If you’ve ever logged into an advertising or analytics platform, you’ve probably complained about how it’s designed. Finding an answer to the simplest question can be challenging and leave you staring at the home screen of the platform far too long.

Now, these platforms are embedding their AI Models into the platform and giving you a text box to enter your question. A few seconds later and BOOM an answer to your question appears. It can seem like magic and open your eyes to new possibilities, but is it really magic and are there any downsides?

What’s Happening Behind the Scenes

To understand where these in-platform AI advisors fit, we need to first understand how they work. In the interest of simplicity, I’ll over-simplify:
1) You enter a question

2) The advisor sends your question to their large-language model (LLM). This is almost always a model built by the platform itself.

3) The LLM uses tools the platform gives it, like running code to do large scale data analysis

4) The advisor runs its tools (like coding), and works in an agentic manner, looping around-and-around the problem (using the LLM) until it reaches what appears to be an answer to your question

5) The advisor writes the answer back to you in plain text (and/or includes a chart/graph)

What These Advisors Work Well For

These can be helpful tools when you have a strong knowledge of the data architecture of these platforms. For example, how the metrics, dimensions, and filters work together.

Here are common use-cases where these advisors can be helpful:
A) You have a very specific question and can provide all of the parameters and constraints. For example: “Give me sessions by source/medium for this year to date, in a table by month, filtered to traffic from the United States.”

B) You’re troubleshooting an anomaly and know how the data is structured in the tool.

C) You’re looking for inspiration and want to find opportunities worth digging into further.

There are many other ways to use them, but I’ve found a few things to be critical:

  • You understand the platform’s data architecture.
  • You know which parameters you need to get the answer you’re looking for.
  • You know what good looks like and roughly what the answer should look like.
  • The answer doesn’t depend on logic unique to your business, goals or strategy.

Common Challenges That Arise

I’ve seen these advisors be helpful, and I’ve also seen them lead people astray. Here are a few of the more common issues I see when people rely to heavily on these them, or when they use them in situations that they aren’t best positioned to help:

  1. You don’t know which parameters your answer needs. When you click through reports, the metrics, dimensions and filters are right in front of you. They remind you to set a date range, filter by geography or page group, exclude certain traffic, and so on. With an empty text box, it’s easy to forget to include those settings in your question. You end up with a correct answer to an incomplete question.
  2. Your question requires business logic the platform doesn’t have. These platforms don’t know your strategy, goals, past lessons learned, company culture and much more. They only see your advertising or analytics data. So when you ask something like “What should we do differently in our ad campaigns based on this data?” they’re working in a vacuum. Think of the advisor as a new team member. Would you ask that question of someone on their first day, and then give them access to your ads and analytics data and nothing else about the company or your strategy?
  3. Each advisor only sees one channel. Your customers don’t live inside one platform. They search, scroll, click an ad, come back through an email and talk to sales. Each advisor sees its own piece of that journey and nothing else, so questions like “where should my next dollar go?” are outside what it can answer.

Whose Interests is the Advisor Serving?

Last but not least, is bias. I am seeing too many people think that these systems don’t have bias built into them. Someone who was always skeptical of what a Google spokesperson would say, is suddenly willing to take their AI advisors answer as gold. Whether it’s intentional or not, these advisors are free tools designed to get you using the platform more. That creates an inherent bias. And as I mentioned above, they have a narrow view of your business, so they’ll over-index on the data they can see, which is their own.

Would you unquestionably trust an ad platforms human advisor that tells you to spend more money with them? What about an attribution model that only has access to the data from it’s own platform?

In Conclusion

Use these single platform advisors to save you time clicking buttons and assisting with siloed-channel analysis, but don’t outsource complex analysis to them. For that, you’re far better suited to leveraging a combined human and technology approach that bridges data across channels, brings in your unique business logic, qualitative and quantitative data, and takes accountability for specific next steps and results. That’s the role a strong agency partner should play.