For twenty years, the deal was simple: search engines send crawlers, websites send pages, and ads pay the bills. Sundar Pichai now says that deal is about to get a second channel. In an interview on the Sources podcast with Alex Heath, Google’s CEO said he expects AI agents to reach website content in two ways: by browsing pages the way humans do, and through protocols built specifically for agents. Both will grow, he said, but protocols will scale much better, and they will take longer to arrive.
What Pichai Actually Said
Heath asked whether the web needs to evolve for AI agents that use browsers like people do, noting that the current setup can feel like trying to fit a triangle into a square. Pichai called it a complex moment.
“When we say agents working at scale, that means many, many different parts of the economy interoperating with each other in some way, in a way that has to make sense for all the players involved,” Pichai said. He added that he thinks the technology will get ahead of the ecosystem in how those parts come to work together.
He pointed to a lot of momentum behind MCP, agent protocols and payment protocols, citing the Universal Commerce Protocol in particular. His prediction: there is going to be a chaotic evolution there, hopefully working towards more interoperable standards.
Browsing Agents vs Protocols: Two Roads to Your Content
Pichai described both routes as legitimate. On the first road, agents use a browser much like a human would: they open pages, click through flows and fill out forms. He called browser-based computer use a very elegant way to work, because that is what a user would do, and it allows controls that keep users in the loop.
On the second road, agents skip the browser entirely and talk to websites through protocols designed for machine access. Google has been backing the Universal Commerce Protocol (UCP) to enable agents to shop online. Google also lists Google-Agent as the user agent it uses when hosted agents browse the web.
“I think you have both ways and they are going to develop,” Pichai said. He called better protocols a far more scalable approach that would work anywhere and instantly, but said he expects them to take longer to develop than browsing agents.
The Ads Question and the Amazon vs Meta Precedent
Heath also asked how Google protects its own interfaces when AI agents do the browsing, given that Google has a very large ads business that requires engagement. The question is not theoretical. In September, Amazon cut off Meta’s personal agent Muse from shopping on Amazon.com, saying the agent does not identify itself when it browses. Muse can open a browser and fill out forms, exactly the kind of tool Pichai described.
Pichai’s answer was that the right answer longer term is to put users first and ask what users would want. He added that ultimately it has to work for all the businesses providing value, and that the market will reach an equilibrium, though that equilibrium will shift.
He compared this period to the internet’s infancy, when people believed buying and selling homes would change almost instantly. Two decades later most of those methods have stayed the same, a reflection of what he called real constraints in the system.
What This Means for Website Owners
For anyone who runs a website, the practical takeaway is that agent traffic is coming down both roads at once, and your current setup probably handles neither deliberately.
Agents that browse like humans will arrive looking like users. Some identify themselves, as Google’s hosted agents do with the Google-Agent user agent. Others, like the Muse agent Amazon blocked, do not. Agents that use protocols will expect structured, machine-readable access to your products, content or services, and standards like UCP and MCP are still in the messy phase Pichai described.
Publishers already know the stakes. Search traffic has been reshaped by AI answers over the past two years, and Pichai has separately described a future where search gets absorbed into an agentic layer. The sites that survive that shift will be the ones that agents can actually use.
How to Spot Agent Traffic in Your Logs
The first concrete step is measurement. Pull your web server logs for the past week and group requests by user agent string. Google’s hosted agents identify themselves as Google-Agent. Other agents may announce their names or purpose in the user agent or in robots.txt directives, and some, like Meta’s Muse before Amazon blocked it, do not identify themselves at all.
Once you have the list, compare it against your traffic and conversion data. If an agent reads product pages but never converts, that is a cost centre. If a commerce agent places real orders through a protocol, that is a channel worth supporting properly. Pichai’s point about putting users first works here too: the agents worth keeping are the ones your customers are actually using.
Tips: How to Prepare Your Site for AI Agents
- Decide your agent policy and write it down. Look at your server logs for agent user agents today. Decide which agents you welcome and which you rate-limit or block, and make the rule explicit instead of accidental.
- Keep robots.txt honest and current. Google’s own crawling expert Gary Illyes has explained the limits of robots.txt for controlling AI content use. Treat it as one signal, not a security gate, and pair it with server-side rules where it matters.
- Make key flows work without a full browser. If your checkout, booking or signup only works through a complex JavaScript UI, you are invisible to protocol-based agents. Structured data and clean HTML still matter more than ever.
- Watch the protocol space, especially UCP and MCP. Commerce and content sites that support emerging agent protocols early will benefit from the same first-mover advantage that early SEO players enjoyed.
- Return proper status codes. If your site is down for maintenance, serve a 503 so agents and crawlers retry later instead of indexing an error page.
- Protect the business model. With agents in the loop, ad impressions and engagement can drop even when traffic holds. Think now about what value agents should be allowed to take for free and what should require a relationship.
Pichai’s framing is useful because it separates the two battles. The browsing battle is about identity, control and blocking. The protocol battle is about being ready before the standards settle. Neither is optional anymore.
Frequently Asked Questions
How does Sundar Pichai expect AI agents to access websites?
He expects two parallel methods: agents that browse pages the way humans do, and agents that use protocols built for machines. He said protocols such as the Universal Commerce Protocol will ultimately scale better, but will take longer to develop than browser-based agents.
What is the Universal Commerce Protocol (UCP)?
UCP is a protocol that Google has been backing to let AI agents shop online through direct machine access rather than by driving a browser. It sits alongside other emerging agent and payment protocols, including MCP, in what Pichai described as a chaotic but interoperable future.
What is the Google-Agent user agent?
Google-Agent is the user agent string Google lists for its hosted agents when they browse the web. Identifying agent traffic like this in your logs is the first step in deciding which automated visitors you want to allow.
Should I block AI agents from my website?
There is no single answer. Amazon blocked Meta’s Muse agent for not identifying itself, while Google’s own agents announce themselves. Decide per agent based on whether it identifies itself, respects your rules and gives value back, and enforce the decision server-side rather than relying on robots.txt alone.
