On November 5 Visa published what happened when it gave 22 AI models 15 ordinary shopping jobs. They failed 42.8% of the runs, and in more than half of those failures the agent bought something anyway. When an agent buys the wrong thing from you, the return lands on your side of the counter.
Visa's Agentic Commerce Benchmark set AI models the kind of job a shopper might hand an agent, like buying a TV or a laptop to fit particular needs. The 22 models came from 12 labs, and each got three attempts at each of the 15 tasks. Across all the runs the pass rate was 57.2%. Only 34.2% of model and task pairings passed all three times. Visa said the most common failures were recommending a product that didn't fit what the shopper needed, and not asking enough questions to find out what that was.
For anyone who sells things, the next number is the one to read twice. About 54% of the failed runs still ended in a purchase. The agent misread the job, picked something and paid for it.
An agent that guesses still checks out. The wrong order ships from your warehouse, not the agent's.
Shopping agents are spreading faster than they're getting accurate.
Shopify said on November 3 that AI traffic to its merchants grew 2.6 times in the third quarter, and that shoppers who arrive from AI channels convert 64% more often, by the company's own figures. On November 6 Amazon let US customers text its Alexa Shopping agent to search, build a cart, buy, track a price and buy automatically when the price drops to a target. Through Shop Direct, that agent also buys from other stores on the web. Google brought UCP checkout to Canada on November 4, in AI Mode and the Gemini app, with lululemon, Shoppers Drug Mart, Simons and some Shopify stores at launch. Zalando launched an agent that buys from other retailers inside its own app, Reuters reported, and The Times reports that Meta's Muse reaches the UK within weeks.
Every one of those agents works from whatever it can read about your products. None of them knows your store the way your own systems do, so when the facts it finds are thin, stale or missing, it fills the gap with a guess. Visa's test shows how often that guess is wrong.
22 AI models, 15 shopping tasks, three attempts each. Failed runs split by whether the agent still made a purchase.
Source: Visa Agentic Commerce Benchmark, Visa press release, November 5, 2026. Visa reported a 57.2% pass rate and that about 54% of failed runs ended in a purchase. The split of the 42.8 failed runs is calculated from those two figures and rounded.
Last week the question was whether to let agents in. This week we learned how often the ones that get in get it wrong.
Our last briefing covered retailers blocking AI shopping agents, and Forrester's advice that commerce teams should decide which agents to serve and what those agents may do. Visa's results put weight behind the second half of that advice. An agent you let in and leave to work from guesses can do real damage, because it doesn't stop at a bad recommendation. It completes the order.
Shoppers don't give second chances either. ACI found that 60% of UK adults would abandon an AI shopping agent after a single mistake, PPC Land reported on November 6. Dropping the agent doesn't undo the order, though. The shopper who walks away from it still has your box to send back, and the memory of buying the wrong thing from your store.
You can't fix the agent. You can decide what it reads and what it's allowed to do.
A wrong order can cost more than the sale was worth. There's the return shipping, the restocking, the support ticket, and a customer who now trusts both the agent and your store a little less. You have no say over which model a shopper uses or how well it asks questions. What you do control is your side of the conversation: whether the agent gets your real price, stock, sizing and return terms from the source, and which actions it can take on its own.
The volumes are still small, which is the good news. Booking Holdings said OpenTable seated 190,000 diners through its ChatGPT integration in September, one of the few real agent numbers anywhere, and its CFO still described AI traffic as "very small in aggregate." According to a write-up of Uber's third-quarter call, its CFO said agents booked four rides in Halloween week. Black Friday is November 27, and agents will be busier than they've ever been. Setting your rules now means the first busy week runs on your facts instead of an agent's best guess.
We give agents your facts and your rules, straight from your systems.
BeaconSpec is the discovery layer for agentic commerce. You already run APIs behind your catalog, booking engine or ordering system: REST, OpenAPI or GraphQL. We turn them into one curated MCP server, the open protocol AI assistants use to call outside tools, and publish its UCP profile at your own /.well-known/ucp, so an agent can ask your systems directly instead of piecing your store together from web pages.
That changes what the agent knows. Prices and stock come from your systems at the moment the agent asks, not from a page it read last month. Curation decides what it can do. You pick exactly which operations an agent may call, and anything you leave off doesn't exist as far as the agent is concerned. If you want agents searching your catalog and checking stock but not placing orders yet, that's the list. When a task needs the shopper signed in, like checking an order, we handle the standard OAuth login so the agent works inside that shopper's own account, with no shared passwords.
We host it, or hand you a sealed container to run in your own environment. BeaconSpec isn't a platform, a marketplace or a payment network, so the interface belongs to you. We can't make an agent ask a shopper better questions. We can make sure the answers it gets about your business are right, and keep your interface correct as the standards underneath it change.
AI shopping agents failed 42.8% of Visa's test runs and often bought anyway. You can't fix the agent, but you can decide what it reads about you and what it's allowed to do.
Shoppers give an agent one chance: 60% in ACI's UK survey would drop it after a single mistake.
Agent volumes are small before the holidays, which makes this the cheap time to set your rules.
Worth watching over the next month: agent order counts from Black Friday week; the date Muse reaches the UK and which retailers sign on; how far Amazon's text agent goes in buying from other stores; and whether Visa reruns its benchmark as the models improve.
BeaconSpec exists for exactly this shift: making your business discoverable and transactable by AI agents over UCP, on an interface you publish and control, kept correct as the standards underneath it keep changing.
David Soden writes about automation, web strategy, and building durable technical systems for businesses. Photography via Pexels (Polina Tankilevitch, Mateusz Dach, EqualStock IN). Figures cited are drawn from Visa's Agentic Commerce Benchmark press release (November 5, 2026); PPC Land on ACI's UK consumer survey (November 6, 2026); Constellation Research on Shopify's third-quarter results (November 3, 2026); Amazon's announcement of Alexa for Shopping by text (November 6, 2026); Search Engine Journal and lululemon on Google's UCP checkout in Canada (November 4, 2026); Reuters on Zalando's shopping agent (November 3, 2026); Retail Gazette, citing The Times, on Meta's Muse in the UK (November 9, 2026); OfficeChai on Booking Holdings' third-quarter call; and a published write-up of Uber's third-quarter call (November 4, 2026).