AI Agents Still Lean on Human Workers to Complete Real-World Tasks

A growing number of personal AI agent startups are quietly routing tasks to human contractors when their software falls short, underscoring the gap between the promise of autonomous digital assistants and their current capabilities.

Foundation models developed by companies including Meta and OpenAI still cannot reliably take actions across the web — particularly those involving phone calls, appointment scheduling, or bookings with local service providers, according to executives at several agent startups.

Marissa Mayer, the former Yahoo CEO who now leads agent startup Dazzle AI, said current top-tier models perform well at surfacing information but struggle when they need to act on it. "We're relying on the frontier models to give us good suggestions and a good candidate list, but we're checking that over to make sure that we're really delivering the type of service that we can stand behind," Mayer said, describing how Dazzle routes certain tasks to human contractors.

Meghan Joyce, CEO of digital agent startup Duckbill, said the barriers typically involve any task requiring a phone call, a doctor's appointment reservation, or the scheduling of a local cleaning service. For those situations, she said, humans still outperform software.

Duckbill recently launched an application programming interface intended for companies building personal agents to connect with human contract workers. "We work on a model that is not dissimilar from Uber, where we boil tasks down into atomic bite-sized pieces," Joyce said, referencing the ride-hailing company where she previously worked.

Mayer offered a concrete example of where AI agents break down. A Dazzle user asked the technology to simultaneously manage a roof replacement, a broken window, and the purchase of a glass tabletop. The agent confused the two glass-related items, "trying to figure out if he could make the tabletop fit in the window," Mayer said. Dazzle's system flagged the error and handed the task off to a person. "Within a few seconds, the human had things sorted out. Whereas an agent would have burned a ridiculous number of tokens," Mayer said.

Shivani Poddar, CEO of personal AI agent startup Wajo AI, said her company incorporates human workers to handle phone calls and even CAPTCHA tests designed to block automated software. "I think AI plus humans, collaborative AI, is what you need to actually get the job done," Poddar said.

The human-in-the-loop approach raises privacy questions. Meta employees previously raised concerns about an internal test in which human contractors handled phone calls on behalf of the company's Muse AI agent. A Meta spokesperson said at the time that this kind of testing is "core to the product development process." Meta currently does not use humans to fulfill users' Muse requests, the company said.

Duckbill's Joyce said her startup conducts "full background checking" on the 1099 contractors it employs, comparing them to drivers for Uber and DoorDash. Mayer said Dazzle screens for contractors who are "extremely discreet and have good taste and good judgment," adding that the company looks for qualities similar to those she would seek in a human personal assistant.

Mayer described the current moment as a "transitionary period" while foundation models continue to improve toward handling more complex web tasks and phone calls as reliably as people can. "We're not even trying to hide it, we think the human in the loop is a benefit," she said.

Meta's Muse, OpenAI's Dots, and a range of other agents can handle many web-browsing tasks, but whether the underlying models can close the gap on real-world action-taking — without human backup — remains the central question for the sector heading into 2027.

Disclaimer