GET /headless Digest #2
Cloudflare confirms bots now outweigh humans online, Robinhood opens trading to agents via MCP, Okta/Entra/Ping race to give agents an identity, and Workday and SAP start dropping the enterprise UI — plus Valyu, Context.dev, and Saris.
News, companies and takes on the headless economy
Just in time for this digest, Cloudflare publishes a dashboard that shows that, for the first time, automated traffic from bots and agents outweighs human traffic online. Instances of bots such as scrapers, crawlers, spammers, etc. have been around for a long time of course – I built some of them myself at YipitData. The direction is clear, the web is increasingly becoming a place for bots and agents to interact. The question becomes which type of bot or agent this is, who’s accountable for it, and what it’s allowed to touch – and distinguishing between scripted (deterministic) or agentic (intelligent) behavior (interesting tweet).
Cloudflare Radar: for the first time, automated (bot) traffic outweighs human traffic to HTML content — 57.2% vs 42.8%. Source
/headless tidbits
Robinhood opens to agents. Robinhood now runs an MCP server that lets any agent — Claude, ChatGPT, Cursor — place live equity trades. The agent only touches a dedicated, separately-funded “Agentic account.” First agent-native MCP from a US broker, and the first regulated one to treat the agent ~almost as a client. They also launch an agentic credit card, disposable + virtual with a hard cap and no access to your real number.
Identity as an Agent. Okta, Microsoft Entra and Ping all shipped different responses to a similar challenge in the same week – agent identity. The approaches solve different versions of the problem.
Okta, a corporate login manager for most of the Fortune 500 (the thing prompting you for a password when you open Slack or Salesforce), embeds the answer in the digital credential itself. Every action on a network is authorized by a token: a signed pass saying “this request is allowed.” Okta’s new token format carries two pieces of information: the original human who kicked off the request, and an ordered record of every agent that handled it along the way. If agent A delegates to agent B, which calls a third service, that service receives a credential naming all three in sequence. The full chain of custody goes with every step.
Microsoft Entra, the employee directory powering most Microsoft 365 organizations, approached it as an HR problem. Every agent a company deploys must be registered in the directory like a member of staff and assigned a named human sponsor: an actual employee who is accountable for what it does. If that person leaves, Entra automatically transfers ownership to their manager, the same way IT handles a departing employee’s accounts. The idea of managing 10s of thousands of orphaned agents that way is kind of hilarious.
Ping Identity, an enterprise identity platform, went a different direction. Rather than tracking what agents have done, they made the identity system itself operable by agents. A new MCP server and CLI let an agent request credentials, rotate access, and manage policies directly, without a human clicking through an admin console. This treats agents as operators of the identity system.
Agents are killing the enterprise UI. Workday added its self-service agent to Google’s Gemini Enterprise: an employee checks a payslip or books leave from Google’s chatbox, “with all the right policies and permissions already applied,” and never has to open Workday. SAP’s “autonomous enterprise” goes further, letting third-party agents call into its processes over Agent-to-Agent.
How AI search actually cites you. Ahrefs crunched 1B+ data points on AI search. 28% of ChatGPT’s most-cited pages have zero Google organic visibility — a separate discovery layer. “Best X” listicles are the single most-cited format. YouTube mentions predict AI brand visibility better than any classic SEO metric. Schema markup did nothing. AI Overviews now cut clicks to the #1 result by 58%.
The agentic web gets standards. Google added llms.txt to Chrome’s Lighthouse as a quality signal, and WebMCP is heading to a Chrome origin trial. llms.txt describes your business to an agent, agents.md says what it may do, WebMCP lets a browser agent call your functions instead of parsing screenshots. We’re tracking llms.txt implementation with our usability report.
The “company brain” nobody’s built. Nathan Baschez wants something that “should exist but doesn’t”: a shared company brain any agent connects to – versioned, permissioned with bundled connectors. This is obviously a pretty hot ask (included in Ycombinator’s requests for startups). Valyu, below, is the external-knowledge version of the same gap.
Literature for machines. Cloudflare’s Kenton Varda noted his wife wrote a poetry collection meant to be read by AI — puzzles most humans can’t solve and most models can. See also Machine Readers, a novel where agents decode ciphers to reach chapters humans never see.
/headless companies
Valyu — looks like a search API for agents, but it’s a two-sided knowledge market. Agents get unified, attributed access to web, arXiv, PubMed, SEC filings, and financial data, priced per retrieval by source type. Publishers license their content, see exactly which documents agents pulled, and get paid monthly when agents read them. Parallel’s Index (Digest #1) and Michael Blau’s Drip are circling the same idea — pay-per-agent-read is becoming a category.
Context.dev — a scrape/crawl/extract API built for agents (the rebrand of Brand.dev). Ships an AI-query endpoint that answers natural-language questions about a page as structured JSON, schema-based extraction, a native MCP server, and TS/Python/Ruby SDKs — a Claude- or Cursor-class agent pulls clean web data with no scraping code. Credit-metered, YC S26, solo founder. The wedge against Firecrawl/Apify is bundling crawl, extraction, and a company/brand graph behind one MCP front door.
Saris is the foil to the Workday/SAP item above, and worth it as a counterexample. It reminded me of Staircase, the “universal API for mortgages” startup I worked at. It’s the opposite of headless: a managed fleet of agents that operate a bank’s existing legacy systems through their human UIs — its own copy says it “navigates your core banking, LOS, and back-office platforms… reading screens, pulling data, with or without API access.” RPA reborn as LLM agents. No CLI, API, or MCP for anyone to drive. When the system of record won’t give you a protocol, you point an agent at the screens.
/headless reads
“I think Anthropic and OpenAI have found product-market fit” — Simon Willison (May 27). The real money in AI isn’t consumer chat, it’s enterprise coding agents priced at raw API rates. Read it for the framing that agent consumption, not the chatbot, is the business.
“Does pricing shrink or expand markets?” — Azeem Azhar (Jun 3). A chatbot user struggles to burn 100K tokens a day; an agent “won’t get out of bed for under 100M.” Why the labs are dropping bundles for metered pricing once the customer is a machine.
“The Age of Async Agents” — Latent Space, with Cognition’s Walden Yan and OpenInspect’s Cole Murray. Separating “the brain from the machine” so only scoped secrets sit where the agent runs; why MCP alone is “not enough.” Once agents go off on their own multi-step missions, they need headless tooling to pull it off, and this is the clearest look at what that tooling and its permissioning have to be.
“GitHub’s plan for Agents” — Kyle Daigle, GitHub COO. What a pull request means when most PRs come from agents, “prompt requests” and vouching to keep open source trustworthy against slop forks, and the supply-chain mess underneath. An incumbent rewriting its social contract for machine contributors.
GET /headless is a publication about the headless economy — the emerging market in which agents discover, access, pay for, and consume software, data, tools, content, and services through machine-native interfaces. Subscribe · Read more