Crawler rules
robots.txt allow rules for OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot and Applebot-Extended, with cart routes closed to scrapers.
stacktxt builds the layer AI agents read, so they can find, use and buy from your store.
Get a free scanstacktxt builds the layer agents read: crawler rules, plain-text manifests, structured data and product feeds, delivered to your site so AI agents can see, use and buy from your brand.
Start with what crawlers read. Add your catalog next, then real-time sync and checkout as agents start to buy.
A Lighthouse audit, robots.txt allow rules for AI search crawlers, a /llms.txt manifest and an ARD manifest at /.well-known/ard.json. No setup fee.
Everything in Stage 1, plus a normalized product feed, /llms-full.txt, Product and Offer JSON-LD, a daily feed push to ChatGPT Business Hub and Google Merchant Center, and regional demand by DMA. No setup fee.
Everything in Stages 1 and 2, plus edge webhooks that sync catalog changes to Cloudflare KV, multi-region manifests, UCP cart endpoints and weekly checkout simulations. One-time setup fee.
We write, test and deploy the machine layer as a ZIP bundle, through your Cloudflare zone or as a pull request, and push your feeds to the shopping platforms.
See how we deliverrobots.txt allow rules for OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-SearchBot and Applebot-Extended, with cart routes closed to scrapers.
A daily server-to-server push of your catalog to your ChatGPT Business Hub account as jsonl.gz, tsv.gz or Parquet, so your catalog stays inside OpenAI’s 14-day retention window.
/llms.txt and /llms-full.txt strip markup and scripts, so a model reads your catalog with 90 to 95% fewer tokens.
Search tools registered in document.modelContext, so an agent on your site can query inventory directly.
A discovery file at /.well-known/ard.json that points agents to everything else.
Product, Offer, return and shipping JSON-LD with zero Rich Results errors.
Page speed and Core Web Vitals, measured the way a crawler loads your pages.
Whether an agent can reserve a cart through /.well-known/ucp.
Where your visits from chatgpt.com and perplexity.ai come from, by region.
Catalog changes pushed to Cloudflare KV in under 500ms.
“A client quote goes here once they approve it.”
Your developer uploads a ZIP, you invite us to Cloudflare, or we open a pull request. Feeds go to ChatGPT Business Hub and Google Merchant Center.
The robots.txt patch, /llms.txt, /llms-full.txt, ard.json and feeds in one bundle for your web root.
Invite [email protected] to your zone and we serve the machine routes from the edge.
For Vercel, Netlify or theme repos, we open a branch with the manifests ready to merge.
Your catalog sent server to server to ChatGPT Business Hub and Google Merchant Center every day.
“A second client quote goes here.”
Enter a URL. We check robots.txt, /llms.txt, structured data and feeds the way an agent reads them.
What agents can and cannot read on your site, with the fix for each gap.
We ship the files as a ZIP, through Cloudflare or as a pull request, then check every route.
Monthly drift checks, crawler triage and a short email on what changed.
Every stage ships as plain files you own. Read them, keep them, move them.