Research HubHow AI agents interact with your website.
Field research on how AI and the open web actually meet. Each report works from raw data: access logs, HTTP fingerprints, traffic patterns, and direct testing.
Do LLM training pages actually work? Yes, 17% more crawls in a controlled test
Yes. In a controlled eight-week test, the category that got training pages was crawled about 17% more than a comparable one held back as a baseline. One ecommerce product category got prerendered LLM training pages; a comparable category was held out of the rollout as a baseline. Eight weeks later the treated category was being crawled 17% more than the holdout, and crawling rose site-wide over the same period, so the holdout is what separates the effect from the background. Training crawls are how a model learns your products well enough to recommend them without looking them up. The build applies to ecommerce category pages on Shopify, Magento, Adobe Commerce, Salesforce Commerce Cloud, BigCommerce, WooCommerce, and headless storefronts.
What Shopify checks for agentic readiness
We ran a Shopify storefront through shopify.com/agentic-readiness, the public agentic-readiness scanner. Here are the 41 URLs Shopify probes and what to prep on your store.
The bots crawling Shopify's agentic stack
Two months of edge logs from a Shopify research storefront. Microsoft Copilot is the only organic agentic verifier polling on its own. Common LLM crawlers (GPTBot, ClaudeBot, Bingbot) still hit the legacy llms-txt path and ignore the newer agent-discovery files.
