A recent experiment using Nginx logs revealed how major AI assistants handle user queries that reference specific websites. The analysis distinguished between AI models directly fetching web pages and users clicking through from AI-generated answers. ChatGPT, Claude, and Perplexity were observed to fetch pages directly, with Claude notably checking robots.txt first. Gemini, however, did not make any direct fetches during the test, relying solely on its internal index and providing citations for users to click through. AI
Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →
RANK_REASON The article details an experiment analyzing AI traffic patterns based on server logs, which falls under research and analysis rather than a new product or model release.