# Intuist Website - robots.txt # https://intuist.ai User-agent: * Allow: / # Disallow admin/private paths (if any) # /_next/ is deliberately NOT blocked: /_next/static carries the CSS and JS # Google needs to render pages, and /_next/image serves every optimized image. Disallow: /api/ Disallow: /_next/data/ Disallow: /private/ # Sitemap location Sitemap: https://intuist.ai/sitemap.xml # Crawl delay for respectful crawling Crawl-delay: 1 # Google specific User-agent: Googlebot Allow: / Crawl-delay: 0 # Bing specific User-agent: Bingbot Allow: / Crawl-delay: 1