User-agent: * Allow: / Allow: /api Disallow: /run Disallow: /api/ai-query Disallow: /api/contact Disallow: /api/detect-business Disallow: /api/submit-review Disallow: /api/csp-report Disallow: /api/runs Disallow: /api/status Disallow: /api/domain-history Disallow: /api/psi-status Disallow: /api/health Disallow: /report/ Disallow: /progress Disallow: /stream Sitemap: https://website-auditor.io/sitemap.xml