# Allow Social Media Bots full access to scrape images and meta tags User-agent: Twitterbot User-agent: facebookexternalhit User-agent: LinkedInBot User-agent: WhatsApp Allow: / # Block Aggressive Commercial Scrapers & AI Crawlers (Zero Search Value) User-agent: MJ12bot User-agent: DotBot User-agent: AhrefsBot User-agent: SemrushBot User-agent: Bytespider User-agent: PetalBot User-agent: ExaSearchBot User-agent: CCBot User-agent: GPTBot User-agent: ClaudeBot User-agent: anthropic-ai User-agent: Amazonbot Disallow: / # Rules for all other crawlers (Google, Bing, etc.) User-agent: * Disallow: /secret-admin-panel-f9z3e/ Disallow: /accounts/ Disallow: /users/edit/ Disallow: /users/restricted/ Disallow: /users/profile/delete/ Disallow: /users/feedback/ Disallow: /u/ Disallow: /quizzes/tag/ Disallow: /*?from_home=* Disallow: /*&from_home=* Disallow: /*?page=* Disallow: /*&page=* Disallow: /*?sort=* Disallow: /*&sort=* Disallow: /*?rows=* Disallow: /*&rows=* Disallow: /*?creator=* Disallow: /*&creator=* # Ensure media and static are allowed (usually allowed by default if not disallowed) Allow: /media/ Allow: /static/ Sitemap: https://quizzme.org/sitemap.xml