User-agent: * Disallow: /admin/ Disallow: /admin012341/ Disallow: /api/ Disallow: /backup/ Disallow: /inc/ Disallow: /data/ Disallow: /static/ Disallow: /template/ Disallow: /docs/ Disallow: /_migrate.js Disallow: /_analyze_mdb.js Disallow: /_db_schema.json Disallow: /_api_key.txt Disallow: /_extract_*.js Disallow: /_query_db.js Disallow: /_read_sessions.js Disallow: /progress.md Disallow: /package.json Disallow: /package-lock.json # ===== 垃圾爬虫屏蔽（共 69 条）===== # 清单维护：C:\工作总结\1-规则和技能\爬虫屏蔽清单.md # 新增后运行 同步robots.js 重新生成，勿手工编辑本段 User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: SEOkicks Disallow: / User-agent: Screaming Frog Disallow: / User-agent: SerpstatBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: MegaIndex.ru Disallow: / User-agent: XoviBot Disallow: / User-agent: LinkpadBot Disallow: / User-agent: SERankingBacklinksBot Disallow: / User-agent: ZoominfoBot Disallow: / User-agent: aiHitBot Disallow: / User-agent: GrapeshotCrawler Disallow: / User-agent: Owlin Disallow: / User-agent: Dataprovider.com Disallow: / User-agent: YandexBot Disallow: / User-agent: Mail.RU_Bot Disallow: / User-agent: SeznamBot Disallow: / User-agent: PetalBot Disallow: / User-agent: SurdotlyBot Disallow: / User-agent: GPTBot Disallow: / User-agent: OAI-SearchBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Google-Extended Disallow: / User-agent: CCBot Disallow: / User-agent: Amazonbot Disallow: / User-agent: Amzn-SearchBot Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: facebookexternalhit Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Diffbot Disallow: / User-agent: cohere-ai Disallow: / User-agent: omgili Disallow: / User-agent: omgilibot Disallow: / User-agent: Timpibot Disallow: / User-agent: YouBot Disallow: / User-agent: VelenPublicWebCrawler Disallow: / User-agent: Webzio-Extended Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: img2dataset Disallow: / User-agent: FriendlyCrawler Disallow: / User-agent: ExaSearchBot Disallow: / User-agent: Python-requests Disallow: / User-agent: Python-urllib Disallow: / User-agent: curl Disallow: / User-agent: Go-http-client Disallow: / User-agent: Scrapy Disallow: / User-agent: okhttp Disallow: / User-agent: libwww-perl Disallow: / User-agent: httrack Disallow: / User-agent: MauiBot Disallow: / User-agent: TurnitinBot Disallow: / User-agent: ShapBot Disallow: / User-agent: AteveSearchSourceUrlDiscovery Disallow: / User-agent: GeneralCrawlBot Disallow: / User-agent: dwnupd Disallow: / User-agent: NavCrawl Disallow: / User-agent: VexelSearchBot Disallow: / User-agent: IbouBot Disallow: / User-agent: iaskspider Disallow: / User-agent: YodaoBot Disallow: / User-agent: dead-link-checker Disallow: / Sitemap: http://www.nai17.com/sitemap.xml