=== AI Site Scout ===
Tags: ai, llm, sitemap, crawler, gptbot, claudebot, seo, llms-txt, ai-crawler, perplexity
Requires at least: 6.0
Tested up to: 6.9
Stable tag: 2.0.0
Requires PHP: 7.4
License: GPLv2 or later
License URI: https://www.gnu.org/licenses/gpl-2.0.html

Prepare your WordPress site for AI. Auto-generates llms.txt, llms-full.txt, and llms-sitemap.xml. Logs all AI crawler visits site-wide.

== Description ==

AI Site Scout prepares your WordPress site to be read, indexed, and cited by AI systems — including ChatGPT, Claude, Perplexity, Gemini, and 20+ other crawlers.

= What it does =

* Generates `/llms.txt`, `/llms-full.txt`, and `/llms-sitemap.xml` dynamically from your published content — no physical files written to disk
* Logs AI crawler visits across **every page of your site**, not just the LLM files
* Detects 25+ known AI bots including GPTBot, ClaudeBot, Anthropic-AI, PerplexityBot, Meta ExternalAgent, Cohere AI, ByteSpider, AmazonBot, and more
* Respects Yoast SEO and Rank Math noindex settings — excluded content stays excluded
* Dashboard widget with bot visit stats, summary pills, settings, and one-click regeneration
* Auto-regenerates on publish and on a configurable cron schedule

= Free tier =

* Up to 100 pages indexed
* 200 log entries stored
* 80-word content excerpts in llms-full.txt
* All core features

= Pro tier =

* Unlimited pages indexed
* 5,000 log entries
* CSV export of crawler logs
* 300-word content excerpts
* Image tags in llms-sitemap.xml
* Tag data included in LLM files

= Why llms.txt? =

The emerging [llms.txt standard](https://llmstxt.org) gives AI systems a clean, structured entry point to your site's content — similar to what robots.txt did for search engines. This plugin automates generating and maintaining that file.

== Installation ==

1. Upload the `ai-site-scout` folder to `/wp-content/plugins/`
2. Activate via **Plugins** menu in WordPress
3. Go to your **Dashboard** — find the **AI Site Scout** widget
4. Click **Regenerate Now** to build your LLM files
5. Verify: `https://yoursite.com/llms.txt`

**Note:** If the LLM file URLs return 404, go to **Settings → Permalinks → Save Changes** to flush rewrite rules.

== Frequently Asked Questions ==

= Why aren't any crawlers showing up in the logs? =

Earlier versions of this plugin only logged bots when they visited the `/llms.txt` endpoint directly. v2.0 fixes this — bots are now logged on every page they visit. Make sure you're running v2.0 and that your site is live/public.

= Does this create physical files on my server? =

No. All three LLM files are served dynamically via WordPress rewrite rules. No files are written to your filesystem.

= Will this conflict with my existing sitemap plugin? =

No. The `llms-sitemap.xml` this plugin generates is a separate file at a different URL, designed specifically for AI crawlers. It does not replace your standard XML sitemap.

= Does this work with WooCommerce? =

Yes. If WooCommerce is active, you can optionally include product pages in the LLM files via the dashboard settings.

= How does Pro licensing work? =

Pro is sold as an annual subscription via Stripe. After purchase you receive a license key by email. Enter it in the dashboard widget to unlock Pro features on that site.

== Changelog ==

= 2.0.0 =
* BUG FIX: AI crawlers now logged on ALL front-end page requests, not just LLM file hits
* Added 20+ new bot signatures (PerplexityBot, Anthropic-AI, Meta ExternalAgent, Cohere AI, ByteSpider, AmazonBot, Mistral AI, HuggingFaceBot, and more)
* Refactored into separate classes (AISS_Bot_Detector, AISS_File_Generator, AISS_License)
* Replaced inline style/script tags with wp_add_inline_style() and wp_add_inline_script()
* Added Pro tier with Stripe license activation/deactivation in dashboard
* Added CSV export of crawler logs (Pro)
* Bot summary pills and stats bar in dashboard widget
* Improved llms.txt format with author, categories, and tags
* Shortcode supports limit, type, and category attributes
* Log table shows URL path with clickable link
* Full i18n / translation-ready

= 1.0.4 =
* Initial public release

== Upgrade Notice ==

= 2.0.0 =
Important bug fix: crawler logging now works correctly on all page types. Update recommended for all users.
