Robots.txt Checker

Fetch robots.txt, inspect allow/disallow rules, and spot crawl directives that may block search engines or AI crawlers.