ZenovayTools

Meta Robots Checker

Dedicated indexability checker. Detects X-Robots-Tag HTTP header, meta robots/googlebot/bingbot tags, noindex/nofollow/noarchive/nosnippet directives. Checks canonical consistency. Grade A-F.

How to Use Meta Robots Checker

  1. 1Enter the URL to check for indexability.
  2. 2The tool fetches the page and reads the X-Robots-Tag HTTP response header.
  3. 3All meta robots-related tags are parsed (robots, googlebot, bingbot, etc.).
  4. 4Review each directive (noindex, nofollow, noarchive, nosnippet) and canonical tag.
Zenovay

Track your website performance

Real-time analytics, session replay, heatmaps, and AI insights. 2-minute setup, privacy-first.

Try Zenovay Analytics — Free

Related Tools

Meta Tag AnalyzerAnalyze meta tags of any webpage. Check title, description, Open Graph, Twitter cards, and get SEO recommendations.
Open Graph CheckerPreview how your page looks when shared on Facebook, Twitter, and LinkedIn. Check all OG and Twitter Card tags.
HTTP Header CheckerInspect HTTP response headers of any URL. Check security headers, caching, content type, and more.
Robots.txt ValidatorValidate and analyze your robots.txt file. Check rules, sitemaps, and common crawling issues.

Frequently Asked Questions

What is the difference between meta robots and X-Robots-Tag?▾
Meta robots is an HTML tag in the <head> section: <meta name="robots" content="noindex">. X-Robots-Tag is an HTTP response header that does the same thing but at the server level. X-Robots-Tag is the only way to control indexing of non-HTML files (PDFs, images). Both can target specific crawlers (googlebot, bingbot) with separate directives.
Why is my page indexed even though I have a noindex tag?▾
Google will not index a noindexed page, but it takes time (up to several weeks) for the change to take effect after Google recrawls the page. Also, if the page is blocked by robots.txt, Google may not see the noindex tag — the tag must be crawlable. Check that your robots.txt doesn't block Googlebot from the affected pages.
What is the canonical tag and why does it matter?▾
A canonical tag (<link rel='canonical' href='...'>) tells search engines which URL is the 'preferred' version of a page when multiple URLs show the same content. If the canonical points to a different URL, the page won't rank under the current URL — all ranking signals are attributed to the canonical. Always use absolute URLs in canonical tags.
Can different bots have different noindex rules?▾
Yes. You can use <meta name="googlebot" content="noindex"> to block only Google while allowing Bing to index the page. Or vice versa with <meta name="bingbot">. The generic <meta name="robots"> applies to all crawlers that respect the robots meta tag standard. X-Robots-Tag headers support the same bot-specific syntax.
What does nofollow mean for PageRank?▾
nofollow in the robots meta tag means crawlers won't follow the links on this page (they won't transfer PageRank to linked pages). This is different from rel='nofollow' on individual links. A page-level nofollow applies to all links on the entire page, while link-level nofollow is selective. Use link-level nofollow for fine-grained control.