Submitted URL Blocked by Robots.txt
Fix one of the most common Google Search Console coverage errors.
- Home
- Submitted URL Blocked by Robots.txt
Fixing "Submitted URL blocked by robots.txt"
This error occurs when a URL listed in your XML sitemap is explicitly disallowed in your robots.txt file. Because sitemaps represent your "priority" indexation list, while robots.txt acts as a "block" list, Google flags this as a critical conflict that prevents crawling.
Why this issue matters for SEO
When you submit a URL in a sitemap, you are telling Google: "This page is important, please index it." However, if your robots.txt contains a Disallow rule for that same URL, Google's crawlers are instructed to stay away. This results in an indexation deadlock where the page cannot be crawled or updated in the search results.
Common causes of robots.txt blocks
- Broad Disallow Rules: A rule like
Disallow: /products/might accidentally block all product pages even if you want them indexed. - Staging Environment Rules: Forgetting to remove a
Disallow: /rule after migrating from a development server to production. - Dynamic Parameter Blocking: Blocking URLs with certain parameters that are also being generated as canonical URLs in your sitemap.
Resolution Workflow
Identify the blocked URL
Open Google Search Console > Indexing > Pages and click on the 'Submitted URL blocked by robots.txt' status.
Test your robots.txt
Use a robots.txt validator to see exactly which rule is triggering the block for the specific URL.
Update Directives
Remove the Disallow rule or add an Allow rule to give Googlebot access to the priority URL.
Validate in GSC
Click 'Validate Fix' in Search Console to notify Google that the directive conflict has been resolved.
Frequently Asked Questions
Can blocked pages still appear in Google?
Yes, but they will likely show 'No information is available for this page' as Google cannot crawl the content to generate a description.
Does robots.txt stop indexing?
No, it stops crawling. To stop indexing, you should use a 'noindex' tag, but the page MUST be crawlable for Google to see that tag.
Recommended Tools
Related Guides
Continue with these guides to strengthen your technical SEO workflow.