Skip to content
IndexDoctor
Sign in

Blocked due to access forbidden (403)

Blocked due to access forbidden (403)

The server refused Googlebot. robots.txt saying Allow does not undo a 403.

A 403 means the host, firewall, or app rejected the request. Search Console records that as a fetch failure. The URL cannot be indexed while the fetch fails.

Cloudflare bot rules, a country block, a basic-auth wall, and an allowlist that omits Googlebot all look like this. A spoofed user-agent from another IP can miss the same rule, so confirm in the firewall logs.

Allow Googlebot to fetch the URL. Do not “fix” a 403 by submitting the URL somewhere else.

What to check

  • Fetch the URL with a Googlebot user-agent and read the status and any challenge page.
  • Check WAF, bot fight, and hotlink rules for a block on Googlebot.
  • Remove basic auth from any URL you want indexed.

Fix prompt

Example prompt

This URL returns 403 to Google. Allow Googlebot through the firewall and return the page with HTTP 200. A robots.txt Allow does not override a 403. Do not call an indexing API.

Check a URL · Crawlers · Glossary

Related