Indexing
Why is my page not indexed by Google?
The usual reasons a URL stays out of Google, and which one a public check can see.
Google indexes a URL only after it can fetch the page and decides the page is worth keeping. A missing result is not one error. It is one of a few different stops.
The page may never have been discovered. It may be blocked in robots.txt, or it may return an error, a login wall, or a redirect. It may carry noindex. Or Google fetched it and chose not to keep it, which Search Console calls crawled or discovered and currently not indexed.
Start with what anyone can see. Fetch the URL, read robots.txt, and read the HTML before JavaScript runs. IndexDoctor's free check does that and names the stop it can prove. Google's own reason, such as “Discovered - currently not indexed”, needs Search Console and is separate from the public check.
None of these stops is fixed by asking an API to index the URL. Change the response, the robots rule, or the page, then wait for the next crawl.
What to check
- Confirm the URL returns HTTP 200 to a browser that is not signed in.
- Search the HTML and the HTTP headers for noindex.
- Read the Googlebot group in robots.txt, not only the wildcard group.
- View the page with JavaScript off. If the words disappear, that is what a cautious crawler may index.
Fix prompt
Example prompt
This URL is not indexed. Check HTTP status, robots.txt for Googlebot, noindex in the HTML and headers, the canonical, and whether the main copy is in the server HTML. Fix the one that is blocking. Do not call an indexing API.
Check a URL · Crawlers · Glossary
Questions
- How long should I wait after a fix?
- Google recrawls on its own schedule. Important URLs are usually fetched again sooner than thin ones. There is no switch that forces the next crawl.
- Is a new site supposed to be missing?
- Often, yes. A new host has little history. Links from pages Google already knows, and a sitemap of real URLs, are how discovery starts.