It’s a brand new page that hasn’t been crawled yet It’s on the site but not in the sitemap The page exists but it is not linked to from anywhere on the site
A hard rule that Google must follow, no matter what A directive that tells Google the preferred version of the page A tag that tells Google the main keyword you want to rank for
To stop the crawler being blocked and keep your developers happy To save money on SEMrush credits The slower the crawler, the more information it retrieves