Glossary / Crawl budget

What is Crawl budget?

Crawl budget is how many pages a search engine will bother to visit on your site in a given period. Most small sites never come close to the limit.

Search engines send out bots that follow links and read pages. Those bots do not have unlimited time, so each site gets a rough allowance of attention. That allowance is the crawl budget.

If your site has a few hundred pages, this is not your problem. It starts to matter on large sites: stores with thousands of filter combinations, or systems that spin up a fresh address for every sort order.

The fixes are cleanup, not tricks. Remove dead pages, stop generating junk addresses, keep your sitemap accurate, and speed up slow responses so the bot gets more done per visit.