{"id":12201,"date":"2026-07-22T10:52:30","date_gmt":"2026-07-22T08:52:30","guid":{"rendered":"https:\/\/mybox.com\/help\/?post_type=manual_kb&#038;p=12201"},"modified":"2026-07-22T10:52:35","modified_gmt":"2026-07-22T08:52:35","slug":"how-to-tell-if-an-indexing-problem-is-caused-by-crawl-budget-site-quality-or-technical-signals","status":"publish","type":"manual_kb","link":"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-to-tell-if-an-indexing-problem-is-caused-by-crawl-budget-site-quality-or-technical-signals\/","title":{"rendered":"How to tell if an indexing problem is caused by crawl budget, site quality, or technical signals"},"content":{"rendered":"\n<div class=\"translation-block translation-block-merged\">\n<p class=\"wp-block-paragraph\">When pages are missing from Google, the cause usually falls into one of three groups: crawl budget limits, weak site quality signals, or technical signals that tell Google not to index a page. These causes can overlap, so the most useful first step is to compare the signs and decide which group best fits your situation before making changes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This matters because the right fix depends on the reason pages are not indexed. A large site with many low-priority URLs often points to crawl budget issues. A small site with thin or repetitive pages often points to weak value signals. A page blocked by noindex, robots rules, canonicals, or redirects points to technical signals. Search Console usually gives the clearest starting point.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_86 ez-toc-wrap-left counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-to-tell-if-an-indexing-problem-is-caused-by-crawl-budget-site-quality-or-technical-signals\/#What_each_indexing_cause_means\" >What each indexing cause means<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-to-tell-if-an-indexing-problem-is-caused-by-crawl-budget-site-quality-or-technical-signals\/#Quick_comparison_of_the_three_causes\" >Quick comparison of the three causes<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-to-tell-if-an-indexing-problem-is-caused-by-crawl-budget-site-quality-or-technical-signals\/#Signs_that_point_to_crawl_budget_limits\" >Signs that point to crawl budget limits<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-to-tell-if-an-indexing-problem-is-caused-by-crawl-budget-site-quality-or-technical-signals\/#Signs_that_point_to_weak_site_quality_signals\" >Signs that point to weak site quality signals<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-to-tell-if-an-indexing-problem-is-caused-by-crawl-budget-site-quality-or-technical-signals\/#Signs_that_point_to_technical_blockers\" >Signs that point to technical blockers<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-to-tell-if-an-indexing-problem-is-caused-by-crawl-budget-site-quality-or-technical-signals\/#How_to_decide_what_to_investigate_first\" >How to decide what to investigate first<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-to-tell-if-an-indexing-problem-is-caused-by-crawl-budget-site-quality-or-technical-signals\/#What_is_normal_and_when_to_look_deeper\" >What is normal and when to look deeper<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_each_indexing_cause_means\"><\/span>What each indexing cause means<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<\/div>\n\n<div id=\"mybox-285553756\" class=\"mybox-content mybox-entity-placement\"><div class=\"early-access-banner-inpost\">\r\n  <div class=\"banner-left-inpost\">\r\n    <div class=\"icon-box-inpost\">\r\n      <img decoding=\"async\" src=\"https:\/\/mybox.com\/help\/wp-content\/uploads\/2026\/02\/square-info-icon.svg\" alt=\"Info\">\r\n    <\/div>\r\n    <div class=\"text-box-inpost\">\r\n      <span class=\"label-inpost\"><span class=\"translation-block translation-block-banner-text\">Early access<\/span><\/span>\r\n      <h4><span class=\"translation-block translation-block-banner-text\">Still need help?<\/span><\/h4>\r\n      <p><span class=\"translation-block translation-block-banner-text\">Contact our customer service team.<\/span><\/p>\r\n    <\/div>\r\n  <\/div>\r\n\r\n  <div class=\"banner-right-inpost\">\r\n    <a href=\"https:\/\/panel.mybox.com\/helpdesk2\/v\/list\/\" class=\"banner-button-inpost\"><span class=\"translation-block translation-block-banner-text\">Message us<\/span><\/a>\r\n  <\/div>\r\n<\/div><\/div>\n\n<div class=\"translation-block translation-block-merged\"><p class=\"wp-block-paragraph\"><strong>Crawl budget<\/strong> is the amount of crawling Googlebot is willing and able to spend on your site. Googlebot works under resource limits, so it does not crawl every URL equally. This usually matters more on larger sites, on sites with many filter or parameter pages, or on sites that create many low-value URLs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Site quality signals<\/strong> are the signs Google uses to judge whether a page adds enough value to be indexed. If pages are very similar, very short, outdated, or provide little original value, Google may crawl them but still decide not to index them.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Technical signals<\/strong> are direct instructions or technical conditions that affect crawling and indexing. Common examples include a noindex directive, robots.txt blocking, a canonical pointing elsewhere, redirect chains, soft 404 behaviour, or server responses that prevent normal access.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Quick_comparison_of_the_three_causes\"><\/span>Quick comparison of the three causes<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Cause group<\/th><th>Common signs<\/th><th>More likely when<\/th><th>What to check first<\/th><\/tr><\/thead><tbody><tr><td>Crawl budget<\/td><td>Many discovered URLs stay unindexed, crawling is uneven, deep pages are ignored<\/td><td>The site is large or creates many low-value URLs<\/td><td>Search Console coverage patterns, internal linking, URL volume<\/td><\/tr><tr><td>Site quality signals<\/td><td>Pages are crawled but not indexed, similar pages drop out, low-value page types struggle<\/td><td>Pages are thin, repetitive, or weak compared with the rest of the site<\/td><td>Page uniqueness, usefulness, duplication, template-heavy content<\/td><\/tr><tr><td>Technical signals<\/td><td>Clear indexing exclusions, blocked crawling, canonical conflicts, redirects, server issues<\/td><td>Affected pages share one setup or one template<\/td><td>Noindex, robots.txt, canonicals, status codes, redirect behaviour<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Signs_that_point_to_crawl_budget_limits\"><\/span>Signs that point to crawl budget limits<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Crawl budget issues are usually pattern-based, not page-based. The strongest sign is scale. If your site has many thousands of URLs, many faceted pages, pagination paths, search result pages, or URL parameters, Googlebot may spend time on low-priority URLs instead of your important pages.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Search Console evidence often shows pages discovered but not indexed, or crawled inconsistently across sections. Important pages may be indexed slowly while duplicate or near-duplicate URLs keep appearing. Deep pages that require many clicks to reach can also be delayed because Googlebot tends to focus on stronger, easier paths first.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Start by checking which page types are affected. If the problem is concentrated in low-priority archives, filters, tags, or parameter URLs, crawl budget is a likely factor. If only a few key pages are affected on a small site, crawl budget is less likely to be the main cause.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Signs_that_point_to_weak_site_quality_signals\"><\/span>Signs that point to weak site quality signals<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Site quality issues are more about evaluation than access. Google may be able to crawl a page but still decide it does not need to be indexed. This is common when many pages use the same structure with only small wording changes, when category or location pages add little unique value, or when content exists mainly to target keywords rather than help users.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Search Console may show that pages were crawled but are still not indexed. This often means the page was seen, but Google did not find enough reason to keep it in the index. On small and medium sites, this is often a more likely cause than crawl budget.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Compare affected pages with the pages that are indexed. Look at page type, content depth, internal links, and how distinct the page is from similar URLs. If the missing pages are the weakest set on the site, quality signals are a stronger lead than technical blocking.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Signs_that_point_to_technical_blockers\"><\/span>Signs that point to technical blockers<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Technical signals are the clearest to confirm because they leave direct evidence. If a page has a noindex tag, is blocked in robots.txt, redirects to another URL, returns an unexpected status code, or declares another URL as canonical, Google may not index it even if the content is good.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These issues often affect groups of pages that share the same template, plugin behaviour, or CMS rule. If an entire section dropped out at once, or if indexing changed after a site update, migration, or redirect change, technical signals should be checked first.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Look for consistency. If every affected page has the same canonical setup, the same meta robots directive, or the same response pattern, that is a strong sign of a technical cause rather than a crawl budget or quality problem.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_to_decide_what_to_investigate_first\"><\/span>How to decide what to investigate first<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<ol class=\"wp-block-list\">\n<li><p>Check Search Console for the exact exclusion or status pattern. If it shows direct blocking or indexing instructions, start with technical signals.<\/p><\/li>\n\n\n\n<li><p>Measure the scope. If the site is large and generates many low-value URLs, review crawl budget next.<\/p><\/li>\n\n\n\n<li><p>Compare page types. If the missing pages are thin, repetitive, or low-value compared with indexed pages, review site quality signals.<\/p><\/li>\n\n\n\n<li><p>Look for section-wide patterns. Template-level issues usually point to technical setup. Scattered low-value pages usually point to quality. Large URL sets with weak internal priority often point to crawl budget.<\/p><\/li>\n\n\n\n<li><p>Use a few sample URLs from each affected group. Do not diagnose the whole site from one page. Indexing problems are often different across page types.<\/p><\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_is_normal_and_when_to_look_deeper\"><\/span>What is normal and when to look deeper<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Not every page on a site will be indexed. Google evaluates pages and may choose not to keep low-value or duplicate URLs. This is normal behaviour. It becomes a problem when important pages, core templates, or whole sections that should rank remain unindexed.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If the cause is still unclear, split the affected URLs into groups by page type and check each group separately in Search Console. That usually shows whether the pattern is about scale, page value, or a technical instruction. For related background on why pages may not appear in Google at all, see <a href=\"https:\/\/mybox.com\/help\/knowledgebase\/why-google-doesnt-index-a-website\/\">Why Google doesn\u2019t index a website<\/a>. If you need to verify how Google is treating specific URLs, see <a href=\"https:\/\/mybox.com\/help\/knowledgebase\/how-to-check-a-websites-indexing-in-google\/\">How to check a website\u2019s indexing in Google<\/a>.<\/p>\n<\/div>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"template":"","format":"standard","manualknowledgebasecat":[42],"manual_kb_tag":[],"class_list":["post-12201","manual_kb","type-manual_kb","status-publish","format-standard","hentry","manualknowledgebasecat-miscellaneous"],"_links":{"self":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb\/12201","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb"}],"about":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/types\/manual_kb"}],"author":[{"embeddable":true,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":1,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb\/12201\/revisions"}],"predecessor-version":[{"id":12213,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb\/12201\/revisions\/12213"}],"wp:attachment":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/media?parent=12201"}],"wp:term":[{"taxonomy":"manualknowledgebasecat","embeddable":true,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manualknowledgebasecat?post=12201"},{"taxonomy":"manual_kb_tag","embeddable":true,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb_tag?post=12201"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}