{"id":988,"date":"2025-12-22T08:37:35","date_gmt":"2025-12-22T07:37:35","guid":{"rendered":"https:\/\/mybox.com\/help\/?post_type=manual_kb&#038;p=988"},"modified":"2026-06-07T19:44:24","modified_gmt":"2026-06-07T17:44:24","slug":"what-is-indexing","status":"publish","type":"manual_kb","link":"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/","title":{"rendered":"What is indexing?"},"content":{"rendered":"\n<div class=\"translation-block translation-block-merged\">\n<p class=\"wp-block-paragraph\">Indexing is the process by which search engines collect, analyze, and store information about websites so that their content can appear in search results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Without indexing, a website cannot be found through search engines such as Google, Bing, or other search providers.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_86 ez-toc-wrap-left counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Contents\" >Contents<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Common_Terms\" >Common Terms<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Crawling\" >Crawling<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Web_Crawler_Spider\" >Web Crawler (Spider)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Googlebot\" >Googlebot<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#How_Indexing_Works\" >How Indexing Works<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#1_Discovery\" >1. Discovery<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#2_Crawling\" >2. Crawling<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#3_Processing\" >3. Processing<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#4_Indexing\" >4. Indexing<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#What_Information_Is_Indexed\" >What Information Is Indexed?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Why_Is_Indexing_Important\" >Why Is Indexing Important?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Example\" >Example<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#How_to_Help_Search_Engines_Index_Your_Website\" >How to Help Search Engines Index Your Website<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Creating_an_XML_Sitemap\" >Creating an XML Sitemap<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Using_Internal_Links\" >Using Internal Links<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-17\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Publishing_Quality_Content\" >Publishing Quality Content<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-18\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Avoiding_Crawl_Restrictions\" >Avoiding Crawl Restrictions<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-19\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Submitting_Your_Website_to_Search_Engines\" >Submitting Your Website to Search Engines<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-20\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/what-is-indexing\/#Summary\" >Summary<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Contents\"><\/span>Contents<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>How Indexing Works<\/li>\n\n\n\n<li>Common Terms<\/li>\n\n\n\n<li>What Is a Web Crawler?<\/li>\n\n\n\n<li>What Information Is Indexed?<\/li>\n\n\n\n<li>Why Is Indexing Important?<\/li>\n\n\n\n<li>How to Help Search Engines Index Your Website<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Common_Terms\"><\/span>Common Terms<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Crawling\"><\/span>Crawling<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<\/div>\n\n<div id=\"mybox-1643105026\" class=\"mybox-content mybox-entity-placement\"><div class=\"early-access-banner-inpost\">\r\n  <div class=\"banner-left-inpost\">\r\n    <div class=\"icon-box-inpost\">\r\n      <img decoding=\"async\" src=\"https:\/\/mybox.com\/help\/wp-content\/uploads\/2026\/02\/square-info-icon.svg\" alt=\"Info\">\r\n    <\/div>\r\n    <div class=\"text-box-inpost\">\r\n      <span class=\"label-inpost\"><span class=\"translation-block translation-block-banner-text\">Early access<\/span><\/span>\r\n      <h4><span class=\"translation-block translation-block-banner-text\">Still need help?<\/span><\/h4>\r\n      <p><span class=\"translation-block translation-block-banner-text\">Contact our customer service team.<\/span><\/p>\r\n    <\/div>\r\n  <\/div>\r\n\r\n  <div class=\"banner-right-inpost\">\r\n    <a href=\"https:\/\/panel.mybox.com\/helpdesk2\/v\/list\/\" class=\"banner-button-inpost\"><span class=\"translation-block translation-block-banner-text\">Message us<\/span><\/a>\r\n  <\/div>\r\n<\/div><\/div>\n\n<div class=\"translation-block translation-block-merged\"><p class=\"wp-block-paragraph\">Crawling is the process of discovering and analyzing content on websites.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">During crawling, a search engine follows links between pages and examines their content.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Web_Crawler_Spider\"><\/span>Web Crawler (Spider)<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A web crawler, also known as a spider or bot, is an automated program that visits websites and collects information for indexing.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Googlebot\"><\/span>Googlebot<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Googlebot is the web crawler used by Google to discover and index content across the web.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Other search engines use their own crawlers, including:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Microsoft Bing \u2013 Bingbot<\/li>\n\n\n\n<li>Baidu \u2013 Baiduspider<\/li>\n\n\n\n<li>Yandex \u2013 YandexBot<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_Indexing_Works\"><\/span>How Indexing Works<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The indexing process typically consists of several stages:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"1_Discovery\"><\/span>1. Discovery<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The search engine discovers a page through:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Links from other websites<\/li>\n\n\n\n<li>Internal website links<\/li>\n\n\n\n<li>XML sitemaps<\/li>\n\n\n\n<li>Previously indexed pages<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"2_Crawling\"><\/span>2. Crawling<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A crawler visits the page and analyzes its content.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">During this process, the crawler behaves similarly to a regular visitor by requesting pages and following links.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"3_Processing\"><\/span>3. Processing<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The search engine analyzes the page and extracts information such as:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Page title<\/li>\n\n\n\n<li>Headings<\/li>\n\n\n\n<li>Main content<\/li>\n\n\n\n<li>Images<\/li>\n\n\n\n<li>Metadata<\/li>\n\n\n\n<li>Internal and external links<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"4_Indexing\"><\/span>4. Indexing<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The collected information is stored in the search engine&#8217;s database, known as the index.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Once indexed, the page becomes eligible to appear in search results.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_Information_Is_Indexed\"><\/span>What Information Is Indexed?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Search engines can store various types of information, including:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Page titles<\/li>\n\n\n\n<li>Meta descriptions<\/li>\n\n\n\n<li>URLs<\/li>\n\n\n\n<li>Text content<\/li>\n\n\n\n<li>Images<\/li>\n\n\n\n<li>Structured data<\/li>\n\n\n\n<li>Keywords and topics<\/li>\n\n\n\n<li>Internal links<\/li>\n\n\n\n<li>External links<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For example, when a crawler visits a page, it may record:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Page Title: Web Hosting Services\nDescription: Fast and reliable hosting solutions\nURL: https:\/\/example.com\/hosting\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">This information can later be displayed in search results.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Why_Is_Indexing_Important\"><\/span>Why Is Indexing Important?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Indexing is essential because it determines whether users can discover your website through search engines.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Benefits of successful indexing include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Increased visibility<\/li>\n\n\n\n<li>More organic traffic<\/li>\n\n\n\n<li>Faster content discovery<\/li>\n\n\n\n<li>Improved search engine presence<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A page that is not indexed cannot appear in search results, regardless of how useful its content may be.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Example\"><\/span>Example<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">When someone searches for:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>wordpress hosting\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">the search engine does not scan the entire internet in real time.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Instead, it searches its previously created index and returns the most relevant indexed pages.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This allows search results to be displayed almost instantly.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_to_Help_Search_Engines_Index_Your_Website\"><\/span>How to Help Search Engines Index Your Website<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">You can improve indexing by:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Creating_an_XML_Sitemap\"><\/span>Creating an XML Sitemap<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A sitemap helps search engines discover important pages more efficiently.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Using_Internal_Links\"><\/span>Using Internal Links<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Linking pages together helps crawlers navigate your website.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Publishing_Quality_Content\"><\/span>Publishing Quality Content<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Search engines prioritize useful, unique, and relevant content.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Avoiding_Crawl_Restrictions\"><\/span>Avoiding Crawl Restrictions<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Make sure important pages are not blocked through:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>robots.txt\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">or meta robots tags.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Submitting_Your_Website_to_Search_Engines\"><\/span>Submitting Your Website to Search Engines<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">You can submit your sitemap using tools such as:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Google Search Console<\/li>\n\n\n\n<li>Bing Webmaster Tools<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Summary\"><\/span>Summary<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Indexing is the process by which search engines collect, analyze, and store information about websites in their databases.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The process involves:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Discovering pages.<\/li>\n\n\n\n<li>Crawling their content.<\/li>\n\n\n\n<li>Analyzing the information.<\/li>\n\n\n\n<li>Storing it in a searchable index.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Once indexed, a page can appear in search results and become accessible to users searching for related information online.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n<\/div>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"template":"","format":"standard","manualknowledgebasecat":[25],"manual_kb_tag":[588],"class_list":["post-988","manual_kb","type-manual_kb","status-publish","format-standard","hentry","manualknowledgebasecat-hosting","manual_kb_tag-website"],"_links":{"self":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb\/988","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb"}],"about":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/types\/manual_kb"}],"author":[{"embeddable":true,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":2,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb\/988\/revisions"}],"predecessor-version":[{"id":4077,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb\/988\/revisions\/4077"}],"wp:attachment":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/media?parent=988"}],"wp:term":[{"taxonomy":"manualknowledgebasecat","embeddable":true,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manualknowledgebasecat?post=988"},{"taxonomy":"manual_kb_tag","embeddable":true,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb_tag?post=988"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}