{"id":279,"date":"2025-12-17T10:47:31","date_gmt":"2025-12-17T09:47:31","guid":{"rendered":"https:\/\/mybox.com\/help\/?post_type=manual_kb&#038;p=279"},"modified":"2026-02-11T06:37:36","modified_gmt":"2026-02-11T05:37:36","slug":"how-do-i-block-a-wordpress-site-from-being-indexed","status":"publish","type":"manual_kb","link":"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-do-i-block-a-wordpress-site-from-being-indexed\/","title":{"rendered":"How do I block a WordPress site from being indexed?"},"content":{"rendered":"\n<div class=\"translation-block translation-block-merged\">\n<p class=\"wp-block-paragraph\">Securing your WordPress site from being crawled by search engine bots can be crucial in certain situations. Below, we discuss why it is worth doing it and how to implement such a solution step by step.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_86 ez-toc-wrap-left counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-do-i-block-a-wordpress-site-from-being-indexed\/#Why_is_it_important_to_block_a_page_from_indexing\" >Why is it important to block a page from indexing?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-do-i-block-a-wordpress-site-from-being-indexed\/#How_do_I_block_a_WordPress_site_from_being_indexed\" >How do I block a WordPress site from being indexed?<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-do-i-block-a-wordpress-site-from-being-indexed\/#Settings_in_the_WordPress_panel\" >Settings in the WordPress panel<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-do-i-block-a-wordpress-site-from-being-indexed\/#Editing_a_robotstxt_file\" >Editing a robots.txt file<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-do-i-block-a-wordpress-site-from-being-indexed\/#htaccess_file_settings\" >.htaccess file settings<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-do-i-block-a-wordpress-site-from-being-indexed\/#Using_WordPress_plugins\" >Using WordPress plugins<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/mybox.com\/help\/en\/knowledgebase\/how-do-i-block-a-wordpress-site-from-being-indexed\/#Summary\" >Summary<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Why_is_it_important_to_block_a_page_from_indexing\"><\/span>Why is it important to block a page from indexing?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Blocking a page from being indexed may be justified in many cases, such as:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Pages under construction or testing<\/strong>: If your site is in development, indexing it in search engines could result in incomplete content, which will negatively impact your company&#8217;s image.<\/li>\n\n\n\n<li><strong>Confidential or internal content<\/strong>: Some pages or subpages may contain data for the internal use of the company or users with permissions.<\/li>\n\n\n\n<li><strong>Avoid SEO penalties for duplicating content<\/strong>: If there are copies of content available on other pages on the page (e.g., for testing purposes), indexing can lead to problems with cannibalization or duplication of content in search results.<\/li>\n\n\n\n<li><strong>Managing website traffic<\/strong>: Pages can generate excessive traffic from search engine bots, which in the case of limited server resources can negatively affect the accessibility of the website for users.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_do_I_block_a_WordPress_site_from_being_indexed\"><\/span>How do I block a WordPress site from being indexed?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<\/div>\n\n<div id=\"mybox-2529664919\" class=\"mybox-content mybox-entity-placement\"><div class=\"early-access-banner-inpost\">\r\n  <div class=\"banner-left-inpost\">\r\n    <div class=\"icon-box-inpost\">\r\n      <img decoding=\"async\" src=\"https:\/\/mybox.com\/help\/wp-content\/uploads\/2026\/02\/square-info-icon.svg\" alt=\"Info\">\r\n    <\/div>\r\n    <div class=\"text-box-inpost\">\r\n      <span class=\"label-inpost\"><span class=\"translation-block translation-block-banner-text\">Early access<\/span><\/span>\r\n      <h4><span class=\"translation-block translation-block-banner-text\">Still need help?<\/span><\/h4>\r\n      <p><span class=\"translation-block translation-block-banner-text\">Contact our customer service team.<\/span><\/p>\r\n    <\/div>\r\n  <\/div>\r\n\r\n  <div class=\"banner-right-inpost\">\r\n    <a href=\"https:\/\/panel.mybox.com\/helpdesk2\/v\/list\/\" class=\"banner-button-inpost\"><span class=\"translation-block translation-block-banner-text\">Message us<\/span><\/a>\r\n  <\/div>\r\n<\/div><\/div>\n\n<div class=\"translation-block translation-block-merged\"><p class=\"wp-block-paragraph\">There are several ways to effectively block your WordPress website from being indexed by search engine bots.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Settings_in_the_WordPress_panel\"><\/span>Settings in the WordPress panel<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Log in to your WordPress admin panel.<\/li>\n\n\n\n<li>Go to <strong>Settings &gt; Reading<\/strong>.<\/li>\n\n\n\n<li>Check <strong>Ask search engines not to index this site<\/strong>.<\/li>\n\n\n\n<li>Save your changes.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">This is the easiest way, but it&#8217;s worth remembering that some bots may ignore this tip. Therefore, in more advanced cases, additional steps are necessary.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Editing_a_robotstxt_file\"><\/span>Editing a robots.txt file<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A <strong>robots.txt<\/strong> file is a tool that allows you to target the traffic of search engine bots on your website. To block an entire page from being indexed, follow these steps:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1. In the root directory of your WordPress installation, find the <strong>robots.txt<\/strong> file. If it doesn&#8217;t exist, create a new file.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">2. Add the following content to the file:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>User-agent: * Disallow: \/<\/code><\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">3. Save the file and make sure it&#8217;s accessible to bots (through the correct file permissions).<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"htaccess_file_settings\"><\/span>.htaccess file settings<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The <strong>.htaccess<\/strong> file can be used for more advanced bot blocking. Here&#8217;s an example configuration:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">1. Open the <strong>.htaccess<\/strong> file in the root directory of WordPress.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">2. Add the following code:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>RewriteEngine On RewriteCond %{HTTP_USER_AGENT} googlebot &#91;NC] RewriteRule .* - &#91;F,L] <\/code><\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">In this case, only the Google bot will be blocked. To block other bots, you need to add the appropriate conditions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">3. Save your changes and test the operation.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Using_WordPress_plugins\"><\/span>Using WordPress plugins<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">There are many plugins that make it easy to manage bots&#8217; access to the site. Popular ones include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Yoast SEO<\/strong>: The plugin allows you to set indexing at the level of individual subpages or the entire site.<\/li>\n\n\n\n<li><strong>All in One SEO Pack<\/strong>: Offers bot blocking and robots.txt file management features.<\/li>\n\n\n\n<li><strong>WP Robots Txt<\/strong>: A simple plugin for editing robots.txt file without having to access the server directly.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Summary\"><\/span>Summary<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Blocking a WordPress site from being crawled by search engine bots is an essential part of website management in many cases, such as protecting sensitive content, preventing SEO errors, or protecting against unwanted traffic. Depending on the user&#8217;s needs and level of proficiency, methods such as settings in the WordPress dashboard, editing the robots.txt file, using plugins, or implementing authentication can be used. Proper configuration not only increases the security of the website, but also allows you to avoid problems related to its visibility on the web.<\/p>\n<\/div>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"template":"","format":"standard","manualknowledgebasecat":[25,58],"manual_kb_tag":[3727,3728,3729,3730,190,1292,1628,2796,3721,3722,3723,3724,3725,3726],"class_list":["post-279","manual_kb","type-manual_kb","status-publish","format-standard","hentry","manualknowledgebasecat-hosting","manualknowledgebasecat-web-applications-cms","manual_kb_tag-search-engines","manual_kb_tag-wordpress-settings","manual_kb_tag-robots-file","manual_kb_tag-search-engine-bots","manual_kb_tag-wordpress","manual_kb_tag-server-configuration","manual_kb_tag-htaccess-rules","manual_kb_tag-wordpress-dashboard","manual_kb_tag-block-indexing","manual_kb_tag-prevent-indexing","manual_kb_tag-robots-txt","manual_kb_tag-noindex-tag","manual_kb_tag-page-noindex","manual_kb_tag-block-crawlers"],"_links":{"self":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb\/279","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb"}],"about":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/types\/manual_kb"}],"author":[{"embeddable":true,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":1,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb\/279\/revisions"}],"predecessor-version":[{"id":5342,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb\/279\/revisions\/5342"}],"wp:attachment":[{"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/media?parent=279"}],"wp:term":[{"taxonomy":"manualknowledgebasecat","embeddable":true,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manualknowledgebasecat?post=279"},{"taxonomy":"manual_kb_tag","embeddable":true,"href":"https:\/\/mybox.com\/help\/en\/wp-json\/wp\/v2\/manual_kb_tag?post=279"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}