{"id":11868,"date":"2026-07-01T10:45:44","date_gmt":"2026-07-01T08:45:44","guid":{"rendered":"https:\/\/mybox.com\/help\/?post_type=manual_kb&#038;p=11868"},"modified":"2026-07-01T10:45:46","modified_gmt":"2026-07-01T08:45:46","slug":"cum-sa-impiedici-botilor-ai-sa-acceseze-continutul-site-ului-web","status":"publish","type":"manual_kb","link":"https:\/\/mybox.com\/help\/ro\/knowledgebase\/how-to-block-ai-bots-from-accessing-website-content\/","title":{"rendered":"Cum s\u0103 \u00eempiedica\u021bi boturile bazate pe IA s\u0103 acceseze con\u021binutul site-ului web"},"content":{"rendered":"<div class=\"translation-block translation-block-merged\">\n<p class=\"wp-block-paragraph\">Blocking AI bots works best as a layered setup. One method alone is usually not enough, because some crawlers follow published standards and some do not. A combined setup using robots.txt, meta tags, HTTP headers, newer standards such as TDMRep, and server-side or Cloudflare filtering can reduce access for compliant AI crawlers and make scraping harder for more aggressive bots.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This topic is useful if you publish original content and want to limit how it is crawled or reused by AI systems. The methods below do not all work in the same way. Some communicate your preferences, while others actively deny access.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_86 ez-toc-wrap-left counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/mybox.com\/help\/ro\/knowledgebase\/how-to-block-ai-bots-from-accessing-website-content\/#Block_AI_bots_in_the_panel\" >Block AI bots in the panel<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/mybox.com\/help\/ro\/knowledgebase\/how-to-block-ai-bots-from-accessing-website-content\/#Layer_1_%E2%80%93_robotstxt_for_known_AI_bots\" >Layer 1 &#8211; robots.txt for known AI bots<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/mybox.com\/help\/ro\/knowledgebase\/how-to-block-ai-bots-from-accessing-website-content\/#Layer_2_%E2%80%93_meta_tags_and_noai_headers\" >Layer 2 &#8211; meta tags and noai headers<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/mybox.com\/help\/ro\/knowledgebase\/how-to-block-ai-bots-from-accessing-website-content\/#Layer_3_%E2%80%93_TDMRep_llmstxt_and_aitxt\" >Layer 3 &#8211; TDMRep, llms.txt, and ai.txt<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/mybox.com\/help\/ro\/knowledgebase\/how-to-block-ai-bots-from-accessing-website-content\/#Layer_4_%E2%80%93_server-side_blocking_and_Cloudflare\" >Layer 4 &#8211; server-side blocking and Cloudflare<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/mybox.com\/help\/ro\/knowledgebase\/how-to-block-ai-bots-from-accessing-website-content\/#Layer_5_%E2%80%93_terms_of_use_paywall_and_conditional_access\" >Layer 5 &#8211; terms of use, paywall, and conditional access<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/mybox.com\/help\/ro\/knowledgebase\/how-to-block-ai-bots-from-accessing-website-content\/#robotstxt_meta_tags_and_server_blocks_%E2%80%93_what_is_the_difference\" >robots.txt, meta tags, and server blocks &#8211; what is the difference?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/mybox.com\/help\/ro\/knowledgebase\/how-to-block-ai-bots-from-accessing-website-content\/#How_to_start_in_practice\" >How to start in practice<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Block_AI_bots_in_the_panel\"><\/span>Block AI bots in the panel<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"929\" height=\"494\" src=\"https:\/\/mybox.com\/help\/wp-content\/uploads\/2026\/07\/260701-121119.png\" alt=\"\" class=\"wp-image-11880\" srcset=\"\" sizes=\"auto, (max-width: 929px) 100vw, 929px\" data-srcset=\"\" \/><\/figure>\n<\/div>\n\n\n\n\n<div id=\"mybox-3147639365\" class=\"mybox-content mybox-entity-placement\"><div class=\"early-access-banner-inpost\">\r\n  <div class=\"banner-left-inpost\">\r\n    <div class=\"icon-box-inpost\">\r\n      <img decoding=\"async\" src=\"https:\/\/mybox.com\/help\/wp-content\/uploads\/2026\/02\/square-info-icon.svg\" alt=\"Info\">\r\n    <\/div>\r\n    <div class=\"text-box-inpost\">\r\n      <span class=\"label-inpost\"><span class=\"translation-block translation-block-banner-text\">Acces timpuriu<\/span><\/span>\r\n      <h4><span class=\"translation-block translation-block-banner-text\">Mai ave\u021bi nevoie de ajutor?<\/span><\/h4>\r\n      <p><span class=\"translation-block translation-block-banner-text\">Contacta\u021bi echipa noastr\u0103 de servicii pentru clien\u021bi.<\/span><\/p>\r\n    <\/div>\r\n  <\/div>\r\n\r\n  <div class=\"banner-right-inpost\">\r\n    <a href=\"https:\/\/panel.mybox.com\/helpdesk2\/v\/list\/\" class=\"banner-button-inpost\"><span class=\"translation-block translation-block-banner-text\">Trimite mesaj<\/span><\/a>\r\n  <\/div>\r\n<\/div><\/div>\n\n<div class=\"translation-block translation-block-merged\"><p class=\"wp-block-paragraph\">After signing in to the panel, you can use the built-in AI bot blocking system. If you need other options or want additional protection, use the methods below as separate layers.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Layer_1_%E2%80%93_robotstxt_for_known_AI_bots\"><\/span>Layer 1 &#8211; robots.txt for known AI bots<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The robots.txt file is the basic way to communicate with crawlers. It is placed in the root directory of your website. In this file, you can define rules that block specific AI crawlers such as GPTBot, ClaudeBot, or PerplexityBot. This method works for bots that respect the robots.txt standard, which includes many large model providers.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An example robots.txt section that blocks selected AI bots can look like this:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>User-agent: GPTBot\nDisallow: \/\n\nUser-agent: ChatGPT-User\nDisallow: \/\n\nUser-agent: OAI-SearchBot\nDisallow: \/\n\nUser-agent: ClaudeBot\nDisallow: \/\n\nUser-agent: PerplexityBot\nDisallow: \/\n\nUser-agent: *\nDisallow:\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">With this setup, the listed AI bots should not crawl your site. Other bots can still have full access if that is what you want.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Layer_2_%E2%80%93_meta_tags_and_noai_headers\"><\/span>Layer 2 &#8211; meta tags and noai headers<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A robots.txt rule does not directly state that you object to your content being used for AI training. For that purpose, some sites also use meta tags and X-Robots-Tag headers with directives such as noai and noimageai. These act as an additional signal to AI-related tools and crawlers that you do not allow this type of use.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">To apply this globally, you can add the following meta tag in the head section of your WordPress theme:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"robots\" content=\"noai, noimageai\"&gt;\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If you only want to protect part of the site, such as a premium content directory, you can apply an X-Robots-Tag header on the server side. In Nginx, an example configuration can look like this:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>location \/premium\/ {\n    add_header X-Robots-Tag \"noai, noimageai\" always;\n}\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">With this configuration, requests to the premium directory always receive a header that signals the content should not be used for AI training.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Layer_3_%E2%80%93_TDMRep_llmstxt_and_aitxt\"><\/span>Layer 3 &#8211; TDMRep, llms.txt, and ai.txt<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Newer standards are being developed to help website owners express objections to text and data mining, which means large-scale processing of content by AI tools and models. One example is TDMRep. It uses a simple meta tag or HTTP header to indicate that the site owner does not allow this kind of use.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An example TDMRep meta tag looks like this:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"tdm-reservation\" content=\"1\"&gt;\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">You can also add the equivalent HTTP header:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>TDM-Reservation: 1\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">At the same time, files such as llms.txt and ai.txt are appearing as similar mechanisms. They work in a way that is comparable to robots.txt, but they can also describe the intended use of content, not only access. In these files, you can specify which AI providers may fetch content and under what conditions.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Layer_4_%E2%80%93_server-side_blocking_and_Cloudflare\"><\/span>Layer 4 &#8211; server-side blocking and Cloudflare<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Not all bots follow robots.txt rules or meta directives. Because of that, it can help to add a stricter block at server level or through a CDN provider. In Cloudflare, for example, you can enable a rule in the Security section that blocks known AI bots and filters part of this traffic before it reaches your site.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If your site uses Apache, you can use the .htaccess file to return a 403 response for selected user agents. An example rule looks like this:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>RewriteEngine On\n\nRewriteCond %{HTTP_USER_AGENT} GPTBot &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ChatGPT-User &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} OAI-SearchBot &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ClaudeBot &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} PerplexityBot &#91;NC]\nRewriteRule .* - &#91;F,L]\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Similar rules can be added in Nginx. This allows you to stop requests from specific AI bots even if they ignore robots.txt.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Layer_5_%E2%80%93_terms_of_use_paywall_and_conditional_access\"><\/span>Layer 5 &#8211; terms of use, paywall, and conditional access<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The last layer is not mainly technical. It is about the rules for using your content. In your terms of use and copyright policy, you can state clearly that you do not allow the website content to be used for AI model training or large-scale scraping. This does not block bots by itself, but it creates an additional legal basis if someone ignores your directives.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For especially valuable or sensitive content, you can also consider limiting access to signed-in users, subscriptions, or a paywall. Most bots do not pass through these protections, so they do not have a direct technical path to the full content.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"robotstxt_meta_tags_and_server_blocks_%E2%80%93_what_is_the_difference\"><\/span>robots.txt, meta tags, and server blocks &#8211; what is the difference?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>robots.txt<\/strong><br>Communicates crawl rules to bots that respect the standard.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Meta tags and X-Robots-Tag headers<\/strong><br>Signal how content should be treated, including directives related to AI use.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>TDMRep, llms.txt, and ai.txt<\/strong><br>Express content use preferences in newer or developing formats.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Server-side rules and Cloudflare<\/strong><br>Actively deny or filter requests, including requests from bots that ignore published directives.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_to_start_in_practice\"><\/span>How to start in practice<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A layered approach usually gives the best result. Start with an up-to-date robots.txt file that includes rules for known AI bots. Then add a noai and noimageai meta tag in your template, or apply X-Robots-Tag headers to selected directories.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The next step can be TDMRep and, as these standards develop further, llms.txt and ai.txt. In parallel, enable traffic filtering in tools such as Cloudflare or configure simple rules in .htaccess. It also helps to support the technical setup with clear terms of use and a deliberate access model for your most valuable content.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This setup does not guarantee complete protection. It does, however, make large-scale AI scraping significantly harder and gives clear signals about how your content may be used.<\/p>\n<\/div>","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"template":"","format":"standard","manualknowledgebasecat":[125],"manual_kb_tag":[],"class_list":["post-11868","manual_kb","type-manual_kb","status-publish","format-standard","hentry","manualknowledgebasecat-websites"],"_links":{"self":[{"href":"https:\/\/mybox.com\/help\/ro\/wp-json\/wp\/v2\/manual_kb\/11868","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mybox.com\/help\/ro\/wp-json\/wp\/v2\/manual_kb"}],"about":[{"href":"https:\/\/mybox.com\/help\/ro\/wp-json\/wp\/v2\/types\/manual_kb"}],"author":[{"embeddable":true,"href":"https:\/\/mybox.com\/help\/ro\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":3,"href":"https:\/\/mybox.com\/help\/ro\/wp-json\/wp\/v2\/manual_kb\/11868\/revisions"}],"predecessor-version":[{"id":11870,"href":"https:\/\/mybox.com\/help\/ro\/wp-json\/wp\/v2\/manual_kb\/11868\/revisions\/11870"}],"wp:attachment":[{"href":"https:\/\/mybox.com\/help\/ro\/wp-json\/wp\/v2\/media?parent=11868"}],"wp:term":[{"taxonomy":"manualknowledgebasecat","embeddable":true,"href":"https:\/\/mybox.com\/help\/ro\/wp-json\/wp\/v2\/manualknowledgebasecat?post=11868"},{"taxonomy":"manual_kb_tag","embeddable":true,"href":"https:\/\/mybox.com\/help\/ro\/wp-json\/wp\/v2\/manual_kb_tag?post=11868"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}