{"id":20553,"date":"2020-01-30T13:00:32","date_gmt":"2020-01-30T13:00:32","guid":{"rendered":"https:\/\/www.arimetrics.com\/glosario-digital\/googlebot"},"modified":"2026-06-26T15:47:03","modified_gmt":"2026-06-26T15:47:03","slug":"googlebot","status":"publish","type":"encyclopedia","link":"https:\/\/www.arimetrics.com\/en\/digital-glossary\/googlebot","title":{"rendered":"GoogleBot"},"content":{"rendered":"<p><b>Definition:<i><img decoding=\"async\" class=\"boxpad wp-image-14415 size-full alignright\" src=\"https:\/\/www.arimetrics.com\/wp-content\/uploads\/2020\/01\/google_bot.jpg\" alt=\"google bot\" width=\"300\" height=\"300\" srcset=\"https:\/\/www.arimetrics.com\/wp-content\/uploads\/2020\/01\/google_bot.jpg 300w, https:\/\/www.arimetrics.com\/wp-content\/uploads\/2020\/01\/google_bot-150x150.jpg 150w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/i><\/b><\/p>\n<p><b><i>Googlebot <\/i><\/b>is Google&#8217;s web crawling robot, allowing you to search, add, scan, and index new web pages. Crawling is the process by which Googlebot discovers new and updated pages and adds them to Google&#8217;s index.<\/p>\n<p>In addition to locating and indexing web pages, Googlebot also indexes files, such as .doc formats, .zip, xls, etc.<\/p>\n\n<h2 class=\"indexed\"><span class=\"heading_text\">How Googlebot works<\/span><\/h2>\n<p>Googlebot works through an algorithm that tells you which web pages to crawl, downloading those pages to a database so that they can later be interpreted by the search engine. As it progresses in its tracking, it follows the routes marked by the links, achieving greater and greater depth. All this is done from Google&#8217;spowerful servers, spread throughout the planet.<\/p>\n<p>The frequency with which <a href=\"https:\/\/www.google.com\/about\/datacenters\/locations\/\" target=\"_blank\" rel=\"noopener\">Google<\/a> visits the different pages that make up the Internet network depends on the importance that Google assigns to those pages. The more authority the page has, the more frequency and depth of crawling it will have from Googlebot.<\/p>\n<h2 class=\"indexed\"><span id=\"versiones-de-googlebot\" class=\"anchor\"><\/span><span class=\"heading_text\">Versions of Googlebot<\/span><\/h2>\n<p>There are two versions of Google&#8217;s bot:<\/p>\n<ul>\n<li><em>Deepbot<\/em>\u2013 crawls the web in depth to include it in Google&#8217;s cache.<\/li>\n<li><em>Freshbot<\/em>&#8211; search for new content. It does this more frequently on sites that are updated regularly (such as media), and less frequently (days or weeks) on sites with few updates.<\/li>\n<\/ul>\n<h2 class=\"indexed\"><span id=\"como-saber-si-googlebot-visita-una-web\" class=\"anchor\"><\/span><span class=\"heading_text\">How to know if Googlebot visits a website<\/span><\/h2>\n<p>The tracking by Googlebot is essential to be able to achieve a <a href=\"https:\/\/www.arimetrics.com\/en\/seo-agency-spain\">positioning in Google<\/a>.\u00a0 That is why sometimes, when the indexing of a website is not achieved, the first step should be to check if the Googlebot has managed to access that website.<\/p>\n<p>When Googlebot visits a website, it leaves in the log a record of the type <em>crawl1.googlebot.com,<\/em>with its corresponding ip. If there are no Googlebot logs, it is advisable to perform a review of the <a href=\"https:\/\/www.arimetrics.com\/en\/digital-glossary\/robots-txt\">robots.txt<\/a> file.<\/p>\n<h2 class=\"indexed\"><span id=\"rastreadores-de-google\" class=\"anchor\"><\/span><span class=\"heading_text\">Google Crawlers<\/span><\/h2>\n<p>We can identify which <a href=\"https:\/\/support.google.com\/webmasters\/answer\/1061943?hl=es\" target=\"_blank\" rel=\"noopener\">google crawler<\/a> has visited a page by viewing the user-agent in the log:<\/p>\n<ul>\n<li>APIs-Google: <em>APIs-Google<\/em><\/li>\n<li>Adsense: <em>Mediapartners-Google<\/em><\/li>\n<li>Adsbot for Mobile Web: <em>Adsbot-Google-Mobile<\/em><\/li>\n<li>Adsbot:<em> Adsbot-Google<\/em><\/li>\n<li>Googlebot:<em> Googlebot<\/em><\/li>\n<li>Googlebot for images:<em> Googlebot-Image<\/em><\/li>\n<li>Googlebot for news: <em>Googlebot-News<\/em><\/li>\n<li>GoogleBot for video: Googlebot-Video<\/li>\n<li>Feedtecher: <em>Feedfecher-Google<\/em><\/li>\n<li>Google Read Aloud:<em> Google-Read-Aloud<\/em><\/li>\n<\/ul>\n<h2 class=\"indexed\"><span id=\"como-bloquear-a-googlebot\" class=\"anchor\"><\/span><span class=\"heading_text\">How to block Googlebot<\/span><\/h2>\n<p>There are times when for privacy reasons you do not want certain content to appear on Google. In this case, various actions can be carried out:<\/p>\n<ul>\n<li>Password protect directories: Googlebot will never crawl password-protected sites.<\/li>\n<li>Use <em>a &#8220;noindex&#8221;<\/em> tag on html pages that you do not want to index.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h2>Frequently asked questions about GoogleBot<\/h2>\n<div class=\"geo-faq-block\">\n<details class=\"geo-faq-item\">\n<summary>What is GoogleBot?<\/summary>\n<p>Definition: Googlebot is Google&#x27;s web crawling robot, allowing you to search, add, scan, and index new web pages. Crawling is the process by which Googlebot discovers new and updated pages and adds them to Google&#x27;s index. In the Arimetrics glossary it is placed in a digital marketing context to clarify its role, uses and practical implications.<\/p>\n<\/details>\n<details class=\"geo-faq-item\">\n<summary>What is GoogleBot used for in digital marketing?<\/summary>\n<p>It is used to better analyse an action, tool, channel or behaviour related to acquisition, measurement, communication, sales or user experience. Its value depends on applying it to a concrete decision.<\/p>\n<\/details>\n<details class=\"geo-faq-item\">\n<summary>How is GoogleBot related to a digital strategy?<\/summary>\n<p>It is related to digital strategy when it affects objectives, data, content, technology, campaigns or conversion processes. That is why it should be reviewed together with the business context, not as an isolated term.<\/p>\n<\/details>\n<details class=\"geo-faq-item\">\n<summary>What should be considered when working with GoogleBot?<\/summary>\n<p>It is advisable to review its definition, context, associated metrics, limitations and possible risks. It is also useful to validate whether the concept has a real impact on performance, user experience or decision-making.<\/p>\n<\/details>\n<\/div>\n<p><script type=\"application\/ld+json\">\n{\n  \"@context\": \"https:\/\/schema.org\",\n  \"@graph\": [\n    {\n      \"@type\": \"DefinedTerm\",\n      \"@id\": \"https:\/\/www.arimetrics.com\/en\/digital-glossary\/googlebot#definedterm\",\n      \"name\": \"GoogleBot\",\n      \"description\": \"Definition of GoogleBot in the Arimetrics Digital Glossary.\",\n      \"inDefinedTermSet\": {\n        \"@type\": \"DefinedTermSet\",\n        \"name\": \"Arimetrics Digital Glossary\",\n        \"url\": \"https:\/\/www.arimetrics.com\/en\/digital-glossary\"\n      }\n    },\n    {\n      \"@type\": \"FAQPage\",\n      \"@id\": \"https:\/\/www.arimetrics.com\/en\/digital-glossary\/googlebot#faq\",\n      \"mainEntity\": [\n        {\n          \"@type\": \"Question\",\n          \"name\": \"What is GoogleBot?\",\n          \"acceptedAnswer\": {\n            \"@type\": \"Answer\",\n            \"text\": \"Definition: Googlebot is Google's web crawling robot, allowing you to search, add, scan, and index new web pages. Crawling is the process by which Googlebot discovers new and updated pages and adds them to Google's index. In the Arimetrics glossary it is placed in a digital marketing context to clarify its role, uses and practical implications.\"\n          }\n        },\n        {\n          \"@type\": \"Question\",\n          \"name\": \"What is GoogleBot used for in digital marketing?\",\n          \"acceptedAnswer\": {\n            \"@type\": \"Answer\",\n            \"text\": \"It is used to better analyse an action, tool, channel or behaviour related to acquisition, measurement, communication, sales or user experience. Its value depends on applying it to a concrete decision.\"\n          }\n        },\n        {\n          \"@type\": \"Question\",\n          \"name\": \"How is GoogleBot related to a digital strategy?\",\n          \"acceptedAnswer\": {\n            \"@type\": \"Answer\",\n            \"text\": \"It is related to digital strategy when it affects objectives, data, content, technology, campaigns or conversion processes. That is why it should be reviewed together with the business context, not as an isolated term.\"\n          }\n        },\n        {\n          \"@type\": \"Question\",\n          \"name\": \"What should be considered when working with GoogleBot?\",\n          \"acceptedAnswer\": {\n            \"@type\": \"Answer\",\n            \"text\": \"It is advisable to review its definition, context, associated metrics, limitations and possible risks. It is also useful to validate whether the concept has a real impact on performance, user experience or decision-making.\"\n          }\n        }\n      ]\n    }\n  ]\n}\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Definition: Googlebot is Google&#8217;s web crawling robot, allowing you to search, add, scan, and index new web pages. Crawling is the process by which Googlebot discovers new and updated pages and adds them to Google&#8217;s index. In addition to locating and indexing web pages, Googlebot also indexes files, such as .doc formats, .zip, xls, etc. [&hellip;]<\/p>\n","protected":false},"author":6,"featured_media":0,"template":"","encyclopedia-tag":[1004],"class_list":["post-20553","encyclopedia","type-encyclopedia","status-publish","hentry","encyclopedia-tag-indexacion-seo"],"_links":{"self":[{"href":"https:\/\/www.arimetrics.com\/en\/wp-json\/wp\/v2\/encyclopedia\/20553","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.arimetrics.com\/en\/wp-json\/wp\/v2\/encyclopedia"}],"about":[{"href":"https:\/\/www.arimetrics.com\/en\/wp-json\/wp\/v2\/types\/encyclopedia"}],"author":[{"embeddable":true,"href":"https:\/\/www.arimetrics.com\/en\/wp-json\/wp\/v2\/users\/6"}],"wp:attachment":[{"href":"https:\/\/www.arimetrics.com\/en\/wp-json\/wp\/v2\/media?parent=20553"}],"wp:term":[{"taxonomy":"encyclopedia-tag","embeddable":true,"href":"https:\/\/www.arimetrics.com\/en\/wp-json\/wp\/v2\/encyclopedia-tag?post=20553"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}