Definition:
Google BERT is a language model developed by Google that represents words by taking into account the context in which they appear. Its name stands for Bidirectional Encoder Representations from Transformers.
BERT is used in natural language processing tasks. Google also applies this technology in Search to understand how combinations of words express different meanings and intentions.
Table of contents
Origins of BERT
Google introduced BERT in 2018 and released its code and pre-trained models to support its use in research and development.
Its contribution built on the transformer architecture and the training of contextual language representations. Rather than always assigning the same representation to a word, BERT takes the surrounding words into account.
In October 2019, Google announced the application of BERT models to ranking results and featured snippets. It is important to distinguish these dates: the model was developed before it was incorporated into the search engine.
How bidirectional understanding works
BERT uses information before and after a word to represent its meaning within a sequence. It does not simply read a sentence twice in opposite directions; it considers the relationships between its elements together.
For example, the Spanish word “banco” does not mean the same thing in “sentarse en un banco” (sitting on a bench) as in “abrir una cuenta en el banco” (opening an account at the bank). Context distinguishes these uses without treating the word as a unit with a fixed meaning.
During the original pre-training process, one task involves masking certain elements of the text and learning to predict them from those that remain visible. Another works with the relationship between pairs of sentences.
The model can then be adapted to a specific task through fine-tuning. This adaptation requires additional training; it does not happen automatically when BERT is used to analyse a text.
What BERT is used for
BERT provides language representations that can be used in different applications. Its uses include the following:
- Text classification: assigning categories to documents or messages.
- Sentiment analysis: identifying the sentiment expressed in a text, according to the categories used during fine-tuning.
- Entity recognition: locating references to people, organisations, places or other elements.
- Question answering: identifying a passage in a document that answers a query.
These functions depend on the adaptation and the data used. The original BERT is not a conversational assistant designed to write lengthy responses like today’s generative systems.
How Google uses BERT in Search
In the search engine, BERT helps interpret the relationships between the words in a query. This is particularly relevant when a preposition, a negation or the order of the terms changes what is being asked.
In the announcement of its introduction to Search, Google explained how this technology helped distinguish a query about someone travelling from Brazil to the United States from one about the opposite journey. The countries were the same, but the relationship between them changed the information needed.
BERT is part of a collection of systems. It does not, on its own, replace link analysis, relevance assessment or the other technologies involved in search results.
It also differs from RankBrain: Google describes the latter as a system that relates words to concepts, while BERT helps understand how their combinations express meanings and intentions. These are related functions, not interchangeable names.
What BERT means for SEO
The application of BERT reinforces the importance of clearly expressing what a page explains and which need it addresses. Descriptive titles, precise terminology and well-explained relationships help communicate the content without turning it into a collection of keywords.
This does not mean that there is a keyword density, sentence length or special structure that guarantees better rankings through BERT. Nor does it require every page to be turned into questions and answers.
BERT is not a penalty or a score that a website owner can look up. A change in traffic is not enough to establish BERT as the cause: the queries, pages and other circumstances of the site must be examined.
The editorial priority remains explaining the topic accurately and addressing the reader’s needs, without introducing artificial wording to try to activate a particular system.