Embeddings in advertising: how a page becomes a point in space
Part of Contextual targeting
The short answer
An embedding is a list of numbers that a language model produces for a piece of text, placing it as a point in a space where texts with similar meaning sit close together. In advertising, embeddings let a planner compare an article with a description of the ideal page by meaning, even when the two share no keywords.
Key takeaways
- An embedding turns text into coordinates, and nearness between coordinates stands in for nearness in meaning.
- Similarity can be measured by the angle between two vectors, and a score is only comparable with other scores from the same model.
- A multilingual model can place an English description and a French article in the same neighborhood, which matters for bilingual Canadian plans.
- Embeddings capture what a page is about, not whether it is true, well made or suitable, and they can miss negation and carry bias from their training text.
What is a vector embedding in plain words?
Definition
- Vector embedding
- A fixed-length list of numbers that a trained model assigns to a word, sentence or document, so that texts used in similar ways get similar lists.
Think of a map. Every article gets a pin, and the model decides where the pin goes from the words and how they relate. Articles about preserving fruit land in one neighborhood, articles about trade policy in another, television news in a third. Real models use hundreds or thousands of directions rather than two, and no single direction has a tidy label, but the idea holds: closeness on the map stands in for closeness in meaning.
The idea grew from work on single words. Mikolov and colleagues at Google described, in their 2013 paper on word representations in vector space (opens in a new tab), model architectures for computing continuous vector representations of words from very large data sets, and measured quality with a word similarity task. Later models embed whole sentences and passages, which is what contextual advertising needs, since a buyer cares about what an article says rather than which words it contains.
| Text | Food preserving | Television | Trade policy |
|---|---|---|---|
| Guide to water bath canning for jam | High | Low | Low |
| Putting up the summer's blueberries for winter | High | Low | Low |
| Network cans its weekend cooking show | Low | High | Low |
| Tariffs raise the price of canned goods | Medium | Low | High |
How is similarity between two articles measured?
Once two texts are vectors, similarity is arithmetic. One standard measure is cosine similarity: draw an arrow from the origin to each point and measure the angle between the arrows. Arrows pointing the same way score near 1, unrelated directions score near 0. The Sentence-BERT paper by Reimers and Gurevych (2019) (opens in a new tab) describes producing semantically meaningful sentence embeddings that can be compared using cosine similarity, and reports cutting the time to find the most similar pair in a collection from 65 hours to about 5 seconds, which is what makes comparing many pages practical.
Other measures exist. Straight line distance between points works when vectors have similar lengths. The dot product rewards both direction and length. For a planner the choice matters less than consistency: the brief and every candidate page must be embedded by the same model and compared by the same measure.
What does a multilingual embedding model change for Canadian plans?
A multilingual model is trained so that the same meaning lands in the same place whatever the language. Reimers and Gurevych's 2020 paper on making sentence embeddings multilingual (opens in a new tab) states the goal directly: a translated sentence should be mapped to the same location in the vector space as the original sentence, and the authors demonstrate it for more than 50 languages. In practice an English description of the ideal article can find French pages, and the reverse, without a translated keyword list.
That matters where both languages are part of daily media. The province's Office of the Commissioner of Official Languages (opens in a new tab) notes that New Brunswick's first Official Languages Act made it Canada's first, and only, officially bilingual province. A plan for a New Brunswick brief that reads only English pages leaves out the province's French language publishers; one that compares both languages in one space can rank them on the same footing.
- Quality varies by language. A model trained mostly on English and European French may place Acadian or Quebec usage less precisely, so a fluent reader should check French matches.
- Regional words matter. Canadian French terms for everyday things can differ from European usage, and a model's grasp of them depends on its training text.
- One description, two languages. The ideal article can be written once, but the avoid topics should be tested on pages in both languages.
- Language still has to be recorded. A match in the shared space says nothing about which language the creative must be in, so record each page's language separately.
What can embeddings get wrong about a page?
Embeddings compress a page into one point, and compression loses things. The failures below come from how the vectors are made, not from any one product, which is why the checks are the same whatever system a team uses. Broader method misreads are covered in our guide to semantic targeting.
| Limit | Why it happens | Check |
|---|---|---|
| Negation | A page saying a method is not safe sits close to one saying it is; Ettinger (2019) found BERT shows clear insensitivity to the contextual impacts of negation | Read the claim in matched passages, not only the topic |
| Truth and quality | A vector records what a page is about, not whether it is accurate or well written | Judge editorial quality and made-for-ads risk separately |
| Learned bias | Models absorb associations from training text; Bolukbasi and colleagues (2016) found gender stereotypes in news-trained word embeddings to a disturbing extent | Review which publishers and topics a brief leaves out, not only what it finds |
| New words and names | A product, slang term or event that appeared after training may be placed by its spelling, not its meaning | Add exact names to the brief and check those pages by hand |
| Numbers and specifics | Two reviews of different model years or prices can sit almost on top of each other | Use filters or human review for dates, prices and specifications |
| Suitability | A sensitive news story about the right subject is close to the brief by design | Run suitability checks after the topic match, never instead of it |
How does PlacementGraph use embeddings?
What should a planner ask about an embedding based result?
Questions before trusting a vector match
- Which text was embedded: the article body, the whole page or only the title and description?
- Were the brief and the pages embedded by the same model and compared by the same measure?
- Does the model cover every language in the brief, and has a fluent reader checked matches in each one?
- Are the matching passages shown, so a score can be checked against the words on the page?
- Were negation, dates and product names checked by hand on the top matches?
- Did suitability and quality checks run separately from the topic match?
Frequently asked questions
Do embeddings use personal data?
Not by themselves. An embedding of an article is built from the article's text, with no information about who reads it. Personal data enters a campaign only through other layers, such as audience segments, tracking pixels or conversion tags, and those need their own privacy review.
Can an embedding represent a whole website?
It can, but a single site vector blurs everything the site covers into one point. Comparing several sampled articles with the brief, and counting how many match, stops a broad publisher whose coverage only touches the topic once from looking like a specialist. Site level vectors work better as a first filter than as a final judgment.
Does a longer brief description give better matches?
Not necessarily. A description that covers several subjects is embedded as one point, which lands somewhere between them and may match none well. Write one short description per topic, phrased like the opening of the ideal article, and embed each one separately. Then compare candidate pages with every topic vector rather than with a blended one.
Are embeddings the same as AI generated content?
No. An embedding model reads text and outputs numbers; it does not write articles. The same family of language models can do both jobs, which causes the confusion. Detecting machine-written filler is a separate question about page quality, covered in our guide to AI generated content sites.
Sources
- Efficient Estimation of Word Representations in Vector Space (opens in a new tab)Mikolov, Chen, Corrado and Dean, arXiv, 2013
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (opens in a new tab)Reimers and Gurevych, EMNLP 2019
- Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation (opens in a new tab)Reimers and Gurevych, EMNLP 2020
- MTEB: Massive Text Embedding Benchmark (opens in a new tab)Muennighoff, Tazi, Magne and Reimers, arXiv, 2022
- What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models (opens in a new tab)Ettinger, arXiv, 2019
- Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings (opens in a new tab)Bolukbasi, Chang, Zou, Saligrama and Kalai, arXiv, 2016
- History of official languages in New Brunswick (opens in a new tab)Office of the Commissioner of Official Languages for New Brunswick
Researched and written by PlacementGraph Research. Read how we research, source and update these pages. Corrections: diana@placementgraph.com.