2026-09-20 · 6 min read
The model was fluent, confident, and wrong
lepochi just learned Spanish, Portuguese and Italian. Six thousand words each, imported from Wiktionary and ranked by how often they turn up in film subtitles, which is a decent proxy for how often somebody says them out loud.
That import leaves one honest gap. Wiktionary's English edition defines words in English, and its Spanish edition is a much thinner extract — so every word arrived with an English definition sitting in the Spanish slot. Which means the single most likely user of the feature, a Spanish speaker learning Portuguese, opens the app and reads English.
Closing that gap is 18,000 short translations. Too many to write by hand, too few to be interesting, and exactly the shape of work a small local model should be good at. There's a box with a graphics card that renders images overnight and sits idle the rest of the day. It can do this between pictures.
So I asked it for two things per word: the Spanish translation, and one short sentence telling a Spanish speaker the most useful thing about that word.
What it said
The first batch came back in confident, fluent Spanish, with emoji.
não — Es un falso amigo: "não" significa "no" (negación), mientras que "no" en español es "noche".
Spanish no does not mean noche. It means no. That is the kind of error that doesn't look like an error — it's grammatical, it's plausible, it's formatted exactly like the eleven correct answers around it, and a learner has no way to catch it.
estar — Es un falso amigo: "estar" en portugués indica ubicación o estado temporal.
estar is the same word in both languages. There is no false friend. It had invented the warning and then invented a justification for it.
Three of the first twelve were wrong this way. Not garbled — wrong.
Where the mistake actually was
The obvious reading is "small models hallucinate, use a bigger one." I don't think that's the lesson, because the thing I'd asked for was the problem.
I had asked it to recall what a Spanish word means. That's a lookup against something it half-remembers from training, with no signal distinguishing a fact it knows from one it's assembling. It can't tell the difference, so it can't decline.
And I didn't need it to remember. I had both word lists sitting on disk — six thousand Spanish words and six thousand Portuguese ones, each with an English gloss. The relationship between pais and país isn't something to be recalled. It's something to be computed.
So I wrote the check: find a Spanish word spelled the same, compare the two English glosses, and hand the answer to the model as a fact to phrase rather than a question to answer.
Three rounds of being wrong more quietly
The detector had to be beaten into shape, and every round was found by reading its output rather than by a test.
Round one. I compared "content words", dropping grammatical filler like of, the, and. For function words the gloss is entirely filler, so the content set came out empty on both sides, zero overlap, and every one of them looked like a false friend. It flagged Portuguese de against Spanish de. Both mean "of".
The bug is a category error I'd written without noticing. No overlap is not the same as different. An empty comparison means I can't tell, and the right output for I can't tell is silence.
Round two. I'd filtered out words shorter than three letters as noise. "Go" is two letters, and "go" is the entire meaning of ir. So ir was a false friend of ir.
Round three. I had spelling rules for regular correspondences — Portuguese -ção to Spanish -ción, Italian gli to Spanish ll. Useful for spotting cognates, where a shared meaning confirms the link. On their own they manufacture lookalikes: Italian mese (month) matched Spanish mesa (table) because one of my rules turned a final e into an a. Two words that no learner would ever confuse, presented as a trap.
Now a false-friend candidate has to match letter for letter before any rule runs.
Then it copied its homework
Detector fixed. I handed the model verified facts in brackets and asked it to write them as a sentence.
It printed the brackets.
que — 🤔 [VERIFIED: same as Spanish "que"]
That went out to ten words as the learner-facing text, emoji and all.
So I made the instruction sharper: this is a note to you, not the answer. And it did write real sentences:
meu — 👤📖 Este libro es mío.
A perfectly good Spanish sentence, correctly punctuated, and completely useless. The reader is a Spanish speaker. They already know what este libro es mío means. It says nothing at all about the Portuguese word meu.
Three attempts, three different ways of being wrong, each one a reasonable response to what I'd literally asked for.
The fix was to stop asking
I already had the fact. I'd built the detector precisely because the model couldn't be trusted with it. At that point, asking the model to phrase the fact was still handing it a job it could fail — and it kept finding new ways.
Writing the sentence from a known fact isn't a language problem. It's a template:
cognate → Igual que el español «estar».
false friend → Ojo: se escribe como el español «país»,
pero significa «padres».Correct every time. Same wording every time, which matters more than it sounds — a learner seeing the same phrasing twice learns the pattern instead of re-reading the sentence. It cost nothing to regenerate every hook already written, with no model call at all. And it's fast enough that the hooks appear the moment the translations land, instead of queueing behind a graphics card.
The model kept exactly one job: translate a word into Spanish. It's very good at that.
The part I didn't expect
There was one case my data genuinely could not settle.
Portuguese lindo is glossed "beautiful". Spanish lindo is glossed "pretty". Same spelling, no shared word between the two definitions — so my detector flagged a false friend. It isn't one. They're the same word; two dictionaries just reached for different synonyms.
No amount of gloss comparison fixes that. Deciding whether beautiful and pretty are the same idea needs language understanding.
Which the model has. So the test became: ask it to translate lindo into Spanish, and look at what comes back. It says lindo. Same word. Not a false friend.
That's the same model that couldn't tell me what no means, quietly resolving a case that defeated the deterministic check.
Recall is the unreliable half. Judgement isn't.
That's the whole thing, and it's narrower and more useful than "models hallucinate".
"What does the Spanish word no mean?" is recall — retrieval from weights, with no internal signal for I'm not sure, and a fluent wrong answer is indistinguishable from a right one.
"Here are two definitions. Are they the same idea?" is judgement — the evidence is in front of it, the answer is a comparison rather than a retrieval, and being wrong looks wrong.
Both are "ask the LLM". Only one is reliable, and the difference isn't the model's size. It's whether the facts are in the prompt or in the weights.
I'm about to need this again. These languages want graded short stories — a few hundred words recombined into something worth reading, which is real generation and the part where a model earns its place. The rule transfers intact: let it write the prose, and check every word it used against the list it was given. Invention where invention is the point, verification where truth is.
A hallucinated mnemonic is a bad hint. A hallucinated word is a sentence somebody says out loud in a shop.