你的本地 LLM 在輸出陸式用語嗎？Gemma 4 e2b vs 12b 繁中偏移審計Is Your Local LLM Quietly Outputting Mainland Vocabulary? Auditing Gemma 4 e2b vs 12b

Setup

Prompt design:

• All Traditional Chinese; deliberately avoid using any mainland term in the prompt itself (otherwise we'd seed the model)
• Each prompt designed to elicit specific vocabulary: "How do I contact Apple support about a problem?" targets 聯繫; "How do I hook up the printer to my Mac?" targets 打印
• 20 prompts cover 17 hard terms + some soft

Hard terms (from knowledge/master/prohibited.md):

維權 (rights advocacy), 信息 (info), 軟件 (software), 硬件 (hardware), 視頻 (video), 聯繫 (contact), 打印 (print), 內存 (memory), 固件 (firmware), 屏幕 (screen), 鼠標 (mouse), 賬號 (account), 默認 (default), 移動端 (mobile), 互聯網 (internet), 雲計算 (cloud computing), 服務器 (server)

Soft terms (context-dependent — logged but not counted as violations):

搞定, 用戶, 程序, 通過, 質量, 客戶端

Audit method: for each output, run text.count(term) for every hard and soft term, record frequencies. Any prompt with ≥1 hard hit counts as "drifted from the Taiwan baseline."

Script: tests/scripts/run_audit.py. Prompts: tests/data/prompts.json. Results: tests/outputs/responses.jsonl, tests/outputs/scores.jsonl, tests/outputs/summary.json.

Results

Three counterintuitive observations:

1. 12B drifts more than e2b without prefix (7 vs 3). Bigger model ≠ better Taiwanese vocabulary instinct — likely because the larger model produces longer responses, giving more chances to land on a trigger word.
2. Prefix raises e2b's drift rate while lowering 12B's — opposite directions, net effect close to zero. The prefix influences style and length, not vocabulary preference.
3. Hard hits ≠ prompts with drift: e2b/none has only 3 prompts that drifted but 10 hard hits — a single affected prompt uses the same prohibited word multiple times (P03 used 聯繫 six times, P10 used it three times).

Most common offenders

Hard-term frequencies across all 80 runs:

This is the headline finding: of the 17 prohibited terms on the list, one word causes 95%+ of the drift — 聯繫. The ones I expected to show up — 軟件, 打印, 視頻, 屏幕, 默認, 服務器, 互聯網, 移動端, 鼠標, 賬號, 內存 — didn't appear once across 80 runs.

Practical translation: Gemma 4 actually handles Taiwan vocabulary well at the surface level — it correctly uses 軟體, 列印機, 影片, 螢幕, 預設, 伺服器, 網際網路, 行動裝置, 滑鼠, 帳號, 記憶體. But it insists 聯繫 is neutral Chinese and no prefix dislodges it.

Soft-term counts (both models combined):

用戶 is "soft" (acceptable in Taiwan) but more frequent than 聯繫. If your editorial policy mandates 使用者, that's a much bigger post-process load than the hard-term cleanup.

How much does the prefix actually help?

The numbers are stark:

1. The prefix has almost no effect on vocabulary choice. Both models stay at 9-10 hard hits — Δ ≤ 1, well within sampling noise even at temperature 0.2.
2. 聯繫's stickiness is consistent across models. Whether e2b or 12B, prefix or no prefix, 聯繫 lands 9-10 times. This isn't a model-size problem, it isn't a prompt-engineering problem — it's a training-data distribution problem.
3. What the prefix does change are other things — it does prevent English replies (verified in the quickstart), and it does suppress some soft terms (客戶端 4→0, 通過 3→0) — but it does not touch the main drift source.

Practical recommendation for the Resonance Stack pipeline

• Keep the prefix (it stops English output) but don't expect it to fix vocabulary
• Add a post-process replacement step in the article generator (tools/generate_article.py) — at minimum, map 聯繫 → 聯絡
• Term list can start with just 聯繫. The other 15 hard terms saw zero drift; adding them to the table is cheap insurance against future model versions but low-priority now
• 用戶 → 使用者 is a separate editorial line. Whether to change it depends on how strongly you want the "Taiwan-flavored" voice — that's aesthetic, not compliance

Verdict

Waveform: INTERFERENCE

Not "local models can't write Traditional Chinese," not "the prefix fully solves it." A more practical picture: Gemma 4's Taiwan-vocabulary coverage is higher than expected, but 1-2 stubborn terms cross any prompt setting. Don't rely on prompt-level constraints alone for language quality — add a small term-replacement step in the generator.

Concrete actions for the Resonance Stack:

1. Add a _normalize_taiwanese_terms() step in tools/generate_article.py with at least 聯繫 → 聯絡
2. Don't change existing prompt templates — keep the prefix (it serves other purposes)
3. Defer the 用戶 / 使用者 decision until voice-master.md is settled
4. Quarterly re-audit: when ollama library publishes a new model version, rerun this audit and watch the term frequencies

Full audit script and data →