把共振站變成可問答的知識庫：本地 RAG 聊天機器人實作Turn Your Blog Into a Searchable Knowledge Base: Local RAG with Ollama

I've written 5 articles on Ollama, n8n, and LangGraph. A month later, when I want to remember "what was LangGraph's retry success rate again?" — I open the blog, hit Cmd+F, scroll up and down hunting for it.

That's exactly the problem RAG (Retrieval-Augmented Generation) solves: turn everything you've written into a searchable knowledge base for an LLM.

This article does it with fully local tooling: Ollama (gemma4 + nomic-embed-text) + ChromaDB + ~200 lines of Python.

The Three Steps of RAG

1. Chunk: split articles into 500-800 token pieces, each self-contained
2. Embed: convert each chunk into a vector (768-dim) using an embedding model
3. Retrieve: at query time, embed the question too, find the top-3 most similar chunks, feed them as context to the LLM

Key insight: the embedding model and the generation LLM are different. nomic-embed-text only produces semantic vectors; it can't answer questions. gemma4:e2b only generates; it can't do semantic search.

Ingest: Writing Articles to the Vector Store

ingest_articles.py does this:

1. Scans content///index.md
2. Strips frontmatter and shortcodes (keeps clean text)
3. Chunks (500 tokens each, 50-token overlap)
4. Calls ollama embeddings nomic-embed-text for each chunk
5. Writes to ChromaDB with metadata (article slug, category, chunk index)

5 articles → 50 chunks → ~30 seconds total.

RAG Chatbot: Asking Questions

rag_chatbot.py flow:

question = input("> ")
# 1. embed the question
q_vec = ollama_embed(question)
# 2. retrieve top-3 chunks
chunks = chroma.query(q_vec, n_results=3)
# 3. build context-augmented prompt
prompt = f"Reference material:\n{chunks}\n\nQuestion: {question}\nAnswer based on the material."
# 4. generate with gemma4
answer = ollama_generate(prompt)
print(answer)

RAG vs Pure LLM

Question: "What security vulnerabilities does OpenClaude have?"

Pure gemma4: "Since I cannot perform real-time security audits on specific running software, I cannot provide an accurate, up-to-date list of vulnerabilities for OpenClaude." (Refusal, followed by generic LLM risk categories.)

RAG version: "Vulnerability type: Sandbox Bypass → Path Traversal. Severity: High. ID: GHSA-m6rx-7pvw-2f73. Disclosed: 2026-04-20. Mitigation: Docker container isolation." (Cites the openclaude-ollama article.)

This is RAG at its best: pure LLM refuses to answer, RAG gives verifiable specifics. But… this is only one of 2 successes out of 5 in-corpus questions.

Two Kinds of Failure: OOC vs Chunk Granularity

After running 7 questions, failures fall into two categories with completely different natures:

Class 1 — Out-of-corpus (topics Resonance Stack never covered)

Q: "How does Stripe charge?" — dists [0.274, 0.319, 0.334] look reasonably close, but the LLM reads the actual chunks and correctly responds "not in the knowledge base" ✅

Meanwhile, pure gemma4 (without RAG) "tries to answer" and produces a generic Stripe overview that may contain outdated or fabricated specifics. This is RAG's real value — the ability to refuse.

Class 2 — In-corpus but wrong chunk granularity

Q: "How many times did LangGraph's retry trigger?" — Should retrieve from static-vs-dynamic, but top-3 returned mcp-local-workflow and openclaude-ollama. The fact "retry triggered 0 times" exists in the corpus, but not in the top-3 chunks. RAG answers "no answer in the material" ❌

This isn't an embedding model problem (all distances < 0.32), nor an LLM reasoning problem. It's a chunking strategy problem — chunk size, overlap, metadata all affect retrieval quality.

Actual hit rate:

Decision Framework: RAG vs Fine-tune

90% of "I want the AI to know about our company" needs can be solved with RAG.