rag that never leaves the machine

local brain started as a weekend question: could i search everything i've saved, by meaning, without uploading any of it? the answer was yes, with a few constraints that made it better.

the index follows the files

a file watcher sees every create, edit, and delete in the folders i point it at. new or changed files are split into passages and embedded into chromadb; deleted files take their passages with them.

def on_change(path: Path) -> None:
    collection.delete(where={"path": str(path)})
    if path.exists():
        chunks = split(read(path), max_tokens=400, overlap=60)
        collection.add(
            ids=[f"{path}#{i}" for i in range(len(chunks))],
            documents=chunks,
            metadatas=[{"path": str(path)}] * len(chunks),
        )

deleting before re-adding is the whole trick. without it, an edited note answers with both its old and new versions, and you stop trusting it.

small models need good context

a local model through ollama is slower and less capable than a hosted one, so retrieval has to do more of the work: fewer, better passages, each with its file path, and an answer that cites them.

• retrieve more than you need, then keep the best few.
• always show where an answer came from.
• let it say "i couldn't find that".

it works on a plane. that was the test, and it passes.