Big Brain v0.4.0

README.md

Markdown · 114 lines · 6.8 KB
Outline

Big Brain (brain)

Shared, searchable long-term memory for AI coding agents (Claude Code, Codex, Grok, Copilot, or any agent that can run a command). Agents record how systems were built (with refs to the real code), workflows, fixes, gotchas, decisions and runbooks; other agents search it before they start, so nothing is rediscovered and nothing is lost when a context window is wiped.

Created by Matthew Q Cannon at Coinopolis. MIT licensed.

Install

Needs Rust (rustup) and git. On Windows, Rust also needs the Visual Studio C++ build tools (the rustup installer offers them); on macOS, the Xcode command-line tools (xcode-select --install).

git clone https://bigbrain.coinopolis.io/git/big-brain.git && cd big-brain
cargo install --locked --path . --features local-embeddings   # recommended: search by meaning, on your machine
brain setup                       # one-time: your name, AI provider, agent tools, projects
brain doctor                      # check everything is wired up

The local-embeddings build adds a small CPU embedding model (bge-base-en-v1.5), so search finds entries by meaning with nothing sent anywhere. It downloads ONNX Runtime while building and the model from Hugging Face on first use. ONNX Runtime needs a recent system: on Linux, glibc 2.38+ and GCC 13+ (Ubuntu 24.04+, Debian 13+; it fails to link on Ubuntu 22.04 and Debian 12). On older systems, where those downloads are blocked, or if you'll use Azure/OpenAI for embeddings instead, build without it: cargo install --locked --path . (keyword search until you set a provider in brain setup).

On Linux x86_64 you can skip the build: download the prebuilt binary (the local-embeddings build; Ubuntu 24.04+, Debian 13+ or newer) and check it against its published SHA-256:

curl -fLO https://bigbrain.coinopolis.io/downloads/brain-x86_64-linux.tar.gz
curl -fLO https://bigbrain.coinopolis.io/downloads/brain-x86_64-linux.tar.gz.sha256
sha256sum -c brain-x86_64-linux.tar.gz.sha256
tar -xzf brain-x86_64-linux.tar.gz && install -Dm755 big-brain-*/brain ~/.local/bin/brain

brain still needs git (the brain is a git repository). brain upgrade needs a clone, so binary installs update by downloading again. Your memory lives in the brain folder, not beside the binary, so replacing or moving the binary never touches it; after switching to a binary in a different place (say from a cargo install to the download), run brain setup again so the agent hooks call the new one (brain doctor warns when they don't).

The clone-and-build commands work in PowerShell. Where an example pipes a file in (brain record < e.md), use --file e.md instead, which works in every shell.

Tested most on Linux; macOS and Windows are supported (paths, hooks and az handled) but have had less real-world use so far. Reports welcome.

What brain setup asks

  1. Your name, as agents should refer to you, and who reviews runbook changes (or none).
  2. Embeddings (search by meaning): azure (an Azure OpenAI embedding deployment), openai (any OpenAI-compatible server, e.g. LM Studio), local (with the feature above), or none (keyword search).
  3. Librarian (optional; reviews stale entries, powers search --deep): azure, openai, lmstudio, none.
  4. Auth for remote providers: azure = keyless (managed identity on an Azure VM, else your az login), env:VAR = API key in an environment variable, command:<cmd> = a command that prints a token.
  5. Agent tools to connect: installs the skill, always-on instructions and hooks where supported, merging into existing settings:
    • Claude Code: skill, plus SessionStart/UserPromptSubmit/Stop hooks in ~/.claude/settings.json.
    • Codex: skill, a managed block in ~/.codex/AGENTS.md, and hooks in ~/.codex/hooks.json (Codex asks you to trust them once).
    • Grok: an always-on rule in ~/.grok/rules/ (it reads Claude's skill and hooks; hooks run in trusted folders).
    • Copilot / others: prints an instruction block for AGENTS.md, .github/copilot-instructions.md, or the tool's own instructions.
  6. Projects: scans a folder for git repos to register.

Re-run brain setup any time; brain setup --help shows the flags for unattended setup (--yes). AZURE_OPENAI_ENDPOINT pre-fills the Azure endpoint.

How it works

  • Entries are markdown files with YAML frontmatter in a git repo (default ~/.local/share/big-brain, %LOCALAPPDATA%\big-brain on Windows; override with BRAIN_HOME; installs from before the rename keep their giga-brain folder). Every write is a commit authored by the agent, so brain log shows who wrote what and git can undo anything.
  • Search fuses per-section embedding similarity with SQLite FTS5 keyword ranking. index.db is derived and rebuilt automatically. If the embedding provider is unreachable, search falls back to keywords and catches up later.
  • Staleness: each entry fingerprints the files it references. When they change, the entry is flagged; hooks tell the owning agent at session start and with the next message (again after 10 messages if still stale), and brain review can have the librarian explain what changed.
  • Hook cost (measured on a 245-entry brain, Linux): session start takes about 7 ms and adds about 100–130 tokens once per session, plus a line per stale entry (10 at most). The per-message hook takes about 7 ms and adds nothing unless an entry has newly gone stale. The stop hook takes about 1 ms and asks for a record once after 5 file edits, then after every 25 more (about 100 tokens each time). The skill itself (about 2,400 tokens) loads only when an agent uses it.
  • Guards: near-duplicate entries of the same kind are refused; obvious secrets are refused; runbook changes can require review (brain proposals). Entries are reference material for agents, never instructions or approval.
  • Machine-specific skill text: put it in <brain dir>/skill-local.md; brain setup appends it to the installed skill under "On this machine". Because every agent reads that text and any agent can write the file, setup shows it to you and asks before including it, and again whenever it changes; --yes and non-terminal runs never approve it.

For AI agents

AGENTS.md tells an agent how to install Big Brain for its user, use it, and work on the code.

Documentation

The full guide (concepts, how it works, agent tools, settings, Azure troubleshooting, command reference): docs/GUIDE.md.

Commands

brain search | get | related | list | record | update | edit | ack | rm | stale | review | who | uses | projects | project add | proposals | template | kinds | doctor | stats | digest | check | log | reindex | setup — see brain --help. Plain brain shows an overview. What changed between versions: CHANGELOG.md.

License

MIT