I Gave Claude Access to My Notes. Here's What I Had to Build to Make It Safe.

My notes were a mess of markdown files. Flat, unlinked, unsearchable in any meaningful way. I could grep them. I could ask an LLM to read them. But I couldn't ask them a relationship question: what connects these two ideas, what does this concept depend on, which of my projects touch this topic.

A knowledge graph fixes that. Instead of files, you have nodes and edges. Nodes are things — topics, projects, emails, calendar events, people. Edges are relationships — depends on, related to, belongs to, triggers. You can traverse those edges. You can answer questions like "everything connected to this concept within two hops" in milliseconds.

But I didn't just want to query it myself. I wanted Claude to query it. That's where MCP comes in, and where things get complicated.

What MCP gives you

MCP (Model Context Protocol) is a way to give an AI agent access to tools and data sources. Instead of pasting content into a prompt, the agent calls a function — search, lookup, traverse — and gets back structured results. The agent decides what to query and when.

This is powerful. It's also a problem. If you give an agent unrestricted access to your knowledge graph, it sees everything. Your personal notes. Your financial data. Your emails. Every agent you ever create gets the full picture.

You need scoping. But scoping at the query level — filtering results — is not safe enough. A filtering bug or a prompt injection can leak data. I wanted something stronger.

The security gate

The core idea in kgraph is simple: don't filter what the agent sees — make out-of-scope nodes structurally invisible at the engine level.

Every agent node in the graph has a scope. That scope is a subgraph boundary — a set of node IDs the agent is allowed to see. When an agent makes any query, the engine first evaluates the gate: does this agent exist, does it belong to the right consumer context, what is its boundary?

result = g.gate_check(consumer_id="ctx:1", agent_id="agent:finance")

if not result.allowed:
    return {"error": result.denial_reason}

# The subgraph IS the graph for this agent
# Nodes outside it don't exist in any traversal
retriever = GraphRAGRetriever.from_graph(result.subgraph)

The traversal engine takes that boundary as a parameter. When it walks edges, it skips any neighbor not in the boundary set. Silently. No error, no filtered result — the node is just not there. You can't ask for something you can't see.

How scoping works in the graph

Scopes aren't stored in a config file. They're stored in the graph itself as edges. An agent node has a BELONGS_TO edge to a consumer context node. It has SCOPED_TO edges pointing to domain nodes. The gate evaluates those edges at query time.

This means you can change an agent's scope by adding or removing edges — no restarts, no config changes. And the scope definition is auditable. You can traverse the graph to see exactly what any agent can reach.

# Add a node
g.add_node("agent:research", NodeType.AGENT, domain="research")
g.add_node("ctx:personal", NodeType.CONTEXT)

# Wire up scope
g.add_edge("agent:research", "ctx:personal", RelType.BELONGS_TO)
g.add_edge("agent:research", "research-domain", RelType.SCOPED_TO)

# Now agent:research can only see nodes in the research domain
# ctx:personal owns that agent — no other context can use it

The engine: Rust under Python

The graph engine is written in Rust, exposed to Python through PyO3 bindings. Edges are stored as CSR (Compressed Sparse Row) matrices — one matrix per edge type. It's the same format used in scientific computing for sparse graphs. Row lookups are O(1), the memory footprint is small, and traversal is fast even on large graphs.

Every mutation writes to a WAL (Write-Ahead Log) before touching in-memory state. On startup, the engine loads the latest snapshot and replays only the delta. The WAL handles power loss mid-write gracefully — truncated records are skipped, not corrupted.

// WAL entry types
pub enum WalEntry {
    AddNode { idx, id, label, props },
    AddEdge { edge_type, from, to, weight, attrs },
    RemoveNode { idx, id },
    RemoveEdge { edge_type, from, to },
    Checkpoint { snapshot_ts },
}

The read and write paths are split into two separate binaries. Rust handles all reads on port 8080 — stateless, loads snapshots, checks WAL for freshness, reloads if stale. Python handles writes on port 8081 — mutations, enrichment, connectors. Reads that don't match known routes get proxied to Python. This lets you scale reads without touching the write side.

Connectors: pulling in the world

A knowledge graph is only useful if it has your actual data. The connector system pulls from external sources and writes them as nodes and edges. Gmail, Google Drive, Google Calendar, filesystem, IMAP, bank transactions — each is a connector that implements two methods: authenticate and sync.

Sync is incremental. Each connector stores a cursor on disk so it picks up where it left off. When a new item comes in, the connector yields a node and optionally edges. The enrichment pipeline runs after sync — an LLM tags, summarizes, and extracts entities from the new node.

Adding a connector automatically creates an agent node, wires up the right scoping edges, and saves a YAML spec. The agent is ready to query the connected data with no manual configuration.

The MCP tools

The data-plane MCP server exposes seven tools to the AI agent. All of them run through the gate before doing anything else.

• search — semantic search using vector embeddings plus graph expansion for context
• lookup — get a node by ID
• traverse — follow edges using the step DSL (out, in, repeat, filter)
• related — find neighbors of a node
• list_nodes — list with optional attribute filters
• catalog — list what node types exist in this agent's subgraph
• brief — structured digest for dashboard use

There's also an admin MCP — mutations, connector management, checkpoints, chat logs. That one is not gate-enforced. It's localhost only and not exposed through the data plane.

What I learned

• Storing scope as graph edges instead of config makes access control auditable and changeable at runtime
• Structural invisibility is stronger than filtering — the difference matters in adversarial cases
• Splitting read and write into separate binaries (Rust + Python) lets each do what it's good at
• CSR matrices are the right data structure for graphs where reads heavily outnumber writes
• Connectors with incremental cursors are easier to reason about than full re-syncs
• The WAL + snapshot pattern from databases works just as well for graph engines

The graph now has my notes, emails, calendar, and documents. Claude can query it through MCP and only sees what I've scoped it to see. Adding a new agent means adding a node and a few edges. The scope is in the data, not hidden in a config file somewhere.

Open questions