enhanced-memory

# Enhanced Memory

Drop-in enhancement for OpenClaw's memory system. Replaces flat vector search with a 4-signal hybrid retrieval pipeline that achieved **0.782 MRR** (vs ~0.45 baseline vector-only).

## Setup

```bash
# Install Ollama and pull the embedding model
ollama pull nomic-embed-text

# Index your memory files (run from workspace root)
python3 skills/enhanced-memory/scripts/embed_memories.py

# Optional: build cross-reference graph
python3 skills/enhanced-memory/scripts/crossref_memories.py build
```

Re-run `embed_memories.py` whenever memory files change significantly.

## Scripts

### `scripts/search_memory.py` — Primary Search

Hybrid 4-signal retrieval with automatic adaptation:

```bash
python3 skills/enhanced-memory/scripts/search_memory.py "query" [top_n]
```

**Signals fused:**
1. **Vector similarity** (0.4) — cosine similarity via nomic-embed-text embeddings
2. **Keyword matching** (0.25) — query term overlap with chunk text
3. **Header matching** (0.1) — query terms in section headers
4. **Filepath scoring** (0.25) — query terms matching file/directory names

**Automatic behaviors:**
- **Temporal routing** — date references ("yesterday", "Feb 8", "last Monday") get 3x boost on matching files
- **Adaptive weighting** — when keyword overlap is low, shifts to 85% vector weight
- **Pseudo-relevance feedback (PRF)** — when top score < 0.45, expands query with terms from initial results and re-scores

### `scripts/enhanced_memory_search.py` — JSON-Compatible Search

Same pipeline with JSON output format compatible with OpenClaw's memory_search tool:

```bash
python3 skills/enhanced-memory/scripts/enhanced_memory_search.py --json "query"
```

Returns `{results: [{path, startLine, endLine, score, snippet, header}], ...}`.

### `scripts/embed_memories.py` — Indexing

Chunks all `.md` files in `memory/` plus core workspace files (MEMORY.md, AGENTS.md, etc.) by markdown headers and embeds them:

```bash
python3 skills/enhanced-memory/scripts/embed_memories.py
```

Outputs `memory/vectors.json`. Batches embeddings in groups of 20, truncates chunks to 2000 chars.

### `scripts/memory_salience.py` — Salience Scoring

Surfaces stale/important memory items for heartbeat self-prompting:

```bash
python3 skills/enhanced-memory/scripts/memory_salience.py          # Human-readable prompts
python3 skills/enhanced-memory/scripts/memory_salience.py --json   # Programmatic output
python3 skills/enhanced-memory/scripts/memory_salience.py --top 5  # More items
```

Scores `importance × staleness` considering: file type (topic > core > daily), size, access frequency, and query gap correlation.

### `scripts/crossref_memories.py` — Knowledge Graph

Builds cross-reference links between memory chunks using embedding similarity:

```bash
python3 skills/enhanced-memory/scripts/crossref_memories.py build          # Build index
python3 skills/enhanced-memory/scripts/crossref_memories.py show <file>    # Show refs for file
python3 skills/enhanced-memory/scripts/crossref_memories.py graph          # Graph statistics
```

Uses file-representative approach (top 5 chunks per file) to reduce O(n²) to manageable comparisons. Threshold: 0.75 cosine similarity.

## Configuration

All tunable constants are at the top of each script. Key parameters:

| Parameter | Default | Script | Purpose |
|-----------|---------|--------|---------|
| `VECTOR_WEIGHT` | 0.4 | search_memory.py | Weight for vector similarity |
| `KEYWORD_WEIGHT` | 0.25 | search_memory.py | Weight for keyword overlap |
| `FILEPATH_WEIGHT` | 0.25 | search_memory.py | Weight for filepath matching |
| `TEMPORAL_BOOST` | 3.0 | search_memory.py | Multiplier for date-matching files |
| `PRF_THRESHOLD` | 0.45 | search_memory.py | Score below which PRF activates |
| `SIMILARITY_THRESHOLD` | 0.75 | crossref_memories.py | Min similarity for cross-ref links |
| `MODEL` | nomic-embed-text | all | Ollama embedding model |

To use a different embedding model (e.g., `mxbai-embed-large`), change `MODEL` in each script and re-run `embed_memories.py`.

## Integration

To replace the default memory search, point your agent's search tool at these scripts. The scripts expect:
- `memory/` directory relative to workspace root containing `.md` files
- `memory/vectors.json` (created by `embed_memories.py`)
- Ollama running locally on port 11434

All scripts use only Python stdlib + Ollama HTTP API. No pip dependencies.