OMGDB DOCS
// Docs

Roadmap

What recently shipped in OMGDB, what comes next, and the principles that decide the order.

This page is kept honest: “shipped” means merged, tested, and covered by the benchmark gate; “next” lists the real reason each item is not done yet. Dated summaries of each pass live in the changelog.

Recently shipped

The 2026-07 application pass:

  • Immediate Mongo-style writes everywhere. update-one/update-many/find-one-and-update/delete-many with explicit scan/mutation bounds, explicit-seed upsert, and ordered atomic bulk-write — on the CLI, over MCP, and in the TypeScript/Python clients.
  • One-request transactions. An ordered insert/get/replace/delete program with read-your-writes visibility, one durable commit, and canonical-log authorization that never trusts derived cache files.
  • Collection and index lifecycle. create/list/drop-collection and stable named indexes (create-index --name, list-indexes, drop-index) as authoritative op-log operations; count-documents and distinct round out the read surface.
  • A real dangerous MCP scope. drop_collection exists only above read-write — an agent cannot even see it unless the server was deliberately started with the higher ceiling.

The 2026-07 native-search pass:

  • Full-text search. define-search-index writes a versioned, typed index definition into the op-log — replayed, transacted, and compacted like any other operation. Queries are a strict AST (term, phrase, prefix, fuzzy, regex, boolean) scored by deterministic BM25, with per-field weights, English/Greek/keyword/n-gram analyzers, bounded highlighted snippets, and --explain score breakdowns.
  • One call, both worlds. --filter intersects a MongoDB-style structured filter with the text query before ranking, at one consistent log boundary.
  • Semantic and hybrid modes. search --mode semantic ranks by exact cosine over locally computed embeddings; --mode hybrid fuses lexical and semantic rankings with versioned deterministic reciprocal-rank fusion.
  • Pinned local models. omgdb models pull installs bge-small-en-v1.5 or multilingual-e5-small with checksum, license, and read-back verification — the only network path in the engine. Search itself never downloads, and builds without model support say so instead of silently substituting a weaker embedder.
  • Honest derived generations. The accelerated index records its exact log boundary, definition/analyzer digests, and rebuild reason; search-index-status reports them, and a stale or corrupt generation is rebuilt, never served.

The 2026-07 performance pass:

  • Much faster writes. Bulk import commits in batches with one fsync each — 50,000 documents load in about 2 seconds — and single durable writes now run at the fsync wall, the same physical wall every embedded database hits.
  • Deferred cache persistence. Derived cache sidecars are no longer rewritten on every operation; checkpoints may lag the durable log by a bounded tail, and readers heal them lazily. Persistence is an optimization, never a correctness event.
  • describe shows the write contract. The live manual now includes each collection’s indexes and validation rules, so an agent knows what a write must satisfy before attempting it.
  • Incremental vector sync. vsync skips embeddings that are already fresh, and vsearch/context reuse persisted vectors instead of re-embedding on every call.
  • MCP contract hardening. Unknown tool names report “unknown tool”, destructive tools carry real destructive hints, and every tool declares its required parameters in the schema.
  • Structured missing-collection errors. explain and diagnose fail loudly on a missing collection with a did-you-mean hint, instead of silently returning nothing.
  • Compaction preserves history. Compacting the op-log keeps each surviving record’s original timestamp, so compaction no longer rewrites when things happened.

Next

Each item was deferred for a real reason, listed with it:

  1. Warm-open fast path. Opening from a checkpoint is benchmarked but not yet measurably faster than replaying the log — it ships when it actually wins.
  2. Paged caches, then a paged store. Point reads now decode only the entries they probe (per-entry offset tables); paging or memory-mapping the remaining whole-file reads (full scans, the secondary-index cache) is the stepping stone to datasets bigger than RAM — the main gap to production scale.
  3. Uniform --json output flag. Done (2026-07): explain, verify, validate and inspect emit stable, test-pinned JSON schemas.
  4. A real neural embedder (ONNX). Done (2026-07): pinned, checksum-verified local models (bge-small-en-v1.5, multilingual-e5-small) with ONNX inference behind search --mode semantic|hybrid on supported native builds. The deterministic hashing embedder remains the dependency-free baseline, and targets without model support (browser WASI, Windows GNU) report the capability as unavailable instead of downloading or substituting silently.
  5. TypeScript + Python clients — in final testing. Both ride a versioned stdio driver protocol (omgdb driver) with a written contract and fixtures, instead of parsing CLI text: @omgdb/client for Node (ESM + CommonJS, zero runtime dependencies) and an omgdb Python package (sync + native asyncio). They ship with the first public release.
  6. HNSW vector index. Flat exact kNN is correct and fine at current scale; an approximate index must stay a derived, rebuildable artifact like every other binary file.
  7. Log-native rollback. Undo expressed in the op-log itself, rather than only in sidecar change files.
  8. Splitting the two largest source files. Done (2026-07): both the store and query crates are split into concern modules small enough to read whole — the codebase now passes its own legibility bar.

Principles

  • The text stays canonical. The NDJSON op-log is the single source of truth. Binary artifacts — caches, vector indexes, future paged checkpoints — are allowed only as derived, deletable, rebuildable files that verify can check at runtime.
  • Honest limitations. Every feature ships with a plain statement of its current limit, and this site never claims past it. If the docs and the engine disagree, that’s a bug in the docs.
  • Every claim is verified. Behavior is proven by tests, and performance by a benchmark harness with a regression gate — a reintroduced hot-path cost fails loudly. A number you read here was measured, not estimated.