MooFile

A lightweight embedded document store — single file, zero daemons. MongoDB-style queries, vector search, on-device embeddings — all in one BSON file you can scp or commit to Git.

# What is MooFile?

MooFile is an embedded document database that lives in a single BSON file on disk — no server, no daemon, no configuration. Think SQLite, but for JSON documents with MongoDB-style queries, vector search, text search, and on-device AI embeddings.

📦 Single-file storage

Append-only BSON file, never modified in place. Safe to cp, scp, or commit to version control. Compaction rewrites a pristine copy.

Rust core

🔍 Rich query API

MongoDB-style filter operators: $eq, $gt, $gte, $lt, $lte, $in, $nin, $or, $and, $not, $exists, $regex.

📐 Indexes (B-Tree + Vector + Text)

Regular B-Tree indexes for fast lookups, HNSW vector indexes for similarity search, and full-text search with stemming & stop-words.

🧠 Autoembedding

On-device embedding via llama.cpp (GGUF models). Insert a document with text — MooFile auto-generates the vector and stores it.

New

🧪 Atomic batches

Group inserts, updates, and deletes into atomic batches. Commit writes everything in one flush. Rollback discards cleanly.

🔒 Multi-process safe

Advisory file locking serializes concurrent writes. Multiple processes can read simultaneously. Cross-platform, no external dependencies.

8
Language bindings
550+
Tests across all backends
~2.8 MB
Library size (no embeddings)
~8.3 MB
Library size (with embeddings)

# Pick Your Language

Eight language bindings — all share the same Rust core compiled into a C shared library. Documents cross the FFI boundary as JSON strings. Install once, use from anywhere.

LanguageMechanismRequirementVibe
Python PyO3 (in-process)
+ pure-Python fallback
Reference pip install moofile
Rust Native crate
Builder pattern API
High-perf Core library
C / C++ extern "C"
+ header-only RAII wrapper
C11 / C++17 Lean 30 functions, 0 deps
Node.js koffi FFI Node 18+ Cursors are iterable
Go cgo Go 1.21+, C toolchain Baked-in rpath
Java Foreign Function & Memory API JDK 22+ Zero third-party jars
C# P/Invoke .NET 8+ Strong-typed documents

All bindings locate libmoofile.so automatically — or override with the MOOFILE_LIB environment variable. Pre-built releases for Linux, macOS, and Windows on the GitHub Releases page.

# Query API

MongoDB-style filter syntax, a chainable query builder with sort, skip, limit, group, and aggregation — all on a single file.

Filter operators

$eq
Equal
$ne
Not equal
$gt
Greater than
$gte
Greater or equal
$lt
Less than
$lte
Less or equal
$in
In list
$nin
Not in list
$or
Logical OR
$and
Logical AND
$not
Logical NOT
$exists
Field exists
$regex
Regex match
$all
Array contains all
$elemMatch
Array elem matches
$size
Array length

Query builder stages

sort
Sort ascending/desc
skip
Skip N docs
limit
Limit to N docs
group
Group by field
count
Aggregation: count
sum
Aggregation: sum
mean
Aggregation: average
min
Aggregation: minimum
max
Aggregation: maximum
collect
Collect values
first
First value
last
Last value

Python example

from moofile import Collection, count, mean, sum

with Collection("data.bson", indexes=["email", "age"]) as db:
    db.insert({"name": "Alice", "email": "alice@example.com", "age": 30})

    # Rich queries with chainable builder
    results = (db.find({"age": {"$gte": 25}})
                  .sort("age", descending=True)
                  .limit(10)
                  .to_list())

    # Aggregation pipeline
    by_dept = (db.find({})
                .group("dept")
                .count()
                .sum("pay")
                .mean("age")
                .to_list())

Rust example

use moofile::Collection;
use bson::doc;

let db = Collection::builder("data.bson")
    .index("email")
    .vector_index("embedding", 384)
    .text_index("content")
    .open()
    .unwrap();

db.insert(doc! { "name": "Alice", "age": 30 }).unwrap();

// Lazy query — no work until terminal method
let results = db.find(doc! { "age": { "$gt": 25 } })
    .unwrap()
    .sort("age", true)
    .limit(10)
    .to_list()
    .unwrap();

# On-Device Embeddings

MooFile runs embedding models locally via llama.cpp (GGUF). No external API calls, no data leaving your machine. Models are auto-downloaded and cached on first use.

🧠 Autoembed

Configure a source text field and a target vector field. Documents inserted with text automatically get their embedding computed and stored — zero code changes.

Supports quantization (int8, binary), normalization, and MRL (Matryoshka) truncation.

🔍 Semantic search

Query by meaning, not just keywords. The semantic() method embeds your query text and runs vector search in one step.

HNSW vector indexes for fast approximate nearest-neighbor search.

# Python — semantic search with auto-downloaded model
from moofile import Collection

db = Collection("semantic.bson", vector_indexes={"embedding": 1024},
    auto_embed={"content": {
        "model": "hf:jsonMartin/voyage-4-nano-gguf:voyage-4-nano-q8_0.gguf",
        "target": "embedding",
        "dims": 1024,
        "precision": "int8",
    }})

db.insert({"content": "Machine learning is fascinating"})
db.insert({"content": "The weather today is sunny"})

# Semantic search — embed query + vector search in one call
results = (db.find({})
            .semantic("content", "deep learning", 5)
            .to_list())

# How It Works

Append-only BSON storage, in-memory indexes rebuilt on open, lazy query evaluation.

📄 Storage

Every write appends a new record to the BSON file — original records are never modified in place. Records are typed: LIVE, REPLACEMENT, or TOMBSTONE. Compaction rewrites a pristine copy with only the latest version of each document.

🔎 Indexes

Regular indexes are in-memory B-Trees mapping field values to document IDs. Vector indexes use HNSW graphs. Text indexes use an inverted index with stemming and stop-word filtering. All rebuilt on open — or loaded from a pickle cache for fast startup.

Atomic batches

batch_begin() Buffer inserts, updates, deletes in memory batch_commit() Single append + flush to disk All index mutations applied

Python and Rust both support context-manager syntax: with db.batch(): commits on success, rolls back on exception. Multi-process writes are serialized via advisory file locking.

# CLI Tools

Four command-line tools ship with the Python package — no GUI needed.

moosh
Interactive REPL shell — open a collection and run queries, inserts, updates right from the terminal.
moo2json
Export/import collections to/from JSON files. Great for backups, migrations, and inspection.
moo2mongo
Bidirectional sync between MooFile collections and MongoDB. Move data between embedded and server environments.
moo2sqlite
Export/import between MooFile and SQLite databases. Flatten nested documents for relational storage.
# Export a collection to JSON
moo2json path/to/data.bson > backup.json

# Import back from JSON
moo2json --import backup.json path/to/data.bson

# Interactive shell
moosh data.bson
>>> db.find({"age": {"$gte": 30}}).to_list()

# See It In Action

MooFile works the same way across all eight language bindings. Here's CRUD, sorting, aggregation, vector search, and atomic batches in five languages.

Python

from moofile import Collection

with Collection("data.bson", indexes=["email"]) as db:
    # Insert
    doc = db.insert({"name": "Alice", "email": "a@test.com", "age": 30})

    # Read
    found = db.find_one({"email": "a@test.com"})

    # Update
    db.update_one({"_id": doc["_id"]}, set={"age": 31})

    # Delete
    db.delete_one({"name": "Bob"})

    # Atomic batch
    with db.batch():
        db.insert({"name": "Charlie"})
        db.update_one({"name": "Alice"}, set={"status": "active"})
    # Both operations committed atomically

Node.js

const { Collection } = require('./moofile');

const db = new Collection('data.bson', {
    indexes: ['email'],
    vector_indexes: { 'embedding': 384 },
});

db.insert({ name: 'Alice', email: 'a@test.com', age: 30 });

// Cursors are iterable
for (const doc of db.find({ age: { $gte: 30 } }))
    console.log(doc);

// Atomic batch
db.batch(() => {
    db.insert({ _id: 'a' });
    db.insert({ _id: 'b' });
});

db.close();

Go

import "github.com/patw/moofile/bindings/go/moofile"

db, err := moofile.Open("data.bson", &moofile.Config{
    Indexes: []string{"email"},
})
defer db.Close()

db.Insert(map[string]any]{"name": "Alice", "age": 30})

// Sorting and aggregation
oldest, _ := db.Find(Document{}, &moofile.FindOptions{
    Sort:  "age",
    Desc:  true,
    Limit: 10,
})

byDept, _ := db.Find(nil, &moofile.FindOptions{
    Group: "dept",
    Agg:   []moofile.Agg{moofile.Count(), moofile.Sum("pay")},
})

Java

import com.moofile.*;

try (Collection db = Collection.open("data.bson",
        Config.create().index("email"))) {

    db.insert(Document.of("name", "Alice", "age", 30));

    // Filter factories following MongoDB Java driver style
    Document activeAdults = Filters.and(
        Filters.gte("age", 30),
        Filters.eq("status", "active"));

    for (Document d : db.find(activeAdults))
        System.out.println(d);
}

C#

using Moofile;

using var db = Collection.Open("data.bson", new Config {
    Indexes = new[] { "email" },
});

db.Insert(Document.Of("name", "Alice", "email", "a@example.com", "age", 30));

// Strong-typed property-expression filters
var adults = Builders<Person>.Filter.Gte(person => person.age, 30);
var matches = db.Find(adults);

// Aggregation
var byDept = db.Find(null,
    FindOptions.Create().Group("dept").Count().Sum("pay"));