Why SQLite for an LLM Proxy — No Postgres, No Redis, No Problem

The usual objection

"SQLite is not a production database." You have probably heard this, or said it yourself. The concern is reasonable: SQLite is an embedded database designed for single-process access. It does not have a connection pool. It does not have replication. It does not have a query planner designed for complex analytics.

All of that is true. And none of it matters for an LLM proxy.

Why it works for an LLM proxy

The upstream API is always the bottleneck. A typical LLM request takes 1 to 30 seconds waiting for the provider to respond. SQLite can handle thousands of reads and writes per second. Your database will never be the slow part — OpenAI will.

Single-process is the point. Stockyard runs as one binary, one process. There is no connection pool problem because there is no network hop to the database. SQLite is an in-process library call. Read latency is microseconds, not milliseconds.

WAL mode handles concurrency. Stockyard runs SQLite in WAL (Write-Ahead Logging) mode, which allows concurrent readers while a writer is active. For a proxy workload — many reads, occasional writes — WAL mode performs well without contention.

The data model is simple. Traces, costs, module configs, audit logs, encrypted keys. These are append-heavy, read-occasionally workloads. Not the complex joins and aggregations that benefit from Postgres's query optimizer. SQLite handles this pattern natively.

What you get from this choice

Zero infrastructure. No database server to provision, monitor, patch, or pay for. No connection string to configure. No pg_hba.conf to get wrong. Stockyard creates the database file on first boot.

Backups are a file copy. Your entire database — every trace, every cost record, every audit entry, every encrypted provider key — is one file. Backing up is cp. Restoring is cp the other direction.

# Backup
cp /data/stockyard.db /backups/stockyard-$(date +%Y%m%d).db

# Restore
cp /backups/stockyard-20260327.db /data/stockyard.db
  

Portability. Move your entire Stockyard instance to a new server by copying one binary and one database file. No export/import pipeline, no migration scripts between database versions, no dump format compatibility issues.

No managed database bill. An RDS Postgres instance costs $15–50/month at minimum. A managed database on any cloud provider adds recurring cost and operational complexity. SQLite costs nothing and runs anywhere the binary runs.

Encryption at rest. Provider API keys are stored with AES-256-GCM encryption inside the SQLite file. The keys never appear in logs or API responses. All of this happens without a separate secrets manager or encrypted volume.

What we do to make it reliable

Using SQLite in production is not just "use SQLite." Stockyard applies specific configurations and patterns to make it work well for a proxy workload:

WAL mode is enabled on every database open. This gives concurrent read access during writes, which matters for a proxy that is recording traces while serving requests.

Integer cents for money. All cost values are stored as integer cents, not floating point. No rounding errors on your cost tracking.

Pure Go driver. Stockyard uses modernc.org/sqlite, a pure Go implementation of SQLite. No CGO, no C compiler dependency, no cross-compilation headaches. The SQLite engine is compiled into the same static binary.

Automatic migrations. Schema changes run on startup. Upgrading Stockyard means downloading the new binary and starting it — the database migrates itself. No manual migration step, no version coordination.

When SQLite is not enough

SQLite is a single-writer database. If you need multiple Stockyard instances writing to the same database simultaneously, SQLite is not the right choice. Specifically:

Multi-node writes. If you are running Stockyard on multiple servers that need to share state in real time, you need Postgres or a similar networked database. SQLite does not do replication.

Heavy analytics. If you want to run complex aggregation queries across millions of traces while the proxy is serving live traffic, a dedicated analytics database (ClickHouse, DuckDB, a data warehouse) will perform better than SQLite for that specific job.

Compliance audits that require a managed database. Some compliance frameworks require a "managed database service" with specific backup SLAs, point-in-time recovery, and audit logging at the database level. SQLite can meet the spirit of these requirements but may not satisfy auditors who expect to see an RDS instance.

For most teams using LLM APIs — solo developers, startups, small-to-medium teams running on one or a few servers — SQLite is not a limitation. It is a simplification that removes an entire category of operational problems.

Frequently Asked Questions

Why does Stockyard use SQLite instead of Postgres?

An LLM proxy is not a high-write web app. It handles tens to low hundreds of requests per second, and SQLite in WAL mode handles that comfortably. Using SQLite means zero external services, zero connection pool configuration, and zero database ops.

Can SQLite handle production traffic for an LLM proxy?

Yes. SQLite in WAL mode supports concurrent reads with serialized writes. An LLM proxy bottleneck is always the upstream provider (1-30 second responses), not the database. SQLite handles the metadata writes easily.

How do I back up a SQLite-based proxy?

Copy one file. That is the entire backup. No pg_dump, no point-in-time recovery configuration. Just cp stockyard.db backup.db.

When would I outgrow SQLite?

If you need horizontal scaling across multiple proxy instances sharing state, or if you need to handle thousands of concurrent write-heavy operations beyond proxy request logging. For a single-instance proxy, SQLite handles millions of traced requests.

Why SQLite for an LLM proxy.