WAL And SQLite

2023-09-21

The Pocket Base, an open-source backend, made an interesting choice for their persistent storage: SQLite. In the industry, SQLite is usually used for simple client-side storage because it's lightweight, portable, and doesn't need much setup. However, it lacks some features, like OUTER JOINs, and doesn't support multiple writers or user management. This choice raises some questions.

Here's their response:

PocketBase uses embedded SQLite (in WAL mode) and has no plans to support other databases. For most queries, SQLite (in WAL mode) performs better than traditional databases like MySQL, MariaDB, or PostgreSQL, especially for read operations. If you need replication and disaster recovery, you can consider using Litestream.

Basically, they're writing into a WAL (Write-Ahead Logging) instead of using a rollback journal to address some critical issues with SQLite:

  1. Increased Concurrency: Using WAL allows multiple readers and a writer to work at the same time. Readers don't block writers, and writers don't block readers, so reading and writing can happen at the same time.

  2. Improved Performance: Writing a commit to the end of the WAL file is faster than writing to the middle of the main database file. This makes transactions faster with WAL.

  3. Crash Recovery: It's more robust if there's a crash. Changes are first written to the WAL file and then transferred to the database file, reducing the risk of database corruption.

  4. Disk I/O Reduction: With WAL, disk operations are more sequential, which reduces the time spent on disk seek operations.

Subscribe for daily updates on software development, productivity, and more.