Comparing CoW Filesystems

With bcachefs now entering the ring, how does it compare to the incumbent CoW filesystems, Btrfs and ZFS, in practice? Each of these systems was built with slightly different priorities, so a nuanced comparison is needed in terms of performance, reliability, and complexity.

Performance

Bcachefs aims to rival the speed of traditional filesystems such as ext4/XFS while delivering CoW features. Initial benchmarks have been promising. In many tests, bcachefs has demonstrated performance equal or superior to Btrfs. This suggests the optimizations in bcachefs's B-tree and write path are paying off. That said, bcachefs is still catching up to decades-old filesystems in raw throughput: It generally cannot beat ext4 or XFS in every test (those non-CoW filesystems won in essentially all the benchmarks against bcachefs in that same comparison). This is expected, given ext4/XFS do less work (no checksums, simpler allocation) and bcachefs is young. On the flip side, bcachefs's read performance (especially random reads) and metadata operations are highly optimized, often outpacing Btrfs.

Compared to ZFS, bcachefs tends to have an upper hand in raw speed on Linux because ZFS's kernel integration is not as tight. ZFS is known to be heavy on memory (ARC cache) and CPU, whereas bcachefs's more lightweight design and ability to use the page cache like other Linux filesystems can translate to lower overhead. Real-world performance between ZFS and bcachefs will depend on workload and tuning, but bcachefs should feel more native on Linux, with low latency and without the complexity of ZFS's userland/kernel split.

Reliability and Data Integrity

ZFS is the long-established champion of data integrity because it's famous for never losing data undetected, thanks to checksumming of everything and CoW with transactional writes. Btrfs also has checksums and CoW, but it had a rocky start with reliability. Where does bcachefs stand on this spectrum? Bcachefs's design philosophy puts reliability first and its motto literally emphasizes not eating your data. In theory, bcachefs should offer equivalent protection to ZFS/Btrfs: Every read is verified by checksum and, with multiple device redundancy, any corrupt block can be detected and reconstructed from a good copy. Bcachefs also provides an fsck (filesystem check) tool, though notably its approach to fsck is novel – the same codebase runs in the kernel and user space, meaning the offline fsck is essentially reusing the filesystem's own code to check consistency. This can reduce the chances of fsck getting out of sync with the filesystem logic, a problem that sometimes afflicts filesystems with separate check utilities.

That said, bcachefs has an advantage in having learned from Btrfs's mistakes. For example, bcachefs's RAID (replication/erasure coding) is designed to avoid write hole issues and does online recovery better. It even includes a mechanism to explicitly heal a degraded volume when a missing device is replaced, instead of relying on mount or scrub operations doubling as recovery (as Btrfs does). Snapshots in bcachefs are more scalable than Btrfs's, meaning you can take many snapshots with less performance penalty. And unlike ZFS's CoW that can fragment data over time, bcachefs has a background garbage collection (copy GC) process that can defragment and reclaim space, somewhat akin to what a log-structured filesystem would do, thereby mitigating performance loss due to fragmentation.