Posts

Showing posts with the label zfs

Long-standing ZFS bug involving encrypted datasets and send/rcv fixed!

A bug opened since May 2021, just a bit over 4 years ago, has finally been fixed:  https://github.com/openzfs/zfs/issues/12014#issuecomment-2889132540 The fix itself is trivial, even though I am sure that finding the cause surely was not. But what really caught my attention was the CodeQL integration test  that was written to prevent this issue from happening again. I find CodeQL and similar tools (such as Opengrep/Semgrep) can be incredibly powerful in integration pipelines to prevent subtle, difficult to reproduce issues from happening again probably saving a lot of time in debugging and to write expensive integration tests. I'd be curious to understand if CodeQL fares better in this context than Opengrep because it is aware of the code flow, as opposed to "just" matching patterns (I think Opengrep understands the code structure, to some extent but I'm not 100% sure). I do find CodeQL intimidating and more complex than Opengrep though: with the latter I was able to ...

HTTM: CLI Time Machine for ZFS, BTRFS/NILFS2 now supports restic

httm added support for restic with version 0.4.0. I've been a long time restic user, and even used restic to backup large datasets (TB) encrypted to S3. While backup is very fast, restore is where things get a little bit more complicated (and slow). Using httm with restic is pretty simple. First you mount the restic repository: restic -r /your/restic/repo/path mount /mnt/restic then you can run httm in browse mode for example for the current directory (assuming it's a directory backed up by restic): httm -b --alt-store=restic . easy peasy.

From 0 to ZFS replication in 5m with syncoid

Image
The ZFS filesystem has many features that once you try them you can never go back. One of the lesser known is probably the support for replicating a zfs filesystem by sending the changes over the network with zfs send/receive. Technically the filesystem changes don't even need to be sent over a network: you could as well dump them on a removable disk, then receive  from the same removable disk.

How to automatically import a ZFS pool built on top of iSCSI devices with systemd

When using ZFS on top of iSCSI devices one needs to deal with the fact that iSCSI devices usually appear late in the boot process. ZFS on the other hand is loaded early and the iSCSI devices are not present at the time ZFS scans available devices for pools to import. This means that not all ZFS pools might be imported after the system has completed boot, even if the underlying devices are present and functional. A quick and dirty solution would be to run  zpool import <poolname> after boot, either manually or from cron. A better, more elegant solution is instead to hook into systemd events and trigger zpool import as soon as the devices are created.

Fun with Postgresql and ZFS

Image
I will show how to use ZFS instant snapshotting and cloning functionality to effortlessly clone a running postgres database regardless of its size. Setup Install your Linux OS of choice then ZFS and Postgres. I use Centos 7 but most commands used in this post are distro-indipendent. Create a zfs pool called tank or use whatever name suits you. In the pool create a filesystem called pgdata . For the sake of following a minimalist ZFS best practice apply the following settings: zfs set compression=lz4 tank/pgdata zfs set xattr=sa tank/pgdata

A not so short guide to ZFS on Linux

Image
Updated Oct 16 2013: shadow copies, memory settings and links for further learning. Updated Nov 15 2013: shadow copies example, samba tuning. Unless you've been living under a rock you should have by now heard many stories about how awesome ZFS is and the many ways it can help with saving your bacon . The downside is that ZFS is not available (natively) for Linux because the CDDL license under which it is released is incompatible with the GPL. Assuming you are not interested in converting to one of the many Illumos distributions or FreeBSD this guide might serve you as a starting point if you are attracted  by ZFS features but are reluctant to try it out on production systems. Basically in this post I note down both the tought process and the actual commands for implementing a fileserver for a small office. The fileserver will run as a virtual machine in a large ESXi host and use ZFS as the filesystem for shared data.