zfs

Pools are managed with the zpool(8) command and have the +following hierarchy:

pool: consists of one or more virtual devices (vdev)
vdev: consists of one or more physical devices (dev) and come in +different kinds such as disk, mirror, raidzX, ... +
- disk: single physical disk (vdev == dev)
- mirror: data is identically replicated on all devs (requires at least 2 +physical devices).
+

Data stored in a pool is distributed and stored across all vdevs by zfs. +Therefore a total failure of a single vdev can lead to total loss of a pool.

A dataset is a logical volume which can be created on top of a pool. Each +dataset can be configured with its own set of properties like +encryption, quota, .... +Datasets are managed with the zfs(8) command.

zfs pool management

Pools are by default mounted at /<POOL>.

Create, modify and destroy zfs pools

# Create a pool MOOSE with a two mirror vdevs.
+zpool create moose mirror <dev1> <dev2> mirror <dev3> <dev4>..
+
+# Add new raidz1 vdev to a pool.
+zpool add moose raidz1 <devA> <devB> <devC>..
+
+# Remove a vdev from a pool.
+zpool remove moose <vdevX>
+
+# Destroy a pool.
+zpool destroy moose
+

+
For stable device names in small home setups it is recommended to use names +from /dev/disk/by-id.
+

Inspect zfs pools

# Show status of all pools or a single one.
+zpool status [<pool>]
+
+# Show information / statistics about pools or single one.
+zpool list [<pool>]
+
+# Show statistics for all devices.
+zpool list -v
+
+# Show command history for pools.
+zpool history
+

Modify `vdevs`

# vdev MIRROR-0 with two devs.
+zpool status
+    NAME            STATE     READ WRITE CKSUM
+    moose           ONLINE       0     0     0
+      mirror-0      ONLINE       0     0     0
+        virtio-200  ONLINE       0     0     0
+        virtio-300  ONLINE       0     0     0
+
+# Attach new device to an existing vdev.
+zpool attach moose virtio-200 virtio-400
+
+# vdev MIRROR-0 with three devs.
+zpool status
+    NAME            STATE     READ WRITE CKSUM
+    moose           ONLINE       0     0     0
+      mirror-0      ONLINE       0     0     0
+        virtio-200  ONLINE       0     0     0
+        virtio-300  ONLINE       0     0     0
+        virtio-400  ONLINE       0     0     0
+
+# Detach device from vdev.
+zpool detach moose virtio-200
+

Replace faulty disk

# MIRROR-0 is degraded as one disk failed, but still intact.
+zpool status
+    NAME            STATE     READ WRITE CKSUM
+    moose           DEGRADED     0     0     0
+      mirror-0      DEGRADED     0     0     0
+        virtio-200  UNAVAIL      0     0     0  invalid label
+        virtio-300  ONLINE       0     0     0
+
+# Replace faulty disk, in mirror.
+# No data is lost since mirror still has one good disk.
+zpool replace moose virtio-200 virtio-400
+
+# MIRROR-0 back in ONLINE (good) state.
+zpool status
+    NAME            STATE     READ WRITE CKSUM
+    moose           ONLINE       0     0     0
+      mirror-0      ONLINE       0     0     0
+        virtio-400  ONLINE       0     0     0
+        virtio-300  ONLINE       0     0     0
+

Import or export zfs pools

When moving pools between hosts, the pool must be exported on the currently +active host and imported on the new host.

# Export a pool called MOOSE.
+zpool export moose
+
+# List pools that can be imported using BY-ID deivce names (for example).
+zpool import -d /dev/disk/by-id
+
+# Import pool MOOSE using BY-ID device names (for example).
+zpool import -d /dev/disk/by-id moose
+

+
Device names used by an existing pool can be changed by exporting and +importing a pool again.
+

zfs dataset management

Datasets are by default mounted at /<POOL>/<DATASET>.

Create and destroy zfs datasets

# Create dataset FOO on pool MOOSE.
+zfs create moose/foo
+
+# Destroy dataset.
+zfs destroy moose/foo
+

List all zfs datasets

# List all zfs datasets.
+zfs list
+

Mount zfs datasets

# List currently mounted datasets.
+zfs mount
+
+# Mount dataset.
+zfs mount moose/foo
+
+# Unmount dataset.
+zfs unmount moose/foo
+

Encrypted datasets

Encryption is a readonly property, can only be set when creating a dataset.

# Create encrypted dataset ENC on pool MOOSE.
+zfs create -o encryption=on -o keyformat=passphrase moose/foo
+
+# Mount encrypte dataset (if key is not loaded).
+zfs mount -l moose/foo
+
+# Unmount dataset and unload encryption key (unload is optional).
+zfs umount -u moose/foo
+

Manage zfs encryption keys

# Preload encryption key for dataset.
+zfs load-key moose/foo
+
+# Preload encryption key for all datasets.
+zfs load-key -a
+
+# Change encryption key for dataset.
+zfs change-key moose/foo
+
+# Unload encryption key for dataset.
+zfs unload-key moose/foo
+

Manage dataset properties

# Get all properties for dataset.
+zfs get quota moose/foo
+
+# Get single property for dataset.
+zfs get all moose/foo
+
+# Get single property for all datasets.
+zfs get quota
+
+# Set property on dataset.
+zfs set quota=10G moose/foo
+

Snapshots

# Create snapshot called V2 for dataset moose/foo.
+zfs snapshot moose/foo@v2
+
+# List all snapshots.
+zfs list -t snapshot
+
+# Make .zfs direcotry visible in the root of the dataset.
+zfs set snapdir=visible moose/foo
+
+# Browse available snapshots in visible .zfs direcotry (readonly).
+ls /moose/foo/.zfs/snapshot
+v1/  v2/
+
+# Create a new dataset based on the V1 snapshot
+zfs clone moose/foo@v1 moose/foov1
+
+# Destroy snapshot.
+zfs destroy moose/foo@v1
+

Access control list

Focus on posix acl.

# Set the ACL type for the FOO dataset to POSIXACL.
+zfs set acltype=posixacl moose/foo
+
+# Get the ACL type of a given dataset.
+zfs get acltype moose/foo
+

+
For performance reasons it is recommended to also set zfs set xattr=sa moose/foo [ref].
+

Example: zfs pool import during startup (`systemd`)

The default zpool cache file is /etc/zfs/zpool.cache. When pools are imported +the cache is updated.

Enable the following targets / services to automatically import pools from the +cache.

systemctl list-dependencies
+  ...
+    └─zfs.target
+      └─zfs-import.target
+        └─zfs-import-cache.service
+

+ +

Notes