Encrypted Containers
Users can build a secure, confidential container environment by encrypting the root file system.
Overview
Apptainer provides a feature to build and run encrypted containers to allow users to encrypt the file system image within a SIF. This encryption can be performed using either a passphrase or asymmetrically via an RSA key pair in Privacy Enhanced Mail (PEM/PKCS1) format. The container is encrypted in transit, at rest, and even while running. In other words, there is no intermediate, decrypted version of the container on disk. Container decryption occurs at runtime in memory.
Note
This feature utilizes the Linux dm-crypt library and cryptsetup
utility and requires cryptsetup version of >= 2.0.0. This version should be
standard with recent Linux versions, but users of older Linux
versions may have to update.
Note
We use Privileged Encryption to represent the case of an encrypted image built by the
root user and decrypted with a suid installation. The partition type in a SIF image in this
case wil be shown as Encrypted squashfs.
We use Unprivileged Encryption to represent the case of an encrypted image built by
normal unprivileged users and decrypted either without a suid installion or with the --userns
option. The partition type in a SIF image in this case will be shown as Gocryptfs squashfs.
This case requires
unprivileged user namespaces.
Encrypting a container
A container can be encrypted either by supplying a plaintext passphrase or a PEM file containing an asymmetric RSA public key. Of these two methods the PEM file is more secure and is therefore recommended for production use.
An -e|--encrypt flag to apptainer build is used to indicate
that the container needs to be encrypted.
A passphrase or a key-file used to perform the encryption is supplied at build time via an environment variable or a command line option.
Encryption Method |
Environment Variable |
Commandline Option |
Passphrase |
|
|
Asymmetric Key (PEM) |
|
|
The -e|--encrypt flag is implicitly set when the --passphrase or
--pem-path flags are passed with the build command. If multiple
encryption related flags and/or environment variables are set, the
following precedence is respected.
--pem-path--passphraseAPPTAINER_ENCRYPTION_PEM_PATHAPPTAINER_ENCRYPTION_PASSPHRASE
Passphrase Encryption
Note
Passphrase encryption is less secure than encrypting containers using an RSA key pair (detailed below). Passphrase encryption is provided as a convenience, and as a way for users to familiarize themselves with the encrypted container workflow, but users running encrypted containers in production are encouraged to use asymmetric keys.
In case of plaintext passphrase encryption, a passphrase is supplied by one of the following methods.
Encrypting with a passphrase interactively (Privileged Encryption)
$ sudo apptainer build --passphrase encrypted.sif encrypted.def
Enter encryption passphrase: <secret>
INFO: Starting build...
Using an environment variable (Privileged Encryption)
$ sudo APPTAINER_ENCRYPTION_PASSPHRASE=<secret> apptainer build --encrypt encrypted.sif encrypted.def
Starting build...
In this case it is necessary to use the --encrypt flag since the
presence of an environment variable alone will not trigger the encrypted
build workflow.
While this example shows how an environment variable can be used to set
a passphrase, you should set the environment variable in a way that will
not record your passphrase on the command line. For instance, you could
save a plain text passphrase in a file (e.g. secret.txt) and use it
like so.
$ export APPTAINER_ENCRYPTION_PASSPHRASE=$(cat secret.txt)
$ sudo -E apptainer build --encrypt encrypted.sif encrypted.def
Starting build...
Image info verification (Privileged Encryption)
$ apptainer sif list encrypted.sif
Example:
------------------------------------------------------------------------------
ID |GROUP |LINK |SIF POSITION (start-end) |TYPE
------------------------------------------------------------------------------
1 |1 |NONE |32176-32223 |Def.FILE
2 |1 |NONE |32223-34202 |JSON.Generic
3 |1 |NONE |34202-34292 |JSON.Generic
4 |1 |NONE |36864-17608704 |FS (Encrypted squashfs/*System/arm64)
5 |1 |4 |17608704-17609449 |Cryptographic Message (PEM/RSA-OAEP)
Note that partition 4 is type Encrypted squashfs.
Encrypting with a passphrase interactively (Unprivileged Encryption)
$ apptainer build --passphrase encrypted.sif encrypted.def
Enter encryption passphrase: <secret>
INFO: Starting build...
Using an environment variable (Unprivileged Encryption)
$ APPTAINER_ENCRYPTION_PASSPHRASE=<secret> apptainer build encrypted.sif encrypted.def
Starting build...
$ export APPTAINER_ENCRYPTION_PASSPHRASE=$(cat secret.txt)
$ apptainer build encrypted.sif encrypted.def
Starting build...
Image info verification (Unprivileged Encryption)
$ apptainer sif list encrypted.sif
Example:
------------------------------------------------------------------------------
ID |GROUP |LINK |SIF POSITION (start-end) |TYPE
------------------------------------------------------------------------------
1 |1 |NONE |32176-32215 |Def.FILE
2 |1 |NONE |32215-36269 |JSON.Generic
3 |1 |NONE |36269-36465 |JSON.Generic
4 |1 |NONE |36864-26386432 |FS (Gocryptfs squashfs/*System/arm64)
Note that partition 4 is type Gocryptfs squashfs.
PEM File Encryption
Apptainer currently supports RSA encryption using a public/private key-pair. Keys are supplied in PEM format. The public key is used to encrypt containers that can be decrypted on a host that has access to the secret private key.
You can create a pair of RSA keys suitable for encrypting your container
with the ssh-keygen command, and then create a PEM file with a few
specific flags like so:
# Generate a key pair
$ ssh-keygen -t rsa -b 4096 -m pem -N ''
Generating public/private rsa key pair.
Enter file in which to save the key (/home/vagrant/.ssh/id_rsa): rsa
[snip...]
# Convert the public key to PEM PKCS1 format
$ ssh-keygen -f ./rsa.pub -e -m pem >rsa_pub.pem
# Rename the private key (already PEM PKCS1) to a nice name
$ mv rsa rsa_pri.pem
You would use the rsa_pub.pem file to encrypt your container and the
rsa_pri.pem file to run it. Be sure to keep the private key safe
and private, because it is not encrypted itself.
Encrypting with a command line option (Privileged Encryption)
$ sudo apptainer build --pem-path=rsa_pub.pem encrypted.sif encrypted.def
Starting build...
Encrypting with an environment variable (Privileged Encryption)
$ sudo APPTAINER_ENCRYPTION_PEM_PATH=rsa_pub.pem apptainer build --encrypt encrypted.sif encrypted.def
Starting build...
In this case it is necessary to use the --encrypt flag since the
presence of an environment variable alone will not trigger the encrypted
build workflow.
$ apptainer sif list encrypted.sif
Example:
------------------------------------------------------------------------------
ID |GROUP |LINK |SIF POSITION (start-end) |TYPE
------------------------------------------------------------------------------
1 |1 |NONE |32176-32215 |Def.FILE
2 |1 |NONE |32215-36269 |JSON.Generic
3 |1 |NONE |36269-36465 |JSON.Generic
4 |1 |NONE |36864-42954752 |FS (Encrypted squashfs/*System/arm64)
5 |1 |4 |42954752-42955497 |Cryptographic Message (PEM/RSA-OAEP)
Note that partition 4 is type Encrypted squashfs.
Encrypting with a command line option (Unprivileged Encryption)
$ apptainer build --pem-path=rsa_pub.pem encrypted.sif encrypted.def
Starting build...
Encrypting with an environment variable (Unprivileged Encryption)
$ APPTAINER_ENCRYPTION_PEM_PATH=rsa_pub.pem apptainer build --encrypt encrypted.sif encrypted.def
Starting build...
In this case it is necessary to use the --encrypt flag since the
presence of an environment variable alone will not trigger the encrypted
build workflow.
$ apptainer sif list encrypted.sif
Example:
------------------------------------------------------------------------------
ID |GROUP |LINK |SIF POSITION (start-end) |TYPE
------------------------------------------------------------------------------
1 |1 |NONE |32176-32215 |Def.FILE
2 |1 |NONE |32215-36269 |JSON.Generic
3 |1 |NONE |36269-36465 |JSON.Generic
4 |1 |NONE |36864-26386432 |FS (Gocryptfs squashfs/*System/arm64)
5 |1 |4 |26386432-26387177 |Cryptographic Message (PEM/RSA-OAEP)
Note that partition 4 is type Gocryptfs squashfs.
Running an encrypted container
To run, shell, or exec an encrypted image, credentials to
decrypt the image need to be supplied at runtime either in a key-file or
a plaintext passphrase.
Running a container encrypted with a passphrase
A passphrase can be supplied at runtime by either of the ways listed in the sections above.
Running with a passphrase interactively
$ apptainer run --passphrase encrypted.sif
Enter passphrase for encrypted container: <secret>
Running with a passphrase in an environment variable
$ APPTAINER_ENCRYPTION_PASSPHRASE="secret" apptainer run encrypted.sif
While this example shows how an environment variable can be used to set
a passphrase, you should set the environment variable in a way that will
not record your passphrase on the command line. For instance, you could
save a plain text passphrase in a file (e.g. secret.txt) and use it
like so.
$ export APPTAINER_ENCRYPTION_PASSPHRASE=$(cat secret.txt)
$ apptainer run encrypted.sif
Running a container encrypted with a PEM file
A private key is supplied using either of the methods listed in the Encryption section above.
Running using a command line option
$ apptainer run --pem-path=rsa_pri.pem encrypted.sif
Running using an environment variable
$ APPTAINER_ENCRYPTION_PEM_PATH=rsa_pri.pem apptainer run encrypted.sif
When executing an Unprivileged Encryption encrypted image with an Apptainer suid
installation, --userns is required.
$ apptainer run --pem-path=rsa_pri.pem encrypted.sif
The following error will be shown
FATAL: container creation failed: mount hook function failure: mount /proc/self/fd/3->/usr/local/var/apptainer/mnt/session/rootfs error: while mounting image /proc/self/fd/3: gocryptfs requires user namespace, please add `--userns` option
Add --userns option
$ apptainer run --pem-path=rsa_pri.pem --userns encrypted.sif