Installing Apptainer
This section will guide you through the process of installing Apptainer 1.2.5 via several different methods. (For instructions on installing earlier versions of Apptainer please see earlier versions of the docs.)
Installation on Linux
Apptainer can be installed on any modern Linux distribution, on bare-metal or inside a Virtual Machine. It can even often be run nested inside another Apptainer container or inside some other container system.
System Requirements
Apptainer requires ~150MiB disk space once compiled and installed.
There are no specific CPU or memory requirements at runtime, though at least 2GB of RAM is recommended when building from source.
Full functionality of Apptainer requires that the kernel supports:
Unprivileged user namespaces - (minimum kernel >=3.8, >=4.18 or 3.10.0-1127 on RHEL7 recommended) Required to run containers without root or setuid privilege. The recommended versions are required for unprivileged SIF file mounts.
OverlayFS mounts - (minimum kernel >=3.18, >=5.11 recommended) Required for full flexibility in bind mounts to containers in suid mode, and to support persistent overlays for writable containers in suid mode. Kernel 5.11 enables support for persistent overlay unprivileged, but otherwise fuse-overlayfs will be used for that.
Instructions to install without or with setuid privileges are below. Please make sure you are familiar with the discussion on Setuid & User Namespaces in the Security section of the user guide, and the User Namespace Requirements in this guide.
Non-standard ldconfig / Nix & Guix Environments
If Apptainer is installed under a package manager such as Nix or
Guix, but on top of a standard Linux distribution (e.g. RHEL or
Debian), it may be unable to correctly find the libraries for --nv
and --rocm
GPU support. This issue occurs as the package manager
supplies an alternative ldconfig
, which does not identify GPU
libraries installed from host packages.
To allow Apptainer to locate the host (i.e. RHEL / Debian) GPU
libraries correctly, set binary path
in apptainer.conf
to
point to the directory of the host’s ldconfig
before the
$PATH:
which is replaced by the user’s PATH.
I.E., /sbin
should be before any Nix or Guix
related path or the user’s PATH.
Filesystem support / limitations
Apptainer supports most filesystems, but there are some limitations when installing Apptainer on, or running containers from, common parallel / network filesystems. In general:
We recommend installing Apptainer on local disk on each compute node.
If Apptainer is installed to a network location, a
--localstatedir
should be provided on each node, and Apptainer configured to use it.The
--localstatedir
filesystem should support overlay mounts.TMPDIR
/APPTAINER_TMPDIR
should be on a local filesystem wherever possible.
Note
Set the --localstatedir
location by by providing
--localstatedir my/dir
as an option when you configure your
Apptainer build with ./mconfig
.
Disk usage at the --localstatedir
location is negligible (<1MiB).
The directory is used as a location to mount the container root
filesystem, overlays, bind mounts etc. that construct the runtime
view of a container. You will not see these mounts from a host shell,
as they are made in a separate mount namespace.
Overlay support
Some features of Apptainer, such as the --writable-tmpfs
and
--overlay
options, try to use the Linux overlay
filesystem driver to
construct a container root filesystem that combines files from different
locations.
Not all filesystems can be used with the overlay
driver,
so when containers are run from these filesystems some Apptainer
features may not be available.
If the overlay
kernel driver does not work, but the setuid flow
is not being used and unprivileged user namespaces are available, then
Apptainer will use the fuse-overlayfs
command if it can be found.
That works with additional filesystem types than the overlay
kernel driver does not work with, and also works in cases where
older kernels do not support using the overlay
driver unprivileged.
Overlay support has two aspects, referenced below:
lowerdir
support for a filesystem allows a directory on that filesystem to act as the ‘base’ of a container. A filesystem must support overlaylowerdir
for you be able to run an Apptainer sandbox container on it, while using functionality such as--writable-tmpfs
/--overlay
.upperdir
support for a filesystem allows a directory on that filesystem to be merged on top of alowerdir
to construct a container. If you use the--overlay
option to overlay a directory onto a container, then the filesystem holding the overlay directory must supportupperdir
.
Fakeroot with uid/gid mapping on Network filesystems
When Apptainer is run using the fakeroot option and
mappings are available in /etc/subuid
and /etc/subgid
then uids / gids
inside the container are mapped to different host uids / gids.
Most local filesystems (ext4/xfs etc.) have no problem with this uid/gid mapping. Most network filesystems (NFS/Lustre/GPFS etc.), however, only support a single uid for each user. When additional user ids are attempted to be used the fileserver will deny many operations, with ‘permission denied’ errors. This is currently a generic problem for rootless container runtimes.
This is only a problem when sandbox images are stored on the network filesystems. For that case it is probably better to use one of the other fakeroot modes that Apptainer supports. Alternatively, use SIF images instead of sandbox images because they don’t have the problem.
Apptainer cache / atomic rename
Apptainer will cache SIF container images generated from remote
sources, and any OCI/docker layers used to create them. The cache is
created at $HOME/.apptainer/cache
by default. The location of the
cache can be changed by setting the APPTAINER_CACHEDIR
environment
variable.
The directory used for APPTAINER_CACHEDIR
should be:
A unique location for each user. Permissions are set on the cache so that private images cached for one user are not exposed to another. This means that
APPTAINER_CACHEDIR
cannot be shared.Located on a filesystem with sufficient space for the number and size of container images anticipated.
Located on a filesystem that supports atomic rename, if possible.
The Apptainer cache is concurrency safe.
Parallel runs of Apptainer that would create overlapping cache
entries will not conflict, as long as the filesystem used by
APPTAINER_CACHEDIR
supports atomic rename operations.
Support for atomic rename operations is expected on local POSIX filesystems, but varies for network / parallel filesystems and may be affected by topology and configuration. For example, Lustre supports atomic rename of files only on a single MDT. Rename on NFS is only atomic to a single client, not across systems accessing the same NFS share.
If you are not certain that your $HOME
or APPTAINER_CACHEDIR
filesystems support atomic rename, do not run apptainer
in parallel
using remote container URLs. Instead use apptainer pull
to create
a local SIF image, and then run this SIF image in a parallel step. An
alternative is to use the --disable-cache
option, but this will
result in each Apptainer instance independently fetching the
container from the remote source, into a temporary location.
NFS
NFS filesystems support overlay mounts as a lowerdir
only, and do
not support --fakeroot
.
Containers run from SIF files located on an NFS filesystem do not have restrictions.
In setuid mode, you cannot use
--overlay
to overlay a directory onto a container when the overlay (upperdir) directory is on an NFS filesystem. In non-setuid mode with fuse-overlayfs it is allowed but will be read-only.When building a container, or running a container with
--fakeroot
, yourTMPDIR
/APPTAINER_TMPDIR
should not be set to an NFS location.You should not run a sandbox container with
--fakeroot
from an NFS location.
Lustre / GPFS / PanFS
Lustre, GPFS, and PanFS do not have sufficient upperdir
or
lowerdir
overlay support for certain Apptainer features, and
do not support --fakeroot
.
In setuid mode, you cannot use
--overlay
or--writable-tmpfs
with a sandbox container that is located on a Lustre, GPFS, or PanFS filesystem. SIF containers on Lustre, GPFS, and PanFS will work correctly with these options. It works with fuse-overlayfs in non-setuid mode.In setuid mode, you cannot use
--overlay
to overlay a directory onto a container, when the overlay (upperdir) directory is on a Lustre, GPFS, or PanFS filesystem. In non-setuid mode with fuse-overlayfs it is allowed but will be read-only.When building a container, or running a container with
--fakeroot
, yourTMPDIR / APPTAINER_TMPDIR
should not be a Lustre, GPFS, or PanFS location.
FUSE-based filesystems
The kernel overlay driver does not allow the upperdir to be a FUSE-based filesystem, so in setuid mode that is disallowed. It does work in non-setuid mode with fuse-overlayfs.
Install unprivileged from pre-built binaries
A script is available to make a relocatable unprivileged binary installation of Apptainer, including all dependencies that are not normally installed on Linux systems. The script works on current Red Hat Enterprise Linux-derived systems, Fedora, SUSE/OpenSUSE, Debian, and Ubuntu.
This is the easiest way to use it:
$ curl -s https://raw.githubusercontent.com/apptainer/apptainer/main/tools/install-unprivileged.sh | \
bash -s - install-dir
where install-dir is the directory you want to install it into.
Once installed, you can run it with install-dir/bin/apptainer
.
The script requires a few tools to be in the user’s PATH that aren’t
always available: curl
, rpm2cpio
, and cpio
. If they are
not available and you do not have privileges to install them, you can
copy them from another compatible machine or you can run the script
on another machine and copy the installed files to the machine you
want them on.
Multiple architectures can be installed into the same directory on a shared filesystem and it will automatically pick the correct architecture when it runs. Architectures supported by EPEL and Fedora are supported. By default it installs from the oldest supported EPEL version available, for maximum compatibility when working with containers based on old or new operating system distributions.
Install from pre-built packages
Prebuilt packages are available for released versions of Apptainer on a variety of host operating systems.
Install RPM from EPEL or Fedora
Multiple architectures of RPMs are available for Red Hat Enterprise Linux and Fedora.
First, on Red Hat Enterprise Linux derived systems enable the EPEL repositories like this:
$ sudo yum install -y epel-release
Then to install a non-setuid installation of Apptainer do:
$ sudo yum install -y apptainer
or for a setuid installation do:
$ sudo yum install -y apptainer-suid
Install from GitHub release RPMs
Alternatively, x86_64 RPMs are available on GitHub immediately after each Apptainer release and they can be installed directly from there. For the non-setuid installation:
$ sudo yum install -y https://github.com/apptainer/apptainer/releases/download/v1.2.5/apptainer-1.2.5-1.x86_64.rpm
For the setuid installation do above command first and then this one:
$ sudo yum install -y https://github.com/apptainer/apptainer/releases/download/v1.2.5/apptainer-suid-1.2.5-1.x86_64.rpm
Install Debian packages
Pre-built Debian packages are only available on GitHub and only for the amd64 architecture. For the non-setuid installation use these commands:
$ sudo apt update
$ sudo apt install -y wget
$ cd /tmp
$ wget https://github.com/apptainer/apptainer/releases/download/v1.2.5/apptainer_1.2.5_amd64.deb
$ sudo apt install -y ./apptainer_1.2.5_amd64.deb
For the setuid installation do above commands first and then these:
$ wget https://github.com/apptainer/apptainer/releases/download/v1.2.5/apptainer-suid_1.2.5_amd64.deb
$ sudo dpkg -i ./apptainer-suid_1.2.5_amd64.deb
Install Ubuntu packages
Pre-built Ubuntu packages are available on PPA (Personal Package Archive) for the amd64 and arm64 architectures on all current Ubuntu releases.
First, on Ubuntu based containers install software-properties-common package to obtain add-apt-repository command. On Ubuntu Desktop/Server derived systems skip this step.
$ sudo apt update
$ sudo apt install -y software-properties-common
For the non-setuid installation use these commands:
$ sudo add-apt-repository -y ppa:apptainer/ppa
$ sudo apt update
$ sudo apt install -y apptainer
For the setuid installation do above commands first and then these:
$ sudo add-apt-repository -y ppa:apptainer/ppa
$ sudo apt update
$ sudo apt install -y apptainer-suid
Install from Source
To install from source, follow the instructions in INSTALL.md on GitHub.
Relocatable Installation
An unprivileged (non-setuid) Apptainer installation built from source is
relocatable. As long as the structure inside the installation directory
(--prefix
) is maintained, it can be moved to a different location
and Apptainer will continue to run normally.
Relocation of a setuid installation is not supported, as restricted location / ownership of configuration files is important to security.
Source bash completion file
If you installed from source, then to enjoy bash shell completion with Apptainer commands and options, source the bash completion file (assuming the default installation prefix):
$ . /usr/local/share/bash-completion/completions/apptainer
Add this command to your ~/.bashrc
file so that bash completion
continues to work in new shells. (Adjust the path if you installed
Apptainer to a different location.)
Build an RPM
If you use RHEL, CentOS or SUSE, building and installing an Apptainer RPM allows your Apptainer installation be more easily managed, upgraded and removed.
The instructions on how to build the RPM from source are in a INSTALL.md section on GitHub.
Build a Debian package
Packaging for Debian and Ubuntu can also be built from source. The instructions on how to do that are in a separate file DEBIAN_PACKAGE.md on GitHub.
Testing & Checking the Build Configuration
After installation you can perform a basic test of Apptainer functionality by executing a simple alpine container:
$ apptainer exec docker://alpine cat /etc/alpine-release
3.9.2
See the user guide for more information about how to use Apptainer.
apptainer buildcfg
Running apptainer buildcfg
will show the build configuration of an
installed version of Apptainer, and lists the paths used by
Apptainer. Use apptainer buildcfg
to confirm paths are set
correctly for your installation, and troubleshoot any ‘not-found’ errors
at runtime.
$ apptainer buildcfg
PACKAGE_NAME=apptainer
PACKAGE_VERSION=1.2.5
BUILDDIR=/home/dtrudg/Sylabs/Git/apptainer/builddir
PREFIX=/usr/local
EXECPREFIX=/usr/local
BINDIR=/usr/local/bin
SBINDIR=/usr/local/sbin
LIBEXECDIR=/usr/local/libexec
DATAROOTDIR=/usr/local/share
DATADIR=/usr/local/share
SYSCONFDIR=/usr/local/etc
SHAREDSTATEDIR=/usr/local/com
LOCALSTATEDIR=/usr/local/var
RUNSTATEDIR=/usr/local/var/run
INCLUDEDIR=/usr/local/include
DOCDIR=/usr/local/share/doc/apptainer
INFODIR=/usr/local/share/info
LIBDIR=/usr/local/lib
LOCALEDIR=/usr/local/share/locale
MANDIR=/usr/local/share/man
APPTAINER_CONFDIR=/usr/local/etc/apptainer
SESSIONDIR=/usr/local/var/apptainer/mnt/session
PLUGIN_ROOTDIR=/usr/local/libexec/apptainer/plugin
APPTAINER_CONF_FILE=/usr/local/etc/apptainer/apptainer.conf
APPTAINER_SUID_INSTALL=0
Note that the LOCALSTATEDIR
and SESSIONDIR
should be on local,
non-shared storage.
Test Suite
The Apptainer codebase includes a test suite that is run during development using CI services.
If you would like to run the test suite locally you can run the test
targets from the builddir
directory in the source tree:
make check
runs source code linting and dependency checksmake unit-test
runs basic unit testsmake integration-test
runs integration testsmake e2e-test
runs end-to-end tests, which exercise a large number of operations by calling the Apptainer CLI with different execution profiles.
Note
Running the full test suite requires a docker
installation, and
nc
in order to test docker and instance/networking functionality.
Apptainer must be installed with suid in order to run the full test suite,
as it must run the CLI with setuid privilege for the starter-suid
binary.
Warning
sudo
privilege is required to run the full tests, and you should
not run the tests on a production system. We recommend running the
tests in an isolated development or build environment.
Installation on Windows or Mac
Linux container runtimes like Apptainer cannot run natively on Windows or Mac because of basic incompatibilities with the host kernel. (Contrary to a popular misconception, MacOS does not run on a Linux kernel. It runs on a kernel called Darwin originally forked from BSD.)
In order to use Apptainer on these platforms, you can install Vagrant Boxes via Vagrant Cloud, one of Hashicorp’s open source tools, by following the instructions below. Then you can install Apptainer in the base VM of your choice by following the linux installation instructions above.
You can also use Lima to install both the Linux VM and Apptainer for access from the host operating system. It will install the “cloud” version of the selected VM operating system.
Mac
Vagrant
Apptainer is available via Vagrant (installable with Homebrew or manually)
To use Vagrant via Homebrew:
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
$ brew install --cask virtualbox vagrant vagrant-manager
Use vagrant init
to create a new Vagrantfile, or use this example:
Then do vagrant up
, and vagrant ssh
to access the virtual machine.
Lima
Apptainer is available via Lima (installable with Homebrew or manually)
To use Lima via Homebrew:
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
$ brew install qemu lima
Then do limactl start template://apptainer
and limactl shell apptainer
.
See the lima apptainer template for more details.