Troubleshooting

A little bit of help.

No space left on device

Sometimes when you are building an image, Singularity tells you that it runs out of space on the device:

sudo singularity build fatty.simg Singularity

IOError: [Errno 28] No space left on device

ABORT: Aborting with RETVAL=255

The issue here is that during build of a squashfs image, Singularity is using the $TMPDIR . If your $TMPDIR is overflowing (or the mount is very small!) then you would see this error. As a test, you can try building a sandbox. If this is the issue, then the sandbox should work.

sudo singularity build --sandbox [fatty] Singularity

Solution You simply need to set the $SINGULARITY_CACHEDIR to a different location that you have more room.

Segfault on Bootstrap of Centos Image

If you are bootstrapping a centos 6 docker image from a debian host, you might hit a segfault:

$ singularity shell docker://centos:6

Docker image path: index.docker.io/library/centos:6

Cache folder set to /home/jbdenis/.singularity/docker

Creating container runtime...

Singularity: Invoking an interactive shell within container...


Segmentation fault

The fix is on your host, you need to pass the variable vsyscall=emulate to the kernel, meaning in the file /etc/default/grub (note, this file is debian specific), add the following:

GRUB_CMDLINE_LINUX_DEFAULT="vsyscall=emulate"

and then update grub and reboot:

update-grub && reboot

Please note that this change might have security implications that you should be aware of. For more information, see the original issue.

How to use Singularity with GRSecurity enabled kernels

To run Singularity on a GRSecurity enabled kernel, you must disable several security features:
$ sudo sysctl -w kernel.grsecurity.chroot_caps=0

$ sudo sysctl -w kernel.grsecurity.chroot_deny_mount=0

$ sudo sysctl -w kernel.grsecurity.chroot_deny_chmod=0

$ sudo sysctl -w kernel.grsecurity.chroot_deny_fchdir=0

The container isn’t working on a different host!

Singularity by default mounts your home directory. While this is great for seamless communication between your host and the container, it can introduce issues if you have software modules installed at $HOME. For example, we had a user run into this issue.

Solution 1: Specify the home to mount A first thing to try is to point to some “sanitized home,” which is the purpose of the -H or –home option. For example, here we are creating a home directory under /tmp/homie, and then telling the container to mount it as home:

rm -rf /tmp/homie && mkdir -p /tmp/homie && \

singularity exec -H /tmp/homie analysis.img /bin/bash

Solution 2: Specify the executable to use It may be the issue that there is an executable in your host environment (e.g. python) that is being called in preference to the containers. To avoid this, in your runscript (the %runscript section of the bootstrap file) you should specify the path to the executable exactly. This means:

%runscript


# This specifies the python in the container

exec /usr/bin/python "$@"


# This may pick up a different one

exec python "$@"

This same idea would be useful if you are issuing the command to the container using exec.

Invalid Argument or Unknown Option

When I try mounting my container with the -B or --bind option I receive an unknown option or Invalid argument error. Make sure that you are using the most recent Singularity release to mount your container to the host system, and that the --bind argument is placed after the execution command. An example might look like this:

$ singularity run -B $PWD:/data my_container.img

Also, make sure you are using an up-to-date Singularity to bootstrap your container. Some features (such as --bind ) will not work in earlier versions.

Error running Singularity with sudo

This fix solves the following error when Singularity is installed into the default compiled prefix of /usr/local:

$ sudo singularity instance.start container.img daemon1

sudo: singularity: command not found

The cause of the problem is that sudo sanitizes the PATH environment variable and does not include /usr/local/bin in the default search path. Considering this program path is by default owned by root, it is reasonable to extend the default sudo PATH to include this directory. To add /usr/local/bin to the default sudo search path, run the program visudo which will edit the sudoers file, and search for the string ‘secure_path’. Once found, append :/usr/local/bin to that line so it looks like this:

Defaults    secure_path = /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin