Bug 2417548

Please give complete instructions on how to reproduce.

This change is from upstream so preferably instead report an issue at https://github.com/apptainer/apptainer.  There were bugs with read-only and fuse2fs in 1.4.3 that were patched in 1.4.4 so perhaps there are more.

this is an example command
apptainer --debug shell --overlay /path/to/overlay.img /path/to/image.sif

we will likely report upstream too as there are a couple similar issues already, as you said with "fixed in 1.4.4"

thanks

That works for me so more details are needed, such as how do you create the overlay image and sif images.  Complete instructions from scratch to reproduce.

here is a full reproduction as we received from the internal group that manages the containers, full execution with 1.4.4:

===
# Apptainer version
$ apptainer --version
apptainer version 1.4.4-1.el9

# Container destination path ("outdir" variable)
$ outdir="./my_dir/"

# Dockerfile content
$ cat fuse_err/Dockerfile 
FROM registry.access.redhat.com/ubi9/ubi:9.7
ENV LANG=C.UTF-8 \
    LC_ALL=C.UTF-8

# Script to "fill" the image
$ cat installer_examples/fuse_err_img.sh 
example text

# As "user1" make container (.sif)
$ echo ${USER}
user1
$ podman build --cgroup-manager cgroupfs -t fuserr -f fuse_err/Dockerfile fuse_err/
STEP 1/2: FROM registry.access.redhat.com/ubi9/ubi:9.7
STEP 2/2: ENV LANG=C.UTF-8     LC_ALL=C.UTF-8
--> Using cache 4919ea81dc0aaabb30657fe52dc3d1ec53f753aee5ff369880114b2904b7c576
COMMIT fuserr
--> 4919ea81dc0a
Successfully tagged localhost/fuserr:latest
4919ea81dc0aaabb30657fe52dc3d1ec53f753aee5ff369880114b2904b7c576
$ podman tag fuserr localhost:5000/fuserr
$ podman save fuserr -o ${outdir}/fuserr.tar
Copying blob 5a45357d82b6 done   | 
Copying config 4919ea81dc done   | 
Writing manifest to image destination
$ apptainer build ${outdir}/fuserr.sif docker-archive:${outdir}/fuserr.tar
INFO:    Starting build...
INFO:    Fetching OCI image...
INFO:    Extracting OCI image...
2025/12/03 13:57:01  warn rootless{usr/bin/newgidmap} ignoring (usually) harmless EPERM on setxattr "security.capability"
2025/12/03 13:57:01  warn rootless{usr/bin/newuidmap} ignoring (usually) harmless EPERM on setxattr "security.capability"
2025/12/03 13:57:03  warn xattr{usr/bin/write} ignoring ENOTSUP on setxattr "user.rootlesscontainers"
2025/12/03 13:57:03  warn xattr{${outdir}/apptainer_tmpdir/build-temp-4238526265/rootfs/usr/bin/write} destination filesystem does not support xattrs, further warnings will be suppressed
INFO:    Inserting Apptainer configuration...
INFO:    Creating SIF file...
[==============================================================================================================================================================] 100 % 0s
INFO:    Build complete: ${outdir}/fuserr.sif

# As "user1" makes image (.img)
$ echo ${USER}
user1
$ apptainer overlay create --create-dir /opt/ --size 5000 fuse_err_overlay.img                        
INFO:    Creating overlay image for use without fakeroot.
INFO:    Consider re-running with --fakeroot option.
$ chmod a+rx /clusterstorage/t-rcds/cpetrini/containers/fuserr.sif
$ apptainer exec --bind ${PWD} --overlay fuse_err_overlay.img ${outdir}/fuserr.sif sh installer_examples/fuse_err_img.sh
e2fsck -y -f fuse_err_overlay.img 
e2fsck 1.46.5 (30-Dec-2021)
Pass 1: Checking inodes, blocks, and sizes 
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
fuse_err_overlay.img: 15/320000 files (0.0% non-contiguous), 39003/1280000 blocks
$ resize2fs -M fuse_err_overlay.img 
resize2fs 1.46.5 (30-Dec-2021)
Resizing the filesystem on fuse_err_overlay.img to 17227 (4k) blocks.
The filesystem on fuse_err_overlay.img is now 17227 (4k) blocks long.
$ chmod a+rx fuse_err_overlay.img

# As user1 run container loading image
$ echo ${USERNAME}
user1
$ apptainer shell --overlay fuse_err_overlay.img:ro ${outdir}/fuserr.sif
Apptainer> ls /opt/
fuserr.txt
Apptainer> 

# As "user2" run container loading the image (both owned by "user1", rx for "user2")
$ echo ${USER}
user2
$ apptainer shell --overlay fuse_err_overlay.img:ro ${outdir}/fuserr.sif
FATAL:   container creation failed: image driver mount failure: image driver fuse2fs instance exited with error: fuse2fs exited: FUSE2FS (3): Permission denied.
FUSE2FS (3): Please run e2fsck -fy. 
===

Thank you for those details.  It turns out this is not an upstream problem, it is only a problem with the EPEL/Fedora build.  The patch files that were supposed to fix this problem in 1.4.4 were empty in src.fedoraproject.org!  I built 1.4.5 yesterday so will fix this in a 1.4.5-2.

FEDORA-2025-df330356b2 (apptainer-1.4.5-2.fc41) has been submitted as an update to Fedora 41.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-df330356b2

FEDORA-EPEL-2025-32198453b0 (apptainer-1.4.5-2.el10_2) has been submitted as an update to Fedora EPEL 10.2.
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2025-32198453b0

FEDORA-EPEL-2025-473cf23bc7 (apptainer-1.4.5-2.el9) has been submitted as an update to Fedora EPEL 9.
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2025-473cf23bc7

FEDORA-EPEL-2025-8e02728fbe (apptainer-1.4.5-2.el10_1) has been submitted as an update to Fedora EPEL 10.1.
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2025-8e02728fbe

FEDORA-2025-ff963b3775 (apptainer-1.4.5-2.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-ff963b3775

FEDORA-EPEL-2025-32198453b0 has been pushed to the Fedora EPEL 10.2 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2025-32198453b0

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-EPEL-2025-473cf23bc7 has been pushed to the Fedora EPEL 9 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2025-473cf23bc7

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-EPEL-2025-8e02728fbe has been pushed to the Fedora EPEL 10.1 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2025-8e02728fbe

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-EPEL-2025-44da2a402e has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2025-44da2a402e

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2025-cf169a01e8 has been pushed to the Fedora 43 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-cf169a01e8`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-cf169a01e8

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2025-ff963b3775 has been pushed to the Fedora 42 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-ff963b3775`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-ff963b3775

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2025-df330356b2 has been pushed to the Fedora 41 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-df330356b2`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-df330356b2

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-EPEL-2025-32198453b0 (apptainer-1.4.5-2.el10_2) has been pushed to the Fedora EPEL 10.2 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-EPEL-2025-473cf23bc7 (apptainer-1.4.5-2.el9) has been pushed to the Fedora EPEL 9 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-2025-ff963b3775 (apptainer-1.4.5-2.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-EPEL-2025-8e02728fbe (apptainer-1.4.5-2.el10_1) has been pushed to the Fedora EPEL 10.1 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-2025-cf169a01e8 (apptainer-1.4.5-2.fc43) has been pushed to the Fedora 43 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-EPEL-2025-44da2a402e (apptainer-1.4.5-2.el8) has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-2025-df330356b2 (apptainer-1.4.5-2.fc41) has been pushed to the Fedora 41 stable repository.
If problem still persists, please make note of it in this bug report.

*** Bug 2423028 has been marked as a duplicate of this bug. ***

*** Bug 2423029 has been marked as a duplicate of this bug. ***

*** Bug 2423030 has been marked as a duplicate of this bug. ***

*** Bug 2423031 has been marked as a duplicate of this bug. ***

*** Bug 2423032 has been marked as a duplicate of this bug. ***