Steps to retest:
1. Have a satellite spin up to work on
2. Spin up a rhel box for the NFS share
3. On both systems, run: "rpm -q nfs-utils"
a. This checks if the system has nfs already installed (it should)
4. On NFS share, run "systemctl status nfs-server" to check if NFS is running or enabled
5. If not, then enabled the it via "systemctl enable nfs-server --now" and recheck status. It should be active.
6. run: “systemctl start nfs-server”, then “systemctl status nfs-server” to ensure that running NFS-server is running.
7. Ensure that the dir /mnt is blank because we're using this location for the export but you can put it anywhere or create a separate dir for this
8. Create an export point by adding "/mnt <sat-server>(rw)" to /etc/exports using vim
9. Stop firewall just in case: systemctl stop firewalld
10. Run: exportfs -r
11. set permission to write into /mnt with chmod 777 /mnt
12. On the satellite, create a separate folder (pulp1)
13. Grant all permission to pulp1 and /var/lib/pulp/ by running: chown -R pulp:pulp /var/lib/pulp/ and chown -R pulp:pulp pulp1
14. run: mount -t nfs <nfs-share server name>:/mnt pulp1 -v
a. It should output something like this for status:
mount.nfs: timeout set for Fri Mar 18 14:25:52 2022
mount.nfs: trying text-based options
15. copy everything in /var/lib/pulp to the share: cp -r /var/lib/pulp/* pulp1
16. On NFS, check /mnt to ensure that the folder structure of /var/lib/pulp is in there (it should have assets, import, exports, tmp, etc)
17. Check permission of /var/lib/pulp and it's varying directory-> ls -lZ /var/lib/pulp
18. Check permission on /mnt on the NFS to compare the permissions between the two servers
19. Check the uid an gid of sat for any of the users from steps 17: grep <user> /etc/passwd
20. On NFS, create group, user, and grant necessary permissions: groupadd -g <gid> <user>, adduser -g <user> -u <uid> <user>, chown -R <user:user> /mnt
21. Ensure that the uid and gid matches between NFS share and sat: grep <user> /etc/passwd
22. run: umount <NFS share hostname>:/mnt
23. Remount the NFS share to /var/lib/pulp/ with: mount -t nfs <nfs-share server name>:/mnt /var/lib/pulp/ -v
a. Steps 14-18 is necessary so the NFS share has the same file structure as /var/lib/pulp in the export in /mnt. Then we can mount to /var/lib/pulp/ so that any writing into that dir in satellite can
be directed to the NFS share one. If we don't do this, and we mount directly to /var/lib/pulp/, then pulp cannot work if all its contents was removed by mounting the empty filesystem
24. run: restorecon -Rv /var/lib/pulp
25. Restart pulp services or all services on satellite: foreman-maintain service restart
26. run: showmount -e <NSF share hostname> to ensure mount is still active
27. Create a custom repo and sync
28. Create a cv, add the repos, and publish
Expected result:
Syncing should complete successfully.
Publishing cv should complete successfully
Actual result:
Syncing completed successfully
Publishing cv completed successfully
I also tried doing migration and it seems to work fine.
Verified on 6.9.9 snap 2
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Important: Satellite 6.9.9 Async Bug Fix Update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2022:1478
Steps to retest: 1. Have a satellite spin up to work on 2. Spin up a rhel box for the NFS share 3. On both systems, run: "rpm -q nfs-utils" a. This checks if the system has nfs already installed (it should) 4. On NFS share, run "systemctl status nfs-server" to check if NFS is running or enabled 5. If not, then enabled the it via "systemctl enable nfs-server --now" and recheck status. It should be active. 6. run: “systemctl start nfs-server”, then “systemctl status nfs-server” to ensure that running NFS-server is running. 7. Ensure that the dir /mnt is blank because we're using this location for the export but you can put it anywhere or create a separate dir for this 8. Create an export point by adding "/mnt <sat-server>(rw)" to /etc/exports using vim 9. Stop firewall just in case: systemctl stop firewalld 10. Run: exportfs -r 11. set permission to write into /mnt with chmod 777 /mnt 12. On the satellite, create a separate folder (pulp1) 13. Grant all permission to pulp1 and /var/lib/pulp/ by running: chown -R pulp:pulp /var/lib/pulp/ and chown -R pulp:pulp pulp1 14. run: mount -t nfs <nfs-share server name>:/mnt pulp1 -v a. It should output something like this for status: mount.nfs: timeout set for Fri Mar 18 14:25:52 2022 mount.nfs: trying text-based options 15. copy everything in /var/lib/pulp to the share: cp -r /var/lib/pulp/* pulp1 16. On NFS, check /mnt to ensure that the folder structure of /var/lib/pulp is in there (it should have assets, import, exports, tmp, etc) 17. Check permission of /var/lib/pulp and it's varying directory-> ls -lZ /var/lib/pulp 18. Check permission on /mnt on the NFS to compare the permissions between the two servers 19. Check the uid an gid of sat for any of the users from steps 17: grep <user> /etc/passwd 20. On NFS, create group, user, and grant necessary permissions: groupadd -g <gid> <user>, adduser -g <user> -u <uid> <user>, chown -R <user:user> /mnt 21. Ensure that the uid and gid matches between NFS share and sat: grep <user> /etc/passwd 22. run: umount <NFS share hostname>:/mnt 23. Remount the NFS share to /var/lib/pulp/ with: mount -t nfs <nfs-share server name>:/mnt /var/lib/pulp/ -v a. Steps 14-18 is necessary so the NFS share has the same file structure as /var/lib/pulp in the export in /mnt. Then we can mount to /var/lib/pulp/ so that any writing into that dir in satellite can be directed to the NFS share one. If we don't do this, and we mount directly to /var/lib/pulp/, then pulp cannot work if all its contents was removed by mounting the empty filesystem 24. run: restorecon -Rv /var/lib/pulp 25. Restart pulp services or all services on satellite: foreman-maintain service restart 26. run: showmount -e <NSF share hostname> to ensure mount is still active 27. Create a custom repo and sync 28. Create a cv, add the repos, and publish Expected result: Syncing should complete successfully. Publishing cv should complete successfully Actual result: Syncing completed successfully Publishing cv completed successfully I also tried doing migration and it seems to work fine. Verified on 6.9.9 snap 2