Hide Forgot
Description of problem: # atomic --debug scan --scanner openscap --scan_type cve registry.access.redhat.com/rhel7 Created /run/atomic/2016-10-04-14-31-44-840717 docker run -t --rm -v /etc/localtime:/etc/localtime -v /run/atomic/2016-10-04-14-31-44-840717:/scanin -v /var/lib/atomic/openscap/2016-10-04-14-31-44-840717:/scanout:rw,Z -v /etc/oscapd:/etc/oscapd:ro rhel7/openscap oscapd-evaluate scan --no-standard-compliance --targets chroots-in-dir:///scanin --output /scanout Created /run/atomic/2016-10-04-14-31-44-840717/98a88a8b722a71835dd761c88451c681a8f1bc6e577f90d4dc8b234100bd4861 Unmounted /run/atomic/2016-10-04-14-31-44-840717/98a88a8b722a71835dd761c88451c681a8f1bc6e577f90d4dc8b234100bd4861 Atomic mount is not supported on the overlay2 docker storage backend. Traceback (most recent call last): File "/usr/bin/atomic", line 184, in <module> sys.exit(_func()) File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 156, in scan self._mount_scan_rootfs(scan_list) File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 244, in _mount_scan_rootfs self.mount(mountpoint=mount_path, image=docker_object['Id']) File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 455, in mount m.mount() File "/usr/lib/python2.7/site-packages/Atomic/mount.py", line 150, in mount raise ValueError(dme) ValueError: Atomic mount is not supported on the overlay2 docker storage backend Version-Release number of selected component (if applicable): atomic-1.12.1-3.el7.x86_64 How reproducible: always
Not sure if there is a plan to fix this code, but I just create this BZ for tracking and document as a known issue in 7.3 note.
Brent is this fixed in atomic-1.13?
No, I wasnt aware we support overlay2 now, do we?
Yes devicemapper,overlay and overlay2. I fixed all of the atomic mount code, I have a feeling that this will just work.
Any progress?
Should be fixed in atomic-1.14.
Unfortunately, it is still broken as version atomic-1.14.1-5.el7.x86_64 # atomic --debug scan --scanner openscap --scan_type cve registry.access.redhat.com/rhel7 Created /run/atomic/2017-01-26-10-46-01-673026 docker run -t --rm -v /etc/localtime:/etc/localtime -v /run/atomic/2017-01-26-10-46-01-673026:/scanin -v /var/lib/atomic/openscap/2017-01-26-10-46-01-673026:/scanout:rw,Z -v /etc/oscapd:/etc/oscapd:ro rhel7/openscap oscapd-evaluate scan --no-standard-compliance --targets chroots-in-dir:///scanin --output /scanout -j1 Created /run/atomic/2017-01-26-10-46-01-673026/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 Mounted {u'Created': 1484254315, u'Labels': {u'com.redhat.component': u'rhel-server-docker', u'authoritative-source-url': u'registry.access.redhat.com', u'distribution-scope': u'public', u'vendor': u'Red Hat, Inc.', u'Name': u'rhel7', u'io.k8s.display-name': u'Red Hat Enterprise Linux 7', u'description': u'The Red Hat Enterprise Linux Base image is designed to be a fully supported foundation for your containerized applications. This base image provides your operations and application teams with the packages, language runtimes and tools necessary to run, maintain, and troubleshoot all of your applications. This image is maintained by Red Hat and updated regularly. It is designed and engineered to be the base layer for all of your containerized applications, middleware and utilites. When used as the source for all of your containers, only one copy will ever be downloaded and cached in your production environment. Use this image just like you would a regular Red Hat Enterprise Linux distribution. Tools like yum, gzip, and bash are provided by default. For further information on how this image was built look at the /root/anacanda-ks.cfg file.', u'summary': u'Provides the latest release of Red Hat Enterprise Linux 7 in a fully featured and supported base image.', u'vcs-type': u'git', u'name': u'rhel7', u'vcs-ref': u'06e55ffd458c665f861599ac9c7550a037d85ac7', u'release': u'66', u'Version': u'7.3', u'architecture': u'x86_64', u'version': u'7.3', u'Release': u'66', u'BZComponent': u'rhel-server-docker', u'build-date': u'2017-01-12T15:36:30.088642', u'io.openshift.tags': u'base rhel7', u'com.redhat.build-host': u'rcm-img-docker02.build.eng.bos.redhat.com'}, 'ImageId': u'e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3', u'VirtualSize': 192540107, u'ParentId': u'', 'input': 'registry.access.redhat.com/rhel7', u'RepoTags': [u'registry.access.redhat.com/rhel7:latest'], u'RepoDigests': [u'registry.access.redhat.com/rhel7@sha256:0614d58c96e8d1a04a252880a6c33b48b4685cafae048a70dd9e821edf62cab9'], u'Id': u'e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3', 'ImageType': 'Docker', u'Size': 192540107} to /run/atomic/2017-01-26-10-46-01-673026/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 Creating the output dir at /var/lib/atomic/openscap/2017-01-26-10-46-01-673026 Unable to find image 'rhel7/openscap:latest' locally Trying to pull repository registry.access.redhat.com/rhel7/openscap ... sha256:db1206c551c8117a53cb89c4f3ccd6072d2e03e5d6d0ad029b751864429dbd55: Pulling from registry.access.redhat.com/rhel7/openscap 7bd78273b666: Already exists c196631bd9ac: Already exists db7cef4d643b: Pull complete e3b3e87ac388: Pull complete Digest: sha256:db1206c551c8117a53cb89c4f3ccd6072d2e03e5d6d0ad029b751864429dbd55 Status: Downloaded newer image for registry.access.redhat.com/rhel7/openscap:latest INFO:OpenSCAP Daemon one-off evaluator 0.1.6 INFO:Autodetected "oscap" in path "/usr/bin/oscap". INFO:Autodetected "oscap-ssh" in path "/usr/bin/oscap-ssh". INFO:Autodetected "oscap-vm" in path "/usr/bin/oscap-vm". INFO:Autodetected "oscap-docker" in path "/usr/bin/oscap-docker". INFO:Autodetected "oscap-chroot" in path "/usr/bin/oscap-chroot". WARNING:Can't import the 'docker' package. Container scanning functionality will be disabled. INFO:Autodetected SCAP content at "/usr/share/openscap/cpe/openscap-cpe-oval.xml". INFO:Autodetected SCAP content in path "/usr/share/xml/scap/ssg/content". INFO:Creating tasks directory at '/var/lib/oscapd/tasks' because it didn't exist. INFO:Creating results directory at '/var/lib/oscapd/results' because it didn't exist. INFO:Creating results work in progress directory at '/var/lib/oscapd/work_in_progress' because it didn't exist. INFO:Evaluated EvaluationSpec, exit_code=0. INFO:Had a local version of /var/lib/oscapd/cve_feeds/com.redhat.rhsa-RHEL7.xml but it wasn't new enough INFO:Evaluated EvaluationSpec, exit_code=0. INFO:[100.00%] Scanned target 'chroot:///scanin/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3' registry.access.redhat.com/rhel7 (e4b79d4d89ab9b0) registry.access.redhat.com/rhel7 passed the scan Files associated with this scan are in /var/lib/atomic/openscap/2017-01-26-10-46-01-673026. The device mounted at /run/atomic/2017-01-26-10-46-01-673026/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 is not a docker container. Traceback (most recent call last): File "/bin/atomic", line 187, in <module> sys.exit(_func()) File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 174, in scan self._unmount_rootfs_in_dir() File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 253, in _unmount_rootfs_in_dir self.unmount(rootfs_dir) File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 461, in unmount m.unmount() File "/usr/lib/python2.7/site-packages/Atomic/mount.py", line 210, in unmount raise ValueError(dme) ValueError: The device mounted at /run/atomic/2017-01-26-10-46-01-673026/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 is not a docker container.
This is probably a different issue. Could you see if atomic mount works with overlay2?
# atomic mount --storage docker registry.access.redhat.com/rhel7/openscap /mnt/ # atomic umount /mnt/ The device mounted at /mnt/ is not a docker container. # ls /mnt/ bin dev home lib64 media opt root sbin sys usr boot etc lib lost+found mnt proc run srv tmp var # mount | grep overlay /dev/vda1 on /var/lib/docker-latest/overlay2 type xfs (rw,relatime,seclabel,attr2,inode64,noquota) overlay on /run/atomic/2017-01-26-10-46-01-673026/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 type overlay (ro,relatime,seclabel,lowerdir=/var/lib/docker-latest/overlay2/463b4fd8f57c9500b213eb58264288247c442a01647985f4af77b9641c044397-init/diff:/var/lib/docker-latest/overlay2/ce73c6685a214a803fb20508ddfabd283ba016d898506bb564eb64b12f0bf498/diff:/var/lib/docker-latest/overlay2/4c2009b40a94e8f81e1afd7e3f52d91f4b55e294dc500c547635cc582e298dd1/diff,upperdir=/var/lib/docker-latest/overlay2/463b4fd8f57c9500b213eb58264288247c442a01647985f4af77b9641c044397/diff,workdir=/var/lib/docker-latest/overlay2/463b4fd8f57c9500b213eb58264288247c442a01647985f4af77b9641c044397/work) overlay on /run/atomic/2017-01-26-10-48-12-762259/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 type overlay (ro,relatime,seclabel,lowerdir=/var/lib/docker-latest/overlay2/c4f640ff1809e085931c8ae20ccadb13f8f013cfe05290c5527b258e929fa92e-init/diff:/var/lib/docker-latest/overlay2/ce73c6685a214a803fb20508ddfabd283ba016d898506bb564eb64b12f0bf498/diff:/var/lib/docker-latest/overlay2/4c2009b40a94e8f81e1afd7e3f52d91f4b55e294dc500c547635cc582e298dd1/diff,upperdir=/var/lib/docker-latest/overlay2/c4f640ff1809e085931c8ae20ccadb13f8f013cfe05290c5527b258e929fa92e/diff,workdir=/var/lib/docker-latest/overlay2/c4f640ff1809e085931c8ae20ccadb13f8f013cfe05290c5527b258e929fa92e/work) overlay on /mnt type overlay (ro,relatime,seclabel,lowerdir=/var/lib/docker-latest/overlay2/c450efdb4a3396d02a26347230b2855308d85cc130923faee1a3c6c75936d332-init/diff:/var/lib/docker-latest/overlay2/85fdd0a227fa59ade8f738ce97ba473a4e110de802a331bb0911b1c5e8ded468/diff:/var/lib/docker-latest/overlay2/4905133fc0e56b008a6efceebd02e8ad236a8142540dcdf93825c98177eadcc3/diff:/var/lib/docker-latest/overlay2/ce73c6685a214a803fb20508ddfabd283ba016d898506bb564eb64b12f0bf498/diff:/var/lib/docker-latest/overlay2/4c2009b40a94e8f81e1afd7e3f52d91f4b55e294dc500c547635cc582e298dd1/diff,upperdir=/var/lib/docker-latest/overlay2/c450efdb4a3396d02a26347230b2855308d85cc130923faee1a3c6c75936d332/diff,workdir=/var/lib/docker-latest/overlay2/c450efdb4a3396d02a26347230b2855308d85cc130923faee1a3c6c75936d332/work)
It looks atomic umount NOT work well with overlay2.
On the other hand, umount works fine. # umount /mnt # umount /run/atomic/2017-01-26-10-48-12-762259/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 # umount /run/atomic/2017-01-26-10-48-12-762259/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 # mount | grep overlay /dev/vda1 on /var/lib/docker-latest/overlay2 type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
OK, I found culprit. There is several hard-coded /var/lib/docker/ places in Atomic/mount.py which won't work with docker-latest. Once I fixed them, it works fine again. We will just need a patch to copy with both docker and docker-latest. # diff -u /usr/lib/python2.7/site-packages/Atomic/mount.py.orig /usr/lib/python2.7/site-packages/Atomic/mount.py --- /usr/lib/python2.7/site-packages/Atomic/mount.py.orig 2017-01-26 12:28:26.717999574 -0500 +++ /usr/lib/python2.7/site-packages/Atomic/mount.py 2017-01-26 12:29:40.449999574 -0500 @@ -403,15 +403,15 @@ @staticmethod def _no_gd_api_dm(cid): - desc_file = os.path.join('/var/lib/docker/devicemapper/metadata', cid) + desc_file = os.path.join('/var/lib/docker-latest/devicemapper/metadata', cid) desc = json.loads(open(desc_file).read()) return desc['device_id'], desc['size'] @staticmethod def _no_gd_api_overlay(cid, driver): - prefix = os.path.join('/var/lib/docker/%s/' % driver, cid) + prefix = os.path.join('/var/lib/docker-latest/%s/' % driver, cid) ld_metafile = open(os.path.join(prefix, 'lower-id')) - ld_loc = os.path.join('/var/lib/docker/%s/' % driver, ld_metafile.read()) + ld_loc = os.path.join('/var/lib/docker-latest/%s/' % driver, ld_metafile.read()) return (os.path.join(ld_loc, 'root'), os.path.join(prefix, 'upper'), os.path.join(prefix, 'work')) @@ -679,7 +679,7 @@ upperdir = [o.replace('upperdir=', '') for o in optstring.split(',') if o.startswith('upperdir=')][0] cdir = upperdir.rsplit('/', 1)[0] - if not cdir.startswith('/var/lib/docker/%s/' % driver ): + if not cdir.startswith('/var/lib/docker-latest/%s/' % driver ): raise MountError('The device mounted at %s is not a ' 'docker container.' % self.mountpoint )
I have created a pull request which should fix this. I tested locally with overlay2. Could you please test it and provide feedback? The code can be found at: https://github.com/projectatomic/atomic/pull/852
Let me know if you are able to test this.
Unfortunately, it is still broken with overlay + docker-latest after applied the patch at /usr/lib/python2.7/site-packages/Atomic/mount.py # atomic --debug scan --scanner openscap --scan_type cve registry.access.redhat.com/rhel7 Created /run/atomic/2017-01-30-15-40-08-703680 docker run -t --rm -v /etc/localtime:/etc/localtime -v /run/atomic/2017-01-30-15-40-08-703680:/scanin -v /var/lib/atomic/openscap/2017-01-30-15-40-08-703680:/scanout:rw,Z -v /etc/oscapd:/etc/oscapd:ro rhel7/openscap oscapd-evaluate scan --no-standard-compliance --targets chroots-in-dir:///scanin --output /scanout -j1 Created /run/atomic/2017-01-30-15-40-08-703680/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 Mounted {u'Created': 1484254315, u'Labels': {u'com.redhat.component': u'rhel-server-docker', u'authoritative-source-url': u'registry.access.redhat.com', u'distribution-scope': u'public', u'vendor': u'Red Hat, Inc.', u'Name': u'rhel7', u'io.k8s.display-name': u'Red Hat Enterprise Linux 7', u'description': u'The Red Hat Enterprise Linux Base image is designed to be a fully supported foundation for your containerized applications. This base image provides your operations and application teams with the packages, language runtimes and tools necessary to run, maintain, and troubleshoot all of your applications. This image is maintained by Red Hat and updated regularly. It is designed and engineered to be the base layer for all of your containerized applications, middleware and utilites. When used as the source for all of your containers, only one copy will ever be downloaded and cached in your production environment. Use this image just like you would a regular Red Hat Enterprise Linux distribution. Tools like yum, gzip, and bash are provided by default. For further information on how this image was built look at the /root/anacanda-ks.cfg file.', u'summary': u'Provides the latest release of Red Hat Enterprise Linux 7 in a fully featured and supported base image.', u'vcs-type': u'git', u'name': u'rhel7', u'vcs-ref': u'06e55ffd458c665f861599ac9c7550a037d85ac7', u'release': u'66', u'Version': u'7.3', u'architecture': u'x86_64', u'version': u'7.3', u'Release': u'66', u'BZComponent': u'rhel-server-docker', u'build-date': u'2017-01-12T15:36:30.088642', u'io.openshift.tags': u'base rhel7', u'com.redhat.build-host': u'rcm-img-docker02.build.eng.bos.redhat.com'}, 'ImageId': u'e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3', u'VirtualSize': 192532823, u'ParentId': u'', 'input': 'registry.access.redhat.com/rhel7', u'RepoTags': [u'registry.access.redhat.com/rhel7:latest'], u'RepoDigests': [u'registry.access.redhat.com/rhel7@sha256:0614d58c96e8d1a04a252880a6c33b48b4685cafae048a70dd9e821edf62cab9'], u'Id': u'e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3', 'ImageType': 'Docker', u'Size': 192532823} to /run/atomic/2017-01-30-15-40-08-703680/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 Creating the output dir at /var/lib/atomic/openscap/2017-01-30-15-40-08-703680 INFO:OpenSCAP Daemon one-off evaluator 0.1.6 INFO:Autodetected "oscap" in path "/usr/bin/oscap". INFO:Autodetected "oscap-ssh" in path "/usr/bin/oscap-ssh". INFO:Autodetected "oscap-vm" in path "/usr/bin/oscap-vm". INFO:Autodetected "oscap-docker" in path "/usr/bin/oscap-docker". INFO:Autodetected "oscap-chroot" in path "/usr/bin/oscap-chroot". WARNING:Can't import the 'docker' package. Container scanning functionality will be disabled. INFO:Autodetected SCAP content at "/usr/share/openscap/cpe/openscap-cpe-oval.xml". INFO:Autodetected SCAP content in path "/usr/share/xml/scap/ssg/content". INFO:Creating tasks directory at '/var/lib/oscapd/tasks' because it didn't exist. INFO:Creating results directory at '/var/lib/oscapd/results' because it didn't exist. INFO:Creating results work in progress directory at '/var/lib/oscapd/work_in_progress' because it didn't exist. INFO:Evaluated EvaluationSpec, exit_code=0. INFO:Had a local version of /var/lib/oscapd/cve_feeds/com.redhat.rhsa-RHEL7.xml but it wasn't new enough INFO:Evaluated EvaluationSpec, exit_code=0. INFO:[100.00%] Scanned target 'chroot:///scanin/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3' registry.access.redhat.com/rhel7 (e4b79d4d89ab9b0) registry.access.redhat.com/rhel7 passed the scan Files associated with this scan are in /var/lib/atomic/openscap/2017-01-30-15-40-08-703680. The device mounted at /run/atomic/2017-01-30-15-40-08-703680/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 is not a docker container. Traceback (most recent call last): File "/bin/atomic", line 187, in <module> sys.exit(_func()) File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 174, in scan self._unmount_rootfs_in_dir() File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 253, in _unmount_rootfs_in_dir self.unmount(rootfs_dir) File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 461, in unmount m.unmount() File "/usr/lib/python2.7/site-packages/Atomic/mount.py", line 210, in unmount raise ValueError(dme) ValueError: The device mounted at /run/atomic/2017-01-30-15-40-08-703680/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 is not a docker container.
can I get access to this vm/machine? im not able to replicate anymore with the latest code.
Brent, does atomic-1.15.2-4.el7.x86_64 in atomic host 7.3.3 include the fix for this?
It should. I have been testing against upstream so I cannot speak to those specific versions. Are you able to test it for me and determine if it works?
lets assume it does and let QE prove us wrong... Fixed in atomic-1.15.2-4.el7.x86_64
Unfortunately, it is still broken with overlay2 + docker-latest. The device mounted at /run/atomic/2017-02-23-16-41-15-099039/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 is not a docker container. Traceback (most recent call last): File "/bin/atomic", line 188, in <module> sys.exit(_func()) File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 174, in scan self._unmount_rootfs_in_dir() File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 253, in _unmount_rootfs_in_dir self.unmount(rootfs_dir) File "/usr/lib/python2.7/site-packages/Atomic/scan.py", line 461, in unmount m.unmount() File "/usr/lib/python2.7/site-packages/Atomic/mount.py", line 210, in unmount raise ValueError(dme) ValueError: The device mounted at /run/atomic/2017-02-23-16-41-15-099039/e4b79d4d89ab9b0aa873fd8dc99c652483bb645f317712ef30d5904ac2eafef3 is not a docker container.
can you provide access details so I can verify this and/or a pointer to the image you used? I need to be able to replicate this exactly.
Sent you an email with the access detail.
This works with 1.15.3 [cloud-user@localhost ~]$ sudo atomic mount registry.access.redhat.com/rhel7 /tmp/foo [cloud-user@localhost ~]$ sudo atomic unmount /tmp/foo [cloud-user@localhost ~]$ rpm -q docker-latest docker-latest-1.12.6-10.el7.x86_64 [cloud-user@localhost ~]$ atomic -v 1.15.3
Well, it can only be reproduced if you used overlay or overlay2.
I believe that https://github.com/projectatomic/atomic/pull/912 fixes this.
Fixed in atomic-1.16.2
Since there is no rpm build with it in brew yet, it probably better to move to POST instead of MODIFIED.
I always move things to modified. Never used POST. Does this mean fixed in the next release?
In RHEL CDW, it usually use, POST: the patch has been posted upstream. MODIFIED: the rpm has been build in brew. ON_QA: the build has been attached to the erratum. Not a big deal if you want to use something different in Agile mode.
Lokesh, would you like to add this to the 7.3.4 atomic errata?