RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1311544 - atomic scan tracebacks on scanning all images
Summary: atomic scan tracebacks on scanning all images
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: atomic
Version: 7.2
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Brent Baude
QA Contact: atomic-bugs@redhat.com
URL:
Whiteboard:
Depends On: 1343939 1343944 1346942
Blocks: 1296594 1313485
TreeView+ depends on / blocked
 
Reported: 2016-02-24 12:39 UTC by Jan Černý
Modified: 2016-06-23 16:21 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-06-23 16:21:23 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1273 0 normal SHIPPED_LIVE atomic bug fix and enhancement update 2016-06-23 20:13:11 UTC

Description Jan Černý 2016-02-24 12:39:56 UTC
Description of problem:

I use atomic scan command to scan containers and images for vulnerabilities.
I scan using super privileged container which is built with Dockerfile from here:
https://github.com/OpenSCAP/openscap-daemon/tree/master/atomic/rhel7_spc
I can scan of a single image or container successfully.
However, when I try to scan all images using
atomic scan --images or atomic scan --all,
the command crashes and I got a traceback.

Version-Release number of selected component (if applicable):

atomic-1.6-6.gitca1e384.el7.x86_64

How reproducible:

always

Steps to Reproduce:

1. Build, run and install RHEL7 SPC as described on https://github.com/OpenSCAP/openscap-daemon/blob/master/atomic/
2. atomic scan --images
3. see the traceback

Actual results:

[root@localhost ~]# atomic scan --images

Scanning...

Traceback (most recent call last):
  File "/usr/bin/atomic", line 420, in <module>
    sys.exit(args.func())
  File "/usr/lib/python2.7/site-packages/Atomic/atomic.py", line 485, in scan
    clean = util.print_scan_summary(scan_return, input_resolve)
  File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 124, in print_scan_summary
    template = "{0:" + str(max_width) + "}   {1:5} {2:5} {3:5} {4:5}"
UnboundLocalError: local variable 'max_width' referenced before assignment


Expected results:

Displayed results of CVE scan of all images. No traceback.


Additional info:

Comment 2 Jan Černý 2016-02-24 14:43:59 UTC
Applying this patch: https://github.com/projectatomic/atomic/commit/5fd44bbb58bbc86213bd726563d7bdc554d73cfa to /usr/lib/python2.7/site-packages/Atomic/util.py fixes the traceback.

I spent some time playing with atomic scan, scanning multiple images and containers and it worked OK for me with that patch applied.

Comment 3 Alex Jia 2016-02-25 08:10:01 UTC
(In reply to Jan Černý from comment #0)

> [root@localhost ~]# atomic scan --images
> 
> Scanning...
> 
> Traceback (most recent call last):
>   File "/usr/bin/atomic", line 420, in <module>
>     sys.exit(args.func())
>   File "/usr/lib/python2.7/site-packages/Atomic/atomic.py", line 485, in scan
>     clean = util.print_scan_summary(scan_return, input_resolve)
>   File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 124, in
> print_scan_summary
>     template = "{0:" + str(max_width) + "}   {1:5} {2:5} {3:5} {4:5}"
> UnboundLocalError: local variable 'max_width' referenced before assignment
> 

I have ever given a test for atomic-1.6-3.gitea18c14.el7.x86_64 w/ docker-1.9.0-8.el7.x86_64, it works well w/o above error, and also work on atomic-1.8-4.git958d939.el7.x86_64 w/ docker-1.9.1-12.el7.x86_64.

Comment 4 Jan Černý 2016-03-01 16:53:32 UTC
(In reply to Alex Jia from comment #3)
> (In reply to Jan Černý from comment #0)
> 
> > [root@localhost ~]# atomic scan --images
> > 
> > Scanning...
> > 
> > Traceback (most recent call last):
> >   File "/usr/bin/atomic", line 420, in <module>
> >     sys.exit(args.func())
> >   File "/usr/lib/python2.7/site-packages/Atomic/atomic.py", line 485, in scan
> >     clean = util.print_scan_summary(scan_return, input_resolve)
> >   File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 124, in
> > print_scan_summary
> >     template = "{0:" + str(max_width) + "}   {1:5} {2:5} {3:5} {4:5}"
> > UnboundLocalError: local variable 'max_width' referenced before assignment
> > 
> 
> I have ever given a test for atomic-1.6-3.gitea18c14.el7.x86_64 w/
> docker-1.9.0-8.el7.x86_64, it works well w/o above error, and also work on
> atomic-1.8-4.git958d939.el7.x86_64 w/ docker-1.9.1-12.el7.x86_64.

Why have you tested these particular versions of atomic? My bug occurs on 
atomic-1.6-6.gitca1e384.el7.x86_64. Which version is the most recent version available in RHEL 7.2 ?

I had a look into the code in Atomic/util.py, I can see that the variable max_width is undefined if len(names) == 0, so the traceback is inevitable.

Did you test scanning without specifying image ids or names?
eg. # atomic scan --all

Comment 5 Alex Jia 2016-03-02 03:10:08 UTC
(In reply to Jan Černý from comment #4)
> > I have ever given a test for atomic-1.6-3.gitea18c14.el7.x86_64 w/
> > docker-1.9.0-8.el7.x86_64, it works well w/o above error, and also work on
> > atomic-1.8-4.git958d939.el7.x86_64 w/ docker-1.9.1-12.el7.x86_64.
> 
> Why have you tested these particular versions of atomic? My bug occurs on 
> atomic-1.6-6.gitca1e384.el7.x86_64. Which version is the most recent version
> available in RHEL 7.2 ?
> 

The atomic-1.6-3.gitea18c14.el7.x86_64 is a fixed version for bug 1272037, and the atomic-1.8-4.git958d939.el7.x86_64 is a errata version in that time. But anyway, you're right, I also should double check recent atomic version in RHEL 7.2.

> I had a look into the code in Atomic/util.py, I can see that the variable
> max_width is undefined if len(names) == 0, so the traceback is inevitable.
> 

Yes, I also noticed this, it's a python syntax error.

> Did you test scanning without specifying image ids or names?
> eg. # atomic scan --all

Yes, I indeed done this, for details, please see  Comment 24 in bug 1272037.

Comment 6 Daniel Walsh 2016-06-03 19:55:58 UTC
Fixed in atomic-1.10

Comment 11 Alex Jia 2016-06-16 09:31:39 UTC
Well, rhel7/openscap image is managed by atomic, so we need to run atomic install and run to execute INSTALL and RUN method firstly.

The current question is atomic scan will automatically pull rhel7/openscap image from repo if it doesn't exist on the local, but atomic scan hasn't execute atomic install and run method automatically.

It will be nice if we can help complete atomic install and run together, or if we don't wanna do this, at least, we should document this for users, and we should let users to pull rhel7/openscap image rather than automatically pull it by atomic scan.

Comment 14 Alex Jia 2016-06-16 15:01:52 UTC
Moving the bug to VERIFIED status per above workarround.

Comment 15 Daniel Walsh 2016-06-16 20:37:53 UTC
A workaround that set setenforce 0 is not a work around.  Is this testing using the unlock/overlayfs?  SELinux will not work with that, If this is fully up2date system with the latest packages, can you attach the avc's that you are seeing?  If this works on a RHEL system it should work on a RHELAH system in enforcing mode.

Comment 16 Alex Jia 2016-06-17 03:52:02 UTC
(In reply to Daniel Walsh from comment #15)
> A workaround that set setenforce 0 is not a work around.  Is this testing
> using the unlock/overlayfs?  SELinux will not work with that, If this is
> fully up2date system with the latest packages, can you attach the avc's that
> you are seeing?  If this works on a RHEL system it should work on a RHELAH
> system in enforcing mode.

Daniel, yes, because the current rhelah 7.2.5(5b82b4035f) hasn't included atomic-1.10.5-5, and it's failed[1] to upgrade Atomic Host since yesterday, in order to continue to push the testing forward, so I use atomic host unlock to install latest atomic on Atomic Host, for details, please see Comment 12. 

Daniel, need I wait for new rhelah 7.2.5 build(ship atomic-1.10.5-5) then give a try again? BTW, it indeed works on RHEL7 system w/ SELinux enforcing mode.


[1] error: Server returned status 500: Internal Server Error

[cloud-user@atomic-host-001 ~]$ sudo atomic host status
  TIMESTAMP (UTC)         VERSION     ID             OSNAME               REFSPEC                                                   
* 2016-06-13 17:33:11     7.2.5       5b82b4035f     rhel-atomic-host     rhel-atomic-host-ostree:rhel-atomic-host/7/x86_64/standard
  2016-05-06 05:57:30     7.2.4       b060975ce3     rhel-atomic-host     rhel-atomic-host-ostree:rhel-atomic-host/7/x86_64/standard
[cloud-user@atomic-host-001 ~]$ rpm -q atomic docker
atomic-1.10.5-3.el7.x86_64
docker-1.10.3-40.el7.x86_64

Comment 18 Daniel Walsh 2016-06-17 13:58:36 UTC
So you are not seeing the devicemapper problem?

Comment 19 Alex Jia 2016-06-17 16:48:13 UTC
(In reply to Daniel Walsh from comment #18)
> So you are not seeing the devicemapper problem?

Daniel, I still can see devicemapper problem[1], but atomic scanner can get scanning report w/o original issue in Comment 0.

[1] Error deleting container: Error response from daemon: Driver devicemapper failed to remove root filesystem a6c92e1b9b13e3351752b3b92b8dc14bf2ffeda8f34c777a3e1ce57d1854223d: remove /var/lib/docker/devicemapper/mnt/3680338ab00c3f2a4a6b3f2a74cc1bde6de53fd13f922b727da4f4d13358dff0: device or resource busy

The issue is tracked in bug 1347037.

500 Server Error: Internal Server Error ("devmapper: Unknown device 3797efd119f0691a9b73c868fecc1f29649dc3d7d2109830c6f562f5ac69fd40")

Sometimes, I can see above issue.

Comment 20 Daniel Walsh 2016-06-17 19:17:09 UTC
Alex add 

MountFlags=slave

to docker.service file and then restart docker daemon
Then try the scan again.

This is the fix we are working on now.

Comment 21 Alex Jia 2016-06-18 10:24:08 UTC
(In reply to Daniel Walsh from comment #20)
> Alex add 
> 
> MountFlags=slave
> 
> to docker.service file and then restart docker daemon
> Then try the scan again.
> 
> This is the fix we are working on now.

Daniel, thanks, I gave a try, but I can't use cloud-user account to edit /usr/lib/systemd/system/docker.service, even though using sudo.


[cloud-user@atomic-host-001 ~]$ sudo ls -lZ /usr/lib/systemd/system/docker.service
-rw-r--r--. root root system_u:object_r:docker_unit_file_t:s0 /usr/lib/systemd/system/docker.service

I will give a try w/ root user on rhelah or RHEL7 system later today.

Comment 22 Alex Jia 2016-06-18 15:01:34 UTC
Daniel, the root user also can't edit /usr/lib/systemd/system/docker.service in rhelah 7.2.5.

-bash-4.2# atomic host status
  TIMESTAMP (UTC)         VERSION     ID             OSNAME               REFSPEC                                                   
* 2016-06-15 21:08:10     7.2.5       c6530479e2     rhel-atomic-host     rhel-atomic-host-ostree:rhel-atomic-host/7/x86_64/standard
  2016-04-12 14:20:45     7.2.3-1     644fcc6035     rhel-atomic-host     rhel-atomic-host-ostree:rhel-atomic-host/7/x86_64/standard

-bash-4.2# id
uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
-bash-4.2# ll -Z /usr/lib/systemd/system/docker.service
-rw-r--r--. root root system_u:object_r:docker_unit_file_t:s0 /usr/lib/systemd/system/docker.service

And it seems MountFlags=slave works in my rhel7 system, I haven't see devicemapper errors, but I still see many mount points haven't been unmounted.

<slice>

/dev/dm-77             10474496  1030740   9443756  10% /run/atomic/2016-06-18-10-55-05-624109/sha256:a05ba1bc496e2bfb07ecae7ebabad56173da3b317576eea7cd00caf366352469
/dev/dm-78             10474496   228808  10245688   3% /run/atomic/2016-06-18-10-55-05-624109/sha256:bdfeab864e127ee6c87368d98fa1338b673e408254be52e22e8730b48e7ac499

</slice>

Comment 23 Daniel Walsh 2016-06-20 18:35:28 UTC
What I did to test this was to create a copy of docker.service to /etc/systemd/system/docker1.service

Add the mount line to docker1.service
systemctl stop docker
systemctl start docker1

Comment 24 Alex Jia 2016-06-22 07:00:56 UTC
(In reply to Daniel Walsh from comment #23)
> What I did to test this was to create a copy of docker.service to
> /etc/systemd/system/docker1.service

Daniel, it works well for me, need I open a bug to track the issue? or fix it with bug 1347037 together?

Comment 26 errata-xmlrpc 2016-06-23 16:21:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1273


Note You need to log in before you can comment on or make changes to this bug.