Bug 690286
| Summary: | augeas occasionally leaks fd's and returns errors during initialization of netcf | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Laine Stump <laine> | ||||
| Component: | augeas | Assignee: | David Lutterkort <lutter> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 6.1 | CC: | adevolder, apevec, ddugger, hbrock, hui.xiao, jane.lv, jvillalo, jwilleford, jzheng, leiwang, luyu, prc, rwu, whuang, yang.z.zhang | ||||
| Target Milestone: | rc | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | augeas-0.7.2-6.el6 | Doc Type: | Bug Fix | ||||
| Doc Text: |
augeas was not thread safe, and could leak file descriptors when multiple programs attempted to use the libvirt library simultaneously. This resulted in the failure of the calling program. augeas has been modified to remove the global variable that caused this threading issue.
|
Story Points: | --- | ||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2011-05-19 13:54:26 UTC | Type: | --- | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 670961 | ||||||
| Attachments: |
|
||||||
There is one more upstream patch needed to fix this:
commit 35c86e740c268ec7fefa5b32a964419cc5e112c0
Author: David Lutterkort <lutter>
Date: Tue Mar 8 13:44:51 2011 -0800
Be more defensive about closing input files for the lexer
Built augeas-0.7.2-6.el6 to address this issue; see https://brewweb.devel.redhat.com//taskinfo?taskID=3203845 *** Bug 687973 has been marked as a duplicate of this bug. *** verify this bug in RHEL6 with augeas-0.7.2-6.el6 steps : 1.update augeas to 0.7.2-6 2.run the "while true; do virsh iface-list; done" in two shell windows No more show error message 3.run the libvirtd with NETCF_DEBUG=1 No more show warning message like: "augeas initialization had errors" 4.lsof |grep aug libvirtd 2812 root mem REG 8,3 301944 1705059 /usr/lib64/libaugeas.so.0.11.0 perl 17548 root DEL REG 8,3 1733213 /usr/lib64/libaugeas.so.0.11.0 Created attachment 488983 [details]
augeas make check
augeas make check all tests passed
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
New Contents:
augeas was not thread safe, and could leak file descriptors when multiple programs attempted to use the libvirt library simultaneously. This resulted in the failure of the calling program. augeas has been patched to be more threadsafe, so this issue no longer occurs.
Change the last sentence to 'Augeas has been modified to remove the global variable that caused this threading issue' (It is by all accounts not just more threadsafe, but just plain threadsafe now)
Technical note updated. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
Diffed Contents:
@@ -1 +1 @@
-augeas was not thread safe, and could leak file descriptors when multiple programs attempted to use the libvirt library simultaneously. This resulted in the failure of the calling program. augeas has been patched to be more threadsafe, so this issue no longer occurs.+augeas was not thread safe, and could leak file descriptors when multiple programs attempted to use the libvirt library simultaneously. This resulted in the failure of the calling program. augeas has been modified to remove the global variable that caused this threading issue.
*** Bug 635669 has been marked as a duplicate of this bug. *** An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2011-0650.html |
there were reports from a couple of users of a leak of fd's owned by libvirtd, as illustrated by this example output of "lsof | grep aug": libvirtd 20159 root 42r REG 253,1 5478 2112374 /usr/share/augeas/lenses/dist/build.aug libvirtd 20159 root 43r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug libvirtd 20159 root 44r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug libvirtd 20159 root 45r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug libvirtd 20159 root 46r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug libvirtd 20159 root 47r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug libvirtd 20159 root 48r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug libvirtd 20159 root 49r REG 253,1 5478 2112374 /usr/share/augeas/lenses/dist/build.aug libvirtd 20159 root 50r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug libvirtd 20159 root 51r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug libvirtd 20159 root 52r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug libvirtd 20159 root 53r REG 253,1 5478 2112374 /usr/share/augeas/lenses/dist/build.aug libvirtd 20159 root 54r REG 253,1 3557 2112732 /usr/share/augeas/lenses/dist/util.aug This problem can be reproduced by opening two shell windows, and running this command in both: while true; do virsh iface-list; done You will occasionally notice an error scroll past in the windows: error: Failed to list active interfaces error: internal error failed to get number of interfaces on host (netcf: unspecified error - errors in loading some config files) If you run libvirtd with NETCF_DEBUG defined in its environment, you will be able to see more specifics of the errors reported by augeas. They are always similar (but not identical) to this: ======================================= warning: augeas initialization had errors please file a bug with the following lines in the bug report: /augeas/load/Ifcfg/error = "/usr/share/augeas/lenses/dist/util.aug:149:0: Missing *)\n/usr/share/netcf/lenses/sysconfig.aug:7.12-.20:Could not load module Util for Util.eol\n/usr/share/netcf/lenses/sysconfig.aug:7.12-.20:Undefined variable Util.eol\n/usr/share/netcf/lenses/sysconfig.aug:10.11-.23:Could not load module Util for Util.del_str\n/usr/share/netcf/lenses/sysconfig.aug:10.11-.23:Undefined variable Util.del_str\n/usr/share/netcf/lenses/sysconfig.aug:11.16-.28:Could not load module Util for Util.comment\n/usr/share/netcf/lenses/sysconfig.aug:11.16-.28:Undefined variable Util.comment\n/usr/share/netcf/lenses/sysconfig.aug:12.16-.26:Could not load module Util for Util.empty\n/usr/share/netcf/lenses/sysconfig.aug:12.16-.26:Undefined variable Util.empty\n/usr/share/netcf/lenses/sysconfig.aug:13.16-.27:Could not load module Build for Build.xchgs\n/usr/share/netcf/lenses/sysconfig.aug:13.16-.27:Undefined variable Build.xchgs\n/usr/share/netcf/lenses/sysconfig.aug:14.16-.28:Could not load module Util for Util.del_str\n/usr/share/netcf/lenses/sysconfig.aug:14.16-.28:Undefined variable Util.del_str\n/usr/share/netcf/lenses/sysconfig.aug:33.4-.8:Undefined variable dels\n/usr/share/netcf/lenses/sysconfig.aug:35.32-.47:Could not load module Util for Util.del_ws_spc\n/usr/share/netcf/lenses/sysconfig.aug:35.32-.47:Undefined variable Util.del_ws_spc\n/usr/share/netcf/lenses/sysconfig.aug:36.26-.32:Undefined variable export\n/usr/share/netcf/lenses/sysconfig.aug:37.15-.17:Undefined variable kv\n/usr/share/netcf/lenses/sysconfig.aug:40.6-.11:Undefined variable xchgs\n/usr/share/netcf/lenses/sysconfig.aug:42.14-.24:Undefined variable var_action\n/usr/share/netcf/lenses/sysconfig.aug:43.20-.30:Undefined variable var_action\n/usr/share/netcf/lenses/sysconfig.aug:48.6-.21:Could not load module Util for Util.del_ws_spc\n/usr/share/netcf/lenses/sysconfig.aug:48.6-.21:Undefined variable Util.del_ws_spc\n/usr/share/netcf/lenses/sysconfig.aug:51.12-.20:Undefined variable comment" ========================================== The leaked fd's are caused by improper error recovery from the parsing errors. The parsing errors are caused by the parser using a global variable to keep track of comment nesting level - if multiple threads in libvirtd are calling augeas at the same time, they can stomp on each others' use of comment nesting level, and the parser gets confused. Both of these problems (the leaked fd's when recovering from the error, and the use of the global variable that causes this particular error) have been fixed in upstream augeas: commit 39ad6cd46357961dda67a1c9eda65229a25858ae Author: David Lutterkort <lutter> Date: Tue Mar 22 17:18:44 2011 -0700 Eliminate global variable 'comment_depth' in lexer commit 9a5d5834d6a15a0d751233dc7d617b75fd4d7564 Author: David Lutterkort <lutter> Date: Wed Mar 16 14:06:50 2011 -0700 * src/lexer.l (augl_init_lexer): really close the input file on error Test runs have verified that these two fixes solve the issue.