Bug 1061911

Summary: [abrt] NetworkManager-0.9.9.0-20.git20131003.fc20: _g_log_abort: Process /usr/sbin/NetworkManager was killed by signal 6 (SIGABRT)
Product: [Fedora] Fedora Reporter: Javier Orlando Ramirez Martinez <javier.ramirez>
Component: NetworkManagerAssignee: Jirka Klimes <jklimes>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 20CC: danw, dcbw, jklimes, rkhan
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
URL: https://retrace.fedoraproject.org/faf/reports/bthash/47fe6387bc1e884429b7a14a50ec4d08ac3011f5
Whiteboard: abrt_hash:ec58b7b53ebccaa391ebd1ebd4c3f9015556d5f2
Fixed In Version: NetworkManager-0.9.9.0-38.git20131003.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-04-22 03:59:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
File: backtrace
none
File: cgroup
none
File: core_backtrace
none
File: dso_list
none
File: environ
none
File: limits
none
File: maps
none
File: open_fds
none
File: proc_pid_status
none
File: var_log_messages none

Description Javier Orlando Ramirez Martinez 2014-02-05 21:09:50 UTC
Version-Release number of selected component:
NetworkManager-0.9.9.0-20.git20131003.fc20

Additional info:
reporter:       libreport-2.1.9
backtrace_rating: 4
cmdline:        /usr/sbin/NetworkManager --no-daemon
crash_function: _g_log_abort
executable:     /usr/sbin/NetworkManager
kernel:         3.11.10-301.fc20.x86_64
runlevel:       N 5
type:           CCpp
uid:            0

Truncated backtrace:
Thread no. 1 (4 frames)
 #2 _g_log_abort at gmessages.c:255
 #5 get_next_cb at settings/nm-agent-manager.c:1023
 #6 get_callback at settings/nm-secret-agent.c:253
 #7 complete_pending_call_and_unlock at dbus-connection.c:2314

Comment 1 Javier Orlando Ramirez Martinez 2014-02-05 21:09:55 UTC
Created attachment 859871 [details]
File: backtrace

Comment 2 Javier Orlando Ramirez Martinez 2014-02-05 21:09:59 UTC
Created attachment 859872 [details]
File: cgroup

Comment 3 Javier Orlando Ramirez Martinez 2014-02-05 21:10:05 UTC
Created attachment 859873 [details]
File: core_backtrace

Comment 4 Javier Orlando Ramirez Martinez 2014-02-05 21:10:23 UTC
Created attachment 859874 [details]
File: dso_list

Comment 5 Javier Orlando Ramirez Martinez 2014-02-05 21:10:24 UTC
Created attachment 859875 [details]
File: environ

Comment 6 Javier Orlando Ramirez Martinez 2014-02-05 21:10:26 UTC
Created attachment 859876 [details]
File: limits

Comment 7 Javier Orlando Ramirez Martinez 2014-02-05 21:10:28 UTC
Created attachment 859877 [details]
File: maps

Comment 8 Javier Orlando Ramirez Martinez 2014-02-05 21:10:30 UTC
Created attachment 859878 [details]
File: open_fds

Comment 9 Javier Orlando Ramirez Martinez 2014-02-05 21:10:34 UTC
Created attachment 859879 [details]
File: proc_pid_status

Comment 10 Javier Orlando Ramirez Martinez 2014-02-05 21:10:37 UTC
Created attachment 859880 [details]
File: var_log_messages

Comment 11 Jirka Klimes 2014-02-11 13:09:55 UTC
Please consider updating NetworkManager. There were some changes in the code.
Nevertheless, there is still an issue upstream. jk/secret-agent-fix branch contains a fix.

Comment 12 Dan Winship 2014-02-14 14:56:19 UTC
the change seems generally correct, but can you explain how the current code is causing problems?

Comment 13 Dan Williams 2014-02-14 22:01:21 UTC
Agreed that the change seems correct, and may actually "fix" the bug but not the actual root cause.

---

So the actual crash here appears to be due to agent_dbus_owner == NULL, and thus the creation of the auth chain fails because NM can't create an auth chain for the agent in get_next_cb().  So the question is why agent_dbus_owner is NULL...

agent_dbus_owner comes from nm_secret_agent_get_dbus_owner(), and when creating the agent, the owner must be valid.  The only place the owner is freed is nm-secret-agent.c in dispose().  So for agent_dbus_owner to be NULL, the current agent must have been already disposed.

For the current agent to already be disposed, it must have had remove_agent() called on it, since (barring refcount problems) that's the only place that agents are disposed of.  But that's really odd, because the code takes pains to ensure that when an agent is disposed, it gets removed from req->pending, and can never become the current agent.  And if it's already the current agent, then it should get dropped from the request, and the next agent asked.

----

I did see one thing though.  get_done_cb() uses:

	g_return_if_fail (call_id == parent->current_call_id);

to ensure that a response from some agent that we don't care about anymore is ignored.  Well, it turns out that call_id and parent->current_call_id are the return value from dbus_g_proxy_begin_call(), which is a counter that starts at 1 and counts up for each call to that proxy object.

But *each* agent (and therefore each proxy) will start counting at 1.  So NM sends a request to agent A, and then drops agent A, and then sends a request to agent B, and then finally agent A replies, there's a chance that NM thinks that agent A's response is actually for agent B.

I don't see how that could cause the crash above, but it's worth noting and probably worth fixing, by having NMSecretAgent pass back the pointer to 'r' (which would still be opaque) instead of r->call, to ensure uniqueness.

Comment 14 Jirka Klimes 2014-02-25 13:39:34 UTC
Pushed to upstream master:
4141e69 settings: free memory in finalize(), not in dispose() in NMSecretAgent (rh #1061911)

However, I leave the bug opened, so that it can be inspected deeper with regards to comment 13.

Comment 15 Fedora Update System 2014-04-08 20:12:25 UTC
NetworkManager-0.9.9.0-34.git20131003.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/NetworkManager-0.9.9.0-34.git20131003.fc20

Comment 16 Fedora Update System 2014-04-09 13:23:57 UTC
Package NetworkManager-0.9.9.0-34.git20131003.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing NetworkManager-0.9.9.0-34.git20131003.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-4964/NetworkManager-0.9.9.0-34.git20131003.fc20
then log in and leave karma (feedback).

Comment 17 Fedora Update System 2014-04-10 09:28:36 UTC
NetworkManager-0.9.9.0-35.git20131003.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/NetworkManager-0.9.9.0-35.git20131003.fc20

Comment 18 Fedora Update System 2014-04-14 14:51:12 UTC
NetworkManager-0.9.9.0-36.git20131003.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/NetworkManager-0.9.9.0-36.git20131003.fc20

Comment 19 Fedora Update System 2014-04-15 11:57:16 UTC
NetworkManager-0.9.9.0-37.git20131003.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/NetworkManager-0.9.9.0-37.git20131003.fc20

Comment 20 Fedora Update System 2014-04-17 16:41:44 UTC
NetworkManager-0.9.9.0-38.git20131003.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/NetworkManager-0.9.9.0-38.git20131003.fc20

Comment 21 Fedora Update System 2014-04-22 03:59:34 UTC
NetworkManager-0.9.9.0-38.git20131003.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.