Bug 673544 - [abrt] udisks-1.0.2-2.fc14: write_interface: Process /usr/libexec/udisks-daemon was killed by signal 11 (SIGSEGV)
[abrt] udisks-1.0.2-2.fc14: write_interface: Process /usr/libexec/udisks-daem...
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: udisks (Show other bugs)
14
x86_64 Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: David Zeuthen
Fedora Extras Quality Assurance
abrt_hash:3344a4f2bec250207b1b663fba2...
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2011-01-28 11:40 EST by Tomáš Trnka
Modified: 2013-03-05 23:06 EST (History)
6 users (show)

See Also:
Fixed In Version: udisks-1.0.2-4.fc14
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-02-15 16:23:59 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
File: backtrace (22.96 KB, text/plain)
2011-01-28 11:40 EST, Tomáš Trnka
no flags Details
fixed backtrace (22.98 KB, text/plain)
2011-01-29 07:14 EST, Tomáš Trnka
no flags Details
Diff between generated code (26.65 KB, text/plain)
2011-02-01 09:42 EST, David Zeuthen
no flags Details

  None (edit)
Description Tomáš Trnka 2011-01-28 11:40:43 EST
abrt version: 1.1.14
architecture: x86_64
Attached file: backtrace
cmdline: /usr/libexec/udisks-daemon
component: udisks
crash_function: write_interface
executable: /usr/libexec/udisks-daemon
kernel: 2.6.35.10-74.fc14.x86_64
package: udisks-1.0.2-2.fc14
rating: 4
reason: Process /usr/libexec/udisks-daemon was killed by signal 11 (SIGSEGV)
release: Fedora release 14 (Laughlin)
time: 1296232489
uid: 0

How to reproduce
-----
1. update to udisks-1.0.2-2.fc14.x86_64 (updates-testing)
2. do anything that touches udisks (e.g. logging in to KDE or opening a File->Open dialog in any KDE app)
3. watch a storm of crashes as dbus keeps respawning udisks-daemon over and over again
Comment 1 Tomáš Trnka 2011-01-28 11:40:49 EST
Created attachment 475828 [details]
File: backtrace
Comment 2 David Zeuthen 2011-01-28 12:22:17 EST
Please install all the required debuginfo packages.
Comment 3 Tomáš Trnka 2011-01-29 07:14:04 EST
Created attachment 475944 [details]
fixed backtrace

The only thing that was missing in the previous one is the crashing source line:
597	      dbus_type = _dbus_gtype_to_signature (G_PARAM_SPEC_VALUE_TYPE (spec));

But how come ABRT didn't report this properly, even though it stated that all necessary debuginfos are present and the backtrace is otherwise complete with locals and stuff? An ABRT or GDB bug?
Comment 4 monts 2011-01-30 09:44:49 EST
Package: udisks-1.0.2-2.fc14
Architecture: x86_64
OS Release: Fedora release 14 (Laughlin)


How to reproduce
-----
1.since updated to KDE 4.6 
2. on every reboot to init 5
3. doing a killall udisks-daemon after every boot to init 5 as a stop gap :( 



Comment
-----
the system was reinstalled still happennig ... did a fresh reinstall without any DE  to init 3 and yummed up to KDE 4.6 .... udisk-daemon keeps crashing ... making the system crawl :(
Comment 5 monts 2011-01-30 09:56:19 EST
Package: udisks-1.0.2-2.fc14
Architecture: x86_64
OS Release: Fedora release 14 (Laughlin)


How to reproduce
-----
1.since updated to KDE 4.6 
2. on every reboot to init 5
3. doing a killall udisks-daemon after every boot to init 5 as a stop gap :( 



Comment
-----
the system was reinstalled still happennig ... did a fresh reinstall without any DE  to init 3 and yummed up to KDE 4.6 .... udisk-daemon keeps crashing ... making the system crawl :(

[root@localhost ~]# dmesg | tail 
[  618.388541] udisks-daemon[8863]: segfault at 18 ip 000000300280aba8 sp 00007fff2afc9100 error 4 in libdbus-glib-1.so.2.1.0[3002800000+21000]
[  618.445530] udisks-daemon[8873]: segfault at 18 ip 000000300280aba8 sp 00007fff5e234db0 error 4 in libdbus-glib-1.so.2.1.0[3002800000+21000]
[  618.507748] udisks-daemon[8882]: segfault at 18 ip 000000300280aba8 sp 00007fff8313fd90 error 4 in libdbus-glib-1.so.2.1.0[3002800000+21000]
[  618.554417] udisks-daemon[8890]: segfault at 18 ip 000000300280aba8 sp 00007fff34cc2930 error 4 in libdbus-glib-1.so.2.1.0[3002800000+21000]
[  618.600083] udisks-daemon[8899]: segfault at 18 ip 000000300280aba8 sp 00007fffb76936a0 error 4 in libdbus-glib-1.so.2.1.0[3002800000+21000]
[  618.653611] udisks-daemon[8907]: segfault at 18 ip 000000300280aba8 sp 00007fffe6275c50 error 4 in libdbus-glib-1.so.2.1.0[3002800000+21000]
[  618.702917] udisks-daemon[8915]: segfault at 18 ip 000000300280aba8 sp 00007fff6abbbbe0 error 4 in libdbus-glib-1.so.2.1.0[3002800000+21000]
[  618.752287] udisks-daemon[8923]: segfault at 18 ip 000000300280aba8 sp 00007fffabeb7c30 error 4 in libdbus-glib-1.so.2.1.0[3002800000+21000]
[ 1031.960736] lo: Disabled Privacy Extensions
[ 1049.592435] lo: Disabled Privacy Extensions
Comment 6 David Zeuthen 2011-01-31 14:09:42 EST
What version of dbus-glib are you using?
Comment 7 David Zeuthen 2011-01-31 14:17:42 EST
Also, please try downgrading _only_ udisks to udisks-1.0.1-4.fc14.x86_64 and see if the bug is still there.
Comment 8 Tomáš Trnka 2011-01-31 14:30:44 EST
I'm using stock F14 dbus-glib-0.86-4.fc14.x86_64.
Downgrading udisks makes the bug go away (actually, that's what I had to do to get a working system). 

yum update udisks (to 1.0.2-2)
killall udisks-daemon
=> repeating crashes
yum downgrade udisks (to 1.0.1-4)
killall udisks-daemon
=> no crashes

Tried this routine several times around, still the same behavior (1.0.2 - crashing, 1.0.1 - working). Backtrace suggests this is LUKS related (I have a LUKS-encrypted home).

If you can't reproduce, I'll try to come up with a patch myself in a few days...
Comment 9 monts 2011-01-31 15:19:08 EST
@Tomáš Trnka Tx a lot.
I have downgraded udisks (to 1.0.1-4) and things are 
back to normal no more crashes so far :)

dmesg | tail does not show any crashes with udisks 

dbus-glib x86_64 ver 0.86 4.fc14

btw this is my disk map if it helps non of the disks are encrypted,
 all seems to mount well and works..

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63      614462      307200   83  Linux
/dev/sda2          614463   410214462   204800000   83  Linux
/dev/sda3       410216448   693493759   141638656   83  Linux
/dev/sda4       693493920   976768064   141637072+   5  Extended
/dev/sda5       693493983   976768064   141637041   83  Linux

Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048      206847      102400    7  HPFS/NTFS
/dev/sdb2          206848   209922047   104857600    7  HPFS/NTFS
/dev/sdb3       209922048  1953519615   871798784    7  HPFS/NTFS


I did a complete reinstall before as I was using a lot of custom build stuff kernel 2.6.37, dbus-1.4.0,coreutils-8.9,binutils-2.21 and few more...

Right now on stock FC14 and KDE 4.6, after the downgrade udisks (to 1.0.1-4)
all seems good. All standard KDE apps like digikam, amarok working fine no crashing of the udisks-daemon. Would update if any other issues are noted.
Comment 10 David Zeuthen 2011-01-31 15:21:30 EST
(In reply to comment #8)
> I'm using stock F14 dbus-glib-0.86-4.fc14.x86_64.
> Downgrading udisks makes the bug go away (actually, that's what I had to do to
> get a working system). 
> 
> yum update udisks (to 1.0.2-2)
> killall udisks-daemon
> => repeating crashes
> yum downgrade udisks (to 1.0.1-4)
> killall udisks-daemon
> => no crashes

Thanks for trying this.

> Tried this routine several times around, still the same behavior (1.0.2 -
> crashing, 1.0.1 - working). Backtrace suggests this is LUKS related (I have a
> LUKS-encrypted home).

I have a LUKS-encrypted home too. Can't reproduce.

Please try running udisks under valgrind like this

 # valgrind /usr/libexec/udisks-daemon --replace

and see if anything comes up (for me: nothing unusual happens). Thanks!

I'll try to go through the code-changes from 1.0.1-4 to 1.0.2-2 with a fine-toothed comb.
Comment 11 Tomáš Trnka 2011-01-31 16:55:45 EST
(In reply to comment #10)
> Please try running udisks under valgrind like this
> 
>  # valgrind /usr/libexec/udisks-daemon --replace
> 
> and see if anything comes up (for me: nothing unusual happens). Thanks!

As expected, got to the same point as gdb:

==6559== Invalid read of size 8
==6559==    at 0x391940ABA8: write_interface (dbus-gobject.c:597)
==6559==    by 0x390D031F82: g_hash_table_foreach (in /lib64/libglib-2.0.so.0.2600.
0)
==6559==    by 0x391940BAC9: object_registration_message (dbus-gobject.c:716)
==6559==    by 0x391281DE10: ??? (in /lib64/libdbus-1.so.3.5.2)
==6559==    by 0x391280FBE1: dbus_connection_dispatch (in /lib64/libdbus-1.so.3.5.2
)
==6559==    by 0x39194099D4: message_queue_dispatch (dbus-gmain.c:101)
==6559==    by 0x390D041E32: g_main_context_dispatch (in /lib64/libglib-2.0.so.0.2600.0)
==6559==    by 0x390D04260F: ??? (in /lib64/libglib-2.0.so.0.2600.0)
==6559==    by 0x390D042C81: g_main_loop_run (in /lib64/libglib-2.0.so.0.2600.0)
==6559==    by 0x428C54: main (main.c:227)
==6559==  Address 0x18 is not stack'd, malloc'd or (recently) free'd

Thanks for pointing --replace out, debugging gets so much easier now...Using GDB I have confirmed that the problem is really spec == NULL being returned from g_object_class_find_property(), which is caused by g_param_spec_pool_lookup not finding anything in the pool.

#0  g_param_spec_pool_lookup (pool=0x649cd0, param_name=0x673fe0 "read", 
    owner_type=6725760, walk_ancestors=1) at gparam.c:1040
#1  0x000000390f010dec in g_object_class_find_property (
    class=<value optimized out>, property_name=0x673fe0 "read") at gobject.c:629
#2  0x000000391940ab90 in write_interface (key=0x432bad, val=0x674140, 
    user_data=<value optimized out>) at dbus-gobject.c:593

Perhaps this new udisks-daemon is just exposing a bug in dbus-glib...
Comment 12 Tomáš Trnka 2011-01-31 17:13:00 EST
Oh, something is terribly rotten here...I've rebuilt udisks-1.0.2-2.fc14.src.rpm
locally and installed that, and...no crashes!
Comment 13 David Zeuthen 2011-01-31 17:46:52 EST
(In reply to comment #12)
> Oh, something is terribly rotten here...I've rebuilt
> udisks-1.0.2-2.fc14.src.rpm
> locally and installed that, and...no crashes!

That's... weird... I checked the build + root logs

 http://kojipkgs.fedoraproject.org/packages/udisks/1.0.2/2.fc14/data/logs/x86_64/

and they both seem fine. Maybe the build root just had some broken toolchain packages at build time? I don't know. Anyway, I've tried rebuilding the package - any chance you can try and see if the work? If they do, I'll push another F14 update. The build is here

 http://koji.fedoraproject.org/koji/taskinfo?taskID=2753830

Thanks!
Comment 14 Tomáš Trnka 2011-01-31 18:39:18 EST
Sorry, forget that last comment of mine, something was probably a bit confused here. After reboot it started crashing again, even with the local build...

Let's hope it will now keep crashing properly, I'm going to try a git build & bisect
Comment 15 Tomáš Trnka 2011-02-01 08:04:35 EST
Found it (well, probably)!

I wasn't able to reproduce using 1.0.2 from git, even with the same CFLAGS&stuff...Diffing the source trees between 1.0.2-2 RPM and git revealed that the *-glue.h files shipped within the 1.0.2 tarball/srpm are significantly different from those generated by dbus-binding-tool 0.86 during the git build here. Using a git tree + those shipped glue files in fact produces a crashing udisks-daemon. (tried to switch between shipped/generated several times to be really sure. Generated glues are okay, the tarball ones aren't.)

Adding a simple
rm -f src/*-glue.h
into %prep produces a working (not a single crash) RPM. If anyone experiencing crashes with 1.0.2-2 wants to give it a try, the RPM is here:
https://ksicht.natur.cuni.cz/~tomasus/udisks/udisks-1.0.2-3.fc14.x86_64.rpm
(the SRPM is there, too, should anyone want to build in on i686)
Comment 16 David Zeuthen 2011-02-01 09:26:42 EST
(In reply to comment #15)
> Found it (well, probably)!
> 
> I wasn't able to reproduce using 1.0.2 from git, even with the same
> CFLAGS&stuff...Diffing the source trees between 1.0.2-2 RPM and git revealed
> that the *-glue.h files shipped within the 1.0.2 tarball/srpm are significantly
> different from those generated by dbus-binding-tool 0.86 during the git build
> here. Using a git tree + those shipped glue files in fact produces a crashing
> udisks-daemon. (tried to switch between shipped/generated several times to be
> really sure. Generated glues are okay, the tarball ones aren't.)
> 
> Adding a simple
> rm -f src/*-glue.h
> into %prep produces a working (not a single crash) RPM. If anyone experiencing
> crashes with 1.0.2-2 wants to give it a try, the RPM is here:
> https://ksicht.natur.cuni.cz/~tomasus/udisks/udisks-1.0.2-3.fc14.x86_64.rpm
> (the SRPM is there, too, should anyone want to build in on i686)

Very nice detective work. It sounds plausible this is why but adding Colin as the Cc to get his confirmation. Colin?

I'll do an udisks build with the hack for nuking src/*-glue.[ch] in %prep. Then I'll file it upstream once we hear back from Colin.
Comment 17 David Zeuthen 2011-02-01 09:38:39 EST
Here's the build

 http://koji.fedoraproject.org/koji/taskinfo?taskID=2754608
Comment 18 David Zeuthen 2011-02-01 09:42:30 EST
Created attachment 476408 [details]
Diff between generated code

This diff is generated this way

$ rm -f diff; for i in src/adapter-glue.h src/daemon-glue.h src/device-glue.h src/expander-glue.h src/port-glue.h tools/udisks-daemon-glue.h tools/udisks-device-glue.h ; do diff -u udisks-1.0.2/$i udisks-1.0.2-regen/$i >> diff ; done
Comment 19 David Zeuthen 2011-02-01 09:44:40 EST
 - udisks-1.0.2 is the shipped code (I'm asking Martin Pitt about this. He
   did the 1.0.2 release. Would be nice if dbus-glib included the version...)

 - udisks-1.0.2-regen is the regenerated code (dbus-glib-0.86-4.fc14.x86_64)
Comment 20 David Zeuthen 2011-02-01 09:46:00 EST
Got word back from Martin. The shipped code is generated using dbus-glib 0.88.

  - udisks-1.0.2 is the shipped code (dbus-glib 0.88 on Ubuntu 10.10)
 
  - udisks-1.0.2-regen is the regenerated code (dbus-glib-0.86-4.fc14.x86_64)
Comment 21 Fedora Update System 2011-02-01 10:01:01 EST
udisks-1.0.2-4.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/udisks-1.0.2-4.fc14
Comment 22 Fedora Update System 2011-02-01 15:57:22 EST
udisks-1.0.2-4.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update udisks'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/udisks-1.0.2-4.fc14
Comment 23 monts 2011-02-02 02:28:21 EST
updated to udisks-1.0.2-4.fc14 ... no more crashes :) thanks for the good work
Comment 24 Colin Walters 2011-02-03 09:21:23 EST
To be clear this is *new* metadata against an old library; there's no sane way for me to make that work.  Don't ship the generated metadata in tarballs.
Comment 25 Fedora Update System 2011-02-15 16:23:52 EST
udisks-1.0.2-4.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.