Created attachment 137624 [details]
syslog with BUG output 33,772 bytes

Also happens with fc6-test3-dvd-i386 DVD.

Perhaps Eric has some clues (and maybe he needs a break from ext3 bugs :)

Looks like lockdep problems, I'll look into it.

xfs looks completely DOA w/ lockdep enabled at this point (and probably also w/
slab debugging) - a simple mount/unmount will oops.  I'll look into it more
tomorrow, probably something straightforward.

We are blowing up in mark_lock because when it does:

        if (likely(this->class->usage_mask & new_mask))
                return 1;

this->class is 0x6b6b6b6b, meaning freed memory.

This got assigned previously in __lock_acquire,

        if (!subclass)
                class = lock->class_cache;

but class_cache is freed (0x6b6b6b6b) at this point.  I've not been able to
figure out where/why this happened yet...

Doing a little debugging in lock_release_non_nested:

lockdepth_depth 4 held_locks dde9cff4

first loop:

testing i==3, hlock dde9d06c, hlock->instance d1400ea4
testing i==2, hlock dde9d044, hlock->instance d1400edc
testing i==1, hlock dde9d01c, hlock->instance d1691110
Matched lock at d1691110 (i==1)

curr lockdep_depth now 1

i==2, hlock dde9d044, hlock->instance d1400edc, our lock d1691110
lock map at d1400edc lock->class_cache 6b6b6b6b lock->key 6b6b6b6b lock->name
6b6b6b6b?

somehow locks 2 and 3 above are freed, and -that- is why we are oopsing, not due
to the mutex_unlock(&sb->s_lock);.  Perhaps this means xfs has freed some locks
w/o unlocking them first....

Found out over the weekend that this is due to this code in xfs_ireclaim:

xfs_ireclaim(xfs_inode_t *ip)
{
   ...
        /*
         * Here we do a spurious inode lock in order to coordinate with
         * xfs_sync().  This is because xfs_sync() references the inodes
         * in the mount list without taking references on the corresponding
         * vnodes.  We make that OK here by ensuring that we wait until
         * the inode is unlocked in xfs_sync() before we go ahead and
         * free it.  We get both the regular lock and the io lock because
         * the xfs_sync() code may need to drop the regular one but will
         * still hold the io lock.
         */
        xfs_ilock(ip, XFS_ILOCK_EXCL | XFS_IOLOCK_EXCL)
...
        /*
         * Free all memory associated with the inode.
         */
        xfs_idestroy(ip);
}

So, lock & free.  This frees memory that lockdep is still pointing to.

Will huddle w/ the xfs guys when I get a chance to make sure unlocking the inode
just before freeing is acceptable.  (it does seem to fix this problem at least).

Created attachment 138154 [details]
unlock xfs inode locks before freeing the inode

this fixes the problem and has been ACKed by the sgi xfs folks.

attached patch is in rawhide now.