Bug 503169
| Summary: | libguestfs /dev is too sparse for kernel installation/upgrade | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Community] Virtualization Tools | Reporter: | Charles Duffy <charles_duffy> | ||||||||
| Component: | libguestfs | Assignee: | Richard W.M. Jones <rjones> | ||||||||
| Status: | CLOSED UPSTREAM | QA Contact: | |||||||||
| Severity: | medium | Docs Contact: | |||||||||
| Priority: | low | ||||||||||
| Version: | unspecified | CC: | virt-maint | ||||||||
| Target Milestone: | --- | ||||||||||
| Target Release: | --- | ||||||||||
| Hardware: | All | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2010-03-27 10:15:17 UTC | Type: | --- | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
|
Description
Charles Duffy
2009-05-29 11:28:26 UTC
(In reply to comment #0) > The following commands, run in the order given, appear to create all entries in > /dev necessary for make-initramfs and grubby to work on RHEL5: > > MAKEDEV -x /dev/mapper /dev/null $(cat /proc/partitions | gawk '{print "/dev/" > $4}' | egrep -v '/dev/(dm-|name|$)') > lvm pvscan > lvm vgscan > lvm vgchange -a y These commands are run after the /init script, using guestfs_command or similar? What actual device nodes get added by the above? eg. try 'll /../dev' before and after and look at the diff between the two. AFAICT /dev/null should exist - not much would work without it. /dev/mapper exists. ><fs> ll /../dev/mapper total 0 drwxrwxr-x 2 root root 0 May 29 07:03 . drwxr-xr-x 5 root root 0 May 29 07:57 .. ><fs> ll /../dev/null crw-rw-rw- 1 root root 1, 3 May 29 07:08 /sysroot/../dev/null I need to know what needs to be run in the actual /init script to fix things. The issue is not with /dev, but with /sysroot/dev. By default, /sysroot/dev has only the LVM-created devices, mapper, and null: VolGroup00 mapper null After running the previously-given commands via a script invoked from guestfs_command: VolGroup00 mapper null hda hda1 hda2 hdb I've tried "mount-options bind /dev /dev" to pull in the much more thoroughly populated contents of /../dev to /sysroot/dev, but the command fails as given. I think you need to run this with LIBGUESTFS_DEBUG=1 to see what's really happening. When we run commands in the guest, we first bind-mount /dev into /sysroot/dev, /proc into /sysroot/proc and /sys into /sysroot/sys. See: http://git.et.redhat.com/?p=libguestfs.git;a=blob;f=daemon/command.c;h=1a50264b3c4c100b6d0c6f467ccc3e06a4c5faa1;hb=HEAD#l48 Now it may be for some reason this isn't happening or is not doing the right thing. The only way to find out for sure is to look for messages printed out by the guest. Created attachment 346117 [details]
LIBGUESTFS_DEBUG log demonstrating sparse /dev
In the given log (with LIBGUESTFS_DEBUG=1), we successfully mount /dev/hda1, but /dev contains no such device.
Thanks for doing that. The output is seriously odd: VolGroup00 mapper null I actually see almost the same thing. For example with a Fedora 10 guest: ><fs> command "ls -l /dev" total 8 drwxr-xr-x 2 root root 4096 May 10 21:36 VolGroup00 drwxr-xr-x 2 root root 4096 May 10 21:36 mapper This is a very odd one because it looks as if the bind-mount is somehow only copying directories, not device nodes. It might be something to do with the permissions on /dev, or something about bind-mounts in the kernel, or something else. -- By the way, in the test you try to run the command "ls /../dev", but because the command runs inside the chroot, that's exactly the same as running "ls /dev". The same goes for the direct ls /../dev command which already runs in a chroot[1]. The only command which doesn't run in a chroot is ll[2]. It's very helpful to look at the source to see the precise semantics. [1] http://git.et.redhat.com/?p=libguestfs.git;a=blob;f=daemon/ls.c;h=261bc96481f2e97b7ec93a31ac44590af3f12746;hb=HEAD#l32 [2] http://git.et.redhat.com/?p=libguestfs.git;a=blob;f=daemon/ls.c;h=261bc96481f2e97b7ec93a31ac44590af3f12746;hb=HEAD#l87 I should have recognized this one, having run into it in the past: initramfs can't be bind mounted. If the sysroot is modified to put /dev on a different filesystem (such as tmpfs), that should resolve the issue. Created attachment 346475 [details]
mount /dev via tmpfs before MAKEDEV invocation
Created attachment 346476 [details]
Use --rbind rather than --bind for bind mounting /dev (to get /dev/pts)
Thanks to Charles Duffy for the two patches: http://git.et.redhat.com/?p=libguestfs.git;a=commitdiff;h=9d6de016ccd932c18001c08c371bf83c6dbff8f0 http://git.et.redhat.com/?p=libguestfs.git;a=commitdiff;h=80e15757879c4cf21e0511c62e57c19efb4aab63 This causes a regression. There are existing /dev nodes in the initramfs which are hidden by the tmpfs. The existing nodes are: $ ls -l initramfs/dev/ total 12 -rw------- 1 rjones rjones 0 2009-06-04 09:10 console -rw-rw-rw- 1 rjones rjones 0 2009-06-04 09:10 full drwxrwxr-x 2 rjones rjones 4096 2009-06-04 09:10 mapper -rw-rw-rw- 1 rjones rjones 0 2009-06-04 09:11 null -rw-rw-rw- 1 rjones rjones 0 2009-06-04 09:10 ptmx drwxrwxr-x 2 rjones rjones 4096 2009-06-04 09:10 pts -rw-rw-rw- 1 rjones rjones 0 2009-06-04 09:10 random drwxrwxr-x 2 rjones rjones 4096 2009-06-04 09:10 shm lrwxrwxrwx 1 rjones rjones 15 2009-06-04 09:10 stderr -> /proc/self/fd/2 lrwxrwxrwx 1 rjones rjones 15 2009-06-04 09:10 stdin -> /proc/self/fd/0 lrwxrwxrwx 1 rjones rjones 15 2009-06-04 09:10 stdout -> /proc/self/fd/1 -rw-rw-rw- 1 rjones rjones 0 2009-06-04 09:10 tty -rw-r--r-- 1 rjones rjones 0 2009-06-04 09:10 urandom -rw-rw-rw- 1 rjones rjones 0 2009-06-04 09:10 zero Moving back to ASSIGNED. I think this should fix the regression: http://git.et.redhat.com/?p=libguestfs.git;a=commitdiff;h=a903368c6db67dc599e5cbefe9e41611fba81faf libguestfs-1.0.40-1.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libguestfs-1.0.40-1.fc11 This doesn't fully fix the regression. /sysroot/dev is not unmounted after the command has run, consequently we get: umount_all: umount: /sysroot/dev: umount: /sysroot/dev: device is busy. Possibly --rbind cannot be reversed by a simple unmount but you have to unmount all the subdevices too. Setting bug back to ASSIGNED. ><fs> mounts /dev/sda2 /dev/sda1 ><fs> command "ls -l /" total 73 drwxr-xr-x 2 root root 4096 Feb 12 13:39 bin drwxr-xr-x 4 root root 1024 Feb 12 13:55 boot drwxrwxrwt 5 root root 173480 Jun 4 13:56 dev drwxr-xr-x 64 root root 4096 Feb 27 13:41 etc drwxr-xr-x 3 root root 4096 Feb 12 13:58 home drwxr-xr-x 2 root root 4096 Jan 24 2003 initrd drwxr-xr-x 11 root root 4096 Feb 12 13:49 lib drwx------ 2 root root 16384 Feb 12 13:35 lost+found drwxr-xr-x 2 root root 4096 Apr 11 2007 misc drwxr-xr-x 4 root root 4096 Feb 12 17:04 mnt drwxr-xr-x 2 root root 4096 Jan 24 2003 opt dr-xr-xr-x 43 root root 0 Jun 4 13:56 proc drwxr-x--- 6 root root 4096 Feb 12 17:29 root drwxr-xr-x 2 root root 8192 Feb 12 13:41 sbin drwxrwxrwt 11 root root 4096 Feb 27 13:41 tmp drwxr-xr-x 15 root root 4096 Feb 12 13:36 usr drwxr-xr-x 22 root root 4096 Feb 12 13:49 var ><fs> mounts /dev/sda2 /dev/sda1 /dev Notice the additional bogus "/dev" mount afterwards. (In reply to comment #10) > This causes a regression. There are existing /dev > nodes in the initramfs which are hidden by the tmpfs. Added a regression test to catch this if it reoccurs in future: http://git.et.redhat.com/?p=libguestfs.git;a=commitdiff;h=96b9d82e2f4f24cdd41040f54a7597202c04cf7e (In reply to comment #13) > This doesn't fully fix the regression. > > /sysroot/dev is not unmounted after the command has run, > consequently we get: > > umount_all: umount: /sysroot/dev: umount: /sysroot/dev: device is busy. Fixed this and added a regression test for it: http://git.et.redhat.com/?p=libguestfs.git;a=commitdiff;h=ab81d09a710242ab61001ef5e8ffd321725b47a7 libguestfs-1.0.41-1.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libguestfs-1.0.41-1.fc11 libguestfs-1.0.51-1.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libguestfs-1.0.51-1.fc11 libguestfs-1.0.54-2.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libguestfs-1.0.54-2.fc11 libguestfs-1.0.55-1.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libguestfs-1.0.55-1.fc11 libguestfs-1.0.58-2.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/libguestfs-1.0.58-2.fc11 libguestfs-1.0.54-2.fc11 has been pushed to the Fedora 11 stable repository. If problems still persist, please make note of it in this bug report. libguestfs-1.0.58-2.fc11 has been pushed to the Fedora 11 stable repository. If problems still persist, please make note of it in this bug report. Long fixed upstream. |