Hide Forgot
Created attachment 515726 [details] vdsm-log Description of problem: trying to initiate connectStoragePool result with a premission error on one of the system domain metadata: Thread-28::DEBUG::2011-07-28 16:56:48,388::blockSD::151::Storage.Misc.excCmd::(readlines) FAILED: <err> = "/bin/dd: opening `/dev/9c84ff7e-7614-4425-863c-d242a 6528aae/metadata': Permission denied\n"; <rc> = 1 happened on several machines- once happened it's constant please note that the exception output isn't describing the actual error" StorageDomainDoesNotExist: Storage domain does not exist: ('9c84ff7e-7614-4425-863c-d242a6528aae',) happened on connectStorageServer to 50-SDs FC pool Version-Release number of selected component (if applicable): vdsm-4.9-86.el6.x86_64 How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
related to bug 726630?
(In reply to comment #2) > related to bug 726630? I don't think so, as here we have a real access error - bug 726630 is about a false negative. Moran, what's stat `readlink -f /dev/vg/metadata` ? And the output of `udevadm test /sys/block/dm-<whatever>/` ?
[root@rhev-a32c-02 ~]# readlink -f /dev/9c84ff7e-7614-4425-863c-d242a6528aae/metadata /dev/dm-660 [root@rhev-a32c-02 ~]# udevadm test /sys/block/dm-660/ run_command: calling: test udevadm_test: version 147 This program is for debugging only, it does not run any program, specified by a RUN key. It may show incorrect results, because some values may be different, or not available at a simulation run. parse_file: reading '/lib/udev/rules.d/10-console.rules' as rules file parse_file: reading '/lib/udev/rules.d/10-dm.rules' as rules file parse_file: reading '/lib/udev/rules.d/11-dm-lvm.rules' as rules file parse_file: reading '/etc/udev/rules.d/12-vdsm-lvm.rules' as rules file parse_file: reading '/lib/udev/rules.d/13-dm-disk.rules' as rules file parse_file: reading '/lib/udev/rules.d/40-isdn.rules' as rules file parse_file: reading '/lib/udev/rules.d/40-multipath.rules' as rules file parse_file: reading '/lib/udev/rules.d/40-redhat.rules' as rules file parse_file: reading '/lib/udev/rules.d/42-qemu-usb.rules' as rules file parse_file: reading '/lib/udev/rules.d/50-firmware.rules' as rules file parse_file: reading '/lib/udev/rules.d/50-udev-default.rules' as rules file parse_file: reading '/etc/udev/rules.d/51-dlm.rules' as rules file parse_file: reading '/lib/udev/rules.d/60-cdrom_id.rules' as rules file parse_file: reading '/etc/udev/rules.d/60-fprint-autosuspend.rules' as rules file parse_file: reading '/lib/udev/rules.d/60-net.rules' as rules file parse_file: reading '/etc/udev/rules.d/60-pcmcia.rules' as rules file parse_file: reading '/lib/udev/rules.d/60-persistent-alsa.rules' as rules file parse_file: reading '/lib/udev/rules.d/60-persistent-input.rules' as rules file parse_file: reading '/lib/udev/rules.d/60-persistent-serial.rules' as rules file parse_file: reading '/lib/udev/rules.d/60-persistent-storage-tape.rules' as rules file parse_file: reading '/lib/udev/rules.d/60-persistent-storage.rules' as rules file parse_file: reading '/lib/udev/rules.d/60-persistent-v4l.rules' as rules file parse_file: reading '/etc/udev/rules.d/60-raw.rules' as rules file parse_file: reading '/lib/udev/rules.d/61-mobile-action.rules' as rules file parse_file: reading '/lib/udev/rules.d/61-option-modem-modeswitch.rules' as rules file parse_file: reading '/lib/udev/rules.d/61-persistent-storage-edd.rules' as rules file parse_file: reading '/lib/udev/rules.d/64-device-mapper.rules' as rules file parse_file: reading '/lib/udev/rules.d/64-md-raid.rules' as rules file parse_file: reading '/lib/udev/rules.d/65-md-incremental.rules' as rules file parse_file: reading '/lib/udev/rules.d/70-acl.rules' as rules file parse_file: reading '/lib/udev/rules.d/70-cups-libusb.rules' as rules file parse_file: reading '/lib/udev/rules.d/70-hid2hci.rules' as rules file parse_file: reading '/etc/udev/rules.d/70-persistent-cd.rules' as rules file parse_file: reading '/lib/udev/rules.d/71-biosdevname.rules' as rules file parse_file: reading '/lib/udev/rules.d/75-cd-aliases-generator.rules' as rules file parse_file: reading '/lib/udev/rules.d/75-net-description.rules' as rules file parse_file: reading '/lib/udev/rules.d/75-persistent-net-generator.rules' as rules file parse_file: reading '/lib/udev/rules.d/75-tty-description.rules' as rules file parse_file: reading '/lib/udev/rules.d/78-sound-card.rules' as rules file parse_file: reading '/lib/udev/rules.d/79-fstab_import.rules' as rules file parse_file: reading '/lib/udev/rules.d/80-drivers.rules' as rules file parse_file: reading '/etc/udev/rules.d/80-kvm.rules' as rules file parse_file: reading '/lib/udev/rules.d/85-regulatory.rules' as rules file parse_file: reading '/lib/udev/rules.d/88-clock.rules' as rules file parse_file: reading '/lib/udev/rules.d/89-microcode.rules' as rules file parse_file: reading '/etc/udev/rules.d/90-alsa.rules' as rules file parse_file: reading '/etc/udev/rules.d/90-hal.rules' as rules file parse_file: reading '/etc/udev/rules.d/91-drm-modeset.rules' as rules file parse_file: reading '/lib/udev/rules.d/95-dm-notify.rules' as rules file parse_file: reading '/lib/udev/rules.d/95-keyboard-force-release.rules' as rules file parse_file: reading '/lib/udev/rules.d/95-keymap.rules' as rules file parse_file: reading '/lib/udev/rules.d/95-udev-late.rules' as rules file parse_file: reading '/etc/udev/rules.d/98-kexec.rules' as rules file parse_file: reading '/dev/.udev/rules.d/99-root.rules' as rules file udev_rules_new: rules use 30732 bytes tokens (2561 * 12 bytes), 18969 bytes buffer udev_rules_new: temporary index used 18820 bytes (941 * 20 bytes) udev_device_new_from_syspath: device 0x7fdda350b7f0 has devpath '/devices/virtual/block/dm-660' udev_device_new_from_syspath: device 0x7fdda34f9990 has devpath '/devices/virtual/block/dm-660' udev_device_read_db: device 0x7fdda34f9990 filled with db file data udev_device_read_db: device 0x7fdda34f9990 filled with db file data udev_rules_apply_to_event: LINK 'mapper/9c84ff7e--7614--4425--863c--d242a6528aae-metadata' /lib/udev/rules.d/10-dm.rules:114 udev_rules_apply_to_event: IMPORT '/sbin/dmsetup splitname --nameprefixes --noheadings --rows 9c84ff7e--7614--4425--863c--d242a6528aae-metadata' /lib/udev/rules.d/11-dm-lvm.rules:21 util_run_program: '/sbin/dmsetup splitname --nameprefixes --noheadings --rows 9c84ff7e--7614--4425--863c--d242a6528aae-metadata' started util_run_program: '/sbin/dmsetup' (stdout) 'DM_VG_NAME='9c84ff7e-7614-4425-863c-d242a6528aae'' util_run_program: '/sbin/dmsetup' (stdout) 'DM_LV_NAME='metadata'' util_run_program: '/sbin/dmsetup' (stdout) 'DM_LV_LAYER=''' util_run_program: '/sbin/dmsetup splitname --nameprefixes --noheadings --rows 9c84ff7e--7614--4425--863c--d242a6528aae-metadata' returned with exitcode 0 udev_rules_apply_to_event: LINK '9c84ff7e-7614-4425-863c-d242a6528aae/metadata' /lib/udev/rules.d/11-dm-lvm.rules:30 udev_rules_apply_to_event: OWNER 36 /etc/udev/rules.d/12-vdsm-lvm.rules:26 udev_rules_apply_to_event: GROUP 107 /etc/udev/rules.d/12-vdsm-lvm.rules:26 udev_rules_apply_to_event: MODE 0600 /etc/udev/rules.d/12-vdsm-lvm.rules:26 udev_rules_apply_to_event: LINK 'disk/by-id/dm-name-9c84ff7e--7614--4425--863c--d242a6528aae-metadata' /lib/udev/rules.d/13-dm-disk.rules:17 udev_rules_apply_to_event: LINK 'disk/by-id/dm-uuid-LVM-gYMJrLyThPJ6OvTxnXV6csBoiYKatbTbFDrLvgSD6yvD4rd9tKZrZSivNflmhsBN' /lib/udev/rules.d/13-dm-disk.rules:18 udev_rules_apply_to_event: IMPORT '/sbin/blkid -o udev -p /dev/dm-660' /lib/udev/rules.d/13-dm-disk.rules:22 util_run_program: '/sbin/blkid -o udev -p /dev/dm-660' started util_run_program: '/sbin/blkid -o udev -p /dev/dm-660' returned with exitcode 2 udev_rules_apply_to_event: PROGRAM '/sbin/multipath -c /dev/dm-660' /lib/udev/rules.d/40-multipath.rules:11 util_run_program: '/sbin/multipath -c /dev/dm-660' started util_run_program: '/sbin/multipath' (stdout) 'Jul 31 11:14:48 | the -c option requires a path to check' util_run_program: '/sbin/multipath -c /dev/dm-660' returned with exitcode 1 udev_rules_apply_to_event: RUN 'socket:/org/kernel/dm/multipath_event' /lib/udev/rules.d/40-multipath.rules:16 udev_rules_apply_to_event: LINK 'block/253:660' /lib/udev/rules.d/50-udev-default.rules:3 udev_rules_apply_to_event: RUN 'socket:@/org/freedesktop/hal/udev_event' /etc/udev/rules.d/90-hal.rules:2 udev_event_execute_rules: no node name set, will use kernel supplied name 'dm-660' udev_device_update_db: created db file for '/devices/virtual/block/dm-660' in '/dev/.udev/db/block:dm-660' udev_node_add: creating device node '/dev/dm-660', devnum=253:660, mode=0660, uid=36, gid=107 udev_node_mknod: preserve file '/dev/dm-660', because it has correct dev_t udev_node_mknod: set permissions /dev/dm-660, 060660, uid=36, gid=107 link_find_prioritized: found '/sys/devices/virtual/block/dm-660' claiming '/dev/.udev/links/mapper\x2f9c84ff7e--7614--4425--863c--d242a6528aae-metadata' link_update: creating link '/dev/mapper/9c84ff7e--7614--4425--863c--d242a6528aae-metadata' to '/dev/dm-660' node_symlink: preserve already existing symlink '/dev/mapper/9c84ff7e--7614--4425--863c--d242a6528aae-metadata' to '../dm-660' link_find_prioritized: found '/sys/devices/virtual/block/dm-660' claiming '/dev/.udev/links/9c84ff7e-7614-4425-863c-d242a6528aae\x2fmetadata' link_update: creating link '/dev/9c84ff7e-7614-4425-863c-d242a6528aae/metadata' to '/dev/dm-660' node_symlink: preserve already existing symlink '/dev/9c84ff7e-7614-4425-863c-d242a6528aae/metadata' to '../dm-660' link_find_prioritized: found '/sys/devices/virtual/block/dm-660' claiming '/dev/.udev/links/disk\x2fby-id\x2fdm-name-9c84ff7e--7614--4425--863c--d242a6528aae-metadata' link_update: creating link '/dev/disk/by-id/dm-name-9c84ff7e--7614--4425--863c--d242a6528aae-metadata' to '/dev/dm-660' node_symlink: preserve already existing symlink '/dev/disk/by-id/dm-name-9c84ff7e--7614--4425--863c--d242a6528aae-metadata' to '../../dm-660' link_find_prioritized: found '/sys/devices/virtual/block/dm-660' claiming '/dev/.udev/links/disk\x2fby-id\x2fdm-uuid-LVM-gYMJrLyThPJ6OvTxnXV6csBoiYKatbTbFDrLvgSD6yvD4rd9tKZrZSivNflmhsBN' link_update: creating link '/dev/disk/by-id/dm-uuid-LVM-gYMJrLyThPJ6OvTxnXV6csBoiYKatbTbFDrLvgSD6yvD4rd9tKZrZSivNflmhsBN' to '/dev/dm-660' node_symlink: preserve already existing symlink '/dev/disk/by-id/dm-uuid-LVM-gYMJrLyThPJ6OvTxnXV6csBoiYKatbTbFDrLvgSD6yvD4rd9tKZrZSivNflmhsBN' to '../../dm-660' node_symlink: preserve already existing symlink '/dev/block/253:660' to '../dm-660' udevadm_test: UDEV_LOG=6 udevadm_test: DEVPATH=/devices/virtual/block/dm-660 udevadm_test: MAJOR=253 udevadm_test: MINOR=660 udevadm_test: DEVNAME=/dev/dm-660 udevadm_test: DEVTYPE=disk udevadm_test: ACTION=add udevadm_test: SUBSYSTEM=block udevadm_test: DM_SBIN_PATH=/sbin udevadm_test: DM_UDEV_PRIMARY_SOURCE_FLAG=1 udevadm_test: DM_UDEV_RULES_VSN=2 udevadm_test: DM_NAME=9c84ff7e--7614--4425--863c--d242a6528aae-metadata udevadm_test: DM_UUID=LVM-gYMJrLyThPJ6OvTxnXV6csBoiYKatbTbFDrLvgSD6yvD4rd9tKZrZSivNflmhsBN udevadm_test: DM_SUSPENDED=0 udevadm_test: DEVLINKS=/dev/mapper/9c84ff7e--7614--4425--863c--d242a6528aae-metadata /dev/9c84ff7e-7614-4425-863c-d242a6528aae/metadata /dev/disk/by-id/dm-name-9c84ff7e--7614--4425--863c--d242a6528aae-metadata /dev/disk/by-id/dm-uuid-LVM-gYMJrLyThPJ6OvTxnXV6csBoiYKatbTbFDrLvgSD6yvD4rd9tKZrZSivNflmhsBN /dev/block/253:660 udevadm_test: DM_VG_NAME=9c84ff7e-7614-4425-863c-d242a6528aae udevadm_test: DM_LV_NAME=metadata udevadm_test: DM_LV_LAYER= udevadm_test: MPATH_SBIN_PATH=/sbin udevadm_test: run: 'socket:/org/kernel/dm/multipath_event' udevadm_test: run: 'socket:@/org/freedesktop/hal/udev_event'
(In reply to comment #4) > [root@rhev-a32c-02 ~]# readlink -f > /dev/9c84ff7e-7614-4425-863c-d242a6528aae/metadata > /dev/dm-660 Please add the stat or `ls -l` of this.
[root@rhev-a32c-02 ~]# ls -l `readlink -f /dev/9c84ff7e-7614-4425-863c-d242a6528aae/metadata` brw-rw----. 1 vdsm qemu 253, 660 Jul 31 11:14 /dev/dm-660 stat `readlink -f /dev/9c84ff7e-7614-4425-863c-d242a6528aae/metadata` File: `/dev/dm-660' Size: 0 Blocks: 0 IO Block: 4096 block special file Device: 5h/5d Inode: 67077 Links: 1 Device type: fd,294 Access: (0660/brw-rw----) Uid: ( 36/ vdsm) Gid: ( 107/ qemu) Access: 2011-07-31 11:14:48.380820340 +0300 Modify: 2011-07-31 11:14:48.380820340 +0300 Change: 2011-07-31 11:14:48.380820340 +0300
seems to be a udev Bug 727425