Bug 1444909

Summary: puppet-ceph isn't handling dev by id so ceph-disk parepare runs multiple times and fails
Product: Red Hat OpenStack Reporter: John Fulton <johfulto>
Component: puppet-cephAssignee: Giulio Fidente <gfidente>
Status: CLOSED ERRATA QA Contact: Yogev Rabl <yrabl>
Severity: unspecified Docs Contact: Don Domingo <ddomingo>
Priority: unspecified    
Version: 11.0 (Ocata)CC: bengland, ddomingo, gfidente, jjoyce, johfulto, jomurphy, jschluet, jtaleric, mburns, sclewis, slinaber, tvignaud, twilkins
Target Milestone: rc   
Target Release: 11.0 (Ocata)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: puppet-ceph-2.3.0-5.el7ost.src.rpm Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1444967 (view as bug list) Environment:
Last Closed: 2017-05-17 20:23:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1444967    

Description John Fulton 2017-04-24 14:21:50 UTC
puppet-ceph isn't handling dev by id so ceph-disk parepare runs multiple times and fails.

puppet-ceph works fine if you pass it paths to block devices like /dev/sdb. It _should_ also handle paths like /dev/disk/by-path/pci-0000:03:00.0-scsi-0:2:16:0. However when dev by id is passed it attempts to try to ceph-disk prepare multiple times because it cannot tell that the disk is already prepared because its clause to determine this fails.

It is necessary to pass dev by id sometimes if a system doesn't have consitent assignment of block devices between boots. So this is a scenario puppet-ceph should support.

Comment 1 Red Hat Bugzilla Rules Engine 2017-04-24 14:28:49 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 8 Yogev Rabl 2017-05-03 19:51:45 UTC
The deployment of the overcloud was successful, the Ceph OSDs has been set properly. When checking the mount the result is: 
/dev/vda1 on /var/lib/ceph/osd/ceph-1 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)
/dev/vdb1 on /var/lib/ceph/osd/ceph-4 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)
/dev/vdc1 on /var/lib/ceph/osd/ceph-6 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)
/dev/vdd1 on /var/lib/ceph/osd/ceph-9 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)
/dev/vde1 on /var/lib/ceph/osd/ceph-12 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)
/dev/vdf1 on /var/lib/ceph/osd/ceph-17 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)
/dev/vdh1 on /var/lib/ceph/osd/ceph-20 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)
/dev/vdg1 on /var/lib/ceph/osd/ceph-21 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)
/dev/vdi1 on /var/lib/ceph/osd/ceph-24 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)
/dev/vdj1 on /var/lib/ceph/osd/ceph-27 type xfs (rw,noatime,seclabel,attr2,inode64,noquota)

Which is mounting by its path to block device and not by path it was set in the environment file: 
    ceph::profile::params::osds:
     '/dev/disk/by-path/virtio-pci-0000:00:05.0':
       journal:
     '/dev/disk/by-path/virtio-pci-0000:00:06.0':
       journal:
     '/dev/disk/by-path/virtio-pci-0000:00:07.0':
       journal:
     '/dev/disk/by-path/virtio-pci-0000:00:08.0':
       journal:
     '/dev/disk/by-path/virtio-pci-0000:00:0b.0':
       journal:
     '/dev/disk/by-path/virtio-pci-0000:00:0d.0':
       journal:
     '/dev/disk/by-path/virtio-pci-0000:00:0e.0':
       journal:
     '/dev/disk/by-path/virtio-pci-0000:00:0f.0':
       journal:
     '/dev/disk/by-path/virtio-pci-0000:00:10.0':
       journal:
     '/dev/disk/by-path/virtio-pci-0000:00:11.0':
       journal:

is that right?

Comment 9 Giulio Fidente 2017-05-03 20:07:22 UTC
(In reply to Yogev Rabl from comment #8)
> The deployment of the overcloud was successful, the Ceph OSDs has been set
> properly. When checking the mount the result is: 
> /dev/vda1 on /var/lib/ceph/osd/ceph-1 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> /dev/vdb1 on /var/lib/ceph/osd/ceph-4 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> /dev/vdc1 on /var/lib/ceph/osd/ceph-6 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> /dev/vdd1 on /var/lib/ceph/osd/ceph-9 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> /dev/vde1 on /var/lib/ceph/osd/ceph-12 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> /dev/vdf1 on /var/lib/ceph/osd/ceph-17 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> /dev/vdh1 on /var/lib/ceph/osd/ceph-20 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> /dev/vdg1 on /var/lib/ceph/osd/ceph-21 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> /dev/vdi1 on /var/lib/ceph/osd/ceph-24 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> /dev/vdj1 on /var/lib/ceph/osd/ceph-27 type xfs
> (rw,noatime,seclabel,attr2,inode64,noquota)
> 
> Which is mounting by its path to block device and not by path it was set in
> the environment file: 
>     ceph::profile::params::osds:
>      '/dev/disk/by-path/virtio-pci-0000:00:05.0':
>        journal:
>      '/dev/disk/by-path/virtio-pci-0000:00:06.0':
>        journal:
>      '/dev/disk/by-path/virtio-pci-0000:00:07.0':
>        journal:
>      '/dev/disk/by-path/virtio-pci-0000:00:08.0':
>        journal:
>      '/dev/disk/by-path/virtio-pci-0000:00:0b.0':
>        journal:
>      '/dev/disk/by-path/virtio-pci-0000:00:0d.0':
>        journal:
>      '/dev/disk/by-path/virtio-pci-0000:00:0e.0':
>        journal:
>      '/dev/disk/by-path/virtio-pci-0000:00:0f.0':
>        journal:
>      '/dev/disk/by-path/virtio-pci-0000:00:10.0':
>        journal:
>      '/dev/disk/by-path/virtio-pci-0000:00:11.0':
>        journal:
> 
> is that right?

hi Yogev,

this is the expected behavior; the aliases are resolved to the real device path during the deployment and this is what Ceph and ceph-disk will refer to in the various reporting commands.

Comment 10 Yogev Rabl 2017-05-04 12:41:48 UTC
Wonderful, in this case, this bug has been verified with puppet-ceph-2.3.0-5.el7ost.src.rpm

Comment 12 errata-xmlrpc 2017-05-17 20:23:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245