Description of problem: On 7.1 "osd activate" is failing due to above subject issue, also partx threw quite a warning messages but it looks ok(check fdisk part info below) Version-Release number of selected component (if applicable): 1.5.25/1.3.0 How reproducible: always Steps to Reproduce: [ubuntu@magna087 cd]$ sudo ceph-deploy disk zap magna087:sdb sdc sdd [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /bin/ceph-deploy disk zap magna087:sdb sdc sdd [ceph_deploy.osd][DEBUG ] zapping /dev/sdb on magna087 [magna087][DEBUG ] connected to host: magna087 [magna087][DEBUG ] detect platform information from remote host [magna087][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Red Hat Enterprise Linux Server 7.1 Maipo [magna087][DEBUG ] zeroing last few blocks of device [magna087][DEBUG ] find the location of an executable [magna087][INFO ] Running command: /usr/sbin/ceph-disk zap /dev/sdb [magna087][WARNIN] Caution: invalid backup GPT header, but valid main header; regenerating [magna087][WARNIN] backup header from main header. [magna087][WARNIN] [magna087][DEBUG ] **************************************************************************** [magna087][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk [magna087][DEBUG ] verification and recovery are STRONGLY recommended. [magna087][DEBUG ] **************************************************************************** [magna087][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or [magna087][DEBUG ] other utilities. [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] partx: specified range <1:0> does not make sense [ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdb [ceph_deploy.osd][INFO ] re-reading known partitions will display errors [magna087][INFO ] Running command: partx -a /dev/sdb [ceph_deploy][ERROR ] RuntimeError: zap command needs both HOSTNAME and DISK but got "sdc None" [ubuntu@magna087 cd]$ sudo ceph-deploy disk zap magna087:sdb magna087:sdc magna087:sdd [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /bin/ceph-deploy disk zap magna087:sdb magna087:sdc magna087:sdd [ceph_deploy.osd][DEBUG ] zapping /dev/sdb on magna087 [magna087][DEBUG ] connected to host: magna087 [magna087][DEBUG ] detect platform information from remote host [magna087][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Red Hat Enterprise Linux Server 7.1 Maipo [magna087][DEBUG ] zeroing last few blocks of device [magna087][DEBUG ] find the location of an executable [magna087][INFO ] Running command: /usr/sbin/ceph-disk zap /dev/sdb [magna087][WARNIN] Caution: invalid backup GPT header, but valid main header; regenerating [magna087][WARNIN] backup header from main header. [magna087][WARNIN] [magna087][DEBUG ] **************************************************************************** [magna087][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk [magna087][DEBUG ] verification and recovery are STRONGLY recommended. [magna087][DEBUG ] **************************************************************************** [magna087][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or [magna087][DEBUG ] other utilities. [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] partx: specified range <1:0> does not make sense [ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdb [ceph_deploy.osd][INFO ] re-reading known partitions will display errors [magna087][INFO ] Running command: partx -a /dev/sdb [ceph_deploy.osd][DEBUG ] zapping /dev/sdc on magna087 [magna087][DEBUG ] connected to host: magna087 [magna087][DEBUG ] detect platform information from remote host [magna087][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Red Hat Enterprise Linux Server 7.1 Maipo [magna087][DEBUG ] zeroing last few blocks of device [magna087][DEBUG ] find the location of an executable [magna087][INFO ] Running command: /usr/sbin/ceph-disk zap /dev/sdc [magna087][WARNIN] Caution: invalid backup GPT header, but valid main header; regenerating [magna087][WARNIN] backup header from main header. [magna087][WARNIN] [magna087][DEBUG ] **************************************************************************** [magna087][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk [magna087][DEBUG ] verification and recovery are STRONGLY recommended. [magna087][DEBUG ] **************************************************************************** [magna087][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or [magna087][DEBUG ] other utilities. [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] partx: specified range <1:0> does not make sense [ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdc [ceph_deploy.osd][INFO ] re-reading known partitions will display errors [magna087][INFO ] Running command: partx -a /dev/sdc [ceph_deploy.osd][DEBUG ] zapping /dev/sdd on magna087 [magna087][DEBUG ] connected to host: magna087 [magna087][DEBUG ] detect platform information from remote host [magna087][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Red Hat Enterprise Linux Server 7.1 Maipo [magna087][DEBUG ] zeroing last few blocks of device [magna087][DEBUG ] find the location of an executable [magna087][INFO ] Running command: /usr/sbin/ceph-disk zap /dev/sdd [magna087][WARNIN] Caution: invalid backup GPT header, but valid main header; regenerating [magna087][WARNIN] backup header from main header. [magna087][WARNIN] [magna087][DEBUG ] **************************************************************************** [magna087][DEBUG ] Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk [magna087][DEBUG ] verification and recovery are STRONGLY recommended. [magna087][DEBUG ] **************************************************************************** [magna087][DEBUG ] GPT data structures destroyed! You may now partition the disk using fdisk or [magna087][DEBUG ] other utilities. [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] partx: specified range <1:0> does not make sense [ceph_deploy.osd][INFO ] calling partx on zapped device /dev/sdd [ceph_deploy.osd][INFO ] re-reading known partitions will display errors [magna087][INFO ] Running command: partx -a /dev/sdd [ubuntu@magna087 cd]$ sudo ceph-deploy osd prepare magna087:sdb magna087:sdc magna087:sdd [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /bin/ceph-deploy osd prepare magna087:sdb magna087:sdc magna087:sdd [ceph_deploy.osd][DEBUG ] Preparing cluster ceph disks magna087:/dev/sdb: magna087:/dev/sdc: magna087:/dev/sdd: [magna087][DEBUG ] connected to host: magna087 [magna087][DEBUG ] detect platform information from remote host [magna087][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Red Hat Enterprise Linux Server 7.1 Maipo [ceph_deploy.osd][DEBUG ] Deploying osd to magna087 [magna087][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [magna087][INFO ] Running command: udevadm trigger --subsystem-match=block --action=add [ceph_deploy.osd][DEBUG ] Preparing host magna087 disk /dev/sdb journal None activate False [magna087][INFO ] Running command: ceph-disk -v prepare --fs-type xfs --cluster ceph -- /dev/sdb [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type [magna087][WARNIN] INFO:ceph-disk:Will colocate journal with data on /dev/sdb [magna087][WARNIN] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdb [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:c478b86f-b05d-4f21-8d4e-c74beb1c122a --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdb [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] INFO:ceph-disk:calling partx on prepared device /dev/sdb [magna087][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/partx -a /dev/sdb [magna087][WARNIN] partx: /dev/sdb: error adding partition 2 [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/udevadm settle [magna087][WARNIN] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/c478b86f-b05d-4f21-8d4e-c74beb1c122a [magna087][WARNIN] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/c478b86f-b05d-4f21-8d4e-c74beb1c122a [magna087][WARNIN] DEBUG:ceph-disk:Creating osd partition on /dev/sdb [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/sgdisk --largest-new=1 --change-name=1:ceph data --partition-guid=1:67111e4b-5861-406c-8831-fc65e3d45ce4 --typecode=1:89c57f98-2fe5-4dc0-89c1-f3ad0ceff2be -- /dev/sdb [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] INFO:ceph-disk:calling partx on created device /dev/sdb [magna087][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/partx -a /dev/sdb [magna087][WARNIN] partx: /dev/sdb: error adding partitions 1-2 [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/udevadm settle [magna087][WARNIN] DEBUG:ceph-disk:Creating xfs fs on /dev/sdb1 [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/mkfs -t xfs -f -i size=2048 -- /dev/sdb1 [magna087][DEBUG ] meta-data=/dev/sdb1 isize=2048 agcount=4, agsize=60719917 blks [magna087][DEBUG ] = sectsz=512 attr=2, projid32bit=1 [magna087][DEBUG ] = crc=0 finobt=0 [magna087][DEBUG ] data = bsize=4096 blocks=242879665, imaxpct=25 [magna087][DEBUG ] = sunit=0 swidth=0 blks [magna087][DEBUG ] naming =version 2 bsize=4096 ascii-ci=0 ftype=0 [magna087][DEBUG ] log =internal log bsize=4096 blocks=118593, version=2 [magna087][DEBUG ] = sectsz=512 sunit=0 blks, lazy-count=1 [magna087][DEBUG ] realtime =none extsz=4096 blocks=0, rtextents=0 [magna087][WARNIN] DEBUG:ceph-disk:Mounting /dev/sdb1 on /var/lib/ceph/tmp/mnt.Y9uepV with options noatime,inode64 [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/mount -t xfs -o noatime,inode64 -- /dev/sdb1 /var/lib/ceph/tmp/mnt.Y9uepV [magna087][WARNIN] DEBUG:ceph-disk:Preparing osd data dir /var/lib/ceph/tmp/mnt.Y9uepV [magna087][WARNIN] DEBUG:ceph-disk:Creating symlink /var/lib/ceph/tmp/mnt.Y9uepV/journal -> /dev/disk/by-partuuid/c478b86f-b05d-4f21-8d4e-c74beb1c122a [magna087][WARNIN] DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.Y9uepV [magna087][WARNIN] INFO:ceph-disk:Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.Y9uepV [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/sgdisk --typecode=1:4fbd7e29-9d25-41b8-afd0-062c0ceff05d -- /dev/sdb [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] INFO:ceph-disk:calling partx on prepared device /dev/sdb [magna087][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/partx -a /dev/sdb [magna087][WARNIN] partx: /dev/sdb: error adding partitions 1-2 [magna087][INFO ] checking OSD status... [magna087][INFO ] Running command: ceph --cluster=ceph osd stat --format=json [ceph_deploy.osd][DEBUG ] Host magna087 is now ready for osd use. [magna087][DEBUG ] connected to host: magna087 [magna087][DEBUG ] detect platform information from remote host [magna087][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Red Hat Enterprise Linux Server 7.1 Maipo [ceph_deploy.osd][DEBUG ] Preparing host magna087 disk /dev/sdc journal None activate False [magna087][INFO ] Running command: ceph-disk -v prepare --fs-type xfs --cluster ceph -- /dev/sdc [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type [magna087][WARNIN] INFO:ceph-disk:Will colocate journal with data on /dev/sdc [magna087][WARNIN] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdc [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:5914bd55-e288-47f3-b326-64c7ca981525 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdc [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] INFO:ceph-disk:calling partx on prepared device /dev/sdc [magna087][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/partx -a /dev/sdc [magna087][WARNIN] partx: /dev/sdc: error adding partition 2 [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/udevadm settle [magna087][WARNIN] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/5914bd55-e288-47f3-b326-64c7ca981525 [magna087][WARNIN] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/5914bd55-e288-47f3-b326-64c7ca981525 [magna087][WARNIN] DEBUG:ceph-disk:Creating osd partition on /dev/sdc [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/sgdisk --largest-new=1 --change-name=1:ceph data --partition-guid=1:ea1793ca-9f74-4a89-a3df-8d9bb34dd79a --typecode=1:89c57f98-2fe5-4dc0-89c1-f3ad0ceff2be -- /dev/sdc [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] INFO:ceph-disk:calling partx on created device /dev/sdc [magna087][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/partx -a /dev/sdc [magna087][WARNIN] partx: /dev/sdc: error adding partitions 1-2 [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/udevadm settle [magna087][WARNIN] DEBUG:ceph-disk:Creating xfs fs on /dev/sdc1 [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/mkfs -t xfs -f -i size=2048 -- /dev/sdc1 [magna087][DEBUG ] meta-data=/dev/sdc1 isize=2048 agcount=4, agsize=60719917 blks [magna087][DEBUG ] = sectsz=512 attr=2, projid32bit=1 [magna087][DEBUG ] = crc=0 finobt=0 [magna087][DEBUG ] data = bsize=4096 blocks=242879665, imaxpct=25 [magna087][DEBUG ] = sunit=0 swidth=0 blks [magna087][DEBUG ] naming =version 2 bsize=4096 ascii-ci=0 ftype=0 [magna087][DEBUG ] log =internal log bsize=4096 blocks=118593, version=2 [magna087][WARNIN] DEBUG:ceph-disk:Mounting /dev/sdc1 on /var/lib/ceph/tmp/mnt.W0OMoY with options noatime,inode64 [magna087][DEBUG ] = sectsz=512 sunit=0 blks, lazy-count=1 [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/mount -t xfs -o noatime,inode64 -- /dev/sdc1 /var/lib/ceph/tmp/mnt.W0OMoY [magna087][DEBUG ] realtime =none extsz=4096 blocks=0, rtextents=0 [magna087][WARNIN] DEBUG:ceph-disk:Preparing osd data dir /var/lib/ceph/tmp/mnt.W0OMoY [magna087][WARNIN] DEBUG:ceph-disk:Creating symlink /var/lib/ceph/tmp/mnt.W0OMoY/journal -> /dev/disk/by-partuuid/5914bd55-e288-47f3-b326-64c7ca981525 [magna087][WARNIN] DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.W0OMoY [magna087][WARNIN] INFO:ceph-disk:Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.W0OMoY [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/sgdisk --typecode=1:4fbd7e29-9d25-41b8-afd0-062c0ceff05d -- /dev/sdc [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] INFO:ceph-disk:calling partx on prepared device /dev/sdc [magna087][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/partx -a /dev/sdc [magna087][WARNIN] partx: /dev/sdc: error adding partitions 1-2 [magna087][INFO ] checking OSD status... [magna087][INFO ] Running command: ceph --cluster=ceph osd stat --format=json [ceph_deploy.osd][DEBUG ] Host magna087 is now ready for osd use. [magna087][DEBUG ] connected to host: magna087 [magna087][DEBUG ] detect platform information from remote host [magna087][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Red Hat Enterprise Linux Server 7.1 Maipo [ceph_deploy.osd][DEBUG ] Preparing host magna087 disk /dev/sdd journal None activate False [magna087][INFO ] Running command: ceph-disk -v prepare --fs-type xfs --cluster ceph -- /dev/sdd [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type [magna087][WARNIN] INFO:ceph-disk:Will colocate journal with data on /dev/sdd [magna087][WARNIN] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdd [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:46b75355-bf45-451f-9a43-0fbb79ff0985 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdd [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] INFO:ceph-disk:calling partx on prepared device /dev/sdd [magna087][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/partx -a /dev/sdd [magna087][WARNIN] partx: /dev/sdd: error adding partition 2 [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/udevadm settle [magna087][WARNIN] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/46b75355-bf45-451f-9a43-0fbb79ff0985 [magna087][WARNIN] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/46b75355-bf45-451f-9a43-0fbb79ff0985 [magna087][WARNIN] DEBUG:ceph-disk:Creating osd partition on /dev/sdd [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/sgdisk --largest-new=1 --change-name=1:ceph data --partition-guid=1:f9f858ea-d757-4312-95ca-77bccc7b4d91 --typecode=1:89c57f98-2fe5-4dc0-89c1-f3ad0ceff2be -- /dev/sdd [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] INFO:ceph-disk:calling partx on created device /dev/sdd [magna087][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/partx -a /dev/sdd [magna087][WARNIN] partx: /dev/sdd: error adding partitions 1-2 [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/udevadm settle [magna087][WARNIN] DEBUG:ceph-disk:Creating xfs fs on /dev/sdd1 [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/mkfs -t xfs -f -i size=2048 -- /dev/sdd1 [magna087][DEBUG ] meta-data=/dev/sdd1 isize=2048 agcount=4, agsize=60719917 blks [magna087][DEBUG ] = sectsz=512 attr=2, projid32bit=1 [magna087][DEBUG ] = crc=0 finobt=0 [magna087][DEBUG ] data = bsize=4096 blocks=242879665, imaxpct=25 [magna087][DEBUG ] = sunit=0 swidth=0 blks [magna087][DEBUG ] naming =version 2 bsize=4096 ascii-ci=0 ftype=0 [magna087][DEBUG ] log =internal log bsize=4096 blocks=118593, version=2 [magna087][DEBUG ] = sectsz=512 sunit=0 blks, lazy-count=1 [magna087][DEBUG ] realtime =none extsz=4096 blocks=0, rtextents=0 [magna087][WARNIN] DEBUG:ceph-disk:Mounting /dev/sdd1 on /var/lib/ceph/tmp/mnt.Ikp8GK with options noatime,inode64 [magna087][WARNIN] INFO:ceph-disk:Running command: /usr/bin/mount -t xfs -o noatime,inode64 -- /dev/sdd1 /var/lib/ceph/tmp/mnt.Ikp8GK [magna087][WARNIN] DEBUG:ceph-disk:Preparing osd data dir /var/lib/ceph/tmp/mnt.Ikp8GK [magna087][WARNIN] DEBUG:ceph-disk:Creating symlink /var/lib/ceph/tmp/mnt.Ikp8GK/journal -> /dev/disk/by-partuuid/46b75355-bf45-451f-9a43-0fbb79ff0985 [magna087][WARNIN] DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.Ikp8GK [magna087][WARNIN] INFO:ceph-disk:Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.Ikp8GK [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/sgdisk --typecode=1:4fbd7e29-9d25-41b8-afd0-062c0ceff05d -- /dev/sdd [magna087][DEBUG ] The operation has completed successfully. [magna087][WARNIN] INFO:ceph-disk:calling partx on prepared device /dev/sdd [magna087][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/partx -a /dev/sdd [magna087][WARNIN] partx: /dev/sdd: error adding partitions 1-2 [magna087][INFO ] checking OSD status... [magna087][INFO ] Running command: ceph --cluster=ceph osd stat --format=json [ceph_deploy.osd][DEBUG ] Host magna087 is now ready for osd use. [ubuntu@magna087 cd]$ echo $? 0 [ubuntu@magna087 cd]$ sudo ceph-deploy osd activate magna087:sdb magna087:sdc magna087:sdd [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.25): /bin/ceph-deploy osd activate magna087:sdb magna087:sdc magna087:sdd [ceph_deploy.osd][DEBUG ] Activating cluster ceph disks magna087:/dev/sdb: magna087:/dev/sdc: magna087:/dev/sdd: [magna087][DEBUG ] connected to host: magna087 [magna087][DEBUG ] detect platform information from remote host [magna087][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Red Hat Enterprise Linux Server 7.1 Maipo [ceph_deploy.osd][DEBUG ] activating host magna087 disk /dev/sdb [ceph_deploy.osd][DEBUG ] will use init type: sysvinit [magna087][INFO ] Running command: ceph-disk -v activate --mark-init sysvinit --mount /dev/sdb [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/blkid -p -s TYPE -ovalue -- /dev/sdb [magna087][WARNIN] ceph-disk: Cannot discover filesystem type: device /dev/sdb: Line is truncated: [magna087][ERROR ] RuntimeError: command returned non-zero exit status: 1 [ceph_deploy][ERROR ] RuntimeError: Failed to execute command: ceph-disk -v activate --mark-init sysvinit --mount /dev/sdb # [ubuntu@magna087 cd]$ sudo fdisk -l WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion. Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes, 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: gpt # Start End Size Type Name 1 10487808 1953525134 926.5G unknown ceph data 2 2048 10485760 5G unknown ceph journal Disk /dev/sda: 1000.2 GB, 1000204886016 bytes, 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x0007c287 Device Boot Start End Blocks Id System /dev/sda1 * 2048 40962047 20480000 83 Linux WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion. Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes, 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: gpt # Start End Size Type Name 1 10487808 1953525134 926.5G unknown ceph data 2 2048 10485760 5G unknown ceph journal WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion. Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes, 1953525168 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: gpt # Start End Size Type Name 1 10487808 1953525134 926.5G unknown ceph data 2 2048 10485760 5G unknown ceph journal #mount output /dev/sdb1 on /var/lib/ceph/osd/ceph-0 type xfs (rw,noatime,seclabel,attr2,inode64,noquota) /dev/sdc1 on /var/lib/ceph/osd/ceph-1 type xfs (rw,noatime,seclabel,attr2,inode64,noquota) /dev/sdd1 on /var/lib/ceph/osd/ceph-2 type xfs (rw,noatime,seclabel,attr2,inode64,noquota) Actual results: [magna087][INFO ] Running command: ceph-disk -v activate --mark-init sysvinit --mount /dev/sdb [magna087][WARNIN] INFO:ceph-disk:Running command: /sbin/blkid -p -s TYPE -ovalue -- /dev/sdb [magna087][WARNIN] ceph-disk: Cannot discover filesystem type: device /dev/sdb: Line is truncated: [magna087][ERROR ] RuntimeError: command returned non-zero exit status: 1 [ceph_deploy][ERROR ] RuntimeError: Failed to execute command: ceph-disk -v activate --mark-init sysvinit --mount /dev/sdb Expected results: osd activate should work Additional info:
The problem is that you need to specify the data and journal partitions to "osd activate". Instead of specifying the raw disks: sudo ceph-deploy osd activate magna087:sdb magna087:sdc magna087:sdd Specify the data and journal partitions: sudo ceph-deploy osd activate magna087:sdb1:sdb2 magna087:sdc1:sdc2 magna087:sdd1:sdd2 (Travis, please correct me if I'm wrong here :)
Travis, do you mind confirming my Comment 2 above? /sbin/blkid is going to fail on /dev/sdb because it doesn't have a filesystem to identify.
(In reply to Ken Dreyer (Red Hat) from comment #3) > Travis, do you mind confirming my Comment 2 above? > > /sbin/blkid is going to fail on /dev/sdb because it doesn't have a > filesystem to identify. Yes, comment #2 is correct. When calling the "activate" command, you must give partitions, not raw block devices. As a side node, I still don't understand why we encourage the use of "prepare" then "activate" at all. Those steps, while they work, are really only needed in testing environments where you might be using a folder as an OSD. When using block devices, doing the single command "ceph-deploy osd create" is much easier.
We should correct both 1.2.3 and 1.3.0 document to remove the activate step when its not required when only prepare will do.
Assigning this to John wilkins so that he can remove the 'activate' command and only include 'prepare' for both 1.2.3 and 1.3.0. As per Travis only prepare is required and not 'activate'
Let's not get ahead of ourselves there, as there is some history to this. See: https://bugzilla.redhat.com/show_bug.cgi?id=1201772 We specifically reverted to the two step method, and I think took it as an action to see why "create" wasn't sufficient. Not sure if we ever did that, but I don't think we should ask John to change anything just yet, as we ask him to do it as it is on purpose. :) Just took me a minute to dig up the old bug.
And finally, as for this bug, I don't think this is a bug. I think it was just invalid input to the ceph-deploy command.
(In reply to Travis Rhoden from comment #8) > And finally, as for this bug, I don't think this is a bug. I think it was > just invalid input to the ceph-deploy command. Agreed. If anything, we might be able to update the docs to clarify that "osd prepare" will split a raw block device into two partitions (one for the journal, one for the data) and that those two partitions should then be passed to "osd activate". What do you think Travis?
I will close this as not a bug (invalid input for ceph-deploy ? )