Bug 1269403 - ceph-deploy error during OSD prepare while installing ceph 1.3.1 on RHEL 7.1
ceph-deploy error during OSD prepare while installing ceph 1.3.1 on RHEL 7.1
Status: CLOSED NOTABUG
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Ceph-Installer (Show other bugs)
1.3.1
Unspecified Linux
unspecified Severity unspecified
: rc
: 1.3.2
Assigned To: Travis Rhoden
ceph-qe-bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-07 04:41 EDT by shilpa
Modified: 2017-12-12 19:23 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-10-12 05:58:17 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description shilpa 2015-10-07 04:41:06 EDT
Description of problem:


ceph-deploy prepare command has an error: "partx: /dev/sdb: error adding partitions 1-2" which is flagged as a WARNING. 

See ceph health warnings. The OSD's are up though. 

#ceph health
HEALTH_WARN 64 pgs degraded; 64 pgs stuck degraded; 64 pgs stuck unclean; 64 pgs stuck undersized; 64 pgs undersized

Error snippet:

[magna075][WARNIN] INFO:ceph-disk:calling partx on prepared device /dev/sdb
[magna075][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/sbin/partx -a /dev/sdb
[magna075][WARNIN] partx: /dev/sdb: error adding partitions 1-2



# ceph-deploy osd prepare magna075:/dev/sdb


[magna075][INFO  ] Running command: ceph-disk -v prepare --fs-type xfs --cluster ceph -- /dev/sdb
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mkfs_options_xfs
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mkfs_options_xfs
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=osd_journal_size
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_cryptsetup_parameters
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_key_size
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_dmcrypt_type
[magna075][WARNIN] INFO:ceph-disk:Will colocate journal with data on /dev/sdb
[magna075][WARNIN] DEBUG:ceph-disk:Creating journal partition num 2 size 5120 on /dev/sdb
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/sbin/sgdisk --new=2:0:5120M --change-name=2:ceph journal --partition-guid=2:d1a8fd80-ddb4-4673-b5dd-7d3763a44994 --typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106 --mbrtogpt -- /dev/sdb
[magna075][DEBUG ] The operation has completed successfully.
[magna075][WARNIN] INFO:ceph-disk:calling partx on prepared device /dev/sdb
[magna075][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/sbin/partx -a /dev/sdb
[magna075][WARNIN] partx: /dev/sdb: error adding partition 2
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/udevadm settle
[magna075][WARNIN] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/d1a8fd80-ddb4-4673-b5dd-7d3763a44994
[magna075][WARNIN] DEBUG:ceph-disk:Journal is GPT partition /dev/disk/by-partuuid/d1a8fd80-ddb4-4673-b5dd-7d3763a44994
[magna075][WARNIN] DEBUG:ceph-disk:Creating osd partition on /dev/sdb
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/sbin/sgdisk --largest-new=1 --change-name=1:ceph data --partition-guid=1:0669b7f9-a33e-4689-928b-33d51c09f5c6 --typecode=1:89c57f98-2fe5-4dc0-89c1-f3ad0ceff2be -- /dev/sdb
[magna075][DEBUG ] The operation has completed successfully.
[magna075][WARNIN] INFO:ceph-disk:calling partx on created device /dev/sdb
[magna075][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/sbin/partx -a /dev/sdb
[magna075][WARNIN] partx: /dev/sdb: error adding partitions 1-2
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/udevadm settle
[magna075][WARNIN] DEBUG:ceph-disk:Creating xfs fs on /dev/sdb1
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/sbin/mkfs -t xfs -f -i size=2048 -- /dev/sdb1
[magna075][DEBUG ] meta-data=/dev/sdb1              isize=2048   agcount=4, agsize=60719917 blks
[magna075][DEBUG ]          =                       sectsz=512   attr=2, projid32bit=1
[magna075][DEBUG ]          =                       crc=0        finobt=0
[magna075][DEBUG ] data     =                       bsize=4096   blocks=242879665, imaxpct=25
[magna075][DEBUG ]          =                       sunit=0      swidth=0 blks
[magna075][DEBUG ] naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
[magna075][DEBUG ] log      =internal log           bsize=4096   blocks=118593, version=2
[magna075][DEBUG ]          =                       sectsz=512   sunit=0 blks, lazy-count=1
[magna075][DEBUG ] realtime =none                   extsz=4096   blocks=0, rtextents=0
[magna075][WARNIN] DEBUG:ceph-disk:Mounting /dev/sdb1 on /var/lib/ceph/tmp/mnt.nbe7yk with options noatime,inode64
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/bin/mount -t xfs -o noatime,inode64 -- /dev/sdb1 /var/lib/ceph/tmp/mnt.nbe7yk
[magna075][WARNIN] DEBUG:ceph-disk:Preparing osd data dir /var/lib/ceph/tmp/mnt.nbe7yk
[magna075][WARNIN] DEBUG:ceph-disk:Creating symlink /var/lib/ceph/tmp/mnt.nbe7yk/journal -> /dev/disk/by-partuuid/d1a8fd80-ddb4-4673-b5dd-7d3763a44994
[magna075][WARNIN] DEBUG:ceph-disk:Unmounting /var/lib/ceph/tmp/mnt.nbe7yk
[magna075][WARNIN] INFO:ceph-disk:Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.nbe7yk
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/sbin/sgdisk --typecode=1:4fbd7e29-9d25-41b8-afd0-062c0ceff05d -- /dev/sdb
[magna075][DEBUG ] The operation has completed successfully.
[magna075][WARNIN] INFO:ceph-disk:calling partx on prepared device /dev/sdb
[magna075][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors
[magna075][WARNIN] INFO:ceph-disk:Running command: /usr/sbin/partx -a /dev/sdb
[magna075][WARNIN] partx: /dev/sdb: error adding partitions 1-2
[magna075][INFO  ] checking OSD status...
[magna075][INFO  ] Running command: ceph --cluster=ceph osd stat --format=json


#ceph osd tree

# ceph osd tree
ID WEIGHT  TYPE NAME         UP/DOWN REWEIGHT PRIMARY-AFFINITY 
-1 1.79999 root default                                        
-2 0.89999     host magna062                                   
 0 0.89999         osd.0          up  1.00000          1.00000 
-3 0.89999     host magna075                                   
 1 0.89999         osd.1          up  1.00000          1.00000
Comment 2 Alfredo Deza 2015-10-07 07:34:47 EDT
This is known behavior from ceph-deploy and the output is already saying that partx will complain (form your output):

[magna075][WARNIN] INFO:ceph-disk:re-reading known partitions will display errors

In this case /dev/sdb2 is the one that is already known and indeed, it displays an error:

[magna075][WARNIN] partx: /dev/sdb: error adding partition 2

The origin of this behavior in RHEL is described in http://tracker.ceph.com/issues/7334

The process of deploying OSDs checks for their status and making sure they are all "up", "in", and not full or near full which the output seems to say is not the case.

I am not sure why the health warning but I am inclined to say this is not because there was a failure in deploying the OSD.
Comment 3 shilpa 2015-10-12 05:58:17 EDT
(In reply to Alfredo Deza from comment #2)
> This is known behavior from ceph-deploy and the output is already saying
> that partx will complain (form your output):
> 
> [magna075][WARNIN] INFO:ceph-disk:re-reading known partitions will display
> errors
> 
> In this case /dev/sdb2 is the one that is already known and indeed, it
> displays an error:
> 
> [magna075][WARNIN] partx: /dev/sdb: error adding partition 2
> 
> The origin of this behavior in RHEL is described in
> http://tracker.ceph.com/issues/7334
> 
> The process of deploying OSDs checks for their status and making sure they
> are all "up", "in", and not full or near full which the output seems to say
> is not the case.
> 
> I am not sure why the health warning but I am inclined to say this is not
> because there was a failure in deploying the OSD.

Understood. Thanks Alfredo. Just wanted to make sure health warning has nothing to do with this. Closing Not a bug.

Note You need to log in before you can comment on or make changes to this bug.