1323841 – [RH Ceph 2.0 - 10.1.0] LibRadosTwoPoolsPP.Whiteout and LibRadosTwoPoolsECPP.Whiteout Failed

Bug 1323841 - [RH Ceph 2.0 - 10.1.0] LibRadosTwoPoolsPP.Whiteout and LibRadosTwoPoolsECPP.Whiteout Failed

Summary: [RH Ceph 2.0 - 10.1.0] LibRadosTwoPoolsPP.Whiteout and LibRadosTwoPoolsECPP....

Keywords:
Status:	CLOSED CURRENTRELEASE
Alias:	None
Product:	Red Hat Ceph Storage
Classification:	Red Hat
Component:	RADOS
Sub Component:
Version:	2.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	medium
Target Milestone:	rc
Target Release:	2.0
Assignee:	Samuel Just
QA Contact:	ceph-qe-bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2016-04-04 22:33 UTC by Vasu Kulkarni
Modified:	2022-02-21 18:06 UTC (History)
CC List:	4 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2016-10-05 15:50:07 UTC
Target Upstream Version:
Dependent Products:

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Ceph Project Bug Tracker	15177	0	None	None	None	2016-04-05 14:31:45 UTC

Description Vasu Kulkarni 2016-04-04 22:33:02 UTC

Description of problem:

During Rados smoke test following tests failed

2016-04-04T16:43:07.811 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [  PASSED  ] 44 tests.
2016-04-04T16:43:07.812 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [  FAILED  ] 2 tests, listed below:
2016-04-04T16:43:07.814 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [  FAILED  ] LibRadosTwoPoolsPP.Whiteout
2016-04-04T16:43:07.815 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [  FAILED  ] LibRadosTwoPoolsECPP.Whiteout

Version-Release number of selected component (if applicable):
ceph version 10.1.0-1.el7cp (61adb020219fbad4508050b5f0a792246ba74dae)

How reproducible:
10.1.0

Steps to Reproduce:
Run Rados smoke test


Actual results:

2016-04-04T16:41:56.853 INFO:teuthology.orchestra.run.clara009:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2016-04-04T16:41:56.860 INFO:teuthology.orchestra.run.clara010:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2016-04-04T16:42:14.252 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.ReturnValuePP (7034 ms)
2016-04-04T16:42:14.254 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.Flush
2016-04-04T16:42:14.255 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.Flush (7028 ms)
2016-04-04T16:42:14.256 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.FlushPP
2016-04-04T16:42:14.257 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.FlushPP (7119 ms)
2016-04-04T16:42:14.258 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.FlushAsync
2016-04-04T16:42:14.259 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.FlushAsync (8548 ms)
2016-04-04T16:42:14.259 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.FlushAsyncPP
2016-04-04T16:42:14.260 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.FlushAsyncPP (7043 ms)
2016-04-04T16:42:14.261 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.RoundTripWriteFull
2016-04-04T16:42:14.262 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.RoundTripWriteFull (7032 ms)
2016-04-04T16:42:14.263 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.RoundTripWriteFullPP
2016-04-04T16:42:14.264 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.RoundTripWriteFullPP (7044 ms)
2016-04-04T16:42:14.265 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.RoundTripWriteFullPP2
2016-04-04T16:42:14.265 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3034 ms)
2016-04-04T16:42:14.266 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.SimpleStat
2016-04-04T16:42:14.267 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.SimpleStat (7178 ms)
2016-04-04T16:42:14.268 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.SimpleStatPP
2016-04-04T16:42:14.269 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.SimpleStatPP (7851 ms)
2016-04-04T16:42:14.270 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.SimpleStatNS
2016-04-04T16:42:14.270 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.SimpleStatNS (35233 ms)
2016-04-04T16:42:14.271 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.SimpleStatPPNS
2016-04-04T16:42:14.272 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.SimpleStatPPNS (23161 ms)
2016-04-04T16:42:14.273 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.StatRemove
2016-04-04T16:42:14.274 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.StatRemove (7294 ms)
2016-04-04T16:42:14.274 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.StatRemovePP
2016-04-04T16:42:14.275 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.StatRemovePP (7257 ms)
2016-04-04T16:42:14.276 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.OmapPP
2016-04-04T16:42:14.277 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.OmapPP (5035 ms)
2016-04-04T16:42:14.278 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.MultiWrite
2016-04-04T16:42:14.279 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.MultiWrite (7048 ms)
2016-04-04T16:42:14.279 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [ RUN      ] LibRadosAioEC.MultiWritePP
2016-04-04T16:42:14.280 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [       OK ] LibRadosAioEC.MultiWritePP (7155 ms)
2016-04-04T16:42:14.281 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [----------] 34 tests from LibRadosAioEC (304738 ms total)
2016-04-04T16:42:14.282 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio:
2016-04-04T16:42:14.283 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [----------] Global test environment tear-down
2016-04-04T16:42:14.284 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [==========] 71 tests from 2 test cases ran. (598830 ms total)
2016-04-04T16:42:14.285 INFO:tasks.workunit.client.0.clara010.stdout:                  api_aio: [  PASSED  ] 71 tests.
2016-04-04T16:42:14.285 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:42:14.286 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23142
2016-04-04T16:42:14.287 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:42:14.288 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23144
2016-04-04T16:42:14.289 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:42:14.290 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23155
2016-04-04T16:42:14.291 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:42:14.292 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23160
2016-04-04T16:42:14.292 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:42:14.293 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23165
2016-04-04T16:42:19.322 INFO:tasks.ceph.osd.1.clara002.stderr:2016-04-04 20:42:19.320330 7fe360bcd700 -1 osd.1 484 promote_throttle_recalibrate  po 948757463 pb 3125 avg_size 8
2016-04-04T16:42:22.284 INFO:tasks.workunit.client.0.clara010.stdout:            api_tier: [ RUN      ] LibRadosTwoPoolsECPP.Promote
2016-04-04T16:42:22.285 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.Promote (22141 ms)
2016-04-04T16:42:22.286 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.PromoteSnap
2016-04-04T16:42:22.287 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: waiting for scrub...
2016-04-04T16:42:22.288 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: done waiting
2016-04-04T16:42:22.289 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.PromoteSnap (24136 ms)
2016-04-04T16:42:22.290 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace
2016-04-04T16:42:22.291 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (8997 ms)
2016-04-04T16:42:22.292 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.Whiteout
2016-04-04T16:42:22.293 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: test/librados/tier.cc:3192: Failure
2016-04-04T16:42:22.294 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: Value of: it != cache_ioctx.nobjects_end()
2016-04-04T16:42:22.295 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:   Actual: false
2016-04-04T16:42:22.296 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: Expected: true
2016-04-04T16:42:22.298 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [  FAILED  ] LibRadosTwoPoolsECPP.Whiteout (7024 ms)
2016-04-04T16:42:22.299 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.Evict
2016-04-04T16:42:22.300 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.Evict (7117 ms)
2016-04-04T16:42:22.301 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.EvictSnap
2016-04-04T16:42:22.302 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.EvictSnap (9495 ms)
2016-04-04T16:42:22.303 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.TryFlush
2016-04-04T16:42:22.304 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.TryFlush (7106 ms)
2016-04-04T16:42:22.305 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.FailedFlush
2016-04-04T16:42:22.306 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.FailedFlush (12060 ms)
2016-04-04T16:42:22.307 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.Flush
2016-04-04T16:42:22.308 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.Flush (7065 ms)
2016-04-04T16:42:22.309 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.FlushSnap
2016-04-04T16:42:22.310 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.FlushSnap (13018 ms)
2016-04-04T16:42:22.311 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.FlushTryFlushRaces
2016-04-04T16:42:22.312 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (32204 ms)
2016-04-04T16:42:22.313 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.TryFlushReadRace
2016-04-04T16:42:22.314 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (28118 ms)
2016-04-04T16:42:22.316 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.HitSetRead
2016-04-04T16:42:22.317 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: hmm, no HitSet yet
2016-04-04T16:42:22.318 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: ok, hit_set contains 180:602f83fe:::foo:head
2016-04-04T16:42:22.319 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.HitSetRead (8385 ms)
2016-04-04T16:42:22.320 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.HitSetTrim
2016-04-04T16:42:22.321 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,0
2016-04-04T16:42:22.321 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: first is 1459802516
2016-04-04T16:42:22.322 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,0
2016-04-04T16:42:22.323 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,0
2016-04-04T16:42:22.324 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,1459802518,1459802519,0
2016-04-04T16:42:22.325 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,1459802518,1459802519,0
2016-04-04T16:42:22.326 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,1459802518,1459802519,0
2016-04-04T16:42:22.327 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,1459802518,1459802519,1459802521,1459802522,0
2016-04-04T16:42:22.328 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,1459802518,1459802519,1459802521,1459802522,0
2016-04-04T16:42:22.329 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,1459802518,1459802519,1459802521,1459802522,0
2016-04-04T16:42:22.329 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,1459802518,1459802519,1459802521,1459802522,1459802524,1459802525,0
2016-04-04T16:42:22.330 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,1459802518,1459802519,1459802521,1459802522,1459802524,1459802525,0
2016-04-04T16:42:22.331 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802516,1459802518,1459802519,1459802521,1459802522,1459802524,1459802525,0
2016-04-04T16:42:22.332 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  got ls 1459802519,1459802521,1459802522,1459802524,1459802525,1459802527,1459802528,0
2016-04-04T16:42:22.333 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: first now 1459802519, trimmed
2016-04-04T16:42:22.334 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.HitSetTrim (20375 ms)
2016-04-04T16:42:22.335 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.PromoteOn2ndRead
2016-04-04T16:42:22.336 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: foo0
2016-04-04T16:42:22.337 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: verifying foo0 is eventually promoted
2016-04-04T16:42:22.338 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (13141 ms)
2016-04-04T16:42:26.932 INFO:teuthology.orchestra.run.clara002:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2016-04-04T16:42:26.937 INFO:teuthology.orchestra.run.clara009:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2016-04-04T16:42:26.942 INFO:teuthology.orchestra.run.clara010:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2016-04-04T16:42:45.622 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c
2016-04-04T16:42:45.623 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c
2016-04-04T16:42:45.624 INFO:tasks.ceph.mon.c:Restarting daemon
2016-04-04T16:42:45.625 INFO:teuthology.orchestra.run.clara009:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f -i c'
2016-04-04T16:42:45.630 INFO:tasks.ceph.mon.c:Started
2016-04-04T16:42:45.631 INFO:tasks.mon_thrash:Sending STOP to mon b
2016-04-04T16:42:45.632 INFO:tasks.ceph.mon.b:Sent signal 19
2016-04-04T16:42:45.633 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons
2016-04-04T16:42:45.803 INFO:tasks.ceph.mon.c.clara009.stdout:starting mon.c rank 2 at 10.8.129.9:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid 532ae0c0-b1c3-4aec-b31e-b28b47ea8135
2016-04-04T16:42:57.013 INFO:teuthology.orchestra.run.clara002:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2016-04-04T16:42:57.020 INFO:teuthology.orchestra.run.clara009:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2016-04-04T16:42:57.027 INFO:teuthology.orchestra.run.clara010:Running: 'sudo logrotate /etc/logrotate.d/ceph-test.conf'
2016-04-04T16:43:00.635 INFO:tasks.mon_thrash:Sending CONT to mon b
2016-04-04T16:43:00.637 INFO:tasks.ceph.mon.b:Sent signal 18
2016-04-04T16:43:00.638 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3
2016-04-04T16:43:00.639 INFO:teuthology.orchestra.run.clara002:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph quorum_status'
2016-04-04T16:43:01.873 INFO:tasks.mon_thrash.ceph_manager:quorum_status is {"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","monmap":{"epoch":1,"fsid":"532ae0c0-b1c3-4aec-b31e-b28b47ea8135","modified":"2016-04-04 20:31:26.225792","created":"2016-04-04 20:31:26.225792","mons":[{"rank":0,"name":"a","addr":"10.8.129.2:6789\/0"},{"rank":1,"name":"b","addr":"10.8.129.9:6789\/0"},{"rank":2,"name":"c","addr":"10.8.129.9:6790\/0"}]}}

2016-04-04T16:43:01.874 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3
2016-04-04T16:43:01.875 INFO:teuthology.orchestra.run.clara002:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph -m 10.8.129.2:6789 mon_status'
2016-04-04T16:43:03.104 INFO:teuthology.orchestra.run.clara002:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph -m 10.8.129.9:6789 mon_status'
2016-04-04T16:43:04.335 INFO:teuthology.orchestra.run.clara002:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph -m 10.8.129.9:6790 mon_status'
2016-04-04T16:43:05.567 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub
2016-04-04T16:43:05.568 INFO:teuthology.orchestra.run.clara002:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph scrub'
2016-04-04T16:43:06.800 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing
2016-04-04T16:43:07.802 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.ProxyRead
2016-04-04T16:43:07.803 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.ProxyRead (16168 ms)
2016-04-04T16:43:07.804 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [ RUN      ] LibRadosTwoPoolsECPP.CachePin
2016-04-04T16:43:07.805 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [       OK ] LibRadosTwoPoolsECPP.CachePin (26315 ms)
2016-04-04T16:43:07.807 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [----------] 18 tests from LibRadosTwoPoolsECPP (270236 ms total)
2016-04-04T16:43:07.808 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:
2016-04-04T16:43:07.809 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [----------] Global test environment tear-down
2016-04-04T16:43:07.810 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [==========] 46 tests from 4 test cases ran. (652312 ms total)
2016-04-04T16:43:07.811 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [  PASSED  ] 44 tests.
2016-04-04T16:43:07.812 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [  FAILED  ] 2 tests, listed below:
2016-04-04T16:43:07.814 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [  FAILED  ] LibRadosTwoPoolsPP.Whiteout
2016-04-04T16:43:07.815 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier: [  FAILED  ] LibRadosTwoPoolsECPP.Whiteout
2016-04-04T16:43:07.816 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:
2016-04-04T16:43:07.817 INFO:tasks.workunit.client.0.clara010.stdout:                 api_tier:  2 FAILED TESTS
2016-04-04T16:43:07.818 INFO:tasks.workunit.client.0.clara010.stdout:error in 23165
2016-04-04T16:43:07.819 INFO:tasks.workunit.client.0.clara010.stderr:+ echo 'error in 23165'
2016-04-04T16:43:07.820 INFO:tasks.workunit.client.0.clara010.stderr:+ ret=1
2016-04-04T16:43:07.821 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:43:07.822 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23174
2016-04-04T16:43:07.822 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:43:07.823 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23176
2016-04-04T16:43:07.824 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:43:07.825 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23183
2016-04-04T16:43:07.826 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:43:07.827 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23209
2016-04-04T16:43:07.828 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:43:07.829 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23220
2016-04-04T16:43:07.830 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'
2016-04-04T16:43:07.831 INFO:tasks.workunit.client.0.clara010.stderr:+ wait 23225
2016-04-04T16:43:07.832 INFO:tasks.workunit.client.0.clara010.stderr:+ for p in '$pids'

Expected results:


Additional info:

http://magna002.ceph.redhat.com/vasu-2016-04-04_15:32:20-smoke-jewel---basic-clara/217127/teuthology.log

Comment 2 Samuel Just 2016-04-05 14:31:11 UTC

Test bug, should have been fixed in

c335855e971334c76abd13c722d57ee9e423ea74 (not in the 10.1.0 branch you are testing)

http://tracker.ceph.com/issues/15177

Comment 3 Vasu Kulkarni 2016-04-11 19:06:03 UTC

Verified in 10.1.1

Comment 4 Ken Dreyer (Red Hat) 2016-10-05 15:50:07 UTC

Fixed in the RHCS 2 GA.

https://access.redhat.com/errata/RHBA-2016:1755

Note You need to log in before you can comment on or make changes to this bug.