Bug 1944736

Summary:	Swift Permission denied error after undercloud upgrade 16.1.3->16.1.4
Product:	Red Hat OpenStack	Reporter:	Chris Janiszewski <cjanisze>
Component:	puppet-swift	Assignee:	RHOS Maint <rhos-maint>
Status:	CLOSED DUPLICATE	QA Contact:	David Rosenfeld <drosenfe>
Severity:	unspecified	Docs Contact:	Chuck Copello <ccopello>
Priority:	unspecified
Version:	16.1 (Train)	CC:	cschwede, jjoyce, jschluet, slinaber, tvignaud
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-03-31 12:21:57 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Chris Janiszewski 2021-03-30 15:10:16 UTC

Description of problem:
Swift stops working and reports a following errors on undercloud after 16.1.3->16.1.4 upgrade

Mar 30 00:13:21 chrisj-dcn2-undercloud object-server[496554]: ERROR: Unable to access /srv/node/d1: [Errno 13] Permission denied: '/srv/node/d1'
Mar 30 00:13:21 chrisj-dcn2-undercloud object-server[496554]: Object update sweep completed on /srv/node/d1 in 0.00s seconds:, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 1 errors, 0 redirects (pid: 92)
Mar 30 00:13:21 chrisj-dcn2-undercloud object-server[496554]: Object update sweep of d1 completed: 0.00s, 0 successes, 0 failures, 0 quarantines, 0 unlinks, 1 errors, 0 redirects
Mar 30 00:13:21 chrisj-dcn2-undercloud object-server[6335]: Object update sweep completed: 0.11s
Mar 30 00:14:52 chrisj-dcn2-undercloud account-server[8475]: 10.10.0.10 - - [30/Mar/2021:04:14:52 +0000] "HEAD /d1/281/.expiring_objects" 404 - "HEAD http://localhost/v1/.expiring_objects?format=json" "tx425ff28016b14ade861eb-006062a5bc" "proxy-server 6" 0.0006 "-" 19 -
Mar 30 00:14:52 chrisj-dcn2-undercloud object-expirer[6690]: Pass completed in 0s; 0 objects expired (txn: tx425ff28016b14ade861eb-006062a5bc)
Mar 30 00:17:46 chrisj-dcn2-undercloud container-server[8080]: Begin container update sweep
Mar 30 00:17:46 chrisj-dcn2-undercloud container-server[8080]: ERROR:  Failed to get paths to drive partitions: [Errno 13] Permission denied: '/srv/node/d1/containers'

I did not see any error during the upgrade and the job has finished with successful status 


Version-Release number of selected component (if applicable):
OSP 16.1.3->4

How reproducible:
Everytime

Steps to Reproduce:
1. Deploy undercloud with osp16.1.3
2. Upgrade undercloud to 16.1.4
3.

Actual results:
error

Expected results:


Additional info:
Since glance in undercloud is backed by swift I can no longer complete the upgrade procedure and update the cloud images.

Comment 2 Chris Janiszewski 2021-03-30 15:24:22 UTC

sosreport -> http://chrisj.cloud/sosreport-chrisj-dcn2-undercloud-2021-03-30-mswevsv.tar.xz

Comment 3 Christian Schwede (cschwede) 2021-03-31 12:21:57 UTC


*** This bug has been marked as a duplicate of bug 1941412 ***