Bug 1805186

Summary:	Centos Regression failing with ENOSPC.
Product:	[Community] GlusterFS	Reporter:	yati padia <ypadia>
Component:	project-infrastructure	Assignee:	bugs <bugs>
Status:	CLOSED UPSTREAM	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	mainline	CC:	bugs, dkhandel, gluster-infra, mscherer, pasik, ravishankar
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-03-12 04:16:20 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description yati padia 2020-02-20 13:13:02 UTC

Description of problem: Centos Regression is failing on all the patches with error "mkdir: cannot create directory ‘/build/scratch’: No space left on device".



Actual results:
Centos Regression fails

Expected results:
Centos Regression should pass.


Additional info:
Link to the output console: https://build.gluster.org/job/centos7-regression/9451/console

Comment 1 M. Scherer 2020-02-20 14:31:31 UTC

builder204 is indeed full, Deepshika did disconnect it.

[root@builder204 backends]# du -sh *
11M	brick1
3,7G	brick2
6,3G	brick3
8,0K	patchy1
6,3G	patchy2
6,3G	patchy3
[root@builder204 backends]# pwd
/d/backends


Not sure why this wasn't cleaned sooner.

Comment 2 Ravishankar N 2020-02-24 12:54:05 UTC

Smoke also fails with ENOSPC now:
https://build.gluster.org/job/devrpm-fedora/19527/consoleFull
https://build.gluster.org/job/devrpm-fedora/19525/console

Comment 3 M. Scherer 2020-02-24 13:45:29 UTC

 builder48.int.rht.gluster.org seems to be full. Seems notification did disappear from my mail box :/ 
I am cleaning it.

Comment 4 Deepshikha khandelwal 2020-02-25 04:20:22 UTC

Misc, not just builder48 is full. Builder45 and builder47 seems to be full as well and that is the reason 32-bit-build smoke job is failing https://build.gluster.org/job/32-bit-build-smoke/6723/ during the installation of packages.

Comment 5 M. Scherer 2020-02-25 09:40:10 UTC

Seems the issue is caused by mock cache, due to older fedora cache (F28, F29) who do not get cleaned automatically. Not sure if that's safe to automate.

Comment 6 M. Scherer 2020-02-25 09:48:49 UTC

on builder47, seems the logs are not rotated, not sure why.

Comment 7 Worker Ant 2020-03-12 04:16:20 UTC

This bug is moved to https://api.github.com/repos/gluster/project-infrastructure/issues/2, and will be tracked there from now on. Visit GitHub issues URL for further details