Bug 1555315 - ansible role to deploy jenkins_builder sometime fail
Summary: ansible role to deploy jenkins_builder sometime fail
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: GlusterFS
Classification: Community
Component: project-infrastructure
Version: mainline
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: M. Scherer
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-03-14 13:14 UTC by Michael S.
Modified: 2020-03-11 12:12 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-11 12:12:59 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Michael S. 2018-03-14 13:14:18 UTC
Description of problem:

From time to time, the job fail with:

remote: fatal: [builder109.cloud.gluster.org]: FAILED! => {"changed": false, "msg": "Unable to start service rpc-statd: Job for rpc-statd.service 
                 failed because the control process exited with error code. See \"systemctl status rpc-statd.service\" and \"journalctl -xe\" for details.\n"}

However, it fix itself after a while.

While invetigating, I did see this was caused by selinux:

type=AVC msg=audit(1521031389.894:2003194): avc:  denied  { write } for  pid=5686 comm="rpc.statd" name="rpc.statd.pid" dev="tmpfs" ino=107552625 
                 scontext=system_u:system_r:rpcd_t:s0 tcontext=unconfined_u:object_r:var_run_t:s0 tclass=file

but once the issue is fixed, I can't reproduce it anymore, even after removing the file.

So tracking progress on the issue until it happen again.

Comment 1 Niels de Vos 2018-03-14 14:17:07 UTC
There should be no need to start rpc.statd on builders. Gluster/NFS will try to start rpc.statd directly (without init/systemd). The cleanup of the tests kill rpc.statd in preparation for the next test.

Comment 2 Nigel Babu 2018-03-14 14:20:46 UTC
The rpc.statd start is a canary in the coal mine for IPV6 issues. It fails to start when IPV6 is disabled without doing a `dracut -v -f`. So without this step the entire process is un-predictable.

Comment 3 Michael S. 2018-03-14 16:45:46 UTC
So, if we do not need them, I can safely remove this then.

Comment 4 M. Scherer 2018-09-03 13:56:09 UTC
Related : https://bugzilla.redhat.com/show_bug.cgi?id=1623410

Comment 5 Worker Ant 2020-03-11 12:12:59 UTC
This bug is moved to https://api.github.com/repos/gluster/project-infrastructure/issues/1, and will be tracked there from now on. Visit GitHub issues URL for further details


Note You need to log in before you can comment on or make changes to this bug.