Created attachment 1283773 [details] hostname/api/1.0/jobs/ab50ec84-ea42-4b96-9d11-691225a65c80/messages Description of problem: I created Create gluster cluster job with 4 nodes. After while there are no new messages and job never finishes. In attachment is file containing response from `hostname/api/1.0/jobs/ab50ec84-ea42-4b96-9d11-691225a65c80/messages`. Version-Release number of selected component (if applicable): tendrl-node-agent-3.0-alpha.4.el7scon.noarch tendrl-dashboard-3.0-alpha.2.el7scon.noarch tendrl-api-3.0-alpha.2.el7scon.noarch tendrl-commons-3.0-alpha.5.el7scon.noarch tendrl-node-monitoring-3.0-alpha.1.el7scon.noarch tendrl-performance-monitoring-3.0-alpha.2.el7scon.noarch etcd-3.1.7-1.el7.x86_64 How reproducible: 80% Steps to Reproduce: 1. Try to create gluster cluster with 4 nodes 2. Check job status and messages via API Actual results: Job remains in `processing` state. Expected results: Job should finish or fail. Additional info:
Hi, In the logs attached at https://bugzilla.redhat.com/attachment.cgi?id=1283773 I can see there's more than 4 nodes in the CreateCluster API Search in above logs for: "SDS install and config completed, Waiting for tendrl-node-agent to detect newly installed sds details" Nodes participating in cluster creation: [u'e273b8ee-0609-44a5-8cae-ea9e6e877ead', u'75aefa6c-5e9e-4e6f-bab3-425eaa0c25af', u'3dad7284-fa21-4764-add3-493dde643081', u'6d230106-4c0f-4cd3-976d-37ac8463e7e2', u'4b2e71c4-a5d9-4e8c-af3c-b331428b11b5', u'807ab842-1716-4de0-820d-fba387177de8', u'50b6e29a-94c7-4398-b966-1b720c8a22f2', u'05e1e393-cfbc-49cf-849c-74d85698ab40', u'a3c6b579-67f1-4e04-b653-c9597010cc5b', u'4eda52b9-fa47-4b3d-8dfa-f7ea41e435a7', u'2c489b2d-162f-43bd-a43c-1ec29740366a'] Filip, please provide the job payload for job id ab50ec84-ea42-4b96-9d11-691225a65c80 Filip I can see the ssh setup was completed for all above nodes, and the gluster cluster was also created by python-gdeploy, can you confirm if all the nodes have gluster repo setup? Tendrl Gluster provisioner is node (4b2e71c4-a5d9-4e8c-af3c-b331428b11b5)
Hi, there were 7 other nodes intended to be used as ceph nodes (4 osds and 3 mons). Unfortunately I have reinstalled machines but I have tested the same testcase today with new packages and cluster was correctly created. I will do some further testing and try to reproduce the issue. I will update after I get more results.
Today I wasn't able to create CreateCluster job as described in https://bugzilla.redhat.com/show_bug.cgi?id=1460197 I will try it again tomorrow and update this bz accordingly.
I was able to create dashboard via dashboard. There were no errors. --> VERIFIED If I run across the issue again, I will reopen this bug. Tested with: tendrl-alerting-3.0-alpha.3.el7scon.noarch tendrl-api-3.0-alpha.4.el7scon.noarch tendrl-api-doc-3.0-alpha.4.el7scon.noarch tendrl-api-httpd-3.0-alpha.4.el7scon.noarch tendrl-commons-3.0-alpha.9.el7scon.noarch tendrl-dashboard-3.0-alpha.4.el7scon.noarch tendrl-node-agent-3.0-alpha.9.el7scon.noarch tendrl-performance-monitoring-3.0-alpha.7.el7scon.noarch