Bug 1255779

*** Bug 1254712 has been marked as a duplicate of this bug. ***

So the proper procedure would include deleting the hosts from Satellite first, allowing them to re-PXE into foreman discovery, and then try again. 

We also opened bug 1255893 and bug 1255889 which are related

Grr, I meant to mention, QE is going to test this out and see how it goes, this probably will need to move to documentation but I want to see it work before doing that.

After delete and re-discovery of the hosts and start of a second deployment, the Actions::Fusor::Deployment::Rhev::Deploy tasks hangs on timeout:

Foreman::Exception: ERF42-7017 [Foreman::Exception]: You've reached the timeout set for this action. If the action is still ongoing, you can click on the "Resume Deployment" button to continue.

This is because one of the machines is trying to download a kickstart file, which fails. foreman-proxy/proxy.log says:

ERROR -- : Failed to retrieve provision template for e4d36a7f-44b1-4f01-a314-0df432e9954f: Error retrieving provision for e4d36a7f-44b1-4f01-a314-0df432e9954f from sat.rhci.com: Net::HTTPNotFound
    "GET /unattended/provision?token=e4d36a7f-44b1-4f01-a314-0df432e9954f HTTP/1.1" 500 184 0.0567

with corresponding production.log message:

   Parameters: {"token"=>"e4d36a7f-44b1-4f01-a314-0df432e9954f", "unattended"=>{}}
unattended: unable to find a host that matches the request from 192.168.77.1
Filter chain halted as :get_host_details rendered or redirected
Completed 404 Not Found in 3ms (ActiveRecord: 0.9ms)

I tried to download the kickstart multiple times, also restarting the host, nothing helped.

(In reply to Antonin Pagac from comment #4)
> After delete and re-discovery of the hosts and start of a second deployment,
> the Actions::Fusor::Deployment::Rhev::Deploy tasks hangs on timeout:
> 
> Foreman::Exception: ERF42-7017 [Foreman::Exception]: You've reached the
> timeout set for this action. If the action is still ongoing, you can click
> on the "Resume Deployment" button to continue.
> 
> This is because one of the machines is trying to download a kickstart file,
> which fails. foreman-proxy/proxy.log says:
> 
> ERROR -- : Failed to retrieve provision template for
> e4d36a7f-44b1-4f01-a314-0df432e9954f: Error retrieving provision for
> e4d36a7f-44b1-4f01-a314-0df432e9954f from sat.rhci.com: Net::HTTPNotFound
>     "GET /unattended/provision?token=e4d36a7f-44b1-4f01-a314-0df432e9954f
> HTTP/1.1" 500 184 0.0567
> 
> with corresponding production.log message:
> 
>    Parameters: {"token"=>"e4d36a7f-44b1-4f01-a314-0df432e9954f",
> "unattended"=>{}}
> unattended: unable to find a host that matches the request from 192.168.77.1
> Filter chain halted as :get_host_details rendered or redirected
> Completed 404 Not Found in 3ms (ActiveRecord: 0.9ms)
> 
> I tried to download the kickstart multiple times, also restarting the host,
> nothing helped.


This sounds similar to when a provisioning token times out, I wonder if the deletion was not complete and an older token was used for fetching the kickstart.

I'd recommend chatting with Sat6 QE to learn how they test discovery, how do they delete a host and rediscover.

After a second try, I think we made a mistake when deleting hosts from Satellite. All content hosts should be deleted, as well as all hosts that are under Hosts -> All hosts menu in Satellite. Then the host machines can be rebooted, rediscovered and successfully used in second deployment.

However, this stands only for hosts provision and deployment of RHEV. When deployment continues and CFME installation begins, we hit an error:

Net::SSH::HostKeyMismatch: fingerprint 31:68:48:1e:e4:af:65:34:fb:1c:4c:3c:8c:4c:4a:4b does not match for "192.168.77.102"

while running Actions::Fusor::Deployment::Rhev::TransferImage task.
This makes me think that something still persists somewhere, and second deployment can not be successfully completed.

Cloned to 1331555 for leaving SSH Fingerprint in known hosts.  Closing this RFE, as host was not properly deleted.