Bug 1881027

As per dev investigation the install command is ran twice
First time: 
time="2020-09-21T11:09:09Z" level=info msg="Submitting step <install>
Second time:
time="2020-09-21T11:11:10Z" level=info msg="Submitting step <install>

Moving to service

It looks like the installer didn't started in a 2 minutes window so the agent called to GetNextStep again go another install command before the first installer started and reported anything.

{code}
time="2020-09-21T11:09:09Z" level=info msg="GetNextSteps cluster: ,<d0b05ec0-7676-4b10-82cc-e4723cc7b41e> host: <40b84db8-fc45-421b-a488-f751a07b9b97>, host status: <installing>" func="github.com/openshift/assisted-service/internal/host.(*InstructionManager).GetNextSteps" file="/go/src/github.com/openshift/origin/internal/host/instructionmanager.go:89" go-id=1863456 pkg=instructions request_id=9b3a2219-86dd-48a9-8a17-9d67bebf3cd9
time="2020-09-21T11:09:09Z" level=info msg="Submitting step <install> id <install-5dda01d5> to cluster <d0b05ec0-7676-4b10-82cc-e4723cc7b41e> host <40b84db8-fc45-421b-a488-f751a07b9b97> Command: <bash> Arguments: <[-c podman run -v /dev:/dev:rw -v /opt:/opt:rw -v /run/systemd/journal/socket:/run/systemd/journal/socket --privileged --pid=host --net=host -v /var/log:/var/log:rw --env PULL_SECRET_TOKEN --name assisted-installer registry.stage.redhat.io/openshift4/assisted-installer-rhel8:v4.6.0-19 --role worker --cluster-id d0b05ec0-7676-4b10-82cc-e4723cc7b41e --boot-device /dev/sda --host-id 40b84db8-fc45-421b-a488-f751a07b9b97 --openshift-version 4.6 --controller-image registry.stage.redhat.io/openshift4/assisted-installer-reporter-rhel8:v4.6.0-15 --url https://api.stage.openshift.com --insecure=false --agent-image registry.stage.redhat.io/openshift4/assisted-installer-agent-rhel8:v4.6.0-15 --host-name worker-0-0 || ( returnCode=$?; podman run --rm --privileged -v /run/systemd/journal/socket:/run/systemd/journal/socket -v /var/log:/var/log --env PULL_SECRET_TOKEN --name logs-sender registry.stage.redhat.io/openshift4/assisted-installer-agent-rhel8:v4.6.0-15 logs_sender -url https://api.stage.openshift.com -cluster-id d0b05ec0-7676-4b10-82cc-e4723cc7b41e -host-id 40b84db8-fc45-421b-a488-f751a07b9b97 --insecure=false -bootstrap=false; exit $returnCode; )]>" func=github.com/openshift/assisted-service/internal/host.logSteps file="/go/src/github.com/openshift/origin/internal/host/instructionmanager.go:125" go-id=1863456 pkg=instructions request_id=9b3a2219-86dd-48a9-8a17-9d67bebf3cd9
time="2020-09-21T11:11:10Z" level=info msg="GetNextSteps cluster: ,<d0b05ec0-7676-4b10-82cc-e4723cc7b41e> host: <40b84db8-fc45-421b-a488-f751a07b9b97>, host status: <installing>" func="github.com/openshift/assisted-service/internal/host.(*InstructionManager).GetNextSteps" file="/go/src/github.com/openshift/origin/internal/host/instructionmanager.go:89" go-id=1865112 pkg=instructions request_id=8d4dfd21-da7f-4387-9a5e-c3efa82405e0
time="2020-09-21T11:11:10Z" level=info msg="Submitting step <install> id <install-9aa410a1> to cluster <d0b05ec0-7676-4b10-82cc-e4723cc7b41e> host <40b84db8-fc45-421b-a488-f751a07b9b97> Command: <bash> Arguments: <[-c podman run -v /dev:/dev:rw -v /opt:/opt:rw -v /run/systemd/journal/socket:/run/systemd/journal/socket --privileged --pid=host --net=host -v /var/log:/var/log:rw --env PULL_SECRET_TOKEN --name assisted-installer registry.stage.redhat.io/openshift4/assisted-installer-rhel8:v4.6.0-19 --role worker --cluster-id d0b05ec0-7676-4b10-82cc-e4723cc7b41e --boot-device /dev/sda --host-id 40b84db8-fc45-421b-a488-f751a07b9b97 --openshift-version 4.6 --controller-image registry.stage.redhat.io/openshift4/assisted-installer-reporter-rhel8:v4.6.0-15 --url https://api.stage.openshift.com --insecure=false --agent-image registry.stage.redhat.io/openshift4/assisted-installer-agent-rhel8:v4.6.0-15 --host-name worker-0-0 || ( returnCode=$?; podman run --rm --privileged -v /run/systemd/journal/socket:/run/systemd/journal/socket -v /var/log:/var/log --env PULL_SECRET_TOKEN --name logs-sender registry.stage.redhat.io/openshift4/assisted-installer-agent-rhel8:v4.6.0-15 logs_sender -url https://api.stage.openshift.com -cluster-id d0b05ec0-7676-4b10-82cc-e4723cc7b41e -host-id 40b84db8-fc45-421b-a488-f751a07b9b97 --insecure=false -bootstrap=false; exit $returnCode; )]>" func=github.com/openshift/assisted-service/internal/host.logSteps file="/go/src/github.com/openshift/origin/internal/host/instructionmanager.go:125" go-id=1865112 pkg=instructions request_id=8d4dfd21-da7f-4387-9a5e-c3efa82405e0
{code}

Installer logs:
{code}
Sep 21 11:11:14 master-0-0 installer[2960]: time="2020-09-21T11:11:14Z" level=info msg="Updating node installation stage: Starting installation - bootstrap"
{code}

Next step called before:
{code}
time="2020-09-21T11:11:10Z" level=info msg="GetNextSteps cluster: ,<d0b05ec0-7676-4b10-82cc-e4723cc7b41e> host: <40b84db8-fc45-421b-a488-f751a07b9b97>, host status: <installing>" func="github.com/openshift/assisted-service/internal/host.(*InstructionManager).GetNextSteps" 
{code}

Eran suggested to add a validate that if docker return specific error that indicate if another instance of the container already running then we don't fail the installation
https://github.com/openshift/assisted-service/blob/master/internal/bminventory/inventory.go#L1556

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633