Please add priority/severity so we can properly prioritize.

(In reply to Steve Goodman from comment #1)
> Please add priority/severity so we can properly prioritize.

I thought it would be handled as part of bug 1715767, so copying from there. There, PM (Martin) set priority to Medium, and Severity was Medium from that start.

(In reply to Yedidyah Bar David from comment #0)

> Under "Replacing the Red Hat Virtualization Manager Apache CA Certificate",
> we should add a step:
> 
> ================================================================
> 13. To make engine-backup update the system on restore:
> 
> # mkdir -p /etc/ovirt-engine-backup/engine-backup-config.d
> 
> cat << __EOF__ >> /etc/ovirt-engine-backup/engine-backup-config.d/update-system-wide-pki.sh
> BACKUP_PATHS="\${BACKUP_PATHS}
> /etc/ovirt-engine-backup"
> cp -f /etc/pki/ovirt-engine/apache-ca.pem /etc/pki/ca-trust/source/anchors/custom-apache-ca.pem
> update-ca-trust
> __EOF__
> 
> You can replace "custom-apache-ca" with whatever applicable name - e.g. your
> 3rd-party ssl ca vendor name, your organization's IT department, etc. You
> should probably use the same name you used in step 1.
> ================================================================


If I understand correctly, you're saying here:

Create a file named "update-system-wide-pki.sh"
in the directory "/etc/ovirt-engine-backup/engine-backup-config.d/"

And copy the following into that file, where <3rd-party-ca-cert>.pem is the same file from step 1.

BACKUP_PATHS="\${BACKUP_PATHS}
/etc/ovirt-engine-backup"
cp -f /etc/pki/ovirt-engine/apache-ca.pem /etc/pki/ca-trust/source/anchors/<3rd-party-ca-cert>.pem
update-ca-trust

Is that correct?

(In reply to Nikolai Sednev from https://bugzilla.redhat.com/show_bug.cgi?id=1715767#c8)
> Works for me on these components:
> Engine Software Version:4.3.5.4-0.1.el7
> ovirt-hosted-engine-ha-2.3.3-1.el7ev.noarch
> ovirt-hosted-engine-setup-2.3.11-1.el7ev.noarch
> Linux 3.10.0-1061.el7.x86_64 #1 SMP Thu Jul 11 21:02:44 UTC 2019 x86_64
> x86_64 x86_64 GNU/Linux
> Red Hat Enterprise Linux Server release 7.7 (Maipo)

Nikolai, can I assume from this that you have tested the entire documented procedure "Replacing the Red Hat Virtualization Manager Apache CA Certificate" at [1], followed by the new step from comment 0?

[1] https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/html-single/administration_guide/index#Replacing_the_Manager_CA_Certificate

Yes

(In reply to Steve Goodman from comment #3)
> (In reply to Yedidyah Bar David from comment #0)
> 
> > Under "Replacing the Red Hat Virtualization Manager Apache CA Certificate",
> > we should add a step:
> > 
> > ================================================================
> > 13. To make engine-backup update the system on restore:
> > 
> > # mkdir -p /etc/ovirt-engine-backup/engine-backup-config.d
> > 
> > cat << __EOF__ >> /etc/ovirt-engine-backup/engine-backup-config.d/update-system-wide-pki.sh
> > BACKUP_PATHS="\${BACKUP_PATHS}
> > /etc/ovirt-engine-backup"
> > cp -f /etc/pki/ovirt-engine/apache-ca.pem /etc/pki/ca-trust/source/anchors/custom-apache-ca.pem
> > update-ca-trust
> > __EOF__
> > 
> > You can replace "custom-apache-ca" with whatever applicable name - e.g. your
> > 3rd-party ssl ca vendor name, your organization's IT department, etc. You
> > should probably use the same name you used in step 1.
> > ================================================================
> 
> 
> If I understand correctly, you're saying here:
> 
> Create a file named "update-system-wide-pki.sh"
> in the directory "/etc/ovirt-engine-backup/engine-backup-config.d/"

Yes, and also create the directory, because it does not normally exist

> 
> And copy the following into that file, where <3rd-party-ca-cert>.pem is the
> same file from step 1.
> 
> BACKUP_PATHS="\${BACKUP_PATHS}
> /etc/ovirt-engine-backup"
> cp -f /etc/pki/ovirt-engine/apache-ca.pem
> /etc/pki/ca-trust/source/anchors/<3rd-party-ca-cert>.pem
> update-ca-trust
> 
> Is that correct?

More or less, yes. If you prefer to supply that as manual instructions instead of a bash command ("Open an editor, copy/paste/edit this into it, etc."), then you should drop the backslash from '"\${BACKUP_PATHS}', e.g.:

BACKUP_PATHS="${BACKUP_PATHS}
/etc/ovirt-engine-backup"
cp -f /etc/pki/ovirt-engine/apache-ca.pem
/etc/pki/ca-trust/source/anchors/<3rd-party-ca-cert>.pem
update-ca-trust

The name "<3rd-party-ca-cert>" is completely up to the user to pick.

Didi, please take a look at the new text at:

https://gitlab.cee.redhat.com/rhci-documentation/docs-Red_Hat_Enterprise_Virtualization/merge_requests/1450

The new text is at step 13:

https://pnt-devops-rad-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/ccs-mr-preview-prod/6447/artifact/doc-Administration_Guide/preview/index.html#Replacing_the_Manager_CA_Certificate

How does it look?

(In reply to Steve Goodman from comment #7)
> Didi, please take a look at the new text at:
> 
> https://gitlab.cee.redhat.com/rhci-documentation/docs-
> Red_Hat_Enterprise_Virtualization/merge_requests/1450
> 
> The new text is at step 13:
> 
> https://pnt-devops-rad-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/ccs-mr-
> preview-prod/6447/artifact/doc-Administration_Guide/preview/index.
> html#Replacing_the_Manager_CA_Certificate
> 
> How does it look?

Looks good to me.

Eli,

Please do a peer review. See comment 7 for details.

(In reply to Steve Goodman from comment #9)
> Eli,
> 
> Please do a peer review. See comment 7 for details.

Looks good to me.

Nikolai,

Please test the procedure as documented.

Merge request is at https://gitlab.cee.redhat.com/rhci-documentation/docs-Red_Hat_Enterprise_Virtualization/merge_requests/1450

The new text is at step 13:

https://pnt-devops-rad-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/ccs-mr-preview-prod/7011/artifact/doc-Administration_Guide/preview/index.html#Replacing_the_Manager_CA_Certificate

The procedure I edited is "Replacing the Red Hat Virtualization Manager Apache CA Certificate"

I added step 13, but please test the entire procedure.

Nikolai, bug 1770959 is now VERIFIED. Can you proceed with this one?

Unable to deploy HE using these components:
Red Hat Enterprise Linux release 8.1 (Ootpa)
Linux 4.18.0-147.3.1.el8_1.x86_64 #1 SMP Wed Nov 27 01:11:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
ovirt-hosted-engine-ha-2.4.0-1.el8ev.noarch
ovirt-hosted-engine-setup-2.4.0-1.el8ev.noarch

Deployment got stuck during connection to the engine at the finish part of the deployment.
Host sees:
yum list | grep appliance
rhvm-appliance.x86_64   2:4.4-20191219.0.el8ev @rhv-4.4.0

Can you please try with 4.3? Thanks.

Just to clarify comment 25, this bug is targeting 4.3.

Nikolai, where does this stand?

Thanks a lot for the support Petr Kubica, Petr Matyas and Lucie Leistnerova.
We ran through https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/html-single/administration_guide/index#Replacing_the_Manager_CA_Certificate and using Didi's https://bugzilla.redhat.com/show_bug.cgi?id=1744522#c0 , I finally verified the bug. CA certificate that Peter generated had been properly embedded by me in to the engine from my first environment, then I ran through the doc and made backup and restore on another host. 
The step 13 as appears in https://bugzilla.redhat.com/show_bug.cgi?id=1744522#c0 should become step 13 in https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/html-single/administration_guide/index#Replacing_the_Manager_CA_Certificate, then existing step 13 in the doc should be moved to 14 and so on, e.g.:

13. To make engine-backup update the system on restore:

# mkdir -p /etc/ovirt-engine-backup/engine-backup-config.d

cat << __EOF__ >> /etc/ovirt-engine-backup/engine-backup-config.d/update-system-wide-pki.sh
BACKUP_PATHS="\${BACKUP_PATHS}
/etc/ovirt-engine-backup"
cp -f /etc/pki/ovirt-engine/apache-ca.pem /etc/pki/ca-trust/source/anchors/custom-apache-ca.pem
update-ca-trust
__EOF__
details and then looked on 
You can replace "custom-apache-ca" with whatever applicable name - e.g. your 3rd-party ssl ca vendor name, your organization's IT department, etc. You should probably use the same name you used in step 1.

14.Restart the ovirt-provider-ovn service:

# systemctl restart ovirt-provider-ovn.service

15.Restart the ovirt-engine service:

# systemctl restart ovirt-engine.service

BTW, regarding HE environments, I'd add that it should be in global maintenance during Replacing the Red Hat Virtualization Manager Apache CA Certificate, as if running through the steps 14&15, engine might be restarted by ha-agent, due to unreachable ovirt-engine.service, liveliness check.

During the verification, I checked that new CA certificate took place before the backup, while I connected to the engine and checked on lock symbol inside the URL, then checked on more information and then view certificate and then validity->not before, and saw there January 23, 2020, 6:25:35 PM GMT+2
(January 23, 2020, 4:25:35 PM GMT), which was correct as we created the certificate that day.
After I had already the environment in global maintenance, made backup and then restored the engine on another host and over different clean NFS storage, the restore was successful and I checked the CA certificate again and it was the same as before the backup, the same which I embedded in to my environment.

Works for me on these components:
Engine Version 4.3.8.1-0.1.master.el7
ovirt-hosted-engine-setup-2.3.12-1.el7ev.noarch
ovirt-hosted-engine-ha-2.3.6-1.el7ev.noarch
Red Hat Enterprise Linux Server release 7.7 (Maipo)
Linux 3.10.0-1062.9.1.el7.x86_64 #1 SMP Mon Dec 2 08:31:54 EST 2019 x86_64 x86_64 x86_64 GNU/Linux

Moving to verified.

(In reply to Nikolai Sednev from comment #40)
> 
> BTW, regarding HE environments, I'd add that it should be in global
> maintenance during Replacing the Red Hat Virtualization Manager Apache CA
> Certificate, as if running through the steps 14&15, engine might be
> restarted by ha-agent, due to unreachable ovirt-engine.service, liveliness
> check.
> 
> During the verification, I checked that new CA certificate took place before
> the backup, while I connected to the engine and checked on lock symbol
> inside the URL, then checked on more information and then view certificate
> and then validity->not before, and saw there January 23, 2020, 6:25:35 PM
> GMT+2
> (January 23, 2020, 4:25:35 PM GMT), which was correct as we created the
> certificate that day.
> After I had already the environment in global maintenance, made backup and
> then restored the engine on another host and over different clean NFS
> storage, the restore was successful and I checked the CA certificate again
> and it was the same as before the backup, the same which I embedded in to my
> environment.

How's this (I added a step at the beginning of the procedure):

Replacing the Red Hat Virtualization Manager Apache CA Certificate

1.  If you are using a self-hosted engine, put the environment into global maintenance mode. For more information, see https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/html-single/administration_guide/index#Maintaining_the_Self-Hosted_Engine.

2.  Add your CA certificate to the host-wide trust store:

    # cp /tmp/3rd-party-ca-cert.pem /etc/pki/ca-trust/source/anchors
    # update-ca-trust

...

Looks good to me. Don't forget to remove the global maintenance, once restore is accomplished.

OK, here's the end of the procedure now, does it look good?

15. Restart the ovirt-provider-ovn service:

    # systemctl restart ovirt-provider-ovn.service

16. Restart the ovirt-engine service:

    # systemctl restart ovirt-engine.service

17. If you are using a self-hosted engine, turn off global maintenance mode.

    # hosted-engine --set-maintenance --mode=none

Looks fine.

Published to https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/html-single/administration_guide/index?lb_target=production#Replacing_the_Manager_CA_Certificate