Bug 1548868

Summary: [Upgrade]4.1.10.1->4.2.2.1 engine upgrade fails.
Product: Red Hat Enterprise Virtualization Manager Reporter: Nikolai Sednev <nsednev>
Component: rhevm-setup-pluginsAssignee: Yedidyah Bar David <didi>
Status: CLOSED ERRATA QA Contact: Jiri Belka <jbelka>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.2.2CC: bugs, didi, gshereme, lsurette, lsvaty, nsednev, ykaul, ylavi
Target Milestone: ovirt-4.2.2Keywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: rhvm-setup-plugins-4.2.4-1.el7ev Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-15 17:34:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
engine logs none

Description Nikolai Sednev 2018-02-25 17:54:13 UTC 
Created attachment 1400621 [details]
engine logs

Description of problem:
[ ERROR ] Failed to execute stage 'Package installation': [u'rhevm-4.1.10.1-0.1.el7.noarch requires rhevm-doc >= 4.0', u'ovirt-engine-4.2.2.1-0.1.el7.noarch requires rhvm = 4.2.2.1-0.1.el7', u'rhevm-4.1.10.1-0.1.el7.noarch requires redhat-support-plugin-rhev >= 4.0', u'rhevm-4.1.10.1-0.1.el7.noarch requires ovirt-engine = 4.1.10.1-0.1.el7']


Then I was still unable to yum update to ovirt-engine-setup-4.2.2.1-0.1.el7.noarch:

Moving to closed as it works for me.

 nsednev-he-1 ~]# engine-setup 
[ INFO  ] Stage: Initializing
[ INFO  ] Stage: Environment setup
          Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-wsp.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf']
          Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20180225123534-2iy676.log
          Version: otopi-1.7.7 (otopi-1.7.7-1.el7ev)
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ INFO  ] Stage: Environment customization
         
          Welcome to the RHV 4.2 setup/upgrade.
          Please read the RHV 4.2 install guide
          https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.2/html/installation_guide/index.html
          Please refer to the RHV Upgrade Helper application
          https://access.redhat.com/labs/rhevupgradehelper/ which will guide you in the upgrading process.
          Would you like to proceed? (Yes, No) [Yes]: 
         
          --== PRODUCT OPTIONS ==--
         
          Configure ovirt-provider-ovn (Yes, No) [Yes]: 
         
          --== PACKAGES ==--
         
[ INFO  ] Checking for product updates...
          Setup needs to install or update the following packages:
          [install] 1:apache-commons-io-2.4-12.el7.noarch will be installed
          [install] archaius-core-0.4.1-1.el7ev.noarch will be installed
          [install] cockpit-dashboard-157-1.el7.x86_64 will be installed
          [install] cockpit-ws-157-1.el7.x86_64 will be installed
          [install] glib-networking-2.50.0-1.el7.x86_64 will be installed
          [install] glusterfs-3.8.4-54.el7.x86_64 will be installed
          [install] glusterfs-api-3.8.4-54.el7.x86_64 will be installed
          [install] glusterfs-client-xlators-3.8.4-54.el7.x86_64 will be installed
          [install] glusterfs-libs-3.8.4-54.el7.x86_64 will be installed
          [install] gnutls-3.3.26-9.el7.x86_64 will be installed
          [install] gsettings-desktop-schemas-3.24.1-1.el7.x86_64 will be installed
          [install] httpcomponents-client-4.2.5-5.el7_0.noarch will be installed
          [install] httpcomponents-core-4.2.4-6.el7.noarch will be installed
          [install] hystrix-core-1.4.21-6.el7ev.noarch will be installed
          [install] hystrix-metrics-event-stream-1.4.21-6.el7ev.noarch will be installed
          [install] jackson-core-2.6.3-1.el7ev.noarch will be installed
          [install] jboss-annotations-1.1-api-1.0.1-0.6.20120212git76e1a2.el7.noarch will be installed
          [install] jctools-1.1-0.3.alpha.el7ev.noarch will be installed
          [install] json-glib-1.2.6-1.el7.x86_64 will be installed
          [install] libmodman-2.0.1-8.el7.x86_64 will be installed
          [install] libproxy-0.4.11-11.el7.x86_64 will be installed
          [install] libssh-0.7.1-3.el7.x86_64 will be installed
          [install] nettle-2.7.1-8.el7.x86_64 will be installed
          [install] openstack-java-cinder-client-3.1.2-2.el7.noarch will be installed
          [install] openstack-java-cinder-model-3.1.2-2.el7.noarch will be installed
          [install] openstack-java-client-3.1.2-2.el7.noarch will be installed
          [install] openstack-java-glance-client-3.1.2-2.el7.noarch will be installed
          [install] openstack-java-glance-model-3.1.2-2.el7.noarch will be installed
          [install] openstack-java-keystone-client-3.1.2-2.el7.noarch will be installed
          [install] openstack-java-keystone-model-3.1.2-2.el7.noarch will be installed
          [install] openstack-java-quantum-client-3.1.2-2.el7.noarch will be installed
          [install] openstack-java-quantum-model-3.1.2-2.el7.noarch will be installed
          [install] openstack-java-resteasy-connector-3.1.2-2.el7.noarch will be installed
          [install] ovirt-ansible-cluster-upgrade-1.1.5-1.el7ev.noarch will be installed
          [install] ovirt-ansible-disaster-recovery-0.2-1.el7ev.noarch will be installed
          [install] ovirt-ansible-engine-setup-1.1.0-1.el7ev.noarch will be installed
          [install] ovirt-ansible-image-template-1.1.5-1.el7ev.noarch will be installed
          [install] ovirt-ansible-infra-1.1.3-1.el7ev.noarch will be installed
          [install] ovirt-ansible-manageiq-1.1.5-1.el7ev.noarch will be installed
          [install] ovirt-ansible-repositories-1.1.0-1.el7ev.noarch will be installed
          [install] ovirt-ansible-roles-1.1.3-1.el7ev.noarch will be installed
          [install] ovirt-ansible-vm-infra-1.1.5-1.el7ev.noarch will be installed
          [install] ovirt-cockpit-sso-0.0.4-1.el7ev.noarch will be installed
          [updated] ovirt-engine-4.1.10.1-0.1.el7.noarch will be updated
          [obsoleting] ovirt-engine-4.2.2.1-0.1.el7.noarch will be installed
          [install] ovirt-engine-api-explorer-0.0.1-1.el7ev.noarch will be installed
          [updated] ovirt-engine-backend-4.1.10.1-0.1.el7.noarch will be updated
          [update] ovirt-engine-backend-4.2.2.1-0.1.el7.noarch is an update
          [updated] ovirt-engine-dashboard-1.1.8-1.el7ev.noarch will be updated
          [update] ovirt-engine-dashboard-1.2.2-1.el7ev.noarch is an update
          [updated] ovirt-engine-dbscripts-4.1.10.1-0.1.el7.noarch will be updated
          [update] ovirt-engine-dbscripts-4.2.2.1-0.1.el7.noarch is an update
          [updated] ovirt-engine-dwh-4.1.9-1.el7ev.noarch will be updated
          [update] ovirt-engine-dwh-4.2.2.1-1.el7ev.noarch is an update
          [updated] ovirt-engine-extension-aaa-jdbc-1.1.6-1.el7ev.noarch will be updated
          [update] ovirt-engine-extension-aaa-jdbc-1.1.7-1.el7ev.noarch is an update
          [updated] ovirt-engine-extensions-api-impl-4.1.10.1-0.1.el7.noarch will be updated
          [update] ovirt-engine-extensions-api-impl-4.2.2.1-0.1.el7.noarch is an update
          [updated] ovirt-engine-metrics-1.0.8-1.el7ev.noarch will be updated
          [update] ovirt-engine-metrics-1.1.3.2-1.el7ev.noarch is an update
          [updated] ovirt-engine-restapi-4.1.10.1-0.1.el7.noarch will be updated
          [update] ovirt-engine-restapi-4.2.2.1-0.1.el7.noarch is an update
          [updated] ovirt-engine-tools-4.1.10.1-0.1.el7.noarch will be updated
          [update] ovirt-engine-tools-4.2.2.1-0.1.el7.noarch is an update
          [updated] ovirt-engine-tools-backup-4.1.10.1-0.1.el7.noarch will be updated
          [update] ovirt-engine-tools-backup-4.2.2.1-0.1.el7.noarch is an update
          [obsoleted] ovirt-engine-userportal-4.1.10.1-0.1.el7.noarch will be removed
          [updated] ovirt-engine-vmconsole-proxy-helper-4.1.10.1-0.1.el7.noarch will be updated
          [update] ovirt-engine-vmconsole-proxy-helper-4.2.2.1-0.1.el7.noarch is an update
          [updated] ovirt-engine-webadmin-portal-4.1.10.1-0.1.el7.noarch will be updated
          [update] ovirt-engine-webadmin-portal-4.2.2.1-0.1.el7.noarch is an update
          [updated] ovirt-engine-websocket-proxy-4.1.10.1-0.1.el7.noarch will be updated
          [update] ovirt-engine-websocket-proxy-4.2.2.1-0.1.el7.noarch is an update
          [updated] ovirt-host-deploy-1.6.7-1.el7ev.noarch will be updated
          [update] ovirt-host-deploy-1.7.2-1.el7ev.noarch is an update
          [updated] ovirt-host-deploy-java-1.6.7-1.el7ev.noarch will be updated
          [update] ovirt-host-deploy-java-1.7.2-1.el7ev.noarch is an update
          [updated] ovirt-iso-uploader-4.0.2-1.el7ev.noarch will be updated
          [update] ovirt-iso-uploader-4.2.0-1.el7ev.noarch is an update
          [install] ovirt-js-dependencies-1.2.0-3.1.el7ev.noarch will be installed
          [updated] ovirt-log-collector-4.1.8-1.el7ev.noarch will be updated
          [update] ovirt-log-collector-4.2.4-1.el7ev.noarch is an update
          [updated] python-ovirt-engine-sdk4-4.1.7-1.el7ev.x86_64 will be updated
          [update] python-ovirt-engine-sdk4-4.2.4-1.el7ev.x86_64 is an update
          [obsoleted] redhat-support-plugin-rhev-4.1.0-1.el7.noarch will be removed
          [install] resteasy-base-jaxrs-3.0.6-4.el7.noarch will be installed
          [install] resteasy-base-jaxrs-api-3.0.6-4.el7.noarch will be installed
          [obsoleted] rhevm-4.1.10.1-0.1.el7.noarch will be removed
          [obsoleted] rhevm-branding-rhev-4.1.8-1.el7ev.noarch will be removed
          [obsoleted] rhevm-dependencies-4.1.1-1.el7ev.noarch will be removed
          [obsoleted] rhevm-doc-4.1.8-1.el7ev.noarch will be removed
          [obsoleting] rhvm-4.2.2.1-0.1.el7.noarch will be installed
          [obsoleting] rhvm-branding-rhv-4.2.2-1.el7ev.noarch will be installed
          [obsoleting] rhvm-dependencies-4.2.1-1.el7ev.noarch will be installed
          [obsoleting] rhvm-doc-4.2.2-1.el7ev.noarch will be installed
          [install] rxjava-1.0.13-3.el7ev.noarch will be installed
          [install] scannotation-1.0.3-0.7.r12.el7.noarch will be installed
          [install] trousers-0.3.14-2.el7.x86_64 will be installed
          [updated] vdsm-jsonrpc-java-1.3.16-1.el7ev.noarch will be updated
          [update] vdsm-jsonrpc-java-1.4.11-1.el7ev.noarch is an update
          Replying "No" will abort Setup. You can pass the option "--offline" to prevent installing or updating packages.
          Do you wish to update them now? (Yes, No) [Yes]: 
[ INFO  ] Checking for an update for Setup...
          Setup will not be able to rollback new packages in case of a failure, because the following installed packages were not found in enabled repositories:
         
          python-ovirt-engine-sdk4-4.1.7-1.el7ev.x86_64
          Do you want to abort Setup? (Yes, No) [Yes]: no
         
          --== NETWORK CONFIGURATION ==--
         
          Setup can automatically configure the firewall on this system.
          Note: automatic configuration of the firewall may overwrite current settings.
          NOTICE: iptables is deprecated and will be removed in future releases
          Do you want Setup to configure the firewall? (Yes, No) [Yes]: 
[ INFO  ] firewalld will be configured as firewall manager.
         
          --== DATABASE CONFIGURATION ==--
         
         
          Found the following problems in PostgreSQL configuration for the Engine database:
           work_mem required to be at least 8192
          Please set:
           work_mem = 8192
          in postgresql.conf on 'localhost'. Its location is usually /var/lib/pgsql/data , or somewhere under /etc/postgresql* .
          The database requires these configurations values to be changed. Setup can fix them for you or abort. Fix automatically? (Yes, No) [Yes]: 
[WARNING] This release requires PostgreSQL server 9.5.9 but the engine database is currently hosted on PostgreSQL server 9.2.23.
          This tool can automatically upgrade PostgreSQL. Automatically upgrade? (Yes, No) [Yes]: 
          PostgreSQL will be upgraded by copying its data to a new directory.
          Do you want to automatically clean up the old data directory on success to reclaim its space (70 MB)? (Yes, No) [Yes]: 
[ INFO  ] Any further action on the DB will be performed only after PostgreSQL has been successfully upgraded to 9.5.
[ INFO  ] Engine DB and DWH one shares the same PostgreSQL instance that is going to be upgraded
          Perform full vacuum on the oVirt engine history
          database ovirt_engine_history@localhost?
          This operation may take a while depending on this setup health and the
          configuration of the db vacuum process.
          See https://www.postgresql.org/docs/9.0/static/sql-vacuum.html
          (Yes, No) [No]: 
         
          --== OVIRT ENGINE CONFIGURATION ==--
         
          Perform full vacuum on the engine database engine@localhost?
          This operation may take a while depending on this setup health and the
          configuration of the db vacuum process.
          See https://www.postgresql.org/docs/9.0/static/sql-vacuum.html
          (Yes, No) [No]: 
          oVirt OVN provider user[admin@internal]: admin
          oVirt OVN provider password: 
         
          --== STORAGE CONFIGURATION ==--
         
         
          --== PKI CONFIGURATION ==--
         
         
          --== APACHE CONFIGURATION ==--
         
         
          --== SYSTEM CONFIGURATION ==--
         
         
          --== MISC CONFIGURATION ==--
         
         
          --== END OF CONFIGURATION ==--
         
[ INFO  ] Stage: Setup validation
          During execution engine service will be stopped (OK, Cancel) [OK]: 
[ INFO  ] Hosted Engine HA is in Global Maintenance mode.
[ INFO  ] Cleaning stale zombie tasks and commands
         
          --== CONFIGURATION PREVIEW ==--
         
          Default SAN wipe after delete           : False
          Firewall manager                        : firewalld
          Update Firewall                         : True
          Host FQDN                               : nsednev-he-1.qa.lab.tlv.redhat.com
          Require packages rollback               : False
          Upgrade packages                        : True
          Engine database secured connection      : False
          Engine database user name               : engine
          Engine database name                    : engine
          Engine database host                    : localhost
          Engine database port                    : 5432
          Engine database host name validation    : False
          Engine installation                     : True
          PKI organization                        : qa.lab.tlv.redhat.com
          Set up ovirt-provider-ovn               : True
          Configure WebSocket Proxy               : True
          DWH installation                        : True
          DWH database secured connection         : False
          DWH database host                       : localhost
          DWH database user name                  : ovirt_engine_history
          DWH database name                       : ovirt_engine_history
          DWH database port                       : 5432
          DWH database host name validation       : False
          Configure Image I/O Proxy               : True
          Configure VMConsole Proxy               : True
         
          Please confirm installation settings (OK, Cancel) [OK]: 
[ INFO  ] Cleaning async tasks and compensations
[ INFO  ] Unlocking existing entities
[ INFO  ] Checking the Engine database consistency
[ INFO  ] Stage: Transaction setup
[ INFO  ] Stopping engine service
[ INFO  ] Stopping ovirt-fence-kdump-listener service
[ INFO  ] Stopping dwh service
[ INFO  ] Stopping Image I/O Proxy service
[ INFO  ] Stopping vmconsole-proxy service
[ INFO  ] Stopping websocket-proxy service
[ INFO  ] Stage: Misc configuration
[ INFO  ] Upgrading PostgreSQL
[ INFO  ] PostgreSQL has been successfully upgraded, starting the new instance (rh-postgresql95-postgresql).
[ INFO  ] Updating PostgreSQL configuration
[ INFO  ] Stage: Package installation
[ ERROR ] Yum [u'rhevm-4.1.10.1-0.1.el7.noarch requires rhevm-doc >= 4.0', u'ovirt-engine-4.2.2.1-0.1.el7.noarch requires rhvm = 4.2.2.1-0.1.el7', u'rhevm-4.1.10.1-0.1.el7.noarch requires redhat-support-plugin-rhev >= 4.0', u'rhevm-4.1.10.1-0.1.el7.noarch requires ovirt-engine = 4.1.10.1-0.1.el7']
[ ERROR ] Failed to execute stage 'Package installation': [u'rhevm-4.1.10.1-0.1.el7.noarch requires rhevm-doc >= 4.0', u'ovirt-engine-4.2.2.1-0.1.el7.noarch requires rhvm = 4.2.2.1-0.1.el7', u'rhevm-4.1.10.1-0.1.el7.noarch requires redhat-support-plugin-rhev >= 4.0', u'rhevm-4.1.10.1-0.1.el7.noarch requires ovirt-engine = 4.1.10.1-0.1.el7']
[ INFO  ] Rolling back to the previous PostgreSQL instance (postgresql).
[ INFO  ] Stage: Clean up
          Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20180225123534-2iy676.log
[ INFO  ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20180225123815-setup.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Execution of setup failed

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.Deploy 4.1.9 SHE over NFS over pair of ha-hosts and add one NFS storage domain.
2.Set hosts in to global maintenance.
3.Upgrade the 4.1.9 engine to the latest bits of 4.1.10.1-0.1, which is ovirt-engine-setup-4.1.10.1-0.1.el7.noarch
4.Run engine-setup to upgrade the engine to latest bits.
5.Upgrade the 4.1.10.1-0.1 engine to 4.2.2.1-0.1 latest bits.
6.Run engine-setup to upgrade the engine to latest bits.

Actual results:
[ ERROR ] Yum [u'rhevm-4.1.10.1-0.1.el7.noarch requires rhevm-doc >= 4.0', u'ovirt-engine-4.2.2.1-0.1.el7.noarch requires rhvm = 4.2.2.1-0.1.el7', u'rhevm-4.1.10.1-0.1.el7.noarch requires redhat-support-plugin-rhev >= 4.0', u'rhevm-4.1.10.1-0.1.el7.noarch requires ovirt-engine = 4.1.10.1-0.1.el7']
[ ERROR ] Failed to execute stage 'Package installation': [u'rhevm-4.1.10.1-0.1.el7.noarch requires rhevm-doc >= 4.0', u'ovirt-engine-4.2.2.1-0.1.el7.noarch requires rhvm = 4.2.2.1-0.1.el7', u'rhevm-4.1.10.1-0.1.el7.noarch requires redhat-support-plugin-rhev >= 4.0', u'rhevm-4.1.10.1-0.1.el7.noarch requires ovirt-engine = 4.1.10.1-0.1.el7']


Expected results:
Engine should go through the upgrade 4.1.10.1->4.2.2.1 without errors.

Additional info:
see the sosreport from the engine.
Comment 1 Yaniv Kaul 2018-02-26 07:29:26 UTC 
Is this upstream or downstream?
Which channels are you subscribed to?
Is that an issue of channels, or something else (wrong deps) ?
Do you see the required packages anywhere?
Comment 2 Sandro Bonazzola 2018-02-26 07:31:55 UTC 
Moving downstream, there's no 4.1.10 released in oVirt.
Didi, it may be related to rhevm-setup-plugins strict requirement on rhv >= 4.1.10 for upgrading to 4.2 but looks more like misconfigured channels.
Can you please have a look?
Comment 3 Yedidyah Bar David 2018-02-26 07:43:30 UTC 
Replying to above questions: This is a simple bug, caused by me, in the fixes for bug 1543526 / bug 1545059. Sorry.
Comment 4 Greg Sheremeta 2018-02-26 16:44:30 UTC 
Didi, check-patch for the new rhv upgrade suite fails, caused by this. dron requests the fix be backported to 4.2
Comment 5 Yedidyah Bar David 2018-02-27 06:32:58 UTC 
It's not a question of backporting, rhvm-setup-plugins didn't branch 4.2 yet. It's only about doing a release and building it in brew. Will handle this soon.
Comment 7 Jiri Belka 2018-03-07 13:50:26 UTC 
ok,

# rpm -q ovirt-engine-setup rhvm-setup-plugins
ovirt-engine-setup-4.2.2.2-0.1.el7.noarch
rhvm-setup-plugins-4.2.5-1.el7ev.noarch

with a little workaround https://bugzilla.redhat.com/show_bug.cgi?id=1552539#c3 but that's different area than respective rhvm-setup-plugins issue
Comment 10 errata-xmlrpc 2018-05-15 17:34:27 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:1473
Comment 11 Franta Kust 2019-05-16 13:08:36 UTC 
BZ<2>Jira Resync