Bug 1287825
| Summary: | After updating from 7.1 -> 7.2 the compute node still has packages available for update | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Marius Cornea <mcornea> | ||||||||||
| Component: | rhosp-director | Assignee: | James Slagle <jslagle> | ||||||||||
| Status: | CLOSED ERRATA | QA Contact: | Marius Cornea <mcornea> | ||||||||||
| Severity: | high | Docs Contact: | |||||||||||
| Priority: | unspecified | ||||||||||||
| Version: | 7.0 (Kilo) | CC: | dmacpher, jcoufal, jslagle, mburns, mcornea, rhel-osp-director-maint, sasha, tdunnon | ||||||||||
| Target Milestone: | y2 | ||||||||||||
| Target Release: | 7.0 (Kilo) | ||||||||||||
| Hardware: | Unspecified | ||||||||||||
| OS: | Unspecified | ||||||||||||
| Whiteboard: | |||||||||||||
| Fixed In Version: | openstack-tripleo-heat-templates-0.8.6-90.el7ost | Doc Type: | Bug Fix | ||||||||||
| Doc Text: | Story Points: | --- | |||||||||||
| Clone Of: | Environment: | ||||||||||||
| Last Closed: | 2015-12-21 16:54:23 UTC | Type: | Bug | ||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||
| Documentation: | --- | CRM: | |||||||||||
| Verified Versions: | Category: | --- | |||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
| Embargoed: | |||||||||||||
| Attachments: |
|
||||||||||||
we need to see the journalctl output for os-collect-config on the compute node also, please show the output of the UpdateDeployment resource of the Compute node from Heat using "heat deployment-show <deployment-uuid>". to get the output of the UpdateDeployment resource, you could do: on the undercloud, source stackrc heat resource-list overcloud You should see a resource named "Compute". Run heat resource-list on the uuid from the physical_resource_id column of the "Compute" resource. heat resource-list <Compute-resource-uuid> You should see a resource named "0". Run heat resource-list on the uuid from the physical_resource_id column of the "0" resource. heat resource-list <0-resource-uuid> You should see a resource named "UpdateDeployment". Run heat deployment-show on the uuid from the physical_resource_id column of the UpdateDeployment resource. heat deployment-show <UpdateDeployment-resource-uuid> The UpdateDeployment is responsible for running the yum_update.sh script that updates packages. Assuming the UpdateDeployment got executed, the deployment-show command will show you useful information such as deploy_stdout, deploy_stderr, deploy_status_code. Created attachment 1101620 [details]
UpdateDeployment output
Attaching the UpdateDeployment output. From what I can tell there were some packages skipped due to dependency problems.
Created attachment 1101621 [details]
os-collect-config.log
Attaching the os-collect-config.log.
Created attachment 1101623 [details]
better formated UpdateDeployment output
can you ssh to the compute node and manually try to update one of the packages that didn't update due to dependency issue (such as openstack-nova-api)? maybe the yum output will tell us something. try: yum update openstack-nova-api openstack-nova-api was already updated (it shows up in the os-collect-config log) I tried manually updating openstack-neutron-lbaas that had available updates and got it successfully updated: yum update openstack-neutron-lbaas Updated: openstack-neutron-lbaas.noarch 0:2015.1.2-1.el7ost Dependency Updated: python-neutron-lbaas.noarch 0:2015.1.2-1.el7ost As a side note: how can I get the UpdateDeployment output in a better format as you attached it? It will be useful for future reports. i notice that the yum update command has several --exclude args. I wonder if that is related to the issue: Running: yum -y update --skip-broken --exclude ceph --exclude libvirt-daemon-config-nwfilter --exclude libvirt-daemon-kvm --exclude net-snmp --exclude ntp --exclude openstack-ceilometer-common --exclude openstack-ceilometer-compute --exclude openstack-neutron --exclude openstack-neutron-ml2 --exclude openstack-neutron-openvswitch --exclude openstack-nova-common --exclude openstack-nova-compute --exclude openvswitch --exclude pm-utils --exclude python-greenlet --exclude python-nova Afaik these are excluded because they get updated by puppet: Notice: /Stage[main]/Ceilometer::Agent::Compute/Package[ceilometer-agent-compute]/ensure: ensure changed '2015.1.1-1.el7ost' to '0:2015.1.2-1.el7ost' Notice: /Stage[main]/Nova/Package[python-greenlet]/ensure: ensure changed '0.4.2-2.el7ost' to '0:0.4.2-3.el7' Notice: /Stage[main]/Nova/Package[python-nova]/ensure: ensure changed '2015.1.1-1.el7ost' to '0:2015.1.2-4.el7ost' Notice: /Stage[main]/Nova/Package[nova-common]/ensure: ensure changed '2015.1.1-1.el7ost' to '0:2015.1.2-4.el7ost' Notice: /Stage[main]/Neutron/Package[neutron]/ensure: ensure changed '2015.1.1-6.el7ost' to '0:2015.1.2-2.el7ost' Notice: /Stage[main]/Snmp/Package[snmpd]/ensure: ensure changed '5.7.2-20.el7_1.1' to '1:5.7.2-24.el7' Notice: /Stage[main]/Nova::Compute::Libvirt/Package[libvirt-nwfilter]/ensure: ensure changed '1.2.8-16.el7_1.4' to '0:1.2.17-13.el7' Notice: /Stage[main]/Ceilometer/Package[ceilometer-common]/ensure: ensure changed '2015.1.1-1.el7ost' to '0:2015.1.2-1.el7ost' Notice: /Stage[main]/Nova::Compute::Libvirt/Package[libvirt]/ensure: ensure changed '1.2.8-16.el7_1.4' to '0:1.2.17-13.el7' Notice: /Stage[main]/Vswitch::Ovs/Package[openvswitch]/ensure: ensure changed '2.3.2-1.git20150730.el7_1' to '0:2.4.0-1.el7' Notice: /Stage[main]/Ntp::Install/Package[ntp]/ensure: ensure changed '4.2.6p5-19.el7_1.1' to '0:4.2.6p5-22.el7' Notice: /Stage[main]/Nova::Compute/Nova::Generic_service[compute]/Package[nova-compute]/ensure: ensure changed '2015.1.1-1.el7ost' to '0:2015.1.2-4.el7ost' Notice: /Stage[main]/Neutron::Plugins::Ml2/Package[neutron-plugin-ml2]/ensure: ensure changed '2015.1.1-6.el7ost' to '0:2015.1.2-2.el7ost' Notice: /Stage[main]/Neutron::Agents::Ml2::Ovs/Package[neutron-ovs-agent]/ensure: ensure changed '2015.1.1-6.el7ost' to '0:2015.1.2-2.el7ost' right, so not everything is going to get updated by puppet. Not all packages will end up as updated, given that the puppet manifest on the compute node doesn't manage every single packages installed on the system such as openstack-neutron-lbaas. openstack-neutron-lbaas was excluded from the update since openstack-neutron was specificed via --exclude, and that broke the deps. Later when puppet ran, it did update the neutron packages it knows about (neutron-ovs-agent for instance), but lbaas doesn't run on compute nodes so it didn't get updated. i think this might be working as designed. yum check-update on my compute after update results in: glusterfs.x86_64 glusterfs-api.x86_64 glusterfs-libs.x86_64 openstack-neutron-lbaas.noarch python-neutron-lbaas.noarch python-werkzeug.noarch rsyslog.x86_64 rsyslog-mmjsonparse.x86_64 (In reply to Alexander Chuzhoy from comment #13) > yum check-update on my compute after update results in: > > glusterfs.x86_64 > glusterfs-api.x86_64 > glusterfs-libs.x86_64 > openstack-neutron-lbaas.noarch > python-neutron-lbaas.noarch > python-werkzeug.noarch > rsyslog.x86_64 > rsyslog-mmjsonparse.x86_64 what images did you start with? i'm seeing the same thing as mcornea after my 7.1 update. afaict, there are definitely packages not updated that probably should have been. libvirt-python is one example. It requires a minimum version of libvirt. When yum-update.sh runs with --exclude libvirt-daemon-config-nwlter --exclude libvirt-daemon-kvm, the updated version of libvirt ends up getting excluded as well, which means the update of libvirt-python is also excluded, so it's not updated. Later when puppet runs with ensure=>latest, it updates the packages it knows about. However, it only knows to to update libvirt-daemon-config-nwlter and libvirt-daemon-kvm. Updating those 2 packages does not pull in the latest libvirt-python, therefore libvirt-python is left at the older version. Given libvirt-python is a dependency of openstack-nova-compute, we probably should be updating it, so I'm requesting blocker for this bz. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:2651 |
Created attachment 1101593 [details] packages list Description of problem: After updating from 7.1 -> 7.2 the compute node still has packages available for update How reproducible: 100% Steps to Reproduce: 1. Deploy 7.1 overcloud 2. Run update procedure to 7.2 3. yum check-update on the compute nodes Actual results: [root@overcloud-compute-0 heat-admin]# yum check-update | wc -l 51 Expected results: There are no available updates as all the packages have been updated during the update procedure. Additional info: Attaching list of packages and repos.