Bug 1151488
| Summary: | [ppc] Host is installed with VDSM version (<UNKNOWN>) and cannot join cluster ppc64... | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Jiri Belka <jbelka> | ||||
| Component: | ovirt-engine | Assignee: | Yaniv Bronhaim <ybronhei> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Jiri Belka <jbelka> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | urgent | ||||||
| Version: | 3.5.0 | CC: | bkorren, danken, ecohen, gklein, iheim, jbelka, lpeer, lsurette, michal.skrivanek, oourfali, pstehlik, rbalakri, Rhev-m-bugs, sherold, ybronhei, yeylon | ||||
| Target Milestone: | --- | Keywords: | Reopened, ZStream | ||||
| Target Release: | 3.5.0 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | infra | ||||||
| Fixed In Version: | org.ovirt.engine-root-3.5.0-18 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1152448 (view as bug list) | Environment: | |||||
| Last Closed: | 2015-02-17 17:17:24 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Infra | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1122979, 1152448 | ||||||
| Attachments: |
|
||||||
Yaniv/Oved, there needs to be a proper engine side fix as well. In 3.5 (in contrast to 3.4.z) the engine requires the packages to be reported for some reason. This is probably a regression in engine We want the vdsm fix for sure, but the engine side needs to be addressed too (In reply to Michal Skrivanek from comment #3) > Yaniv/Oved, there needs to be a proper engine side fix as well. In 3.5 (in > contrast to 3.4.z) the engine requires the packages to be reported for some > reason. This is probably a regression in engine > We want the vdsm fix for sure, but the engine side needs to be addressed too As long as there are package version the engine should know to cope with that. What additional fix you had in mind? 3.4 engine works even when the pkgs are not reported. "packages2" should not be needed for joining the cluster. The vdsm version is reported in other field which should be used. So it is a 3.5 regression in engine I guess; or at least a 3.5-only problem in engine. please verify that with the attached patch and see that engine adds the host properly. afaiu if engine didn't fail the addHost to cluster in 3.4 before, its due to a bug that was fixed regarding the version validation between 3.4 to 3.5. with unknown version engine should fail it Yes it works with 3.4.3 without problem (vdsm-4.14.17-1.pkvm2_1.ppc64). The vdsm-side patch would get to 3.4.4. But it seems that there has been an unintentional behavioral change in Engine. Please consider fixing it in 3.5.1. *** Bug 1151489 has been marked as a duplicate of this bug. *** please also read bug 1151489 for more details we need to fix engine behavior in 3.5 to look at proper variables. Oved, can you please prioritize? We need to work with ppc host for that, and sometimes it delays the work a bit. I move that to 3.5.1 as it requires verification and debugging to understand the change in HandleVdsVersionCommand or the version parsing Starting to verify this issue and I need to clarify the answer: Revisiting this area again (HandleVdsVersionCommand) I see again that firstly engine checks isEngineSupportedByVdsm by checking the supported engine value from getCaps (example from vdsm 3.5: supportedENGINEs = ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5']). this returned normally regarding the ppc bug which lead to unreported rpm versions. In the case where engine supported by vdsm - like vdsm and engine 3.5, all works and you won't see the UNKNOWN eventhough vdsm version is not reported properly (you see it in the UI - VDSM Version: [N/A] in host's tab). But if the supportedByVdsm is false, and vdsm version is not reported, such as this case, you will see the report: Host a1a1 is installed with VDSM version (<UNKNOWN>) and cannot join cluster Default which is compatible with VDSM versions [4.13, 4.14, 4.9, 4.11, 4.15, 4.12, 4.10]. saying that, not sure what is your case. but the code is similar both in 3.5 and 3.4, it just depends on the versions you run I intend to close it as NOTABUG. (In reply to Yaniv Bronhaim from comment #13) > Starting to verify this issue and I need to clarify the answer: > > Revisiting this area again (HandleVdsVersionCommand) I see again that > firstly engine checks isEngineSupportedByVdsm by checking the supported > engine value from getCaps (example from vdsm 3.5: supportedENGINEs = ['3.0', > '3.1', '3.2', '3.3', '3.4', '3.5']). this returned normally regarding the > ppc bug which lead to unreported rpm versions. > > In the case where engine supported by vdsm - like vdsm and engine 3.5, all > works and you won't see the UNKNOWN eventhough vdsm version is not reported > properly (you see it in the UI - VDSM Version: [N/A] in host's tab). > > But if the supportedByVdsm is false, and vdsm version is not reported, such > as this case, you will see the report: > Host a1a1 is installed with VDSM version (<UNKNOWN>) and cannot join cluster > Default which is compatible with VDSM versions [4.13, 4.14, 4.9, 4.11, 4.15, > 4.12, 4.10]. > > saying that, not sure what is your case. but the code is similar both in 3.5 > and 3.4, it just depends on the versions you run > > I intend to close it as NOTABUG. The only claim by Michal was that without applying your fix it worked well in 3.4, but not in 3.5. I also remember that the code is the same, and I think it probably worked well in a bit different use-case. We've been doing this validation a long time now, and that's the correct one. Closing this one. Michal - if you notice anything wrong with the logic/behavior, please reopen. I suppose the statement in #13 was never tested. Reproduces with vt7 (3.5.0-0.17.beta.el6e) and vdsm-4.14.17-10.gitbb25d3b.el7.ppc64 2014-Oct-27, 09:49 Host ibm-p8-rhevm-03.rhts.eng.bos.redhat.com is installed with VDSM version (<UNKNOWN>) and cannot join cluster ppc64 which is compatible with VDSM versions [4.13, 4.14, 4.9, 4.11, 4.15, 4.12, 4.10]. Yes, it is _3.5_ ! According to some discussion it's strange a BZ can be switched from ASSIGNED to CLOSED/WORKSFORME. We need to verify that the following works: 1. Work with engine 3.5 2. Work with the most updated PPC VDSM version 3. Try to add the host to the engine 4. Make sure it works If it works well, we can close this one. If not, we need to apply the fix to make it work. Jiri - Yaniv can verify that if he gets access to the host. Or you help us verify that. The bug was closed due to understanding it works with latest 3.5 release + latest 3.4 PPC vdsm release. If it doesn't, then we have an issue. If it does, it can be either moved to MODIFIED for proper verification, or closed. (In reply to Oved Ourfali from comment #16) > We need to verify that the following works: > 1. Work with engine 3.5 see above > 2. Work with the most updated PPC VDSM version NAME="RHEV for IBM POWER" VERSION="3.4" ID="rhev for ibm power" VERSION_ID="3.4" BUILD_ID="36.0-service" PRETTY_NAME="RHEV for IBM POWER 3.4" ANSI_COLOR="0;34" CPE_NAME="cpe:/o:ibm:service:pkvm2_1" > 3. Try to add the host to the engine > 4. Make sure it works no it does not > If it works well, we can close this one. > If not, we need to apply the fix to make it work. > > Jiri - Yaniv can verify that if he gets access to the host. > Or you help us verify that. > > The bug was closed due to understanding it works with latest 3.5 release + > latest 3.4 PPC vdsm release. If it doesn't, then we have an issue. If it > does, it can be either moved to MODIFIED for proper verification, or closed. "understanding it works" - then it should be CLOSEDCURRENTRELEASE, no? just to clarify (again): - vdsm is rhev for power (3.4) - engine is d/s 3.5 (it is *obvious* the cluster level is 3.4) The reason it works on 3.4 and not 3.5, is because the engine is 3.5, and vdsm is 3.4, so the supportedEngines entry in VDSM doesn't contain 3.5, thus we check for the package version. In 3.4 it isn't checked because 3.4 is a supported engine by VDSM. For verification reasons, I'm moving this one to MODIFIED, to be verified once 3.4.4 is released. Note that it can be verified only once 3.4.4 is out. (In reply to Oved Ourfali from comment #18) > The reason it works on 3.4 and not 3.5, is because the engine is 3.5, and > vdsm is 3.4, so the supportedEngines entry in VDSM doesn't contain 3.5, thus > we check for the package version. In 3.4 it isn't checked because 3.4 is a > supported engine by VDSM. aah, now it makes sense. Thanks for clarification! > For verification reasons, I'm moving this one to MODIFIED, to be verified > once 3.4.4 is released. yes. it's *not* in RHEV for Power build 36, that is based on 3.4.3 > yes. it's *not* in RHEV for Power build 36, that is based on 3.4.3
how can we understand the relation between RHEV for Power builds and vdsm version (without comparing rpm packages inside power build)?
the rpm versions are the same ok, vt9 with vdsm 4.14.18-1 (https://bugzilla.redhat.com/show_bug.cgi?id=1152448#c4) rhev 3.5.0 was released. closing. |
Created attachment 945662 [details] engine.log, vdsm.log Description of problem: << 3.5 engine part of the issue - <UNKNOWN> in event, cannot add into rhevm 3.5 , engine checks not only for 'supportedENGINEs but also for packages2 list for vdsm >> RHEV for IBM POWER release 3.4 build 35 service (pkvm2_1) can't be added to engine,... Host ibm-p8-rhevm-hv-01.lab.bos.redhat.com is installed with VDSM version (<UNKNOWN>) and cannot join cluster ppc64 which is compatible with VDSM versions [4.13, 4.14, 4.9, 4.11, 4.15, 4.12, 4.10]. ... 2014-10-10 16:16:06,088 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) [5139c54f] Correlation ID: 5139c54f, Job ID: 7827e052-c40b-44e2-9d50-e7939b52d326, Call Stack: null, Custom Event ID: -1, Message: Host ibm-p8-rhevm-hv-01.lab.bos.redhat.com is installed with VDSM version (<UNKNOWN>) and cannot join cluster ppc64 which is compatible with VDSM versions [4.13, 4.14, 4.9, 4.11, 4.15, 4.12, 4.10]. seems to be related to getos()... ... if getos() in (OSName.RHEVH, OSName.OVIRT, OSName.FEDORA, OSName.RHEL): KEY_PACKAGES = {'qemu-kvm': ('qemu-kvm', 'qemu-kvm-rhev'), 'qemu-img': ('qemu-img', 'qemu-img-rhev'), 'vdsm': ('vdsm',), 'spice-server': ('spice-server',), 'libvirt': ('libvirt', 'libvirt-daemon-kvm'), 'mom': ('mom',), } ... operatingSystem = {'name': 'PowerKVM', 'release': '3.4.2.35.0.pkvm2_1', 'version': '19_0'} packages2 = {'kernel': {'buildtime': 1412189087.0, 'release': '1709.pkvm2_1.12.ppc64', 'version': '3.10.23'}} workaround: --- /tmp/caps.py.orig 2014-10-10 10:27:47.976434065 -0400 +++ /usr/share/vdsm/caps.py 2014-10-10 10:26:32.235548944 -0400 @@ -443,7 +443,7 @@ def _getKeyPackages(): pkgs = {'kernel': kernelDict()} - if getos() in (OSName.RHEVH, OSName.OVIRT, OSName.FEDORA, OSName.RHEL): + if getos() in (OSName.RHEVH, OSName.OVIRT, OSName.FEDORA, OSName.RHEL, OSName.POWERKVM): KEY_PACKAGES = {'qemu-kvm': ('qemu-kvm', 'qemu-kvm-rhev'), 'qemu-img': ('qemu-img', 'qemu-img-rhev'), 'vdsm': ('vdsm',), with patch: ... packages2 = {'kernel': {'buildtime': 1412189087.0, 'release': '1709.pkvm2_1.12.ppc64', 'version': '3.10.23'}, 'libvirt': {'buildtime': 1412717569, 'release': '1.pkvm2_1.17.9', 'version': '1.1.3'}, 'mom': {'buildtime': 1405446678, 'release': '1.pkvm2_1.1', 'version': '0.4.1'}, 'qemu-img': {'buildtime': 1412195719, 'release': '2.pkvm2_1.17.9', 'version': '1.6.0'}, 'qemu-kvm': {'buildtime': 1412195719, 'release': '2.pkvm2_1.17.9', 'version': '1.6.0'}, 'vdsm': {'buildtime': 1412188587, 'release': '1.pkvm2_1', 'version': '4.14.17'}} reservedMem = '321' rngSources = ['hwrng', 'random'] software_revision = '1' software_version = '4.14' supportedENGINEs = ['3.0', '3.1', '3.2', '3.3', '3.4'] supportedProtocols = ['2.2', '2.3'] supportedRHEVMs = ['3.0'] uuid = '2116A5A' version_name = 'Snow Man' vlans = {} vmTypes = ['kvm'] Version-Release number of selected component (if applicable): vdsm-4.14.17-1.pkvm2_1.ppc64 How reproducible: 100% Steps to Reproduce: 1. add _new_ powerkvm for 3.4 into 3.5 (3.4 clstr obviously) 2. 3. Actual results: fail Expected results: should work Additional info: mskrivanek@ said it's two issues: vdsm reporting and engine checks