Bug 1401963 - installed webadmin-portal-debuginfo is not updated by engine-setup and brokes the engine
Summary: installed webadmin-portal-debuginfo is not updated by engine-setup and brokes...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Setup.Engine
Version: 4.0.6.2
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ovirt-4.1.1
: 4.1.1.3
Assignee: Sandro Bonazzola
QA Contact: Lucie Leistnerova
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-12-06 13:11 UTC by Lucie Leistnerova
Modified: 2019-04-28 13:09 UTC (History)
8 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-04-21 09:41:08 UTC
oVirt Team: Integration
Embargoed:
ylavi: ovirt-4.1+
rule-engine: exception+


Attachments (Terms of Use)
error (17.88 KB, image/png)
2016-12-06 13:11 UTC, Lucie Leistnerova
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 73046 0 master MERGED packaging: spec: ensure debug packages alignment 2017-02-27 10:21:42 UTC
oVirt gerrit 73130 0 ovirt-engine-4.1 MERGED packaging: spec: ensure debug packages alignment 2017-02-27 12:57:34 UTC
oVirt gerrit 73131 0 ovirt-engine-4.1.1.z MERGED packaging: spec: ensure debug packages alignment 2017-02-27 13:55:57 UTC

Description Lucie Leistnerova 2016-12-06 13:11:14 UTC
Created attachment 1228476 [details]
error

Description of problem:
I have installed ovirt-engine-webadmin-portal-debuginfo-4.0.6.1-0.1.el7ev.noarch.rpm with engine 4.0.6.1-0.1.el7ev. Then I upgraded to engine 4.0.6.2-0.1.el7ev

# yum update ovirt-engine-setup ovirt-engine-setup-plugin-websocket-proxy
# engine-setup
...

all seemed OK, no unusual errors in log files. After some work with webadmin UI dialog error occurs: java.lang.reflect.InvocationTargetException. Restart or rerun of engine-setup didn't help.
When I upgraded ovirt-engine-webadmin-portal-debuginfo from 4.0.6.1-0.1.el7ev to 4.0.6.2-0.1.el7ev and run engine-setup everything was OK.

How reproducible: with combination ovirt-engine-webadmin-portal-debuginfo-4.0.6.1-0.1.el7ev and engine 4.0.6.2-0.1.el7ev


Actual results: error displayed, see attachment

Expected results: no error, engine-setup should upgrade ovirt-engine-webadmin-portal-debuginfo if installed


Additional info:
If I kill all the dialogs with error I can go along the engine tabs except of VM tab, then the error was back.

Comment 1 Lucie Leistnerova 2016-12-06 13:12:56 UTC
lost in all the versions

Version-Release number of selected component (if applicable):
ovirt-engine-setup-4.0.6.2-0.1.el7ev.noarch

Comment 2 Yedidyah Bar David 2016-12-06 14:12:48 UTC
It should be easy to add to the spec file, for all the debuginfo packages:

Requires:       %{name} = %{version}-%{release}

But perhaps there is a reason why we don't do this. Alexander?

Comment 3 Yedidyah Bar David 2016-12-06 14:16:35 UTC
I see this was already discussed in [1], when adding these packages. Adding Vojtech, not sure who should know best.

[1] https://gerrit.ovirt.org/#/c/36655/

Comment 4 Alexander Wels 2016-12-06 14:20:29 UTC
So the reason we don't install this by default is because the symbol maps are fairly large ~80M or so. If we don't mind this, then we should add the requires.

Comment 5 Yedidyah Bar David 2016-12-06 14:27:52 UTC
(In reply to Alexander Wels from comment #4)
> So the reason we don't install this by default is because the symbol maps
> are fairly large ~80M or so. If we don't mind this, then we should add the
> requires.

No, that wasn't the question. The question was if we need to be able to install the debuginfo packages without the engine. If the debuginfo packages require engine of same version, you can still install the engine without them, but when you do have them, and update the engine, yum should update them too.

BTW, doing this will lead to a new bug - 'yum update' will then fail, unless we versionlock the debuginfo packages. Perhaps we should indeed versionlock them as well. Yuck.

Comment 6 Alexander Wels 2016-12-06 14:37:03 UTC
Yes we should version lock the debug package with the engine. The debug symbols must match the engine version, and no we don't need to install them without the engine, as they as information about the GWT permutations of that specific engine version.

Comment 7 Sandro Bonazzola 2016-12-06 14:44:58 UTC
(In reply to Alexander Wels from comment #6)
> Yes we should version lock the debug package with the engine. The debug
> symbols must match the engine version, and no we don't need to install them
> without the engine, as they as information about the GWT permutations of
> that specific engine version.


Ok, this settle it. Let's add the missing require.

Comment 8 Alexander Wels 2016-12-06 20:17:58 UTC
Just to add a note, I don't think the debuginfo is the source of the exceptions originally reported by this bz. A restart of the engine temporarily makes the exceptions go away but they return shortly after that. I also have an upstream community member reporting the same issue. I have been update to determine the source as of yet.

Comment 9 Jiri Belka 2016-12-07 08:46:02 UTC
Wouldn't be enough to add this package into versionlock.list ? tl;dr the code but doesn't it update all rpms in versionlock.list?

Comment 10 Jiri Belka 2016-12-07 08:47:28 UTC
(In reply to Alexander Wels from comment #8)
> Just to add a note, I don't think the debuginfo is the source of the
> exceptions originally reported by this bz. A restart of the engine
> temporarily makes the exceptions go away but they return shortly after that.
> I also have an upstream community member reporting the same issue. I have
> been update to determine the source as of yet.

I would agree, I don't have debuginfo packages and my Admin Portal shows UI issue mentioned by others.

Comment 11 Lucie Leistnerova 2016-12-07 08:49:08 UTC
Alexander, Jiri... you are right. I came today and the error is back, but I have the newest debuginfo packages. Can't find error anywhere in logs. I will investigate it more.

Still I think that when I have installed debuginfo engine-setup should update it.

Comment 12 Lucie Leistnerova 2016-12-07 09:03:57 UTC
in response of https://myengine/ovirt-engine/webadmin/xsrf my browser says is bad JSON, could that be it?

//OK[2,1,["com.google.gwt.user.client.rpc.XsrfToken/4254043109","CC08CF9D159582AA7B9B53F70281A2D3CE102E3FCD693C3E769474C367752ADC"],0,7]

after that this script is called
https://myengine/ovirt-engine/webadmin/GenericApiGWTService

function L1e$(_0,_1){_0.g=_1;return _0}throw L1e$(new (ELf(Srb)),"java.lang.reflect.InvocationTargetException");

and this two scripts are called when the error appears.

I have one running VM on the engine. When I stop it error don't show.
It happens only in VM tab, VM showed from Host tab is OK.

Comment 13 Michal Skrivanek 2016-12-07 13:54:45 UTC
AFAIU the debuginfo has nothing to do with the problem. Feel free to add the package by default, but it is not needed, so also feel free to just close this bug. 
The exception is being handled in bug 1402401

Comment 14 Yedidyah Bar David 2016-12-07 14:00:44 UTC
(In reply to Michal Skrivanek from comment #13)
> AFAIU the debuginfo has nothing to do with the problem. Feel free to add the
> package by default,

That wasn't the question - the question was whether having non-compatible engine and debuginfo packages can lead to problems and/or should be prevented.

> but it is not needed, so also feel free to just close
> this bug. 
> The exception is being handled in bug 1402401

Thanks. Closing for now. Please reopen if indeed it's a bug.

Comment 15 Alexander Wels 2016-12-07 15:21:24 UTC
Btw note this is a serious problem for several testers, and at least one community member. I believe the problem lies somewhere on the back-end when the UI makes a query, which causes an exception. This exception is then eaten AND NOT logged, which then returns the InvocationTargetException to the UI. The UI thinks hmm some exception occured this usually means my XSRF token is bad, lets get a new one and try again. Which then repeats a bunch of times.

Something in 4.0.6-2 or 3 is causing this, the UI has nothing to do with it.

Comment 16 Yedidyah Bar David 2016-12-07 15:24:04 UTC
(In reply to Alexander Wels from comment #15)
> Btw note this is a serious problem for several testers, and at least one
> community member. I believe the problem lies somewhere on the back-end when
> the UI makes a query, which causes an exception. This exception is then
> eaten AND NOT logged, which then returns the InvocationTargetException to
> the UI. The UI thinks hmm some exception occured this usually means my XSRF
> token is bad, lets get a new one and try again. Which then repeats a bunch
> of times.
> 
> Something in 4.0.6-2 or 3 is causing this, the UI has nothing to do with it.

Sorry, I do not fully understand. So it's a bug, or not? A bug somewhere else that manifests when they differ?

Comment 17 Alexander Wels 2016-12-07 15:28:49 UTC
(In reply to Yedidyah Bar David from comment #16)
> (In reply to Alexander Wels from comment #15)
> > Btw note this is a serious problem for several testers, and at least one
> > community member. I believe the problem lies somewhere on the back-end when
> > the UI makes a query, which causes an exception. This exception is then
> > eaten AND NOT logged, which then returns the InvocationTargetException to
> > the UI. The UI thinks hmm some exception occured this usually means my XSRF
> > token is bad, lets get a new one and try again. Which then repeats a bunch
> > of times.
> > 
> > Something in 4.0.6-2 or 3 is causing this, the UI has nothing to do with it.
> 
> Sorry, I do not fully understand. So it's a bug, or not? A bug somewhere
> else that manifests when they differ?

So I talked to Oved, and this (the debug info causing a problem) is NOT a bug, however BZ1402401 causes the TargetInvocationException to happen, and that is a bug.

Comment 18 Lucie Leistnerova 2016-12-07 15:36:32 UTC
I admit that the problem of the exception was elsewhere. But still I have to upgrade packages of debuginfo manually. So there is no easy way how to check that the package is installed and upgrade it with engine-setup?

Comment 19 Alexander Wels 2016-12-07 15:44:01 UTC
I think there is a way to check, but I don't know the mechanism. Sandro knows though.

Comment 20 Yedidyah Bar David 2016-12-08 06:58:05 UTC
Of course there is a way, but in general we only update packages that must be updated for the engine to work well. Our upgrade instructions do say to run 'yum update' after engine-setup finishes.

Comment 21 Vojtech Szocs 2016-12-08 17:53:23 UTC
Update: the other bug 1402401 (accessing VM tab triggers error dialog) is now MODIFIED.

As for this bug:

> the question was whether having non-compatible engine and debuginfo packages can lead to problems and/or should be prevented

Yes, having non-compatible Engine vs. debuginfo packages *will* lead to problems.

> Thanks. Closing for now. Please reopen if indeed it's a bug.

Re-opening..

We should version-lock debuginfo packages: when those are installed (optional by default), their version *must* be the same as Engine version.

Comment 22 Yedidyah Bar David 2017-02-07 07:58:58 UTC
A similar issue now came up on [1].

Thinking about this, I am not sure what we'll do.

We can easily add to the debuginfo packages:

Requires:       %{name} = %{version}-%{release}

This will (hopefully, didn't check) solve the flow in [1].

It might break 'yum update' later on, if the package is not version-locked.
But since it's not mandatory, nothing will version-lock it if it's installed manually by yum.

So a next engine-setup can version-lock it, which will probably help future yum updates, but not first one after yum install *debuginfo.

[1] http://lists.ovirt.org/pipermail/users/2017-February/079523.html

Comment 23 Sandro Bonazzola 2017-02-24 13:41:09 UTC
about yum update error messages related to version locking:
debug packages are not meant to be installed on any production
environment other than for debugging purpose.
Once the debug is finished they need to be removed.

Comment 24 Vojtech Szocs 2017-03-06 16:44:03 UTC
(In reply to Yedidyah Bar David from comment #22)
> We can easily add to the debuginfo packages:
> 
> Requires:       %{name} = %{version}-%{release}

Given that `debuginfo` pkgs are yum installed manually on demand, I think this is the most correct solution. When a `debuginfo` pkg is yum installed, we must ensure the user gets the right version, so RPM Requires is a good fit here.

> It might break 'yum update' later on, if the package is not version-locked.
> But since it's not mandatory, nothing will version-lock it if it's installed
> manually by yum.

If we're able to detect optional pkgs when `engine-setup` is run, and version-lock them, we can do that. Otherwise, let users handle the `debuginfo` pkgs manually.

Comment 25 Sandro Bonazzola 2017-03-07 08:52:34 UTC
(In reply to vszocs from comment #24)
> (In reply to Yedidyah Bar David from comment #22)
> > We can easily add to the debuginfo packages:
> > 
> > Requires:       %{name} = %{version}-%{release}
> 
> Given that `debuginfo` pkgs are yum installed manually on demand, I think
> this is the most correct solution. When a `debuginfo` pkg is yum installed,
> we must ensure the user gets the right version, so RPM Requires is a good
> fit here.

This is what has been implemented


> 
> > It might break 'yum update' later on, if the package is not version-locked.
> > But since it's not mandatory, nothing will version-lock it if it's installed
> > manually by yum.
> 
> If we're able to detect optional pkgs when `engine-setup` is run, and
> version-lock them, we can do that. Otherwise, let users handle the
> `debuginfo` pkgs manually.

See comment #23

Comment 26 Lucie Leistnerova 2017-03-20 15:02:24 UTC
Debuginfo packages requires appropriate portal package, so 'yum update' fails without running engine-setup first. It works for me together with documentation 4.1 that says that 'yum update' after upgrade should be done.

verified in ovirt-engine-4.1.1.5-0.1.el7.noarch


Note You need to log in before you can comment on or make changes to this bug.