Bug 1567684 - rebase Beaker's fork of lshw
Summary: rebase Beaker's fork of lshw
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Community
Component: general
Version: 25
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: 25.5
Assignee: Dan Callaghan
QA Contact: Matt Tyson 🤬
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-16 00:26 UTC by Dan Callaghan
Modified: 2018-07-23 06:40 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-07-23 06:40:56 UTC


Attachments (Terms of Use)

Description Dan Callaghan 2018-04-16 00:26:26 UTC
Currently in the Beaker harness repos we are shipping:

lshw-B.02.18-0.2.20150722svn2580.beaker.5

which is an svn snapshot of lshw from several years ago, with some patches written by us that were not (yet) accepted upstream [1]. However RHEL7 now has:

lshw-B.02.18-12.el7

which is a newer NVR. We need to revisit all of the custom Beaker patches in our fork and compare against latest upstream *and* the RHEL7 package (which is also carrying some patches) to determine if any patches are still needed, and if so attempt again to submit them upstream and/or rebase our fork on top of the latest version.

If the RHEL7 version has everything we need, then we can simply stop maintaining our fork. Otherwise, the end result should be a build of lshw with a higher NVR than RHEL7 shipping in our harness repos.

[1] https://github.com/lyonel/lshw/compare/master...beaker-project:master

Comment 1 Dan Callaghan 2018-04-16 00:27:15 UTC
Note that this is somewhat important, because the higher NVR of the RHEL7 package means that it will be installed in favour of our forked lshw package. That means we have potentially regressed in those things which we were carrying patches for.

Comment 2 Dan Callaghan 2018-04-30 23:03:29 UTC
As part of fixing this, you would want to also figure out why lshw-tests is currently broken and/or what updates are needed when rebasing lshw.

https://github.com/beaker-project/lshw-tests

https://beaker-jenkins-rhel7.rhev-ci-vms.eng.rdu2.redhat.com/job/lshw-tests/

Comment 3 Dan Callaghan 2018-06-22 05:42:12 UTC
Okay, so it turns out the lshw RHEL maintainer Petr Oros posted a Github PR with all our Beaker patches:

https://github.com/lyonel/lshw/pull/31

which got squash-merged by Lyonel back in September 2017:

https://ezix.org/src/pkg/lshw/commit/f95aa917a84a8ee74ce79e9b4f9e198d21a2e4d9

although it is not in any release yet.

Comment 4 Dan Callaghan 2018-07-11 00:45:42 UTC
Just for posterity I have pushed a new branch "beaker-2015-fork" for lshw and lshw-tests, to capture the current state before I rebase onto the latest upstream master.

https://github.com/beaker-project/lshw/compare/beaker-2015-fork-starting-point...beaker-2015-fork

https://github.com/beaker-project/lshw-tests/compare/beaker-2015-fork-starting-point...beaker-2015-fork

This shows the history of our contributions which were squash-merged (without proper attribution) in upstream commit f95aa917a84a8ee74ce79e9b4f9e198d21a2e4d9.

Comment 5 Dan Callaghan 2018-07-11 03:44:18 UTC
I've updated the master branch of lshw-tests to match current lshw master. The lshw-tests job in our Jenkins now runs this (lshw master against lshw-tests master) and it is now passing.

In working through all the rebases/test changes I only found one regression which would affect beaker-system-scan:

https://github.com/beaker-project/lshw-tests/commit/850e709ac3e37f08f832334f2ce319311a70f9f7

I've posted patches for lshw to address that, and two other minor issues I noticed as well:

https://ezix.org/src/pkg/lshw/pulls/18
Avoid very long IDE programming interface names as capabilities

https://ezix.org/src/pkg/lshw/pulls/19
Set powerpc logo hint

https://ezix.org/src/pkg/lshw/pulls/20
Fix DIMM info for older IBM POWER systems

Comment 6 Dan Callaghan 2018-07-11 03:58:00 UTC
Oh, I forgot to mention the other very unfortunate, substantial difference I noticed. When Lyonel merged the Beaker patches he renamed <subsysvendor> to <subvendor>, and <subsysproduct> to <subproduct>.

https://github.com/beaker-project/lshw-tests/commit/e968fcedac14f91b934d111fe2b5dfcfb389804d

We will have to adjust beaker-system-scan to handle both variations.

Comment 7 Dan Callaghan 2018-07-11 04:08:35 UTC
As far as RHEL packages go...

RHEL5 never shipped lshw. RHEL6.9 added lshw but a rather old version which not received any of the recent patches. So we will need to continue building our own version for RHEL5 and RHEL6.

RHEL7 ships lshw and it is very heavily patched (74 patches in the spec file). I tried diffing the patched sources from the RHEL7.6 package against upstream master. As far as I can see, it is almost in sync except for a few things:

* RHEL7 is lacking some of the latest fixes on upstream master (fine)
* RHEL7 added a patch for vendor_id which we ourselves abandoned (bug 1212284, bug 1367912)
* RHEL7 still has our original <subsysvendor/> and <subsysproduct/> element names, rather than Lyonel's adjust <subvendor/> and <subproduct/>

The RHEL7 package also seems to have picked up the regression for older IBM POWER memory info.

Comment 8 Dan Callaghan 2018-07-11 04:26:53 UTC
I've updated the master branch of our lshw fork at https://github.com/beaker-project/lshw to be upstream master + the three patches mentioned in comment 5. And I've updated the corresponding beaker branch of lshw-tests to match it.

I added a new job lshw-tests-beaker-fork job to our Jenkins, which will run this combination (beaker-project/lshw fork against lshw-tests beaker branch) and it is now passing too.

Comment 9 Dan Callaghan 2018-07-11 05:58:22 UTC
(In reply to Dan Callaghan from comment #6)
> Oh, I forgot to mention the other very unfortunate, substantial difference I
> noticed. When Lyonel merged the Beaker patches he renamed <subsysvendor> to
> <subvendor>, and <subsysproduct> to <subproduct>.
> 
> https://github.com/beaker-project/lshw-tests/commit/
> e968fcedac14f91b934d111fe2b5dfcfb389804d
> 
> We will have to adjust beaker-system-scan to handle both variations.

I just realised we don't have to change anything in beaker-system-scan. It's already looking at the <hint name="pci.subdevice"/> and <hint name="pci.subvendor"/> elements (not <subsysvendor/> and <subsysproduct/>) and those have not changed names.

Comment 10 Dan Callaghan 2018-07-12 04:12:19 UTC
I have built:

lshw-B.02.19-0.1.20180614git028f6b2.beaker.1.el5bkr
lshw-B.02.19-0.1.20180614git028f6b2.beaker.1.el6bkr
lshw-B.02.19-0.1.20180614git028f6b2.beaker.1.el7bkr
lshw-B.02.19-0.1.20180614git028f6b2.beaker.1.fc27
lshw-B.02.19-0.1.20180614git028f6b2.beaker.1.fc28
lshw-B.02.19-0.1.20180614git028f6b2.beaker.1.fc29

which is the same as the current beaker-project/lshw fork master branch (that is, upstream master plus the same three patches mentioned above). These packages are available in the testing repos:

https://beaker-project.org/yum/harness-testing/

Comment 12 Dan Callaghan 2018-07-19 00:31:29 UTC
For verifying this bug, I would suggest to just run a hardware scan (click the button on the Details Tab of the system page) for every working piece of hardware we have in beaker-devel. Check that the job completes successfully, and then examine the system activity logs to see what has changed. We would expect the hardware scan to produce basically *no* changes over what was there previously (there are no improvements relevant to Beaker in latest lshw master that I could see, but there should also be no regressions).

Just note that the way we record devices in the system activity records is a bit strange. Even a slight tweak in wording of the device name (which can be expected, as the PCIID database gets name tweaks occasionally) will produce a useless activity entry saying device 123 was removed and device 456 was added. In those cases I would suggest to just review the resulting Devices table on the Details tab and just check it for sanity.

Ultimately, it's actually the lshw-tests which give us confidence that we are still producing accurate hardware details -- verifying this bz in Beaker is more just a sanity test to make sure nothing unexpectedly breaks.

Comment 13 Matt Tyson 🤬 2018-07-19 03:57:28 UTC
It looks like beaker is mistakenly recording disk hardware changes.  I've filed Bug 1603009 to track this.  I think this is a Beaker issue and not a lshw issue.

A few systems have reported new CPU flags which look to be spectre / meltdown related. It's also reporting new kernel modules.

There's device ID changes for some machines, but the device hardware page looks sane.

I'll mark this as VERIFIED.

Comment 14 Roman Joost 2018-07-23 06:40:56 UTC
Released with Beaker 25.5: https://beaker-project.org/docs/whats-new/release-25.html#beaker-25-5


Note You need to log in before you can comment on or make changes to this bug.