Bug 541294

Food for thought:

I think a clean way to do this might be to keep to separate versions of the data. 

 Smolt Version      - <date-0>
 Smolt version      - <date-1>
 User Edited Values - <date-2>

The final system view is simply the latest Smolt version with the User edited values overlaid. There could be more than one set of User edited values (by possibly different people)

Pros:
 - you have a history
 - you can clearly view what was smolted vs human edited

Cons:
 - complexity

I have a system with 24GB of physical memory and in Beaker it showed 65536 as the MEMORY value.

*** Bug 593747 has been marked as a duplicate of this bug. ***

Steve,

Can you look into the memory issues specifically.

The memory size comes from smolt: http://git.fedorahosted.org/git/?p=smolt.git;a=blob;f=client/smolt.py#l1347

It takes MemTotal from /proc/meminfo -- memory available to the kernel, i.e. minus reservations etc.

Perhaps we could look to using lshw instead.  It can produce output in XML and reports each physical memory module installed and its size.  It may do a better job of reporting the CPUs as well.

*** Bug 747495 has been marked as a duplicate of this bug. ***

As highlighted by bug 747495, we should also ensure virt guests with memory ballooning report a sensible value.

I think using lshw makes a lot of sense here. smolt does not currently have a release that correctly supports ppc64 (the next branch does, but it's wrong). Additionally, smolt's current releases can't parse the 3.x version strings and thus think know memory is installed. I don't know why it's dependent on the version string at all, tbh.

Generally speaking, the inventory task does not work at all on ppc64 (tested on both FC17 and RH6.3) -- due to the issues mentioned above. So an alternative tool would be great.

The other big limitation of smolt (at least on RHEL <= 6) is that it talks to hald, which rules out running in a kickstart %post environment.

Changing this bug's description to be about switching to lshw. It should hopefully solve, or allow us to solve, the many other inventory-related bugs we have too.

There are a huge number of "devices" that smolt tracks that lshw does not. They are all devices without device ID's and for the most part are either abstractions, ports with nothing connected, or platform devices. I am not certain how many problems their removal from the inventory tasks output will cause for people, and apologise for any inconvenience, but as far as i'm concerned lshw will still be a huge upgrade.

Acking this as it is definitely able to be done. James has done most of the work already, the only remaining problem is tracking down discrepancies between the output of the existing hal-based scripts and the new lshw-based one, and figuring out whether these discrepancies are acceptable or need to be fixed somehow.

This patch is now here: http://gerrit.beaker-project.org/#/c/3139/

More detail on this migration at https://beaker-project.org/dev/proposals/inventory-lshw-migration.html

I'm also going to split out separate issues (blocked on this one) for aarch64 and ppc64le inventory scanning support.

Separate issues for the new architectures have been created as bug 1128002 (aarch64) and bug 1128004 (ppc64le).

ia64 support was dropped in RHEL6, while x86, ppc, s390 support were all dropped in RHEl7, so I suggest we drop those architectures from the scope of this issue and continue to use smolt on those systems.

That leaves this issue covering migration from smolt to lshw for x86_64, ppc64 and s390x

(In reply to Nick Coghlan from comment #16)

I think using smolt sometimes and lshw other times (effectively two separate branches of beaker-system-scan) is a bad idea. It means that the smolt branch would continue reporting all the bad/wrong/missing data that it does now, but the lshw one would be fixed. So for example, i386 would give different results than x86_64 on the same system.

Also, the ia64 machines and the i386-only ones (we have some) will continue to be supported until RHEL5 ELS ends in 2020, so I don't think we can just forget about fixing all the inventory bugs for those systems.

That makes sense, but that's a big test matrix. I'd like to find a way to break this issue down into smaller pieces that can be delivered incrementally.

So how does this sound:

1. All 7 architectures will be updated to use lshw at the same time (for the reasons you gave)
2. x86_64, ppc64 & s390x will be treated as "tier 1" architectures: any regressions noted with them will be considered as failing QA, and we don't ship the updated beaker-system-scan until they're fixed
3. ia64, i386, ppc, and s390 will be treated as "tier 2" architectures: any minor regressions specific to them will be filed as new bugs, and won't block the release of the new beaker-system-scan version

(In reply to Nick Coghlan from comment #18)
> That makes sense, but that's a big test matrix. I'd like to find a way to
> break this issue down into smaller pieces that can be delivered
> incrementally.
> 
> So how does this sound:
> 
> 1. All 7 architectures will be updated to use lshw at the same time (for the
> reasons you gave)
> 2. x86_64, ppc64 & s390x will be treated as "tier 1" architectures: any
> regressions noted with them will be considered as failing QA, and we don't
> ship the updated beaker-system-scan until they're fixed
> 3. ia64, i386, ppc, and s390 will be treated as "tier 2" architectures: any
> minor regressions specific to them will be filed as new bugs, and won't
> block the release of the new beaker-system-scan version

Sounds good to me.

A heads up that lshw is being added in RHEL7.1: BZ 1101590

(In reply to Evan McNabb from comment #20)
> A heads up that lshw is being added in RHEL7.1: BZ 1101590

Please note that the current released version of lshw has very limited support for aarch64 (among others), we have contributed patches to enhance lshw's functionality on non-x86 archs and they have been accepted upstream - however there is no new release with those fixes yet.

Thanks for the upstream fixes. If a release is made, we can help push to get it included in RHEL.

Taking this bug in order to finish it off, although Amit has done quite a substantial portion of the work.

We had to fix quite a few bugs/RFEs in lshw in order to bring it up to the same level as smolt in some cases. All our lshw are posted upstream. For anyone interested, the lshw patches we are still carrying that are not yet merged upstream can be seen here:

https://github.com/beaker-project/lshw/compare/lyonel:master...master

As part of the development work for porting beaker-system-scan to use lshw, Amit wrote a Beaker task /distribution/inventory-compare which runs both versions of beaker-system-scan on a system and highlights differences in the output.

The source for the task is here:

https://git.beaker-project.org/cgit/beaker-meta-tasks/tree/inventory-compare

We are running that task on as many different representative pieces of hardware as we can, to make sure there are no unacceptable regressions in the new version of beaker-system-scan.

Marking this as VERIFIED based on the comparison tests analyzed in comment 26.

All of the differences noted there are either:
(a) improvements
(b) insignificant, because nobody could reasonably be relying on the old values (for example "OTHER" for type)
or (c) minor regressions which are filed as separate bugs.

Even though it is still not perfect, we are close enough that this can be released now (instead of letting it drag on for any longer than the 3 years it already has!)

Beaker 21.0 has been released.

*** Bug 647470 has been marked as a duplicate of this bug. ***