Bug 1794647
| Summary: | [RFE] Support POWER kickstart tree | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Bryan Kearney <bkearney> |
| Component: | Provisioning | Assignee: | Lukas Zapletal <lzap> |
| Status: | CLOSED WORKSFORME | QA Contact: | Roman Plevka <rplevka> |
| Severity: | low | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.4 | CC: | bmbouter, briasmit, dkliban, egolov, ggainey, inecas, ipanova, ltran, lzap, rchan, rplevka, sshtein, ttereshc |
| Target Milestone: | Unspecified | Keywords: | FutureFeature, Triaged |
| Target Release: | Unused | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | foreman-2.0.0 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-04-07 11:32:30 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Bryan Kearney
2020-01-24 08:49:47 UTC
Created from redmine issue https://projects.theforeman.org/issues/28846 Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/28846 has been resolved. FAILED QA on sat6.8.0-9 I can't make this work. I sysnced the PPCle kickstart repos and created a new host record for PPC arch. the orchestration seems to only deploy empty files: [root@dhcp-2-25 boot]# pwd /var/lib/tftpboot/boot [root@dhcp-2-25 boot]# ll total 63148 drwxr-xr-x. 2 foreman-proxy root 139 Jul 28 12:55 fdi-image -rw-r--r--. 1 foreman-proxy foreman-proxy 57894232 Jul 28 07:38 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-1-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 6762800 Jul 28 07:38 red-hat-enterprise-linux-7-server-kickstart-x86_64-7-8-1-vmlinuz -rw-r--r--. 1 foreman-proxy foreman-proxy 0 Jul 29 10:28 red-hat-enterprise-linux-8-for-power-little-endian-baseos-beta-kickstart-4-initrd.img -rw-r--r--. 1 foreman-proxy foreman-proxy 0 Jul 29 10:28 red-hat-enterprise-linux-8-for-power-little-endian-baseos-beta-kickstart-4-vmlinuz I thought this may be the deferred-download thing so I resynced the repo with "immediate" download policy. I don't have a ppc machine handy, but i used x86_64 machine just for the sake of picking these files up via pxe boot. Still no luck - the files stay empty. @lzap, am i doing anything wrong or is this a valid bug? Roman, can you please paste the proxy.log lines where those two files gets downloaded? I am interested in the URLs. Can you try to reach those URLs? Are they 404? Can you tell if the files or paths are different for RHEL 8? Some platforms have different paths and RHEL8 might changed it. Hello, this turns out to be bug in pulp perhaps. I can see that RHEL8 PPC64LE KS is synced properly however kickstart (PXE) files are missing: /content/beta/rhel8/8/ppc64le/baseos/kickstart/images is empty as well as /content/beta/rhel8/8/ppc64le/baseos/kickstart/images/ppc/ppc64. For the record this is the treeinfo file: [checksums] images/boot.iso = sha256:a407c97fd193c66c921fd4be13b6a3f1f123c1e43e083579df5cbd90a28cb20b images/install.img = sha256:f2ebf358b516d3e17b0f64a8884c85d3f048c7d9e7bb35dd591ac10953019e2e ppc/ppc64/initrd.img = sha256:badab8b80aa1bdeadbc6c45401207a1923df6da780067f6e8cfedeeebad0e7d7 ppc/ppc64/vmlinuz = sha256:d597d603b240e3a4077ee77af358a1f1c6044c2302be82c5dd902eb5101004a3 [general] ; WARNING.0 = This section provides compatibility with pre-productmd treeinfos. ; WARNING.1 = Read productmd documentation for details about new format. arch = ppc64le family = Red Hat Enterprise Linux name = Red Hat Enterprise Linux 8.3 packagedir = Packages platforms = ppc64le repository = . timestamp = 1593612907 variant = BaseOS variants = BaseOS version = 8.3 [header] type = productmd.treeinfo version = 1.2 [images-ppc64le] boot.iso = images/boot.iso initrd = ppc/ppc64/initrd.img kernel = ppc/ppc64/vmlinuz [release] name = Red Hat Enterprise Linux short = RHEL version = 8.3 [stage2] mainimage = images/install.img [tree] arch = ppc64le build_timestamp = 1593612907 platforms = ppc64le variants = BaseOS [variant-BaseOS] id = BaseOS name = BaseOS packages = Packages repository = . type = variant uid = BaseOS Upstream bug assigned to lzap I agree this sounds like a Pulp bug. Could we get an upstream Pulp issue filed with reproducer steps including a kickstart we could use for testing? Before we do that, Roman can you do one thing? I remember that there was a problem with OnDemand sync, can you switch this to Immediate? Do we even have this option? BZ does not let me to link the issue, how do I do it? https://pulp.plan.io/issues/7303 The Pulp upstream bug status is at NEW. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. (In reply to Lukas Zapletal from comment #13) > Before we do that, Roman can you do one thing? I remember that there was a > problem with OnDemand sync, can you switch this to Immediate? Do we even > have this option? I already tried that with no luck I am working with Centos8 kickstart currently, and here's what I see on Pulp2, 2-master for pulp and pulp_rpm: #!/bin/bash -v BASE='centos8-baseos-ks' REMOTE1='http://centos.mirror.rafal.ca/8.2.2004/BaseOS/ppc64le/kickstart/' pulp-admin rpm repo create --serve-http=true --repo-id=$BASE --relative-url=$BASE --feed=$REMOTE1 --download-policy immediate pulp-admin rpm repo sync run --repo-id=$BASE +----------------------------------------------------------------------+ Synchronizing Repository [centos8-baseos-ks] +----------------------------------------------------------------------+ blahblahblah ... completed Task Succeeded If I look at the published trees I see the kickstart files where I would expect them: [root@pulp2 yum]# tree | grep -v rpm . ├── http │ └── repos │ ├── centos8-baseos-ks -> /var/lib/pulp/published/yum/master/yum_distributor/centos8-baseos-ks/1598980046.23 │ └── listing ├── https │ └── repos │ ├── centos8-baseos-ks -> /var/lib/pulp/published/yum/master/yum_distributor/centos8-baseos-ks/1598980046.23 │ └── listing └── master └── yum_distributor └── centos8-baseos-ks └── 1598980046.23 ├── images │ ├── boot.iso -> /var/lib/pulp/content/units/distribution/7b/27380410b6008e50444acb8fdd69e351b7cbd4e4f398fa410457f39157e0b9/images/boot.iso │ └── install.img -> /var/lib/pulp/content/units/distribution/7b/27380410b6008e50444acb8fdd69e351b7cbd4e4f398fa410457f39157e0b9/images/install.img ├── Packages │ ├── a │ ... │ ├── z ├── ppc │ └── ppc64 │ ├── initrd.img -> /var/lib/pulp/content/units/distribution/7b/27380410b6008e50444acb8fdd69e351b7cbd4e4f398fa410457f39157e0b9/ppc/ppc64/initrd.img │ └── vmlinuz -> /var/lib/pulp/content/units/distribution/7b/27380410b6008e50444acb8fdd69e351b7cbd4e4f398fa410457f39157e0b9/ppc/ppc64/vmlinuz └── repodata ├── 4bce5e49d718a0f08800330740a8b6ae60e72f494ee4a096793abbdefffc93aa-filelists.xml.gz ├── 549bdf78ee50c7c8d91304bdde84ac8d9e76d0b83fc66353725d4aee2b673e09-comps.xml ├── 7b91eeafeffd01bd9f7cad8a64a2f1609ecfed9c6697ec9113b8d84c2949485f-updateinfo.xml.gz ├── a99296cc5691e272ddc74415f3341891330300d001f6bba57e1bedff23b695e9-primary.xml.gz ├── d797275dc5994c926ade087a3d435c23d5eaf52e96924ab0b7d3e91c6347d907-other.xml.gz └── repomd.xml 1 directories, 2528 files [root@pulp2 yum]# And when I try to access them, they get delivered: [root@pulp2 yum]# wget --no-check-certificate https://pulp2.dev/pulp/repos/centos8-baseos-ks/images/install.img Saving to: ‘install.img’ 100%[================================================================>] 527,413,248 451MB/s in 1.1s 2020-09-01 17:19:44 (451 MB/s) - ‘install.img’ saved [527413248/527413248] [root@pulp2 yum]# wget --no-check-certificate http://pulp2.dev/pulp/repos/centos8-baseos-ks/images/install.img Saving to: ‘install.img.1’ 100%[================================================================>] 527,413,248 768MB/s in 0.7s 2020-09-01 17:19:48 (768 MB/s) - ‘install.img.1’ saved [527413248/527413248] [vagrant@pulp2 1794647] I am going to reset and retry with a RHEL8 kickstart repo, assuming I can find/gain access to such a thing - but so far, this looks ok from Pulp2's side. @lzap, @rplevka, am I missing something here? Pulp2 directory structure looks fine (i.e., "matches what I expect from looking at the treeinfo file") after synching RHEL8-baseos-kickstart repo from https://cdn.redhat.com/content/dist/rhel8/8.2/ppc64le/baseos/kickstart/ as well: #!/bin/bash -v BASE='rhel8-baseos-ks' REMOTE1=https://cdn.redhat.com/content/dist/rhel8/8.2/ppc64le/baseos/kickstart/ pulp-admin rpm repo create --serve-http=true --repo-id=$BASE --relative-url=$BASE --feed=$REMOTE1 --download-policy immediate --feed-key ./cdn.pem --feed-cert ./cdn.pem --feed-ca-cert ./redhat-uep.pem pulp-admin rpm repo sync run --repo-id=$BASE [root@pulp2 yum]# pwd /var/www/pub/yum [root@pulp2 yum]# tree . | grep -v rpm . ├── http │ └── repos │ ├── centos8-baseos-ks -> /var/lib/pulp/published/yum/master/yum_distributor/centos8-baseos-ks/1598980046.23 │ ├── listing │ └── rhel8-baseos-ks -> /var/lib/pulp/published/yum/master/yum_distributor/rhel8-baseos-ks/1598986776.13 ├── https │ └── repos │ ├── centos8-baseos-ks -> /var/lib/pulp/published/yum/master/yum_distributor/centos8-baseos-ks/1598980046.23 │ ├── listing │ └── rhel8-baseos-ks -> /var/lib/pulp/published/yum/master/yum_distributor/rhel8-baseos-ks/1598986776.13 └── master └── yum_distributor ├── centos8-baseos-ks │ └── 1598980046.23 │ ... └── rhel8-baseos-ks └── 1598986776.13 ├── images │ ├── boot.iso -> /var/lib/pulp/content/units/distribution/51/695f7a47d2bb8f43cfee8eb12a711e735c5093c52b7260e105f764cbf13e67/images/boot.iso │ └── install.img -> /var/lib/pulp/content/units/distribution/51/695f7a47d2bb8f43cfee8eb12a711e735c5093c52b7260e105f764cbf13e67/images/install.img ├── install.img ├── Packages │ ├── a ... │ ├── z ├── ppc │ └── ppc64 │ ├── initrd.img -> /var/lib/pulp/content/units/distribution/51/695f7a47d2bb8f43cfee8eb12a711e735c5093c52b7260e105f764cbf13e67/ppc/ppc64/initrd.img │ └── vmlinuz -> /var/lib/pulp/content/units/distribution/51/695f7a47d2bb8f43cfee8eb12a711e735c5093c52b7260e105f764cbf13e67/ppc/ppc64/vmlinuz ├── repodata │ ├── 3ea0cdd6c686aac3c2c3862b81ea7d9fa9a9d1e77a8c411ea4eb9f5a9dfc4991-filelists.xml.gz │ ├── 4df08de286551a837c0da1088bda7f4c6c08bd7745ef2bf5173cbbf7a0a5f2fb-other.xml.gz │ ├── 9b28aa4f2e4d9e018e13997182e1dd6a19f714f943605245cd45cd6e4c1eedc0-comps.xml │ ├── 9cbea1d8ad5317e27521d78f729c68c1909b59f059b1b90f095eac83a657f895-productid.gz │ ├── a41c1ebf9624fbd07f7d8e5b9ba91a3c60ae359d22de36db577333b8e4775abd-updateinfo.xml.gz │ ├── cdf308b88cc57d936afcd37e6da6e70cbd97eef8c55d9628052c6507935f34c3-primary.xml.gz │ └── repomd.xml └── treeinfo -> /var/lib/pulp/content/units/distribution/51/695f7a47d2bb8f43cfee8eb12a711e735c5093c52b7260e105f764cbf13e67/treeinfo 76 directories, 5071 files [root@pulp2 yum]# wget --no-check-certificate http://pulp2.dev/pulp/repos/rhel8-baseos-ks/images/install.img Length: 527704064 (503M) [application/octet-stream] Saving to: ‘install.img’ 100%[================================================================>] 527,704,064 839MB/s in 0.6s 2020-09-01 19:14:55 (839 MB/s) - ‘install.img’ saved [527704064/527704064] [root@pulp2 yum]# wget --no-check-certificate https://pulp2.dev/pulp/repos/rhel8-baseos-ks/images/install.img Length: 527704064 (503M) [application/octet-stream] Saving to: ‘install.img.1’ 100%[================================================================>] 527,704,064 253MB/s in 2.0s 2020-09-01 19:15:04 (253 MB/s) - ‘install.img.1’ saved [527704064/527704064] [root@pulp2 yum]# Unless someone can show that Pulp is putting content in the wrong place, I'm going to say that whatever the problem is here, it isn't a Pulp2 problem. The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug. The entire point of the foreman change (which is what this bug is actually about) is that you can't look for pxeboot files, for ppc64le, under /pxeboot. See https://github.com/theforeman/foreman/pull/7381/files#diff-fd59a92ae681a1fd981d71c04679dd9dR28 from the PR that is attempting to address this bug. Whatever is generating the link for "/content/beta/rhel8/8/ppc64le/baseos/kickstart//images/pxeboot/vmlinuz" is wrong - either the foreman PR we're linked to here isn't on the system-under-test or the fix is incorrect. Pulp is placing the files where PPC64LE wants them to be. Pulp doesn't generate the kickstart profiles, which is where the error is happening. (One note - the ks-files are in ppc/ppc64, but the arch is actually called ppc64le - not sure if that might be why the code in the PR isn't working? because it's looking for arch/ppc64 instead of ppc64le?) Yes, as you wrote the comment I was looking into this and made a patch and tested it on Roman's instance: https://projects.theforeman.org/issues/30763 Apologies for the noise, this turned out to be an issue in Foreman core. Foreman puts "architecture" there, whatever user defines. Roman, update your architecture to be ppc64le so the URL is correct. I have applied the patch on your instance so you can continue testing. The Pulp upstream bug status is at CLOSED - NOTABUG. Updating the external tracker on this bug. Returning this to Provisioning Thanks for attaching the case, this will help PMs to identify which features to implement. We do not currently support other platforms than Intel for provisioning, but some workflows are known to work. For example IBM POWER over BOOTP does work, we do not test it tho. There was a regression recently which was fixed in 6.8: https://projects.theforeman.org/issues/28846 I am going to close this bug, we do not test or support IBM POWER provisioning at the moment, however everything needed for modern IBM POWER (OpenPOWER) systems to boot should be ready in Satellite 6.8. |