Hide Forgot
Description of problem: The latest update of createrepo now grabs a modules.yaml present in the directory and adds it automatically to the repodata, saving the previously required step of using modifyrepo. If this unzipped modules.yaml file is removed, and createrepo --update is run, the modularity is removed from the repodata. modules.yaml must remain in the repository or createrepo, with or without --update, will remove the *-modules.yaml.gz from the repodata. Version-Release number of selected component (if applicable): createrepo_c-0.16.2-2.el8.x86_64 How reproducible: Everytime Steps to Reproduce: ~~~~~~~~~~~~~~~~ Build repodata with a modules.yaml file ~~~~~~~~~~~~~~~~ [root@r8 ~]# gunzip /var/cache/dnf/rhel-8-for-x86_64-appstream-rpms-9d3886b51bb367d7/repodata/e7eecb4464237e9f35c4c2cbf4c3b502bc0bff1fff6b557715a128782e1ddb05-modules.yaml.gz -c > modules.yaml [root@r8 ~]# ls bash-4.4.20-1.el8_4.x86_64.rpm modules.yaml [root@r8 ~]# createrepo . Directory walk started Directory walk done - 1 packages Temporary output repo path: ./.repodata/ Preparing sqlite DBs Pool started (with 5 workers) Pool finished [root@r8 ~]# ls repodata/ 490c178630c4236c27cba1fae8d2c2adafcb24dc6a6419e63e7e94d4a26a48c7-filelists.xml.gz b9b4df684ee0e2585ef547bd5959877cbd301435096aa9bf384a6f0a3f1886b2-primary.sqlite.bz2 4e74d1470408e76d24d92db559b95915a84ad73bcb299338f64b3a2964b19e6f-primary.xml.gz cd02b2dda28b7438e3d49348903b4c3de47c312f3b4b3d77dd7a8c0be3b0aaf0-filelists.sqlite.bz2 80e02b156ca07d57b082ee3231fbae7f53653931943a4ce73ee8c9914b03b3e0-modules.yaml.gz e3691ee3f913c946f52e680cc4cd76e745354ae2846755cc390a96a715c29845-other.xml.gz a04cce7cacf3365758218c1f497d89b681f57d18e389672383e83583acc336a9-other.sqlite.bz2 repomd.xml ~~~~~~~~~~~~~~~~ remove the original modules.yaml file from the directory and update the repodata ~~~~~~~~~~~~~~~~ [root@r8 ~]# mv modules.yaml /tmp [root@r8 ~]# createrepo --update . Directory walk started Directory walk done - 1 packages type modules added to list from path: ./repodata/80e02b156ca07d57b082ee3231fbae7f53653931943a4ce73ee8c9914b03b3e0-modules.yaml.gz Loaded information about 1 packages Temporary output repo path: ./.repodata/ Preparing sqlite DBs Pool started (with 5 workers) Pool finished ~~~~~~~~~~~~~~~~ *modules.yaml.gz is gone ~~~~~~~~~~~~~~~~ [root@r8 ~]# ls repodata/ 490c178630c4236c27cba1fae8d2c2adafcb24dc6a6419e63e7e94d4a26a48c7-filelists.xml.gz b9b4df684ee0e2585ef547bd5959877cbd301435096aa9bf384a6f0a3f1886b2-primary.sqlite.bz2 repomd.xml 4e74d1470408e76d24d92db559b95915a84ad73bcb299338f64b3a2964b19e6f-primary.xml.gz cd02b2dda28b7438e3d49348903b4c3de47c312f3b4b3d77dd7a8c0be3b0aaf0-filelists.sqlite.bz2 a04cce7cacf3365758218c1f497d89b681f57d18e389672383e83583acc336a9-other.sqlite.bz2 e3691ee3f913c946f52e680cc4cd76e745354ae2846755cc390a96a715c29845-other.xml.gz [root@r8 ~]# grep module repodata/repomd.xml -c 0 ~~~~~~~~~~~~~~~~ Move the unzipped modules.yaml back in and it's included again ~~~~~~~~~~~~~~~~ [root@r8 ~]# mv /tmp/modules.yaml . [root@r8 ~]# createrepo --update . Directory walk started Directory walk done - 1 packages Loaded information about 1 packages Temporary output repo path: ./.repodata/ Preparing sqlite DBs Pool started (with 5 workers) Pool finished [root@r8 ~]# ls repodata/ 490c178630c4236c27cba1fae8d2c2adafcb24dc6a6419e63e7e94d4a26a48c7-filelists.xml.gz b9b4df684ee0e2585ef547bd5959877cbd301435096aa9bf384a6f0a3f1886b2-primary.sqlite.bz2 4e74d1470408e76d24d92db559b95915a84ad73bcb299338f64b3a2964b19e6f-primary.xml.gz cd02b2dda28b7438e3d49348903b4c3de47c312f3b4b3d77dd7a8c0be3b0aaf0-filelists.sqlite.bz2 80e02b156ca07d57b082ee3231fbae7f53653931943a4ce73ee8c9914b03b3e0-modules.yaml.gz e3691ee3f913c946f52e680cc4cd76e745354ae2846755cc390a96a715c29845-other.xml.gz a04cce7cacf3365758218c1f497d89b681f57d18e389672383e83583acc336a9-other.sqlite.bz2 repomd.xml Expected results: createrepo --update should not remove *-modules.yaml.gz from repodata, EVEN IF, the unzipped modules.yaml is gone. Additional info: The ability for createrepo to automatically include the modules.yaml came from Bugzilla 1795936 – [RFE] createrepo_c should be able to handle modules information https://bugzilla.redhat.com/show_bug.cgi?id=1795936
I think this behavior is expected. --update controls whether createrepo_c recalculates metadata from rpm files or if it reuses already present metadata (from primary, filelists..). If you want to preserve the *-modules.yaml.gz during an update even when their source was removed from the repo you can use --keep-all-metadata option. Perhaps we could describe this more explicitly in the man page?
RHEL 7 and below don't have that option. The --update would leave any additional repodata files in place. Can we instead change "--update" to use the --keep-all-metadata option by default? This would match the behaviour customers have used for years and help to avoid more differences in the steps for maintaining repositories of RHEL 5-7 and RHEL 8.
We have discussed it and we most likely don't want to do such a change at this point. Maybe in the future if we ever do a bigger revision and clean up of the createrepo_c options.
I am closing this as NOTABUG because we currently don't have any plans to change this. If there is some new development or some new information please feel free to reopen this bug.
I made an upstream PR that proposes the change of using --keep-all-metadata by default and adds --discard-additional-metadata option to cover the opposite use case. https://github.com/rpm-software-management/createrepo_c/pull/287 Tests update: https://github.com/rpm-software-management/ci-dnf-stack/pull/1037 > RHEL 7 and below don't have that option. The --update would leave any additional repodata files in place. I have tried it and while it does leave the additional repodata in place (in the repodata directory) it removes them from repomd.xml so I think they are technically no longer in the repo. To me it seems closer to the current createrepo_c behavior (with --keep-all-metadata off by default). On the other hand we might want to change this because of modules.yaml.
I ran into this modules.yaml when I tried to build a custom .iso for my client. In short, the .iso would contain some rpm's from RHEL 8, and the additional rpm's we are providing as part of our product. The customer has air-gapped their computers for security reasons and can't use the over-the-internet method of installation. As a work-around, I provide a custom entry in /etc/yum.repos.d that lets them loop mount the RedHat 8.5 iso and update from that. Still, if you advertise that createrepo builds a repo, it should also build any needed yaml files. Or perhaps, is there a way I can get past this that I am unaware. Thank You Cal Page
So, the client wants me to DO SOMETHING, a work-around, whatever. Trouble is, the modules.yaml has what looks like an encrypted front end name. I suspect this is part of your licensing system. So, if my customer is asking me to HACK your licensing system, I'd have to say no. I'm a White Hack hacker ONLY. So, can you please suggest a way I can move past this in accordance with your licensing system. Best Regards, Cal Page
(In reply to Cal Page from comment #10) > Trouble is, the modules.yaml has what looks like an encrypted front end > name. I suspect this is part of your licensing system. I am not sure what you mean here but I don't think there should be any encryption or licensing problems. If you are referring to a name like: 3cc6e3f2bb6f513d28f0d50dd151a3ae4641af22e5963d193b4e73dc572378c0-modules.yaml.gz that is just a checksum + modules.yaml + compression suffix. > So, can you please suggest a way I can move past this If you are using modular rpms from some RHEL 8 repo you should take the modules.yaml from that repo as well. In general createrepo_c cannot generate the modules.yaml file because it doesn't know how the user wants to arrange to modules (just like createrepo_c cannot generate groups.xml because it doesn't know what should each group look like and contain). If you want to create your own modules we have some tools here: https://github.com/rpm-software-management/modulemd-tools However unless I am misunderstanding all of this is not related to this bug (which is about changing default of --keep-all-metadata parameter).
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (createrepo_c bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:1866