Bug 1670651

Description of problem:

createrepo_c --keep-all-metadata --update doesn't keep all metadata as the option name suggests.  

Version-Release number of selected component (if applicable):

0.11.1-2.fc29

How reproducible:

100%

Steps to Reproduce:

1. Prepare/obtain a repo with modulemd, as in example:

$ find myrepo/
myrepo/
myrepo/repodata
myrepo/repodata/89ebbf3f135f25a1f600602207318f8b3fc2d0eb0851af5669b47b956abcfffe-primary.xml.gz
myrepo/repodata/4f9bf8da7d86c71fa4f7cb01ffe631ccf3cae6fedb22815beebbc56d30ebd5a8-modules.yaml.gz
myrepo/repodata/a27718cc28ec6d71432e0ef3e6da544b7f9d93f6bb7d0a55aacd592d03144b70-comps.xml
myrepo/repodata/f00d758e99c401ac7d5841770907ef7b3e86e89c8e035a212e693855856d88bf-other.xml.gz
myrepo/repodata/590354d60d7b4be2c1c40f5eaed3ac9209092dc6c94c72f2b9f63aa92f50c745-updateinfo.xml.gz
myrepo/repodata/fdbe8c908a8eb2a680801959112643e35792d39de87730015dd2233716befcee-filelists.xml.gz
myrepo/repodata/repomd.xml
myrepo/Packages
myrepo/Packages/r
myrepo/Packages/r/release-e2e-test-1.0.1640-1.el7.x86_64.rpm


2. Try to add sqlite to the repo with command: createrepo_c -d --update --keep-all-metadata ./myrepo/

3. Observe new contents of repo


Actual results:

sqlite files are added, but modulemd file is lost, as in example:

find myrepo/
myrepo/
myrepo/repodata
myrepo/repodata/9f7e68e69d7e2453caa3865a97f8165c1f34d089010c8eb00e7fef6396a6aafa-comps.xml.gz
myrepo/repodata/61f7528520042e94b2bb02568be957ecc311049a8d91ae71cb361a7cc705b1ec-filelists.sqlite.bz2
myrepo/repodata/06100c7a259671cb36cd84e23954fed4b79be03f626dd703ec0eb6dd56e85cde-filelists.xml.gz
myrepo/repodata/9fcb8e5b757cd06b8f20373d074d4864a10785f7c9ad0f0cf159bc798464c376-primary.xml.gz
myrepo/repodata/a27718cc28ec6d71432e0ef3e6da544b7f9d93f6bb7d0a55aacd592d03144b70-comps.xml
myrepo/repodata/83a86dc28a0ea6758f4abb5e88f4523153dafa9ea6fd8287c3db7e238b2559d4-other.sqlite.bz2
myrepo/repodata/fee28491e9dbeee98e83202de3d79ccb828ddf2b7c893ebeb8ce6cfa035496e3-primary.sqlite.bz2
myrepo/repodata/7bbb9aeda401d03d6c66ef21a3aadc9443273364533d841091db13140eaf4cee-other.xml.gz
myrepo/repodata/590354d60d7b4be2c1c40f5eaed3ac9209092dc6c94c72f2b9f63aa92f50c745-updateinfo.xml.gz
myrepo/repodata/repomd.xml
myrepo/Packages
myrepo/Packages/r
myrepo/Packages/r/release-e2e-test-1.0.1640-1.el7.x86_64.rpm



Expected results:

sqlite files are added, modulemd files are retained.


Additional info:

Current behavior is consistent with the help text which is "Keep groupfile and updateinfo from source repo during update", however the option name suggests keeping *all* metadata and this would be genuinely more useful.

The current behavior impacts Pulp, which attempts to add sqlite to an existing repo like this:

        pipe = subprocess.Popen('createrepo_c -d --update --keep-all-metadata '
                                '--local-sqlite '
                                '-s %(checksum_type)s --skip-stat %(content_dir)s' %
                                {'checksum_type': checksum_type, 'content_dir': self.content_dir},
                                shell=True, stdout=subprocess.PIPE,
                                stderr=subprocess.PIPE)

The Pulp developers apparently took the '--keep-all-metadata' option name at face value and didn't realize this would discard most kinds of metadata from the repo, such as modulemd and productid files.  As a result, enabling publish of sqlite files in Pulp will disable publish of modulemd and productid files.