Bug 1416905

(In reply to Orion Poplawski from comment #0)
> Description of problem:
> 
> This build:
> https://copr-be.cloud.fedoraproject.org/results/orion/protobuf3/fedora-26-
> x86_64/00503007-ola/
> 
> had the following error:
> 
> DEBUG util.py:435:  Failed to synchronize cache for repo
> 'coprbecloudfedoraprojectorg_results_orion_protobuf3_fedora26x86_64_devel',
> disabling.
> 
> but the build ran.  Unfortunately this meant that it was built with an
> incorrect set of packages (missing the updated one from the copr repo)

Hello, it's true that this error may cause fetching different packages than intended but, in your case, it seems that the actual cause is missing devel repo. 

Do or did you have "Manual createrepo" option enabled in your project so that the devel repo could have been created in the fedora-26-x86_64 chroot?

Otherwise `skip_if_unavailable=1` option is set by mockchain when using `-a` to add a repo. We could maybe somehow override it but that might prove to be quite hard.

I do not have manual createrepo set.  Other builds worked fine.  Perhaps a race condition during rebuild of the repo due to another build finishing?  For reproducible and to ensure proper builds, this really needs to get fixed.

(In reply to Orion Poplawski from comment #2)
> I do not have manual createrepo set.  Other builds worked fine.  Perhaps a
> race condition during rebuild of the repo due to another build finishing? 
> For reproducible and to ensure proper builds, this really needs to get fixed.

Ye, but without the "manual createrepo" set the devel repository does not get created but even so it is _always_ added as additional repository with `skip_if_unavailable=1` and that error "Failed to synchronize cache for repo 'coprbecloudfedoraprojectorg_results_orion_protobuf3_fedora26x86_64_devel'" is expected in that case. To make things clear, could you maybe elaborate more on the "incorrect set of packages" that was used for the build and where the "correct set of packages" can be found?

So the purpose of the protobuf3 copr is to test out new versions of protobuf.  So I build a new version of protobuf3 in the copr, and then build dependent packages in the copr.  Because of the above error, the package "ola" was built using the older protobuf in F26 rather than the updated version in the copr which is what I expected.  This caused me to miss the fact that ola fails to build with the updated protobuf.  I would think that it would be expected to be a hard requirement that packages built in a copr are built with the package set in the copr itself.  I was certainly surprised which is why I filed this bug.

If the necessary workflow is to set the "manual createrepo" flag, and then manually create to repo after each build, this would be quite cumbersome.

(In reply to Orion Poplawski from comment #4)
> So the purpose of the protobuf3 copr is to test out new versions of
> protobuf.  So I build a new version of protobuf3 in the copr, and then build
> dependent packages in the copr.  Because of the above error, the package
> "ola" was built using the older protobuf in F26 rather than the updated
> version in the copr which is what I expected.  This caused me to miss the
> fact that ola fails to build with the updated protobuf.  I would think that
> it would be expected to be a hard requirement that packages built in a copr
> are built with the package set in the copr itself.  I was certainly
> surprised which is why I filed this bug.
> 
> If the necessary workflow is to set the "manual createrepo" flag, and then
> manually create to repo after each build, this would be quite cumbersome.

Okay, that error with synchonizing repo 'coprbecloudfedoraprojectorg_results_orion_protobuf3_fedora26x86_64_devel' had nothing to do with it then. It always appears when manual createrepo is off and is kind of annoying.

In your case, the error needed to be: Failed to synchronize cache for repo 'coprbecloudfedoraprojectorg_results_orion_protobuf3_fedora26x86_64'

(without the "devel" suffix)

'coprbecloudfedoraprojectorg_results_orion_protobuf3_fedora26x86_64' is the actual copr repo that packages are built into.

Looking at the root.log: these packages were used:

DEBUG util.py:435:   protobuf                       x86_64 3.1.0-6.fc26                fedora 731 k
DEBUG util.py:435:   protobuf-compiler              x86_64 3.1.0-6.fc26                fedora 668 k
DEBUG util.py:435:   protobuf-devel                 x86_64 3.1.0-6.fc26                fedora 327 k
DEBUG util.py:435:   protobuf-java                  noarch 3.1.0-6.fc26                fedora 1.2 M
DEBUG util.py:435:   protobuf-python                noarch 3.1.0-2.fc26               

coprbecloudfedoraprojectorg_results_orion_protobuf3_fedora26x86_64

Only protobuf-python was retrieved from the copr repo, all the other protobuf packages were taken from Fedora repos. 

It is possible that the error was caused by repodata not being refreshed by createrepo_c: see https://bugzilla.redhat.com/show_bug.cgi?id=1355720. That should be fixed now.

The packages from the copr should be always used. The repository is local so it should be pretty reliable except for bugs somewhere. I think implementing some kind of fail-safe mechanism for this case would be cool but also cumbersome to implement in the current code.

Ah, thanks for the clarification.  Okay, hopefully will be fixed with the createrepo fix.  I'll keep my eye out it case I see it again.

I think this can be closed now but feel free to reopen with a feature request. Glad we got it sorted out!:)