Bug 1816696

Summary: mirrorlists stopped working
Product: [Fedora] Fedora Reporter: Ralf Corsepius <rc040203>
Component: mockAssignee: Pavel Raiskup <praiskup>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 31CC: copr-team, jdisnard, jkeating, mebrown, msuchy, philip.wyett, praiskup, williams
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: mock-2.2-1.fc32 mock-2.2-1.fc31 mock-2.2-1.fc30 mock-2.3-1.fc32 mock-2.3-1.fc31 mock-2.3-1.el7 mock-2.3-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-24 03:28:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
brief log
none
verbose log none

Description Ralf Corsepius 2020-03-24 14:45:21 UTC
Description of problem:

In /etc/mock/*.cfgs, the mirrorslist option seems to have stopped working 

Version-Release number of selected component (if applicable):
mock-2.1-1.fc31.noarch
dnf-4.2.18-1.fc31.noarch

How reproducible:
Always


Actual results:

# mock -r fedora-31-x86_64-packman --init
...
Errors during downloading metadata for repository 'packman':
  - Curl error (37): Couldn't read a file:// file for file:///usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist [Couldn't open file /usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist]
Error: Failed to download metadata for repo 'packman': Cannot prepare internal mirrorlist: Curl error (37): Couldn't read a file:// file for file:///usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist [Couldn't open file /usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist]


The corresponding /etc/mock/fedora-31-x86_64-packman.cfg  contains this:
..
[packman]
name=packman
mirrorlist=file:///usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist
...

/usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist exists and is readable.

Expected results:
Function.

Additional info:
This is a regression, severely reducing mock's usablity.

Comment 1 Pavel Raiskup 2020-03-24 14:53:49 UTC
Thanks for the report.

Please work-around this for now by disabling bootstrap, `--no-bootstrap-chroot` or
`config_opts['use_bootstrap'] = False`.

Comment 2 Ralf Corsepius 2020-03-24 16:23:02 UTC
Thanks for the workaround. Seems to work for me.

Comment 4 Fedora Update System 2020-04-02 07:48:12 UTC
FEDORA-2020-fba9845e22 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-fba9845e22

Comment 5 Fedora Update System 2020-04-02 07:49:22 UTC
FEDORA-2020-6b7c342fb4 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-6b7c342fb4

Comment 6 Fedora Update System 2020-04-02 07:49:49 UTC
FEDORA-2020-85df0014c1 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-85df0014c1

Comment 7 Fedora Update System 2020-04-03 19:56:11 UTC
FEDORA-2020-fba9845e22 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-fba9845e22`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-fba9845e22

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 8 Fedora Update System 2020-04-03 20:35:11 UTC
FEDORA-2020-85df0014c1 has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-85df0014c1`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-85df0014c1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Fedora Update System 2020-04-03 20:49:40 UTC
FEDORA-2020-6b7c342fb4 has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-6b7c342fb4`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-6b7c342fb4

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Fedora Update System 2020-04-07 05:05:09 UTC
FEDORA-2020-fba9845e22 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 11 Fedora Update System 2020-04-07 13:16:05 UTC
FEDORA-2020-fba9845e22 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 12 Fedora Update System 2020-04-20 22:14:52 UTC
FEDORA-2020-6b7c342fb4 has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 13 Fedora Update System 2020-04-20 22:32:04 UTC
FEDORA-2020-85df0014c1 has been pushed to the Fedora 30 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 14 Ralf Corsepius 2020-04-23 04:11:04 UTC
Reopening: The symptoms have changed, but this bug persists:

# mock -r fedora-31-x86_64-packman --init
...
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/mock/fedora-31-x86_64-bootstrap/root/usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist'

Apparently, mock mixes up host and chroot'ed environment and bogusly tries to read the mirrorlist file from inside the chroot.

Comment 15 Pavel Raiskup 2020-04-23 06:55:32 UTC
> Apparently, mock mixes up host and chroot'ed environment and bogusly tries to
> read the mirrorlist file from inside the chroot.

No, the mirror list shoudl be copied into the _bootstrap_ chroot, because mock
executes dnf/yum from withing the _bootstrap_ (see carefully the path).

IOW, when you drop the '...bootstrap/root' part, is this the right location?

    /usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist

If yes, can you help us to diagnose why it was not available in the bootstrap
chroot?  I probably need full traceback ... also full log with --trace --verbose
could be attached (compressed) here.

Comment 16 Ralf Corsepius 2020-04-23 09:52:14 UTC
(In reply to Pavel Raiskup from comment #15)
> > Apparently, mock mixes up host and chroot'ed environment and bogusly tries to
> > read the mirrorlist file from inside the chroot.
> 
> No, the mirror list shoudl be copied into the _bootstrap_ chroot, because
> mock
> executes dnf/yum from withing the _bootstrap_ (see carefully the path).
> 
> IOW, when you drop the '...bootstrap/root' part, is this the right location?
> 
>     /usr/share/mock/mirrorlists/fedora-31-x86_64-packman.mirrorlist
Yes, this is the location, this file is installed on the host.

> If yes, can you help us to diagnose why it was not available in the bootstrap
> chroot?
I'll try.

>  I probably need full traceback ... also full log with --trace
> --verbose
> could be attached (compressed) here.
I'll do so, soonish.

Comment 17 Ralf Corsepius 2020-04-23 09:54:29 UTC
Created attachment 1681050 [details]
brief log

Log created this way:
# mock -r fedora-31-x86_64-packman --scrub=all
# mock -r fedora-31-x86_64-packman --init > mock1.log 2>&1

Comment 18 Ralf Corsepius 2020-04-23 09:57:02 UTC
Created attachment 1681051 [details]
verbose log

Log created this way:
# mock -r fedora-31-x86_64-packman --scrub=all
# mock -r fedora-31-x86_64-packman --trace --verbose --init > mock2.log 2>&1

Comment 19 Pavel Raiskup 2020-04-23 11:10:52 UTC
Thanks, can you please try the following patch?

```
diff --git a/mock/py/mockbuild/mounts.py b/mock/py/mockbuild/mounts.py
index 8290aa02..f5515cbc 100644
--- a/mock/py/mockbuild/mounts.py
+++ b/mock/py/mockbuild/mounts.py
@@ -99,6 +99,8 @@ class BindMountPoint(MountPoint):
             if os.path.isdir(self.srcpath):
                 util.mkdirIfAbsent(self.bindpath)
             elif not os.path.exists(self.bindpath):
+                normbindpath = os.path.normpath(self.bindpath)
+                util.mkdirIfAbsent(os.path.dirname(normbindpath))
                 util.touch(self.bindpath)
             cmd = ['/bin/mount', '-n']
             if self.recursive:

```

I think we can afford to do this in general, because if we bind-mount
directories we create the whole directory path.  It sounds sane to do
the same for non-directory bind mounts.

Comment 20 Ralf Corsepius 2020-04-23 13:00:38 UTC
Thanks for the patch. 

At first glance, this appears to work.

Comment 21 Fedora Update System 2020-05-22 12:14:18 UTC
FEDORA-2020-f109b06dad has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-f109b06dad

Comment 22 Fedora Update System 2020-05-22 12:14:56 UTC
FEDORA-2020-16720483b5 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-16720483b5

Comment 23 Fedora Update System 2020-05-22 12:15:27 UTC
FEDORA-2020-ec92166d87 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-ec92166d87

Comment 24 Fedora Update System 2020-05-22 12:16:01 UTC
FEDORA-EPEL-2020-9a5f274330 has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9a5f274330

Comment 25 Fedora Update System 2020-05-22 12:16:34 UTC
FEDORA-EPEL-2020-9662bcb8a3 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9662bcb8a3

Comment 26 Fedora Update System 2020-05-23 04:05:44 UTC
FEDORA-2020-16720483b5 has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-16720483b5`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-16720483b5

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 27 Fedora Update System 2020-05-23 04:26:22 UTC
FEDORA-EPEL-2020-9662bcb8a3 has been pushed to the Fedora EPEL 7 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9662bcb8a3

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 28 Fedora Update System 2020-05-23 04:32:32 UTC
FEDORA-2020-ec92166d87 has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-ec92166d87`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-ec92166d87

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 29 Fedora Update System 2020-05-23 04:36:47 UTC
FEDORA-2020-f109b06dad has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-f109b06dad`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-f109b06dad

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 30 Fedora Update System 2020-05-23 04:53:57 UTC
FEDORA-EPEL-2020-9a5f274330 has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-9a5f274330

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 31 Fedora Update System 2020-05-24 03:28:38 UTC
FEDORA-2020-f109b06dad has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 32 Fedora Update System 2020-05-31 03:57:06 UTC
FEDORA-2020-16720483b5 has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 33 Fedora Update System 2020-06-07 18:43:31 UTC
FEDORA-EPEL-2020-9662bcb8a3 has been pushed to the Fedora EPEL 7 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 34 Fedora Update System 2020-06-08 00:44:59 UTC
FEDORA-EPEL-2020-9a5f274330 has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.