Bug 2010740 - after enabling third-party repos, the included apps can't be found (except Chrome)
Summary: after enabling third-party repos, the included apps can't be found (except Ch...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: gnome-software
Version: 35
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Milan Crha
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker
Depends On:
Blocks: F35FinalBlocker
TreeView+ depends on / blocked
 
Reported: 2021-10-05 12:38 UTC by Kamil Páral
Modified: 2021-10-21 18:49 UTC (History)
8 users (show)

Fixed In Version: gnome-software-41.0-3.fc35
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-10-21 18:49:26 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
searching for 3rd party apps in gnome-software (video) (5.43 MB, video/webm)
2021-10-05 12:39 UTC, Kamil Páral
no flags Details
rpm-qa output (58.73 KB, text/plain)
2021-10-05 12:40 UTC, Kamil Páral
no flags Details
gnome-software --verbose output (160.04 KB, text/plain)
2021-10-05 12:43 UTC, Kamil Páral
no flags Details


Links
System ID Private Priority Status Summary Last Updated
GNOME Gitlab GNOME gnome-software issues 1486 0 None None None 2021-10-05 17:40:56 UTC

Description Kamil Páral 2021-10-05 12:38:39 UTC
Description of problem:
If I enable all third-party repos in gnome-software (i.e. pycharm, chrome, nvidia and steam repos), only Chrome can be found in a subsequent search. PyCharm, the Nvidia driver and Steam are not found. Reopening gnome-software doesn't help. Rebooting doesn't help. After going to Updates tab and forcing g-s to search for updates, I can then find Nvidia and Steam. PyCharm can't be found, ever. Please see the attached video.

I do understand that the searches are performed locally, not online, and the repo metadata need to be downloaded first. The users don't, though. G-S doesn't inform them that repos need to be synced, doesn't present any progress bar, doesn't offer any button to click. And so it is reasonable to expect that the additional software can be found right after enabling the repos.

Quite interestingly, Chrome can be found immediately. The search delay seems somewhat longer than usual, and I expected that the repos are getting refreshed in the background. But the other apps still show up as non-existent. This doesn't seem to be a race condition, I tried this several times and Chrome could always be found and the others couldn't. I don't know why.

Refreshing the available updates is something that is probably quite a stretch for a general user (how can be updates relevant to app searching, right?) and so I expect general users to simply not being able to find the specified apps until a sufficient time passes and g-s/packagekit decides to refresh the caches (so perhaps even days). This makes a poor experience.


Version-Release number of selected component (if applicable):
gnome-software-41.0-1.fc35.x86_64
PackageKit-1.2.4-2.fc35.x86_64
fedora-workstation-repositories-35-1.fc35.noarch


How reproducible:
always

Steps to Reproduce:
1. install a clean F35 Workstation
2. in gnome-software, go to Software Repositories and enable all four Third Party Repositories
3. search for all third party apps - PyCharm, Google Chrome, Nvidia and Steam. Only Chrome is found.
4. reopen the gnome-software (a naive-user's solution) and search again, no change
5. reboot (a desperate-user's solution) and search again, no change
6. refresh the available updates in the Updates tab (an informed-user's solution), see that Chrome, Nvidia and Steam can be found. PyCharm still can't.

Expected results:
If I enable third party repos, I'm either informed about steps I need to take to make the apps available (or a progress bar to wait for, etc), or everything should happen automagically in the background and I should be able to find the apps when searching for them.

Comment 1 Kamil Páral 2021-10-05 12:39:38 UTC
Created attachment 1829368 [details]
searching for 3rd party apps in gnome-software (video)

Comment 2 Kamil Páral 2021-10-05 12:40:13 UTC
Created attachment 1829369 [details]
rpm-qa output

Comment 3 Kamil Páral 2021-10-05 12:43:22 UTC
Created attachment 1829371 [details]
gnome-software --verbose output

This is the g-s output generated when performing the first 3 reproduction steps.

Comment 4 Kamil Páral 2021-10-05 12:47:48 UTC
Proposing for a blocker discussion. Let's see if we consider "enable an official repo and then find the included app" as a basic functionality:
"All applications that can be launched using the standard graphical mechanism after a default installation of Fedora Workstation on the x86_64 architecture must start successfully and withstand a basic functionality test. "
https://fedoraproject.org/wiki/Fedora_35_Final_Release_Criteria#Default_application_functionality
(or perhaps if the Workstation WG has some opinion on this topic)

Comment 5 Milan Crha 2021-10-05 15:00:33 UTC
Thanks for a bug report. What is the Menu->Update Preferences->Automatic Updates set to, please? It should be checked, which means, when you restart the gnome-software (`gnome-software --quit; gnome-software`, eventually `pkill gnome-software`), you should see the "updating repo data" on the gnome-software start. I'm pretty sure it's checked by default, I'd only like to verify that.

I tried it here, with deleted /var/cache/PackageKit/ when the gnome-software was not running and even with the automatic updates off it had been possible to find the NVidia drivers or Steam, after the PacakgeKit's cache had been updated. That's on f34 with the latest PackageKit and gnome-software though, I do not have f35 machine yet.

Comment 6 Milan Crha 2021-10-05 15:40:03 UTC
Looking around, my f34 has saved /var/cache/app-info/xmls/rpmfusion-nonfree-steam.xml , while the f35 does not have that file at all (there is no file in that directory). The appstream data is used for searching and filling the app details. I do not know what downloads that file, might it be PackageKit or anything else?

Comment 7 Milan Crha 2021-10-05 15:44:20 UTC
I forgot to add, the Chrome is defined in /usr/share/app-info/xmls/google-chrome.xml , which comes from the appstream-data package.

Comment 8 Milan Crha 2021-10-05 16:34:22 UTC
A bit more debugging, my f34 machine has enabled unfiltered Flathub, which also provides Steam, thus what I found here came from the Flathub, not from the .repo file. With the f35 and the file rpmfusion-nonfree-steam.xml added into the /var/cache/app-info/xmls/ I can find the Steam.

As you said in the description, the refresh on the Updates tab adds files to the /var/cache/app-info/xmls/, I've there now: phracek-PyCharm.xml rpmfusion-nonfree-steam.xml rpmfusion-nonfree-nvidia-driver.xml.

Comment 9 Milan Crha 2021-10-05 17:40:57 UTC
Okay, the files in /var/cache/app-info/xmls/ are downloaded by the PackageKit. I can see those added after I run: 

   # pkcon refresh -c 1

which is the equivalent of the gnome-software's:

   pk_client_refresh_cache()

with the cache age also being '1'.

The update monitor (in the gnome-software) calls the refresh once every day. It does not call the refresh on the repository changes.

I opened an upstream bug:
https://gitlab.gnome.org/GNOME/gnome-software/-/issues/1486

Comment 10 Kamil Páral 2021-10-06 06:30:05 UTC
(In reply to Milan Crha from comment #5)
> What is the Menu->Update Preferences->Automatic Updates set to, please?

I'm using a default F35 Workstation install in a VM, so everything is set to default. In this case, automatic updates enabled.

(In reply to Milan Crha from comment #9)
> The update monitor (in the gnome-software) calls the refresh once every day.
> It does not call the refresh on the repository changes.

It sounds like you found the root cause and no more debugging is needed on my side?

Comment 11 Kamil Páral 2021-10-06 06:43:31 UTC
Also, Milan, can you please confirm whether the inability to display PyCharm is caused by a missing appstream metadata, or there could be a different bug related to it?

Comment 12 Milan Crha 2021-10-06 08:11:38 UTC
(In reply to Kamil Páral from comment #10)
> It sounds like you found the root cause and no more debugging is needed on
> my side?

Correct, I provided a proposed patch upstream. Note that the proposed patch (still to be approved) invokes a refresh after the repositories dialog is closed, when there had been done any change in the dialog. This refresh can take some time, depending on the network speed and configured repositories/remotes.

(In reply to Kamil Páral from comment #11)
> Also, Milan, can you please confirm whether the inability to display PyCharm
> is caused by a missing appstream metadata, or there could be a different bug
> related to it?

Checking the content of the /var/cache/app-info/xmls/phracek-PyCharm.xml , where the appstream data is supposed to be, shows the file basically empty, which, I believe, is the reason, why the Software cannot find it. Compare it with /var/cache/app-info/xmls/rpmfusion-nonfree-steam.xml, which contains one <component type="desktop">.

Comment 13 Kamil Páral 2021-10-06 08:44:29 UTC
(In reply to Milan Crha from comment #12)
> Correct, I provided a proposed patch upstream. Note that the proposed patch
> (still to be approved) invokes a refresh after the repositories dialog is
> closed, when there had been done any change in the dialog. This refresh can
> take some time, depending on the network speed and configured
> repositories/remotes.

What will be the user experience? Will the next search take longer, but eventually find the app? Or will the next search still say that e.g. Steam can't be found, but if you try again a minute later, will it then find it? (The first option seems more user-friendly. The best solution would probably include some progress bar and inform the user that "please wait, additional repos are being synchronized").

> Checking the content of the /var/cache/app-info/xmls/phracek-PyCharm.xml ,
> where the appstream data is supposed to be, shows the file basically empty,
> which, I believe, is the reason, why the Software cannot find it. Compare it
> with /var/cache/app-info/xmls/rpmfusion-nonfree-steam.xml, which contains
> one <component type="desktop">.

Thanks. I'll file a fedora-workstation ticket complaining that one of their official repo files doesn't have any appstream metadata. Let's ignore PyCharm in this bug report.

Comment 14 Milan Crha 2021-10-06 09:00:25 UTC
(In reply to Kamil Páral from comment #13)
> What will be the user experience?

The refresh runs in background, there is no indication anywhere. The search made before the refresh is done will not include the app (I understand it's suboptimal).

> I'll file a fedora-workstation ticket...

I'd rather let the PyCharm COPR know, to provide the appstream data. The fedora-workstation won't do much about it. It can be just an oversight on the COPR packaging/side.

Comment 15 Michael Catanzaro 2021-10-06 12:05:51 UTC
(In reply to Milan Crha from comment #12)
> Correct, I provided a proposed patch upstream. Note that the proposed patch
> (still to be approved) invokes a refresh after the repositories dialog is
> closed, when there had been done any change in the dialog. This refresh can
> take some time, depending on the network speed and configured
> repositories/remotes.

What happens if the user enables third-party software using GNOME Software's info bar, then doesn't visit the dialog, or visits the dialog but makes no changes?

Comment 16 Milan Crha 2021-10-06 12:47:56 UTC
(In reply to Michael Catanzaro from comment #15)
> ...then doesn't visit the dialog, or visits the dialog but makes no changes?

No change in the dialog means no need for the refresh.

I missed the infobar part, I added that to the upstream merge request. Together with the reload call after the refresh is done.

Comment 17 Adam Williamson 2021-10-06 17:54:12 UTC
+4 in https://pagure.io/fedora-qa/blocker-review/issue/500 , marking accepted.

Comment 18 Milan Crha 2021-10-07 07:43:14 UTC
The upstream change had not been accepted yet, thus I took its current version and added it to the f35 branch. Any changes from the review will land with the upstream version update.

Comment 19 Fedora Update System 2021-10-07 07:52:24 UTC
FEDORA-2021-db2521bdf4 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-db2521bdf4

Comment 20 Kamil Páral 2021-10-07 12:15:53 UTC
(In reply to Fedora Update System from comment #19)
> FEDORA-2021-db2521bdf4 has been submitted as an update to Fedora 35.
> https://bodhi.fedoraproject.org/updates/FEDORA-2021-db2521bdf4

This solves the problem. After enabling the extra repos, I closed the dialog and search for nvidia. The search took some time (~half a minute, I'd guess), and initially it showed only the old results, but a second afterwards the new results (the nvidia driver entries) appeared. When I searched for Steam after this, it showed up instantly. So the first search seems to be delayed and waiting until the repos get refreshed in the background. The user experience is now much improved compared to the original state.

Comment 21 Fedora Update System 2021-10-07 15:54:41 UTC
FEDORA-2021-db2521bdf4 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-db2521bdf4`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-db2521bdf4

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 22 Fedora Update System 2021-10-07 23:32:02 UTC
FEDORA-2021-db2521bdf4 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 23 Adam Williamson 2021-10-20 20:54:18 UTC
Re-opening this per discussion in https://bugzilla.redhat.com/show_bug.cgi?id=2001837 , re-proposing for blocker vote as the details are slightly different now. Testing so far indicates this works about 50/50. Sometimes when you enable third party repos in g-i-s the packages are visible immediately in g-s, sometimes they are not. Don't think we know what the race or whatever is yet.

Comment 24 Kamil Páral 2021-10-20 21:26:34 UTC
To add more details, the problem is mentioned in bug 2001837 comment 87 and later.

Comment 25 Milan Crha 2021-10-21 06:32:53 UTC
The final upstream patch differs from the one currently used in Fedora. The repo data refresh is part of the repo enable in the new patch, which eliminates the race of "I closed the repos dialog, but the apps showed long after that".

This bug is related to the bug #2001837, but I agree with the notes in the bug #2001837 comment #87, the bug #2001837 is a special case. Enabling repositories with the fedora-third-party should do something with the repository data. The gnome-software has no chance to know that certain repos had been enabled/disabled (I said 'certain', because flatpak backend can detect the change, but PackageKit backend cannot). Let's continue in bug #2001837.

Comment 26 Ben Cotton 2021-10-21 18:35:57 UTC
In today's Go/No-Go meeting, we agreed: This occurs often enough in testing to violate the final criterion: If an initial setup utility is run or intended to be run after the first boot of the installed system, then it must start successfully and each page or panel of the initial setup utility should withstand a basic functionality test.
https://meetbot.fedoraproject.org/fedora-meeting/2021-10-21/f35-final-go_no_go-meeting.2021-10-21-17.00.log.html#l-285

Comment 27 Adam Williamson 2021-10-21 18:49:26 UTC
This bug isn't *really* the same as the problem we're discussing now, so I've filed a new bug:

https://bugzilla.redhat.com/show_bug.cgi?id=2016510

I'm transferring the accepted blocker status there, and re-closing this one.


Note You need to log in before you can comment on or make changes to this bug.