Bug 2074121 - Enabling "Fedora Flathub Selection" repo doesn't show included apps in gnome-software until re-login
Summary: Enabling "Fedora Flathub Selection" repo doesn't show included apps in gnome-...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: gnome-software
Version: 36
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Milan Crha
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker
Depends On:
Blocks: F36FinalBlocker
TreeView+ depends on / blocked
 
Reported: 2022-04-11 15:14 UTC by Kamil Páral
Modified: 2022-04-22 01:19 UTC (History)
5 users (show)

Fixed In Version: gnome-software-42.0-4.fc36
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-04-22 01:19:26 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
system journal (282.72 KB, text/plain)
2022-04-11 15:23 UTC, Kamil Páral
no flags Details
journal with gs from comment 6 (322.75 KB, text/plain)
2022-04-12 11:58 UTC, Kamil Páral
no flags Details
journal with gs from comment 6 including gs restart (350.62 KB, text/plain)
2022-04-12 12:00 UTC, Kamil Páral
no flags Details
journal relevant to comment 15 (live search) (274.01 KB, text/plain)
2022-04-13 14:12 UTC, Kamil Páral
no flags Details
journal relevant to comment 15 (no search ahead, broken search) (280.50 KB, text/plain)
2022-04-13 14:15 UTC, Kamil Páral
no flags Details
journal relevant to comment 17+19 (287.71 KB, text/plain)
2022-04-14 06:27 UTC, Kamil Páral
no flags Details
journal relevant to comment 18 (274.65 KB, text/plain)
2022-04-14 06:33 UTC, Kamil Páral
no flags Details
system journal related to comment 28 (300.20 KB, text/plain)
2022-04-19 14:24 UTC, Kamil Páral
no flags Details
system journal related to comment 35 (375.93 KB, text/plain)
2022-04-20 08:28 UTC, Kamil Páral
no flags Details


Links
System ID Private Priority Status Summary Last Updated
GNOME Gitlab GNOME gnome-software issues 1712 0 None opened Clicking "Fedora Flathub Selection" does not update the filtered Flathub AppStream metadata until reboot 2022-04-11 15:16:50 UTC

Description Kamil Páral 2022-04-11 15:14:44 UTC
Description of problem:
If you enable "Fedora Flathub Selection" repo in gnome-software, the included apps are not searchable in gnome-software until you re-login or reboot. This only affects "Fedora Flathub Selection", all other third-party repos work correctly (if you enable them and search for their apps, gnome-software shows a spinner during repo refresh and then shows you correct search results).

Note: If you enable "Fedora Flathub Selection" during gnome-initial-setup, everything works as expected. This bug is just about manually enabling the repo and searching in it during the same user session.

Version-Release number of selected component (if applicable):
gnome-software-42.0-2.fc36.x86_64
flatpak-libs-1.12.7-1.fc36.x86_64
Fedora-Workstation-Live-x86_64-36-20220410.n.0.iso

How reproducible:
always

Steps to Reproduce:
1. install a clean system, I used Fedora-Workstation-Live-x86_64-36-20220410.n.0.iso
2. in gnome-initial-setup, do NOT enable third party repos
(2b. you can optionally update to latest gnome-software here and reboot, I upgraded to gnome-software-42.0-2.fc36)
3. in your user session, open gnome-software -> software repositories, and enable "Fedora Flathub Selection"
4. search for "Minecraft" in gnome-software, nothing is found
5. log out and back again, search for "Minecraft" in gnome-software, it is found

Comment 1 Kamil Páral 2022-04-11 15:16:51 UTC
Upstream report: https://gitlab.gnome.org/GNOME/gnome-software/-/issues/1712

Proposing as a blocker due to:
> The default graphical package manager for a given software type must appropriately: 
> Configure software sources by enabling/disabling pre-defined official repositories and then adjust the available software pool accordingly
https://fedoraproject.org/wiki/Fedora_36_Final_Release_Criteria#Installing.2C_removing_and_updating_software

(It was also accepted as a blocker in the past in bug 2010740)

Comment 2 Kamil Páral 2022-04-11 15:23:39 UTC
Created attachment 1871832 [details]
system journal

Here's system journal when I cleanly boot the system, go to gnome-software, enable flathub selection, and search for minecraft, then wait some time and search again.

Comment 3 František Zatloukal 2022-04-11 17:10:41 UTC
Discussed during the 2022-04-11 blocker review meeting: [1]

The decision to classify this bug as an AcceptedBlocker was made:

"It violates the following criterion: “The default graphical package manager for a given software type must appropriately… Configure software sources by enabling/disabling pre-defined official repositories and then adjust the available software pool accordingly"."

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2022-04-11/f36-blocker-review.2022-04-11-16.00.log.txt

Comment 4 Milan Crha 2022-04-12 09:44:13 UTC
I see this on my console:

   Gs  failed to get search apps: failed to get permissions: No such ref 'app/com.mojang.Minecraft/x86_64/stable' in remote flathub

when searching for "minec". The "failed to get permissions" is suspicious. Your journal log doesn't show this, it shows a flathub appstream data had been downloaded. This appstream data can be seen at:

   /var/lib/flatpak/appstream/flathub/x86_64/active/appstream.xml

while, when the remote is disabled after Fedora install, there is no:

   /var/lib/flatpak/appstream/flathub/

directory.

When I restart gnome-software the Minecraft can be found. When I disable the Flathub Selection remote in the Software, the "minec" doesn't find anything (as expected) and when I enable it again (still the same Software instance), the Minecraft cannot be found again (even after I give the Software some time to notice the Flatpak changed and it updates its internal data) and there is no runtime warning from the top of this comment.

Comment 5 Kamil Páral 2022-04-12 10:39:21 UTC
Ok, so I experimented more. In my several attempts, it took about 40-80 seconds after enabling the repo before /var/lib/flatpak/appstream/flathub/ was created. During that time, searching for Minecraft returned no results. After the directory appeared, gnome-software was able to find it.
I'm a bit confused here, because in my original attempt, I waited some time and couldn't find it anyway, so either the metadata download speed was very low at that time, or there is some race condition and it sometimes works (running gnome-software picks up the live changes) and sometimes it doesn't.

Also, as you noted, when I disabled the repo, checked that Minecraft was no longer to be found, and then re-enabled the repo, I was able to find Minecraft again after a short while, but only sometimes. In other attempts, I couldn't find it anymore, as you described. I can't be completely sure, but it seems to behave like this:
a) if you wait enough time (1-2 minutes) after enabling the repo, without doing *any search*, then you can find it
b) if you search for it quickly after enabling the repo, you get no results, and you'll always get no results, even after the repo is refreshed (or something) in the background
Is there some search caching, perhaps?

But I'm not sure if this is the same bug or two separate bugs (downloading repo metadata for the first time versus refreshing existing metadata after re-enabling the repo).

Also, a major issue here seems to be that gnome-software doesn't seem to be aware that a flatpak repo refresh is in progress. For packagekit's rpm repos, the spinner is shown until a repo is refreshed, and only then results are displayed. For flatpak repos, it displays search results immediately and doesn't seem to be aware of any refresh.

Comment 6 Milan Crha 2022-04-12 11:00:00 UTC
I've been also debugging this and this build [1] adds some more debug prints into the gnome-software terminal, which can help to narrow some things.

The repo enable is an expensive process, especially when the cache is not already downloaded locally (though even with it the enable can cause full data download). It's not only about downloading 45MB of appstream data locally, and then filter it (it's how I understand the log info, which comes from flatpak/OSTree itself (search your log for "libostree pull from 'flathub' for appstream2/x86_64 complete", where the log claims it took 12 seconds, which is nothing significant)), but there are also other OSTree magics, not talking about about gnome-software and delays in getting file monitor notifications.

Trying it now, when I search for "minec", nothing is found. Then I go to the repositories and enable the filtered flathub, after which, once all the processing of the change in enabling the remote is done, the search page refreshes itself and then finds the Minecraft. It works the same when I disable the remote.

The only thing is that it takes several seconds, even with cached flathub data in /var/lib/flatpak/... .

> Is there some search caching, perhaps?

No search caching is involved here.

> For packagekit's rpm repos, the spinner is shown until a repo is refreshed

Which spinner do you mean, please? Aka where can it be seen, please?

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=85550828

Comment 7 Kamil Páral 2022-04-12 11:32:56 UTC
> Which spinner do you mean, please? Aka where can it be seen, please?

From my previous experience with gnome-software, when I enable a third-party rpm repo and then immediately search for something, I see only a big spinner in the middle of the screen and no search results, until (I assume) the repos are refreshed, and only then search results appear. When I enable a flatpak repo, it immediately shows "no results found" and no spinner. So the user doesn't know that something is happening in the background and that he needs to wait some more.

Comment 8 Milan Crha 2022-04-12 11:39:56 UTC
I see, the spinner on the search page. I just have it spinning after Flatpak remote enable due to the flatpak being stuck in a download:

Thread 18 (Thread 0x7ff4b7fff640 (LWP 6579) "pool-org.gnome."):
#0  0x00007ff4e3d52e8f in poll () from /lib64/libc.so.6
#1  0x00007ff4e4ba00dd in g_main_context_iterate.constprop () from /lib64/libglib-2.0.so.0
#2  0x00007ff4e4b488e0 in g_main_context_iteration () from /lib64/libglib-2.0.so.0
#3  0x00007ff4c22db863 in flatpak_load_http_uri_once () from /lib64/libflatpak.so.0
#4  0x00007ff4c22dbb4f in flatpak_load_uri () from /lib64/libflatpak.so.0
#5  0x00007ff4c22faa8c in _flatpak_dir_get_remote_state.constprop.0 () from /lib64/libflatpak.so.0
#6  0x00007ff4c22b489c in flatpak_dir_update_remote_configuration () from /lib64/libflatpak.so.0
#7  0x00007ff4c22bc3f5 in flatpak_installation_update_remote_sync () from /lib64/libflatpak.so.0
#8  0x00007ff4c23614e5 in gs_flatpak_refresh_appstream_remote () from /usr/lib64/gnome-software/plugins-17/libgs_plugin_flatpak.so
#9  0x00007ff4c2363116 in gs_flatpak_refresh () from /usr/lib64/gnome-software/plugins-17/libgs_plugin_flatpak.so
#10 0x00007ff4c2364514 in gs_flatpak_search () from /usr/lib64/gnome-software/plugins-17/libgs_plugin_flatpak.so
#11 0x00007ff4c237014a in gs_plugin_flatpak_do_search () from /usr/lib64/gnome-software/plugins-17/libgs_plugin_flatpak.so
#12 0x00007ff4e4f6e031 in gs_plugin_loader_run_results () from /usr/lib64/gnome-software/libgnomesoftware.so.17
#13 0x00007ff4e4f72aae in gs_plugin_loader_process_thread_cb () from /usr/lib64/gnome-software/libgnomesoftware.so.17
#14 0x00007ff4e4d43023 in g_task_thread_pool_thread () from /lib64/libgio-2.0.so.0
#15 0x00007ff4e4b77b72 in g_thread_pool_thread_proxy.lto_priv () from /lib64/libglib-2.0.so.0
#16 0x00007ff4e4b75172 in g_thread_proxy () from /lib64/libglib-2.0.so.0
#17 0x00007ff4e3cda017 in start_thread () from /lib64/libc.so.6
#18 0x00007ff4e3d5f6d0 in clone3 () from /lib64/libc.so.6

There's nothing one can do about that, as far as I know.

Comment 9 Kamil Páral 2022-04-12 11:58:45 UTC
Created attachment 1871947 [details]
journal with gs from comment 6

This is system journal after installing GS from Koji from comment 6. This time, I enabled the repo, searched for minecraft, waited until /var/lib/flatpak/appstream/flathub appeared, searched again, and minecraft *did not* show up. So there has to be some race condition in here.

Comment 10 Kamil Páral 2022-04-12 12:00:51 UTC
Created attachment 1871948 [details]
journal with gs from comment 6 including gs restart

This is the same journal as in comment 9, but in that same session I additionaly ran "gnome-software --quit" and started it again from gnome overview, searched for minecraft and it showed up.

Comment 11 Milan Crha 2022-04-13 10:49:47 UTC
The log from comment #9 has this:

> gs_flatpak_add_apps_from_xremote: 0x558411a06d60: failed to refresh metadata 'flathub': Operation was cancelled

which, I believe, is the reason. The log shows that after this refresh cancellation the remote information is not updated again, thus the gnome-software does not know about the apps in this remote.

Now to find the reason why it had been cancelled.

Comment 12 Kamil Páral 2022-04-13 10:52:47 UTC
I wonder if automatic update checking/downloading could interfere with this. I know that while testing this, I sometimes saw "Updates are available" notification show up.

Comment 13 Milan Crha 2022-04-13 12:16:33 UTC
After doing some more debugging, what do you do after you enable the Flathub remote? Do you close the Repositories dialog immediately? If you do, then it can be the reason for the cancellation of the ongoing refresh, because the dialog cancels anything it's working on on its closing.

A simple test of this is:
a) start the search for the "minec"
b) open the Repositories dialog and do the changes there
c) do not close the Repositories dialog, rather wait until the search page reloads with added or removed results.

If this will work, then it's it.

Comment 14 Milan Crha 2022-04-13 12:19:19 UTC
Related upstream bug:
https://gitlab.gnome.org/GNOME/gnome-software/-/issues/821

Comment 15 Milan Crha 2022-04-13 13:03:39 UTC
I do not seem to be able to reproduce this reliably. Could I ask you to test with this [1] patched gnome-software, please?
It prints several debugging information on the console.

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=85612089

Comment 16 Kamil Páral 2022-04-13 14:08:21 UTC
In the past, I used to close the repo config dialog at some random time. This time, I let it open the whole time. I found out a very useful thing - you can actually see when the metadata gets refreshed, because the repo config dialog is then refreshed and it scrolls to the top. So that's how I determined that I can perform further actions. I'll described the findings below together with fresh system journals.

PS: For all my tests, I use a virtual machine snapshot that I restore each time, so that my repeated attempts are more consistent and reliable. In that snapshot, gnome-software hasn't been started (manually) yet, so each time I test it, this tries to mimic the situation when a fresh system installation is used.

Comment 17 Kamil Páral 2022-04-13 14:12:29 UTC
Created attachment 1872193 [details]
journal relevant to comment 15 (live search)

In this case, I did the following:
1. start gnome-software, search for "minecr" (no results)
2. open repo dialog, enable flathub selection, wait (don't close the dialog)
3. see that "no results" changed to a spinner underneath the dialog (you might need to resize it to be much smaller, to see it)
4. see that the spinner changed back to "no results"
5. see that the repo dialog refreshed (scrolled to top)
6. close the repo dialog
7. search again for "minecr", no results

I performed this 5 times, each time it happened the same.

Comment 18 Kamil Páral 2022-04-13 14:15:40 UTC
Created attachment 1872195 [details]
journal relevant to comment 15 (no search ahead, broken search)

In this case, I did the following:
1. start gnome-software
2. open repo dialog, enable flathub selection, wait (don't close the dialog)
3. see that the repo dialog refreshed (scrolled to top)
4. close the repo dialog
5. search for "minecr", no results
6. search for "mine", no results (!!)

If I searched for something else, like "chess" or "calendar", search worked. But for some reason, searching for "mine" returned empty, even though there should've been at least 2 results (gnome mines and kmines), which usually show up.

I performed this 5 times, each time it happened the same.

Comment 19 Milan Crha 2022-04-13 16:03:17 UTC
Both logs show:

> gnome-software[1980]: failed to get search apps: failed to get permissions: No such ref 'app/com.mojang.Minecraft/x86_64/stable' in remote flathub

Form which I guess something broke even lower. The "failed to get permissions" is a gnome-software string, it doesn't mean file system or other permissions, it's about info about permissions the app requests from the user (like the filesystem permissions to access user's home and such).

The "No such ref 'xxx' in remote yyy" comes from the flatpak itself. It looks like the remote summary does not agree with the appstream data.

When you've it in this broken state, could you run from a terminal:

   $ flatpak remote-info flathub com.mojang.Minecraft
   $ flatpak search minec

It's possible the first or the second command will fix the problem under the hood, I do not know. For me, the search finds one hit, the remote-info shows an information used in the GUI of the gnome-software. I do not get the "No such ref" error here.

Comment 20 Kamil Páral 2022-04-14 06:27:23 UTC
Created attachment 1872383 [details]
journal relevant to comment 17+19

In this case, I followed the reproducer from comment 17, and then at the end I executed the commands from comment 19, and performed searches again. It didn't fix it, there were still no results for "minecr".

Comment 21 Kamil Páral 2022-04-14 06:33:25 UTC
Created attachment 1872384 [details]
journal relevant to comment 18

In this case, I followed the reproducer from comment 18 and wanted to combine it with the commands from comment 19. But guess what. It WORKED every bloody time. In step 5 and 6, I received all results properly (with a small exception, see below). I retested 5 times, and each time it worked (yesterday it was broken 5 times in a row).

A slight variation occurred twice, though. In my first search for mine/minecr, I saw "No apps found". And only after waiting ~5 seconds and repeating the search, I saw the results. The attached journal is from this slightly incorrect case. (In the other 3 attempts, the search returned proper results on the first try).

Comment 22 Milan Crha 2022-04-14 10:52:47 UTC
This is getting tricky to reproduce. I made a backup of the "/var/lib/flatpak/" before finishing the initial setup wizard, to which I restore the directory and I also delete ~/.cache/gnome-software/, before I restart the machine, which mimics pretty closely the state after install.



I have one more faulty state, which is, I believe, related to the misbehaviour described in bug #2075439. Depending on the timing after reboot, I can get the gnome-software's repositories list into a state where no system remotes are shown. That happens when gnome-software realizes the user has no enough permissions to work with system flatpak appstream data. From the source code:

	/* if we can't update the AppStream database system-wide don't even
	 * pull the data as we can't do anything with it */

This is asked on org.freedesktop.Flatpak.appstream-update.



The state with `No such ref 'app/com.mojang.Minecraft/x86_64/stable' in remote flathub` - I see it when I wait a bit before I log in with my user (aka when I workaround bug #2075439). I added a change into [1], which stops gnome-software from panic when the ref is not part of the remote (I do not know why exactly it happens, maybe due to ongoing update of the flatpak remote/repos). It seems to work, as I can install the app afterwards, even when this error is shown there.

The [1] adds also more debug information. It can be extracted from the journal with:

   $ journalctl -ex | egrep "gnome-software|.Software"

That being said, I'll propose a change upstream to not panic when an app permission read fails, though I'm not sure whether they'll approve it. In any case, I think the bug #2075439 should be addressed too.

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=85660722

Comment 23 Kamil Páral 2022-04-14 12:17:37 UTC
Sigh, I took the koji build from comment 22 and performed steps from comment 17. But this time, it worked perfectly, i.e. the open search results refreshed correctly and showed Minecraft. I tested it 5 times, and wasn't able to replicate the failure from yesterday. I start to think it behaves completely randomly just to troll us.

Comment 24 Milan Crha 2022-04-14 12:51:02 UTC
This test build [2] contains upstream fixes from [3] and [4], which should address the problem - they did for me, except of the case of the not allowed permissions for the system flatpak remotes (the first ball of the comment #22), but that's something out of my hands. Could you give it a try too, please?

For the reproducer from my side:
a) kill gnome-software (gnome-software --quit)
b) remove ~/.cache/gnome-software
c) restore /var/lib/flatpak/
d) restart the computer
(d.1) on gdm prompt wait for 10 seconds)
e) login
g) run gnome-software from the Activities (you may see "Software catalog is being downloaded" now)
h) search for: mine
   - see two apps found
i) open Repositories and enable filtered flathub remote/repo
j) wait until the search page updates itself
   - you can also try to close the dialog prematurely, then you'd need to search again yourself, but it still should find the app.

* if does not work - no Minecraft shown, journalctl shows a runtime warning about "No such ref"
* if does work - there is Minecraft found and when selected, and waited a bit, also the required size is shown

The build from comment #22 workarounds the "no such ref" error (you can find in in the journal), but it also doesn't show the size (more runtime warnings in the journal). The [2] fixes this in a correct way.

[2] https://koji.fedoraproject.org/koji/taskinfo?taskID=85665227
[3] https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/1321
[4] https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/1325

Comment 25 Adam Williamson 2022-04-18 21:47:19 UTC
Kamil, can you test the test build from #c24? Setting this to POST to reflect that proposed fixes exist (the MRs linked by Milan).

Comment 26 Milan Crha 2022-04-19 11:06:19 UTC
I just started an official build [1], for which I'll create an update soon after the build finishes. That to avoid any delays for the Go/no-GO meeting or such.

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=85916129

Comment 27 Fedora Update System 2022-04-19 11:38:03 UTC
FEDORA-2022-efb93e8c5f has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-efb93e8c5f

Comment 28 Kamil Páral 2022-04-19 14:23:10 UTC
Unfortunately this doesn't seem to help here. Reproducer:
1. install a fresh VM from Fedora-Workstation-Live-x86_64-36-20220418.n.0.iso
2. boot the system
3. using dnf, install gnome-software update from comment 27
4. shut down, make a VM snapshot, boot again
5. run gnome-software, go to software repos dialog and enable flathub selection, close the dialog
6. search for "mine"

During my first attempt, I saw all 3 expected results including minecraft. On my second attempt, only rpm results were visible and minecraft didn't appear. Even after searching again for "minecr" -> no apps found. At the very same time, "flatpak search minecraft" from command line found the game just fine. There's still a race condition there somewhere.

Comment 29 Kamil Páral 2022-04-19 14:24:05 UTC
Created attachment 1873520 [details]
system journal related to comment 28

Journal from the failed attempt described in comment 28.

Comment 30 Milan Crha 2022-04-19 15:34:40 UTC
I do not see any problem being logged in the above journal, things seem to pass fine there. Let me re-try with the new .iso, maybe some things changed there too (I used older .iso before).

Comment 31 Milan Crha 2022-04-19 17:00:58 UTC
(In reply to Kamil Páral from comment #28)
> 5. run gnome-software, go to software repos dialog and enable flathub
> selection, close the dialog

I've been able to reproduce this only when I close the Repositories dialog too early, like immediately after I enter my credentials (when enabling the remote). This cancels the remote enable, thus it behaves like the Flathub not being enabled. When I open the Repositories dialog again and disable and then enable the Flathub, and wait a bit before I close the dialog, then it works properly.

I'm this close to just not cancel the dialog's cancellable to avoid this kind of race. Such modified gnome-software is building (as a scratch build) here:
https://koji.fedoraproject.org/koji/taskinfo?taskID=85930396

Comment 32 Fedora Update System 2022-04-19 17:28:03 UTC
FEDORA-2022-efb93e8c5f has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-efb93e8c5f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-efb93e8c5f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 33 Kamil Páral 2022-04-19 17:49:28 UTC
(In reply to Milan Crha from comment #31)
> I've been able to reproduce this only when I close the Repositories dialog
> too early, like immediately after I enter my credentials (when enabling the
> remote). This cancels the remote enable, thus it behaves like the Flathub
> not being enabled. When I open the Repositories dialog again and disable and
> then enable the Flathub, and wait a bit before I close the dialog, then it
> works properly.

This might've been the case. I'll test more tomorrow. Today, I tried to behave like a regular user would, i.e. I didn't wait a long time before closing the dialog. I enabled the repo and closed it in a normal speed, i.e. after a few seconds.

> I'm this close to just not cancel the dialog's cancellable to avoid this
> kind of race. Such modified gnome-software is building (as a scratch build)
> here:
> https://koji.fedoraproject.org/koji/taskinfo?taskID=85930396

I'll also test tomorrow. What does "not cancel the dialog's cancellable" exactly mean? What if there is a very slow internet and the repo metadata download takes minutes?

Comment 34 Milan Crha 2022-04-20 05:54:59 UTC
(In reply to Kamil Páral from comment #33)
> I'll also test tomorrow. What does "not cancel the dialog's cancellable"
> exactly mean? What if there is a very slow internet and the repo metadata
> download takes minutes?

Then it'll run in the background until the operation is finished or until it fails.

This is not ideal. I wrote a little bit to the long standing bug:
https://gitlab.gnome.org/GNOME/gnome-software/-/issues/821#note_1429373

Note the early closure of the repositories dialog is nothing new, you might not need to call it a regression from the previous Fedora versions (meaning the official build, which is part of the update FEDORA-2022-efb93e8c5f, can be enough for this bug).

Comment 35 Kamil Páral 2022-04-20 08:28:07 UTC
I tested it more today with gnome-software-42.0-3.fc36. When I wait until the repo dialog refreshes (scrolls to top), I can find minecraft every time. So some of the fixes seem to have effect, great. Unfortunately that's not how regular users will behave.

If I close the dialog shortly after enabling the repo, it almost universally fails. In 3 attempts, searching for "mine" only displayed the rpm results (after the spinner disappeared), but not minecraft, even when I waited more and repeated it. In 2 attempts, searching for "mine" only displayed the rpm results initially, but closing the search and searching again showed minecraft as well.

> I'm this close to just not cancel the dialog's cancellable to avoid this
> kind of race. Such modified gnome-software is building (as a scratch build)
> here:
> https://koji.fedoraproject.org/koji/taskinfo?taskID=85930396

I tested with this as well, and immediately the first attempt failed. After closing the dialog and searching for "mine", I again saw just the rpm results (after the spinner disappeared), but not minecraft, even when I waited more and repeated it. At the same time, "flatpak search minecraft" from cmdline worked ok. So your patch is either not working or there's another issue still present. I'll attach the log.

> Note the early closure of the repositories dialog is nothing new, you might not need to call it a regression from the previous Fedora versions (meaning the official build, which is part of the update FEDORA-2022-efb93e8c5f, can be enough for this bug).

Overall if you think fixing the dialog handling properly requires much more work than can be done before release, we can discuss waiving the blocker on the grounds that it's too hard to fix in such a short time. Especially if you believe the problem has been present even before (I think I tested it quite a lot in F35 and I think it worked better with standard rpm repos, but I'm not sure).

Comment 36 Kamil Páral 2022-04-20 08:28:59 UTC
Created attachment 1873756 [details]
system journal related to comment 35

This is when using the custom koji build, as described in comment 35.

Comment 37 Milan Crha 2022-04-20 13:49:59 UTC
Could you try again with this [1] build, please?

I added changes to fix a place I've been able to reproducer it at. It adds also sever debug prints, which will be shown in the journal. To filter the journal for the relevant information only I suggest to use this command:

   $ journalctl -ex | egrep "gnome-softwa|.Software" >log.txt

I'd like to see the log if you happen to reproduce. This time the refresh is postponed after the repo is enabled, thus the search can be slowed down with the appstream data download. You can cancel the download by changing the search term. You can check the log for "NOW-NOW-NOW" (quoted for clarity only). Being it there, you hit the place I've got for the reproducer. It should survive it this time.

I'll add these new changes into the:
https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/1325

> Especially if you believe the problem has been present even before (I think I tested it quite a lot in F35 and I think it worked better with standard rpm repos, but I'm not sure)

The problem with closing the Repositories dialog "prematurely" is there for years, see [2]. The rpm repos are different, they are managed by PackageKit, transparently, while Flatpak is more guided by the gnome-software. The gnome-software 42 contains a lot of changes in the threading model, which play its role here as well. The appstream data refresh on remote enable had been added also in the 42. Simply said, there are many changes involved in the gnome-software 42.

[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=85969540
[2] https://gitlab.gnome.org/GNOME/gnome-software/-/merge_requests/1325

Comment 38 Kamil Páral 2022-04-20 14:33:25 UTC
(In reply to Milan Crha from comment #37)
> Could you try again with this [1] build, please?

I tried it 10 times and it worked 10 times! Maybe I was just lucky, but it is really promising. Can this be a part of the official build? I haven't noticed any delayed search, to be honest, the search seemed to take roughly the same time as before.

I have a small side note, though. In 3 out of 10 attempts, the search screen didn't contain just a spinner in the middle, but it said "Search for Apps" with an icon. The search was happening in the background, and the screen later refreshed with search results. But the spinner was not visible, and it was slightly confusing, because the screen was static (didn't seem to be doing anything) and the term "Search for Apps" seems to indicate that no search is happening yet, and I should start it somehow. Or people can also understand it as that no results were found. (I saw this screen before, and it might've been the source of some of my confusion when reporting this bug - I thought this screen means "no results"). This is not a new thing with this build, it was there before, I just wanted to make you aware. It seems that the spinner screen is not always shown reliably. (And the "Search for Apps" term should probably be rephrased to be clearer, if it's supposed to be visible sometimes).

> I'd like to see the log if you happen to reproduce. 

It worked every time, so I didn't upload one. If you want it anyway, I can.

> search term. You can check the log for "NOW-NOW-NOW" (quoted for clarity

Yes, that was there every time I checked.

Comment 39 Milan Crha 2022-04-20 15:20:34 UTC
> the search screen didn't contain just a spinner in the middle, but it said "Search for Apps" with an icon.

Yeah, I noticed it too, some time ago, but I didn't consider it a major thing. There is a special text for "no apps found", thus this just misinterprets the inner state of the search. A new (upstream) bug will be helpful.

> It worked every time, so I didn't upload one. If you want it anyway, I can.

Right, if ti works, then no log needed. I'll update the update.

> Yes, that was there every time I checked.

Good, then you've been facing what I reproduced. That's truly promising.

Comment 40 Milan Crha 2022-04-20 15:46:55 UTC
Unless I made a mistake while removing those debug prints and splitting the change into two separate commits, the gnome-software-42.0-4.fc36 works the same as the previous scratch build. I just added it into the update [1].

[1] https://bodhi.fedoraproject.org/updates/FEDORA-2022-efb93e8c5f

Comment 41 Fedora Update System 2022-04-20 23:05:39 UTC
FEDORA-2022-efb93e8c5f has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-efb93e8c5f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-efb93e8c5f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 42 Kamil Páral 2022-04-21 07:58:49 UTC
(In reply to Milan Crha from comment #40)
> Unless I made a mistake while removing those debug prints and splitting the
> change into two separate commits, the gnome-software-42.0-4.fc36 works the
> same as the previous scratch build. I just added it into the update [1].
> 
> [1] https://bodhi.fedoraproject.org/updates/FEDORA-2022-efb93e8c5f

I tested it 5 more times and it worked perfectly for me ;-)

Comment 43 Fedora Update System 2022-04-22 01:19:26 UTC
FEDORA-2022-efb93e8c5f has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.