Bug 1762751 - Gnome-software should provide workaround for libgit2 for F29/F30->F31 upgrade
Summary: Gnome-software should provide workaround for libgit2 for F29/F30->F31 upgrade
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: PackageKit
Version: 31
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
Assignee: Kalev Lember
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedPreviousReleaseBlocker
Depends On:
Blocks: F31FinalBlocker 1747408
TreeView+ depends on / blocked
 
Reported: 2019-10-17 11:48 UTC by Miroslav Suchý
Modified: 2019-10-31 14:17 UTC (History)
23 users (show)

Fixed In Version: PackageKit-1.1.12-3.fc29 PackageKit-1.1.12-6.fc30
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-10-27 01:09:15 UTC


Attachments (Terms of Use)
F30 to F31 upgrade log - with ktexteditor (985.34 KB, text/plain)
2019-10-23 19:32 UTC, František Zatloukal
no flags Details

Description Miroslav Suchý 2019-10-17 11:48:58 UTC
See the discussion in 
  https://bugzilla.redhat.com/show_bug.cgi?id=1747408
To successfully handle the upgrade from F30 to F31, Gnome-Software should either:

 * reset libgit2, exa, bat modules
 * or do not offer an upgrade to F31 at all and point to command line dnf upgrade.

This can be done as temporary patch in F30 only.

Comment 1 Miro Hrončok 2019-10-17 11:58:33 UTC
And F29.

Comment 2 Richard Hughes 2019-10-17 15:13:23 UTC
GNOME Software is entirely the wrong layer to handle this.

Comment 3 Miro Hrončok 2019-10-18 10:45:52 UTC
(In reply to Richard Hughes from comment #2)
> GNOME Software is entirely the wrong layer to handle this.

We need a layer to workaround the problem. What layer should that be in this case?

Comment 4 Miro Hrončok 2019-10-18 11:01:52 UTC
Changing the version to F31, so it can be tracked as F31 blocker, but the workaround needs to be delivered on F29 and F30.

Comment 5 Jaroslav Mracek 2019-10-18 11:05:56 UTC
I suppose that PackageKit is the correct layer to handle it.

Comment 6 Adam Williamson 2019-10-18 16:53:49 UTC
note, we've already asked if libdnf could do this so we don't have to do it separately in dnf and gnome-software/PackagekKit, but the answer was "no": https://bugzilla.redhat.com/show_bug.cgi?id=1747408#c82

If a FESCo majority votes for https://pagure.io/fesco/issue/2230#comment-605570 , it will become an accepted previous release blocker.

Comment 7 Michael Catanzaro 2019-10-19 14:37:30 UTC
(In reply to Richard Hughes from comment #2)
> GNOME Software is entirely the wrong layer to handle this.

GNOME Software, PackageKit, wrong layer, right layer, doesn't matter because it doesn't need to be elegant. What we need is a hacky downstream patch in one place or the other to unblock the F31 release so that we don't wind up with another no-go decision next week.

Comment 8 Zbigniew Jędrzejewski-Szmek 2019-10-19 19:16:33 UTC
+1 for blocker, same as #1747408.

Comment 9 Stephen Gallagher 2019-10-21 12:25:11 UTC
+1 blocker

Comment 10 Mohan Boddu 2019-10-21 14:04:39 UTC
+1 Blockerbranched/Fedora-31-20191021.n.0/STATUS

Comment 11 Miro Hrončok 2019-10-21 14:22:42 UTC
+1 blocker FTR

Comment 12 Kalev Lember 2019-10-21 16:25:51 UTC
I don't think this can be fixed in PackageKit alone. I looked at this briefly and as much as I can tell, libdnf doesn't expose the necessary API to be able to do reset module streams.

Moving to libdnf.

Comment 13 Geoffrey Marr 2019-10-21 18:45:45 UTC
Discussed during the 2019-10-21 blocker review meeting: [0]

The decision to classify this bug as an "AcceptedPreviousReleaseBlocker" was made as we take FESCo's vote on https://pagure.io/fesco/issue/2230#comment-605570 as a declaration that this should be a blocker. If a decision is taken to use the libgit2_0.28 workaround instead, this will become a Accepted0DayBlocker.

[0] https://meetbot.fedoraproject.org/fedora-blocker-review/2019-10-21/f31-blocker-review.2019-10-21-16.00.txt

Comment 14 Jaroslav Mracek 2019-10-22 06:00:31 UTC
I create required binding for PackageKit (https://github.com/rpm-software-management/libdnf/pull/822). Kalev Lember - please test it and feel free to merge it and back-port it into Fedora 29,30. I believe that rest is up to PackageKit how the function will be used. Returning to PackageKit.

Comment 15 Kalev Lember 2019-10-22 10:45:42 UTC
Nice, thanks Jaroslav!

Comment 16 Kalev Lember 2019-10-23 14:47:50 UTC
OK, so now we have https://github.com/rpm-software-management/libdnf/pull/822 for libdnf, and https://github.com/hughsie/PackageKit/pull/350 for PackageKit. I put together a COPR to test this (completely untested by me right now): https://copr.fedorainfracloud.org/coprs/kalev/reset-libgit2-module/

Comment 17 Adam Williamson 2019-10-23 19:11:51 UTC
So far this does not *seem* to be working for me.

My reproducer steps are these:

1. Install Fedora 30 Workstation live
2. 'dnf -y install kf5-ktexteditor' (this depends on libgit2 0.27, so causes the 0.27 stream of the libgit2 module to be enabled and installs libgit2 from it)
3. Enable the COPR
4. Update system with gnome-software, check packages from the COPR got installed
5. Do `gsettings set org.gnome.software show-upgrade-prerelease true`
6. Run gnome-software and refresh updates so Fedora 31 upgrade shows
7. Hit the button to run the upgrade download

After doing this, I get a warning that kf5-ktexteditor will be removed. I would expect it not to be in this case - if the module reset worked, kf5-ktexteditor's deps should be satisfiable within the F31 repos and it should not be removed.

By contrast, the dnf-system-upgrade workaround *does* seem to work. If from step 5 on I instead do `dnf --releasever=31 system-upgrade download`, with the *older* version of the system-upgrade plugin without the workaround, I get an error about kf5-ktexteditor (because dnf doesn't do --allowerasing by default)...but with the *newer* version of the system-upgrade plugin *with* the workaround, I get:

Resetting modules:
    libgit2

and kf5-ktexteditor is included in the transaction and there is no error.

Comment 18 Adam Williamson 2019-10-23 19:23:08 UTC
Running packagekitd --verbose in a console, I do see this log message:

12:21:33	PackageKit-DNF      resetting libgit2 module

and I do *not* see "failed to reset libgit2 module"...

Comment 19 Adam Williamson 2019-10-23 19:24:14 UTC
Oh, sorry, BTW, there's a step 8 in the reproducer: hit the 'install' button after the download completes. It's at this point that the warning shows up.

Comment 20 František Zatloukal 2019-10-23 19:31:47 UTC
I was able to upgrade from Fedora 30 Workstation GA (just with kf5-ktexteditor installed) to Fedora 31 just fine. libgit2 got updated from 0.27 to 0.28, kf5-ktexteditor wasn't removed.

gnome-software-3.32.1-2.fc30.x86_64
PackageKit-1.1.12-5.fc30.x86_64

I'll add log from that right away.

Comment 21 František Zatloukal 2019-10-23 19:32:27 UTC
Created attachment 1628610 [details]
F30 to F31 upgrade log - with ktexteditor

Comment 22 Adam Williamson 2019-10-23 19:37:48 UTC
The system log does not provide much useful info, I don't think. It shows that a modular libgit2 package is removed and a non-modular one is 'updated', but that's about all. To be clear, here, Frantisek is not saying that the fix works for him: he's saying he can't reproduce the bug in the first place. The PackageKit he has installed is the stock F30 one, it is not the one with the proposed fix.

Did you update the system before trying to upgrade (as in my step 4)?

Comment 23 František Zatloukal 2019-10-23 19:58:21 UTC
(In reply to Adam Williamson from comment #22)
> Did you update the system before trying to upgrade (as in my step 4)?

No, looks like that's the cause. If I update the system before the upgrade, it gets broken and fix provided in copr doesn't help.

Sorry for the noise :)

Comment 24 Adam Williamson 2019-10-23 20:06:02 UTC
OK, so at least we're on the same page now, good :)

Comment 25 Adam Williamson 2019-10-23 20:32:17 UTC
So I went ahead and ran the upgrade, and indeed kf5-ktexteditor is removed. A libgit2 package is installed; it appears to be a build from the 0.27 stream for F31 (libgit2-0.27.8-1.module_f31+3326+1efaac5b.x86_64 ). 'dnf module list libgit2' shows:

Fedora Modular 31 - x86_64
Name           Stream         Profiles       Summary                            
libgit2        0.26                          Library implementation of Git      
libgit2        0.27 [e]                      Library implementation of Git      
libgit2        0.28                          Library implementation of Git      

so, it seems like the reset of the module just doesn't really take effect?

Comment 26 Jaroslav Mracek 2019-10-24 07:16:56 UTC
I changed patch in libdnf. It saves modular changes and updates failsafe data directly. The function now does not require successful transaction after calling. Please could you retest it?

Comment 27 Kalev Lember 2019-10-24 10:06:11 UTC
Thanks! I've updated the COPR with the new patch.

Comment 28 František Zatloukal 2019-10-24 10:51:49 UTC
Kalev, seems updated builds from your COPR work just fine. Can you submit bodhi update with everything you have in your COPR?

Thanks!

I did:

1. Install Fedora 30 Workstation live
2. 'dnf -y install kf5-ktexteditor' (this depends on libgit2 0.27, so causes the 0.27 stream of the libgit2 module to be enabled and installs libgit2 from it)
3. Enable the COPR
4. Update system with gnome-software, check packages from the COPR got installed
5. Do `gsettings set org.gnome.software show-upgrade-prerelease true`
6. Run gnome-software and refresh updates so Fedora 31 upgrade shows
7. Hit the button to run the upgrade download

Package kf5-ktexteditor remained present after the upgrade to F31.

Comment 29 Kalev Lember 2019-10-24 11:48:05 UTC
Great, thanks for testing! Doing the builds now.

Comment 30 Fedora Update System 2019-10-24 12:46:19 UTC
FEDORA-2019-2368fc415f has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-2368fc415f

Comment 31 Fedora Update System 2019-10-24 13:50:23 UTC
FEDORA-2019-2fb39de3ce has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-2fb39de3ce

Comment 32 Fedora Update System 2019-10-25 18:08:38 UTC
PackageKit-1.1.12-3.fc29, libdnf-0.31.0-8.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.

Comment 33 Miro Hrončok 2019-10-25 19:09:40 UTC
F30 update not yet pushed.

Comment 34 Fedora Update System 2019-10-25 19:34:36 UTC
PackageKit-1.1.12-6.fc30, libdnf-0.35.5-4.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-2368fc415f

Comment 35 Fedora Update System 2019-10-27 01:09:15 UTC
PackageKit-1.1.12-6.fc30, libdnf-0.35.5-4.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 36 Steve 2019-10-31 14:17:03 UTC
This version of Packagekit breaks the whole gnome-software. Please see #1767453


Note You need to log in before you can comment on or make changes to this bug.