Bug 1943593 - fedpkg-minimal breaks if "sources" file does not end in newline character
Summary: fedpkg-minimal breaks if "sources" file does not end in newline character
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: fedpkg-minimal
Version: 34
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Ondřej Nosek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-03-26 14:31 UTC by Hans Ulrich Niedermann
Modified: 2021-04-04 00:18 UTC (History)
3 users (show)

Fixed In Version: fedpkg-minimal-1.2.0-1.fc34
Clone Of:
Environment:
Last Closed: 2021-04-04 00:18:38 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Hans Ulrich Niedermann 2021-03-26 14:31:36 UTC
Description of problem:

  fedpkg-minimal requires the "sources" file to end in a newline
  character. If the file does not end in a newline character,
  fedpkg fails to download the source files, and then fails to
  do anything else.

Version-Release number of selected component (if applicable):

  fedpkg-minimal-1.1.0-16.fc34

How reproducible:

  100%

Steps to Reproduce:
0. Install fedpkg-minimal.
1. "fedpkg clone" some package
2. remove the trailing newline from the "sources" file
3. Run "fedpkg --version"

Actual results:

  $ fedpkg --version
  sha512sum: kirigami-gallery-20.12.3.tar.xz: No such file or directory
  kirigami-gallery-20.12.3.tar.xz: FAILED open or read
  sha512sum: WARNING: 1 listed file could not be read
  $ 

Expected results:

  $ fedpkg --version
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
  100  320k  100  320k    0     0   366k      0 --:--:-- --:--:-- --:--:--  366k
  kirigami-gallery-20.12.3.tar.xz: OK
  $ 

Additional info:

  These are the two koji builds of kirigami-gallery which led me to
  discovery of this bug:

    https://koji.fedoraproject.org/koji/taskinfo?taskID=64608367
    https://koji.fedoraproject.org/koji/taskinfo?taskID=64637096

  Fedora developers will usually have fedpkg proper installed on
  their development machines, not fedpkg-minimal, so Fedora
  developers will not run into this issue on their local machines
  and only get tripped up by it once they build on koji builders.

Comment 1 Hans Ulrich Niedermann 2021-03-26 14:39:07 UTC
I see two ways to fix this:

  a) Have fedpkg-minimal treat a "sources" file not ending
     in a newline as if it had a newline at the end. That
     is what fedpkg proper does.

  b) Have git hooks or fedpkg proper or something similar
     block "sources" files not ending in a newline.

Unless fedpkg-minimal needs to be kept so minimal that a few
more bytes of code for a bugfix are an issue, I would favour
option a).

Comment 2 Ondřej Nosek 2021-03-27 21:46:17 UTC
Hi,

there is a sentence "remove the trailing newline from the "sources" file" to reproduce. I think it should be "add the trailing newline". I tried a lot of various repos and I only found sources file without trailing spaces and it always worked well.
I believe, that what you want is a fedpkg-minimal to be more tolerant and accept sources with or without trailing newlines. Is it so?

Comment 3 Ondřej Nosek 2021-03-28 00:07:03 UTC
This fix (under review) can, hopefully, help you:
https://pagure.io/fedpkg-minimal/pull-request/11

Comment 4 Hans Ulrich Niedermann 2021-03-28 00:35:24 UTC
Sorry for the bad writing and the missing hexdumps.

Let me "hexdump -C sources" the two examples from the kirigami-gallery
example (git clone https://src.fedoraproject.org/rpms/kirigami-gallery.git).

This sources file from commit f6fd608e301f5ae37dd81fdacb7a01584a7be5b3
made fedpkg-minimal fail to parse the sources file:

[...]
00000090  38 62 39 63 61 36 39 36  65 37 35 31 63 33 33 32  |8b9ca696e751c332|
000000a0  30 37 66 66 61 30 38 63  31 33 35                 |07ffa08c135|
000000ab

This sources file commit cffb1959e1f692f8a5670f9958f8b7be05fb3508
allowed fedpkg-minimal to parse the sources file:

[...]
00000090  38 62 39 63 61 36 39 36  65 37 35 31 63 33 33 32  |8b9ca696e751c332|
000000a0  30 37 66 66 61 30 38 63  31 33 35 0a              |07ffa08c135.|
000000ac

The fedpkg command from the "fedpkg" package interprets both variants
of the sources file in exactly the same way.

As every Fedora package which builds in koji obviously has a sources
file ending *with* a newline, to create a sources file which triggers
the bug in fedpkg-minimal you need to remove that last newline character
from the sources file.

I should have given a command line command to do that. Sorry.

And thank you for the fixes. From reading the proposed fixes in the
pull request, this appears to fix the problem.

Comment 5 Ondřej Nosek 2021-03-28 17:21:35 UTC
Understood, thanks. I modified some unittests to reflect this better.

Comment 6 Fedora Update System 2021-03-30 15:02:16 UTC
FEDORA-2021-ebade0225c has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-ebade0225c

Comment 7 Ondřej Nosek 2021-03-30 15:07:14 UTC
There are builds "fedpkg-minimal-1.2.0-1" for all available distributions in Koji. Updates are created in Bodhi as well.

Comment 8 Fedora Update System 2021-03-31 01:19:52 UTC
FEDORA-2021-ebade0225c has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-ebade0225c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-ebade0225c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Fedora Update System 2021-04-04 00:18:38 UTC
FEDORA-2021-ebade0225c has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.