Bug 2329783 - Package sources are checked in to distgit repo
Summary: Package sources are checked in to distgit repo
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: golang-modernc-ccorpus2
Version: 45
Hardware: Unspecified
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: W. Michael Petullo
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2024-12-02 02:02 UTC by "FeRD" (Frank Dana)
Modified: 2026-08-17 13:20 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed:
Type: ---
Embargoed:


Attachments (Terms of Use)

Description "FeRD" (Frank Dana) 2024-12-02 02:02:08 UTC
I happened to accidentally discover that the golang-modernc-ccorpus2 packaging repository:

https://src.fedoraproject.org/rpms/golang-modernc-ccorpus2

currently has the dubious distinction of being the single largest package repo in the Fedora collection, exceeding the next-nearest by over 100MB.

This is because the generated `assets.go` source archive is committed to the package repository, instead of having been uploaded to the builder lookaside cache.

I wasn't clear myself on the correct procedure for generated sources like that, but @churchyard reported that it's no different from any other source archive.

To avoid large downloads, upstream source files (or upstream-originating modified source files) should not be committed to the package repo. Rather, they should be added to the builders' lookaside cache using `fedpkg new-sources`, which uploads the file to Koji and adds its name to the local `.gitignore` file so that it can't be checked in.

To correct this (remove the source files from the Git history), a surgical rewrite of the repository history by a Fedora git administrator will be required.

See https://pagure.io/releng/issue/7265 for when this was previously done, for the Blender package.

An important point here is that it will NOT do any good to simply `git rm` the errant file -- it's still part of the commit history, so it'll still live in the `.git` directory and still cause large checkouts. Correcting this requires administrative rewrite of the repo's commit history, which can be done by releng but first has to be authorized by FeSCO because under normal circumstances rewriting package repo history is entirely prohibited.

But I wanted to open this issue first, as a heads-up regarding the handling of that file in the future. Source archives from upstream should never be committed to the package repo itself, only uploaded to the lookaside cache.

See the Package Maintenance Guide section of the Packaging Guidelines, for some brief documentation regarding the lookaside cache (although the packaging docs' coverage of this topic could be better, IMHO):

https://docs.fedoraproject.org/en-US/package-maintainers/Package_Maintenance_Guide/#_upload_new_source_files_to_the_lookaside_cache

Thread in which I discovered these oversize repos, and where the proper handling of `assets.go` was discussed: 

https://discussion.fedoraproject.org/t/package-repo-surgery-due-to-checked-in-source-archive/138772


Reproducible: Always

Comment 1 "FeRD" (Frank Dana) 2024-12-15 16:14:44 UTC
I've filed a releng ticket asking for their assistance in cleaning up the repo's history, that ticket is:

https://pagure.io/releng/issue/12488

Traditionally it's been deemed necessary for releng to seek FESCO approval before performing any such cleanup, so there are a few steps to the whole process and it may take a bit of time.

Comment 2 W. Michael Petullo 2025-02-20 17:31:31 UTC
Mea culpa. I missed this. I agree with Frank's suggestions.

Comment 3 Adam Williamson 2025-12-02 01:49:06 UTC
This message is a reminder that Fedora Linux 41 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 41 on 2025-12-15.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '41'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 41 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 4 W. Michael Petullo 2025-12-02 14:10:52 UTC
Changed version to Rawhide so the bug does not hit EOL before the releng ticket referenced above is completed.

Comment 5 Aoife Moloney 2026-08-17 13:20:03 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 45 development cycle.
Changing version to 45.


Note You need to log in before you can comment on or make changes to this bug.