Looks like two buildrequires are missing from epel9 BR -- (python-libcloud and python-mock).  I requested branching to epel9 for python-libcloud[1]. Python-mock is now a deprecated package since mock is now included in the python standard libraries[2].


[1] - https://bugzilla.redhat.com/show_bug.cgi?id=2104639
[2] - https://fedoraproject.org/wiki/Changes/DeprecatePythonMock

Wondering what is driving this request since the last time Salt was built for EPEL was EPEL-6 ?
There was no Salt in EPEL-7 or EPEL-8, hence why this need for EPEL-9 now ?

The main driver for this is Salt's new Tiamat build system.  New packages built with this system will not support 3rd party python modules installed via RPM.  Some user sites may not be able to accommodate installing 3rd party python modules via pip etc.

Will bring the issue up in the next core meeting for Salt, as that is a fair point which should be addressed.

After discussion within the core team for Salt Project it has been decided that Salt will not provide Tiamat based Salt or classic rpm packages to EPEL-9, or any other EPEL version.

However the source code is open-sourced and anyone is welcome to fork and provide their version of Tiamat based Salt, similar to how Debian/Ubuntu, and SUSE currently build Salt packages for their platforms.


Tiamat based Salt packages cannot use system provided Python packages at present. It would be difficult to achieve such given that some platforms have versions of Python which are even EOL, for example: RHEL 7 is using Python 3.6 which is EOL and given that the Tiamat based Salt packages are internally using Python 3.9.13, this would prove difficult to utilize and match system packages.
Noting that there is no system Python 3.9 packages on RHEL 7, and Python 3.9.13 is the last maintenance fix for Python 3.9.
At some point in the future, a newer version of Python 3 will probably be utilized, for example: if the rumored performance improvements in Python 3.11 are real when it is released.

Tiamat was developed to allow Salt to use it's own version of Python internally to over-come such issues, allowing Salt to utilize currently supported versions of Python and to ensure security fixes are up to date.  It has also allowed support for older platforms which currently do not support recent Python 3 versions and provide for UTF-8 support, examples of which are Solaris 10, Arista routers which only had Python 2.7 and 'C' language environments.

Salt Project understands that customers may not be allowed to directly install Python packages from PyPI. But similar to larger customers rsync'ing their own copy of Salt repositories, I have heard of them having their own internal versions of PyPI with approved and vetted Python packages, which they should be able to install from.  Hence the inability to utilize system Python packages should not be an issue given work-arounds, that may already be in place.

As to 
```
I would be happy to be a co-maintainer of the package (FAS rcallicotte);
please add me through https://src.fedoraproject.org/rpms/salt/adduser
```

Please apply through the regular channels, not in my abilities to approve etc.

https://src.fedoraproject.org/rpms/salt lists dmurphy18 and krionbsd as admins for this repo.  All that is needed at this point is for one of you to log in and add rcallicotte as a co-maintainer.

I am fall back reserve, as in, bus rule, someone get hit by a bus, I get to enter the field of play.
Krionbsd is on the SRE Team and person in charge, but he will probably not be available till middle of next week, due to unforeseen circumstances.  There is also the discussion of contributor vs co-maintainer etc. but that is for that team.
So asking for patience until krionbsd is available to address the issues.

I have opened a PR againsts Rawhide: https://src.fedoraproject.org/rpms/salt/pull-request/2

Per EPEL Packaging Guidelines[1]:

Will you be able to branch and build salt in epel9?
I would be happy to be a co-maintainer if you do not wish
to build it on epel9 (FAS: rcallicotte).


[1] - https://docs.fedoraproject.org/en-US/epel/epel-package-request/#fedora_packagers

Under discussion in core team.

BTW: there is now a Release Candidate for RHEL 9 with the Tiamat based packaging https://repo.saltproject.io/salt_rc/salt/py3/redhat/9/x86_64/latest

This is also open-sourced, please feel free to check it out and report any issues you find.

With the immanent release of Salt 3005 support for RHEL 9, is this issue still relevant ?

Yes this issue is still relevant because the Tiamat build system does not support 3rd party python modules installed via RPM[1].

https://docs.saltproject.io/salt/install-guide/en/latest/topics/upgrade-to-tiamat.html#how-to-upgrade-to-tiamat

David, the existence of EL9 compatible packages in the Salt yum repo is not a blocker for this being branched for EPEL9.  Robby is not asking you to build this in EPEL9, he's offering to do it for you.  This is similar to how puppet is still in EPEL despite Puppet Inc. offering their own yum repos.

Understood that Robby is offering to build the EPEL packages in addition to Salt, however in the past such offers and help has led to issues and expectations with customers after volunteers stop delivering changes, and the expectation that Salt picks up the slack.

The issue is under discussion in the core team still, so don't have an outcome to report yet on the matter.

Saltstack customers misunderstanding EPEL is also not a blocker for this package to be added to epel9.  If Robby is willing to maintain the salt package in epel9, then he should be allowed to do so.  If needed you can restrict his commit access to just the epel9 branch or to a branch pattern such as epel*, but I would encourage granting full access so he can assist with Fedora branch maintenance as well.  If for some reason Robby is unable to maintain the epel9 branch in the future, that branch can be retired at any time.  There is no commitment for the other maintainers to pick up the slack if they are unwilling.

https://docs.fedoraproject.org/en-US/package-maintainers/Package_Retirement_Process/#_epel

So after Salt Core Team discussions the following was the conclusion:

Robby should create EPEL 9 packaging for Salt, independent of Salt, pulling in tagged releases as necessary from Salt on GitHub,  similar to what SUSE and Debian/Ubuntu currently do, that he and whoever else is interested, maintain, independent of Salt.  
Noting that the resultant packages would not be signed with Salt's public/private keys (current SALTSTACK-GPG-KEY.pub are SHA-1 based, they won't work with RHEL 9 since it dropped support for SHA-1). Probably except the packages would be signed with EPEL-9 keys.

Salt Project is moving away from using classic rpm packaging to a self-contained package which includes all of the needed dependencies, since this allows Salt Project to support older platforms with system Python versions which are End-Of-Life, like Python 3.6 on RHEL 7 & 8.  And in the future allow for newer versions of Python 3 too on older or current platforms which don't support that version of Python 3 (hoping Python 3.11 is as speedy as rumors state).  Having to rely on system python has stopped Salt from moving forward, for example: the need to support Debian 9 which used Python 3.5, hence not use f-strings, thankfully Debian 9 is EOL, but similar occurrences happened in the past, on only has to remember RHEL 6 and Python 2.4.

Understandably EPEL-9 doesn't have these limitations. Hence creating a separate build of Salt packages for EPEL-9 independent of Salt Project is probably the best process for EPEL-9.

Sorry for delays in getting answers on this, but eventually got there.

> Robby should create EPEL 9 packaging for Salt, independent of Salt, pulling in tagged releases as necessary from Salt on GitHub,  similar to what SUSE and Debian/Ubuntu currently do, that he and whoever else is interested, maintain, independent of Salt.

This is what Robby is asking to do.  Please add him (rcallicotte) with commit, admin, or collaborator permissions here:

https://src.fedoraproject.org/rpms/salt/adduser

I must be missing something in my understanding.

I am saying go and create Salt for EPEL-9 independently, in its own project / branch etc., that is, without input from Salt Project.

Is this a problem with Fedora infrastructure that there can be only one Salt, and hence he needs to have to be added here instead of its own project/branch.  I mean it is going to be under EPEL repository, which Salt Project is not going to utilize.

Noting that Salt Project will be using onedir (Tiamat just got renamed today - talk tomorrow in Community Open Hours) packaging for Fedora and not regular style rpm packaging (it will be an rpm containing a onedir with Salt, Python and all dependencies self-contained in that one directory - hence the name).

From my recall (when EPEL was independent), there should be no overlap and both Salt projects should be capable of existing, or has everything changed since EPEL no longer became independent. If this so, then have an issue.

Looks like EPEL is no longer the old EPEL

https://docs.fedoraproject.org/en-US/eln/

so everything is coming out of rawhide :(, well that is going to be an issue for Salt Project.

Will need to revisit issue with Salt Core Team, if the above is correct, can you confirm Carl George.
Want to make sure I have it correct before reopening the issue with the Team

ELN is the "Enterprise Linux Next" project and is not directly related to EPEL:

https://docs.fedoraproject.org/en-US/epel/

As far as I know, the EPEL project operates the same way it always has.

> I am saying go and create Salt for EPEL-9 independently, in its own project / branch etc., that is, without input from Salt Project.

EPEL package sources are just different branches of Fedora package sources.  To add salt to EPEL9, a maintainer of https://src.fedoraproject.org/rpms/salt must request an epel9 branch, add whatever commits are appropriate there (usually by merging a Fedora branch such as rawhide), then create a koji build and bodhi update.  If the existing maintainers are unwilling to do that work, then add a co-maintainer that is willing.  Blocking this is not really an option, other than an agreed technical reason such as a dead upstream or the package already existing in RHEL.

> Is this a problem with Fedora infrastructure that there can be only one Salt, and hence he needs to have to be added here instead of its own project/branch.

It's not a problem, that's just how it's structured.  Fedora and EPEL packages must be built from https://src.fedoraproject.org/rpms/<package name>.  They can't be built from arbitrary forks or from other namespaces.

> Noting that Salt Project will be using onedir (Tiamat just got renamed today - talk tomorrow in Community Open Hours) packaging for Fedora and not regular style rpm packaging (it will be an rpm containing a onedir with Salt, Python and all dependencies self-contained in that one directory - hence the name).

Packaging salt in this way is not compatible with Fedora packaging guidelines.  Tiamat/onedir style is not allowed based on multiple sections it would conflict with.

https://docs.fedoraproject.org/en-US/packaging-guidelines/#_filesystem_layout
https://docs.fedoraproject.org/en-US/packaging-guidelines/#bundling

> From my recall (when EPEL was independent), there should be no overlap and both Salt projects should be capable of existing, or has everything changed since EPEL no longer became independent. If this so, then have an issue.

EPEL has never been independent of Fedora.  It started in Fedora and remains in Fedora.

If you and Kirill are only willing to maintain tiamat/onedir style packages, that will have to be done in the Saltstack repo, not in Fedora/EPEL.  At that point, it may be worth considering completely turning the Fedora/EPEL package maintenance over to the community via the orphan process.

https://docs.fedoraproject.org/en-US/package-maintainers/Package_Orphaning_Process/

Thanks for the clear response Carl to my questions.

I am no longer responsible for packaging Salt.

I thought we might have some issues with Tiamat/onedir due to Source issues pulling in all those dependencies and such.  Understand the need to maintain classic rpm style packages, esp. given use of fedxxxxx tools in the past to build Fedora, and the need to continue to do so in the future which are somewhat contradicted by tiamat/style packages.

I will bring the issues up again with the core team, however it will probably be next week before I have answers, as about to go on PTO.

I filed a stalled EPEL package ticket for this at https://pagure.io/releng/issue/10931

No problem, asked to hold off as discussing this on Monday morning, since on vacation.

I'm a bit confused about why the Salt Core Team's closed door decisions are really relevant here. Fedora and the EPEL subproject are independent community entities that are not affiliated with or governed by Salt.

When someone submits an EPEL branch request, the EPEL policy allows for three possible outcomes:

1. The current package maintainers have interest in maintaining the package in EPEL and they branch it and perhaps add the requestor as a co-maintainer for that branch if both parties are interested.
2. The package maintainers provide a compelling reason why the package should not be built. python-mock which Robby brought up is a good example. The package was already deprecated in Fedora, so the maintainers chose not to branch it for EPEL 9 and instead helped packagers migrate to the unittest.mock module from the Python stdlib.
3. The package maintainers are disinterested in maintaining the package in EPEL.
3a. If the requestor does not offer to step up themself, nothing happens.
3b. If the requestor offers to maintain the package, the maintainer should give them the needed permissions. If the maintainer is non-responsive or only responds along the lines of "I don't want to maintain this in EPEL," the requestor can invoke the Stalled EPEL Requests procedure after waiting the prescribed amount of time.

So really, the Fedora salt package maintainers just have to decide whether or not they wish to personally maintain the package in EPEL 9. Based on the previous responses, it seems that answer is no.

Hopefully, that clears up any potential confusion regarding the branching policy. The EPEL SIG is working on clarifying our policy documents[1], but time is a limited resource :).

[1]: https://docs.fedoraproject.org/en-US/epel/epel-policy/

Adding Robby to salt now

Thank you, David. I've assigned this to Robby. I agree with Carl and also "encourage granting full access so he can assist with Fedora branch maintenance as well." This would be either the commit or admin ACL. This way, packaging improvements can be easily synced between Fedora EPEL and Fedora. Robby should still be able to branch the package for EPEL 9 with the permissions he has, though.

Updated Robby to 'commit' ACL.  Currently going through Admin assignments since the current main admin is leaving the company, hence my backup duties just got activated. Hopefully things will settle down and new main admin will be assigned.

Thank you David.

This bug appears to have been reported against 'rawhide' during the Fedora Linux 37 development cycle.
Changing version to 37.

Why this change , the title says it all, EPEL 9, which would be getting taken from Rawhide for branch etc., and from the discussion thread above, will still be doing the classic Fedora rpms for Salt, which includes 36, 36, etc.  Hence unless there is some other reason Ben, can you change this back to Rawhide.

FEDORA-EPEL-2022-84e9baa913 has been submitted as an update to Fedora EPEL 9. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-84e9baa913

FEDORA-EPEL-2022-84e9baa913 has been pushed to the Fedora EPEL 9 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-84e9baa913

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-EPEL-2022-84e9baa913 has been pushed to the Fedora EPEL 9 stable repository.
If problem still persists, please make note of it in this bug report.