Please branch and build corosync in epel8 and epel9. If you do not wish to maintain corosync in epel8 and epel9, or do not think you will be able to do this in a timely manner, the EPEL Packagers SIG would be happy to be a co-maintainer of the package; please add the epel-packagers-sig group through https://src.fedoraproject.org/rpms/corosync/addgroup and grant it commit access, or collaborator access on epel* branches.
Note: this is part of the High Availability add-on in RHEL, so it's eligible for EPEL per the policy.
Hi, I don't really understand the purpose of having corosync in EPEL when it is already included in RHEL (and because we are rebasing regularly it is usually same version as upstream)?
corosync is shipped in the High Availability channel, not in RHEL proper. EPEL packages can't depend on HA (as EPEL itself can and does conflict with it and other layered products): https://docs.fedoraproject.org/en-US/epel/epel-policy/#_policy We need this package in EPEL as it's a BR for asterisk, which we're trying to get branched in EPEL 8 and EPEL 9.
Ok. I have to discuss this more deeply with other stake holders. I mean, for me it's really not a problem at all (at the end of the day, it's yet another git rebase/git push/fedpkg build, pretty easy), but I'm not sure if it is really valid to include corosync in epel or not - but I understand use case now. Just please keep in mind, at least libqb and kronosnet had to be compiled as well. I think both are in ha only, and not in epel/baseos.
Just noticed "depends on" knet bug (closed) was set. Are you sure knet is really already included in epel?
I have an update up for kronosnet: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-8ac3351db4
Perfect. No matter what, I will find out if we can/should have a corosync in the epel and let you know. Otherwise we can also create something similar - corosync-epel. I'm just wondering, is full corosync needed or just corosynclib (and -devel)? If so, are we expecting to have full cluster? With fencing? Maybe the solution could be to turn off clustering support in asterisk?
Thanks! Asterisk uses corosync for https://wiki.asterisk.org/wiki/display/AST/Corosync (which is what pulls in this dependency) and that needs a full corosync cluster. I don't think it needs fencing, but I'm not positive about that. Turning off clustering support in the package is possible, but I do have a usecase that will require res_corosync specifically in the coming months, so I'd rather get it sorted out properly now.
Will you be able to branch and build corosync in epel8? For epel9 we will need to wait for #2121845 first. Thanks!
Hi Davide, I still haven't got the answer from appropriate people, but I'm pretty sure it's not going to be branch - rather it's going to be corosync-epel. Also we have to make sure it properly conflicts with "corosync" package (btw. this is something we should add to kronosnet-epel too) and it needs a bit more test. I will try to let you know hopefully this week and maybe prepare spec.
Sounds good, thanks for the update Jan!
Davide: just an update. I still haven't got answer from my manager (he is on PTO currently) but I hope I will get some answer this week.
@dcavalca Ok, so finally got some answer - corosync is definitively not going to move (at least in RHEL8/9) so we must go corosync-epel path. It looks like you already know the process of asking for new package - could you please ask for new package (corosync-epel) and give me commit/manage permissions then? I will try to prepare spec and send it for your review. My idea is to create package which is 1:1 copy of RHEL, just different name and conflicts with corosync (btw. can you please make similar change to knet package - I don't think it conflicts with rhel knet AND libnoozle - required by corosync - is same name as package in rhel). Also I was thinking about asterisk package itself - I think it make sense to depend on either corosync or corosync-epel or (directly) on /usr/sbin/corosync so users which has HA repo enabled can use official corosync and other users can use corosync-epel - what do you think? Does it make sense to you?
Oh, and my name for fedora is "honzaf" - https://accounts.fedoraproject.org/user/honzaf/
> @dcavalca Ok, so finally got some answer - corosync is definitively not going to move (at least in RHEL8/9) so we must go corosync-epel path. It looks like you already know the process of asking for new package - could you please ask for new package (corosync-epel) and give me commit/manage permissions then? I will try to prepare spec and send it for your review. $ fedpkg request-repo --exception corosync-epel https://pagure.io/releng/fedora-scm-requests/issue/47805 > My idea is to create package which is 1:1 copy of RHEL, just different name and conflicts with corosync (btw. can you please make similar change to knet package - I don't think it conflicts with rhel knet AND libnoozle - required by corosync - is same name as package in rhel). My understanding is that EPEL packages shouldn't conflict with RHEL. For cases like corosync (and kronosnet), the -epel package needs to provide all subpackages that aren't shipped in RHEL (and obviously needs to be built from the same sources or things won't work properly). That's what I did for kronosnet-epel: https://src.fedoraproject.org/rpms/kronosnet-epel/commits/epel8-next (this is the -next branch because RHEL and CS aren't in sync at the moment, but the point stands) > Also I was thinking about asterisk package itself - I think it make sense to depend on either corosync or corosync-epel or (directly) on /usr/sbin/corosync so users which has HA repo enabled can use official corosync and other users can use corosync-epel - what do you think? Does it make sense to you? Yeah this seems reasonable.
Oh actually, this is different from kronosnet. In the case of kronosnet, some subpackages are shipped in RHEL/CentOS proper, so those are the ones we have to drop from the -epel package. For corosync, I believe all packages are shipped in the HA channel. In that case, I don't think we need a -epel package at all, it should be enough to branch corosync itself for epel8 and epel9.
(In reply to Davide Cavalca from comment #16) > Oh actually, this is different from kronosnet. In the case of kronosnet, > some subpackages are shipped in RHEL/CentOS proper, so those are the ones we > have to drop from the -epel package. For corosync, I believe all packages > are shipped in the HA channel. In that case, I don't think we need a -epel I think corosync-vqsim is shipped in CRB. What is actually mistake but ... > package at all, it should be enough to branch corosync itself for epel8 and > epel9. Yup, but please go with -epel path anyway. It would be quite a big trouble if somebody with HA repo enabled would enable also epel and (potentially) get unsupported package (because epel may get newer version sooner).
Ok corosync-epel has been created and you're an admin on it. I've requested branched for epel8 and epel9: $ fedpkg request-branch epel8 https://pagure.io/releng/fedora-scm-requests/issue/47808 $ fedpkg request-branch epel9 https://pagure.io/releng/fedora-scm-requests/issue/47809
@dcavalca sorry for a bit delay, but I was really hardly scratching my head with this. Let me explain where I see the problem(s). 1. https://docs.fedoraproject.org/en-US/epel/epel-policy/#_policy - says it very strictly - "Packages which are known to be in other Red Hat Enterprise Linux channels should be maintained in a similar fashion to limited architecture packages. The package should be gotten from the upstream (either ftp.redhat.com for RHEL-6 or git.centos.org for RHEL-7) and maintained with a NEVR less than that of the Red Hat Enterprise Linux release." 1.1 - "limited architecture packages." should be maintained as "missing built sub-packages" according to https://docs.fedoraproject.org/en-US/epel/epel-policy-missing-sub-packages/ 1.2 - "and maintained with a NEVR less than that" (I think NEVR should be NVR) - this is not the case for kronosnet 2. https://docs.fedoraproject.org/en-US/epel/epel-packaging-examples/#_rename_release - Is (I think) completely wrong for our use case because of "Requires: %{rhel_name}%{?_isa} = %{version}-%{rhel_release}" - user without HA channel doesn't have access to "%{version}-%{rhel_release}" (RHEL_RELEASE is the important part) 3. https://docs.fedoraproject.org/en-US/epel/epel-policy/#policy_for_conflicting_packages - "EPEL packages can conflict with packages in other RHEL channels." - I'm not sure if explicit conflict is meant or implicit one Basically let's say I will follow https://docs.fedoraproject.org/en-US/epel/epel-packaging-examples/#_missing_but_built_spec_examples - this will create "corosynclib-NVR.rpm", NOT corosynclib-epel (expected). Now, let's say I will follow https://docs.fedoraproject.org/en-US/epel/epel-packaging-examples/#_rename_release I will get NVR smaller than RHEL, but corosync (for example) is rebased quite often so to keep NVR really less than RHEL for ALL RHEL customers I would need to either package oldest still supported Z-Stream (so 8.2 for SAP or 8.4 if we ignore SAP) or change VERSION (I've tested to add 0.). Or to explain it from other side. I think it is needed to somehow achieve that corosynclib (for example, but knet is similar story) is NEVER updated to epel version once customer has HA repo enabled. And with described example it is virtually impossible once user has Z-Stream and corosync was rebased in Y-Stream (so we probably update it in EPEL too). I've tried two ways (included in following attachments) how to achieve it: 1. Use VERSION beginning with 0. (ugly, probably not correct but always working) 2. Ignore example and call all subpackages "-epel" + add conflicts - I like this solution a bit more Not sure which way to go (and maybe you have some better one - I'm open to ideas), but we have to sync with knet too - current situation where knet subpackages are named same and have same version as latest RHEL Y-stream one is really very unsafe (doesn't work for Z-Stream because subpackage gets updated making corosync use newer library which may cause problem). Ideas, opinions?
Created attachment 1916004 [details] EPEL 8 file with version prefixed with 0.
Created attachment 1916005 [details] EPEL 8 file with all subpackages prefixed -epel
Oh, and scratch build of all subpackages prefixed is https://koji.fedoraproject.org/koji/taskinfo?taskID=92621218
Thanks for clarifying Jan. Within EPEL, I believe this issue was previously discussed in https://pagure.io/epel/issue/102 which is what led to the currently policy. I will bring this up again in the EPEL Steering Committee meeting tomorrow to see if we can come up with a better solution.
Davide: Thanks - looking forward for committee says. No matter what, if last sentence from https://fedoraproject.org/wiki/EPEL/FAQ#Does_EPEL_replace_packages_provided_within_Red_Hat_Enterprise_Linux_or_layered_products.3F ("EPEL will coordinate with other channels/products to minimize any conflicts, but may replace or cause issues with other channels.") applies then it's probably really easiest to follow current policy/example and simply ignore the fact that some subpackages may "break" users with HA channel subscription.
This was discussed in today's EPEL meeting. Your conclusion: > Thanks - looking forward for committee says. No matter what, if last sentence from https://fedoraproject.org/wiki/EPEL/FAQ#Does_EPEL_replace_packages_provided_within_Red_Hat_Enterprise_Linux_or_layered_products.3F ("EPEL will coordinate with other channels/products to minimize any conflicts, but may replace or cause issues with other channels.") applies then it's probably really easiest to follow current policy/example and simply ignore the fact that some subpackages may "break" users with HA channel subscription. is correct -- by design, packages in EPEL are allowed to duplicate what's in RHEL channels that aren't baseos/appstream/crb, so doing something like what I've done for kronosnet here would work and be compliant. There are a couple of options you could potentially look into to mitigate chance of conflict: - you could add an Epoch to the package in the HA channel, so that it always wins the version race - as you suggested, you could prefix 0. to the version for the -epel package (however, this might be confusing for users, and is somewhat in conflict with https://docs.fedoraproject.org/en-US/packaging-guidelines/Versioning/) - on the client side, you could use vendor pinning in DNF (i.e. set allow_vendor_change=False in dnf.conf), which would prevent packages from being replaced from another channel after the first installation - on the client side, you could use includepkgs (or excludepkgs) in dnf.conf to ensure only one set of these packages is made available It's worth noting that these won't prevent conflicts in the general scenario, as there might well be other packages in HA that end up conflicting with or being replaced by packages in EPEL. On the EPEL side, we're going to add a FAQ entry to our docs to hopefully make this clearer for the next person that has to deal with a similar situation.
Prefixing the release instead of the version could be another option, which is used at https://docs.fedoraproject.org/en-US/epel/epel-packaging-examples/#_rename_release - which might maybe be less confusing compared to a version prefix (and yes, still requires maybe relaxed sub-package dependencies depending on the specific case).
Robert: Kind of, but the problem is with rebases - because (IMHO) it make sense to keep epel package in line with latest Y-Stream. What mean if Y-Stream is rebased, EPEL is rebased and Z-Stream users may get newer version simply because version was increased and it has higher priority than release.
IIRC, EPEL does not support Z-Stream, only the latest Y-Stream.
EPEL targets compatibility with the *latest* Z-Stream, a.k.a. the current RHEL release. There are multiple Z-Streams. The non-latest ones are what you probably know as EUS. Y-Stream is now effectively CentOS Stream (which happens to be the origin of the name). What I believe Jan is describing is if a corosync rebase to a new version has happened in CentOS Stream HA/RS channels, and the same rebase is delivered to EPEL without waiting for it to be released in RHEL HA/RS. This would result in a mismatch between corosynclib/corosync-vqsim (appstream/crb) and corosync/corosynclib-devel (epel). It probably makes more sense to keep the epel packages in lockstep with the released RHEL corosynclib/corosync-vqsim packages. If a significant rebase in CentOS Stream causes incompatibility, corosync-epel can be rebased similarly in epel-next.
(In reply to Robert Scheck from comment #28) > IIRC, EPEL does not support Z-Stream, only the latest Y-Stream. Yup, that's the point. Imagine user of 8.2 ZStream/EUS with enabled EPEL in the time of Y-stream being 8.8. Such user will get newest corosynclib/corosync-vqsim and corosynclib-devel via updates by default (but not corosync package itself, because main package is postfixed with -epel), that is what I was afraid of and why I've suggested to postfix -epel to ALL of the subpackages, not only to main package - or use "0." prefix of "Version" (pretty ugly but sadly "Release" is not enough because of possible rebases in latest Y-Stream (let's say 8.8)).
We wouldn't postfix the main package with -epel in that scenario, just the source package. All the binary subpackages would be named the same, with the exception of corosync-vqsim, which would be dropped as it's already provided in CRB. This way there's no conflict with the main repos (and this is the approach I've used in kronosnet).
(In reply to Jan Friesse from comment #30) > […] Imagine user of 8.2 ZStream/EUS with enabled EPEL in the time of Y-stream being 8.8. […] From my point of view EPEL does not support EUS. Such a user needs then to use excludepkgs/includepkgs in dnf or has otherwise to ensure to avoid installing breaking EPEL packages on its own, because with EUS and EPEL, issues can occur everywhere, especially when RHEL rebases a package with ABI bumps/incompatibilities (which practically happens every now and then) in Y-stream. And personally, EUS-using systems are IMHO kind of "stale", because in such environments any update is reviewed in depth and tested on customer side.
> From my point of view EPEL does not support EUS. This is correct. I thought we had this documented somewhere but I can't seem to find it now. For example, when a Qt rebased from 5.12 to 5.15 in RHEL 8.5, EPEL packages were rebuilt to be compatible with RHEL 8.5, making them uninstallable on RHEL 8.4. Many other EPEL packages do continue to work on older Z-Stream releases, but that is just a side effect of RHEL's overall stability and not a goal for EPEL. There are snapshots of EPEL that are taken when a new minor version of RHEL comes out, which Z-Stream users can use to get packages which in theory should work with their specific minor version (but with no updates). https://dl.fedoraproject.org/pub/archive/epel/
Carl, Robert and Davide: Thank you for your information - it was really helpful and I found out I've (almost completely) misunderstood the EPEL process. But after rereading your comments (and guide) again I think I'm now able to understand that process. Maybe a bit more confusing (for me) was the fact, that (I think) epel process basically describes situation when subpackages are not shipped and main package is. The corosync case is exact opposite - we ship vqsim and corosynclib (double thank you Carl to notice this, I was of opinion that only vqsim is shipped!) but not main package. This also made a bit more complicated to name corosync source as corosync-epel and main package as corosync - so I've converted main package to subpackage and main package is now empty. Anyway, included (next comment) is corosync-epel.spec - I would be really greatfull for review/comments. Scratch build is https://koji.fedoraproject.org/koji/taskinfo?taskID=92951923 Davide: Just to make sure, epel9 is buildable now or we are still waiting for libqb?
Created attachment 1917597 [details] corosync-epel.spec for review
Created attachment 1917601 [details] Fixed corosync-epel.spec for review I've forgot to move "requires" to correct place (new corosync subpackage). Scratch build is https://koji.fedoraproject.org/koji/taskinfo?taskID=92952563
epel9 is still blocked on libqb being added to CRB, see https://bugzilla.redhat.com/show_bug.cgi?id=2121845 The spec you attached lgtm, with one exception: you're disabling vqsim support, which leads to the package being build without --enable-vqsim. This is a deviation from the RHEL package, I'd recommend instead enabling vqsim normally and just not shipping that subpackage.
Davide: Thank you for the review. I've fixed vqsim problem and pushed + built (https://koji.fedoraproject.org/koji/taskinfo?taskID=92980551) + updated (https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-1a8ade9914) package so for rhel8 we are ready now. I'm also pre-preparing rhel9 spec so once we get libqb into CRB and you build the knet I can just build corosync. Please let me know when libqb (kronosnet) for epel9 is ready so I can trigger the build (and close this BZ then). Regards, Honza
Davide: Just noted comment about removing manpages because of doxygen2man in epel9 - doxygen2man is part of libqb so once we ship libqb it may make sense to ship also doxygen2man ... same applies if we decide to make libqb-epel.
Thanks! After discussing this with bstinson yesterday, I'm just going to ship libqb in EPEL 9 for now the same way, as it sounds like it will take a while to get it sorted out properly.
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-22fa6cf82b and https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-NEXT-2022-e58e072f55 should take care of libqb for the time being.
@dcavalca Any news about kronosnet build for EPEL 9? I've seen libqb is now ready but (tested scratch build today) there is still "No matching package to install: 'libnozzle1-devel'"
Working on it. Because the RHEL 9 and CentOS Stream 9 versions of this differ (as they do for libqb), we will need to build this in both epel9 and epel9-next, and I need to fix epel9-next build of libqb first as I just realized it's not installable.
https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-NEXT-2022-c76ac71b3b is the epel9-next update for kronosnet-epel, which includes the fixed libqb-epel
And https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-c83906f47f is the epel9 one
FEDORA-EPEL-2022-61fffa3ec4 has been submitted as an update to Fedora EPEL 9. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-61fffa3ec4
Davide: Perfect. Today I was able to build epel9 package https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-61fffa3ec4 (and referenced this BZ in update so it should close when we package get into stable).
FEDORA-EPEL-2022-61fffa3ec4 has been pushed to the Fedora EPEL 9 testing repository. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-61fffa3ec4 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-EPEL-2022-61fffa3ec4 has been pushed to the Fedora EPEL 9 stable repository. If problem still persists, please make note of it in this bug report.