Bug 1156007 - [rfe] info about fedora in http header
[rfe] info about fedora in http header
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: dnf (Show other bugs)
rawhide
Unspecified Unspecified
low Severity unspecified
: ---
: ---
Assigned To: Jaroslav Rohel
Fedora Extras Quality Assurance
: FutureFeature, Reopened, Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2014-10-23 07:47 EDT by Matthew Miller
Modified: 2017-06-19 09:54 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-06-19 08:23:00 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Matthew Miller 2014-10-23 07:47:25 EDT
It appears that our three products will be all drawing from the same "Everything" tree on the mirrors, at least for F21 and probably for F22.

With this plan, there is no way for us to easily count how many of which product are in use. It would be nice if dnf would either send a custom http header which we could log to mirror manager, or set the user agent, or something, based on the presence of fedora-release-workstation, fedora-release-server, fedora-release-cloud — or the absence of any of these.
Comment 1 Honza Silhan 2014-10-31 09:18:53 EDT
Hi, we could do that but we have to solve more urgent bugs now.
Comment 2 Matthew Miller 2014-10-31 13:45:00 EDT
Hi Jan. This is actually fairly high priority for Fedora overall, as this is essential to our overall promotion and marketing strategy. Thanks!
Comment 3 Jan Zeleny 2014-11-03 02:42:00 EST
Hello Matt. I think Jan could have phrased his comment better. Rest assured that we fully recognize the importance of this feature for Fedora and we do plan to deliver it as soon as possible. However, now we need to focus on things that block dnf from being default in Fedora. The justification for this decision is that the benefits of this feature would be significantly reduced without dnf being on as many systems as possible.
Comment 4 Matthew Miller 2014-11-03 08:52:02 EST
Thanks Jan. I'm going to e-mail the yum devel list about coming up with a solution for F21, and then presumably that will need to be ported forward into F22.

Although we'll also want something for software-center-based package management in F21 -- I'm not actually even sure of the details of how that works. Hmmm; where is the best place to coordinate all of this?
Comment 5 Jan Zeleny 2014-11-03 09:03:17 EST
> Hmmm; where is the best place to coordinate all of this?

Let's try the thread you started on yum-devel mailing list. I believe all the stakeholders are subscribed there.
Comment 6 Fedora Admin XMLRPC Client 2016-07-08 05:24:50 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 7 Jaroslav Mracek 2017-05-18 05:09:16 EDT
The bug is here for more then 2 years, therefore probably requested feature is out of scope. Please if the request is still relevant and has high impact, don't hesitate to reopen the bug report.
Comment 8 Matthew Miller 2017-05-18 08:53:40 EDT
This would still be really helpful and I regard it as high priority even though it has not been addressed in two years.
Comment 9 Matthew Miller 2017-05-18 09:07:23 EDT
What I'd like is ID, VERSION_ID, and VARIANT_ID from /etc/os-release.
Comment 10 Stephen John Smoogen 2017-05-18 15:41:16 EDT
I am not sure this is exactly an HTTP header as much as adding additional information to the User Agent String.

24.194.145.233 - - [09/May/2017:04:45:35 +0000] "GET /metalink?repo=updates-released-f24&arch=x86_64 HTTP/1.1" 200 8747 "-" "dnf/1.1.10"

to 

24.194.145.233 - - [09/May/2017:04:45:35 +0000] "GET /metalink?repo=updates-released-f24&arch=x86_64 HTTP/1.1" 200 8747 "-" "dnf/1.1.10 fedora 25 server"

This looks like updating dnf-2.4.1/const.py or updating 
./dnf/repo.py:        self.useragent = dnf.const.USER_AGENT
./dnf/util.py:    handle.useragent = dnf.const.USER_AGENT

to have that additional data in it. I expect it is non-trivial but not impossible?
Comment 11 Jaroslav Rohel 2017-06-01 06:12:01 EDT
> It would be nice if dnf would either send a custom http header which we could log to mirror manager, or set the user agent, or something, based on the presence of fedora-release-workstation, fedora-release-server, fedora-release-cloud — or the absence of any of these.

Maybe better solution is to add requested information to the metalink.
Eg. add '&rel=fedora-server'

How to do this?
You want information based on presence of 'fedora-release-[server,workstation,...]-<version>.noarch.rpm' package. (Or information from '/etc/os-release' file. But content of this file is provided by 'fedora-release-[server,workstation,...]-<version>.noarch.rpm'.)

There is package 'fedora-repos-<version>.noarch.rpm'. But we can use the same logic as above. That means create 'fedora-repos-[server,workstation,...]-<version>.noarch.rpm' packages. These packages will be contain metalinks with &rel=requested_information. 

This solution is DNF (YUM, ...) independent.
Comment 12 Matthew Miller 2017-06-01 07:11:51 EDT
(In reply to Jaroslav Rohel from comment #11)
> Maybe better solution is to add requested information to the metalink.
> Eg. add '&rel=fedora-server'


That could work too. Smooge, what do you think?
Comment 13 Igor Gnatenko 2017-06-01 07:16:29 EDT
@jrohel, it is not only about metalinks, but also for direct baseurl access I suppose...

Matt, what are you trying to achieve?
Comment 14 Matthew Miller 2017-06-01 08:22:24 EDT
(In reply to Igor Gnatenko from comment #13)
> Matt, what are you trying to achieve?

To get a general sense of the popularity of each edition (and possibly spin) in a low-impact way. I'll defer to Smooge on best implementation from the server side.
Comment 15 Stephen John Smoogen 2017-06-01 10:53:23 EDT
One of the big issues that comes up every release is "What editions are being used?" and how much. Knowing this has multiple benefits to Fedora (and various downstreams)

* This is to help allow groups like QA and Release Engineering know which spins, editions, containers, etc. are getting used the most so that they can work out "OK XYZ container isn't building so we need to slip, ABC container didn't build so we don't need to slip"

* This is to help edition groups to get an idea if their edition makes any sense. While no one wants to know if their release is a "flop" they need to know if they want to improve on the next iteration. 

* This needs to be configurable in some way that survives proxy systems and mirrormanager mangling as we get the data from apache logs.

My current mirrormanager scripts do parse the /metalink?repo=updates-released-f21&arch=x86_64 type lines so a release= or container= for however that is done should be possible to add to this. I wasn't sure how it would affect mirrormanager (or CentOS's mirroring system or the RHEL one) so had gone with putting the data in User Agent as that was less likely to tools I could not fix.

Please let me know how I can give better information for this.
Comment 16 Jaroslav Rohel 2017-06-08 07:38:33 EDT
We discussed this internally in the DNF team and we prefer adding the argument to the metalink.

Both sides
- metalink in repo file
- metalink parser on server
are under control of fedora infra/releng.

Other distros don't have to use this at all.

We recommend using 'variant' variable with values such as 'server', 'workstation', etc.
Keep it close to fedora composes.
Comment 17 Matthew Miller 2017-06-08 08:02:27 EDT
Ok, but we would still need DNF support for that. Currently, the metalink line an repo file looks like:

  metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-debug-$releasever&arch=$basearch

Where $releasever and $basearch are filled in by DNF. We would need to add a $releasevariant or something to that.
Comment 18 Jaroslav Mracek 2017-06-19 08:23:00 EDT
Dear Matthew, I would like to explain the solution little bit in more detail. We propose following:

The package fedora-repos will be splitted into 3 packages (fedora-server-repos, fedora-workstations-repos, and fedora-cloud-repos). Each of them will have /etc/yum.repos.d/fedora.repo with modification in metalink, where you can use something like for fedora-workstations-repos:

metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-debug-$releasever&arch=$basearch&releasevariant=workstation 

or for fedora-server-repos: 

metalink=https://mirrors.fedoraproject.org/metalink?repo=fedora-debug-$releasever&arch=$basearch&releasevariant=server 

That means that there will be no additional variable or requirement from dnf site, but only infrastructure solution. And this is exactly what you need, because your request is Fedora specific (cannot be applied for RHEL), therefore it has to be solved by Fedora delivery approach of distro variant. Hope that it helps.
Comment 19 Matthew Miller 2017-06-19 09:42:36 EDT
(In reply to Jaroslav Mracek from comment #18)
> The package fedora-repos will be splitted into 3 packages
> (fedora-server-repos, fedora-workstations-repos, and fedora-cloud-repos).
> Each of them will have /etc/yum.repos.d/fedora.repo with modification in
> metalink, where you can use something like for fedora-workstations-repos:

I'm not automatically opposed to this, but an upgrade path which replaces the repo definition files seems scary.
Comment 20 Matthew Miller 2017-06-19 09:45:22 EDT
Also this is compounded by https://apps.fedoraproject.org/packages/fedora-workstation-repositories, which adds select third-party repositories.
Comment 21 Neal Gompa 2017-06-19 09:54:40 EDT
Why not just add support for custom substitution variables? It's been requested by other folks from time to time.

See bug 1251774 and the libdnf GitHub issue[1].

[1]: https://github.com/rpm-software-management/libdnf/issues/170

Note You need to log in before you can comment on or make changes to this bug.