Bug 1570619

Summary: wrong data about 2001:470:71::/48 subnets
Product: [Fedora] Fedora Reporter: Dominik 'Rathann' Mierzejewski <dominik>
Component: GeoIP-GeoLite-dataAssignee: Paul Howarth <paul>
Status: CLOSED CANTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: carl, paul
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-22 13:41:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dominik 'Rathann' Mierzejewski 2018-04-23 11:00:10 UTC
Description of problem:
The IPv6 subnets 2001:470:71::/48 are incorrectly listed as located in the US, while in reality they're allocated in different EU countries, e.g.:
2001:470:71:1::/64 - PL
2001:470:71:ff::/64 - UK
2001:470:71:f00::/64 - UA

Version-Release number of selected component (if applicable):
GeoIP-GeoLite-data-extra-2018.04-1.fc27.noarch

How reproducible:
Always.

Steps to Reproduce:
1. $ geoiplookup6 2001:470:71:f00::

Actual results:
GeoIP Country V6 Edition: US, United States
GeoIP ASNum V6 Edition: AS6939 Hurricane Electric LLC
GeoIP City Edition V6, Rev 1: US, N/A, N/A, N/A, 37.750999, -97.821999, 0, 0

Expected results:
Correct country location should be listed.

Additional information:
This causes Fedora mirrormanager to provide far-away mirror URLs instead of local ones.

Comment 1 Paul Howarth 2018-04-23 11:09:52 UTC
What do you propose we do about this? There will be no further releases from upstream so there's no point raising it there.

Comment 2 Dominik 'Rathann' Mierzejewski 2018-04-23 12:49:52 UTC
(In reply to Paul Howarth from comment #1)
> What do you propose we do about this? There will be no further releases from
> upstream so there's no point raising it there.

In this case I can see only two options:

1. Fork upstream data and patch. I'd happily contribute a patch, but the source data seems to be binary. Are there any tools to manipulate it?
2. Stop shipping wrong data.

Comment 3 Dominik 'Rathann' Mierzejewski 2018-04-23 13:16:32 UTC
Or maybe you could switch to the new, still developed databases: https://dev.maxmind.com/geoip/geoip2/geolite2/ 
It looks like the new software is already packaged in Fedora: libmaxminddb . The new databases seem to have accurate data about the IPv6 subnets in question.

Comment 4 Dominik 'Rathann' Mierzejewski 2018-04-23 13:18:37 UTC
(In reply to Dominik 'Rathann' Mierzejewski from comment #3)
> Or maybe you could switch to the new, still developed databases:
> https://dev.maxmind.com/geoip/geoip2/geolite2/ 
> It looks like the new software is already packaged in Fedora: libmaxminddb .
> The new databases seem to have accurate data about the IPv6 subnets in
> question.

Make that "about some of the IPv6 subnets in question". The first two are still listed as located in the US, but the Ukrainian one is correctly identified.

Comment 5 Paul Howarth 2018-04-23 14:27:52 UTC
The GeoIP-GeoLite-data package is providing data to go with the GeoIP library.

Switching from GeoIP to libmaxminddb is certainly the way to go, but that's a different API and, as you say, requires different databases, so those databases should be packaged separately from GeoIP-GeoLite-data. It also needs the upstream library consumers to be changed to use the different API.

Comment 6 Carl George 2018-04-23 23:15:51 UTC
Thanks Paul for your work on geoip related things so far.

> so those databases should be packaged separately from GeoIP-GeoLite-data

Do you know if that work has been started by anyone?  If not I may take a stab at it and submit it for review.

Comment 7 Paul Howarth 2018-04-24 06:13:15 UTC
(In reply to Carl George from comment #6)
> Thanks Paul for your work on geoip related things so far.
> 
> > so those databases should be packaged separately from GeoIP-GeoLite-data
> 
> Do you know if that work has been started by anyone?  If not I may take a
> stab at it and submit it for review.

I'm not aware of any work having been done on this, except for the python-geoip-geolite2 package (Bug #1204447), which included a copy of the GeoLite2 City database, but that package appears to have been retired (https://src.fedoraproject.org/rpms/python-geoip-geolite2/blob/master/f/dead.package).

Comment 8 Carl George 2018-04-24 17:29:21 UTC
Just submitted https://bugzilla.redhat.com/show_bug.cgi?id=1571404.

Comment 9 Carl George 2018-04-25 02:42:15 UTC
Paul, I got the new geolite2 package approved.  The SRPM builds two subpackages, geolite2-city and geolite2-country, corresponding to the database files from Maxmind.  Would you like to co-maintain the package with me?

Comment 10 Paul Howarth 2018-04-25 12:04:43 UTC
(In reply to Carl George from comment #9)
> Paul, I got the new geolite2 package approved.  The SRPM builds two
> subpackages, geolite2-city and geolite2-country, corresponding to the
> database files from Maxmind.  Would you like to co-maintain the package with
> me?

Sure.

Do you plan to update the new package monthly? What I did for GeoIP-GeoLite-data was to update monthly in rawhide/branched, quarterly in stable Fedora releases, and six-monthly in EPEL. The geolite2 package is yours though, so feel free to do more frequent updates if you're sufficiently enthused :-)

Have you branched and built the package for older Fedoras and EPEL?

Comment 11 Carl George 2018-04-25 15:25:13 UTC
Got you added.

I wasn't planning on sticking to a specific schedule.  I usually update my packages evenly across all branches whenever possible.  How about aiming for quarterly for all branches?

Branched, built, and submitted for el6, el7, f27, and f28:
https://bodhi.fedoraproject.org/updates/?packages=geolite2

Comment 12 Carl George 2018-04-25 16:03:22 UTC
Dominik, regarding the original point of this issue, I looked up those subnets using mmdblookup (from libmaxminddb-devel) and the GeoLite2-Country.mmdb database in my new geolite2-country package.  One of them is correct now, but the other two are still listed as "United States".

    $ mmdblookup --file /usr/share/GeoIP/GeoLite2-Country.mmdb --ip 2001:470:71:1:: country names en
    
      "United States" <utf8_string>
    
    $ mmdblookup --file /usr/share/GeoIP/GeoLite2-Country.mmdb --ip 2001:470:71:ff:: country names en
    
      "United States" <utf8_string>
    
    $ mmdblookup --file /usr/share/GeoIP/GeoLite2-Country.mmdb --ip 2001:470:71:f00:: country names en
    
      "Ukraine" <utf8_string>

You could report the error upstream to MaxMind, but I'm sure their response will be to switch to their paid GeoIP2 database.

> GeoLite2 databases are free IP geolocation databases comparable to, but less accurate than, MaxMind's GeoIP2 databases.

- https://dev.maxmind.com/geoip/geoip2/geolite2/

Comment 13 Paul Howarth 2018-05-22 13:41:17 UTC
I'm going to close this because there's nothing I can do about it.

Users of this package need to get their applications moved over to libmaxminddb/geolite2.

Comment 14 Dominik 'Rathann' Mierzejewski 2018-05-22 21:49:15 UTC
Agreed, thanks for your time.