Bug 729786 - python-dateutil should use system-wide zoneinfo data
Summary: python-dateutil should use system-wide zoneinfo data
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: python-dateutil
Version: 19
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jef Spaleta
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-08-10 19:53 UTC by Petr Machata
Modified: 2015-05-05 01:36 UTC (History)
4 users (show)

Fixed In Version: python-dateutil-1.5-3.fc15
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-18 13:36:53 UTC


Attachments (Terms of Use)
Fix-ish (3.29 KB, patch)
2011-08-17 14:14 UTC, Petr Machata
no flags Details | Diff

Description Petr Machata 2011-08-10 19:53:07 UTC
Description of problem:

python-dateutil installs a tarball with compiled zoneinfo files to Python's site-packages directory.  To actually get the data, the code then extracts the requested zone from the tarball and reads it.  This scheme is problematic to keep up to date (the data in current rawhide is from 2010, the data in my f14 from 2008), and needlessly complicated.  I'd like to propose that files from /usr/share/zoneinfo are instead opened directly.  It shouldn't be necessary to install the tarball at all.

Version-Release number of selected component (if applicable):
1.5-1

How reproducible:
Always.

Steps to Reproduce:
$ python -c 'import dateutil.zoneinfo; print dateutil.zoneinfo.gettz ("America/Bahia_Banderas")'
  
Actual results:
None

Expected results:
tzfile('America/Bahia_Banderas')

Comment 1 Jef Spaleta 2011-08-17 01:32:33 UTC
Good catch.

We've made similar changes to pytz package in the past to use the systemwide tzdata package. I'm surprise that noone caught the need for this in dateutil when pytz functionality was amended.  The timezone functions crop up in a lot of places it seems.


If you have a patch to propose that will do the lookup against systemwide tzdata I'm willing to look at it and submit it upstream. Doesn't have to be perfect, but if you can get a patch that works for you as you expect that would help immensely.



-jef

Comment 2 Petr Machata 2011-08-17 14:14:19 UTC
Created attachment 518687 [details]
Fix-ish

Yeah, it was me the last time around, too.  So this is partial fix.  It honors TZDIR just like pytz does, and gives precedence to system-wide zoneinfo files.  It's currently hard-coded to not distribute the tarball at all, and to disregard it even if it is included.  This would ideally be configurable in setup.py, but that escapes my fu.

Comment 3 Jef Spaleta 2011-08-17 16:12:20 UTC
thanks for the patch. that's a good start for our package needs an upstream discussion.


Give me through the weekend to get some testing packages out with this patch?  Ping this bug again Sunday if you haven't seen me move on it.

Do you have a FAS account? Want to get commit rights for this package?

-jef

Comment 4 Petr Machata 2011-08-30 17:45:21 UTC
Ping?  I've been busy myself, so no progress on this front.

I am Fedora maintainer, but I don't really care much about the package beyond the way it uses zoneinfo data.

Comment 5 Jef Spaleta 2011-09-02 16:16:42 UTC
apologizes.  I got blind sided by a science campaign for the past two weeks.  It never fails...as soon as our equipment is a critical need...something goes wrong and I'm pull 20 hour days trying to keep it operational via remote interface.

I swear on whatever holy book you need me to (as long as its published by Orielly Press) that I'll get to this over the long weekend coming up.

-jef

Comment 6 Jef Spaleta 2011-09-15 17:52:57 UTC
Hey,

thanks for the patience.  I've built a rawhide package with a version of the patch with system tzdata turned on and the tarball excluded in the payload...and with a dep on tzdata.


 http://koji.fedoraproject.org/koji/taskinfo?taskID=3353636

You want to take it for a quick spin and make sure its doing what you expect before I spin up F14-F16 versions of this?

I ran the test locally with reproduce steps from the report on my F15 system.


-jef

Comment 7 Petr Machata 2011-09-15 21:40:38 UTC
It looks good to me, thanks.

I still don't know how to support the proper build-time selection, like setup.py build --with-system-zoneinfo or some such.  I don't think setup.py allows anything like that, the people whom I asked all recommended overriding "build" and "install" actions, which is clumsy, but it may well be the only way.

Comment 8 Jef Spaleta 2011-09-15 22:51:06 UTC
I'll worry about that as part of cleaning it up for upstream consumption.
While its a distro specific patch its not a concern...a patch is a patch is patch.

-jef

Comment 9 Fedora Update System 2011-09-15 23:59:37 UTC
python-dateutil-1.5-3.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/python-dateutil-1.5-3.fc15

Comment 10 Fedora Update System 2011-09-18 00:52:09 UTC
Package python-dateutil-1.5-3.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-dateutil-1.5-3.fc15'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/python-dateutil-1.5-3.fc15
then log in and leave karma (feedback).

Comment 11 Fedora Update System 2011-09-27 23:06:19 UTC
python-dateutil-1.5-3.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 12 Thomas Spura 2012-08-24 07:50:34 UTC
Reopening.

I talked with upstream and the common api is dateutils.tz and NOT dateutils.zoneinfo.

The first is working without the patch and the second one is expected to just load the bundled tarball (or will always fail, when you choose to remove the tarball).

So it would be best to drop the patch again and maybe even ship the bundled tarball...

Comment 13 Thomas Spura 2012-08-24 07:51:09 UTC
The api is described over here:

http://labix.org/python-dateutil/#head-b79630fbbda87af6d6d121737510fd7ea5aeea97

Comment 14 Petr Machata 2012-08-27 23:09:43 UTC
So, we _could_ just ship a new version of pytz every time there is a corresponding tzdata update.  This would mostly achieve the same thing as what we are doing now, except it would cost a lot of effort.  What use case are we disregarding by not shipping the tarball and looking into system data instead?  Note that while the user technically can remove the tarball, they probably shouldn't, in packaged environment.

Comment 15 Thomas Spura 2012-08-28 09:31:11 UTC
(In reply to comment #14)
> What use case are we disregarding by not shipping the tarball and looking into
> system data instead?

We are not conform with upstream's intention. You need to use dateutil.tz, when you want to query the system tzdata.
With the patch tz and zoneinfo are basically behaving the same, without any difference.

I'd vote for dropping the patch without replacement, as:
* tz and zoneinfo are behaving the same then, without any difference.
* we diverge from upstreams intention.

Comment 16 Petr Machata 2012-08-28 09:51:57 UTC
How exactly does it serve anyone to ship an out of day copy of tzdata?  What is the use case?  I may be missing something, but this particular upstream intention seems misguided to me.

Comment 17 Thomas Spura 2012-08-28 10:28:10 UTC
IIUC, zoneinfo is intended as a fallback, when dateutil.tz fails (when e.g. the system zoneinfos are unavailable) and shouldn't be used normally directly by another program.

How did you note this? Are you aware of a dependency, which uses zoneinfo directly?

Comment 18 Petr Machata 2012-08-28 11:44:58 UTC
(In reply to comment #17)
> IIUC, zoneinfo is intended as a fallback, when dateutil.tz fails (when e.g.
> the system zoneinfos are unavailable) and shouldn't be used normally
> directly by another program.

If that is the case, then we could simply drop the tarball without replacement.  (I.e. replace the current patch with one that only drops the tarball.)  System timezones will always be available on Fedora.  Having applications fail loud and soon is IMHO better than them silently consuming out of date data.

> How did you note this? Are you aware of a dependency, which uses zoneinfo
> directly?

By accident.  I don't know how exactly python-dateutil is used by its clients.

Comment 19 Fedora End Of Life 2013-04-03 19:26:50 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle.
Changing version to '19'.

(As we did not run this process for some time, it could affect also pre-Fedora 19 development
cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.)

More information and reason for this action is here:
https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19

Comment 20 Fedora End Of Life 2015-01-09 21:52:04 UTC
This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 21 Fedora End Of Life 2015-02-18 13:36:53 UTC
Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.