Various time zones show regression in different years

Fedora 33 changed to end-of-life (EOL) status on 2021-11-30. Fedora 33 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Why do you consider the current behaviour a bug?

Please note that timezone data does and will change, so the exact results of mktime will change over time.

In this specific example you have set tm_isdst, which is to say that you have told mktime to presume that DST was in effect for this time.

This means that the function will attempt to find the *nearest* calendar time that fits.

In this case there is no nearby calendar time that has DST set for Europe/Moscow because, since 2010, Moscow does not observe DST.

Since it has been a long time since a DST time was in effect in that zone the heuristics for finding a calendar time that meet your requirement of tm_isdst fail, and -1 / EOVERFLOW is returned.

In general, if you set tm_isdst to 1 then mktime *may* fail if it can't easily find a calendar time that matches.

Are you trying to fix a specific upstream testsuite issue?

Hi, Carlos!

The "bug" -- maybe too strong )). But the regression at least.
Looking at the earlier glibc versions one can see that mktime does not give the different results in the same situation.

First of all the -EOVERFLOW appears sometimes. All other calls of the mktime with isdst=-1 on non-DST dates return the "almost normal" results.
Using various time zones it looks as "noisy" process that unpredictable to the programmer.
By the way, nothing overflows in that cases to set EOVERFLOW error.

When I tell mktime to presume that DST is not in affect (0) at the summer dates, mktime works wery well, 
newer -EOVERFLOF returns and all timestamps are right! Moreover, mktime changes isdst to the right value (0) when DST is not in effect (and do not touch my timestamp !!!+).
This predictable behavior of the mktime is too contrasting with the unpredictable previous.

You wrote that "the function will attempt to find the *nearest* calendar time that fits". 
Ok, but the man page does not say that mktime should to do this at all. 
Exept the normalisation case (say, Jul 40) mktime shouldn't change the user input (to shift seconds counter).
But the procedure of "finding nearest" do this. 
I think that it is a "free interpretation" of the POSIX.

I think the right behavior of the mktime in the case isdst=1 for non-DST dates will be the same as for the cases isdst=0 for DST dates:
"user, you are mistekenly guess that the DST is in effect, it is not )))) 
But never - to find "something that matches". 
But this is philosophical question, you can disagree )))

My interest in this connected with the astronomical and automation software that I'm writing.
Appearance of the "right" time zones and the "almost TAI" time lets the some things simpler. 
But the time support in glibc is a nutshell!! )) Espetially the time zones interface. )))
I was trying use the mktime in the "ambiguous times detection" task.
Sometimes I understand people and companies which are writing theirs own code for the timezone files parsing and timezones database support.

Sincerely,
Peter.

"Please note that timezone data does and will change, so the exact results of mktime will change over time."

By the way, 
zdump -v Europe/Moscow | grep 2014
shows the following:

Europe/Moscow  Sat Oct 25 21:59:59 2014 UT = Sun Oct 26 01:59:59 2014 MSK isdst=0 gmtoff=14400
Europe/Moscow  Sat Oct 25 22:00:00 2014 UT = Sun Oct 26 01:00:00 2014 MSK isdst=0 gmtoff=10800

As can be seeing, isdst=0 in both strings.
Is this right records in time zone file or it can be a bug?

Oh.. Comment 5 can be discarded...

(In reply to Peter from comment #4)
> Looking at the earlier glibc versions one can see that mktime does not give
> the different results in the same situation.

Correct.

This is because the results of the API query depend on dates, and dates are constantly moving forward.

Eventually you will also cross the y2038 border after which things will change again unless you have 64-bit time_t.
 
> First of all the -EOVERFLOW appears sometimes. All other calls of the mktime
> with isdst=-1 on non-DST dates return the "almost normal" results.
> Using various time zones it looks as "noisy" process that unpredictable to
> the programmer.

What software are you testing?

Are you testing C library implementations for portability?

It is possible to give mktime a broken-down time that it cannot reconcile to produce time_t.

In this case you've asked for a time with DST enabled, but no such time exists *near-enough* (heuristically) to your requested time for the search algorithm to find a sensible time.

> By the way, nothing overflows in that cases to set EOVERFLOW error.

POSIX error code EOVERFLOW means "The result cannot be represented."

You are correct that nothing "overflows" the error code is used to represent the case where the algorithm failed to turn the broken down time into time_t.
 
> When I tell mktime to presume that DST is not in affect (0) at the summer
> dates, mktime works wery well, 

Correct, because the algorithm can find a time at which DST is not in effect, in this case clocks do not change in Moscow (DST is not in effect).

> newer -EOVERFLOF returns and all timestamps are right! Moreover, mktime
> changes isdst to the right value (0) when DST is not in effect (and do not
> touch my timestamp !!!+).

If you have specific results that you feel are incorrect, for a given version of glibc, and a given version of tzdata, we should review those.

> This predictable behavior of the mktime is too contrasting with the
> unpredictable previous.

mktime() results depend on tzdata, and as time advances, results will change.

If we can find an incorrect result I can take this upstream.
 
> You wrote that "the function will attempt to find the *nearest* calendar
> time that fits". 
> Ok, but the man page does not say that mktime should to do this at all. 

Then we can fix this in the man page.

Normalization happens *also* for tm_isdst since the API is trying to comply with your request to find a time that exists when DST matches.

> Exept the normalisation case (say, Jul 40) mktime shouldn't change the user
> input (to shift seconds counter).
> But the procedure of "finding nearest" do this. 
> I think that it is a "free interpretation" of the POSIX.

The code in question is this:

glibc/time/mktime.c:

424   /* We have a match.  Check whether tm.tm_isdst has the requested
425      value, if any.  */
426   if (isdst_differ (isdst, tm.tm_isdst))
427     {
428       /* tm.tm_isdst has the wrong value.  Look for a neighboring
429          time with the right value, and use its UTC offset.
430 
431          Heuristic: probe the adjacent timestamps in both directions,
432          looking for the desired isdst.  This should work for all real
433          time zone histories in the tz database.  */

... and this can produce some some interesting results in an attempt to honour finding tm_isdst that matches.

We have used such search heuristics since 1998 for the implementation.

A few things have changed though in 2018 (glibc 2.29) where we started to more eagerly
return EOVERFLOW if we could not find a matching tm_isdst.
 
> I think the right behavior of the mktime in the case isdst=1 for non-DST
> dates will be the same as for the cases isdst=0 for DST dates:
> "user, you are mistekenly guess that the DST is in effect, it is not )))) 
> But never - to find "something that matches". 
> But this is philosophical question, you can disagree )))

I don't have a strong opinion, but the code from 1998 which has been in place
since then attemps to do the match.

The question we can try to answer is: If the exact tm_idst match fails, then what?

Today you get EOVERFLOW. You can re-try the API query with isdst=-1, which means
you don't know? I tested this and it works as expected.

> My interest in this connected with the astronomical and automation software
> that I'm writing.

Thank you, that answers my question about what software you are testing.

> Appearance of the "right" time zones and the "almost TAI" time lets the some
> things simpler. 
> But the time support in glibc is a nutshell!! )) Espetially the time zones
> interface. )))

I think you must retry with tm_isdst=-1 if you get an EOVERFLOW and let the
API tell you what is in effect.

> I was trying use the mktime in the "ambiguous times detection" task.
> Sometimes I understand people and companies which are writing theirs own
> code for the timezone files parsing and timezones database support.

They will face the same problem ;-)

In summary:
- Does retrying with tm_isdst=-1 a solution for you?

(In reply to Carlos O'Donell from comment #7)
> (In reply to Peter from comment #4)
> > Looking at the earlier glibc versions one can see that mktime does not give
> > the different results in the same situation.
> 
> Correct.
> 
> This is because the results of the API query depend on dates, and dates are
> constantly moving forward.

This is not the case. 
Run the following:
TZ=Europe/Paris ./test_isdst_1 

The result the following (from the middle of the output)

1954 -504151200 0 Sat Jan  9 23:00:00 1954
1955 Value too large for defined data type
1956 Value too large for defined data type
1957 Value too large for defined data type
1958 Value too large for defined data type
1959 Value too large for defined data type
1960 Value too large for defined data type
1961 Value too large for defined data type
1962 Value too large for defined data type
1963 Value too large for defined data type
1964 Value too large for defined data type
1965 Value too large for defined data type
1966 Value too large for defined data type
1967 Value too large for defined data type
1968  -62388000 0 Tue Jan  9 23:00:00 1968

This is a historical data. 
The output with the earlier versions of glibc was without the EOWERFLOW.

> It is possible to give mktime a broken-down time that it cannot reconcile to
> produce time_t.
> 
> In this case you've asked for a time with DST enabled, but no such time
> exists *near-enough* (heuristically) to your requested time for the search
> algorithm to find a sensible time.

Take a look, please, at the following example.
TZ=Europe/Moscow ./test_isdst_1
Last strings shows the following:

2010 1263067200 0 Sat Jan  9 23:00:00 2010
2011 1294603200 0 Sun Jan  9 23:00:00 2011
2012 1326139200 0 Tue Jan 10 00:00:00 2012
2013 1357761600 0 Thu Jan 10 00:00:00 2013
2014 1389297600 0 Fri Jan 10 00:00:00 2014
2015 1420833600 0 Fri Jan  9 23:00:00 2015
2016 1452369600 0 Sat Jan  9 23:00:00 2016
2017 1483992000 0 Mon Jan  9 23:00:00 2017
2018 1515528000 0 Tue Jan  9 23:00:00 2018
2019 1547064000 0 Wed Jan  9 23:00:00 2019
2020 Value too large for defined data type
2021 Value too large for defined data type

The expected result:
1) Jan 10 00:00:00
2) dst_flag is not set according to manpage:
"tm_isdst is set (regardless of its initial value) to a positive value or to 0, respectively, to indicate whether DST is or is not in 
effect at the specified time."

Why this resultis expected?
Because date/time given is normalised and exists, and can be represented by the time_t and manpage says that.. see cite above.
(It should be noted, that the DST rules does not applied in the Moscow timezone since October 2010).

Is the following output correct?

2010 1263067200 0 Sat Jan  9 23:00:00 2010

It is not, because the "Jan  9 23:00:00 2010" is not a date/time of interest
(the required date/time is Jan 10 00:00:00 2010).
The tm_isdst flag set correctly according to manpage.

Is the following output correct?

2012 1326139200 0 Tue Jan 10 00:00:00 2012

Yes, it is, because the date/time is as requested and tm_isdst = 0.

But:
what happens at 2020? I'm can't explain. And 2021 too.

All these 3 different results are obtained from the same algorythm with the same mktime call.

The next question is the following:
can these 3 results be useful to the software developer?
Can he make some decision based on these results?

The answer is: partially yes. Partially - I mean the second result, which is right.
More precisely - only 3 string from the program output:

2012 1326139200 0 Tue Jan 10 00:00:00 2012
2013 1357761600 0 Thu Jan 10 00:00:00 2013
2014 1389297600 0 Fri Jan 10 00:00:00 2014

The remaining strings from this example can not be neither used nor explained.

It seems that the main issue is that mktime tries to find something *near-enough*
in context of tm_isdst flag for already normalised end existing broken-down date-time (which can be represented).
But all that required is simply to look in the timezone database and to make correction according correction of the tm_isdst flag
(as was said in the manpage).

By the way, the same behavior of mktime I found in the Fedora 35.

> In summary:
> - Does retrying with tm_isdst=-1 a solution for you?

No, it does not, because at the ambiguous times (where both isdst=0 and isdst=1 values are valid) 
the only one value is set by mktime in this case (tm_isdst=-1).

This message is a reminder that Fedora Linux 34 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 34 on 2022-06-07.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '34'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 34 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Fedora Linux 34 entered end-of-life (EOL) status on 2022-06-07.

Fedora Linux 34 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release.

Thank you for reporting this bug and we are sorry it could not be fixed.