Bug 1285621

Summary: Bugzilla is sending emails with wrong Date value
Product: [Community] Bugzilla Reporter: Jason McDonald <jmcdonal>
Component: Email NotificationsAssignee: Bug Bot 🤖 <bugbot>
Status: CLOSED WORKSFORME QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: 4.4CC: agk, bberg, mattias.ellert, mleitner, mschorm, pdwyer, rkhan, swadeley, thoger, vdanen, vondruch
Target Milestone: ---Keywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-03 01:05:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1468128, 1610130    
Bug Blocks: 1337584    
Attachments:
Description Flags
Email notification with wrong date none

Description Jason McDonald 2015-11-26 04:53:43 UTC
Description of problem:
Since November 25, I've noticed that Bugzilla has sent me several notification emails with the Date header (which should show the time the email was dispatched from the source) incorrectly set to an earlier date.  (This has probably been happening for longer than that and I've just noticed now because we've been making a lot of changes to our backlog over the past two days.)

The incorrect Date value causes my email client (which I have set to sort emails in date order) to quietly insert the email in the middle of the emails I have already read rather than at the bottom of the list, making it less likely that I'll notice the email and respond in a timely fashion.

Version-Release number of selected component (if applicable):
4.4.10042

How reproducible:
At least three times in the last 24 hours.

Steps to Reproduce:
1. File a bug without Story Points.
2. Have a different user set the Story Points field to a non-default value.
(There may be other ways, but the three I noticed so far were all notifications for changes to the Story Points field).

Actual results:
The notification generated in step 2 has the tiemstamp of step 1 in the Date field.

Expected results:
The notification generated in step 2 has the same (or only slightly later) timestamp as the corresponding entry in the bug history.

Additional info:
none

Comment 2 Jason McDonald 2015-11-26 05:07:56 UTC
(In reply to Jason McDonald from comment #0)
> How reproducible:
> At least three times in the last 24 hours.

To clarify, this bug doesn't always happen.  We set the Story Points field for about 40 bugs and I see wrong timestamps on three of the notifications so far.

Comment 6 Tomas Hoger 2016-01-12 20:14:17 UTC
*** Bug 1297913 has been marked as a duplicate of this bug. ***

Comment 9 Jeff Fearn 🐞 2016-02-21 22:15:34 UTC
*** Bug 1310038 has been marked as a duplicate of this bug. ***

Comment 10 Jeff Fearn 🐞 2016-02-21 22:25:35 UTC
I'm pretty sure this is a consequence of bug 1287318, when BZ sends an email for a change it updates the bugs table with the time it was last emailed out, when that failed to update the table it would reuse the date of the transaction where the table update failed.

There are still some bugs around that have not been updated since the fix for 1287318 went live, so some emails will still be sent out with an incorrect date, but this should disappear over time as they all get back in sync.

This bug is only a current issue if both dates are after the upgrade, which IIRC was 15th Feb.

Comment 14 Muhammad Tahir 2016-03-08 03:14:25 UTC
*** Bug 1315229 has been marked as a duplicate of this bug. ***

Comment 15 Rashid Khan 2016-03-18 18:45:21 UTC
When is 4.4+ getting released? 
ETA?

Comment 23 Jason McDonald 2016-06-07 07:24:10 UTC
*** Bug 1343324 has been marked as a duplicate of this bug. ***

Comment 29 Jeff Fearn 🐞 2016-09-29 09:51:22 UTC
Hi Matt, what's going on with this bug?

Comment 30 Matt Tyson 🤬 2016-09-30 06:21:07 UTC
It's still happening.  I haven't been able to isolate the cause yet.

Comment 31 Jeff Fearn 🐞 2016-10-26 23:49:26 UTC
Moving to post bet5a work as we haven't been able to isolate this.

Comment 32 Matt Tyson 🤬 2016-12-07 23:21:01 UTC
*** Bug 1401838 has been marked as a duplicate of this bug. ***

Comment 33 Tomas Hoger 2017-02-13 09:02:17 UTC
This is still happening.

Comment 34 Jeff Fearn 🐞 2017-02-13 09:21:25 UTC
Yeah we know, but there are no resources to allocate to this problem at this time, and this is likely to remain the case for several quarters.

Comment 35 Jeff Fearn 🐞 2017-04-27 22:35:08 UTC
*** Bug 1446149 has been marked as a duplicate of this bug. ***

Comment 36 Tomas Hoger 2017-06-30 07:37:54 UTC
It seems this problem has become more frequent recently.

Comment 37 Jeff Fearn 🐞 2017-07-13 00:16:47 UTC
I believe this is, to some extent anyway, caused by BZ 1468128, which prevents unsent mails from being sent if the script hits a "bad" bug.

Comment 38 Jeff Fearn 🐞 2017-07-18 04:58:44 UTC
(In reply to Jeff Fearn from comment #37)
> I believe this is, to some extent anyway, caused by BZ 1468128, which
> prevents unsent mails from being sent if the script hits a "bad" bug.

This fix has made it's way to prod now. Hopefully it will address this issue.

Comment 39 Tomas Hoger 2017-07-18 07:32:50 UTC
Regarding the verification of this bug - should this be considered not fixed if I get another email with wrong Date: header, or only if the Date: is after today?  I.e. if mails with bad date are to be expected for bugs that got into some "corrupted" state in the past while no new "corruptions" should happen any more, or the fix should avoid all badly dated mails.

Comment 40 Jeff Fearn 🐞 2017-07-19 00:16:43 UTC
It's still possible for this to happen, although it should be limited to within an hour or so of the last transaction.

We don't know what causes mails to not be sent when the change occurs, but when that triggers then it will get sent when the cron job runs the script. So if you update a bug and the email fails to be sent, then if another change happens before the cron runs, you will get the first changes time.

We are waiting to see if this occurs often and if so if we can track down what prevents the mail from being sent. We could be more aggressive with the cron job, run it more frequently and with a shorter wait time, but we'd like to get a clearer picture of how the script changes impact things before we make a call on that.

Comment 45 Dan Callaghan 2017-07-25 03:47:53 UTC
I just received a bug notification yesterday which was (wrongly) backdated by about a year. It's quite annoying, as I'm more likely to miss those notifications if they get buried under a year's worth of other mails.

Comment 46 Dan Callaghan 2017-07-25 03:50:54 UTC
Also if I'm reading your hypothesis in comment 40 correctly -- the mails would never be backdated by more than one hour. However that is clearly not the case. When this happens, the Date header is the same as the *previous* time the bug changed (which could be days, months, or years in the past).

Comment 47 VĂ­t Ondruch 2017-08-09 08:49:37 UTC
~~~
Date: Fri, 28 Jul 2017 09:01:23 +0000
Precedence: bulk
Auto-Submitted: auto-generated
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: CC pvalena strzibny
X-Bugzilla-Classification: Fedora
X-Bugzilla-Product: Fedora
X-Bugzilla-Component: rubygem-sanitize
X-Bugzilla-Sub-Component: 
X-Bugzilla-Version: 24
X-Bugzilla-Keywords: 
X-Bugzilla-Severity: unspecified
X-Bugzilla-Who: jkurik
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: unspecified
X-Bugzilla-Assigned-To: axilleas
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_status resolution cf_last_closed
X-Bugzilla-Comment: public
X-Bugzilla-Reporter: releng  
X-Loop: bugzilla
X-BeenThere: bugzilla
Message-ID: <bug-1308081-307447-3wJUCdJI3D.com>
In-Reply-To: <bug-1308081-307447.com>
References: <bug-1308081-307447.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: https://bugzilla.redhat.com/
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13

https://bugzilla.redhat.com/show_bug.cgi?id=3D1308081

Fedora End Of Life <jkurik> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |CLOSED
         Resolution|---                         |EOL
        Last Closed|                            |2017-08-08 09:00:13
~~~

Comment 48 VĂ­t Ondruch 2017-09-06 10:24:55 UTC
These two are interesting. Yesterday I got this email from BZ:

~~~
Return-Path: bugzilla
Received: from zmta03.collab.prod.int.phx2.redhat.com (LHLO
 zmta03.collab.prod.int.phx2.redhat.com) (10.5.81.10) by
 zmail26.collab.prod.int.phx2.redhat.com with LMTP; Tue, 5 Sep 2017 08:40:37
 -0400 (EDT)
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12])
	by zmta03.collab.prod.int.phx2.redhat.com (Postfix) with ESMTP id C22704519D
	for <vondruch.redhat.com>; Tue,  5 Sep 2017 08:40:37 -0400 (EDT)
Received: by smtp.corp.redhat.com (Postfix)
	id BED3E835D6; Tue,  5 Sep 2017 12:40:37 +0000 (UTC)
Delivered-To: vondruch
Received: from mx1-phx2.redhat.com (mx01.util.phx2.redhat.com [10.5.26.26])
	by smtp.corp.redhat.com (Postfix) with ESMTPS id B7664835C4
	for <vondruch>; Tue,  5 Sep 2017 12:40:37 +0000 (UTC)
Received: from bugweb-04.app.bz.hst.phx2.redhat.com (bugweb-04.app.bz.hst.phx2.redhat.com [10.5.117.34])
	by mx1-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v85CebUi044932
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO)
	for <vondruch>; Tue, 5 Sep 2017 08:40:37 -0400
Received: from bugweb-04.app.bz.hst.phx2.redhat.com (localhost [127.0.0.1])
	by bugweb-04.app.bz.hst.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v85CebSF035791
	for <vondruch>; Tue, 5 Sep 2017 08:40:37 -0400
Received: (from apache@localhost)
	by bugweb-04.app.bz.hst.phx2.redhat.com (8.14.4/8.14.4/Submit) id v85CebBA035790;
	Tue, 5 Sep 2017 08:40:37 -0400
X-Authentication-Warning: bugweb-04.app.bz.hst.phx2.redhat.com: apache set sender to bugzilla using -f
From: bugzilla
To: vondruch
Subject: [Bug 1487591]
   CVE-2017-0899 CVE-2017-0900 CVE-2017-0901 CVE-2017-0902 rubygems:
   various flaws [fedora-all]
Date: Tue, 05 Sep 2017 12:40:37 +0000
Precedence: bulk
Auto-Submitted: auto-generated
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Fedora
X-Bugzilla-Product: Fedora
X-Bugzilla-Component: rubygems
X-Bugzilla-Sub-Component: 
X-Bugzilla-Version: 26
X-Bugzilla-Keywords: Security, SecurityTracking
X-Bugzilla-Severity: medium
X-Bugzilla-Who: updates
X-Bugzilla-Status: MODIFIED
X-Bugzilla-Priority: medium
X-Bugzilla-Assigned-To: vanmeeuwen+fedora
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: 
X-Bugzilla-Comment: public
X-Bugzilla-Reporter: amaris  
X-Loop: bugzilla
X-BeenThere: bugzilla
Message-ID: <bug-1487591-307447-oLAsLbOJd8.com>
In-Reply-To: <bug-1487591-307447.com>
References: <bug-1487591-307447.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: https://bugzilla.redhat.com/
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12

https://bugzilla.redhat.com/show_bug.cgi?id=3D1487591



--- Comment #2 from Fedora Update System <updates> ---
rubygems-2.6.13-100.fc26 has been submitted as an update to Fedora 26.
https://bodhi.fedoraproject.org/updates/FEDORA-2017-20214ad330

--=20
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=3Dl0e1JTQ=
sv4&a=3Dcc_unsubscribe=
~~~

Today, a few moments ago, I submitted update for F27 and I received email with precisely the same date time as the one from yesterday:

~~~
Return-Path: bugzilla
Received: from zmta04.collab.prod.int.phx2.redhat.com (LHLO
 zmta04.collab.prod.int.phx2.redhat.com) (10.5.81.11) by
 zmail26.collab.prod.int.phx2.redhat.com with LMTP; Wed, 6 Sep 2017 06:14:22
 -0400 (EDT)
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11])
	by zmta04.collab.prod.int.phx2.redhat.com (Postfix) with ESMTP id AF171D0310
	for <vondruch.redhat.com>; Wed,  6 Sep 2017 06:14:22 -0400 (EDT)
Received: by smtp.corp.redhat.com (Postfix)
	id AC43418ABA; Wed,  6 Sep 2017 10:14:22 +0000 (UTC)
Delivered-To: vondruch
Received: from mx1-phx2.redhat.com (mx01.util.phx2.redhat.com [10.5.26.26])
	by smtp.corp.redhat.com (Postfix) with ESMTPS id A60E019E35
	for <vondruch>; Wed,  6 Sep 2017 10:14:22 +0000 (UTC)
Received: from bugweb-04.app.bz.hst.phx2.redhat.com (bugweb-04.app.bz.hst.phx2.redhat.com [10.5.117.34])
	by mx1-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v86AEMEm047378
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO)
	for <vondruch>; Wed, 6 Sep 2017 06:14:22 -0400
Received: from bugweb-04.app.bz.hst.phx2.redhat.com (localhost [127.0.0.1])
	by bugweb-04.app.bz.hst.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v86AEMCt030016
	for <vondruch>; Wed, 6 Sep 2017 06:14:22 -0400
Received: (from apache@localhost)
	by bugweb-04.app.bz.hst.phx2.redhat.com (8.14.4/8.14.4/Submit) id v86AEMPC030015;
	Wed, 6 Sep 2017 06:14:22 -0400
X-Authentication-Warning: bugweb-04.app.bz.hst.phx2.redhat.com: apache set sender to bugzilla using -f
From: bugzilla
To: vondruch
Subject: [Bug 1487591]
   CVE-2017-0899 CVE-2017-0900 CVE-2017-0901 CVE-2017-0902 rubygems:
   various flaws [fedora-all]
Date: Tue, 05 Sep 2017 12:40:37 +0000
Precedence: bulk
Auto-Submitted: auto-generated
X-Bugzilla-Reason: CC
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Fedora
X-Bugzilla-Product: Fedora
X-Bugzilla-Component: rubygems
X-Bugzilla-Sub-Component: 
X-Bugzilla-Version: 26
X-Bugzilla-Keywords: Security, SecurityTracking
X-Bugzilla-Severity: medium
X-Bugzilla-Who: updates
X-Bugzilla-Status: MODIFIED
X-Bugzilla-Priority: medium
X-Bugzilla-Assigned-To: vanmeeuwen+fedora
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: 
X-Bugzilla-Comment: public
X-Bugzilla-Reporter: amaris  
X-Loop: bugzilla
X-BeenThere: bugzilla
Message-ID: <bug-1487591-307447-aarggpRK9u.com>
In-Reply-To: <bug-1487591-307447.com>
References: <bug-1487591-307447.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: https://bugzilla.redhat.com/
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11

https://bugzilla.redhat.com/show_bug.cgi?id=3D1487591



--- Comment #3 from Fedora Update System <updates> ---
rubygems-2.6.13-100.fc27 has been submitted as an update to Fedora 27.
https://bodhi.fedoraproject.org/updates/FEDORA-2017-81cf93b7c2

--=20
You are receiving this mail because:
You are on the CC list for the bug.
Unsubscribe from this bug https://bugzilla.redhat.com/token.cgi?t=3DmGiv5ne=
GLV&a=3Dcc_unsubscribe=
~~~


It looks like if there was some cache somewhere caching fragments from the header ...

Comment 50 VĂ­t Ondruch 2019-05-22 12:39:00 UTC
Created attachment 1571977 [details]
Email notification with wrong date

This still happens with BZ 5.0.4.rh21. See the attachment.

Comment 51 Jeff Fearn 🐞 2019-07-19 02:49:38 UTC
*** Bug 1731140 has been marked as a duplicate of this bug. ***

Comment 53 Jeff Fearn 🐞 2022-04-07 06:32:17 UTC
Hi, recently we changed how Bugzilla sends mail. It now uses a dedicated smtp server to queue and send messages. If anyone sees a message with the wrong date please forward it to me.

Comment 54 Jeff Fearn 🐞 2022-05-03 01:05:23 UTC
I'm going to close this as the new mail system should prevent email hiccups. Feel free to reopen this bug if you see it again. Please include the mail headers in a private attachment.