Bug 1764669

Thanks for a bug report. This is most likely this upstream bug:
https://gitlab.gnome.org/GNOME/evolution-ews/issues/69

The change there looks simple, thus it can go in RHEL7.

Err, I misread the steps. The 3) in the comment #0 means other upstream bug, namely this one:
https://gitlab.gnome.org/GNOME/evolution-ews/issues/52

That changes uses extensive API changes in evolution-data-server, it's not possible to backport it.

If I can ask: why do you Accept/Decline/... the meeting with selected local calendar (like on This Computer/Personal, right)?

The thing is that once you receive a meeting invitation through an EWs account, the server usually adds this meeting into your calendar automatically, then when it's selected in the Evolution the evolution-ews finds it in the EWS calendar and preselects that calendar, instead of the Personal, which might be the default, probably.

The sent invitations/notifications are not sent by Evolution itself, they are sent by the server. The evolution-ews can only instruct the server whether the notification should be sent or not, but it doesn't know it in this case, because this behaves the same like when creating a new event in the calendar.

@mcrha Thanks for your quick analysis.

# Upstream issue

The behavior described in upstream https://gitlab.gnome.org/GNOME/evolution-ews/issues/52 does sounds like the same issue I reported here.

# Local calendar use case

The underlying set of factors that led to meeting records on the local calendar are not completely clear. There are likely additional bugs involved as having meeting records on the local calendar was not intentional.

One use case that applied was a Microsoft Office 365 to hosted Exchange account migration. Before migration, calendar records were exported as .ics from the original Office 365 account via the Office 365 web interface. There were then imported into a dedicated local Evolution calendar for backup. This export/import approach was used because an attempted move operation within Evolution had failed with errors about missing timezone information on some records. Server administrators separately migrated Office 365 calendar records to the new Microsoft Exchange account. 

The original Office 365 account was deactivated, and the local backup calendar deselected by default to prevent showing duplicates in Evolution. However, I suspect this state may have led to issues in the pre-selection of the correct calendar in Evolution when responding to meeting updates. Ultimately, some records were found on what should have been an empty local calendar, some records had been updated on the local backup calendar, and some records on the new Microsoft Exchange account were not being updated with new times as they should have been.

Evolution's pre-selection of the correct calendar when responding to a meeting update remains problematic, but a separate issue.

# Improvement suggestions

I am inclined to agree with the improvement suggestions provided by the reporter of https://gitlab.gnome.org/GNOME/evolution-ews/issues/52. If move (or delete/put) operations lead to new records which are not identical in all ways to the originals, and lead to uncontrollable and unpredictable side-effects on the server, then it would be best to simply not offer a move operation with Evolution.

The severe behavior that needs to be addressed is the potential for Evolution to trigger unintended mass-emails. If this only happens as a result of a calendar meeting move operation in the Evolution GUI, then disabling calendar meeting move operations seems reasonable.

# Additional test cases

I have not tested the case of importing calendar records into an Exchange calendar via Evolution, however some testing may be warranted to ensure that operation does not produce similar behavior at an even greater scale.

Thanks for the information. I think the reason why On This Computer/Personal is preselected for you is that Evolution didn't find the meeting in your calendars (including the EWS calendar(s)) and the On This Computer/Personal is set as the Default calendar. To have checked for a fresh data you can set "Listen for server change notifications" in the mail account Properties->Receiving Options tab, which also the other parts inherit. I think this change requires restart of the background processes, thus something like:

   $ evolution --force-shutdown

I'll add a temporary change to Evolution and evolution-ews, specific only to EWS calendars, to not send notifications when moving meetings into an EWS calendar, regardless whether the meeting organizer is the user or not.

I cannot cover all the cases here, that's why I call it a temporary change. The full change will wait for a rebase of the evolution-ews (and other evolution packages), which might probably happen only in RHEL 8 and later, but I'm not a decision maker here, thus this is just my personal opinion.

I just realized that Exchange server rejects to create meetings with an Organizer other than the user to whom the account belongs, thus moving from one calendar to another meetings organized by some one breaks the meeting information. It looks like a limitation on the Exchange Server for the EWS' CreateItem operation.

That means that moving meetings between calendars is not a good thing to do with EWS calendars.

I still do not want to disable the copy/move operation for EWS calendars, because it works fine for regular events/tasks/notes.

One more observation, when the send of the iTip messages is disabled, the Exchange server (at least my 2019) treats the event as a regular event, not as a meeting, even though the data sent to the server contains attendees. That's quite surprising to me.

The Exchange server seems to be very tight to receive meeting invitations by mail.

With both these limitations (this one and in the previous comment), I can tell the server to not send notifications on meeting copy/move from another calendar if I extend the previous patch and I add a patch to the evolution itself.

I forgot to add:

These two issues are quite bad and I'm afraid the fix would make things worse, not better (nonetheless, the upstream version has it included), thus I hesitate to do this change in RHEL. What do you think, Don?

Note the problems are caused by the EWS protocol, the way the Exchange works, not by evolution-ews itself (except if there are other ways to CreateItem, which I'm not aware of at the moment).

(In reply to Milan Crha from comment #9)
> I just realized that Exchange server rejects to create meetings with an
> Organizer other than the user to whom the account belongs, thus moving from
> one calendar to another meetings organized by some one breaks the meeting
> information. It looks like a limitation on the Exchange Server for the EWS'
> CreateItem operation.
> 
> That means that moving meetings between calendars is not a good thing to do
> with EWS calendars.

RFC 5546, "iCalendar Transport-Independent Interoperability Protocol (iTIP)," Section 6.1.1, "Security Considerations -> Security Threats -> Spoofing the Organizer"(https://tools.ietf.org/html/rfc5546#section-6.1.1) has the following content:

"In iTIP, the "Organizer" (or someone working on the "Organizer's" behalf) is the only person authorized to make changes to an existing "VEVENT", "VTODO", or "VJOURNAL" calendar component and republish it or redistribute updates to the "Attendees".  An iCalendar object that maliciously changes or cancels an existing "VEVENT", "VTODO", or "VJOURNAL" calendar component may be constructed by someone other than the "Organizer" and republished or sent to the "Attendees"."

I am not completely sure of the consequences of that section to this issue, but it might hint at some programmer's rationalization for not allowing a record to be created where the creator is not the "Organizer" (or someone working on the "Organizer's" behalf). Interestingly, because all the other content is copied and the iTIP workflows get triggered, the resulting behavior is essentially an accidental manifestation of the threat they were trying to avoid.

Yeah, exactly, the way the EWS server behaves (in combination with evolution-ews) is the opposite from your RFC paragraph citation.

My personal opinion is that users should be able to add meetings organized by some else received by other means than by the Mail. Like when users download an .ics file with a meeting in a conference, which is organized by the conference itself. It doesn't mean they own the meeting they want to add to the calendar, it only means they want to see when it is happening and whom is attending it. It's nothing against the 6.1.1 section of the RFC citation.

Anyway, my question is (because I cannot decide myself): what is better, keep things as they are, or apply changes, which will not send notifications, but which also turn meetings into regular events when copy/move them between calendars? It's possible evolution-ews does something incorrectly, I'm only not aware what it would be.

(In reply to Milan Crha from comment #13)
> These two issues are quite bad and I'm afraid the fix would make things
> worse, not better (nonetheless, the upstream version has it included), thus
> I hesitate to do this change in RHEL. What do you think, Don?

I would define the metric for resolution as preventing the behavior of a user unknowingly sending emails to others. In Comment #9, you note: "I still do not want to disable the copy/move operation for EWS calendars, because it works fine for regular events/tasks/notes." Would it be possible to just disable the operation for Meetings? A corresponding explanation in the GNOME Evolution documentation or Red Hat Knowledgebase might then be sufficient.

A usability concern is the redefinition of copy/move within this context. In general, a user would likely assume that a copy/move from one storage location to another will result in a record with identical values for each field in the new location. In the case of Microsoft Exchange, it seems the records cannot be the same in both locations. In addition, the side effects triggered due to iTIP work-flow are also unexpected behavior for a user's intent of simply storing a record in a different place.

> Note the problems are caused by the EWS protocol, the way the Exchange
> works, not by evolution-ews itself (except if there are other ways to
> CreateItem, which I'm not aware of at the moment).

Thanks for the investigation of the protocol. That is definitely the right unit-of-analysis. Could you share an authoritative link for the protocol documentation? It would be nice to have it in the comments for reference.

It might also be fun to see how DavMail (http://davmail.sourceforge.net/index.html) handles this case.

(In reply to Milan Crha from comment #17)
> My personal opinion is that users should be able to add meetings organized
> by some else received by other means than by the Mail. Like when users
> download an .ics file with a meeting in a conference, which is organized by
> the conference itself. It doesn't mean they own the meeting they want to add
> to the calendar, it only means they want to see when it is happening and
> whom is attending it. It's nothing against the 6.1.1 section of the RFC
> citation.

I completely agree.

> Anyway, my question is (because I cannot decide myself): what is better,
> keep things as they are, or apply changes, which will not send
> notifications, but which also turn meetings into regular events when
> copy/move them between calendars? It's possible evolution-ews does something
> incorrectly, I'm only not aware what it would be.

This seems to be a pair of design defects in Microsoft Exchange. First, copy/move does not copy or move the data. Instead, the operation modifies the data. Second, the operation triggers unpredictable work-flow activities which are invisible to the user and out of their control.

Given the complexity, and the additional factor that Microsoft Exchange is always a moving target with unpredictable behavior, my suggestion is to make a minimal change to Evolution that just removes Meeting copy/move operations where the target calendar is hosted on a Microsoft Exchange server. Users can then be left to perform Meeting import/copy/move operations in Microsoft Exchange's web interfaces or Microsoft Outlook where it is no longer Evolution's problem/fault.

(In reply to Don Pellegrino from comment #18)
> Could you share an authoritative link for the protocol documentation?

I do not have anything really authoritative. The documentation for the EWS protocol is rather sparse. One can check what XML elements are provided at [1] (for the CreateItem) and then try what it does. For example when I tried to set also the Organizer, then the server rejected the request with a claim that the Set operation on that property is not allowed.

I do not know all the aspects of the EWS protocol, I work on it only rarely, thus it's possible there are ways to import meetings as meetings, I only do not know of them.

(In reply to Don Pellegrino from comment #19)
> Given the complexity, and the additional factor that Microsoft Exchange is
> always a moving target with unpredictable behavior, my suggestion is to make
> a minimal change to Evolution that just removes Meeting copy/move operations
> where the target calendar is hosted on a Microsoft Exchange server.

Or let the evolution-ews backend reject the addition of a meeting where the calendar owner is not the organizer. That would cover both copy/move between calendars and import of such meetings. The downside is that some events could be copied/moved, but some not, stopping on the first "faulty" meeting.

[1] https://docs.microsoft.com/en-us/exchange/client-developer/web-service-reference/createitem

The below blog is rather old, but I believe it's still valid:
https://blogs.msdn.microsoft.com/webdav_101/2011/09/28/howto-set-the-organizer-of-a-meeting-on-the-calendar-of-an-attendee-using-ews/

I made a change upstream [1], only for 3.35.2+ (for the development version). It adds a new translatable string, saying the EWS calendar cannot create meetings organized by other users.

[1] https://gitlab.gnome.org/GNOME/evolution-ews/commit/9070176add1b2823502699eff9f79955ede1ec6a

(In reply to Milan Crha from comment #22)
> I made a change upstream [1], only for 3.35.2+ (for the development
> version). It adds a new translatable string, saying the EWS calendar cannot
> create meetings organized by other users.

I plan to backport this for 7.9. In case user uses multiple addresses on the server, he/she can add them into Aliases section in EWS Mail account Properties, which will let him/her crate meetings for any of such users (in addition to the main mail address of the EWS account). Trying to add a meeting organized by all other users will be rejected by the EWS calendar and an error will be returned.

Unable to reproduce with evolution-ews-3.28.5-6

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (evolution-data-server bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3995