]>

Eledrova 558 Prague 181 00 CZ jkt@flaska.net

General Internet Engineering Task Force IMAP QRESYNC EXISTS EXPUNGE VANISHED ARRIVED This document updates the QRESYNC extension of the IMAP protocol to use a new untagged response, the ARRIVED one, to inform about the UIDs of newly arriving messages. Deprecating the EXISTS response, this extension prevents a possible race condition where clients can lose synchronization of message UIDs in a selected mailbox when new arrivals are immediately expunged.

The QRESYNC extension introduces the VANISHED response, a UID-based supplement to the sequence-number-based EXPUNGED . The VANISHED response is also explicitly allowed to refer to non-existing message UIDs. Such a situation present a possible race condition where clients could lose track of the sequence -> UID mapping where new arrivals are removed in a parallel session. This document updates the QRESYNC extension with a new response to be used instead of EXISTS, the ARRIVED one. It also adds a mechanism for activating this extension and explains backward compatibility concerns.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

This section illustrates an example where a fully-compliant QRESYNC client loses track of UIDs of some messages in its mailbox. Suppose a client has opened a mailbox using the SELECT ... QRESYNC command. The mailbox contains a single message with UID 5; UIDNEXT is 11. The client is fully synchronized with the server. In this situation, the server informs client about new arrival:

Upon receiving the EXISTS reponse, IMAP client would typically proceed towards discovering various message metadata like theirs UIDs and flags, either through the UID SEARCH ALL command, or via UID FETCH (FLAGS). However, when the new arrivals are immediately expunged (perhaps as a result of an activity in a parallel session), the server can possibly process the request for additinal metadata only after the expunge has been reported to the client. The VANISHED response actually complicates matter in this situation:

This response informs about permanent removal of messages with UIDs between 12 and 20, inclusively. Unfortunately, as the VANISHED response is explicitly allowed to reference non-existing UIDs, the client doesn't know whether this VANISHED response affects two newly arriving messages. Immediately after having received these two responses, the client cannot make any assumptions about the rest of the mailbox (i.e. messages with UIDs > 6). Specifically, any of the following can be true: The mailbox now contains only one message with UID 5 The mailbox contains two messages; first of them has UID 5 and nothing is known about the other one The mailbox contains three messages; first of them has UID 5 and nothing is known about the rest of them In absence of the QRESYNC-ARRIVED extension or when talking to servers not offering that, clients MUST be prepared to work around the described race condition. An example of such a behavior is issuing an explicit UID SEARCH command to rediscover the unclear portion of the sequence -> UID mapping:

A single search operation is enough to recover the complete UID mapping, but it induces unnecessary round trip before a full mapping can be established again. In addition, clients would have to ignore any unsolicited FETCH responses not containing UID of the target message:

Clients have no chance but to ignore any FETCH response which affects messages whose UID is not yet known. This race condition can be avoided by switching to UID-based equivalent of EXISTS, the ARRIVED response. Alternatively, the same goal would be achieved if the QRESYNC specification mandated that VANISHED responses MUST NOT refer to UIDs which were not present in the mailbox at the time immediately before the VANISHED response was generated.

The QRESYNC-ARRIVED parameter to SELECT/EXAMINE commands shares syntax with the QRESYNC parameter defined in ; the only exception is that it is identified by the "QRESYNC-ARRIVED" atom. For backward compatibility, IMAP servers supporting this extension MUST support QRESYNC as defined by . In addition, such servers SHOULD NOT sent VANISHED responses containing UIDs of messages which were expunged before, and SHOULD NOT send the untagged EXPUNGED instead of VANISHED.

Contents: ordered list of UIDs The ARRIVED response reports that the specified UIDs have been delivered to the mailbox. It is similar in functionality to the EXISTS response ; however, it can return information about multiple messages, and it returns UIDs instead of message numbers. The first benefit saves bandwidth, while the second elliminates the potential of client losing track of assigned UIDs in a mailbox. If the client has opened the mailbox with the QRESYNC-ARRIVED parameter to SELECT/EXAMINE command, the EXISTS response MUST be still sent during the initial mailbox synchronization, i.e. between having received the SELECT or EXAMINE command and the corresponding tagged response, as in . Servers MAY send the EXISTS response several times, but they SHOULD send it only once to conserve resources. The number given in the EXISTS response MUST NOT shrink. The clients MUST treat the last occurence as the final data. Servers MUST NOT issue ARRIVED prior to issuing a tagged OK for the SELECT/EXAMINE command to prevent uncertanities about the HIGHESTMODSEQ value. New arrivals since the last time the mailbox has been selected are reported using FETCH responses with embedded UID, as in regular QRESYNC. After the mailbox has been selected, all further updates about new arrivals MUST use the ARRIVED response. If servers were allowed to fall back to untagged EXISTS, the possibility of race conditions would return. UIDs sent in the ARRIVED response must be always presented in a sorted order, lowest UID first. The sequence MUST NOT contain duplicate UIDs. Servers SHOULD take advantage of the compression capabilities of the sequence-set syntax and use the range syntax if possible. The ARRIVED response MUST NOT be sent unless the client has passed the QRESYNC-ARRIVED option to the last SELECT or EXAMINE command.

This text builds upon the QRESYNC IMAP extension .

IMAP4 capabilities are registered by publishing a standards track or IESG approved experimental RFC. The registry is currently located at: http://www.iana.org/assignments/imap4-capabilities This document defines the QRESYNC-ARRIVED IMAP capability. IANA will be asked to add this capability to the registry.

The following syntax specification uses the Augmented Backus-Naur Form (ABNF) notation as specified in . Non-terminals referenced but not defined below are as defined by , , or .

This extensions adds no functionality on top of what is already defined in the QRESYNC extension, and therefore we believe that no additional security implications have to be considered.

&RFC2119; &RFC5234; &RFC5162; &RFC3501;

Fixed ABNF syntax for the select-param Better abstract Clearer formatting

ENABLE is still required, otherwise clients have no way of activating VANISHED

Clarified that UIDs presented in the ARRIVED response MUST be always presented in a sorted order Used a special non-terminal in the ABNF grammar Used correct format for FETCH responses Clarify that EXISTS must be sent at least once and SHOULD be sent only once Clarify that ARRIVED MUST NOT occur before the tagged response to SELECT/EXAMINE Clarify that arrivals since the last time are to be reported through usual UID FETCH, as in old QRESYNC