Multicast requirements for control over LLN

The ROLL working group is chartered to design and standardize a routing protocol for resource constrained devices in Low-power and Lossy Networks (LLN) . The requirements for ROLL are documented in . For building control it is recognized that most communication is local to the wireless mesh network, and does not necessarily pass through the edge router. The point-to-point RPL routing algorithm is developed to efficiently support unicast routing in such applications . The Trickle algorithm was initially developed to support the RPL routing algorithm , and later proposed to support general multicast delivery in LLNs in Trickle Multicast Forwarding (TMF) . This draft discusses the multicast requirements for constrained devices participating in M2M building control networks. An important requirement is the delivery of control commands to a subset (group) of neighbouring devices in the LLN within some latency bound. Also, analyses are provided of how well Trickle algorithm and TMF can meet these requirements and suggestions for improvement are made.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. Addtional privileged words are described below. "TMF" is used as an abbreviation for Trickle Multicast Forwarding as described in . A "device" is a physical processor connected to at least one link through a network interface. Each interface has at least one IP unicast address. The IP address is optionally bound to a host name, which may be a Fully Qualified Domain Name (FQDN). One device communicates directly with another device by wirelessly transmitting packets to it over a link. The link quality is divided in three regions : good: where a transmitted packet will be correctly received by a destination with a probability of say 95% or more. transitional: where the probability of correct reception fluctuates. bad: where almost no transmission is successfully received. It is empirically known that good links can become bad occasionally (e.g. once a week for a few minutes)due to dynamic effects such as multipath interference. A distinction is made between reception and delivery of a message. A message is received when it is stored in the reception buffer of the receiver after transmission and all error checks have been succesfully passed. The message is delivered when the message is passed from the reception buffer to the destination application. We also say the application accepts the message. Broadcasting is used for the link-local sending of one packet to all reachable 1-hop neigbours. This is equivalent to the term link-local multicast.

In this draft, we focus and develop discussions on requirements pertaining to IP multicasting requirements and IP multicast routing, in the context of building control applications on LLNs. This draft aims to show potential (latency) improvements for current proposed multicast routing approaches, that can be easily attained.

Multicast is important for building control applications. Two types of applications are considered: Discovery messages to (a subset of) the members of the mesh (multicast GET) Control messages to a subset of the mesh (multicast PUT) The first type requires the message to be sent to a (sub)set which may be randomly distributed over the building area. Some of the destinations return unicast response messages to the source. The second type requires a Non-Confirmable message mostly to be sent to a closely spaced subset. No return messages are generated. This second type is the subject of this draft, although most of the requirements equally apply to case 1. GET and PUT and Confirmable/Non-Confirmable are message types defined for CoAP . They are thought representative for the two applications types, as the multicast GET SHOULD return a unicast response and the multicast PUT typically does not return a response in control applications. An office building typically consist of multiple floors, divided in working areas. The working areas can be open or enclosed by walls. Within a working area sensors measure temperature, presence, humidity and other parameters. On the basis of these measurements, equipment within the working area can receive commands to change settings. A well-known example is presence detection to switch on or dim lights. The equipment configuration is quite stable, because devices are installed in the ceiling, and modifying (or servicing) the installation can be costly. The equipment is interconnected in a wireless network. The RF transmissions pass through the walls and generate interference to the wireless equipment in other working areas. The lay-out of a network may be different from installation to installation. However, it is expected that many wireless networks extend over one floor and include several working areas. Another working hypothesis is that most of the time sensors will multicast their values to a group of devices within the working area. Consequently, multicast messages are often meant for a subset of neigbouring (not necessarily 1-hop) devices. A LoWPAN is a mesh of wireless devices that share the same IPv6 address prefix. A typical LowPAN in a building may cover the area of an entire floor. A commercial installation may cover 1000 m2 per floor. A length of 50 m can easily mean a hop count >5 for a message to pass from end to end. For example, devices may be installed in the ceiling in a grid with a grid pattern distance of 2 m between devices. Messages may consist of sensor measurements performed or commands issued in a given working area, which then must be acted upon by neigbouring devices in the same working area. Under this control pattern, source and sink are located in one working area, and accordingly sink and source of a multicast message are often between 3 - 6 m from each other. Consequently, it is required to send a multicast to a subset of the devices in the LoWPAN. In case of commands to luminaries, a command message must be delivered to all LoWPAN-local multicast group members within a clear deadline of about 200ms. In a deadline of 120 ms is suggested for other building applications. Although control messages are frequently exchanged between closely spaced (less than 6 m) devices, it is sometimes necessary to send a message to a subset of devices covering the whole building. In that case the multicast message will need to pass the edge router of the LoWPAN and to propagate to other subnets. This case is discussed in more detail in .

The multicast requirements are derived from the characteristics of the aforementioned applications. A device is said to be correct it it follows the selected multicast routing algorithm. The application characteristics and the network installation make it possible to add an additional set of network properties to make the multicast algorithm more efficient. The basic traditional multicast requirements (applicable to both PUT and GET) are : Validity: If sender S sends message, m, to a group, g, of destinations, a path exists between S and any destination D, and if S and D are correct, D eventually accepts m. Integrity: A destination D accepts m at most once from sender S and only if S sent m to a group including D. Agreement: If a correct destination of g accepts m, then all correct destinations of g accept m. The set of intended destination devices is identified by the multicast (group) IP address. Every device in the associated multicast group is a destination of the multicast. Each destination accepts messages with as destination the specified IP multicast address. Additional multicast requirements are: Timeliness: There is a known constant C such that if m is sent at time t, no correct destination accepts m after t+C. For lighting control applications the value of C is taken as 200 ms. This requirement only holds for the PUT case without response from a destination, but not for the GET case where a response is returned. Ordering: When m1 and m2 sent to the same group g, and a receiver in g accepts message m1 before m2, every receiver in g accepts m1 before accepting m2 Ordering applies to both the PUT and GET cases. Ordering can be partial or total. Partial ordering means that for specified message pairs, one message of the pair precedes the other. In case of total ordering, every message pair is ordered. Partial ordering is obtained by adding message counters in the message such that destinations can order the messages of a given sender. Messages from different sources are not ordered. Total ordering can be obtained with vector clocks or using synchronized clocks. Vector clocks require a large overhead that increases linearly with the number of devices in the network. As long as no synchronized clocks are available, partial ordering seems the most realistic. Total Ordering is interesting for the discovery application. When two devices announce themselves simultaneously with conflicting properties, all participants can come to the same decision by favoring the first arrival. Partial ordering is necessary when a multicast message needs multiple packets (for example discovery messages) or when multicast messages are sent with intervals shorter than the maximum throughput delay.

In this section we investigate the behavior of the Trickle algorithm when used for multicast routing. Rebroadcasting as defined in Trickle makes meeting tight deadlines a challenge. Simulation results in this section show for particlar configurations and parameter settings which end-to-end communication delays can be expected.

The simplest approach to IP multicast is to broadcast from a source to a set of devices reachable over good links in one hop. This is not sufficient however, because the set of reachable devices is often a subset of the set of destination devices. Consequently, additional measures are needed to make sure that the Agreement requirement is met. A standard technique, to reach all devices instead of a subset, stipulates that every receiver of a broadcast message rebroadcasts this message (flooding). When the multicast destination address of the message corresponds with a specified multicast address in the receiver device, the message is delivered. Thanks to this technique it is assured that when a path exists between the source and the destination device, the destination device will eventually receive the message from the sender. Given the network density described in section 2, the multicast can generate a broadcast storm with lots of interfering senders. The technique to prevent the storm, also used in Trickle, is to randomly delay a message rebroadcast. However, long delays can seriously jeopardize the Timeliness requirement. The following sections give insight under which conditions the Timeliness requirement can be met.

The simulations were done on a general rectangular network topology and on an approximation of known building installations. The IEEE 802.15.4 protocol is simulated with CSMA and the standard back-off intervals specified by IEEE 802.15.4. Packets between A and B arrive with a probability dependent on the distance but independent of the direction. A distance of 70m is at the limit of the transmission range. Two rectangular meshes were tried: (1) 5 x 5 nodes and (2) 10 x 10 nodes. The distance between two adjoining neigbors was varied between 5 and 70 m. The total surface for the 10 x 10 mesh varied accordingly between 45 x 45 m^2 and 630 x 630 m^2. The building installation approximation consist of a rectangular grid of 14 x 7 nodes over a surface of 35 x 15 m^2. Parameters Imin, Imax and k and variables I, t and c are defined as in .

The table below presents some of the results on the 5 x 5 mesh. Imax k Parameter Distance 10m 40m 70m 250ms 1 hopcount 1 2-4 5-9 250ms 1 avg delay 5 ms 40 ms 110 ms 250ms 1 max delay 18 ms 90 ms 1050 ms 250ms 1 msgs sent 0-5 0-11 1-12 250ms 1 msgs received 18-36 3-20 0-20 250ms 3 hopcount 1 2-4 5-9 250ms 3 avg delay 5 ms 40 ms 130 ms 250ms 3 max delay 25 90 ms 260 ms 250ms 3 msgs sent 1-7 3-12 7-13 250ms 3 msgs received 40-60 14-32 9-23 500ms 1 hopcount 1 3-5 5-10 500ms 1 avg delay 5 ms 40 ms 110 ms 500ms 1 max delay 19 ms 100 ms 1500 ms 500ms 1 msgs sent 0-4 0-8 0-10 500ms 1 msgs received 12-26 0-16 0-16 500ms 3 hopcount 1 3-5 5-10 500ms 3 avg delay 5 ms 40 ms 120 ms 500ms 3 max delay 22 80 ms 240 ms 500ms 3 msgs sent 1-8 2-9 5-10 500ms 3 msgs received 28-44 8-27 5-18 The observed behavior is close to what is observed on the 10 x 10 mesh and on the installation configuration. Behavior on, for example, a single row of nodes tends to be quite different and requires quite different parameter settings. The results in the table concern node (4,4) which had the longest end-to-end delays of all nodes. Node (0,0) sent a message every 2 seconds. Individual packets were lost but all messages arrived at all nodes eventually. The Imin was taken to 10 ms and Imax was taken to 250 ms and 500 ms with quite similar results. Changing the Imax has measurable influence on the maximum end-to-end delay. The table shows how many copies of a given message were received by node (4,4) and how many times a given message was rebroadcast. For k=3 more messages were received and sent. Receiving more messages leads to lower maximum delays because the probability of receiving the message early increases with increasing rebroadcast frequency. The causes for the large maximum delays (>400ms), occurring at d=70m, have been investigated in more detail. It is shown that a new packet does not always arrive after the first transmission. This is probably due to the synchronization of nodes when a new message arrives, resulting in hidden terminal effects at the destination node by overlapping sending intervals of its neighbors. For d=70 m, packets are only received by the direct neighbor along the x-axis or the y-axis. Consequently, when node (x, y) receives a new message, it originates probably from (x-1, y) or (x, y-1). When node (x, y) sends, packets are received in nodes (x+1, y) and (x,y+1). Given a Imin value of 10ms there is a large probability that the sending by nodes (x+1, y) and (x, y+1) overlap, leading to collision of the messages at node (x+1, y+1). In the following intervals, nodes (x+1, y) and nodes (x, y+1) receive the last message from their neighbors and do not repeat the message because c is larger than k, thus leading to long delays. The receiving node (x+1, y+1) sends at regular intervals, determined by the Imax value, its last received 'old' message. Often the reception of the old message by a neighbor leads to resending the new message. For that reason the maximum delay is linked to the maximum interval Imax. Increasing the value of k increases the probability of reciveing rebroadcast messages.

The results indicate that for the network configurations we foresee, with Trickle it is quite possible to reach average message delivery latency within the 200 ms range, meeting the Timeliness requirement for most nodes, and to limit the maximum latency by tuning parameter k.

The Trickle Multicast Forwarding (TMF) draft differs from direct application of in the introduction of multi-source, sliding windows, and use of ICMP messages. For Trickle parameter k finite, a transmission event consists of sending a Trickle ICMP advertizement (that summarizes a forwarder's state i.e. buffered IP multicast packets) and in addition any multicast messages that need rebroadcasting. This section analyzes some issues of TMF, in particular its ability to meet the Timeliness requirement for building control scenarios, and proposes improvements to address the issues.

Summarizing state in an ICMP message is clearly useful to reduce network traffic, if many IP multicast packets are being buffered in Trickle multicast forwarders. However, if only one or a few multicast packets are active in the network at a time, a forwarder sending ICMP messages generates unnecessary overhead. As an example, consider a forwarder that stores and needs to rebroadcast a single multicast message m1. According to TMF, it would need to send an ICMP message containing information about m1 (SeedID, sequence number, M bit) and additionally send a Trickle Multicast message with a Trickle Multicast header option which contains exactly the same information (SeedID, sequence number, M bit) plus the useful application data. In such cases were low latency is required, the extra overhead of sending the ICMP message leads to additional delays, for example in dense network topologies due to increased congestion. In a simulation of a building control installation the operation with and without extra ICMP message was compared for the case that a single multicast message was active. Without ICMP messages an average latency of message delivery to the entire group of 131 ms was observed. The extra overhead generated by ICMP messages led to an average delay of 197 ms, quite close to the Timeliness bound of 200 ms. The simulation modeled a single IP multicast message active in a 6LoWPAN network, delivery targeted to a group which is a subset of 13 nodes out of 95 nodes total, with a 40-byte data payload, each node acting as a forwarder, with Trickle parameters k=1, Imin=32 ms, Imax=128 ms. To addresss the latency issue without increasing k (which would lead to increased traffic), we propose that: sending the Trickle ICMP message is made OPTIONAL as part of a transmission event, if a Trickle forwarder has any Trickle Multicast Messages to send in that transmission event. A Trickle Multicast forwarder may decide per transmission event (depending on internal state e.g. number of buffered messages) whether the ICMP message is sent or not. as part of a transmission event, sending the Trickle ICMP message MAY be done after retransmitting Trickle Multicast Messages. Note that the TMF draft does not clearly express a preferred order for Trickle ICMP messages. These proposed changes are still fully compatible with existing implementations of TMF.

The current TMF makes a separation between (IP) hosts and Trickle Multicast forwarders. Nodes that only need to receive IP multicast packets (not wanting to participate in rebroadcasting) therefore can be configured as hosts unaware of the multicast routing protocol. However, this bears the risk that such hosts receive a specific multicast packet very late or never, because they don't have a way to signal missing packets to Trickle forwarders. Implementing a node as a host has the clear advantages that the node does not need to buffer any Trickle Multicast Messages which can considerably reduce memory usage. A solution that enables the best of both worlds is to allow Trickle Multicast forwarders to act as 'data sinks' only i.e. not acting as a repeater. We propose that: a Trickle Multicast forwarder MAY act as a data sink, which means that it does keep sliding window state for messages it accepts, and sends Trickle ICMP messages, but does not buffer any Trickle Multicast Messages for retransmission.

In the TMF draft the notion of 'consistency' (as we read it) is based on information received in Trickle ICMP messages only, not on information received from incoming Trickle Multicast Messages. This operation can lead to unnecessary delays in certain use cases. Consider the following scenario: Nodes A, B, C are Trickle Multicast forwarders; where A cannot hear C and C cannot hear A A stores messages m1,m2,m3, B stores m1,m2,m3, C stores m2,m3 C sends ICMP(m2,m3) B sees an inconsistency based on this and schedules the missing m1 for transmission A sends ICMP (m1,m2,m3) but not any multicast message m_i B sees a consistency and increments c When the Trickle timer at B expires assuming k=1, the scheduled transmission of m1 is cancelled C does not get m1 from B, at least not during this round. Eventually C will get m2, after more rounds (when B transmits before A does), but later than necessary. A first approach to improve latency in this scenario is to apply the suppression only to ICMP messages, not to scheduled multicast messages (such as m1 by B in the example above). A refinement of this approach is to maintain a counter c for each SeedID/Sequence-number combination, in addition to a global Trickle counter c. Then, retransmission of Trickle Multicast Messages is only suppressed for those messages that have been received at least k times. ICMP suppression is still based on the global Trickle counter c as in the current TMF draft.

A forwarder that does not support sending ICMP advertizements could advertize its state by retransmitting the multicasat message with the largest number in its window that has no missing messages relative to the lower bound of the window. So if a forwarder has a window containing m1,m2,m4,m5 it retransmits m2, triggering others to send m3 (and maybe higher numbers). If it encounters an inconsistency, i.e. seeing a multicast with a number lower than its own upperbound, it itself would send out the messages that have a higher number than the received multicast message (excluding the ones that it has received at least k tines during the current Trickle interval).

From the analyses above emerge a number of recommendations that aim to reduce transmission latency of multicast messages and to reduce the probability of missing a multicast message. In summary, the following adaptations to TMF are proposed which can be applied independently of each other: Efficient retransmission: sending the Trickle ICMP message is made OPTIONAL as part of a transmission event if a Trickle forwarder already has any Trickle Multicast Messages to send. Allow data sinks: a Trickle Multicast forwarder MAY refrain from buffering any Trickle Multicast Messages for retransmission. Consistency improvement: When a transmission is suppressed, a forwarder MAY only suppress ICMP but not suppress transmission of a multicast message that was scheduled due to a detected inconsistency. This approach could be refined by keeping in addition to a global Trickle consistency counter c, separate counters c per SeedID/sequence-number combination suppressing only messages seen at least k times. Window handling without ICMP: forwarders without ICMP sending capability can ask for retransmissions by rebroadcasting multicast messages

This document makes no request of IANA. Note to RFC Editor: this section may be removed on publication as an RFC.

TBD

This I-D has benefited from conversations with and comments from Anders Brandt, Kerry Lynn, Zach Shelby, Emmanuel Frimout, Michael Verschoor, Jamie Mc Cormack, Dee Denteneer, Jerald Martocci, Matthieu Vial, and Nicolas Riou.