ICMP black hole: problem position

ICMP is the key protocol to exchange control and error messages over the Internet. ICMP is also implied in several functionalities, like the "Path Maximum Transmission Unit Discovery" (PMTUD) mechanism . For instance, in IPv4, the sender sets the "don’t fragment bit" (this is useless in IPv6 since fragmentation is not authorized). If a router cannot transmit the packet because of its size, it must send back to the source an ICMP "Too Big" (type 3, code 4 with IPv4 and type 2, code 0 with IPv6) packet. Iteratively, the source will lower the packet size until it matches the lowest MTU on the path. An appropriate ICMP's processing throughout a path is therefore a key requirement both for troubleshooting operations (e.g., debugging routing problems) and for several functionalities (e.g., PMTUD). Unfortunately certain routers of the Internet do not handle ICMP as one would expect. For instance, some of them will not forward ICMP packets on a given path, while others will not generate and send ICMP packets in response to errors or solicitations (e.g., an ICMP "echo request"). Note that the presence of ICMP black holes has been the motivation for the design of ICMP-free alternatives to PMTUD, namely the Packetization Layer Path MTU Discovery (PLPMTUD) . In the present document we take the opposite approach and try to improve ICMP-related problem debugging in the Internet. The present document is essentially a problem position document. More precisely, the contributions are twofold: we provide a taxonomy of router behavior with respect to ICMP; we provide examples taken from real world experiments, using existing tools to illustrate the previous discussion and taxonomy;

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

Let us consider a path from a source S to a destination D, and a router R along this path (there are typically several such routers and R is one of them). We assume that the packet flow generated by S and destined to D uses protocol PP. More specifically in this work we consider either ICMP/IP, UDP/IP, or TCP/IP, where IP denotes either IPv4 or IPv6.

|Router R|--------------->|Destination D| +--------+ initial forward +--------+ final forward +-------------+ path path ]]> We can define three different paths (): the initial forward path: this is the path between S and R; the final forward path: this is the path between R and D; the ICMP return path: This is the path taken by ICMP packets destined to S and generated either by R, or by a router on the final forward path, or by D itself, and that go through R (this is not necessarily the case); It is important to note that the initial forward path up to R and the return ICMP path from R are not necessarily identical since routers often have different forwarding rules for each direction. The forward initial or final paths also potentially depend on the nature of the initial packet protocol, since routers often use protocol dependent forwarding rules. Finally, it is important to note that there are potentially as many ICMP return paths as there are routers on the forwarding path.

First of all, it is important to note that ICMP handling is not a global property of a router. Said differently, two interfaces of a given router may behave differently with respect to ICMP processing (e.g., per interface firewall configuration can explain this situation). Therefore we need to work at the interface level. Moreover, for each interface we also differentiate incoming and outcoming traffic. Said differently, a router behavior from S to D may differ when considering the reverse direction, D to S. Let us consider traffic from a source S to a destination D, and a router R along this path. We first identify an ability, which is not directly related to ICMP, but which has impacts on ICMP: Ability A0: "correct IP processing of an incoming packet (not necessarily an ICMP packet) on the initial forward path": even if this is not an ICMP packet, it may have consequences on ICMP behavior. For instance, if R does not decrement the IPv4 TTL/IPv6 Hop Limit field, R will not generate an ICMP packet if this field equals 1 upon being received by R. We then distinguish three ICMP-related abilities for traffic from S to D: Ability A1: "R forwards ICMP packets destined to D on the final forward path when appropriate": upon receiving an ICMP packet from S, after A0 processing, when appropriate, does R forward it to D? Ability A2: "ICMP packet generation at R": if an event requiring the generation of an ICMP packet occurs, does R forge an ICMP packet and does R return it to S, regardless of whether this packet arrives to S or not? For instance this ICMP packet can result from an IPv4 TTL/IPv6 Hop Limit field that decrements to 0 during the forwarding of a packet received on the incoming forward path; Ability A3: "correct ICMP return path from R": when an ICMP packet is sent back to S from R (when it is generated by R) or through R (when it is generated by a router on the final forward path, D included), does this packet reach S? It not necessarily the case, even if R can generate ICMP packets, e.g., because of ICMP filtering rules; As a particular case, we also need to assess the destination D abilities. These abilities are essentially the same as for router R, with the distinction that: Ability A1': "D processes ICMP packets destined to D as required by the upper layers": Forwarding is replaced in that case by the handling of the packet to the upper layers; Those are the four main properties along which we assess R's behavior and D's behavior. As explained, these properties are (S, D) dependent (i.e. they depend on the path and direction), and there is no way to determine for sure all the features of router R with respect to ICMP. For instance, the R's behavior for another interface, that is neither the two ones used for traffic from S to D, may have completely different behavior when considering ICMP. Note that having a full characterization of R is not required, since our goal is to help solving ICMP black holes from S to D, and not to determine R's full configuration policies with respect to ICMP.

It is of common practice to probe each router R directly, e.g., with PING "Echo Request"/"Echo Reply" packets. This case almost corresponds to the previous case where R is the destination (D and R are identical). A limit of this approach is that there is no strong guarantee that the initial forward path to R is identical with a packet destined to R to the initial forward path to R with a packet destined to D. However, this probing method can still be useful to qualify ability A3, no matter whether the initial forward paths are the same or not. Indeed, A3 share in both cases the same ICMP packet source (R), destination (S) and protocol (ICMP).

Let us consider a path from a source S to a destination D, and let R_0, R_1, .. R_n-1 be routers along this path. Let us assume this path is fixed for the duration of the test (i.e., the list of the n routers will not change). Then the path characterization consists in providing the full list of abilities that could be determined for each router plus the destination. Of course, an ability A may not be determined, i.e., it can be either true or false, which is represented by (A + !A).

Let us consider a simple case () with two intermediate routers, R0 and R1, where R0 does not forward ICMP packets on path S to D. In that case there is no way to determine ability A1 on R1 and A1' on D, but other abilities may be determined since they rely on different mechanisms. For example, A0 can be determined by using UDP as a probing protocol, A2 can be determined by using a TTL value that decrements to 0 on R0, R1 or D with UDP or TCP as probing protocol, and since we assume that the ICMP packet returned on the return path arrives to S, A3 is determined too. This examples shows that the determination of abilities for router R can impact our detection of the abilities for router on the final forward path after R. Of course, it does not impact the routers abilities themselves, only our view of their abilities. Going further in the determination of all the router abilities may require additional control of the network, for instance via vintage points (i.e., routers or hosts within the core network that allows us to inject and observe traffic between S and D).

Let us consider the following example where the path between S and D crosses three routers: R0, R1 and R2. The user, located behind S, experiments bandwidth problems when using TCP (e.g., through an HTTP or SSH connection), and more precisely small TCP segments reach D but not large ones. In order to investigate the problem, the user will probably ping D and in that case will obtain a reply. Therefore this test is not of significant value to the user. So the user continues investigating the problem and uses the traceroute tool, with either TCP or ICMP as the probing protocols. The results are presented in . The user can deduce that R1, whose IP address (iP_R1) is obtained by the traceroute/ICMP test, does not forge any ICMP packet in response to TCP packets whose IPv4 TTL/IPv6 Hop Limit is decremented to 0 on R. Thus the user can deduce that its bandwidth problem is due to an erroneous configuration of R1, which by not forging ICMP packets when handling TCP packets, deprives S from using PMTUD.

If the example of can relatively easily be characterized, this is not the case with more complex situations. In we detail the IBTrack (ICMP Black hole Tracker) tool that provides a general methodology to achieve this kind of analysis. Depending on the assumptions that can be made, different tools can also be used. In particular if vantage points within the core network can be used, a more detailed diagnosis can probably be made by refining the observations, using different observations from different vantage points. Similarly, if the destination D can be involved in the process, some abilities of the routers may be refined, even if some of the ICMP return paths are not functioning. Therefore the present document should be regarded as a problem position document. Practical approaches and tools will be introduced and discussed in separate documents.

TBD

The authors want to thank the other authors of , namely Fabrice Schuler and Jean-Louis Roch.