One Administrative Domain

One of the basic assumptions of Internet deployment is that different Autonomous Systems (ASes) belong to different administrative authorities that use independent policies. Therefore, attributes such as LOCAL_PREF are not sent across AS boundary. As networks have evolved, such an assumption may not always hold. A single administrative authority such as a Service Provider (SP) may own equipments in multiple ASes and may instantiate services on and across multiple ASes. As a result, an SP customer's end-points may be connected to multiple ASes even though the customer expects the SP network to behave as a single "domain". For instance, a customer utilizing LOCAL_PREF to influence routing expects that the expressed routing preference be preserved at all of their endpoints whether or not they are connected to same or different ASes. This expectation is reasonable since the ASes, being under the same administrative authority, use consistent policies and LOCAL_PREF set in one AS would be comparable in another AS (when designed to be so). To facilitate such control, this document proposes an approach where non-transitive attributes are tunneled across ASes and are interpreted at traffic ingress points.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

One Administrative Domain (OAD): A collection of autonomous systems (ASes) that are managed by a single administrative entity. They do not appear any different to ASes that belong to a separate administration.

Today a large SP network often consists of multiple ASes, for instance, reflecting the SP's internal management structure. The SP provides services across those ASes to its customers. Some of the sites of a given customer may be connected to one AS whereas some of the other sites of the same customer may be connected to another AS. However, for these customers, the SP network is a single entity. In many instances, the customer desires the routing behavior between two of its sites be uniform whether or not these sites are in the same AS or in different ASes. provides a typical example of a VPN customer. A customer site with equipment, CE1, is dual-homed to the provider in AS1. A second site of the customer with CE2 is also connected to AS1. A third site of the same customer with CE3 is connected to AS2. CE1 advertises a route. The customer sets different LOCAL_PREF for its two links to the provider network and thereby chooses one of the links as the primary path. CE2 receives the LOCAL_PREFs and correctly uses the preferred link for forwarding. However, CE3 doesn't receive the LOCAL_PREFs since LOCAL_PREF is not sent across ASes. So CE3 might start to load balance the traffic to CE1 over both links, or might use the non-preferred link solely. In this scenario, the two ASes are contiguous and under the same administrative domain. So it is desirable that the SP customer be able to use the simple mechanism of setting LOCAL_PREF to influence routing decisions irrespective of the internal design of the provider network. In other words, it is desirable to make the OAD behave essentially as one AS. The SP may be able to solve the issue by mapping LOCAL_PREF to a community in AS1, allowing the community to go across the AS boundary and finally reverse mapping the community to LOCAL_PREF in AS2. However, an approach like that is narrow in scope and is difficult to manage in a large network.

Multiple ASes under the same administrative authority may not always be contiguous. shows a scenario where two ASes, AS1 and AS2, that belong to the same provider, are separated by an AS that is owned by a third party. Such a scenario may arise due to merger of two SPs. While the mechanism proposed in this draft would work in the same way, caution must be exercised in exposing internal parameters of the provider network to a 3rd party transit AS. We acknowledge that one can consider fixing the problem described here by merging the ASes into one AS (i.e., by renumbering them to one ASN). However, in many cases that is not a viable option. Instead, the solution described here allows an OAD consisting of multiple ASes to essentially behave as a single AS.

The problem described in arises because non-transitive attributes that crucially influence routing decisions are dropped at AS boundaries. The key idea is to carry these non-transitive attributes to the traffic ingress point. BGP already supports attribute tunneling by using the ATTR_SET attribute that transperantly carries multiple attributes that need to be preserved across AS boundaries (). However, ATTR_SET can carry only one set of attributes. As shown in the examples later on, a solution for the present problem needs to carry two sets of attributes, (i) the attribute set for the edge (PE to CE connection, to address the problem described in ), and (ii) the attribute set for the core (PE to RR connection). Moreover, a mechanism is needed to differentiate the set of attributes for the core from the set of attributes for the edge. Such distinction is needed even if, say, only the attributes for the core is present. Towards this end, this document generalizes the attribute tunneling mechanism by introducing a new attribute called ATTR_SET_STACK that carries multiple ATTR_SETs by stacking them. This approach allows adding multiple ATTR_SETs as well as preserves the sequence in which they must be used. The attribute is defined as shown in . The 'Length' field of ATTR_SET_STACK includes the cumulative length, in octet, of all the ATTR_SET attributes. In this document we define the rules for stacking two ATTR_SET attributes, which are sufficient for the purpose of OAD. We keep the rules open to future additions to support applications that may require more than two ATTR_SET attributes. Rules: When an AS border router (ASBR) advertises a route that doesn't have an ATTR_SET_STACK attribute to another AS, if allowed by the policy, the ASBR Creates an ATTR_SET_STACK attribute, "Pushes" any existing ATTR_SET attribute in the ATTR_SET_STACK attribute. Encodes the current attributes in an ATTR_SET and "pushes" this ATTR_SET in the ATTR_SET_STACK attribute. Thus, when there are edge attributes to tunnel, the ASBR creates an ATTR_SET_STACK attribute with two ATTR_SET attributes in it with the ATTR_SET for the edge attributes at the bottom. When only core attributes are to be tunneled, it creates an ATTR_SET_STACK attribute with one ATTR_SET attribute in it carrying the core (set by PE) attributes. An ingress PE that imports the route "pops" the top ATTR_SET attributes from the ATTR_SET_STACK. If permitted by the local policy, it uses the attributes from it in its best path selection process. When an ingress PE advertises an imported route to a CE, only the bottom ATTR_SET element is advertised to it (without any ATTR_SET_STACK attribute wrapper). If a router receives a route with an ATTR_SET_STACK attribute, and it propagates that route to one of its peers, then if the peer is trusted, the peer receives the route with the same ATTR_SET_STACK attribute; otherwise the ATTR_SET_STACK is removed from the route. Note that the creation of ATTR_SET_STACK is controlled by local policy (discussed later) and SHOULD be done only for trusted peer ASes.

In this section, we provide some examples of customer accessing VPN service from a provider to illustrate the difference between the existing behavior and the OAD behavior.

This is a simpler case of a customer connected to only one provider network and there is no edge attribute set.

Lbl1, NH=PE1, LP=100 | +------ PE2 | -> Lbl2, NH=PE2, LP=100 | (RD2)A/B, Lbl2 +--------------------------+ ]]>

Lbl1, NH=PE1, LP=100 | +------ PE2 | ATTR_SET: LP=200 | (RD2)A/B, Lbl2 | -> Lbl2, NH=PE2, LP=100 | | ATTR_SET: LP=150 | +--------------------------+ ]]> As shown in , the provider network consisting of two ASes connected by option C technique (). The customer site with CE1 is dual-homed and advertises prefix A/B to PE1 and PE2. Customer prefers the PE1-CE1 link. This preference is expressed by setting LOCAL_PREF to 200 on the route advertised by PE1 whereas PE2 sets LOCAL_PREF to 150. The second customer site with CE2 is connected to PE3 in AS2. Each PE uses a unique RD. So PE3 receives two prefixes: (RD1)A/B and (RD2)A/B, and imports them into (RD3)A/B. Therefore, the prefix (RD3)A/B has two paths. The first path is with nexthop PE1 (in option C, the nexthops remain unchanged), and the second path is with nexthop PE2. When RR1 sends the routes to RR2, since they are in different ASes, RR1 does not send LOCAL_PREFs to RR2. So when RR2 sends the routes to PE3, it sends default LOCAL_PREF (shown as 100). I.e., PE3 loses the route preferences that were set in AS1. When OAD behavior is turned on on RR1 (and RR2 is added as a trusted peer), when RR1 sends the routes to RR2, it creates an ATTR_SET_STACK attribute with one ATTR_SET in it that contains the LOCAL_PREF of the route. When PE3 imports the routes into (RD3)A/B, it extracts the LOCAL_PREFs from the ATTR_SET_STACK (which contains only one ATTR_SET attribute). Therefore, PE3 has both the LOCAL_PREF set by PE1 and PE2 (coming from the ATTR_SET_STACK) and the (default) LOCAL_PREF set by RR2. As per the policy set on PE2, the LOCAL_PREFs coming from AS1 can be used by PE2 for computing best path and hence honor the routing preferences set by the customer. This behavior is depicted in .

Lbl1', NH=ASBR2, LP=100 | +------ PE2 | -> Lbl2', NH=ASBR2, LP=100 | A/B (RD2, Lbl2) +-----------------------------+ ]]>

Lbl1', NH=ASBR2, LP=100 | +------ PE2 | ATTR_SET: LP=200 | A/B (RD2, Lbl2) | -> Lbl2', NH=ASBR2, LP=100 | | ATTR_SET: LP=150 | +-----------------------------+ ]]> shows the same provider network when its two ASes are connected by option B (). Similar to the option C case, on PE3, the prefix (RD3)A/B has two paths, but both with nexthop ASBR2. The VPN label of each route is changed by ASBR2, which allows the packet to ultimately reach PE1 or PE2. Similar to option C, ASBR1 does not send LOCAL_PREFs to ASBR2. So PE3 loses the route preferences that were set in AS1. When OAD behavior is turned on on ASBR1 (and ASBR2 is added as a trusted peer), when ASBR1 sends the routes to ASBR2, it creates an ATTR_SET_STACK attribute with one ATTR_SET in it that contains the LOCAL_PREF of the route. This way PE3 receives both the LOCAL_PREF set by PE1 and PE2 (coming from the ATTR_SET_STACK) and the (default) LOCAL_PREF set by ASBR2. Therefore PE2 can honor the routing preferences set by the customer.

Lbl1', LP=100 | | /LP=150 | | core ATTR_SET: LP=200 | | / | | edge ATTR_SET: LP=100 | | PE2(Lbl2) | | -> Lbl2', LP=100 | ------|---/---------------------------- | core ATTR_SET: LP=150 | | / | | edge ATTR_SET: LP=100 | CE1 CE2 +--------------------------+ \ / +------------------------+ +-----+ \ / |A/B NH=Provider1(LP=100)| |A/B | /----------------------\ | NH=Provider2(LP=90) | |LP=90| | Provider 2 | +------------------------+ +-----+ \----------------------/ ]]> This example considers the scenario when there is both an edge ATTR_SET and a core ATTR_SET. The scenario is shown in where a customer utilizes enterprise VPN service from both Provider 1 and Provider 2. Provider 1 runs an OAD consisting of two ASes, AS1 and AS2, connected by interAS Option B or Option C techniques. To Provider 1, the customer connects one site at AS1 via CE1 and another site at AS2 via CE2. At AS1, CE1 is dual-homed connecting to PE1 and PE2 as IBGP () and prefers PE1. CE1 originates a route, A/B, that it advertises to CE2 via both Provider 1 and Provider 2. CE1 prefers Provider 1 by setting the LOCAL_PREF attribute to 100 towards Provider 1 and to 90 towards Provider 2. Within Provider 1, since PE1 is preferred by the customer, PE1 advertises A/B to RR1 with LOCAL_PREF 200 (and label Lbl1) and PE2 advertises A/B with LOCAL_PREF 150 (and label Lbl2). RR1 preserves both routes since PE1 and PE2 uses different route-distinguishers for the customer VPN route. In Provider 1's OAD, PE3 receives two routes for A/B: the first one with label Lbl1' and a next-hop that takes the packet to PE1, and the second one with label Lbl2' and a next-hop that takes the packet to PE2. CE2 receives one route each from Provider 1 (at AS2) and Provider 2. By using the mechanism described in , CE2 sees the LOCAL_PREF attributes set by CE1 and chooses Provider 1's path and sends traffic to PE3. PE3 does not have any visibility into the LOCAL_PREFs that PE1 or PE2 has set (as LOCAL_PREF is non-transitive attribute) and may choose the path with Lbl2' as its bestpath and send traffic to PE2 violating the intent of the customer to receive traffic via PE1. When OAD is turned on, PE3 receives the ATTR_SET_STACK attribute containing two ATTR_SETs: (i) the top ATTR_SET containing the core attributes (set by PE1 or PE2), (ii) the bottom ATTR_SET containing the edge attributes that comes from the CE. PE3 extracts the top ATTR_SET for its own best path computation and sends the bottom ATTR_SET to CE2. This way PE3 is able to honor the prefences set in AS1.

An implementation MUST allow the operator to identify the neighbors that belong to the same OAD, and/or are trusted. An implementation MUST allow the operator to specify whether the attributes from the ATTR_SET (within an ATTR_SET_STACK) are to be used for best path computation. Note that attributes MUST not be mixed; i.e., either only the attributes from an ATTR_SET are used, or no attribute from an ATTR_SET are used.

When non-transitive attributes such as LOCAL_PREF are tunneled across AS boundary, the values used for these attributes must be consistent across different ASes in an OAD. When the originator sends an ATTR_SET_STACK attribute to a 3rd party peer AS, even if the peer AS is a transit AS with respect to the provider network, the peer AS may extract the ATTR_SETs and use them for its own calculations (e.g., if the customer also has a site connected to the 3rd party AS). If the routing policies of the 3rd party AS is not consistent with the originator AS, routing inconsistencies may occur. Therefore, ATTR_SET_STACK attribute may be sent to a peer AS only if the peer AS is trusted. In this context, a trusted AS is either in the same OAD, or it is contractually bound to treat the ATTR_SET_STACK attribute as an opaque attribute, or its routing policy is consistent with the originator AS. A route carrying an ATTR_SET attribute potentially has two sets of non-transitive attributes for possible use: (i) those in the ATTR_SET, and (ii) those carried by the route. The non-transitive attributes are given a "global" scope when those in the ATTR_SET are used. Sometimes, however, a "local" scope may be preferred in some ASes in a given OAD, in which case the non-transitive attributes carried by the route are used. Local policy must govern which set of attributes should be used.

IANA shall assign a value from the "BGP Path Attributes" registry, to be called "ATTR_SET_STACK", with this document as the reference.

The proposed mechanism allows non-transitive attributes to be sent across AS boundary. Sending the non-transitive attributes to non-trusted peers can create routing inconsistencies and other vulnerabilities and MUST not be done. Procedures and protocol extensions defined in this document do not otherwise affect the BGP security model.