ForCES Packet Parallelization

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

This document follows the terminology defined by the ForCES Model in . The required definitions are repeated below for clarity. FE Model - The FE model is designed to model the logical processing functions of an FE. The FE model proposed in this document includes three components; the LFB modeling of individual Logical Functional Block (LFB model), the logical interconnection between LFBs (LFB topology), and the FE-level attributes, including FE capabilities. The FE model provides the basis to define the information elements exchanged between the CE and the FE in the ForCES protocol [RFC5810]. LFB (Logical Functional Block) Class (or type) - A template that represents a fine-grained, logically separable aspect of FE processing. Most LFBs relate to packet processing in the data path. LFB classes are the basic building blocks of the FE model. LFB Instance - As a packet flows through an FE along a data path, it flows through one or multiple LFB instances, where each LFB is an instance of a specific LFB class. Multiple instances of the same LFB class can be present in an FE's data path. Note that we often refer to LFBs without distinguishing between an LFB class and LFB instance when we believe the implied reference is obvious for the given context. LFB Model - The LFB model describes the content and structures in an LFB, plus the associated data definition. XML is used to provide a formal definition of the necessary structures for the modeling. Four types of information are defined in the LFB model. The core part of the LFB model is the LFB class definitions; the other three types of information define constructs associated with and used by the class definition. These are reusable data types, supported frame (packet) formats, and metadata. Element - Element is generally used in this document in accordance with the XML usage of the term. It refers to an XML tagged part of an XML document. For a precise definition, please see the full set of XML specifications from the W3C. This term is included in this list for completeness because the ForCES formal model uses XML. Attribute - Attribute is used in the ForCES formal modeling in accordance with standard XML usage of the term, i.e., to provide attribute information included in an XML tag. LFB Metadata - Metadata is used to communicate per-packet state from one LFB to another, but is not sent across the network. The FE model defines how such metadata is identified, produced, and consumed by the LFBs, but not how the per-packet state is implemented within actual hardware. Metadata is sent between the FE and the CE on redirect packets. ForCES Component - A ForCES Component is a well-defined, uniquely identifiable and addressable ForCES model building block. A component has a 32-bit ID, name, type, and an optional synopsis description. These are often referred to simply as components. LFB Component - An LFB component is a ForCES component that defines the Operational parameters of the LFBs that must be visible to the CEs. LFB Class Library - The LFB class library is a set of LFB classes that has been identified as the most common functions found in most FEs and hence should be defined first by the ForCES Working Group.

A lot of network devices can process packets in a parallel manner. The ForCES Model presents a formal way to describe the Forwarding Plane's datapath with Logical Function Blocks (LFBs) using XML. This document describes how packet parallelization can be described with the ForCES model. The modelling concept has been influenced by Cilc. Cilc is a programming language that has been developed since 1994 at the MIT Laboratory to allow programmers to identify elements that can be executed in parallel. The two Cilc concepts used in this document is spawn and sync. Spawn being the place where parallel work can start and sync being the place where the parallel work finishes and must collect all parallel output.

This document addresses the following two types of packet parallelization: Flood - where a copy of a packet is sent to multiple LFBs to be processed in parallel. Split - where the packet will be split in equal size chunks specified by the CE and sent to multiple LFB instances probably of the same LFB class to be processed in parallel. It must be noted that the process of copying the packet in the Flood parallel type is implementation depended and is loosely defined here. An implementor may either decide to physical copy the packet and send all packets on the parallel paths, or may decide to logically copy the packet by simply sending for example pointers of the same packet provided that the necessary interlocks are taken into account. The implementor has to take into account the device's characteristics to decide which approach fits bets to the hardware. Additionally in the split parallel type, while harder, the implementor may also decide to logically split the packet and send for example pointers to parts of the packet, provided that the necessary interlocks are managed. This document introduces two LFBs that are used in before and after the parallelization occurs: Splitter - similar to Cilc's spawn. An LFB that will split the path of a packet and be sent to multiple LFBs to be processed in parallel. Merger - similar to Cilc's sync. An LFB that will receive packets or chunks of the same initial packet and merge them into one. Both parallel packet distribution types can currently be achieved with the ForCES model. The splitter LFB has one group output that produces either chunks or packets to be sent to LFBs for processing and the merger LFB has one group input that expects either packets or chunks to aggregate all the parallel packets or chunks and produce a single packet. shows an simple example of a split parallel datapath along with the splitter and merger LFB. shows an example of a flood parallel datapath along with the splitter and merger LFB.

| Regex LFB |----+ +----------+ | +------------+ | +----------+ | |---+ +------>| | | | +------------+ | | --->| Splitter |-------->| Regex LFB |----------->| Merger |---> | LFB | +------------+ | LFB | | |---+ +------>| | +----------+ | +------------+ | +----------+ +---->| Regex LFB |----+ +------------+ ]]>

| LFB |--->| LFB |--->| | --->| Splitter | +------------+ +-------+ | Merger |---> | LFB | | LFB | | | +------------+ | | | |--------->| IPv4 TTL |---------->| | +----------+ | Decrement | +----------+ | LFB | +------------+ ]]> This version of the modelling framework does not allow for nested parallel datapath topologies. This decision was reached by the authors and the ForCES working group as there was no strong use case or need at the time. This led to a more simple metadata definition needed to be transported between the splitter and the corresponding merger. If there is a need for nested parallel datapaths a new version of a splitter and merger will be needed to be defined as well as an augmentation to the defined metadata. One important element to a developer is the ability to define which LFBs can be used in a parallel mode, with which other LFBs can they be parallelized with and the order of the LFBs can be assembled. This information must be accessible in the core LFBs. However instead of appending one more capability in the FEObject LFB and changing the FEObject LFB, we opted for an alternative. We introduced an additional core LFB, the CoreParallelization, that will not have input and output ports, but simply the capability necessary for LFB parallelization. If this LFB is not supported in the FEObjects LFB's SupportedLFBs component and not instantiated in the FE, that means that the FE does not support LFB parallelization. The topology of the parallel datapath can be deferred and manipulated from the FEObject LFB's LFBTopology. The CoreParallelization requires only one capability in order to specify each LFB that can be used in a parallel mode: The Name of the LFB. The Class ID of the LFB. The Version of the LFB. The number of instances that class can support in parallel. A list of LFB classes that can follow this LFB class in a pipeline for a parallel path. A list of LFB classes that can exist before this LFB class in a pipeline for a parallel path. A list of LFB classes that can process packets or chunks in parallel with this LFB class.

ParallelLFBType Table entry for parallel LFBs LFBName The name of an LFB Class string LFBClassID The id of the LFB Class uint32 LFBVersion The version of the LFB Class used by this FE string LFBParallelOccurenceLimit The upper limit of instances of the same parallel LFBs of this class uint32 AllowedParallelAfters List of LFB Classes that can follow this LFB in a parallel pipeline uint32 AllowedParallelBefores List of LFB Classes that this LFB class can follow in a parallel pipeline uint32 AllowedParallel List of LFB Classes that this LFB class be run in parallel with uint32 ParallelLFBs List of all supported parallel LFBs ParallelLFBType ]]> While the ForCES model cannot describe how the splitting or the merging is actually done as that is an implementation issue of the actual LFB, however this document defines operational parameters to control the splitting and merging, namely the size of the chunks, what happens if a packet or chunk has been marked as invalid and whether the merge LFB should wait for all packets or chunks to arrive. Additionally this document defines metadata, which contain necessary information to assist the merging procedure. The following metadata set as a struct is defined: ParallelType - Flood or split Correlator - Identify packets or chunks that belonged to the initial packet that entered the Splitter LFB ParallelNum - Number of packet or chunk for specific Correlator. ParralelPartsCount - Total number of packets or chunks for specific Correlator. This metadata is produced from the Splitter LFB and is opaque to LFBs in parallel paths and is passed along to the merger LFB without being consumed. In case of a packet/chunk being branded invalid by an LFB in a parallel path, it MUST be sent by an output port of said LFB An LFB inside a parallel path decides that a packet or a chunk has to be dropped it MAY drop it but the metadata MUST be sent to the Merger LFB's InvalidIn input port for merging purposes. Additional metadata produced by LFBs inside a datapath MAY be aggregated within the Merger LFB and sent on after the merging process. In case of receiving the same metadata definition with multiple values the merger LFB MUST keep the first received from a valid packet or chunk.

One frame type has been defined in this library. Frame Type Name Synopsis Chunk A chunk is a frame that is part of an original larger frame

One data type has been defined in this library. DataType Name Type Synopsis ParallelTypes Atomic uchar. Special Values Flood (0), Split (1). The type of parallelization this packet will go through

The following metadata structure with ID 16, using the ForCES model extension, is defined for the parallelization library: Metadata Name Type ID Synopsis ParallelType uchar 1 The type of parallelization this packet will go through. 0 for flood, 1 for split. Correlator uint32 2 An identification number to specify that packets or chunks belong to the same parallel work. ParallelNum uint32 3 Defines the number of the specific packet or chunk of the specific parallel ID. ParallelPartsCount uint32 4 Defines the total number of packets or chunks for the specific parallel ID.

A splitter LFB takes part in parallelizing the processing datapath by sending either the same packet or chunks of the same packet to multiple LFBs.

The splitter LFB receives any kind of packet via the singleton input, Input. Depending upon the CE's configuration of the ParallelType component, if the parallel type is of type flood (0), the same packet MUST be sent through all of the group output ParallelOut's instances. If the parallel type is of type split (1), the packet will be split into same size chunks except the last which MAY be smaller, with the max size being defined by the ChunkSize component. All chunks will be sent out in a round-robin fashion through the group output ParallelOut's instances. Each packet or chunk will be accompanied by the following metadata set as a struct : ParallelType - The paralleltype split or flood. Parallel ID - generated by the splitter LFB to identify that chunks or packets belong to the same parallel work. Parallel Num - each chunk or packet of a parallel id will be assigned a number in order for the merger LFB to know when it has gathered them all along with the ParallelPartsCount metadata. ParallelPartsCount - the number of chunks or packets for the specific parallel id.

This LFB has only two components specified. The first is the ParallelType, an uint32 that defines how the packet will be processed by the Splitter LFB. The second is the ChunkSize, an uint32 that specifies the maximum size of a chunk when a packet is split into multiple same size chunks.

This LFB has only one capability specified, the MinMaxChunkSize a struct of a uint32 to specify the minimum chunk size and a uint32 to specify the maximum chunk size.

This LFB has no events specified.

A merger LFB receives multiple packets or multiple chunks of the same packet and merge them into one merged packet.

The Merger LFB receives either a packet or a chunk via the group input ParallelIn, along with the ParallelType metadata to identify whether what was received was a packet or a chunk, the Correlator, the ParallelNum and the ParallelPartsCount. In case that an LFB has dropped a packet or a chunk within a parallel path the merger LFB MAY receive only the metadata or both metadata and packet or chunk through the InvalidIn group input port. It SHOULD receive a metadata specifying the error code. Current defined metadata's in the Base LFB Library are the ExceptionID and the ValidateErrorID. The Merger LFB MAY store the parallel metadata along with the exception metadata as a string in the optional InvalideMetadataSets as a means for the CE to debug errors in the parallel path. If the MergeWaitType is set to false the Merger LFB will initiate the merge process upon receiving the first packet. If false it will wait for all packet in the Correlator to arrive. If one packet or chunk has been received through the InvalidIn port then the merging procedure will be operate as configured by the InvalidAction component. If the InvalidAction component has been set to 0 then if one packet or chunk is not valid all will dropped, else the process will initiate. Once the merging process has been finished the resulting packet will be sent via the singleton output port PacketOutput. If the Merger LFB receives different values for the same metadata from different packets or chunks that has the same correlator then the Merger LFB will use the first metadata from a packet or chunk that entered the LFB through the ParallelIn input port.

This LFB has the following components specified: InvalidAction - a uchar defining what the Merge LFB will do if an invalid chunk or packet is received. If set to 0 (DropAll) the merge will be considered invalid and all chunks or packets will be dropped. If set to 1 (Continue) the merge will continue. MergeWaitType - a boolean. If true the Merger LFB will wait for all packets or chunks to be received prior to sending out a response. If false, when one packet or a chunk with a response is received by the merge LFB it will start with the merge process. InvalidMergesCounter - a uint32 that counts the number of merges where there is at least one packet or chunk that entered the merger LFB through the InvalidIn input port. InvalidAllCounter - a uint 32 that counts the number of merges where all packets/chunks entered the merger LFB through the InvalidIn input port. InvalidIDCounters - a struct of two arrays. Each array has a uint32 per row. Each array counts number of invalid merges where at least one packet or chunk entered through InvalidID per error ID. The first array is the InvalidExceptionID and the second is the InvalidValidateErrorID. InvalideMetadataSets - an array of strings. An optional component that stores metadata sets along with the error id as a string. This could provide a debug information to the CE regarding errors in the parallel paths.

This LFB has no capabilities specified.

This LFB specifies only two event. The first detects whether the InvalidMergesCounter has exceeded a specific value and the second detects whether the InvalidAllCounter has exceeded a specific value. Both error reports will send the respective counter value.

A core LFB that specifies that the FE supports parallelization, instead of updating the FEObject LFB

The CoreParallelization does not handle data. It is a core LFB that has only one capability.

This LFB has no components specified.

This LFB has only one capability specified. The ParallelLFBs is a table which lists all the LFBs that can be parallelized. Each row of the table contains: LFBName - a string. The Name of the parallel LFB. LFBClassID - a uint32. The Class ID of the parallel LFB. LFBVersion - a string. The Version of the parallel LFB. LFBParallelOccurenceLimit - a uint32. The upper limit of instances of the same parallel LFBs of this class. AllowedParallelAfters - a table of uint32s (LFB Class IDs). A list of LFB classes that can follow this LFB class in a pipeline for a parallel path. AllowedParallelBefores - a table of uint32s (LFB Class IDs). A list of LFB classes that can exist before this LFB class in a pipeline for a parallel path. AllowedParallel - a table of uint32s (LFB Class IDs). A list of LFB classes that can process packets or chunks in parallel with this LFB class.

This LFB specifies no events

Chunk A chunk is a frame that is part of an original larger frame ParallelTypes The type of parallelization this packet will go through uchar Flood The packet/chunk has been sent as a whole to multiple recipients Split The packet/chunk has been split into multiple chunks and sent to recipients ParallelLFBType Table entry for parallel LFBs LFBName The name of an LFB Class string LFBClassID The id of the LFB Class uint32 LFBVersion The version of the LFB Class used by this FE string LFBParallelOccurenceLimit The upper limit of instances of the same parallel LFBs of this class uint32 AllowedParallelAfters List of LFB Classes that can follow this LFB in a parallel pipeline uint32 AllowedParallelBefores List of LFB Classes that this LFB class can follow in a parallel pipeline uint32 AllowedParallel List of LFB Classes that this LFB class be run in parallel with uint32 ParallelMetadataSet A metadata Set for parallelization related LFBs 32 ParallelType The type of parallelization this packet/chunk has gone through ParallelTypes Correlator An identification number to specify that packets or chunks originate from the same packet. uint32 ParallelNum Defines the number of the specific packet or chunk of the specific parallel ID. uint32 ParallelPartsCount Defines the total number of packets or chunks for the specific parallel ID. uint32 Splitter A splitter LFB takes part in parallelizing the processing datapath. It will either send the same packet or chunks of one packet to multiple LFBs 1.0 PacketIn An input port expecting any kind of frame Arbitrary ParallelOut An parallel output port that sends the same packet to all output instances or chunks of the same packet different chunk on each instance. Arbitrary Chunk ParallelMetadataSet ParallelType The type of parallelization this packet will go through ParallelTypes ChunkSize The size of a chunk when a packet is split into multiple same size chunks uint32 MinMaxChunkSize The minimum and maximum size of a chunk capable of splitted by this LFB MinChunkSize Minimum chunk size uint32 MaxChunkSize Maximum chunk size uint32 Merger A merger LFB receives multiple packets or multiple chunks of the same packet and merge them into one merged packet 1.0 ParallelIn An parallel input port that accepts packets or chunks from all output instances Arbitrary Chunk ParallelMetadataSet InvalidIn When a packet is sent out of an error port of an LFB in a parallel path will be sent to this output port in the Merger LFB Arbitrary Chunk

ExceptionID ValidateErrorID

PacketOutput An output port expecting any kind of frame Arbitrary InvalidAction What the Merge LFB will do if an invalid chunk or packet is received uchar DropAll Drop all packets or chunks Continue Continue with the merge MergeWaitType Whether the Merge LFB will wait for all packets or chunks to be received prior to sending out a response boolean InvalidMergesCounter Counts the number of merges where there is at least one packet/chunk that entered the merger LFB through the InvalidIn input port uint32 InvalidAllCounter Counts the number of merges where all packets/chunks entered the merger LFB through the InvalidIn input port uint32 InvalidIDCounters Counts number of invalid merges where at least one packet/chunk entered through InvalidID per error ID InvalidExceptionID Per Exception ID uint32 InvalidValidateErrorID Per Validate Error ID uint32 InvalideMetadataSets Buffers metadata sets along with the error id as a string. string ManyInvalids An event that specifies if there are too many invalids InvalidCounter InvalidMergesCounter ManyAllInvalids An event that specifies if there are too many invalids InvalidCounter InvalidAllCounter CoreParallelization A core LFB that specifies that the FE supports parallelization, instead of updating the FEObject LFB 1.0 ParallelLFBs A table which lists all the LFBs that can be parallelized ParallelLFBType ]]>

The authors would like to thank Jamal Hadi Salim and Dave Hood for comments and discussions that made this document better.

LFB classes defined by this document belong to LFBs defined by Standards Track RFCs. According to IANA, the registration procedure is Standards Action for the range 0 to 65535 and First Come First Served with any publicly available specification for over 65535. This specification includes the following LFB class names and LFB class identifiers: LFB Class Identifier LFB Class Name LFB Version Description Reference 18 Splitter 1.0 A splitter LFB will either send the same packet or chunks of one packet to multiple LFBs. This document 19 Merger 1.0 A merger LFB receives multiple packets or multiple chunks of the same packet and merge them into one. This document 20 CoreParallelization 1.0 A core LFB to signify the parallelization capability This document

The Metadata ID namespace is 32 bits long. Values assigned by this specification: Value Name Definition 0x00000010 ParallelMetadataSet This document