]> Telchemy Incorporated

2905 Premiere Parkway, Suite 280 Duluth GA 30097 USA alan.d.clark@telchemy.com

Network Zen

77/440 Soi Phoomjit, Rama IV Road Phra Khanong, Khlong Toie Bangkok 10110 Thailand +66 (0) 87 502 4274 gwz@net-zen.net

Shanghai Research Institute of China Telecom Corporation Limited

No. 1835, South Pudong Road Shanghai 200122 China bijy@sttri.com.cn

Huawei

101 Software Avenue, Yuhua District Nanjing Jiangsu 210012 China sunseawq@huawei.com

Real-time Applications and Infrastructure Area Audio/Video Transport Working Group Real Time Control Protocol This document defines two RTP Control Protocol (RTCP) Extended Report (XR) blocks that allow the reporting of concealment metrics for audio applications of RTP.

At any instant, the audio output at a receiver may be classified as either 'normal' or 'concealed'. 'Normal' refers to playout of audio payload received from the remote end and also includes locally generated signals such as announcements, tones, and comfort noise. 'Concealed' refers to playout of locally generated signals used to mask the impact of network impairments or to reduce the audibility of jitter buffer adaptations. This document defines two new concealment-related block types to augment those defined in for use in a range of RTP applications. These two block types extend the packet loss concealment mechanism defined in Section 4.7.6 of . The first block type, the Loss Concealment Metrics Block, provides metrics for actions taken by the receiver to mitigate the effect of packet loss and packet discard. Specifically, the first metric (On-Time Playout Duration) reports the duration of normal playout of data that the receiver obtained from the sender's stream. A second metric (Loss Concealment Duration) reports the total time during which the receiver played out media data that was manufactured locally, because the sender's data for these periods was not available due to packet loss or discard. A similar metric (Buffer Adjustment Concealment Duration) reports the duration of playout of locally manufactured data replacing data that is unavailable due to adaptation of an adaptive de-jitter buffer. Further metrics (Playout Interrupt Count and Mean Playout Interrupt Size) report the number of times normal playout was interrupted and the mean duration of these interruptions. Loss Concealment Duration and Buffer Adjustment Concealment Duration are reported separately because buffer adjustment is typically arranged to occur in silence periods, so it may have very little impact on user experience, whilst loss concealment may occur at any time. The second block type, the Concealed Seconds Metrics Block, provides metrics for Concealed Seconds, which are measured at the receiving end of the RTP stream. Specifically, the first metric (Unimpaired Seconds) reports the number of whole seconds occupied only with normal playout of data that the receiver obtained from the sender's stream. The second metric (Concealed Seconds) reports the number of whole seconds during which the receiver played out any locally generated media data. A third metric, Severely Concealed Seconds (SCSs), reports the number of whole seconds during which the receiver played out locally generated data to conceal a lost or discarded frame percentage in excess of the configured SCS Threshold. These metrics belongs to the class of transport-related terminal metrics defined in .

The use of RTCP for reporting is defined in . defines an extensible structure for reporting using an RTCP Extended Report (XR). This document defines a new Extended Report block that MUST be used as defined in and .

The Performance Metrics Framework provides guidance on the definition and specification of performance metrics. The RTP Monitoring Framework provides guidelines for reporting block format using RTCP XR. The metrics blocks described in this document are in accordance with those guidelines.

These metrics are applicable to audio applications of RTP and the audio component of audio/video applications in which the packet loss concealment machinery is contained at the receiving end to mitigate the impact of network impairments to user's perception of media quality.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

The report blocks in this document make use of binary fractions. The following terminology is used: Numeric formats S X:Y where S indicates a two's complement signed representation, X the number of bits prior to the decimal place, and Y the number of bits after the decimal place. Hence, 8:8 represents an unsigned number in the range 0.0 to 255.996 with a granularity of 0.0039. S7:8 would represent the range -127.996 to +127.996. 0:16 represents a proper binary fraction with range 0.0 to 1 - 1/65536 = 0.9999847 though note that use of flag values at the top of the numeric range slightly reduces this upper limit. For example, if the 16-bit values 0xfffe and 0xffff are used as flags for "over-range" and "unavailable" conditions, a 0:16 quantity has range 0.0 to 1 - 3/65536 = 0.9999542

The Loss Concealment Metrics Block is intended to be used as described in this section, in conjunction with information from the Measurement Information Block . Instances of this metrics block refer by synchronization source (SSRC) to the separate auxiliary Measurement Information Block , which describes measurement periods in use (see , Section 4.2). This metrics block relies on the measurement period in the Measurement Information Block indicating the span of the report and SHOULD be sent in the same compound RTCP packet as the Measurement Information Block. If the measurement period is not received in the same compound RTCP packet as this metrics block, this metrics block MUST be discarded.

The structure of the Loss Concealment Metrics Block is as follows.

0 1 2 3 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | BT=30 | I |plc| resv | block length=6 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of Source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | On-Time Playout Duration | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Loss Concealment Duration | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Buffer Adjustment Concealment Duration | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Playout Interrupt Count | Reserved | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Mean Playout Interrupt Size | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

A Loss Concealment Metrics Block is identified by the constant 30. This field is used to indicate whether the loss concealment metrics are Sampled, Interval, or Cumulative metrics: I=10: Interval Duration - the reported value applies to the most recent measurement interval duration between successive metrics reports. I=11: Cumulative Duration - the reported value applies to the accumulation period characteristic of cumulative measurements. I=01: Sampled Value - the reported value is a sampled instantaneous value (not allowed in this block). I=00: Reserved value - this value is reserved for future use. In this document, Loss Concealment metrics can only be measured over definite intervals and cannot be sampled. Senders MUST NOT use the values I=00 or I=01. If a block is received with I=00 or I=01, the receiver MUST discard the block. This field is used to identify the packet loss concealment method in use at the receiver, according to the following code: bits 014-015 0 = silence insertion 1 = simple replay, no attenuation 2 = simple replay, with attenuation 3 = enhancement Other values are reserved. Note that the enhancement method (plc=3) for packet loss concealment offers an improved audio quality and better robustness against packet losses and is equivalent to "enhanced" in Section 4.7.6 of . These bits are reserved. They MUST be set to zero by senders and ignored by receivers (see , Section 4.2). The length of this report block in 32-bit words, minus one. For the Loss Concealment Metrics Block, the block length is equal to 6. As defined in Section 4.1 of . 'On-time playout' is the uninterrupted, in-sequence playout of valid decoded audio information originating from the remote endpoint. This includes comfort noise during periods of remote talker silence, if Voice Activity Detection (VAD) is used, and locally generated or regenerated tones and announcements. An equivalent definition is that on-time playout is playout of any signal other than those used for concealment. On-time playout duration is expressed in units of RTP timestamp and MUST include both speech and silence intervals, whether VAD is used or not. Two values are reserved: a value of 0xFFFFFFFE indicates out of range (that is, a measured value exceeding 0xFFFFFFFD), and a value of 0xFFFFFFFF indicates that the measurement is unavailable. The duration, expressed in units of RTP timestamp, of audio playout corresponding to Loss-Type concealment. Loss-Type concealment is reactive insertion or deletion of samples in the audio playout stream due to effective frame loss at the audio decoder. Effective frame loss is the event in which a frame of coded audio is simply not present at the audio decoder when required. In this case, substitute audio samples are generally formed, at the decoder or elsewhere, to reduce audible impairment. Two values are reserved: a value of 0xFFFFFFFE indicates out of range (that is, a measured value exceeding 0xFFFFFFFD), and a value of 0xFFFFFFFF indicates that the measurement is unavailable. The duration, expressed in units of RTP timestamp, of audio playout corresponding to Buffer Adjustment-Type concealment, if known. Buffer Adjustment-Type concealment is proactive or controlled insertion or deletion of samples in the audio playout stream due to jitter buffer adaptation, re-sizing decisions, or re-centering decisions within the endpoint. Because this insertion is controlled, rather than occurring randomly in response to losses, it is typically less audible than Loss-Type concealment. For example, jitter buffer adaptation events may be constrained to occur during periods of talker silence, in which case only silence duration is affected, or sophisticated time-stretching methods for insertion/deletion during favorable periods in active speech may be employed. Concealment events that cannot be classified as Buffer Adjustment-Type MUST be classified as Loss-Type. Two values are reserved: a value of 0xFFFFFFFE indicates out of range (that is, a measured value exceeding 0xFFFFFFFD), and a value of 0xFFFFFFFF indicates that the measurement is unavailable. The number of interruptions to normal playout that occurred during the reporting period. Two values are reserved: a value of 0xFFFE indicates out of range (that is, a measured value exceeding 0xFFFD), and a value of 0xFFFF indicates that the measurement is unavailable. These bits are reserved. They MUST be set to zero by senders and ignored by receivers (see , Section 4.2). The mean duration, expressed in units of RTP timestamp, of interruptions to normal playout that occurred during the reporting period. Two values are reserved: a value of 0xFFFFFFFE indicates out of range (that is, a measured value exceeding 0xFFFFFFFD), and a value of 0xFFFFFFFF indicates that the measurement is unavailable.

The Concealed Seconds Metrics Block is intended to be used as described in this section, in conjunction with information from the Measurement Information Block . It provides a description of potentially audible impairments due to lost and discarded packets at the endpoint, expressed on a time basis analogous to a traditional Public Switched Telephone Network (PSTN) T1/E1 errored seconds metric. Instances of this metrics block refer by synchronization source (SSRC) to the separate auxiliary Measurement Information Block that describes measurement periods in use (see , Section 4.2). This metrics block relies on the measurement period in the Measurement Information Block indicating the span of the report and SHOULD be sent in the same compound RTCP packet as the Measurement Information Block. If the measurement period is not received in the same compound RTCP packet as this metrics block, this metrics block MUST be discarded. The following metrics are based on successive one-second intervals as declared by an RTP clock. This RTP clock does not need to be synchronized to any external time reference. The starting time of this clock is unspecified. Note that this implies that the same loss pattern could result in slightly different count values, depending on where the losses occur relative to the particular one-second demarcation points. For example, two loss events occurring 50 ms apart could result in either one Concealed Second or two, depending on the particular one-second boundaries used. The seconds in this sub-block are not necessarily calendar seconds. At the tail end of a session, periods of time of less than one second shall be incorporated into these counts if they exceed 500 ms and shall be disregarded if they are less than 500 ms.

The structure of the Concealed Seconds Metrics Block is as follows.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | BT=31 | I |plc| resv | block length=4 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SSRC of Source | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Unimpaired Seconds | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Concealed Seconds | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Severely Concealed Seconds | Reserved | SCS Threshold | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

A Concealed Seconds Metrics Block is identified by the constant 31. This field is used to indicate whether the Concealed Seconds metrics are Sampled, Interval, or Cumulative metrics: I=10: Interval Duration - the reported value applies to the most recent measurement interval duration between successive metrics reports. I=11: Cumulative Duration - the reported value applies to the accumulation period characteristic of cumulative measurements. I=01: Sampled Value - the reported value is a sampled instantaneous value (Not allowed in this block). I=00: Reserved value - this value is reserved for future use. In this document, Concealed Seconds metrics can only be measured over definite intervals and cannot be sampled. Senders MUST NOT use the values I=00 or I=01. If a block is received with I=00 or I=01, the receiver MUST discard the block. This field is used to identify the packet loss concealment method in use at the receiver, according to the following code: bits 014-015 0 = silence insertion 1 = simple replay, no attenuation 2 = simple replay, with attenuation 3 = enhancement Other values are reserved. Note that the enhancement method (plc=3) for packet loss concealment offers an improved audio quality and a better robustness against packet losses and is equivalent to "enhanced" in Section 4.7.6 of . These bits are reserved. They MUST be set to zero by senders and ignored by receivers (see , Section 4.2). The length of this report block in 32-bit words, minus one. For the Concealed Seconds Metrics Block, the block length is equal to 4. As defined in Section 4.1 of . A count of the number of Unimpaired Seconds that have occurred.An Unimpaired Second is defined as a continuous period of one second during which no frame loss or discard due to late arrival has occurred. Every second in a session must be classified as either OK or Concealed. Normal playout of comfort noise or other silence-concealment signals during periods of talker silence, if VAD is used, shall be counted as Unimpaired Seconds. Two values are reserved: a value of 0xFFFFFFFE indicates out of range (that is, a measured value exceeding 0xFFFFFFFD), and a value of 0xFFFFFFFF indicates that the measurement is unavailable. A count of the number of Concealed Seconds that have occurred.A Concealed Second is defined as a continuous period of one second during which any frame loss or discard due to late arrival has occurred. Equivalently, a Concealed Second is one in which some Loss-Type concealment has occurred. Buffer Adjustment-Type concealment SHOULD NOT cause Concealed Seconds to be incremented, with the following exception. An implementation MAY cause Concealed Seconds to be incremented for 'emergency' buffer adjustments made during talkspurts. Loss-Type concealment is reactive insertion or deletion of samples in the audio playout stream due to effective frame loss at the audio decoder. "Effective frame loss" is the event in which a frame of coded audio is simply not present at the audio decoder when required. In this case, substitute audio samples are generally formed, at the decoder or elsewhere, to reduce audible impairment. Buffer Adjustment-Type concealment is proactive or controlled insertion or deletion of samples in the audio playout stream due to jitter buffer adaptation, re-sizing decisions, or re-centering decisions within the endpoint. Because this insertion is controlled, rather than occurring randomly in response to losses, it is typically less audible than Loss-Type concealment. For example, jitter buffer adaptation events may be constrained to occur during periods of talker silence, in which case only silence duration is affected, or sophisticated time-stretching methods for insertion/deletion during favorable periods in active speech may be employed. For these reasons, Buffer Adjustment-Type concealment MAY be exempted from inclusion in calculations of Concealed Seconds and Severely Concealed Seconds. However, an implementation SHOULD include Buffer Adjustment-Type concealment in counts of Concealed Seconds and Severely Concealed Seconds if the event occurs at an 'inopportune' moment, such as an emergency or large, immediate adaptation during active speech or an unsophisticated adaptation during speech without regard for the underlying signal. In these cases, the assumption of low audibility cannot hold. In other words, jitter buffer adaptation events that may be presumed to be audible SHOULD be included in Concealed Seconds and Severely Concealed Seconds counts. Concealment events that cannot be classified as Buffer Adjustment-Type MUST be classified as Loss-Type. For clarification, the count of Concealed Seconds MUST include the count of Severely Concealed Seconds. Two values are reserved: a value of 0xFFFFFFFE indicates out of range (that is, a measured value exceeding 0xFFFFFFFD), and a value of 0xFFFFFFFF indicates that the measurement is unavailable. A count of the number of Severely Concealed Seconds.A Severely Concealed Second is defined as a non-overlapping period of one second during which the cumulative amount of time that has been subject to frame loss or discard due to late arrival exceeds the SCS Threshold. Two values are reserved: a value of 0xFFFE indicates out of range (that is, a measured value exceeding 0xFFFD), and a value of 0xFFFF indicates that the measurement is unavailable. These bits are reserved. They MUST be set to zero by senders and ignored by receivers (see , Section 4.2). The SCS Threshold is defined as the percentage of packets corresponding to lost or discarded frames that must occur within a one second period in order for the second to be classified as a Severely Concealed Second. This is expressed in numeric format 0:8 and hence can represent a range of 0 to 99.6 percent loss or discard. A default threshold of 5 percent effective frame loss (50 ms effective frame loss ) per second is suggested. This corresponds to an SCS Threshold in hexadecimal of 0x0D.

defines the use of SDP (Session Description Protocol) for signaling the use of XR blocks. XR blocks MAY be used without prior signaling.

This section augments the SDP attribute "rtcp-xr" by providing two additional values of "xr-format" to signal the use of the two report blocks defined in this document.

]]>

When SDP is used in Offer/Answer context, the SDP Offer/Answer usage defined in applies. Note that "thresh" is declared by the offer.

New block types for RTCP XR are subject to IANA registration. For general guidelines on IANA considerations for RTCP XR, refer to .

This document assigns two block type values in the IANA "RTP Control Protocol Extended Reports (RTCP XR) Block Type Registry" under the subregistry "RTCP XR Block Type":

Name: LCB Long Name: Loss Concealment Metrics Block Value 30 Reference: Section 3.1 Name: CSB Long Name: Concealed Seconds Metrics Block Value 31 Reference: Section 4.1

This document also registers two new parameters in the "RTP Control Protocol Extended Reports (RTCP XR) Session Description Protocol (SDP) Parameters Registry": "loss-conceal" "conc-sec"

The contact information for the registrations is: RAI Area Directors rai-ads@tools.ietf.org

It is believed that the RTCP XR blocks defined in this document introduce no new security considerations beyond those described in . These blocks do not provide per-packet statistics, so the risk to confidentiality documented in Section 7, Paragraph 3 of does not apply.

Geoff Hunt wrote the initial version of this document.

The authors gratefully acknowledge reviews and feedback provided by Bruce Adams, Philip Arden, Amit Arora, Bob Biskner, Kevin Connor, Alissa Cooper, Claus Dahm, Randy Ethier, Roni Even, Adrian Farrel, Jim Frauenthal, Albert Higashi, Tom Hock, Shane Holthaus, Paul Jones, Rajesh Kumar, Keith Lantz, Alfred C. Morton, Mohamed Mostafa, Amy Pendleton, Colin Perkins, Mike Ramalho, Ravi Raviraj, Pete Resnick, Albrecht Schwarz, Meral Shirazipour, Tom Taylor, and Hideaki Yamada.

Wikipedia ITU-T

On-Time Playout Duration Metric Metric Name: On-Time Playout Duration Metric Description: 'On-time playout' is the uninterrupted, in-sequence playout of valid decoded audio information originating from the remote endpoint. On-time playout duration is playout duration of any signal other than those used for concealment. Method of Measurement or Calculation: See , On-Time Playout Duration definition. Units of Measurement: See , On-Time Playout Duration definition. Measurement Point(s) with Potential Measurement Domain: See , 3rd paragraph. Measurement Timing: See , 1st paragraph for measurement timing and for Interval Metric flag. Use and Applications: See . Reporting Model: See RFC 3611. Loss Concealment Duration Metric Metric Name: Loss Concealment Duration Metric Description: The duration of audio playout corresponding to Loss-Type concealment. Method of Measurement or Calculation: See , Loss Concealment Duration definition. Units of Measurement: See , Loss Concealment Duration definition. Measurement Point(s) with Potential Measurement Domain: See , 3rd paragraph. Measurement Timing: See , 1st paragraph for measurement timing and for Interval Metric flag. Use and Applications: See . Reporting Model: See RFC 3611. Buffer Adjustment Concealment Duration Metric Metric Name: Buffer Adjustment Concealment Duration Metric Description: The duration of audio playout corresponding to Buffer Adjustment-Type concealment. Method of Measurement or Calculation: See , Buffer Adjustment Concealment Duration definition. Units of Measurement: See , Buffer Adjustment Concealment Duration definition. Measurement Point(s) with Potential Measurement Domain: See , 3rd paragraph. Measurement Timing: See , 1st paragraph for measurement timing and for Interval Metric flag. Use and Applications: See . Reporting Model: See RFC 3611. Playout Interrupt Count Metric Metric Name: Playout Interrupt Count Metric Description: The number of interruptions to normal playout that occurred during the reporting period. Method of Measurement or Calculation: See , Playout Interrupt Count definition. Units of Measurement: See , Playout Interrupt Count definition. Measurement Point(s) with Potential Measurement Domain: See , 3rd paragraph. Measurement Timing: See , 1st paragraph for measurement timing and for Interval Metric flag. Use and Applications: See . Reporting Model: See RFC 3611. Mean Playout Interrupt Size Metric Metric Name: Mean Playout Interrupt Size Metric Description: The mean duration of interruptions to normal playout that occurred during the reporting period. Method of Measurement or Calculation: See , Playout Interrupt Count definition. Units of Measurement: See , Playout Interrupt Count definition. Measurement Point(s) with Potential Measurement Domain: See , 3rd paragraph. Measurement Timing: See , 1st paragraph for measurement timing and for Interval Metric flag. Use and Applications: See . Reporting Model: See RFC 3611. Unimpaired Seconds Metric Metric Name: Unimpaired Seconds Metric Description: A count of the number of Unimpaired Seconds that have occurred. Method of Measurement or Calculation: See , Unimpaired Seconds definition. Units of Measurement: See , Unimpaired Seconds definition. Measurement Point(s) with Potential Measurement Domain: See , 5th paragraph. Measurement Timing: See , 1st paragraph for measurement timing and paragraph for Interval Metric flag. Use and Applications: See . Reporting Model: See RFC 3611. Concealed Seconds Metric Metric Name: Concealed Seconds Metric Description: A count of the number of Concealed Seconds that have occurred. Method of Measurement or Calculation: See , Concealed Seconds definition. Units of Measurement: See , Concealed Seconds definition. Measurement Point(s) with Potential Measurement Domain: See , 5th paragraph. Measurement Timing: See , 1st paragraph for measurement timing and for Interval Metric flag. Use and Applications: See . Reporting Model: See RFC 3611. Severely Concealed Seconds Metric Metric Name: Severely Concealed Seconds Metric Description: A count of the number of Severely Concealed Seconds that have occurred. Method of Measurement or Calculation: See , Severely Concealed Seconds definition. Units of Measurement: See , Severely Concealed Seconds definition. Measurement Point(s) with Potential Measurement Domain: See , 5th paragraph. Measurement Timing: See , 1st paragraph for measurement timing and for Interval Metric flag. Use and Applications: See . Reporting Model: See RFC 3611. SCS Threshold Metric Metric Name: SCS Threshold Metric Description: The amount of time corresponding to lost or discarded frames that must occur within a one-second period in order for the second to be classified as a Severely Concealed Second. Method of Measurement or Calculation: See , SCS Threshold definition. Units of Measurement: See , SCS Threshold definition. Measurement Point(s) with Potential Measurement Domain: See , 5th paragraph. Measurement Timing: See , 1st paragraph for measurement timing and for Interval Metric flag. Use and Applications: See . Reporting Model: See RFC 3611.