Fact-checked by Grok 2 weeks ago

Explicit Congestion Notification

Explicit Congestion Notification (ECN) is a network protocol extension that enables routers to explicitly signal impending congestion to endpoints without dropping packets, thereby improving the efficiency of congestion control in IP networks.^[1] ECN was first proposed on an experimental basis in RFC 2481 in January 1999, before being advanced to standards track and defined in RFC 3168, published in September 2001.^[2]^[1] ECN integrates with both the Internet Protocol (IP) and Transmission Control Protocol (TCP) by repurposing two bits in the IP header—specifically bits 6 and 7 of the Differentiated Services (DS) field in IPv4 or the Traffic Class field in IPv6—to indicate ECN capability and congestion status.^[1] These bits define four codepoints: 00 for Not-ECN-Capable Transport (Not-ECT), indicating non-ECN support; 10 for ECN-Capable Transport (ECT(0)); 01 for ECT(1), which provides differentiation for potential future uses; and 11 for Congestion Experienced (CE), which routers set to mark packets when queue thresholds signal congestion.^[1] In operation, a TCP sender negotiates ECN capability during the connection setup by setting the ECN-Echo (ECE) and Congestion Window Reduced (CWR) flags in the SYN and SYN-ACK segments; upon agreement, the sender marks outgoing packets with an ECT codepoint.^[1] If a router detects congestion via mechanisms like Random Early Detection (RED), it sets the CE codepoint on the packet instead of discarding it, allowing the receiver to echo this indication back to the sender using the ECE flag in acknowledgments.^[1] The sender then responds by reducing its congestion window as if a packet loss had occurred and signals confirmation to the receiver via the CWR flag, ensuring end-to-end feedback without interrupting data flow.^[1] This approach addresses the limitations of traditional TCP congestion control, which relies solely on packet drops to infer network overload, by providing proactive notification that reduces latency and packet loss, particularly benefiting real-time and delay-sensitive applications.^[1] ECN supports incremental deployment, as non-ECN devices can coexist by treating ECN-marked packets as normal, though middleboxes may require configuration to avoid interfering with ECN signaling.^[1] Subsequent standards, such as RFC 6679 for RTP over UDP^[3] and RFC 9331 for Low Latency, Low Loss, Scalable throughput (L4S) service,^[4] have extended ECN's applicability to other protocols and advanced congestion management scenarios.

Introduction

Definition and Purpose

Explicit Congestion Notification (ECN) is an extension to the Internet Protocol (IP) and transport protocols like TCP that enables routers to signal impending congestion to endpoints by marking IP packets instead of discarding them.^[5] This mechanism utilizes two bits in the IP header—the ECN-Capable Transport (ECT) codepoint to indicate ECN support and the Congestion Experienced (CE) codepoint to mark packets when congestion is detected.^[5] The primary purpose of ECN is to provide end-to-end notification of network congestion, allowing senders to reduce their transmission rates proactively before packet loss occurs, thereby enhancing overall network performance and efficiency.^[5] In contrast to traditional drop-based signaling in TCP congestion control algorithms like Reno or CUBIC, which interpret packet drops as congestion indicators and trigger window reductions after loss, ECN avoids the associated inefficiencies such as retransmission delays and throughput degradation.^[6]^[7]^[8] A central concept in ECN is the CE mark, which routers apply to packets when queue buildup signals incipient congestion, informing the receiver and ultimately the sender without interrupting the data flow.^[5] For instance, in a router facing queue overflow, an ECN-capable packet with the ECT codepoint would have its header marked with CE rather than being dropped, enabling the transport protocol to relay this feedback to the sender for rate adjustment.^[5] This approach mitigates the delays and resource waste from lost packets, supporting better utilization in IP networks.^[8]

Historical Development

The concepts underlying Explicit Congestion Notification (ECN) emerged from efforts to improve TCP's congestion control by providing proactive signals from routers, rather than relying solely on packet loss as an implicit indicator. In 1993, Sally Floyd and Van Jacobson published a seminal paper introducing Random Early Detection (RED), a gateway algorithm that probabilistically discards packets during early stages of congestion to notify TCP senders, addressing the limitations of TCP's loss-based feedback in preventing buffer overflows and global synchronization.^[9] This work laid the foundation for explicit signaling mechanisms. Building directly on RED, Floyd's 1994 paper, "TCP and Explicit Congestion Notification," proposed replacing probabilistic drops with bit markings in packet headers to explicitly notify endpoints of congestion, enabling faster and more efficient responses without packet loss.^[10] The path to standardization began in the late 1990s within the IETF. In January 1999, RFC 2481 was issued as an experimental specification by K. K. Ramakrishnan and Sally Floyd, outlining the addition of ECN to the IP header's Type of Service octet and corresponding modifications to TCP for negotiating and handling ECN signals.^[11] The IETF's Explicit Congestion Notification (ECN) Working Group, chartered around 1999 and active through 2001, coordinated these efforts to ensure compatibility across network elements.^[12] In September 2001, RFC 3168, authored by Ramakrishnan, Floyd, and David Black, obsoleted RFC 2481 and advanced ECN to Proposed Standard status, providing comprehensive definitions for ECN codepoints in IP, detailed TCP semantics, and interoperability guidelines to support backward compatibility with non-ECN traffic. Despite standardization, ECN deployment remained limited for over a decade due to challenges such as middlebox interference, where firewalls and NAT devices often dropped or rewrote ECN-marked packets, mistaking them for malformed traffic.^[13] Adoption began accelerating in the 2010s amid surging Internet traffic from data centers, video streaming, and mobile networks, which amplified the inefficiencies of loss-based congestion control. In January 2018, RFC 8311 updated RFC 3168 by relaxing experimental restrictions, permitting both ECN-Capable Transport (ECT) codepoints for marking and encouraging broader testing to support low-latency applications and more precise congestion feedback. This revival reflected the growing need for scalable, drop-avoiding mechanisms in high-bandwidth environments.

Standards and Specifications

Core RFCs

The foundational standards for Explicit Congestion Notification (ECN) are defined in a series of IETF RFCs that establish its operation at the IP layer and integration with transport protocols. RFC 3168, published in 2001, introduces ECN as an extension to IP and TCP, enabling routers to mark packets to signal congestion without dropping them.^[5] It specifies the use of two bits in the IP header—the Type of Service (TOS) field in IPv4 or the Traffic Class field in IPv6—to encode ECN codepoints: Not-ECT (00, indicating non-ECN-capable transport), ECT(0) (10, ECN-capable transport), ECT(1) (01, an alternate ECN-capable codepoint for potential future use), and CE (11, congestion experienced).^[5] Routers detect incipient congestion using active queue management (AQM) algorithms such as Random Early Detection (RED) and probabilistically mark ECN-capable packets by setting the CE codepoint instead of dropping them, while dropping packets only when queues are full.^[5] At the transport layer, TCP endpoints negotiate ECN capability during the SYN/SYN-ACK exchange using the ECN-Echo (ECE) and Congestion Window Reduced (CWR) flags in the TCP header; receivers echo CE marks via the ECE flag in ACKs, prompting the sender to reduce its congestion window (typically by half) and acknowledge the response with CWR.^[5] ECN's integration with other transport protocols builds on this IP-layer foundation. For the Stream Control Transmission Protocol (SCTP), ECN is specified in RFC 4960 (2007), which adapts the IP ECN bits for SCTP's chunk-based structure.^[14] SCTP uses the IP header's ECN codepoints directly, with receivers reporting CE marks via ECNE (ECN Echo) chunks and senders responding with CWR chunks to indicate congestion window reduction; negotiation occurs during association setup using parameter types in the INIT and INIT-ACK chunks.^[14] Similarly, for the Datagram Congestion Control Protocol (DCCP), designed for unreliable, congestion-controlled flows, ECN support is defined in RFC 4340 (2006).^[15] DCCP packets carry ECN capability in feature negotiation during connection setup and use the IP ECN bits for marking, with receivers confirming CE via the ECE bit in acknowledgments and senders reducing their sending rate upon receipt; it emphasizes ECN for applications like streaming media where packet loss is undesirable.^[15] All core ECN RFCs emphasize interoperability and backward compatibility to support incremental deployment. Non-ECN devices treat ECN bits as don't-cares or low-priority bits, ignoring marks without adverse effects, while ECN-capable endpoints fall back to non-ECN operation if negotiation fails or middleboxes block ECN setup packets.^[5]^[14]^[15] For instance, if a SYN packet with ECN negotiation is dropped by a non-supporting middlebox, the sender retransmits without ECN flags to maintain connectivity.^[5] These rules ensure ECN traffic coexists seamlessly with legacy non-ECN flows across diverse networks.^[5]

Extensions and Recent Developments

ECN support for the Real-time Transport Protocol (RTP) over UDP is specified in RFC 6679 (2012), enabling congestion notification for delay-sensitive applications such as voice and video without packet drops.^[16] In 2018, RFC 8311 updated the original ECN specification in RFC 3168 by relaxing restrictions that previously prohibited widespread deployment of experimental ECN capabilities, thereby enabling broader testing and innovation in congestion signaling beyond basic loss avoidance.^[17] The Accurate ECN (AccECN) protocol, detailed in draft-ietf-tcpm-accurate-ecn (advancing toward standardization as of 2025), enhances TCP feedback by encoding multiple ECN signals per round-trip time within existing TCP header fields, allowing for more precise detection and response to congestion without requiring additional options.^[18] The Low Latency, Low Loss, Scalable Throughput (L4S) architecture, formalized in RFC 9330 in 2023, leverages ECN markings in a dual-queue approach at network devices to isolate classic congestion-responsive traffic from scalable control algorithms, achieving sub-millisecond queuing delays while maintaining high throughput. ECN++, specified in draft-ietf-tcpm-generalized-ecn (updated in 2025), experimentally extends ECN support to TCP control packets such as SYN and ACK, facilitating ECN negotiation during connection setup and enabling congestion signaling on retransmissions for improved initial window sizing.^[19] Ongoing IETF work includes draft-halmir-mpls-ecn (2025), which integrates ECN into MPLS networks via network actions labels to propagate congestion markings across label-switched paths without packet drops.^[20] Additionally, QUIC's core specification in RFC 9000 incorporates ECN feedback mechanisms, with extensions in RFC 9002 enabling endpoints to report ECN-CE counts for congestion control, supporting low-latency applications over UDP.^[21] These developments are driven by increasing demands in data centers, 5G networks, and AI workloads, where low-loss congestion signaling is essential for handling high-volume, latency-sensitive traffic without traditional packet drops.

How ECN Works

ECN in the IP Layer

Explicit Congestion Notification (ECN) operates at the IP layer by utilizing two bits in the IP header to indicate congestion without dropping packets. These bits, known as the ECN field, are located in the Differentiated Services (DS) field in IPv4 or the Traffic Class octet in IPv6: specifically, bits 6 and 7. The ECN field supports four codepoints: 00 for Not-ECT (Not ECN-Capable Transport), indicating packets ineligible for ECN marking; 01 for ECT(1) and 10 for ECT(0), both denoting ECN-Capable Transport where packets may be marked during congestion; and 11 for CE (Congestion Experienced), signaling that congestion has occurred along the path. Packets marked as ECT are eligible for routers to set the CE codepoint instead of discarding them, allowing end systems to react to congestion signals.^[22] Routers integrate ECN with Active Queue Management (AQM) mechanisms, such as Random Early Detection (RED) or Controlled Delay (CoDel), to detect and respond to incipient congestion based on queue occupancy. In AQM-enabled routers, the probability of applying an ECN mark increases as the average queue length grows, providing an early warning before queues overflow. For instance, in RED with ECN support (RED-ECN), the marking decision uses the average queue size to probabilistically select ECT packets for CE marking rather than dropping them, thereby preserving packet delivery while notifying endpoints of building congestion. Similarly, CoDel monitors queue delay and applies ECN marks when delays exceed a target threshold, aiming to control latency without relying solely on drops.^[22]^[23] The marking process occurs when a router determines that an ECT packet would otherwise be dropped due to congestion; instead, it sets the ECN field to CE and forwards the packet. This probabilistic marking helps avoid synchronized bursts from multiple flows. Non-ECT packets (codepoint 00) are treated traditionally: they may be dropped if the queue is full, but not marked. Once set to CE, the codepoint must not be altered by subsequent routers or tunnels, ensuring the congestion signal propagates to the destination.^[24] The ECN field placement is identical in IPv4 and IPv6, facilitating consistent handling across protocol versions. In IPv4, setting the CE codepoint requires recalculating the header checksum due to the change in the DS field. IPv6, lacking a header checksum, avoids this step. For tunneling protocols like IP-in-IP, ECN marks are preserved by copying the outer header's ECN field to the inner header upon decapsulation, or setting CE if either header indicates congestion, to prevent mark loss in encapsulated traffic.^[25]^[26] In RED-ECN, the marking probability P_{\text{mark}} for an average queue length q between the minimum threshold \min_{th} and maximum threshold \max_{th} is given by:

P_{\text{mark}} = \max_P \times \frac{q - \min_{th}}{\max_{th} - \min_{th}}

where \max_P is the maximum marking probability (typically 0.02). No marking occurs if q < \min_{th}, and all packets are eligible for marking (or dropping) if q > \max_{th}. This linear increase helps signal moderate congestion gradually.^[23]

ECN in the Transport Layer

In the transport layer, receivers inspect the Explicit Congestion Notification (ECN) field in the IP header of incoming packets to detect congestion experienced (CE) marks set by network nodes. Upon detecting one or more CE marks, the receiver echoes this information back to the sender using dedicated fields or flags in the transport protocol header, ensuring timely feedback without altering the underlying data flow. This mechanism allows endpoints to respond to congestion signals proactively, treating them as equivalent to packet drops for the purpose of rate adjustment.^[5] Senders integrate ECN feedback into their congestion control algorithms by reducing the congestion window (cwnd) upon receipt of echoed marks, typically halving it once per round-trip time in a process akin to fast recovery after a detected loss. This response decreases the amount of outstanding data, alleviating network pressure while maintaining throughput stability. The sender may also acknowledge the congestion reduction to the receiver via a specific flag, confirming that the signal has been acted upon and preventing persistent echoing.^[5] Similar high-level feedback mechanisms are employed in non-TCP protocols like Datagram Congestion Control Protocol (DCCP), where receivers use Ack Vectors to echo CE marks, allowing senders to invoke comparable congestion responses.^[27] ECN handling in the transport layer emphasizes distinguishing marks from actual packet losses, as a CE-marked packet is successfully delivered and requires no retransmission—only a congestion control adjustment. This separation ensures that ECN enhances efficiency for delay-sensitive or loss-intolerant applications by avoiding unnecessary recovery actions triggered by drops.^[5]

Negotiation and Compatibility

The negotiation of Explicit Congestion Notification (ECN) capability in TCP occurs during the three-way handshake to ensure both endpoints support the mechanism before enabling it. The initiating endpoint sends a SYN packet with both the ECN-Echo (ECE) and Congestion Window Reduced (CWR) flags set in the TCP header to signal its ECN capability. If the responding endpoint supports ECN, it replies with a SYN-ACK packet that sets the ECE flag while clearing the CWR flag; the initiator then sends an ACK to complete the handshake, confirming mutual support. If the responder does not support ECN, it clears both ECE and CWR flags in the SYN-ACK, allowing the connection to proceed without ECN.^[5] According to the rules in RFC 3168, endpoints must not set the ECN-Capable Transport (ECT) codepoint in the IP header of data packets unless ECN capability has been successfully negotiated in both directions—meaning the endpoint has sent an ECN-setup SYN or SYN-ACK and received a corresponding response. If negotiation fails or is not attempted, endpoints are required to clear any ECN bits (such as Congestion Experienced, or CE) in outgoing packets to prevent unintended signaling. Routers, in turn, ignore ECN markings on non-ECT packets and treat them as non-ECN-capable, potentially dropping them during congestion rather than marking them.^[5] Compatibility challenges arise primarily from middleboxes, such as firewalls and network address translators (NATs), which may strip ECN bits, drop ECN-setup SYN packets, or respond with resets (RSTs), often mistaking them for port-scanning attempts or invalid traffic. To mitigate this, RFC 8311 updates RFC 3168 by relaxing restrictions on ECN usage, permitting experimentation with ECN on TCP control packets (like SYNs) and retransmissions even before full negotiation, and advising middleboxes against discarding such packets unless responding to a verified attack; this encourages broader deployment without immediate blocking.^[5]^[17] If ECN negotiation fails—due to lack of support, middlebox interference, or no response—the connection automatically falls back to traditional packet drop-based congestion control, ensuring reliable operation without any inherent penalty for attempting ECN. For validating endpoint ECN support and detecting compatibility issues like middlebox stripping, tools such as the TCP Behavior Inference Tool (TBIT) can probe remote hosts over the Internet to infer ECN capability and path behavior.^[5]^[28]

Applications and Protocols

ECN with TCP

Explicit Congestion Notification (ECN) integrates with TCP primarily through modifications to the TCP header and specific behaviors during connection establishment and data transfer. Two bits in the 6-bit Reserved field of the TCP header are repurposed for ECN: bit 9 for the ECN-Echo (ECE) flag and bit 8 for the Congestion Window Reduced (CWR) flag. The receiver sets the ECE flag in acknowledgment (ACK) packets to report the receipt of one or more IP packets marked with the Congestion Experienced (CE) codepoint, informing the sender of network congestion without packet loss.^[24] In response, the sender sets the CWR flag in the subsequent data packet's TCP header to acknowledge that it has reacted to the congestion indication by reducing its transmission rate.^[24] ECN capability must be negotiated during the TCP three-way handshake to ensure both endpoints support the mechanism and to prevent misinterpretation of flags. The SYN packet from the client includes both ECE and CWR flags set to indicate ECN support. The server replies with a SYN-ACK packet setting only the ECE flag, confirming its capability while avoiding CWR to prevent confusion with data transmission responses. Neither endpoint sets the Explicit Congestion Notification (ECT) codepoint in the IP header of SYN or SYN-ACK packets under the original specification, which helps avoid complications with initial congestion window sizing and potential attacks.^[29] This negotiation ensures ECN is activated only for data packets after successful handshake completion. During data transfer, the receiver echoes congestion marks by setting ECE in every ACK following the receipt of a CE-marked packet, entering an ECN-Echo state until the sender acknowledges the reduction. Upon receiving the first ECE-marked ACK in a round-trip time (RTT), the sender reduces its congestion window (cwnd) by half—equivalent to the response for a detected packet loss—and enters a state where it ignores subsequent ECE marks until the next RTT to prevent over-reaction. The sender then sets CWR in the TCP header of the next full-sized data packet, prompting the receiver to cease setting ECE in further ACKs. This process is formalized as:

\text{new_cwnd} = \frac{\text{cwnd}}{2}

applied to the first ECN mark per RTT, with the minimum cwnd clamped at one maximum segment size (MSS) and retransmission timers reset if necessary.^[30] The original ECN specification prohibits setting ECT on non-data TCP control packets, such as pure ACKs, window probes, retransmissions, SYN, SYN-ACK, FIN, and RST, to mitigate denial-of-service risks and ensure control packet reliability. The ECN++ experimental extension addresses this limitation by permitting ECT marking on these packets, including during handshakes and graceful shutdowns, while preserving negotiation integrity. Specifically, it allows SYN and SYN-ACK packets to carry ECT after negotiation confirmation, and extends CE marking and feedback to FIN and RST for end-to-end congestion signaling in control flows.^[19] ECN behaviors remain compatible with modern TCP congestion control variants, which adapt the core window reduction mechanism to their algorithms. For instance, CUBIC employs the standard ECN response of halving cwnd upon ECE receipt to maintain its cubic scaling function for long-distance networks.^[7] BBRv2 similarly integrates ECN by treating CE marks as primary congestion signals, halving cwnd on ECE and using them to refine bandwidth and RTT estimates for better queue management.^[31] Across these variants, the per-RTT adjustment

\text{new_cwnd} = \frac{\text{cwnd}}{2}

on the first mark ensures consistent congestion avoidance.

ECN with Other Protocols

Explicit Congestion Notification (ECN) has been adapted for various transport protocols beyond TCP to enable congestion signaling in diverse network environments, particularly those requiring reliability, unreliability, or real-time performance. These adaptations leverage the IP-layer ECN marking while incorporating protocol-specific mechanisms for negotiation, feedback, and congestion response, ensuring compatibility with the underlying unreliable datagram service of IP.^[5] In the Stream Control Transmission Protocol (SCTP), ECN support is integrated to provide reliable, message-oriented transport with congestion awareness. SCTP uses a dedicated ECN-Echo (ECE) chunk for receivers to report congestion experienced (CE) marks from incoming packets, providing cumulative feedback on marked data chunks. Upon receiving an ECE chunk, the sender responds by reducing its slow-start threshold and congestion window, mirroring TCP's congestion avoidance but adapted to SCTP's multi-streaming model. This mechanism, specified in the SCTP standard, enhances throughput in multi-homed scenarios without relying on packet loss.^[32] The Datagram Congestion Control Protocol (DCCP) incorporates ECN to support unreliable, congestion-controlled delivery suitable for applications like streaming media and telephony. ECN capability is negotiated during connection setup using DCCP's feature negotiation framework, where endpoints exchange ECN Inc and ECN Confirm values to establish support and verify nonces for feedback integrity. Receivers report ECN marks via Confirm packets, which include byte counts of confirmed data and handle nonce echoes to prevent misreporting; this allows senders to adjust rates promptly for low-latency flows. DCCP's design makes it particularly apt for real-time media, where ECN reduces jitter compared to loss-based signaling.^[15] QUIC, a UDP-based multiplexed transport protocol, embeds ECN feedback directly into its acknowledgment (ACK) frames to support modern web applications, including HTTP/3. In QUIC version 1, endpoints negotiate ECN during connection setup via transport parameters, and receivers report cumulative counts of ECN-Capable Transport (ECT) and CE-marked packets in ACK frames, enabling precise congestion window adjustments without head-of-line blocking. Extensions like Accurate ECN (AccECN) further refine this by providing per-packet feedback signals, allowing more granular responses to congestion and integration with low-latency services in HTTP/3 deployments. This approach leverages QUIC's encryption and stream multiplexing for robust, privacy-preserving congestion control.^[21] For UDP-based protocols lacking built-in transport headers, such as those used in real-time applications, ECN is implemented at the application level, exemplified by the Real-time Transport Protocol (RTP) for VoIP. RTP over UDP sets the IP ECN field to ECT(0) for sent packets and uses out-of-band signaling, often via Session Description Protocol (SDP) attributes, to negotiate ECN capability between endpoints. Receivers feedback ECN marks through RTP Control Protocol (RTCP) reports, such as extended RTCP feedback messages tallying CE packets, prompting senders to reduce rates adaptively. This setup avoids modifying UDP but relies on application-layer coordination for congestion responsiveness in bandwidth-constrained scenarios like voice calls.^[16] A key challenge in applying ECN to UDP-based protocols stems from UDP's connectionless nature, which lacks a standardized handshake for capability negotiation. Unlike connection-oriented protocols, UDP requires custom out-of-band mechanisms—such as SDP for RTP or application-specific signaling—to agree on ECN use, increasing implementation complexity and potential for incompatibility across diverse endpoints. Additionally, ensuring end-to-end ECN propagation through middleboxes demands careful handling of feedback without native acknowledgments, often leading to reliance on periodic reports that may delay congestion response.^[16]^[8]

Data Center and Specialized Uses

In data centers, Explicit Congestion Notification (ECN) enables efficient congestion management in environments with shallow switch buffers and high-speed links. Data Center TCP (DCTCP), proposed by Microsoft Research in 2010, leverages ECN to address buffer pressure in such networks. Switches mark packets with the Congestion Experienced (CE) codepoint when queue occupancy exceeds a small threshold (e.g., 65 packets for 10 Gbps links), providing multi-bit feedback on congestion levels. The sender estimates the fraction of marked packets (α) using an exponential moving average: α ← (1 − g) × α + g × F, where g is a smoothing factor (typically 1/16) and F is the observed mark fraction in recent acknowledgments. Upon congestion, the congestion window (cwnd) is reduced proportionally: cwnd ← cwnd × (1 − α/2). This approach maintains low queue lengths (e.g., tens of packets) while achieving near line-rate throughput, reducing buffer usage by up to 90% compared to standard TCP.^[33]^[34] RDMA over Converged Ethernet (RoCE), particularly RoCEv2, integrates ECN to support lossless fabrics in data center storage and compute interconnects, reducing reliance on Priority Flow Control (PFC) which can cause head-of-line blocking. ECN marking signals congestion via the IP Differentiated Services (DS) field, allowing endpoints to adjust rates without packet drops. In RoCE networks, switches enable ECN on specific priorities (e.g., 0-7), and adapters like NVIDIA ConnectX series implement ECN reaction points (RP) and notification points (NP) to process marks and generate congestion notification packets (CNPs). This setup mitigates PFC storms in high-throughput scenarios, such as GPU clusters, by enabling proactive rate limiting based on CE marks. NVIDIA/Mellanox implementations, deployed in production data centers, configure ECN per port and priority via sysfs interfaces, ensuring compatibility with shallow-buffered Ethernet switches for RDMA traffic.^[35]^[36] Low Latency, Low Loss, Scalable throughput (L4S) extends ECN for data center applications by using a dual-queue architecture to isolate latency-sensitive "mice" flows (short, interactive) from bandwidth-intensive "elephant" flows (long-lived). The L4S queue applies frequent ECN marking (ECT(1) codepoint) with a Coupled AQM that balances marking rates between queues: the Classic queue (ECT(0)) uses drop-tail or mild ECN, while the L4S queue enforces sub-millisecond queuing delay (e.g., <1 ms at 99th percentile) via aggressive, scalable controls like DCTCP-style reductions. This separation prevents elephant flows from inflating latency for mice flows, enabling near-zero queuing delay in controlled data center environments. RFC 9332 specifies the coupling mechanism, where L4S marking probability (p_L) relates to Classic drop probability (p_C) as p_C ≈ (p_L / 2)^2 for fairness.^[4]^[37] Emerging 2025 trends highlight ECN's role in 5G edge computing for AI networks, where low-latency inference demands sub-10 ms end-to-end delays. In 5G RAN, ECN-like signaling (e.g., via L4Span) estimates per-user queuing delay and marks packets to throttle rates, supporting real-time AI tasks like distributed inference at the edge. Integration of 5G with Time-Sensitive Networking (TSN) in industrial IoT enables converged networks for low-latency control loops in factories using TSN's deterministic scheduling (IEEE 802.1Qbv).^[38] For inter-data center traffic, Google's B4 WAN employs ECN-enabled TCP variants to optimize global flows, achieving high utilization across continents. NVIDIA/Mellanox hardware further supports these uses in AI fabrics, with ECN tuned for RoCE in GPU-accelerated clusters.^[38]

Performance Considerations

Benefits and Improvements

Explicit Congestion Notification (ECN) enhances network efficiency by signaling congestion through packet marking rather than dropping, thereby avoiding the delays associated with retransmissions and enabling proactive congestion control. This mechanism reduces overall latency, particularly in scenarios involving bursty traffic or mild congestion, where traditional drop-based methods like tail-drop queuing lead to unnecessary packet loss and recovery times. Studies indicate that ECN can lower tail latency in web services compared to non-ECN TCP, as it mitigates the impact of head-of-line blocking and retransmission timeouts (RTOs).^[8] In data center environments, ECN supports protocols like Data Center TCP (DCTCP), which uses ECN markings to estimate fractional buffer occupancy and adjust sending rates accordingly. DCTCP achieves near-line-rate throughput (e.g., 95% of link capacity at 1 Gbps) on buffered links during congestion, while reducing average and tail latencies for short flows to under 1 ms, compared to 19 ms with standard TCP. This results in up to a 90% reduction in required buffer space without performance degradation, allowing networks to operate with smaller queues and higher utilization.^[33] The Low Latency, Low Loss, Scalable throughput (L4S) architecture further improves ECN by enabling scalably responsive congestion controls that maintain standing queue delays below 1 ms—even under heavy load—versus 10-100 ms in classic ECN or drop-tail systems. Simulations and evaluations show that L4S supports 2x buffer size reduction without increased loss, promoting better link utilization through rapid rate adaptations to early congestion signals.^[39] In real-world applications, ECN improves VoIP quality by reducing jitter and bursty losses that degrade audio, as markings allow endpoints to throttle rates before drops occur, preserving low-delay paths. Similarly, for video streaming, ECN minimizes artifacts from packet loss by sustaining higher throughput during congestion, leading to smoother playback without frequent rebuffering.^[8]

Limitations and Challenges

One significant limitation of Explicit Congestion Notification (ECN) is interference from middleboxes, such as firewalls and NAT devices, which often drop or alter packets marked with ECN bits, mistaking them for malformed traffic. This issue persists in the Internet core, where studies indicate that ECN feedback loops fail approximately 40% of the time, particularly at autonomous system boundaries, leading to unreliable congestion signaling and fallback to packet loss-based mechanisms.^[40] Recent measurements confirm that while ECN support has grown threefold on servers since earlier assessments, middlebox-induced traversal problems still affect a notable fraction of paths, complicating end-to-end deployment.^[41] Asymmetric routing poses another challenge for ECN, as the mechanism relies on bidirectional feedback where acknowledgments carry congestion marks from the reverse path; if the return path includes devices that block or modify these signals, the sender receives lost or incomplete feedback, potentially causing underutilization of bandwidth. This requires precise tuning of Active Queue Management (AQM) algorithms, such as adjusting marking thresholds dynamically, to compensate for path asymmetries and maintain effective congestion control without excessive latency spikes.^[42] In environments with frequent route changes, like mobile or multi-homed networks, such tuning becomes even more critical to avoid feedback loops breaking entirely. ECN introduces additional overhead through its negotiation process during connection setup and the extra processing for marking and feedback interpretation, which is minimal in standard scenarios but can become noticeable in ultra-low-latency applications like real-time gaming or financial trading where sub-millisecond delays are essential. This overhead arises from the need to inspect and set ECN bits in IP headers and TCP options, potentially conflicting with optimized hardware acceleration paths.^[43] Security risks in ECN stem from its susceptibility to adversarial manipulation, where malicious middleboxes or on-path attackers can inject false congestion marks to throttle legitimate traffic, mimicking denial-of-service effects without overt packet drops. Mitigations are addressed through extensions like Accurate ECN (AccECN), which enhances feedback accuracy by providing multiple signals per round-trip time and includes validation checks to detect and ignore anomalous markings, thereby preserving signal integrity in untrusted networks.^[18] Despite progress, adoption gaps remain a key barrier, with low ECN support in ISP core routers limiting widespread use; as of 2025, only select providers like Comcast have begun rolling out Low Latency, Low Loss, Scalable throughput (L4S) capabilities in limited areas, with ongoing trials by others such as Vodafone demonstrating up to 94% delay reductions in live networks.^[44]^[45] This fragmented deployment hinders L4S's potential for interactive applications, as uncoordinated upgrades risk interworking issues with classic ECN paths.^[46]^[47]

Deployment and Implementations

End-Host Support

End-host support for Explicit Congestion Notification (ECN) has been integrated into major operating systems, enabling endpoints to negotiate and utilize ECN capabilities during connection establishment. In Linux, kernel support for ECN was introduced in version 2.4 in 2001, allowing TCP to respond to congestion signals without packet loss.^[48] The configuration is managed via the sysctl parameter net.ipv4.tcp_ecn, where a value of 2 (the kernel default since early implementations) enables ECN negotiation for incoming connections that request it, while outgoing connections do not initiate negotiation unless explicitly configured otherwise.^[49]^[50] This setting persists across kernel series, including the 5.x releases from the late 2010s onward, promoting broader adoption in server and client environments.^[51] Windows provides ECN support for TCP starting with Windows Vista and Windows Server 2008, with the feature enabled by default in Windows Server editions since 2012 but disabled by default in client versions like Windows 10 and 11 to mitigate compatibility issues with legacy networks.^[52] Users can enable it globally using the command netsh int tcp set global ecn=enabled, and its status can be verified via PowerShell with Get-NetTCPSetting, which reports ECN capability across TCP profiles.^[53] On Apple platforms, ECN support has been available in the kernel since macOS 10.5 (2007), though it remained disabled by default until macOS 10.11 El Capitan and iOS 9 in 2015, when Apple enabled it for all supported devices to improve performance over Wi-Fi and cellular networks, including during handoffs.^[54] Subsequent versions, such as macOS Sierra and iOS 10 (2016), extended this to 100% enablement on Wi-Fi and select cellular providers, prioritizing low-latency scenarios.^[55] Android, built on the Linux kernel, inherits ECN support from kernel versions integrated since its early releases, with full usability available since Android 4.0 (2011) through kernel configurations, though user-facing enablement requires root access or custom builds.^[56] BSD variants like FreeBSD and NetBSD include ECN in their TCP stacks by default, with FreeBSD's net.inet.tcp.ecn.enable set to 2, permitting incoming ECN requests while outgoing connections do not initiate negotiation unless explicitly configured.^[57]^[58] Web applications and browsers further enhance end-host ECN usage, particularly with QUIC; for instance, Google Chrome and Mozilla Firefox negotiate ECN during QUIC handshakes as per the protocol's specification, enabling congestion feedback in HTTP/3 sessions without relying solely on TCP.^[59] As of 2025, ECN support is near-universal among client operating systems, though actual deployment remains limited due to middlebox interference, such as legacy firewalls that drop ECN-marked packets, with APNIC measurements indicating low TCP ECN option rates of around 2-3% in client-initiated connections from February to August 2025, though server-side adoption remains opt-in to avoid interactions with legacy infrastructure.^[60]^[46]

Network Device Support

Explicit Congestion Notification (ECN) support in network devices, particularly routers and switches, enables intermediate hardware to mark packets indicating congestion rather than dropping them, integrating with mechanisms like Weighted Random Early Detection (WRED) for proactive traffic management. In routers, Cisco IOS and IOS XE platforms have supported ECN since version 12.2(8)T in 2001, with integration into WRED for marking packets based on queue thresholds to signal impending congestion without loss.^[61] Juniper Junos operating system provides Class of Service (CoS) ECN capabilities, allowing per-queue marking for IP traffic since version 8.0, enhancing end-to-end notification in TCP/IP networks.^[62] Open-source routing platforms like VyOS, built on Linux, include default Active Queue Management (AQM) features that support ECN marking through kernel-level configurations for congestion avoidance.^[63] Data center switches from NVIDIA (formerly Mellanox) have incorporated ECN since around 2012, particularly for RoCEv2 deployments, where it enables end-to-end congestion control in high-performance fabrics by marking packets in hardware queues.^[64] Arista and Cumulus Linux-based switches support ECN in leaf-spine architectures, optimizing RoCE traffic in AI and storage networks through buffer threshold-based marking to prevent loss in oversubscribed environments.^[65] Application-Specific Integrated Circuits (ASICs) like Broadcom's Jericho series perform ECN marking directly in hardware, as seen in Jericho-based AI networking solutions that use ECN signaling for proactive congestion control across distributed data centers.^[66] In Network Function Virtualization (NFV) environments, cloud providers such as Google Cloud Platform (GCP) enable ECN in virtual routers and Kubernetes Engine (GKE) nodes, where TCP stacks can be configured to use ECN for queue management in containerized workloads.^[67] As of 2025, ECN adoption in enterprise routers continues to grow, driven by needs for low-latency AI and cloud applications, though specific penetration rates vary by vendor.^[46] Internet Service Provider (ISP) deployment is increasing through Low Latency, Low Loss, Scalable throughput (L4S) pilots, which rely on ECN for real-time congestion signaling in production networks.^[68] Monitoring tools like sFlow provide ECN statistics by sampling traffic and queue metrics on supported devices, aiding in visibility of marking rates and congestion events.^[69] ECN configuration in network devices typically involves threshold-based marking, where a minimum threshold (min-th) triggers initial Congestion Experienced (CE) marks, ramping up probability until a maximum threshold (max-th). For example, setting min-th to 10 packets and max-th to 30 packets allows gradual CE marking as queue depth increases, configurable via scheduler profiles in systems like Junos.^[70]

References

[1]
RFC 3168: The Addition of Explicit Congestion Notification (ECN) to IP
Summary of each segment:
[2]
RFC 6679: Explicit Congestion Notification (ECN) for RTP over UDP
This memo specifies how Explicit Congestion Notification (ECN) can be used with the Real-time Transport Protocol (RTP) running over UDP.
[3]
RFC 3168 - The Addition of Explicit Congestion Notification (ECN) to ...
This memo specifies the incorporation of ECN (Explicit Congestion Notification) to TCP and IP, including ECN's use of two bits in the IP header.
[4]
RFC 5681 - TCP Congestion Control - IETF Datatracker
This document defines TCP's four intertwined congestion control algorithms: slow start, congestion avoidance, fast retransmit, and fast recovery.
[5]
RFC 8312 - CUBIC for Fast Long-Distance Networks
CUBIC is an extension to the current TCP standards. It differs from the current TCP standards only in the congestion control algorithm on the sender side.
[6]
RFC 8087 - The Benefits of Using Explicit Congestion Notification ...
The goal of this document is to describe the potential benefits of applications using a transport that enables Explicit Congestion Notification (ECN).
[7]
[PDF] Random early detection gateways for congestion avoidance
Sally Floyd and Van Jacobson. Abstract-This paper presents Random Early Detection (RED) gateways for congestion avoidance in packet-switched networks. The ...
[8]
[PDF] TCP and Explicit Congestion Notification
This paper discusses the use of Explicit Congestion Noti- fication (ECN) mechanisms in the TCP/IP protocol. The first part proposes new guidelines for TCP's ...
[9]
RFC 2481: A Proposal to add Explicit Congestion Notification (ECN ...
This discussion of ECN and IPsec tunnel considerations draws heavily on related discussions and documents from the Differentiated Services Working Group.
[10]
Explicit Congestion Notification (ecn) - IETF Datatracker
Concluded WG Explicit Congestion Notification (ecn). About · Documents · Meetings · History · Photos · Email expansions. Group history. Date, By, Action. 2000- ...
[11]
[PDF] Measuring ECN++ Good News for ++, Bad News for ECN over Mobile
A new proposal called ECN++ [4] proposes safe ways to remove all the original prohibitions on using ECN on each type of TCP packet. As with any new protocol, ...
[12]
RFC 4960 - Stream Control Transmission Protocol - IETF Datatracker
RFC 4960 Stream Control Transmission Protocol September 2007 12 - Reserved for Explicit Congestion Notification Echo (ECNE) 13 - Reserved for Congestion ...
[13]
RFC 4340 - Datagram Congestion Control Protocol (DCCP)
RFC 4340 Datagram Congestion Control Protocol (DCCP) March 2006 12. Explicit Congestion Notification The DCCP protocol is fully ECN-aware [RFC3168]. Each ...
[14]
RFC 8311 - Relaxing Restrictions on Explicit Congestion Notification ...
This memo updates RFC 3168, which specifies Explicit Congestion Notification (ECN) as an alternative to packet drops for indicating network congestion to ...
[15]
More Accurate Explicit Congestion Notification (AccECN) Feedback ...
This document updates the original ECN specification in RFC 3168 to specify a scheme that provides more than one feedback signal per RTT in the TCP header.
[16]
draft-ietf-tcpm-generalized-ecn-17 - ECN++: Adding Explicit ...
Apr 21, 2025 · Dec 2025: Submit document on adding Explicit Congestion Notification (ECN) to TCP Control Packets to the IESG for publication as Experimental ...Missing: 9264 | Show results with:9264
[17]
draft-halmir-mpls-ecn-00 - Explicit Congestion Notification Using ...
Jun 26, 2025 · MPLS Working Group J. Halpern Internet-Draft G. Mirsky Intended status: Standards Track Ericsson Expires: 28 December 2025 26 June 2025 ...Missing: integration | Show results with:integration
[18]
RFC 9000 - QUIC: A UDP-Based Multiplexed and Secure Transport
QUIC is a secure general-purpose transport protocol. This document defines version 1 of QUIC, which conforms to the version-independent properties of QUIC.Table of Contents · Streams · Cryptographic and Transport... · Connection Migration
[19]
https://datatracker.ietf.org/doc/draft-ietf-tcpm-generalized-ecn/
[20]
https://datatracker.ietf.org/doc/draft-halmir-mpls-ecn/
[21]
https://datatracker.ietf.org/doc/html/rfc9000
[22]
https://datatracker.ietf.org/doc/html/rfc3168#section-5
[23]
https://datatracker.ietf.org/doc/html/rfc2309#section-4
[24]
RFC 6824: TCP Extensions for Multipath Operation with Multiple Addresses
Summary of each segment:
[25]
RFC 9260 - Stream Control Transmission Protocol - IETF Datatracker
RFC 9260. Stream Control Transmission Protocol. Abstract. This document describes the Stream Control Transmission Protocol (SCTP) and obsoletes RFC 4960.
[26]
TBIT, the TCP Behavior Inference Tool: ECN
TBIT, the TCP Behavior Inference Tool: ECN. ECN: The problem is that some Internet hosts are not reachable from an ECN-Capable TCP client.
[27]
https://datatracker.ietf.org/doc/html/rfc4341
[28]
https://www.icir.org/tbit/ecn-tbit.html
[29]
[PDF] BBR v2 A Model-based Congestion Control - IETF Datatracker
BBR v2 improvements: using ECN as a signal. - BBR v1: does not use ECN. - BBR v2: uses DCTCP-style ECN, if available, to help keep queues short. 6. 20 BBR v2 ...
[30]
RFC 6679 - Explicit Congestion Notification (ECN) for RTP over UDP
This memo specifies how Explicit Congestion Notification (ECN) can be used with the Real-time Transport Protocol (RTP) running over UDP.
[31]
[PDF] Data Center TCP (DCTCP) - People | MIT CSAIL
Aug 30, 2010 · In this paper, we proposed a new variant of TCP, called Data. Center TCP (DCTCP). Our work was motivated by detailed traf- fic measurements ...
[32]
RFC 8257 - Data Center TCP (DCTCP) - IETF Datatracker
Sep 29, 2021 · This Informational RFC describes Data Center TCP (DCTCP): a TCP congestion control scheme for data-center traffic.
[33]
Explicit Congestion Notification (ECN) - NVIDIA Docs
Oct 23, 2023 · ECN is an extension to the IP protocol. It allows reliable communication by notifying all ends of communication when congestion occurs.
[34]
RoCE Storage Implementation over NX-OS VXLAN Fabrics - Cisco
In summary, ECN and PFC are used in RoCEv2 traffic to provide congestion control and to ensure that critical traffic is not delayed or lost due to congestion.
[35]
The Explicit Congestion Notification (ECN) Protocol for Low Latency ...
This specification defines the protocol to be used for a new network service called Low Latency, Low Loss, and Scalable throughput (L4S).
[36]
RFC 9332 - Dual-Queue Coupled Active Queue Management (AQM ...
Dec 12, 2023 · This specification defines a framework for coupling the Active Queue Management (AQM) algorithms in two queues intended for flows with different responses to ...
[37]
5G TSN - integrating for industrial automation - Ericsson
Aug 27, 2019 · This article explains how our approach to 5G TSN integration helps factories meet real-time requirements for combined deployment and ...Missing: ECN | Show results with:ECN
[38]
RFC 9330: Low Latency, Low Loss, and Scalable Throughput (L4S) Internet Service: Architecture
Summary of each segment:
[39]
Enabling Internet-Wide Deployment of Explicit Congestion Notification
Aug 7, 2025 · Even though ECN support is increasing among web-servers, there are still many limitations due to the existence of ECN-unfriendly middleboxes. ...
[40]
[PDF] A Fresh Look at ECN Traversal in the Wild - Semantic Scholar
Aug 30, 2022 · The Explicit Congestion Notification (ECN) field has taken on new importance due to Low Latency, Low Loss, and Scalable throughput (L4S) ...
[41]
[PDF] ACC: Automatic ECN Tuning for High-Speed Datacenter Networks
To meet the operational challenge, in this paper we report the design and implementation of an automatic run-time optimization scheme, ACC, which leverages the ...
[42]
Achieving low latency and high throughput with ECN and AQM
The design does not require per-packet extra processing, so it incurs very small overhead and is simple to implement in both hardware and software. In ...
[43]
Measuring Explicit Congestion Notification - APNIC Blog
Sep 11, 2025 · This work was taken up by RFC 2481, describing an experimental protocol that proposed to add Explicit Congestion Notification (ECN) to IP. The ...<|separator|>
[44]
The Internet Resilience Report 2025 - Catchpoint
Jun 24, 2025 · Adopt a phased approach to resilience improvements 44%. Use an ... ECN and L4S need to be supported by the client and server but also ...
[45]
Re: sysctl should disable ECN by default - Debian Mailing Lists
Sep 1, 2001 · This option adds ECN support to the Linux kernel, as well as a sysctl (/proc/sys/net/ipv4/tcp_ecn) which allows ECN support to be disabled at runtime.Missing: history | Show results with:history
[46]
Network configuration/Ethernet - ArchWiki
Aug 31, 2025 · # sysctl net.ipv4.tcp_ecn=1. To enable ECN only when requested by incoming connections (the reasonably safe, kernel default): # sysctl net ...
[47]
IP sysctl - The Linux Kernel Archives
Increase this when using large numbers of interfaces and/or routes. From linux kernel 3.6 onwards, this is deprecated for ipv4 as route cache is no longer used.Missing: history | Show results with:history
[48]
Windows 10,11 TCP/IP Tweaks - SpeedGuide
ECN is disabled by default in modern Windows TCP/IP implementations, as it is possible that it may cause problems with some outdated routers that drop packets ...
[49]
How to Resolve ECN Negotiation Failure Between Windows( as ...
Mar 17, 2025 · The Windows operating system sets the ECN in the IP header to '10' in the first packet and sets the ECE and CWR in the TCP SYN packet to 1.<|control11|><|separator|>
[50]
El Capitan: does it deal with network congestion at last?
Dec 31, 2015 · OS X has had support for ECN built into its kernel since version 10.5, but it has remained disabled by default. Back in June 2015, Apple boldly ...
[51]
iOS 10 and macOS Sierra: Networking for the modern Internet
Jun 24, 2016 · In the iOS and macOS betas, ECN is enabled 100 percent of the time over Wi-Fi and several cellular providers' networks. International text (in ...
[52]
[PDF] The Congestion Crisis and the Internet's Preparedness for ECN
– ECN support added by Google Summer of Code project in 2006. • Mobile operating systems. – Linux kernel of Android has ECN support but no easy way for users ...
[53]
TransportProtocols/tcp_rfc_notes - FreeBSD Wiki
Oct 19, 2024 · The concept was introduced along with a discussion of the queue management algorithm "RED" (Random Early Detect/Drop) by RFC 2309. ... RED/ECN CE ...Missing: equation | Show results with:equation
[54]
https://eclecticlight.co/2015/12/31/el-capitan-does-it-deal-with-network-congestion-at-last/
[55]
ECN in QUIC · quicwg/base-drafts Wiki - GitHub
Jun 8, 2025 · The capability check uses the ECN Feedback function to verify that one direction of the path between the endpoints is free from issues with ECN ...Missing: validation | Show results with:validation
[56]
Measuring ECN | blabs - APNIC Labs
Sep 8, 2025 · ... ECN field is used as an identifier for L4S packets to be distinguished from classic packets. The queue for L4S traffic has a shallower (or ...
[57]
CoS Explicit Congestion Notification (ECN) | Junos OS
Explicit congestion notification (ECN) enables end-to-end congestion notification between two endpoints on IP based networks.
[58]
QoS/QoE support in VyOS? - Development
Oct 10, 2023 · Short answer is anywhere there is a fast->slow transition on any network, fq+aqm technologies are needed to manage it.
[59]
HowTo Configure ECN on Mellanox Ethernet Switches (Spectrum)
ECN can be configured, with or without RED, using the following commands for specific traffic (eg traffic-class 0) per interface (eg 1/1).
[60]
[PDF] AI Network Fabric Deployment Guide - Arista
Jan 13, 2025 · Sample interface configuration for the GPU POD leafs and spines referenced in the topology. ECN Counter Platform Support. Global config is ...
[61]
HyperPorts Supercharge the Multi-Building AI Training Clusters
Sep 16, 2025 · Broadcom's DNX Jericho-based AI networking solution, introduced in ... (ECN) signaling to enable network-level proactive congestion control between ...
[62]
Customizing node system configuration | Google Kubernetes Engine ...
This setting controls the use of Explicit Congestion Notification (ECN) by TCP. ECN is used only when both ends of the TCP connection indicate support for it.
[63]
L4S: Real-Time Congestion Management at the Speed of Software
Sep 30, 2025 · Learn how L4S-aware applications improve performance by adapting to network conditions; Understand how L4S reduces delay and enables fairer ...
[64]
NetFlow & sFlow Config for Cisco 8000 Series Routers
Jun 13, 2025 · This page helps you with information about the key concepts, advantages and limitations of sFlow, and steps to configure sFlow on your router.
[65]
Example: Configuring Static and Dynamic ECN | Junos OS
Static ECN requires you to manually set the threshold levels that trigger a notification event. Dynamic ECN automatically adjusts the thresholds based on real- ...Missing: sFlow statistics, th