View source for Explicit Congestion Notification

{{short description|Extension to the Internet Protocol to signal network congestion}}
{{IPstack}}

'''Explicit Congestion Notification''' ('''ECN''') is an extension to the [[Internet Protocol]] and to the [[Transmission Control Protocol]] and is defined in {{IETF RFC|3168}} (2001). ECN allows end-to-end notification of [[network congestion]] without dropping packets. ECN is an optional feature that may be used between two ECN-enabled endpoints when the underlying network infrastructure also supports it.

Conventionally, TCP/IP networks signal congestion by dropping packets.  When ECN is successfully negotiated, an ECN-aware router may set a mark in the IP header instead of dropping a packet in order to signal impending congestion.  The receiver of the packet echoes the congestion indication to the sender, which reduces its transmission rate as if it detected a dropped packet.

Rather than responding properly or ignoring the bits, some outdated or faulty network equipment has historically dropped or mangled packets that have ECN bits set.<ref>{{cite web |url=http://conferences.sigcomm.org/imc/2011/docs/p171.pdf |title=Measuring the State of ECN Readiness in Servers, Clients, and Routers |author1=Steven Bauer |author2=Robert Beverly |author3=Arthur Berger |publisher=Internet Measurement Conference 2011 |date=2011 |url-status=live |archive-url=https://web.archive.org/web/20140322043254/http://conferences.sigcomm.org/imc/2011/docs/p171.pdf |archive-date=2014-03-22 }}</ref><ref>{{cite web|url=http://conferences.sigcomm.org/imc/2004/papers/p336-medina.pdf|title=Measuring Interactions Between Transport Protocols and Middleboxes|author1=Alberto Medina|author2=Mark Allman|author3=Sally Floyd|publisher=Internet Measurement Conference 2004|url-status=live|archive-url=https://web.archive.org/web/20160304084141/http://conferences.sigcomm.org/imc/2004/papers/p336-medina.pdf |archive-date=2016-03-04}}</ref><ref>{{cite web |url=http://www.icir.org/tbit/ecn-tbit.html |title=TBIT, the TCP Behavior Inference Tool: ECN |publisher=Icir.org |access-date=2014-03-22 |url-status=live |archive-url=https://web.archive.org/web/20130311202346/http://www.icir.org/tbit/ecn-tbit.html |archive-date=2013-03-11 }}</ref> {{As of|2015}}, measurements suggested that the fraction of [[web server]]s on the public Internet for which setting ECN prevents network connections had been reduced to less than 1%.<ref>{{cite web |url=http://ecn.ethz.ch/ecn-pam15.pdf |title=Enabling Internet-Wide Deployment of Explicit Congestion Notification |author1=Brian Trammell |author2=Mirja Kühlewind |author3=Damiano Boppart |author4=Iain Learmonth |author5=Gorry Fairhurst |author6=Richard Scheffenegger |publisher=Proceedings of the Passive and Active Measurement Conference 2015 |date=2015 |access-date=14 June 2015 |url-status=dead |archive-url=https://web.archive.org/web/20150615083912/http://ecn.ethz.ch/ecn-pam15.pdf |archive-date=15 June 2015 }}</ref>

Passive support has existed in Ubuntu Linux since 12.04 and in Windows Server since 2012.<ref name="2017stats">{{cite web |url=http://profiles.murdoch.edu.au/myprofile/david-murray/files/2012/06/An_Analysis_of_Changing_Enterprise_Network_Traffic_Characteristics-22.pdf |title=An Analysis of Changing Enterprise Network Traffic Characteristics |author1=David Murray |author2=Terry Koziniec |author3=Sebastian Zander |author4=Michael Dixon |author5=Polychronis Koutsakis |publisher=The 23rd Asia-Pacific Conference on Communications (APCC 2017) |date=2017 |access-date=3 October 2017 |url-status=live |archive-url=https://web.archive.org/web/20171003124654/http://profiles.murdoch.edu.au/myprofile/david-murray/files/2012/06/An_Analysis_of_Changing_Enterprise_Network_Traffic_Characteristics-22.pdf |archive-date=3 October 2017 }}</ref> Passive support in the most popular websites has increased from 8.5% in 2012 to over 70% in May 2017.<ref name="2017stats" /> Adoption across the Internet now requires clients to actively request ECN. In June 2015, [[Apple Inc.|Apple]] announced that ECN will be enabled by default on its supported and future products, to help drive the adoption of ECN signaling industry-wide.<ref name="apple-wwdc-719">{{cite web|url=https://developer.apple.com/videos/wwdc/2015/?id=719|title=Your App and Next Generation Networks|publisher=Apple Inc.|date=2015|url-status=live|archive-url=https://web.archive.org/web/20150615162041/https://developer.apple.com/videos/wwdc/2015/?id=719|archive-date=2015-06-15}}</ref>

==Operation==
ECN requires specific support at both the Internet layer and the [[transport layer]] for the following reasons:
* In TCP/IP, routers operate within the Internet layer, while the transmission rate is handled by the endpoints at the transport layer.
* Congestion may be handled only by the transmitter, but since it is known to have happened only after a packet was sent, there must be an echo of the congestion indication by the receiver to the transmitter.

Without ECN, congestion indication echo is achieved indirectly by the detection of lost packets. With ECN, the congestion is indicated by setting the ECN field within an IP packet to CE (Congestion Experienced) and is echoed back by the receiver to the transmitter by setting proper bits in the header of the transport protocol. For example, when using TCP, the congestion indication is echoed back by setting the ECE bit.

===Operation of ECN with IP===
ECN uses the two [[Least significant bit|least significant]] (right-most) bits of the [[IPv6 packet#Fixed header|Traffic Class]] field in the [[IPv4 header|IPv4]] or [[IPv6 header]] to encode four different code points:
* <code>00</code> &ndash; Not ECN-Capable Transport, Not-ECT
* <code>01</code> &ndash; ECN Capable Transport(1), ECT(1)
* <code>10</code> &ndash; ECN Capable Transport(0), ECT(0)
* <code>11</code> &ndash; Congestion Experienced, CE.

When both endpoints support ECN they mark their packets with ECT(0) or ECT(1). Routers treat the ECT(0) and ECT(1) codepoints as equivalent. If the packet traverses an [[active queue management]] (AQM) queue (e.g., a queue that uses [[random early detection]] (RED)) that is experiencing congestion and the corresponding [[router (computing)|router]] supports ECN, it may change the code point to <code>CE</code> instead of [[packet loss|dropping the packet]]. This act is referred to as "marking" and its purpose is to inform the receiving endpoint of impending [[network congestion|congestion]]. At the receiving endpoint, this congestion indication is handled by the upper layer protocol ([[transport layer]] protocol) and needs to be echoed back to the transmitting node in order to signal it to reduce its transmission rate.

Because the CE indication can only be handled effectively by an upper layer protocol that supports it, ECN is only used in conjunction with upper layer protocols, such as [[Transmission Control Protocol|TCP]], that support congestion control and have a method for echoing the CE indication to the transmitting endpoint.

===Operation of ECN with TCP===
TCP supports ECN using two flags in the TCP header.  The first, ''ECN-Echo'' (ECE) is used to echo back the congestion indication (i.e., signal the sender to reduce the transmission rate).  The second, ''Congestion Window Reduced'' (CWR), to acknowledge that the congestion-indication echoing was received.  Use of ECN on a TCP connection is optional; for ECN to be used, it must be negotiated at connection establishment by including suitable options in the SYN and SYN-ACK segments.

When ECN has been negotiated on a TCP connection, the sender indicates that IP packets that carry TCP segments of that connection are carrying traffic from an ECN Capable Transport by marking them with an ECT code point. This allows intermediate routers that support ECN to mark those IP packets with the CE code point instead of dropping them in order to signal impending congestion.

Upon receiving an IP packet with the ''Congestion Experienced'' code point, the TCP receiver echoes back this congestion indication using the ECE flag in the TCP header.  When an endpoint receives a TCP segment with the ECE bit it reduces its congestion window as for a packet drop.  It then acknowledges the congestion indication by sending a segment with the CWR bit set.

A node keeps transmitting TCP segments with the ECE bit set until it receives a segment with the CWR bit set.

To see affected packets with [[tcpdump]], use the filter predicate <code>(tcp[13] & 0xc0 != 0)</code>.

====ECN and TCP control packets====
Since the [[Transmission Control Protocol]] (TCP) does not perform congestion control on control packets (pure ACKs, SYN, FIN segments), control packets are usually not marked as ECN-capable.

A 2009 proposal{{Ref RFC|5562}} suggests marking SYN-ACK packets as ECN-capable.  This improvement, known as ECN+, has been shown to provide dramatic improvements to performance of short-lived TCP connections.<ref name="kuzmanovic">Aleksandar Kuzmanovic.  The power of explicit congestion notification. In ''Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications''. 2005.</ref>

===Operation of ECN with other transport protocols===
ECN is also defined for other transport layer protocols that perform congestion control, notably [[Datagram Congestion Control Protocol|DCCP]] and [[Stream Control Transmission Protocol]] (SCTP).  The general principle is similar to TCP, although the details of the on-the-wire encoding differ.

It is possible to use ECN with protocols layered above [[User Datagram Protocol|UDP]]. However, UDP requires that congestion control be performed by the application, and early UDP based protocols such as [[Domain Name Services|DNS]] did not use ECN. More recent UDP based protocols such as [[QUIC]] are using ECN for congestion control.

==Effects on performance==
Since ECN is only effective in combination with an [[Active Queue Management]] (AQM) policy, the benefits of ECN depend on the precise AQM being used.  A few observations, however, appear to hold across different AQMs.

As expected, ECN reduces the number of packets dropped by a TCP connection, which, by avoiding a retransmission, reduces latency and especially jitter.  This effect is most drastic when the TCP connection has a single outstanding segment,{{Ref RFC|2884}} when it is able to avoid an [[Transmission Control Protocol#Congestion control|RTO]] timeout; this is often the case for interactive connections, such as remote logins, and transactional protocols, such as HTTP requests, the conversational phase of SMTP, or SQL requests.

Effects of ECN on bulk throughput are less clear<ref>Marek Małowidzki, Simulation-based Study of ECN Performance in RED Networks, In ''Proc. SPECTS'03''. 2003.</ref> because modern TCP implementations are fairly good at resending dropped segments in a timely manner when the sender's [[Transmission Control Protocol#Flow control|window]] is large.

Use of ECN has been found to be detrimental to performance on highly congested networks when using AQM algorithms that never drop packets.<ref name="kuzmanovic" />  Modern AQM implementations avoid this pitfall by dropping rather than marking packets at very high load.

==Implementations==
Many modern implementations of the TCP/IP protocol suite have some support for ECN; however, they usually ship with ECN disabled.

===ECN support in TCP by hosts===

====Microsoft Windows====
Windows versions since Windows Server 2008 and Windows Vista support ECN for TCP.<ref name=win>{{cite web | url = https://technet.microsoft.com/en-us/library/bb726965.aspx | title = New Networking Features in Windows Server 2008 and Windows Vista | url-status = live | archive-url = https://web.archive.org/web/20100115195304/http://technet.microsoft.com/en-us/library/bb726965.aspx | archive-date = 2010-01-15 }}</ref> Since Windows Server 2012, it is enabled by default in Windows Server versions, because [[#DCTCP|Data Center Transmission Control Protocol]] (DCTCP) is used.<ref>{{cite web | url = https://technet.microsoft.com/en-us/library/hh997028.aspx | title = Data Center Transmission Control Protocol (DCTCP) (Windows Server 2012) | url-status = live | archive-url = https://web.archive.org/web/20170826132305/https://technet.microsoft.com/en-us/library/hh997028.aspx | archive-date = 2017-08-26 }}</ref> In previous Windows versions and non-server versions it is disabled by default.

ECN support can be enabled using a shell command such as {{code|netsh interface tcp set global ecncapability{{=}}enabled}}.

====BSD====
On [[FreeBSD]], ECN for TCP can be configured using the {{Mono|net.inet.tcp.ecn.enable}} [[sysctl]]. By default, it is enabled only for incoming connections that request it. It can also be enabled for all connections or disabled entirely.<ref>{{cite web |title=tcp(4) - Internet Transmission Control Protocol |url=https://www.freebsd.org/cgi/man.cgi?query=tcp |website=FreeBSD Kernel Interfaces Manual |access-date=3 April 2020}}</ref>

[[NetBSD]]&nbsp;4.0 implements ECN support for TCP; it can be activated through the [[sysctl]] interface by setting 1 as value for the {{code|sysctl net.inet.tcp.ecn.enable}} parameter.<ref>{{cite web|url=http://netbsd.org/releases/formal-4/NetBSD-4.0.html|title=Announcing NetBSD 4.0|date=2007-12-19|access-date=2014-10-13|url-status=live|archive-url=https://web.archive.org/web/20141031073922/http://netbsd.org/releases/formal-4/NetBSD-4.0.html|archive-date=2014-10-31}}</ref>

Likewise, the sysctl {{Mono|net.inet.tcp.ecn}} can be used in [[OpenBSD]].<ref>{{cite book|url=https://books.google.com/books?id=PN6Xy9zWAbsC&pg=PA347 |title=Absolute OpenBSD: UNIX for the Practical Paranoid |author=Michael Lucas |year=2013 |publisher=No Starch Press |isbn=9781593274764 |access-date=2014-03-22}}</ref>

====Linux====
Since version 2.4.20 of the [[Linux kernel]], released in November 2002,<ref>{{cite web
 |url         = https://datatag.web.cern.ch/datatag/papers/tr-datatag-2004-1.pdf
 |title       = A Map of the Networking Code in Linux Kernel 2.4.20, Technical Report DataTAG-2004-1, FP5/IST DataTAG Project
 |date        = March 2004
 |access-date  = 1 September 2015
 |website     = datatag.web.cern.ch
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20151027124019/http://datatag.web.cern.ch/datatag/papers/tr-datatag-2004-1.pdf
 |archive-date = 27 October 2015
}}</ref> Linux supports three working modes of the ECN for TCP, as configured through the [[sysctl]] interface by setting parameter {{Mono|/proc/sys/net/ipv4/tcp_ecn}} to one of the following values:<ref name="linux">{{cite web
 |url         = https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt
 |title       = Documentation/networking/ip-sysctl.txt: /proc/sys/net/ipv4/* Variables
 |access-date  = 2016-02-15
 |publisher   = [[kernel.org]]
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20160305080444/https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt
 |archive-date = 2016-03-05
}}</ref>
* {{Mono|0}}{{snd}} disable ECN and neither initiate nor accept it
* {{Mono|1}}{{snd}} enable ECN when requested by incoming connections, and also request ECN on outgoing connection attempts
* {{Mono|2}}{{snd}} (default) enable ECN when requested by incoming connections, but do not request ECN on outgoing connections

Beginning with version 4.1 of the Linux kernel, released in June 2015, the {{Mono|tcp_ecn_fallback}} mechanism{{Ref RFC|3168|rsection=6.1.1.1}} is enabled by default<ref>{{cite web
 |url         = http://man7.org/linux/man-pages/man7/tcp.7.html
 |title       = Linux man pages
 |date        = 2015-12-05
 |access-date  = 2016-02-15
 |website     = man7.org
 |url-status     = live
 |archive-url  = https://web.archive.org/web/20160216231741/http://man7.org/linux/man-pages/man7/tcp.7.html
 |archive-date = 2016-02-16
}}</ref> when ECN is enabled (the value of 1).  The fallback mechanism attempts ECN connectivity in the initial setup of outgoing connections, with a graceful fallback for transmissions without ECN capability, mitigating issues with ECN-intolerant hosts or firewalls.

====Mac OS X====
[[Mac OS X]] 10.5 and 10.6 implement ECN support for TCP. It is controlled using the Boolean [[sysctl]] variables {{Mono|net.inet.tcp.ecn_negotiate_in}} and {{Mono|net.inet.tcp.ecn_initiate_out}}.<ref name="mac">{{cite web | url = http://www.icir.org/floyd/ecn.html#implementations | title = ECN (Explicit Congestion Notification) in TCP/IP | url-status = live | archive-url = https://web.archive.org/web/20120619200913/http://www.icir.org/floyd/ecn.html#implementations | archive-date = 2012-06-19 }}</ref>  The first variable enables ECN on incoming connections that already have ECN flags set; the second one tries to initiate outgoing connections with ECN enabled. Both variables default to {{Mono|0}},{{Failed verification|date=February 2025}} but can be set to {{Mono|1}} to enable the respective behavior.

In June 2015, [[Apple Inc.]] announced that [[OS X 10.11]] would have ECN turned on by default,<ref name="apple-wwdc-719" /> but the OS shipped without that default behavior. In macOS Sierra, ECN is enabled for half of TCP sessions.<ref>{{cite web|url=https://arstechnica.com/gadgets/2016/09/macos-10-12-sierra-the-ars-technica-review/6/|title=macOS 10.12 Sierra: The Ars Technica review|website=Ars Technica|date=20 September 2016|access-date=25 April 2018|url-status=live|archive-url=https://web.archive.org/web/20180426012040/https://arstechnica.com/gadgets/2016/09/macos-10-12-sierra-the-ars-technica-review/6/|archive-date=26 April 2018}}</ref>

====iOS====
In June 2015, [[Apple Inc.]] announced that [[iOS 9]], its next version of iOS, would support ECN and have it turned on by default.<ref name="apple-wwdc-719" /> TCP ECN negotiation is enabled on 5% of randomly selected connections over Wi-Fi / Ethernet in iOS 9 and 50% of randomly selected connections over Wi-Fi / Ethernet and a few cellular carriers in [[iOS 10]]<ref>{{cite web|url=https://developer.apple.com/videos/play/wwdc2016/714/|title=Networking for the Modern Internet - WWDC 2016 - Videos - Apple Developer|first=Apple|last=Inc.|website=Apple Developer|access-date=18 April 2018|url-status=live|archive-url=https://web.archive.org/web/20180418161153/https://developer.apple.com/videos/play/wwdc2016/714/|archive-date=18 April 2018}}</ref><ref>{{cite web |url=https://www.ietf.org/proceedings/98/slides/slides-98-maprg-tcp-ecn-experience-with-enabling-ecn-on-the-internet-padma-bhooma-00.pdf |title=TCP ECN — Experience with enabling ECN on the Internet |first=Padma|last=Bhooma |date=March 2017 |access-date=2017-05-03 |url-status=live |archive-url=https://web.archive.org/web/20180509153808/https://www.ietf.org/proceedings/98/slides/slides-98-maprg-tcp-ecn-experience-with-enabling-ecn-on-the-internet-padma-bhooma-00.pdf |archive-date=2018-05-09 }}</ref> and 100% for [[iOS 11]]<ref>{{cite web|url=https://developer.apple.com/videos/play/wwdc2017/707/|title=Advances in Networking, Part 1 - WWDC 2017 - Videos - Apple Developer|first=Apple|last=Inc.|website=Apple Developer|access-date=18 April 2018|url-status=live|archive-url=https://web.archive.org/web/20180131043815/https://developer.apple.com/videos/play/wwdc2017/707/|archive-date=31 January 2018}}</ref>

====Solaris====
The [[Solaris (operating system)|Solaris]] kernel supports three states of ECN for TCP:<ref>{{cite web |title=ipadm(8) |url=https://docs.oracle.com/cd/E88353_01/html/E72487/ipadm-8.html |website=Oracle Solaris 11.4 Information Library |publisher=Oracle |access-date=6 May 2021}}</ref>
* {{Mono|never}}{{snd}} no ECN
* {{Mono|active}}{{snd}} use ECN
* {{Mono|passive}}{{snd}} only advertise ECN support when asked for.

As of Solaris 11.4, the default behavior is {{Mono|active}}. ECN usage can be modified via {{Mono|ipadm set-prop -p ecn{{=}}active tcp}}.<ref>{{cite web |title=Administering TCP/IP Networks, IPMP, and IP Tunnels in Oracle® Solaris 11.4, Using the TCP ECN Feature |url=https://docs.oracle.com/cd/E37838_01/html/E60991/gqyxf.html |website=Oracle Solaris 11.4 Information Library |publisher=Oracle |access-date=6 May 2021}}</ref>

===ECN support in IP by routers===
Since ECN marking in routers is dependent on some form of [[active queue management]], routers must be configured with a suitable queue discipline in order to perform ECN marking.

Cisco IOS routers perform ECN marking if configured with the [[Weighted random early detection|WRED]] queuing discipline since version 12.2(8)T.

Linux routers perform ECN marking if configured with one of the [[Random Early Detection|RED]] or GRED queue disciplines with an explicit ''ecn'' parameter, by using the [[Stochastic Fair Blue|sfb]] discipline, by using the [[CoDel]] Fair Queuing (fq_codel) discipline, or the CAKE<ref name=":0">{{cite arXiv|title=Piece of CAKE: A Comprehensive Queue Management Solution for Home Gateways|last1=Høiland-Jørgensen|first1=Toke|last2=Täht|first2=Dave|last3=Morton|first3=Jonathan|year=2018|class=cs.NI|eprint=1804.07617v2}}</ref> queuing discipline.

Modern BSD implementations, such as [[FreeBSD]], [[NetBSD]] and [[OpenBSD]], have support for ECN marking in the [[ALTQ]] queueing implementation for a number of [[queuing discipline]]s, notably [[Random Early Detection|RED]] and [[Blue (queue management algorithm)|Blue]]. [[FreeBSD]] 11 included [[CoDel]], PIE, FQ-CoDel and FQ-PIE [[queuing discipline]]s implementation in [[ipfw]]/dummynet framework with ECN marking capability.<ref>{{cite web|title=Import Dummynet AQM version 0.2.1 (CoDel, FQ-CoDel, PIE and FQ-PIE) to FreeBSD 11|url=https://svnweb.freebsd.org/base?view=revision&revision=300779|website=The FreeBSD Project, FreeBSD r300779|access-date=5 August 2016}}</ref>

=== {{Anchor|DCTCP}}Data Center TCP ===
''Data Center Transmission Control Protocol'' (''Data Center TCP'' or ''DCTCP'') utilizes ECN to enhance the [[Transmission Control Protocol]] congestion control algorithm.<ref>{{cite web|title=Data Center TCP (DCTCP)|url=http://netbsd.org/releases/formal-4/NetBSD-4.0.html|archive-url=https://web.archive.org/web/20141031073922/http://netbsd.org/releases/formal-4/NetBSD-4.0.html |access-date=March 7, 2023|archive-date=2014-10-31 }}</ref> It is used in [[data center]] networks. Whereas the standard [[TCP congestion control algorithm]] is only able to detect the ''presence'' of congestion, DCTCP, using ECN, is able to gauge the ''extent'' of congestion.<ref>{{cite IETF|rfc=8257|title=Data Center TCP (DCTCP): TCP Congestion Control for Data Centers|access-date=August 21, 2021}}</ref>

DCTCP modifies the TCP receiver to always relay the exact ECN marking of incoming packets at the cost of ignoring a function that is meant to preserve signalling reliability. This makes a DCTCP sender vulnerable to loss of ACKs from the receiver, which it has no mechanism to detect or cope with.<ref name="rfc7560">{{Cite IETF |rfc = 7560 |title = Problem Statement and Requirements for Increased Accuracy in Explicit Congestion Notification (ECN) Feedback |date = August 26, 2015 |access-date = August 21, 2021}}</ref> {{As of|2014|7}}, algorithms that provide equivalent or better receiver feedback in a more reliable approach are an active research topic.<ref name="rfc7560"/>

==See also==

* [[Backward ECN]] (BECN)
* [[L4S]], which uses the ECN bits
* [[Network congestion avoidance]]
* [[Type of service]] (ToS)

==References==
{{Reflist|30em}}

==External links==
* [http://www.icir.org/floyd/ecn.html ECN web page by Sally Floyd]
* {{IETF RFC|4774|link=no}} (BCP 124), ''Specifying Alternate Semantics for the Explicit Congestion Notification (ECN) Field'', S. Floyd, (November 2006)
* [https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=81164413ad096bafe8ad1068f3f095a7dd081d8b Linux kernel support for defining a per-route/destination congestion control algorithm] (merged in Linux kernel 4.0)

[[Category:Network performance]]
[[Category:Transport layer protocols]]
[[Category:Flow control (data)]]