Disable Nagle in Low Latency High Frequency network applications February 10, 2012
Posted by anywhereinfo in Linux, TCP.Tags: nagle algorithm
add a comment
Most of the low latency, high frequency network applications that i work with, disable Nagle Algorithm. The reason is that “Nagle Algo”,”Delayed ACKs” and “Application ACKs” do not work together nicely.
The concept behind delayed ACKs is to bet, when receiving some data from the net, that the local application will send a reply very soon. So there’s no need to send an ACK immediately; the ACK can be piggybacked on the next data going the other way. If that doesn’t happen, after a 500ms delay, an ACK is sent anyway.
The concept behind the Nagle algorithm is that if the sender is doing very tiny writes (like single bytes, from Telnet), there’s no reason to have more than one unacknowledged packet outstanding on the connection. This prevents slow links from choking with huge numbers of outstanding tinygrams.
With both enabled, you begin to see a fixed latency as below.
Application A gives the first packet to TCP. TCP Sends it, because there is no outstanding packet.The packet size does not matter
Application A gives the second packet to TCP. Here 2 possibilities are
1. Packet Size = MSS :- TCP Sends Packet.
2. Packet Size < MSS :- Nagle prevents TCP from sending packet
If Application A continues to pump data, then once Packet Size >= MSS, TCP will send the packet.
If Application A expects an application level ACK before handing more data after the second packet, the second packet is queued.
Receiver does not acknowledge the first packet, due to delayed ACK.
Finally, Delayed ACK timer indicates timeout and the receiver sends an ACK after 500 ms.
So from this we can conclude that
1. As long as Application Acknowledgments don’t impose additional constraints
2. The sender always has pending data to send
3. Receives has not set un-usual delayed ACK Settings, which causes sender TCP to execute re-transmissions
4. Network behaves normally
Nagle should have no impact.
In low latecny, high frequency, network applications pt 2 is normally not an issue. Pt3 is also a rare case. Pt4 impacts network applications and it requires its own blog. Pt1, is typically the main issue.
Sooner or later when your code runs into the scenario where sender despite having data to send, will not hand any data to sender TCP because of Application level Receiver acknowledgment, and the Receiver is waiting on sender to send data before it can send an ACK, you will experience the full delay from Delayed ACK timeout settings.
How big is your byte? January 23, 2012
Posted by anywhereinfo in C++, Memory.Tags: addressable unit
add a comment
The size of the byte is hardware dependent and no standards exist that dictate it’s size. In C/C++ programming languages, byte is an “addressable unit of data storage large enough to hold any member of the basic character set of the execution environment” (3.6 of the C standard).
Here is an interesting C++ FAQ on Byte
This ambiguity is the main reason as to why IP Networks use “Octet” and not a “Byte” to describe their internal data such as IP Address.
Extract RPM without installing it January 23, 2012
Posted by anywhereinfo in Linux, rpm.add a comment
While doing C++ development on RHEL Platform, i came across the need to extract code out of custom RPM. A single liner solution is
rpm2cpio custom_code.rpm | cpio -idmv
At time of writing this post, both the tools (rpm2cpio and cpio) are available via yum install.
The above command will typically produce 2 types of files
- custom_code.spec
- custom_code.tar.gz
You can extract the code in the point 2 by
tar -zxvf custom_code.tar.gz
