|
Mac GeekeryGet your geek on. |
|
blog advertising is good for you
recent popular content
User login
|
Apple slipped this one in right under my nose. As of Mac OS X 10.4.6, we have SACKs! That is to say, the primary thing I lamented MOSX lacking in my Mac OS X Network Tuning Guide has been remedied, and is enabled by default. So, here’s the lowdown on how to tune a post-10.4.6 install of Mac OS X. Editing These SettingsAll of the following are
Be very sure you agree with the change before doing so. These are my settings. Think long and hard about making them yours as well. This is not flagged as an Easy or Trivial article for a reason; nothing here is something you just drop in. You must understand what you’re doing at some level for this to be helpful to you. New Reno Fast Recovery Methodnet.inet.tcp.newreno implements the “New Reno” modification to the “Reno” TCP Fast Recovery Protocol (RFC 2582). New Reno is the name given to an enhancement to TCP partial acknowledgments (Tahoe/Reno) that is used in modern server TCP stacks in the absence of a client that supports SACKs. It’s a bit complicated to explain if you’re not familiar with TCP recovery techniques, but suffice it to say that it’s a hack based on assumptions that aren’t always true, as was its predecessor. Now that Mac OS X supports SACKs, this is essentially obsolete. However, if you cannot presume your clients will mostly be SACKs-capable machines then (and only then) you should turn on New Reno support to aid those client machines if you primarily serve this class of machine and are on a link that is not heavily lossy. Client systems that do not support SACKs include: Mac OS prior to 10.4.6, Linux prior to 2.1, Windows prior to Windows 95 + Windows Sockets 2.0. This is only helpful over a congested link with very light packet loss issues and clients without SACKs. For your local LAN, you won’t need this (unless you have a rather horrible LAN). This is something an Internet server would use rather than a LAN server. For LAN servers, especially gigabit, this will reduce your performance, even on good clients. If you have a couple of old machines, let them suffer rather than hurt the common user. SACKs is far, far better for this job. This is off by default, and should stay that way unless you have good reason to change it. In my testing, New Reno on a functional gigabit network reduced performance 50%. Use when needed, not before. Read why. Selective AcknowledgementsMac OS X 10.4.6 and later supports Selective Acknowledgments, so be sure to upgrade if you have not. TCP Selective Acknowledgement is a feature of the TCP protocol whereby the client can tell the server (at the protocol level) exactly which parts of a fragmented packet did not make it and ask for exactly those parts to be resent, rather than the whole packet. On a bad network with higher-than-normal packet loss, this can make a noticeable difference in performance. SACKs was first introduced as an idea in RFC 1323 and modified in RFC 2018. The two together comprise the most common implementation of SACKs in the wild, even though there are other RFC additions available. This defaults to on, and has the following controls:
SACKs are already on, and the default settings are perfect. Leave it be. Window Sizesnet.inet.tcp.recvspace and net.inet.tcp.sendspace control how much buffer space is allotted per socket connection, per direction. This is how much data the kernel will cache on a socket while the application chews on it. For data streams over a slow link (even 10Mb enet) this won’t matter as the processor can keep up rather well. On really fast links like gigabit, this is essential. This setting defaults to 32K each way but moving that higher lets the system get more data cached and ready for your system if it falls behind the network traffic. As a test, I rose this to 250KB for each direction on my local machine and another on my local gigabit network. Transfers went from 12MB/s to 25MB/s. Thinking I could get more, I then transferred data from both hard drives on the remote machine. Sure enough, the remote hard drive was the limiting factor and it jumped to 35MB/s. Just be very sure to update kern.ipc.maxsockbuf to match the total of both settings before changing these settings, especially in To resolve such a scenario, startup in Single User Mode, remount the drive as read/write, and edit/remove the The following are the settings I use on my machines on a gigabit network: kern.ipc.maxsockbuf=500000 net.inet.tcp.sendspace=250000 net.inet.tcp.recvspace=250000 Maximum Segment Sizenet.inet.tcp.mssdflt sets the default Maximum Segment Size, or the largest that the system will set the data portion of a TCP packet to. By default, many BSD-derivatives will set this to 512 bytes. Bad. Ethernet supports a frame size of 1,500 bytes on links up to 100 Mb or 9,000 bytes on 1000Mb links so if either is what you’re connected to then up this bad boy to the size of your frame minus the size of a TCP header and options (60 bytes). For 10/100Mb links, set it to 1,440; for 1000Mb links that have been explicitly set to use jumbo frames on all participating computers, set it to 8940. Remember: if you set your frames larger than the frames your router allows (on a large LAN) then it could fragment the packet when re-sending or outright crash, and either would suck. If you fail to calculate this correctly and it goes over the size of the hardware frame, then you’ll be fragmenting every non-trivial packet you send, slowing down your network. Be careful. Even though I’m on a gigabit network, I use: net.inet.tcp.mssdflt=1440 The reason is that I transition between AirPort and wired often, so I’d rather not kill my AirPort performance (a 1,500-byte frame) for those few extra bits on ethernet. Besides, because of the larger buffer mentioned previously, I’ve already exceeded the speed of the hard drive on the server. Extra tweaking is not needed. Sockets and Connectionsnet.inet.tcp.sockthreshold sets the number of open sockets needed for the system to actually obey your sendspace and recvspace marks. If the number of open sockets is below this number then the buffers are set to 64K, regardless of what you’ve set them to. This means that the system allocates 64K of buffer space to each socket connection direction, or 128K total, until you get to 256 open sockets, at which point they’re set to 32K (the default send/recvspace values). Set to 0 to disable and always use your custom sizes (not really a good idea). kern.ipc.somaxconn controls the size of the connection listening queue and typically only needs to be adjusted in high-performance server environments. The default value of 128 is more than adequate for a home/work machine and most workgroup servers. If, however, you are running a high-volume server and connections are getting refused at a TCP level, then you want to increase this. This is a very tweakable setting in such a case. Too high and you’ll get resource problems as it tries to notify a server of a large number of connections and many will remain pending, and too low and you’ll get refused connections. Real-Word WinsUsing an unmodified Mac OS X and connecting over gigabit ethernet to an unmodified Mac OS X over AFP, I transferred a 4GB file.
Remember, change these settings at both ends for maximum benefit. Create a AddendumWhy Reno & New Reno Break GigabitNew Reno is a modification to Reno, which is a modification to Tahoe, which is a congestion recovery algorithm for TCP (a way of determining when packets are lost and coordinating a retransmit of the lost packets). TCP Tahoe said that if a client hasn’t received a packet in an expected amount of time that the connection must be dropping packets due to full buffers somewhere down the line. In order to prevent future losses, the TCP stack should reduce its speed by entering a state called “slow start” where it drops the speed and slowly ramps it back up again. It does this in hopes of reducing the number of retransmits required over the course of the transaction. Tahoe also features a concept called Fast Retransmit. When it detects three negatively-acknowledged packets (NACKs) from the client the server will instantly start retransmitting older unacknowledged packets, presuming they’ve been lost. The TCP Reno modification introduced Fast Recovery, which said that if it received a NACK packet from the client that the simple admission of a missing packet meant that the link was still not truly congested and that it should not fall back to a complete slow start but only reduce throughput slightly. Only when the server starts missing out on NACKs completely should the stack fall back to slow start. If data is getting through, then use that state to try and repair the link rather than starting over, essentially. Once a NACK is received from the client, Reno presumes the link is good again and negotiates a return to full speed. Both Tahoe and Reno suck at multiple packet losses. Both presume a constant stream of data rather than bursts of data. With so many clients on today’s Internet, data is only ever really sent in short bursts from any one client, so it would be a reasonable conclusion that it’s possible for a whole section of a stream to be lost. Neither congestion avoidance method has a qualification for this problem. Additionally, Reno forces a delay equal to the calculated Round-Trip Time (RTT) of a packet from the server to the client whenever there is a loss. This means that for every loss, the connection is stalled for a duration of at least the RTT for every occurrence. TCP New Reno changes Reno in several ways. In New Reno, the Fast Recovery method was changed so that NACKs from the client will only exit Fast Recovery if certain conditions were met as to which packet was NACKed (old or new, essentially). Only when all outstanding packets from the initial loss period have been acknowledged should the stack return to normal. This has a consequence of eliminating the RTT delay. By not exiting Fast Recovery as easily, New Reno does not penalize the connection when related losses would have triggered a re-entry into Fast Recovery as in the Reno method. However, at gigabit speeds, the network can be significantly faster than the TCP stack or application layer, so buffers get full and packets get dropped (ignored) regularly. This will then trigger New Reno’s still-agressive Fast Recovery to prevent future losses and find a speed that will not lose any packets at all. At the speeds involved, however, Selective Acknowledgements would allow the client to return a list of packets that have not been received and request a retransmit of those packets with no performance penalty. This would permit full-speed transmits over well-maintained lines whereas Reno/New Reno would drop the connection into Fast Recovery mode at the slightest hint of trouble. Thus, with so much bandwidth available to the client and server, having a full discussion of what packets are missing is cheaper than making educated guesses as to what could be lost, simply because of the sheer volume of data being transmitted in a given moment in time (up to 113 MB/s at 95% capacity). For more information, including comparative examples, read Simulation-based Comparisons of Tahoe, Reno, and SACK TCP [PDF] from Sally Floyd’s SACKS resources page.
About Adam Knight
 Author Biography Adam Knight is one of the founders of Mac Geekery and is a geek at heart. Programmer by day, hacker by night, his daily life revolves around the Macintosh platform, which he has been a user and programmer for since the early days of System 7 when his LCII replaced his Apple //c. In-between tech jobs, he’s managed to learn the basics of any web hacker: PHP, MySQL, Perl, Apache, Linux, *BSD, and the intricacies of ./configure —prefix=~/bombshelter/. Today, codepoet is concentrating on blogging again, writing some software for the Mac by himself (including Notae) and for his company (such as Switchblade) and has a few other toys coming out soon. Bug him over AIM or email [link fixed]. |
You might want to make some of those parms powers of two
net.inet.tcp.recvspace=262144
etc. I’d be interested to see if setting on power of two boundaries is where the performance changes.
it seems that i cannot have more than 256 connections opened on a single server socket on my powermac 10.4.8
if((client_socket = accept(server_socket, NULL, NULL)) < 0) { cout << “accept failed\n”; exit(-1); } r++; printf ( “ open %d\n”, r); } ————————— and client code ————————— int r=0; while( r < 260) { if(connect(client_socket, (struct sockaddr*)&srv_addr, sizeof(srv_addr)) < 0) { cout << “connect failed\n”; exit(-1); } r++; if ((client_socket = socket(PF_INET, SOCK_STREAM, 0)) < 0) { cout << “Error creating client socket end!\n”; exit(-1); } }i cannot figure what is the kernel parameter to modify , tried some but without result.
take for example this code like http://www.citi.umich.edu/u/aglo/eecs482-fall04/11.15-17.txt
to create a simple server with accept() function . you can easy modify cient and server code .
————— int r=0; while(1) {
—————————-
i was testing this good library http://www.aspl.es/vortex/ but regression test fail to work due to this sw limit.
any idea?
Best Regards
Attilio
How do you feel about the defaults for Broadband tuner?
http://www.apple.com/support/downloads/broadbandtuner10.html
thanks alot for your work !! Have you considered / tested setting your send/recv buffers higher such as this tutorial lays out ?
http://www.psc.edu/networking/projects/tcptune/#MacOS