Mac Geekery

Answer

The CPU isn’t the only bottleneck on the system, not by a long shot. Commonly, we associate a high CPU load with a lot of processing and then a system slowdown, but feeding the CPU too slowly and carrying out its tasks lethargically is just as common a cause for a slowdown as is feeding the CPU too quickly.

The first other possible bottleneck is the system memory. While a gigabyte of RAM works out well for most daily tasks, when you’re doing a lot of memory-hungry tasks at once, say using Photoshop and then trying to browse a few graphic-heavy sites, the system will be shuffling a lot of data in and out of RAM from the hard drive, and then trying to get that data to the CPU for processing. Some of that data, in the example, will be Photoshop’s own scratch data that it uses because VM is slow, and some will be VM because you switched programs and the web browser had paged out of active memory. So then you have two large streams of disk access, two large streams of data going to the main memory, and both coming back out of RAM to be processed by the CPU in that limited amount of memory, possibly paging more out to disk in the process.

That might get a little slow, as you can see, and it has nothing to do with your CPU speed or CPU load. It has to do with an evil little number called I/O Wait, or a statistic that estimates how much time your CPU is spending sitting idly by waiting on the rest of the computer to catch up to the I/O commands it has issued, be it RAM, disk, or PCI card. Sadly, I can’t find a way to get this statistic in Tiger. Neither top nor sar report it. In Linux, both do, and it’s quite handy. So in Mac OS X, we guess how much time is spent dealing with I/O Wait.

The fundamental concept behind I/O causing a system to slow down is called blocking. When the CPU issues a request for data, it can block execution of a command until that request is complete. This means that no further actions are processed, at all, until the response comes back from that request. This can happen within the processor as a synchronous I/O request or in an execution thread as a blocking asynchronous I/O request (which would block the thread, but not the machine).

It’s still somewhat common to use blocking methods for requesting both data from RAM and from the network or disks because the data requested is usually about to be used in a computation. Sometimes a cache of some form or another will issue a non-blocking I/O request to get data it anticipates the CPU will need and stores it somewhere handy. The speed of that request isn’t of concern, so non-blocking is fine and you’ll never know it did it (generally). The issue is purely with calls for data that block that thread.

This extends outside of I/O, as well. There are calls within the system frameworks for computation, graphics, inter-process communication, and others that can block thread execution and cause the program or system to become unresponsive. Commonly, programmers will spin off a new thread to handle this so that the main thread never becomes blocked (a blocked main thread is the single cause of the Spinning Gay Pride Pinwheel of Death). If a programmer does something silly on the main thread, the whole app will lock up until it’s done. If that thread-blocking action requires I/O and that gets blocked, then you get a no-load hang of the program. If that action is a blocking inter-process communication that results in an I/O block in the other program, then both hang. If the other program is the kernel, the system may stall, depending on the request (almost impossible, but only almost).

So, if something is waiting on something that is waiting on the kernel, then what happens if the kernel is busy when the answer comes in and it somehow misses the response? It keeps waiting, and keeps blocking, and you have a hung system with no load. Maybe it’ll time out, or try again, or realize it got the response, or maybe it’ll just sit there until you kick it in the power button, there’s no telling.

But, dear reader, that’s why you can get a lethargic computer with no load at all. The computer is a system of components, and you’re only watching the pretty CPU chart. Something like iStat might be a little more helpful, but what I really, really want is GKrellM ported to the Mac…

For the retentive sphincters in the crowd, a blocking I/O request is synchronous I/O and a non-blocking request is asynchronous I/O. While more descriptive of their functionality, I feel those are less descriptive concerning the importance of their differences in this dicussion.