My interest in NUMA (Non-Uniform Memory Access) has picked up quite a bit as of late. This technology has a huge potential for application in large scale Web hosting and Internet solutions. In brief, NUMA is a technology that was conceived to get around bus contention in general. What this means is that, with NUMA, programs are no longer limited by memory size or disk IOPS on the same host node and can, therefore, harness the power of other nearby servers as if they were part of the same chassis. In fact, IO scales up as a result!

The problem with uniform memory access architectures is that in a single chassis multi-processor system with a large amount of memory the bus tends to saturate very quickly. Processor cores addressing the memory bus are thus contending for the same resource, which becomes a hot spot. Take a 32-core server with 128GB of DDR3 memory. This server will under perform should it run a parallel memory hungry application. The issue being that each task on a core will at some point need to address memory via the bus (DDR3 is a three-frequency or three-channel bus). So out of 32 tasks that are all attempting to access the bus, there will be (32 -3 = 29/3 ~ 9) 9 tasks that contend for the same channel. The end result is a saturated bus and poor performance.

NUMA solves this bus contention problem by creating a virtual network of processor nodes and giving each node a certain weight based on its locality. The most interesting aspect is its true scalability potential. Most importantly, NUMA is the only technology that can scale up. Amazon’s EC2 and most other technologies in the market nowadays can only scale out. In other words, one has to implement load balancing, replication, and data synchronization.

Here’s where it gets interesting. IBM has already comoditised NUMA with its x86 offering: the IBM x440. Although one can only chain two x440 chassis together to form one NUMA network, what was once a concept is now practical! The x440′s can be chained together via their SMP expansion module using remote IO cables. Now imagine having the ability to chain 5, 10, 100 16GB dual quad core boxes! Not only does one take advantage of of a few terrabytes of memory but also scale core compute and IO.

A challenge remains. The remote IO cables that allow formation of a NUMA node cluster is based on proprietary technology. The research and development effort required to deliver such technology is colossal for a smaller firm. Or maybe not!

The major flaw of the cloud, and ironically its touted strength, is its inability to scale up. For example, the promise of cloud providers is that you can start out with a 512MB nodes and scale up however much you want! The sky is the limit they say! Oh, but no more than 15GB of RAM on the same node! Truth be told, you’re going to hit bottlenecks way before you start using that much memory. Something is going to saturate whether it’s the IO subsystem, CPU, network IO, or the bus. And that’s probably the main reason for putting a cap at 15GB (15GB for domU + 1GB for dom0). It’s balanced and fair. But what if one needs 72GB of RAM on one node without having to worry about hitting IO / CPU bottlenecks? What to do? You’d be out luck!

You can scale out by allocating new nodes on other blades and load balancing the application across. This is when one needs to dig in the cloud, get down and dirty. You’d have to worry about provisioning scripts, data replication, sync, etc. So the answer is no, you can’t scale up with IO / CPU / and RAM on the same blade!

This is where NUMA comes in. It stands for non-uniform memory access. This is what I believe the next cloud will be based on. Imagine hundreds of stacked blades connected with very high speed IO and memory buses. Each server has 16GB of RAM. Process 108796 on server103 already consuming 132GB RAM needs an additional 30GB RAM to run, server105 (closest and less costly in terms of bus access) responds and delivers! IO is scalable since each blade has its own IO chip. So IOPS can scale up not down, as is the case today.

The good news is that the Nehalem architecture is the first NUMA capable Intel CPU. Perhaps paving the way for NUMA across blades. Linux has had NUMA code for ages. The next vendor to turn a bunch of Linux blades into a scalable super computer for hosting will take it all home. I think motherboard vendors and Intel will have to coordinate to come up with a bus that connects the blades’ fabric. Of course, it’s easier said than done.

NUMA