Jim O'Reilly


Upcoming Events

Where the Cloud Touches Down: Simplifying Data Center Infrastructure Management

Thursday, July 25, 2013
10:00 AM PT/1:00 PM ET

In most data centers, DCIM rests on a shaky foundation of manual record keeping and scattered documentation. OpManager replaces data center documentation with a single repository for data, QRCodes for asset tracking, accurate 3D mapping of asset locations, and a configuration management database (CMDB). In this webcast, sponsored by ManageEngine, you will see how a real-world datacenter mapping stored in racktables gets imported into OpManager, which then provides a 3D visualization of where assets actually are. You'll also see how the QR Code generator helps you make the link between real assets and the monitoring world, and how the layered CMDB provides a single point of view for all your configuration data.

Register Now!

A Network Computing Webinar:
SDN First Steps

Thursday, August 8, 2013
11:00 AM PT / 2:00 PM ET

This webinar will help attendees understand the overall concept of SDN and its benefits, describe the different conceptual approaches to SDN, and examine the various technologies, both proprietary and open source, that are emerging. It will also help users decide whether SDN makes sense in their environment, and outline the first steps IT can take for testing SDN technologies.

Register Now!

More Events »

Subscribe to Newsletter

  • Keep up with all of the latest news and analysis on the fast-moving IT industry with Network Computing newsletters.
Sign Up

See more from this blogger

Are InfiniBand's Days Numbered?

InfiniBand (IB) has been something of an anomaly for years now. While it's clearly the fastest storage interface, it's mainly supplied only by Mellanox with Intel providing some products. The reason for this is that IB services a few niches where ultimate performance and low latency matter, such as high-performance computing (HPC) and financial systems. The broader markets are serviced by Fibre Channel, which is gradually being supplanted by Ethernet, due to cost and performance.

IB technology leads the storage industry, with 56Gb/sec links available, which leads Fibre Channel’s 16Gb/sec product by a wide margin. InfiniBand is priced fairly aggressively against Fibre Channel, but the latter’s entrenched position a few years back slowed IB’s acceptance in the market, coupled with ease of deployment issues.

More Insights


More >>

White Papers

More >>


More >>

With HPC, financial systems, and big data growing at good rates, it would seem that IB should enjoy a glowing future, but there are dark clouds on the horizon.

IB gets its performance from Remote Direct Memory Access (RDMA), which allows an application to write data directly into the memory of the target system, without having in-memory copies and I/O stack software to slow it down. That technology is now appearing in the Ethernet market, provided by Chelsio and, surprisingly, Mellanox itself.

Chelsio and Mellanox have rival architectures, with Mellanox’ RoCE (RDMA over Converged Ethernet) requiring a lossless Ethernet infrastructure, though it might perform slightly faster than iWARP from Chelsio. Both of the Ethernet solutions are available in 40Gb/sec configurations; Mellanox also supports a 56Gb/sec capability.

Performance benchmarking indicates that Ethernet RDMA performs close enough to IB that the advantage enjoyed by IB in the past is negated.

Both Ethernet solutions at 40Gb/sec are relatively new, though both companies have years of experience in RDMA designs. The issue is projecting whether we will see Ethernet take business from IB, or if IB will sail serenely onward.

[Read Jim O'Reilly's thoughts on the rise of 10GbE in "Will 2014 Be The Year Of 10 Gigabit Ethernet?"]

Mellanox is firmly on both sides of the debate, with a versatile solution that can be tailored to either; it should thrive whichever way the decision goes. At the same time, Mellanox's recent emphasis on Ethernet and its creation of RoCE indicate the company's sense of the market.

With a version of Fibre Channel over Ethernet (FCoE) and cheap alternatives such as iSCSI providing good performance for block I/O, the SAN space is moving to Ethernet and Fibre Channel looks to be fading. Also, there is massive expansion of object storage, again using Ethernet interfaces. The net result is that in about five years, Ethernet will be the strongly dominant storage interface.

That isolates IB, and the fact that Ethernet is close in performance and latency removes the strongest raison d’etre for IB. Ethernet should enjoy the benefits of huge volumes, and has the advantage of many vendors driving its evolution. As a result, IB looks also to be a candidate for Ethernet convergence.

The next step in the Ethernet community appears to be 100Gb/sec using four connections per link. I suspect IB will have a product with those parameters, but the cost of PHY development likely means that there won’t be a “140Gb/sec” IB product. This is probably the cross-over point where Ethernet is preferable to IB.

The Fibre Channel folks are talking up 32Gb/sec and beyond, but their schedule will give them a slower product than 40Gb/sec Ethernet/IB in maybe two years. To get to anything like parity with Ethernet and IB, Fibre Channel has to go to multiple connections per link, and that is even further out. And even when it gets there, there’s that pesky RDMA advantage that both Ethernet and IB enjoy. Enough said.

There will still be IB (and Fibre Channel) around a decade from now, since the industry is surprisingly conservative and technologies have passionate adherents, but the writing is on the wall, and Ethernet looks likely to own storage at some point.

Related Reading

Network Computing encourages readers to engage in spirited, healthy debate, including taking us to task. However, Network Computing moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing/SPAM. Network Computing further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | Please read our commenting policy.
Vendor Comparisons
Network Computing’s Vendor Comparisons provide extensive details on products and services, including downloadable feature matrices. Our categories include:

Research and Reports

Network Computing: April 2013

TechWeb Careers