Demartek Commentary: FCoE Technology Math Lesson

3 February 2010

On the February 2, 2010 FCoE Fact vs. Fiction call, we discussed some of the potential technology advantages of Fibre Channel over Ethernet (FCoE). FCoE is the first major application to take advantage of the enhancements to Ethernet described by Data Center Bridging (DCB), and this is being implemented on a 10GbE technology base. This promises the ability to converge LAN and SAN technology, which suggests that CIOs should examine what can be done today and begin to plan for the future.

In a large datacenter, each server on a rack may have four, six, eight, or more 1GbE NIC ports and two-to-four 4Gb Fibre Channel (FC) ports. The NIC ports would probably be a combination of ports on the server motherboard and some number of single, dual, or possibly quad-port NIC add-in adapters. The Fibre Channel ports would be in the form of single or dual-port add-in adapters. These NIC and FC ports would be connected to different top-of-rack switches, the NICs connected to an Ethernet switch and the FC adapters connected to a Fibre Channel switch.

A typical rack with ten servers each having six 1GbE NIC ports (two on the motherboard and 2 dual-port NICs) and 1 dual-port 4Gb FC adapter then has:

140Gbps of total available I/O bandwidth,
80 cables (and switch ports consumed),
30 add-in adapters,
2 different top-of-rack switches.

In an FCoE design, this same rack of ten servers with a pair of dual-port FCoE adapters in each server would have:

400Gbps of total available I/O bandwidth,
40 cables (and switch ports consumed),
20 add-in adapters,
1 top-of-rack switch.

The number of cables has been reduced by 50%, the number of adapters reduced by 33% and the number of switches in the rack reduced by 50%. However, the total available I/O bandwidth has increased by 285%. There is also a reduction in power consumption, and therefore heat generated and air conditioning required for this new solution. This example is a relatively simple one, with only ten servers in a rack. Consider the impact this would have on a row of server racks that had more than ten servers in each rack.

Action Item: As you plan new computing racks or new data centers, consider the long-term savings in equipment and expense by implementing a converged network using FCoE.

The podcast of the Wikibon FCoE Fact vs Fiction call is now available at: http://wikibon.org/w/images/7/74/02-02-10_Peer_Incite.mp3.