Get Active - Lowering Networking Power in Data Centers
In the past I’ve discussed topics such as virtualization and digital power to help improve data center processing efficiency. I may even have discussed additions to the 802.3 standard to idle Ethernet drops when they were not in use. However, I have not addressed the interconnect power itself and it was surprising what I found.
In medium scale data centers such as those run by financial institutions, large retailers or corporations you will find thousands of server blades and the networking equipment to connect them together. What is interesting about this architecture is that the majority of networking traffic occurs within the data center itself. The reason for this is partially due to the litigious nature of our society and the never ending quest for information to help us understand ourselves. For example, simply performing an on-line stock trade - which to the user is a single transaction - will spawn dozens of additional inter-server transactions to secure, execute, verify and log the event as well as extract statistics used in market analysis. So when millions of people are on-line day trading stocks, billions of transactions are occurring within the data centers.
This huge amount of traffic need bandwidth and traditionally this has been accomplished by employing fiber optic cable. Fiber has the advantage of a very small diameter thus providing space for air-flow to cool the systems. Larger copper wire could be used for short hauls, but the diameter would block the air-flow and cause over-heating.
Fiber requires light (lasers) to operate and different distances and data rates require different modes of optical transmission. To allow flexibility, equipment manufacturers have created connectors that accept a module that contains the laser and receiver electronics. These are many variants, but the most accepted standards are SFP+ (Small Form-factor pluggable), QSFP (Quad SFP), CFP ("C" or x100 Form-factor Pluggable), XFP (10 Gigabit small Form-factor Pluggable), and CXP. These modules are actively powered and consume 400-500 milliwatts of power each! When you have thousands of them the power quickly adds up. Additionally, the heat generated must be dealt with and the modules are also very expensive.
Now what’s most interesting is that the majority of interconnects within the data center are only a few meters long! Normally passive copper cables would work fine but as mentioned above they would decrease the airflow at the back of the equipment. So a clever solution is to use smaller diameter copper wire (28-30 AWG) which suffers from higher loss and place active drivers and equalizers such as the DS64BR401 in the connectors which fit these standard module sockets. This technique is called "Active Copper" or "Active Cable" and has many benefits in less than 20 meters runs. The first benefit is cost - these cables can be less than half the cost of the fiber module and cable. The second is power - active cables can reduce the power consumption significantly if properly designed (< 200 mW vs. 400mW for fiber).
Fiber will always have a place for carrying data long distances for which it excels. However, in the data center copper wire is regaining ground with the help of active electronics may be the majority of media carrying your next stock trade! Till next time...