Next generation data centre networking
Driven by Moore’s Law, the microelectronics industry has continually pushed down the cost of compute, however data centre operators face a technically challenging and costly future as they try to keep pace with evolving data centre network infrastructure requirements.
It’s generally fair to say that experts in the industry believe that the traditional approach to data centre implementation simply won’t scale economically to meet the needs of tomorrow’s Internet. While data centres are managing to keep pace with the vast quantities of data being generated, processed and stored, they are doing so at enormous cost. This expenditure is predicted to continue as bandwidth demands show no signs of levelling in the foreseeable future. Here Andrew Rickman, chief executive of Rockley Photonics outlines the industry’s key concerns and the technology innovation that will transform the future design of data centres whilst delivering scalability, cost and power reductions.
The data explosion
Let’s put things into perspective. In 2015, annual global IP traffic crossed the zettabyte threshold for the first time. To give you an idea of just how much data this relates to, an exabyte is equivalent to roughly 36,000 years’ worth of HD quality video. So 1,024 exabytes (1 zettabyte) would be equivalent to watching Netflix’s entire video catalogue every day, for the next 3 million years.
Social media channels and increasingly new entertainment and gaming platforms are having a profound impact on the amount of information data centres are required to process, store, distribute and deliver to the client. In addition to this, it’s thought that the Internet of Things (IoT) applications will generate a further 3.9 exabytes of data by 2017.
All this Internet activity creates in excess of 1,820 terabytes of new data every minute, which has to be stored, processed and shared between a burgeoning number of data centres located across the world. Without the data centre, there simply is no cloud.
As the volume of data being created, stored, and processed continues to escalate, so too do the capacity and bandwidth requirements of data centres. The Internet has grown 100-fold over the past ten years. To accommodate that growth, data centre capacity has grown 1,000-fold.
Current forecasts predict that data centres may require a further 1,000 times more bandwith and compute capacity over the next ten years. Enter the mega data centre. These enormous facilities will make more use of software to define the infrastructure and take advantage of open architectures for both software and hardware. However, when it comes to the issue of scalability and whether existing data centre architecture is capable of providing the required capacity economically, serious concerns are raised by the industry.
Networking: a perfect storm
To simplify matters, let’s divide a typical mega data centre into its two constituent parts: the server, or the compute function, which performs data processing and storage, and the network, which interconnects the vast number of servers (typically 100,000+) to each other and the world outside the data centre.
Moore’s Law has enabled microprocessor manufacturers to build higher performance compute machines with more storage capacity to be manufactured at less cost. However, when applied to the networking part of the data centre, Moore’s Law does not offer the same capability because this function also involves data transport over distance and therefore optics are also utilised — a completely different physical medium from the CMOS electronic domain where Moore’s Law has held true.
So while Moore’s Law sees to it that we can scale compute, network architecture has not fully benefited from the Law and is therefore becoming an issue of increasing focus for data centre cost containment.
We’ve now reached the stage where data centre operators have to focus on the network component in the face of the anticipated scaling required to meet future growth.
The problem with mega scale data centre networks is that they are being inefficiently built out of equipment that was never originally intended for networks of this size. This network complexity can be traced all the way down to a component level where the hardware vendors are still operating in silos and building network equipment with discreet CMOS switch chips, transceivers and high-speed electronic chassis constructions/backplanes.
Switching is still, by and large, performed via electronic CMOS switches that are currently limited from 24 to 36 ports each. A data centre with 100,000+ servers all needing connections to each other (and the outside world) requires a vast number of chips interconnected through multi-layer architectures. Needless to say, this generally leads to incredibly complex, expensive and inefficient network fabrics.
All the switch chips required, together with the high-speed circuit boards, cables and transceivers needed to build these network architectures are adding to network cost, network management complexity and power consumption, and it does not scale in sync with the rest of the data centre.
Spiralling data centre costs and power consumption are limiting factors that have been well documented and are serious issues. However, it is the fundamental continuing scalability of data centres that is under question from this perfect storm in networking.
Scaling efficiencies meet the network
Rockley Photonics has taken a holistic view of the networking functionality and is developing technologies and products that could finally put an end to the perpetually increasing costs and complexity of the data centre network (DCN).
Rockley has used a unique integrated combination of silicon photonics and traditional CMOS technology to break through the barriers that have until recently defined the current marketplace.
The packet switching approach consolidates the majority of the functional elements (and value) of the network within a single module. These modules combine the packet processing, switching, backplane and transceiver functionalities into a fiber-in/fiber-out (optical IO) modular unit that can be interconnected to operate like a single switch of virtually any size. The size of this switch can range from those standardly available from current vendors all the way up to unprecedented levels of scaling capable of interconnecting all the racks (or servers for that matter) in a megascale data centre.
This ‘smart integration’ of electronics and photonics will enable each technology to play to its strengths: CMOS’s density and ability to perform complex processing, and photonics’ outright speed and capacity of transmission at highly efficient power levels.
Re-inventing the data centre network
To enable future data centres, we need to significantly simplify the network. Fortunately, the industry is already migrating towards simpler and more efficient network architectures. One example is the move from classical single-rooted tree topologies with bandwidth bottlenecks and single points of failure, to folded Clos-based topologies that increase the capacity of the network by providing redundancy with multiple paths through the network.
Software-defined networks allow architects to separate the application, control, and physical transport layers and move them from proprietary hardware to open software. This approach allows the control plane processing, which guides packets to their destination, to be performed on a set of commodity servers, thus simplifying the design of the switches.
These are both examples of important, but fundamentally incremental improvements. To enable real scalability, we need much more. We need to deploy innovative, disruptive technologies in key areas of the network.
All hands on deck
In order to meet the demands of tomorrow in an environmentally responsible way, we must adopt a multi-faceted strategy.
The software-defined data centre (SDDC) is already driving greater efficiencies, while investment in renewable energy is taking an increasing number of data centres off the grid altogether.
In line with Moore’s Law, compute and storage are making strides in driving down energy consumption while increasing capacity.
There is already much to be optimistic about; however, in this ongoing quest to meet the digital demands of tomorrow, while at the same time, keeping our carbon footprint as small as possible, the network at the heart of the Internet remains the single largest obstacle we have yet to overcome.
However, developing this technology will take a system-level, multi-disciplinary approach, which is why Rockley has brought together world-class experts in photonics, electronics, software, system architecture and semiconductor/photonics manufacturing.
Data centre networking is a $10 bn and growing market. Through new thinking and the application of new technologies throughout the entire datacentre network fabric, it will be possible to recalibrate the cost and power trajectories of networking and enable the potential 1,000x increase in compute capacity needed to fuel the next 10 years of Internet growth.
Andrew Rickman OBE