There is a new trend for network topology design standards — creating a fast, predictable, scalable, and efficient communication architecture in a data center environment. This article provides an introduction to Leaf-Spine network topology.
With the increased focus on massive data transfers and instantaneous data travel in the network, the aging three-tier design within a data center is being replaced with what is being called the Leaf-Spine design. Leaf-Spine architecture is adaptable to the continuously changing needs of companies in big data industries with evolving data centers.
The Other Model
A traditional three-tiered model was designed for use in general networks, usually segmented into pods which constrained the location of devices such as virtual servers. The architecture consists of Core routers, Aggregation (sometimes called Distribution) routers, and Access switches. These devices are interconnected by pathways for redundancy which can create loops in the network. As part of the design, a protocol (Spanning Tree) that prevents looped paths is implemented. However, doing so deactivates all but the primary route. A backup path is then only brought up and utilized when the active path experiences an outage.
Leaf-Spine Network Topology Introduction
With Leaf-Spine configurations, all devices are exactly the same number of segments away and contain a predictable and consistent amount of delay or latency for traveling information. This is possible because of the new topology design that has only two layers, the Leaf layer and Spine layer. The Leaf layer consists of access switches that connect to devices like servers, firewalls, load balancers, and edge routers. The Spine layer (made up of switches that perform routing) is the backbone of the network, where every Leaf switch is interconnected with each and every Spine switch.
To allow for the predictable distance between devices in this two-layered design, dynamic Layer 3 routing is used to interconnect the layers. Dynamic routing allows the best path to be determined and adjusted based on responses to network change. This type of network is for data center architectures with a focus on “East-West” network traffic. “East-West” traffic contains data designed to travel inside the data center itself and not outside to a different site or network. This new approach is a solution to the intrinsic limitations of Spanning Tree with the ability to utilize other networking protocols and methodologies to achieve a dynamic network.
Advantages of Leaf-Spine
With Leaf-Spine, the network uses Layer 3 routing. All routes are configured in an active state through the use of Equal-Cost Multipathing (ECMP). This allows all connections to be utilized at the same time while still remaining stable and avoiding loops within the network. With traditional Layer 2 switching protocols like Spanning Tree on three-tiered networks, it must be configured on all devices correctly and all of the assumptions that Spanning Tree Protocol (STP) relies on must be taken into account (one of the easy mistakes to make when configuring STP is with mislabeling device priorities which could lead to an inefficient path setup). The removal of STP between the Access and Aggregation layers in lieu of Layer 3 routing results in a much more stable environment.
Another advantage is the ease of adding additional hardware and capacity. When oversubscription of links occurs (meaning that more traffic is generated than can be aggregated onto the active link at one time), the ability to expand capacity is straightforward. An additional Spine switch may be added and uplinks may be extended to every Leaf switch, resulting in the addition of interlayer bandwidth and reduction of the oversubscription. When device port capacity becomes an issue, a new Leaf switch can be added by connecting it to every spine and adding the network configuration to the switch. The ease of expansion optimizes the IT department’s process of scaling the network without managing or disrupting the Layer 2 switching protocols.
There are some concerns around utilizing the Leaf-Spine network architecture. The first comes from the sheer amount of cable needed to connect each Spine with every Leaf. The cable glut will only worsen in time as new Leaf and Spine switches are added to expand capacity. Considerations should be given for where to strategically locate the Spine switches within a data center, especially for large deployments. This is to ensure cabling is planned, organized, and manageable when scaling out capacity as the network grows.
The other major disadvantage comes from the use of Layer 3 routing. This eliminates the spanning of VLANs (Virtual LAN) across a network. VLANs in a Leaf-Spine network are localized to each individual Leaf switch; any VLAN segments that are left on a Leaf switch are not accessible by the other Leafs. This can create issues with a scenario such as guest virtual machine mobility within a data center.
Use Cases for Leaf-Spine
Web scale applications where server location within the network is static would benefit from the implementation of Leaf-Spine. The use of Layer 3 routing between layers of the architecture does not hinder web scale applications because they do not require server mobility. The removal of Spanning Tree Protocol (STP) results in a more stable and reliable network performance of East-West traffic flows. Scalability of the architecture is also improved.
Enterprise applications leveraging mobile virtual machines (e.g. vMotion) create an issue when a server needs to be supportable anywhere within the data center. The use of Layer 3 routing and lack of VLANs extending between Leafs breaks this requirement. To work around this issue, a solution such as Software Defined Networking (SDN) can be employed, which creates a virtual Layer 2 above/on top of the Leaf-Spine network. This allows servers to move around within the environment with impunity at no detriment to “East-West” performance, scalability, and stability attributes of a Leaf-Spine network topology. More details around SDN will be discussed in a future blog article.
As an alternative to Layer 3 routing, a Leaf-and-Spine topology can leverage other protocols such as Transparent Interconnection of Lots of Links (TRILL) or Shortest Path Bridging (SPB) to accomplish similar functionality. This is done by reducing the use of Spanning Tree and enabling Layer 2 ECMP while also supporting the spanning of VLANs between Leaf switches. More about this will be discussed in future blog posts.
Leaf-Spine networks offer many unique benefits over the traditional 3-tier model. The use of Layer 3 routing with Equal Cost Multipathing (ECMP) improves the total available bandwidth by utilizing all available uplinks. With easily adaptable configurations and design, Leaf-Spine has improved the IT department’s management of oversubscription and scalability. Eliminating the Spanning Tree Protocol (STP) has led to drastically improved network stability. Utilizing new tools and the ability to overcome inherent limitations with other solutions such as SDN, Leaf-Spine environments allow IT departments and data centers to thrive while accomplishing all needs and wants of the business.