A Beginner’s Guide to Understanding the Leaf-Spine Network Topology

There is a new trend for network topology design standards — creating a fast, predictable, scalable, and efficient communication architecture in a data center environment. This article provides an introduction to Leaf-Spine network topology.

Why Leaf-Spine?

With the increased focus on massive data transfers and instantaneous data travel in the network, the aging three-tier design within a data center is being replaced with what is being called the Leaf-Spine design. Leaf-Spine architecture is adaptable to the continuously changing needs of companies in big data industries with evolving data centers.

The Other Model

A traditional three-tiered model was designed for use in general networks, usually segmented into pods which constrained the location of devices such as virtual servers. The architecture consists of Core routers, Aggregation (sometimes called Distribution) routers, and Access switches. These devices are interconnected by pathways for redundancy which can create loops in the network. As part of the design, a protocol (Spanning Tree) that prevents looped paths is implemented. However, doing so deactivates all but the primary route. A backup path is then only brought up and utilized when the active path experiences an outage.

3-Tier leaf spine network design

Figure 1: Traditional three-tier network design

Leaf-Spine Network Topology Introduction

With Leaf-Spine configurations, all devices are exactly the same number of segments away and contain a predictable and consistent amount of delay or latency for traveling information. This is possible because of the new topology design that has only two layers, the Leaf layer and Spine layer. The Leaf layer consists of access switches that connect to devices like servers, firewalls, load balancers, and edge routers. The Spine layer (made up of switches that perform routing) is the backbone of the network, where every Leaf switch is interconnected with each and every Spine switch.

To allow for the predictable distance between devices in this two-layered design, dynamic Layer 3 routing is used to interconnect the layers. Dynamic routing allows the best path to be determined and adjusted based on responses to network change. This type of network is for data center architectures with a focus on “East-West” network traffic. “East-West” traffic contains data designed to travel inside the data center itself and not outside to a different site or network.  This new approach is a solution to the intrinsic limitations of Spanning Tree with the ability to utilize other networking protocols and methodologies to achieve a dynamic network.

Figure 2: Leaf-Spine architecture design

Figure 2: Leaf-Spine architecture design

Advantages of Leaf-Spine

With Leaf-Spine, the network uses Layer 3 routing. All routes are configured in an active state through the use of Equal-Cost Multipathing (ECMP). This allows all connections to be utilized at the same time while still remaining stable and avoiding loops within the network. With traditional Layer 2 switching protocols like Spanning Tree on three-tiered networks, it must be configured on all devices correctly and all of the assumptions that Spanning Tree Protocol (STP) relies on must be taken into account (one of the easy mistakes to make when configuring STP is with mislabeling device priorities which could lead to an inefficient path setup). The removal of STP between the Access and Aggregation layers in lieu of Layer 3 routing results in a much more stable environment.

Another advantage is the ease of adding additional hardware and capacity. When oversubscription of links occurs (meaning that more traffic is generated than can be aggregated onto the active link at one time), the ability to expand capacity is straightforward. An additional Spine switch may be added and uplinks may be extended to every Leaf switch, resulting in the addition of interlayer bandwidth and reduction of the oversubscription. When device port capacity becomes an issue, a new Leaf switch can be added by connecting it to every spine and adding the network configuration to the switch. The ease of expansion optimizes the IT department’s process of scaling the network without managing or disrupting the Layer 2 switching protocols.

Leaf-Spine Concerns

There are some concerns around utilizing the Leaf-Spine network architecture. The first comes from the sheer amount of cable needed to connect each Spine with every Leaf. The cable glut will only worsen in time as new Leaf and Spine switches are added to expand capacity. Considerations should be given for where to strategically locate the Spine switches within a data center, especially for large deployments. This is to ensure cabling is planned, organized, and manageable when scaling out capacity as the network grows.

The other major disadvantage comes from the use of Layer 3 routing. This eliminates the spanning of VLANs (Virtual LAN) across a network. VLANs in a Leaf-Spine network are localized to each individual Leaf switch; any VLAN segments that are left on a Leaf switch are not accessible by the other Leafs. This can create issues with a scenario such as guest virtual machine mobility within a data center.

Use Cases for Leaf-Spine

Web scale applications where server location within the network is static would benefit from the implementation of Leaf-Spine. The use of Layer 3 routing between layers of the architecture does not hinder web scale applications because they do not require server mobility. The removal of Spanning Tree Protocol (STP) results in a more stable and reliable network performance of East-West traffic flows. Scalability of the architecture is also improved.

Enterprise applications leveraging mobile virtual machines (e.g. vMotion) create an issue when a server needs to be supportable anywhere within the data center.  The use of Layer 3 routing and lack of VLANs extending between Leafs breaks this requirement.  To work around this issue, a solution such as Software Defined Networking (SDN) can be employed, which creates a virtual Layer 2 above/on top of the Leaf-Spine network.  This allows servers to move around within the environment with impunity at no detriment to “East-West” performance, scalability, and stability attributes of a Leaf-Spine network topology.  More details around SDN will be discussed in a future blog article.

Other Considerations

As an alternative to Layer 3 routing, a Leaf-and-Spine topology can leverage other protocols such as Transparent Interconnection of Lots of Links (TRILL) or Shortest Path Bridging (SPB) to accomplish similar functionality.  This is done by reducing the use of Spanning Tree and enabling Layer 2 ECMP while also supporting the spanning of VLANs between Leaf switches.  More about this will be discussed in future blog posts.

Summary

Leaf-Spine networks offer many unique benefits over the traditional 3-tier model. The use of Layer 3 routing with Equal Cost Multipathing (ECMP) improves the total available bandwidth by utilizing all available uplinks. With easily adaptable configurations and design, Leaf-Spine has improved the IT department’s management of oversubscription and scalability. Eliminating the Spanning Tree Protocol (STP) has led to drastically improved network stability. Utilizing new tools and the ability to overcome inherent limitations with other solutions such as SDN, Leaf-Spine environments allow IT departments and data centers to thrive while accomplishing all needs and wants of the business.

5 Comments

  • Dan March 23, 2015 12:00 pm

    This is a great introduction to Leaf-Spine. You have covered a ton of material clearly and concisely. I especially appreciate the “Use Cases” section and look forward to read more on Software Defined Networking. Thank you for this great resource!

  • Matthias July 18, 2016 6:11 am

    Hi Trevor,

    thanks for your good introduction and the pros and cons of Leaf-Spine.

    There is still one question, when reading your introduction.
    I do not understand the disadvantage when using Layer3 Routing with VLANs on the Leaf Switches.

    We will start with two Spine Switches (Layer3 gateways of the local networks within the VLANs and VRRP) and a lot of Leaf Switches with the VLAN switchports for the access nodes (managed with VTP).
    Where is the disadvantage in this setup when using this design model?

    Have a nice day
    Matthias

  • Trevor Husseman August 1, 2016 11:54 am

    Hello Matthias,
    Having Layer 3 routing is not an inherent disadvantage of the Leaf-Spine network topology. By utilizing Layer 3 routing, Leaf switches can go beyond their typical workload and employ dynamic routing protocols. This allows the network to instantaneously determine the most efficient route for packets.
    Thanks for your question!
    Trevor Husseman

  • Noemi Berry October 7, 2016 3:20 pm

    How are dual-homed servers physically attached and logically configured in leaf-spine?
    Our Linux servers are dual-homed to separate physical switches, using mode 1 (active/passive) bonding. This requires the upstream
    switching / VLAN and routing layer to be the same for both NICs in the bond. Other devices use link aggregation, including some se
    rvers in mode 4 (active/active with LACP) bonding. Other methods depend on the same VLAN/routing upstream on separate switches, su
    ch as HA-paired firewalls.

    How are dual-homed servers, LAGs and other HA methods handled in leaf-spine?

    • Sumedh February 25, 2017 7:32 pm

      For simple dual-homed, or anytime you have >2 interfaces bonded (active-standby or active-active – doesn’t matter) like you are describing, you need 2x the number of leaf switches. If you use a leafs with 2x the number of ports, then you don’t achieve failover between the leafs – which I imagine is one of the objectives of dual-homing and/or bonding. Each pair of leafs should be connected to each other via VLAN trunking or MLAG, so that when one leaf fails, the other one will take over (at 1/2 the capacity, but the the server will at least be available and reachable). The 2x the normal number of leafs will need to be connected to spines with 2x the number of ports on the spines (but the number of spines must the same as before). Capacity (bandwidth) planning at the spine will need to be 2x than normal, unbonded methods. There are many trade-offs to to consider when you want to combine dual homing, bonding, and leaf/spine.

Your email address will not be published. Required fields are marked *

Phone: 312-602-4000
Email: marketing@westmonroepartners.com
222 W. Adams
Chicago, IL 60606
Show Buttons
Share On Facebook
Share On Twitter
Share on LinkedIn
Hide Buttons