Cisco ACI upgrade considerations

Hardware evolutions

For those who follow the Cisco ACI development lifecycle, you'll know that a lot has changed since companies first implemented the platform. The leaf and spine hardware has been through generational evolutions that include bandwidth increases from 1/10-Gbps for front-facing server ports to 10/25/100-Gbps, whilst fabric uplink bandwidth increases from 40-Gbps to 100/400-Gbps (with 800Gbps on the horizon), vast improvements in scalability offering customers more capacity and features whilst removing some of the initial ASIC limitations in Generation 1 hardware. Switches can now have dual roles, meaning that some switches can be either a Spine or a Leaf, offering customers greater flexibility in their deployment options and choices.

While we cannot escape the hardware requirement for a Cisco ACI fabric's Leaf and Spine components, we can now migrate the controllers (APICs) to virtualised components. Why would we call this out? Because these are the conversations we have been having surrounding the APIC-SERVER-M2/L2 hardware, which has an EOL date that has now passed.

Yes, you heard us right, it passed. As of June 30th, 2024, your APIC-SERVER-M2/L2 appliances are now out of support, meaning you should take immediate action to check your estate and remediate. The EOL notification shows the recommended hardware upgrade option to the APIC-SERVER-M4/L4. Currently, these appliances are on a 20-day lead time for shipment and delivery; if this is too long, the other option is to explore the virtualised APIC solution, which does have software and architectural considerations to also take into account.

In the example of the APICs, we appreciate that a broader company preference often determines the hardware/virtual appliance conversation but we now have an extra tool in the toolbox when considering the options.

Navigating the challenges and benefits of software upgrades

Software version selection for upgrades can mean many things to different people, and the thought process of OS selection for traditional network and security infrastructures often fails to translate to Cisco ACI. This is mainly because of the interdependencies with other platforms.

However, the benefits of software upgrades are still relevant: new versions offer new features, security enhancements, and defect resolutions to give you the confidence that your environment is stable and protected. New software versions can bring significant enhancements, such as new API capabilities and native integrations with other products.

Although this can be very useful and help streamline workflows, we can quickly build a spider's web of software inter-dependencies across the infrastructure when integrating the network, the server and security controls together. These inter-dependencies can provide additional challenges when it comes to Cisco ACI software upgrades. These integrations may result in stepped upgrade paths for the inter-connected components to stay within the vendor-validated compatibility matrix. Irrespective of the dependencies, we advocate using native integrations where possible for the ease of end-to-end simplicity and provisioning across VMWare DVS, Nuntanix, Hyper-V, F5, and so on. The extra effort required to track interoperability often outweighs the effort required to implement siloed, non-integrated processes.

Cumbersome stepped upgrades can also apply to Cisco ACI if you have not upgraded your fabric since the initial deployment.

The stepped upgrade has a subsequent knock-on effect. The broader integrated eco-system will also need to be stepped, which is another article in itself. However, Cisco provides helpful tools to navigate this challenging landscape and decision process. The first is the Cisco ACI upgrade matrix <link>, the second is the Cisco ACI and VMware compatibility matrix, and the third is the hardware compatibility matrix <link>, which we will cover shortly.

As you may have anticipated, there is an overlap between hardware and software progression; new hardware support, such as APIC-SERVER-M4/L4, is naturally not programmed into older software versions. Why would it be? The APIC-SERVER-M4/L4 did not exist when the software was released. Also, support for older hardware may no longer be maintained in new software versions. Therefore, we have to be aware of the linkage and ensure that we recognise that there could be a situation where the software upgrade path is compatible, but the hardware support is not, meaning that you still have to follow a stepped upgrade path whilst also considering the hardware migration approach.

For one customer, we had this exact issue. There was no end-to-end vendor support for the full hardware and software upgrade cycle. We were able to engineer a solution, but the upgrade required much more time and effort than initially anticipated.

When selecting software versions, the final thing to consider is the behavioural aspects of newer software versions and the features you are currently running. The rate of change within IT today means that software evolves quickly; things need to improve, often incorporating fixes for software defects and security vulnerabilities whilst also making efficiency gains within the software. Just because it works for you doesn't mean that that specific feature and its functionality have not affected 50 other customers, and you have just been lucky.

Deployment complexity. Or is it?

Cisco ACI deployment types have been standard for a long time, driven by the controller deployment model: Standalone Fabric, Multi-Pod fabric, and Multi-Site Fabric. Whilst we will not cover these from a configuration perspective due to their architectural and component differences, they are a consideration when discussing software and hardware upgrades.

Cisco ACI Multi-Pod is a stretched topology, which means a single APIC cluster stretched across multiple locations. When it comes to software and hardware upgrades, people often overlook a few areas. Focusing on the topic of the end-of-life APIC-SERVER-M2/L2 appliances, you can immediately see that physically replacing your APIC cluster with the APIC-SERVER-M4/L4 appliances introduces resource complexity; you need to be in at least two places simultaneously. Now, introduce the need to replace spine or leaf hardware; you now have the consideration of potentially breaking your APIC cluster connectivity, entering a minority state, and depending on the generation, you may find that you have software dependencies that will oppose your switch replacement against your APIC replacement and some tough decisions will need to be made.

On the other hand, Cisco ACI Multi-Site offers some of the benefits of standalone fabrics when replacing end-of-life APIC-SERVER-M2/L2 or Spine/Leaf hardware. Still, it may present different challenges from a software and Multi-Site Controller perspective. On the controller front, the fundamental change is the platform move from Multi-Site Orchestrator (MSO) to Nexus Dashboard Orchestrator (NDO), which requires a few architectural design and feature considerations, as deploying NDO requires the deployment Nexus Dashboard, which is a unified platform that combines operational and management simplicity into a single solution, introducing additional consumption options such as Nexus Dashboard Insights (NDI) which can enable you to pre-empt unintentional errors, monitor and analyse your Cisco ACI fabrics. Moving to NDO from MSO, when done right, should be a simple transition, but as with everything new, there are considerations with how some of the newer NDO versions drive tenant policy to be built and deployed.

Next steps