Lately I have been spending some time with LAB-testing new Cisco ACI environment (more about ACI in further posts). As a multi-DC service provider DC-interconnect is of utmost importance. I have implemented some Nexus 7000/5000 environments using vPC DC-interconnect (dark fiber, CWDM) which have worked quite nicely. Especially if there are requirements of MACSEC (802.1AE) and such, N7K M-series linecards are quite nice with 10G line rate MACSEC.
However there has been some cases where direct L2-interconnect is not possible. Especially in this ACI-case if you want to stretch the fabric (one fabric) to other DC you need 8 x 40G (!) in between, which is quite a lot. You could do the ACI interconnect (two separate ACI fabrics) from Leaf to Leaf with N x 10G, but still the L2 over L3 brings some advantages. Therefore I have been looking into different technologies for DCI. Lately I have lab-tested some PBB-EVPN (Provider Backbone Bridging – Ethernet VPN) DCI with Cisco ASR’s, and it works quite nicely I must say. In the lab tests I used 1 x 10G towards DC’s in both ends and 2 x 10G bundled between the ASR’s simulating the EVPN/MPLS in between. As there is no “real” MPLS in between, this simply can be thought of as two directly connected PE-routers. There are two extended VLAN’s in this example; 503 and 751.