《ACI 多站点架构和部署.pdf》由会员分享,可在线阅读,更多相关《ACI 多站点架构和部署.pdf(77页珍藏版)》请在三个皮匠报告上搜索。
1、#CiscoLive#CiscoLiveMax Ardica,Distinguished EngineermaxardicaBRKDCN-2980ACI Multi-Site Architecture and Deployment 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveEnter your personal notes hereCisco Webex App Questions?Use Cisco Webex App to chat with the speaker after th
2、e sessionFind this session in the Cisco Live Mobile AppClick“Join the Discussion”Install the Webex App or go directly to the Webex spaceEnter messages/questions in the Webex spaceHowWebex spaces will be moderated by the speaker until June 9,2023.12343https:/ 2023 Cisco and/or its affiliates.All righ
3、ts reserved.Cisco PublicBRKDCN-29803 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveSession Objectives At the end of the session,the participants should be able to:Articulate the different deployment options to interconnect Cisco ACI networks(Multi-Pod and Multi-Site)and
4、when to choose one vs.the otherUnderstand the functionalities and specific design considerations associated to the ACI Multi-Site architectureInitial assumption:The audience already has a good knowledge of ACI main concepts(Tenant,BD,EPG,L2Out,L3Out,etc.)BRKDCN-29804Agenda 2023 Cisco and/or its affi
5、liates.All rights reserved.Cisco PublicIntroductionInter-Site Connectivity Deployment ConsiderationsNexus Dashboard Orchestrator(NDO)ACI Multi-Site Control and Data PlaneProvisioning Policies on NDOConnecting to the External L3 DomainNetwork Services Integration(Stretch Goal)BRKDCN-29805Introduction
6、 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveACI Architectural OptionsFabric and Policy Domain EvolutionSingle Fabric,Single Controller DomainACI Single Pod Fabric 1Pod AMP-BGP-EVPNIPNPod nACI Multi-Pod FabricAPIC Cluster2IPNACI Remote LeafRemote Leaf Location3Multiple
7、 Fabrics,Multiple Controller DomainsFabric AMP-BGP-EVPNISNFabric nACI Multi-Site4ACI Hybrid Cloud/Multi-Cloud5BRKDCN-29807 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveACI Multi-SiteThe Ideal Architecture for“Loosely Coupled”DCs17BRKDCN-2980Separate ACI Fabrics with ind
8、ependent APIC clustersNo latency limitation between FabricsACI Multi-Site Orchestrator pushes cross-fabric configuration to multiple APIC clusters providing scoping of all configuration changesMP-BGP EVPN control-plane between sitesData-Plane VXLAN encapsulation across sitesEnd-to-end policy definit
9、ion and enforcementRegion 1Region 2Inter-Site NetworkSite 1Site 2RESTAPI GUINexus Dashboard OrchestratorL3L3214455MP-BGP-EVPNVXLAN33Want to know how to provision Multi-Pod and Multi-Site from scratch?Come to BRKDCN-2919(Wed 10.30 am)2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#C
10、iscoLiveACI Multi-SiteNDO Provisioning Configuration for“Autonomous Sites”18BRKDCN-2980If the fabrics are operated as independent(“autonomous”)sites,NDO could still be used as a single point of provisioningNo use of ISN and VXLAN EVPN for east-west communicationLayer 3 communication still possible v
11、ia the L3Out data path NDO can be used to“replicate”configuration across sites by associating the same“autonomous template”to those fabricsRegion 1Region 2Site 1Site 2RESTAPI GUINexus Dashboard OrchestratorL3L3Cisco Nexus Dashboard Orchestrator 4.0(1)2023 Cisco and/or its affiliates.All rights reser
12、ved.Cisco Public#CiscoLiveData Center Interconnect(DCI)Extend connectivity/policy between loosely coupled DC sitesDisaster Recovery and IP mobility use casesCompartmentation/ScaleBuilding Multiple Fabrics inside a single Data CenterOptimized and controlled L2/L3 connectivity(including optimized/cont
13、rolled BUM forwarding),scale out total number of leaf nodes(SP use case)ACI Multi-Site ArchitectureMost Common Use CasesHybrid-Cloud and Multi-CloudIntegration between on-prem and public clouds(AWS,Azure,GCP)Nexus Dashboard OrchestratorSP 5G Telco DC/Cloud*Centralized DC Orchestration for“Autonomous
14、 Fabrics”Optional SR-MPLS/MPLS Handoff on Border Leaf nodesNexus Dashboard Orchestrator*May also apply to Enterprise deploymentsBRKDCN-298019Inter-Site Connectivity Deployment Considerations 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveNot managed by APIC or NDO,must be
15、 independently configured(day-0 configuration)IP topology can be arbitrary,not mandatory to connect all the spine nodes to the ISNISN main functional requirements:OSPF/BGP*to peer with the spine nodes and exchange TEP address reachabilityMust use sub-interfaces(with VLAN tag 4)toward the spinesNo mu
16、lticast requirement for BUM traffic forwarding across sitesIncreased end-to-end MTU support(at least 50/54 extra Bytes)Inter-Site NetworkMP-BGP-EVPNNexus Dashboard OrchestratorSub-interfaces(VLAN tag 4)*Requires ACI 5.2(1)and NDO 3.5(1)21Inter-Site Network(ISN)Functional RequirementsBRKDCN-2980 2023
17、 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveISN1.Data-Plane MTU:MTU of the traffic generate by endpoints(servers,routers,service nodes,etc.)connected to ACI leaf nodesNeed to account for 50B of overhead(VXLAN encapsulation)for inter-site communication2.Control-Plane MTU:fo
18、r CPU generated traffic like MP-BGP sessions across sitesControl plane traffic is not VXLAN encapsulatedThe default value is 9000B,9000B,can be tuned on APIC to match the maximum MTU value supported in the ISNMPMP-BGPBGP2111Nexus Dashboard OrchestratorACI Multi-Site and MTU SizeDifferent MTU Meaning
19、s22BRKDCN-2980 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveControl-Plane MTU can be set leveraging the“Control Plane MTU Policy”on APICThe setting applies to all the control-plane traffic generated by ACI leaf/spine nodesThe required MTU in the ISN would hence depend o
20、n this setting and on the MTU of the traffic generated by endpoints/devices connected to the fabricAlways need to consider the VXLAN encapsulation overhead for data plane trafficModify the default 9000B MTU value(if needed)ISNConfigurable MTUMPMP-BGPBGPMulti-Site OrchestratorACI Multi-Site and MTU S
21、izeTuning MTU Size for EVPN Control-Plane Traffic23BRKDCN-2980Nexus Dashboard Orchestrator(NDO)2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveIP networkMulti-Site Orchestrator Cluster.Site 1Site 2Site nESXi HypervisorESXi HypervisorESXi HypervisorMSO Node1MSO Node2MSO Nod
22、e3Supported from the beginning(MSO release 1.0(1)Each Cisco Multi-Site Orchestrator node is packaged in a VMware vSphere virtual appliance(OVA)For high availability,you should deploy each Cisco Multi-Site Orchestrator virtual machine on its own VMware ESXi hostRequirements for MSO Release 1.2(x)and
23、above:VMware ESXi 6.0 or laterMinimum of eight virtual CPUs(vCPUs),48 Gbps of memory,and 100 GB of disk spaceMSO 3.1(1)last supported release with this form factor,MSO 3.1(1)last supported release with this form factor,now EoL/EoSnow EoL/EoSOriginal Multi-Site Orchestrator OptionVM Based MSO Cluster
24、(OVA),Now EoL/EoSBRKDCN-298027 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCisco Multi-SiteOrchestratorUp to release 3.1(1)Cisco Nexus Dashboard OrchestratorFrom release 3.2(1)Cisco Multi-Site Orchestrator has become Cisco Nexus Dashboard OrchestratorBRKDCN-298028 2023
25、 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCustom/third-partyPublic cloudPrivate cloudCisco Nexus DashboardData BrokerOrchestratorInsightsFabric DiscoveryFabric ControllerSAN ControllerPowering automationUnified agile platformCisco Nexus DashboardCisco Nexus DashboardSimp
26、le to Automate,Simple to ConsumeBRKDCN-298029 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveCisco Nexus DashboardDeployment EvolutionShippingPhysical Cisco ND Platform ClusterShippingVirtual/Cloud Cisco ND Platform Cluster ND cloud cluster supported for AWS and AzureND v
27、irtual cluster supported on ESXi and KVM hypervisorsSpec:16 vCPUs,64Gb ram and 500Gb diskBRKDCN-298030 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveNoteThis procedure is only supported for SW upgrades but not for SW downgradesOriginal MSO ClusterCisco Nexus Dashboard Or
28、chestrator 4.1(2)3 3Rollback config to the backup file(includes also a”DB cleanup”procedure)1 1Export backupBackup file2 2Import backuphttps:/ 4Take care of Templates with drifts(if any)Migrating the MSO Cluster to Cisco NDO32BRKDCN-2980All MSO releases are officially End-of-Life(EOL)Customer should
29、(and must)migrate from MSO to NDONDO 4.1(2)release is the recommended target release for this migrationMigration Procedure 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveRecommended Releases per ScenarioRecommended Releases per ScenarioCurrent ReleaseCurrent ReleaseTarget
30、 ReleaseTarget ReleaseMSO/NDO 1.1.x to 3.7(2)*NDO 4.1(2)None-GreenfieldNDO 4.1(2)Nexus Dashboard OrchestratorWhat NDO Release to Choose?*NDO 4.1(2)supports direct GUI upgrade from NDO 3.x releases12BRKDCN-298033 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveNDO 4.0(x)has
31、 gone through a series of design and implementation changes aiming to improve the performance and scalability figures1.Execution Service Manages Deployments to Sites(APIC)2.Schema Service Manages Schemas and Templates3.pcTagVnid Service Manages Policy Translation between sites4.Added a new service f
32、or notification handling notification Engine(for 100 sites scale)Why Recommending to Adopt NDO 4.x?NDO 4.x Design and Implementation ChangesBRKDCN-298034ACI Multi-Site Control-and Data-Plane 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveInter-Site NetworkVTEP IPVNIDTenan
33、t PacketMP-BGP-EVPNNo Multicast Requirement in Backbone,Head-End Replication(HER)for any Layer 2 BUM traffic)Class-IDIdentity information carried across Fabrics(Availability Zones)Network information carried across Fabrics(Availability Zones)Nexus Dashboard OrchestratorACI Multi-SiteNetwork and Iden
34、tity Extended between Fabrics36BRKDCN-2980 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveSite 1Site 2EP1ISNEP2Inter-Site policies defined on the ACI Nexus Dashboard Orchestrator are pushed to the respective APIC domainsEnd-to-end policy consistencyCreation of Shadow obje
35、cts to locally recreate the policies in each APIC domainInter-site communication requires the installation of translation table entries on the spines(namespace normalization)Translation entries are populated in different cases:Stretched EPGs/BDsCreation of a contract between site-local(not stretched
36、 EPGs)Preferred Group or vzAny deploymentsEP1 EPGEP2 EPGCShadow EPGsVRF VNID:16678781BD VNID:13543235Class-ID:49153VRF VNID:16678781BD VNID:15434518Class-ID:31564EP1 EPGEP2 EPGCShadow EPGsVRF VNID:15434256BD VNID:13762843Class-ID:32770VRF VNID:15434256BD VNID:12753426Class-ID:36784VNID 16678781Class
37、-ID:49153VNID 15434256Class-ID:32770Site 2 Spines Translation TableACI Multi-SiteInter-Site Policies and Spines Translation Tables37BRKDCN-2980 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveWebAppDBMulti-Site Preferred GroupFree communicationContract required to communic
38、ate with EPG(s)external to the Preferred Group”VRF unenforced”not supported with Multi-SiteMulti-Site Preferred Group configuration can be provision directly from NDOCreates shadow EPGs and translation table entries under the hood to allow free inter-site communication5000 total EPGs part of preferr
39、ed group supported in NDO 4.x releaseTypically desired in legacy to ACI migration scenariosC1C2Non-PG EPGACI Multi-SiteSimplify Policy Enforcement:Preferred Groups38BRKDCN-2980 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveSite 1Site 2EP1EP2L3Out Site 1L3Out Site 2Ext-EP
40、GExt-EPGAdding internal EPGs and External EPGs(associated to L3Outs)to the Preferred Group allows to enable free east-west and north-south connectivityWhen adding the Ext-EPG to the Preferred Group:Cant use 0.0.0.0/0 for classification,needs more specific prefixes As workaround it is possible to use
41、 0.0.0.0/1 and 128.0.0.0/1 to achieve the same resultMust ensure Ext-EPG is a stretched objectIntersite L3Out not supported if the Ext-EPG is part of a Preferred GroupInter SiteNetworkMulti-Site Preferred GroupOn NDOEPG1EPG2Ext-EPGSimplify Policy EnforcementPreferred Groups for E-W and N-S Flows39BR
42、KDCN-2980 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveWhat is vzAny?Logical object representing all the EPGs in a VRFSimplify Policy EnforcementvzAny Support40BRKDCN-2980Permit-AnyEPG1vzAny(VRF1)EPG2Ext-EPGUse case 2:Enable free communication inside a VRFEPG1vzAny(VRF1
43、)EPG2Ext-EPGPCC1vzAny provides and consumes a contract with an associated“Permit-any”filterUse ACI fabric only for network connectivity without policy enforcementEquivalent to“VRF unenforced”No current No current ServiceService-Graph Graph support in NDO*support in NDO*Multiple EPGs part of a specif
44、ic VRF1 consume the services provided by a shared EPG(part of VRF1 or of a VRF-shared)VRF-shared can be part of the same tenant or of a different tenantUse case 1:Many-to-One communication(Shared Services)EPG1vzAny(VRF1)C1Shared EPGEPG2VRF1 or VRF-SharedEPG3CPNo current No current ServiceService-Gra
45、ph Graph support in NDO*support in NDO*Permit-Any*Committed for NDO 4.2(1)release 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveInter SiteNetworkSite1Site2EPG1Shared-EPGEPG3Proper translation entries are created on the spines of both fabrics to enable east-west communica
46、tionSupported also for Shared Services behind an L3OutL3Out-Site1Ext-EPGShared-ResourceL3Out-Site2Ext-EPGACI Multi-Site and vzAnyMany-to One Communication(Shared Services)41BRKDCN-2980 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveInter SiteNetworkSite1Site2EPG1EPG2Prope
47、r translation entries are created on the spines of both fabrics to enable east-west communicationSupported also for connecting to the external Layer 3 domainvzAny+PBR support for any-to-any communication planned for a future NDO releaseL3Out-Site1L3Out-Site2Ext-EPGExt-EPGACI Multi-Site and vzAnyEnab
48、le Inter-Site Free Communication Inside a VRF42BRKDCN-2980Underlay and Overlay Control-Plane Considerations 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveACI Multi-SiteBGP Inter-Site Peers49BRKDCN-2980Inter-Site NetworkEVPN-RID 1EVPN-RID 2EVPN-RID 3EVPN-RID 4Anycast VTEP
49、 Addresses:O-UTEP&O-MTEPSpines connected to the Inter-Site Network perform two main functions:1.Establishment of MP-BGP EVPN peerings with spines in remote sites One dedicated Control-Plane address(EVPN-RID)is assigned to each spine running MP-BGP EVPN 2.Forwarding of inter-sites data-plane traffic
50、Anycast Overlay Unicast TEP(O-UTEP):assigned to all the spines connected to the ISN and used to source and receive L2/L3 unicast traffic Anycast Overlay Multicast TEP(O-MTEP):assigned to all the spines connected to the ISN and used to receive L2 BUM trafficEVPN-RID,O-UTEP and O-MTEP addresses are as
51、signed from the Nexus Dashboard Orchestrator and must be routable across the ISN 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveACI Multi-SiteExchanging TEP Information across Sites50BRKDCN-2980Site 1Site 2OSPF/BGPOSPF/BGPLeaf Routing TableIS-IS to OSPF/BGP mutual redistr
52、ibutionLeaf Routing TableIP PrefixNext-HopO-UTEP A,O-MTEP A,EVPN-RID S1-S4Site2-S5,Site2-S6,Site2-S7,Site2-S8TEP Pool 1TEP Pool 2S1S2S6S7S8S3S5S4IP PrefixNext-HopO-UTEP B,O-MTEP B,EVPN-RID S5-S8Site1-S1,Site1-S2,Site1-S3,Site1-S4Inter-Site NetworkIP Network Routing TableO-UTEP A,O-MTEP AEVPN-RID S1-
53、S4O-UTEP B,O-MTEP BEVPN-RID S5-S8Filter out the advertisement of internal TEP pools into the ISNNexus Dashboard OrchestratorOSPF or BGP peering between spines and Inter-Site network Mandates the use of L3 sub-interfaces(with VLAN 4 tag)between the spines and the ISNExchange of External Spine TEP add
54、resses(EVPN-RID,O-UTEP and O-MTEP)across sites Internal TEP Pool information not needed to establish inter-site communication(should be filtered out on the first-hop ISN router)Use of overlapping internal TEP Pools across sites possible and fully supported 2023 Cisco and/or its affiliates.All rights
55、 reserved.Cisco Public#CiscoLiveACI Multi-SiteInter-Site MP-BGP EVPN Control Plane51BRKDCN-2980Site 1Site 2MP-BGP EVPNEP1Leaf 1EP1O-UTEP ACOOPEP1EP2O-UTEP BEP2Leaf 4EP1 EPGEP2 EPGCEP2COOPS3-S4 TableS5-S8 TableS1S2S4S3S6S7S8S5O-UTEP AO-UTEP BDefine and push inter-site policyInter-Site NetworkNexus Da
56、shboard OrchestratorMP-BGP EVPN used to communicate Endpoint(EP)information across Sites MP-iBGP or MP-EBGP peering options supported Required MP-BGP configuration fully automated via NDO Remote host route entries(EVPN Type-2)are associated to the remote site Anycast O-UTEP addressAutomatic filterin
57、g of endpoint information across Sites Host routes are exchanged across sites onlyonly if there is a cross-site contract requiring communication between endpointsData-Plane Communication across Sites 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveEP2EP1*BUM frame is assoc
58、iated to GIPo1 and flooded intra-site along the corresponding FTAG tree2EP1 generates a BUM frame 14S7 translates the VNID and the GIPo values to locally significant ones and associates the frame to an FTAG treeS1S2S3S4S5S6S7S86EP2 receives the BUM frameGIPo1=Multicast Group associated to EP1s BDInt
59、er-Site NetworkO-MTEP BO-UTEP A53S3 is elected as Multi-Site forwarder for GIPo 1 BUM traffic it creates a unicast VXLAN packet with O-UTEP A as S_VTEP and Multicast O-MTEP B as D_VTEP3ACI Multi-SiteInter-Site Layer 2 BUM*ForwardingNexus Dashboard OrchestratorBRKDCN-2980Inter-Site BUM traffic source
60、d from O-UTEP A and destined to O-MTEP B 5BUM frame is flooded along the tree associated to GIPo.VTEP learns VM1 remote locationEP1O-UTEP AProxy B*Site 1Site 2*BUM Broadcast,Unknown Unicast,Multicast 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveBRKDCN-298054EP110.10.10.
61、10Proxy AProxy BEP1 sends traffic to EP21EP2 unknown,traffic is encapsulated to the local Proxy A Spine VTEP(adding S_Class information)Proxy A20.20.20.0/2420.20.20.0/242EP1e1/3=VXLAN Encap/Decap5EP1O-UTEP AProxy B10.10.10.0/2410.10.10.0/24Leaf learns remote Site location info for EP1EP2e1/1Site 1Si
62、te 2S1S2S3S4S5S6S7S8EP1 EPGEP2 EPGCO-UTEP B4EP2S2-L4-TEPEP1O-UTEP AS6 translates the VNID and Class-ID to local values and sends traffic to the local leaf Inter-Site NetworkO-UTEP AVTEP IPVTEP IPClassClass-IDIDTenant PacketTenant PacketVNIDVNIDPolicy information carried across PodsS2 has remote info
63、 for EP2 and encapsulates traffic to remote O-UTEP B Address(also changes src TEP to be O-UTEP A)EP1Leaf 4EP2O-UTEP BVXLAN Inter-Site unicast traffic sourced from O-UTEP A and destined to O-UTEP B 3EP220.20.20.20110.10.10.1020.20.20.2010.10.10.1020.20.20.20S1-L4-TEPProxy-A210.10.10.1020.20.20.20O-UT
64、EP AO-UTEP B310.10.10.1020.20.20.20O-UTEP AS2-L4-TEP410.10.10.1020.20.20.2066If policy allows it,EP2 receives the packetNexus Dashboard OrchestratorACI Multi-SiteInter-Site Unicast Data-Plane(1)2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveBRKDCN-298055*7EP2 sends traffi
65、c back to remote EP1EP1 receives the packet12*EP1O-UTEP AProxy B*Leaf applies the policy and,if allowed,encapsulates traffic to remote O-UTEP address8Site 2Leaf learns remote Site location info for EP2Proxy A11EP2O-UTEP BEP1e1/3S1S2S3S4S5S6S7S8Site 1=VXLAN Encap/DecapInter-Site NetworkVTEP IPClass-I
66、DTenant PacketVNIDPolicy information(EP1s Class-ID)carried across Pods9S6 rewrites the S-VTEP to be O-UTEP BO-UTEP BO-UTEP AEP1 EPGEP2 EPGC1220.20.20.2010.10.10.1020.20.20.2010.10.10.10O-UTEP BS1-L4-TEP1020.20.20.2010.10.10.10O-UTEP BO-UTEP A920.20.20.2010.10.10.10S2-L4-TEPO-UTEP A820.20.20.2010.10.
67、10.107S3 translates the VNID and S_Class to local values and sends traffic to the local leaf EP1S1-L4-TEPEP2O-UTEP A10VXLAN Inter-Site unicast traffic sourced from O-UTEP B and destined to O-UTEP A EP110.10.10.10EP220.20.20.20Nexus Dashboard OrchestratorACI Multi-SiteInter-Site Unicast Data-Plane(2)
68、2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLive*Site 2S1S2S3S4S5S6S7S8From this point EP1 to EP2 communication is encapsulated Leaf to Remote Spine O-UTEPs in both directionsSite 1Inter-Site Network=VXLAN Encap/DecapO-UTEP BO-UTEP AProxy AEP2O-UTEP BEP1e1/3EP1O-UTEP AProx
69、y B*EP2e2/5EP1 EPGEP2 EPGCFlows between O-UTEP A and O-UTEP B(and vice versa)56EP110.10.10.10EP220.20.20.20Nexus Dashboard OrchestratorBRKDCN-2980ACI Multi-SiteInter-Site Unicast Data-Plane(3)Layer 3 Only Communication between Autonomous Sites 2023 Cisco and/or its affiliates.All rights reserved.Cis
70、co Public#CiscoLiveAutonomous deployment mode,NDO used as for”configuration replication”Routing across sites via the WAN backboneWANL3OutL3OutNeed to apply a contract between internal EPG and Ext-EPG associated to the L3Out in Fabric 1Need to apply a contract between Ext-EPG associated to the L3Out
71、in Fabric 2 and internal EPGMandates the use of a multi-VRF capable backbone network(VRF-Lite,MPLS-VPN,etc.)to extend multiple VRFs across fabricsACI Multi-SiteL3 Only across Sites(“Autonomous Sites”)58BRKDCN-2980Provisioning Policies on NDO 2023 Cisco and/or its affiliates.All rights reserved.Cisco
72、 Public#CiscoLiveSupporting Different Types of PoliciesApplication Management PoliciesUsed to define tenant policies(Application Network Profiles,EPGs,BDs,VRFs,etc.)Fabric Management PoliciesUsed to define fabric access policies,interface and monitoring policiesCisco Nexus Dashboard Orchestrator 4.0
73、(1)BRKDCN-298063 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveBenefitsFeaturesGranular roll back of templates specific configuration NDO 3.4(1)Template versioning and rollbackSupport rollback of template from newer to older version-idLabel a template as GoldenBetter vis
74、ibility to reduce errors and seize the impact of a templates deploymentNDO 3.4(1)Template deployment plan visibilityShows preview of what NDO is going to provisioningto each site More structured deployments which enables increased flexibilityNDO 3.4(1)Change control workflowNew personas for manageme
75、nt and provisioning of configurationEase of use for migrationNDO 3.4(1)Detach templatesfrom SitesConfiguration is not removed from the APIC/NDFC domainsSimplify the understanding and reconciliation of config drifts between NDO and APIC/NDFCNDO 3.6(1)Configuration drift reconciliation workflowNDO wor
76、kflow that synchronizes and merges any config changes made in APIC or NDFC domainsProvisioning Policies on NDOWhy do we“Templatize”the Configuration?2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveSite 1SchemaEFFECTIVEPOLICYSite 2EFFECTIVEPOLICYApplication Template=ACI pol
77、icy definition(ANP,EPGs,BDs,VRFs,etc.)Schema=container of Application Templates sharing a common use-caseAs a typical use case,a schema can(and should)be dedicated to a TenantThe template is the atomic unit of change for policies A Multi-Site template associated to a single site can be pushed only t
78、o that siteA Multi-Site template associated to multiple sites is concurrently pushed to all those sitesApplication TemplatesMulti-Site Templatest0t1t1Tenant1Site Local TemplateStretched TemplateTenant1BRKDCN-298067 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveSite 1Site
79、 2EPG1EPG2EPG3EPG4EPG5EPG6BD1BD2BD3BD4BD5BD6C1Site 1 TemplateSchema(dedicated to Tenant1)ANP1ANP1ANP1Best Practices for Multi-Site TemplatesOne Template per Site,plus Two Templates for“Stretched Objects”VRFStretched Template 1(EPGs,BDs,)Site 2 TemplateStretched Template 2(VRFs,contracts,)SGExt-EPGL3
80、Out-S1*L3Out-S2*L3Out defined in a separate“L3Out Template”from NDO 4.1(1)BRKDCN-298068 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveSite 1EFFECTIVEPOLICYSite 2EFFECTIVEPOLICYAutonomous templates can also be associated to one or more fabricsDifferently than for Multi-Si
81、te templates,the deployment of an Autonomous template to different sites wont cause the“stretching”of configuration objects(VRFs,BDs,EPGs,)NDO performs a“configuration replication”function to multiple sitesAutonomous Templates can be deployed to different fabrics at different points in time*Other te
82、mplate types behave as Application Autonomous templatesAutonomous TemplateTenant1Application TemplatesAutonomous Templates*Roadmap feature planned for CY23t0t1SchemaBRKDCN-298069Connecting to the External L3 Domain 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveWANClientL
83、3OutBorder LeafsConnecting to WAN Edge routers from Border Leaf nodesVRF-Lite hand-off for extending L3 multi-tenancy outside the ACI fabric Up to 800 L3Outs/VRFs currently supported on the same BL nodes pairSupport for host routes advertisement out of the ACI Fabric from ACI release 4.0(1)Enabled a
84、t the BD levelSupport for L3 Multicast and Shared L3OutWAN Edge RoutersConnecting to the External Layer 3 DomainTraditional IP-Based L3Outs(Recommended Option)BRKDCN-298073 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveWANClientMPLS tagged trafficBorder LeafsConnecting t
85、o WAN Edge(PE)routers from Border Leaf nodesTypically connected directlySingle control plane session(MP-BGP EVPN)for all tenant VRFsBGP EVPN address family to carry DC prefixes,MPLS label for VRF(VPN label)and color community MPLS tagged traffic between the BL nodes and the WAN Edge routers,one tag
86、per VRFCouple of current limitationsNo support for host-based route advertisementNo support for Layer 3 Multicast communication WAN Edge RoutersNCS5500,NCS540/560 or ASR9K with ACI 5.0(1)releaseFX,FX2,FX3,GX Leaf modelsConnecting to the External Layer 3 DomainSR-MPLS/MPLS Hand-Off on the BL NodesBRK
87、DCN-298075https:/ External EPG(s)Associated to the L3Out 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveL3Out Site 1WANInter-Site NetworkL3Out Site 2BD-RedBD-GreenThe Ext-EPG can be defined in a template associated to multiple sites(stretched object)The Ext-EPG must then
88、be mapped to the local L3Outs in the“site level”section of the template configurationL3Outs remain independent objects defined in each siteRecommended when the L3Outs in the separate sites provide access to a common set of external resources(as the WAN)Simplifies the policy definition and external t
89、raffic classificationStill allows to apply route-map polices on each L3Out(since we have independent APIC domains)Ext-EPGACI Multi-Site and L3OutStretching or Not Stretching the Ext-EPG?BRKDCN-298079 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveL3Out Site 1WANInter-Site
90、 NetworkL3Out Site 2BD-RedBD-GreenSeparate Ext-EPGs can be defined in templates mapped to separate sites(non stretched objects)Each Ext-EPG can be mapped to the local L3Out in the“global”or“site level”section of the template configurationAllows to apply different policies to each Ext-EPGs at differe
91、nt time Can still use the same 0.0.0.0/0 network configuration for classification on both sitesMay require enablement of Intersite L3OutExt-EPGExt-EPGACI Multi-Site and L3OutStretching or Not Stretching the Ext-EPG?BRKDCN-298080Solving Asymmetric Routing Issues with the External Network 2023 Cisco a
92、nd/or its affiliates.All rights reserved.Cisco Public#CiscoLiveSite 1Site 2Inter-Site NetworkActive/StandbyActive/StandbyExt-EPGC1Web-EPG10.10.10.1010.10.10.11IP Subnet10.10.10.0/24IP Subnet10.10.10.0/24L3Out Site 1L3Out Site 2Traffic dropped because of lack of state in the FWACI Multi-Site and L3Ou
93、tTypical Deployment of Perimeter FWsBRKDCN-298082 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveSite 1Site 2Inter-Site NetworkActive/StandbyIngress optimization requires host-routes advertisement on the L3OutNative support on ACI Border Leaf nodes available from ACI rele
94、ase 4.0(1)Not currently supported for SR-MPLS L3OutsL3Out Site 1Active/Standby10.10.10.1010.10.10.11Host routes10.10.10.10/32Host routes10.10.10.11/32Host-routes injected into the WAN*L3Out Site 2Ext-EPGC1Web-EPG*Alternative could be running an overlay solution(LISP,GRE,etc.)ACI 4.0(1)Release Enable
95、d on MSO at the BD level in each siteSolving Asymmetric Routing IssuesUse of Host-Routes AdvertisementBRKDCN-298083Intersite L3Out Support 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveStarting with ACI Release 4.2(1)it is possible for endpoints in a site to send traffic
96、 to resources(WAN,Mainframes,FWs/SLBs,etc.)accessible via a remote L3Out connectionExternal prefixes are exchanged across sites via MP-BGP VPNV4/VPNv6 sessions between spinesTraffic will be directly encapsulated to the TEP of the remote BL nodesThe BL nodes will get assigned an address part of an ad
97、ditional(configurable)prefix that must be routable across the ISNSame solution will also support transit routing across sites(L3Out to L3Out)L3Out Site 1WANInter-Site NetworkWAN,Mainframes,FW/SLB,etcMP-BGP VPNv4/VPNv6ACI Multi-Site and L3OutSupport of Intersite L3OutBRKDCN-298087 2023 Cisco and/or i
98、ts affiliates.All rights reserved.Cisco Public#CiscoLiveACI 4.2(1)Release Endpoint to remote L3Out communication(intra-VRF)Endpoint to remote L3Out communication(inter-VRF)L3Out Site 1WANInter-Site NetworkWAN,Mainframes,FW/SLB,etcACI Multi-Site and Intersite L3OutSupported ScenariosInter-site transi
99、t routing(intra-VRF)Inter-site transit routing(inter-VRF)L3Out Site 1WANInter-Site NetworkL3Out Site 2WAN,Mainframes,FW/SLB,etcWAN,Mainframes,FW/SLB,etcBRKDCN-298088Network Services IntegrationIntegration Models 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveACI Multi-Sit
100、e and Network ServicesIntegration ModelsDeployment options fully supported with ACI Multi-Pod Active and Standby pair deployed across PodsLimited supported optionsActiveStandbyISNActive/Active ClusterActive/Active FW cluster nodes stretched across Sites(single logical FW)Limited supported optionsISN
101、ActiveActiveTypical deployment model for ACI Multi-Site,each fabric leverages a dedicated service node functionUse of PBR to avoid creating asymmetric paths through stateful devices(FWs,LBs,etc.)for both North-South and East-West communicationActive/StandbyActive/StandbyISNBRKDCN-2980104 2023 Cisco
102、and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveBRKDCN-2980L3 ModeActive/Standby ClusterSite1L3 ModeActive/Active ClusterSite1L3 ModeActive Node 1Site1Active/Standby ClusterActive/Active ClusterIndependent Active NodesL3 ModeActive Node 2L3 Mode Active/Standby Node 3The Active/Standb
103、y pair represents a single MAC/IP entry in the PBR policyThe Active/Active cluster represents a single MAC/IP entry in the PBR policySpanned Ether-Channel Mode supported with Cisco ASA/FTD platformsAll ASA/FTD nodes must be connected to the same leaf nodes pairEach Active node represent a unique MAC
104、/IP entry in the PBR policyUse of Symmetric PBR to ensure each flow is handled by the same Active node in both directionsUse of Service Graph and Policy Based RedirectionResilient Service Node Deployment in Each SitePBR redirection only supported to a local service function,hence it is important to
105、deploy such function in a resilient way106Use of Service Graph and PBRNorth-South and East-West 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveL3Out-Site1L3 ModeActive/StandbyL3 ModeActive/StandbyInter SiteNetworkSite1Site2Compute leaf always applies the PBR policyEPGExtE
106、PGWebCCompute leaf always applies the PBR policyInbound traffic can enter any site when destined to a stretched subnet(if ingress optimization is not deployed or possible)PBR policy is always applied on the compute leaf node where the destination endpoint is connectedRequires the VRF to have the def
107、ault policies for enforcement preference and directionExt-EPG and Web EPG can indifferently be provider or consumer of the contractProvider(Consumer)Consumer(Provider)10.10.10.1010.10.10.11L3Out-Site2North-South CommunicationInbound Traffic111BRKDCN-2980 2023 Cisco and/or its affiliates.All rights r
108、eserved.Cisco Public#CiscoLiveInter SiteNetworkL3Out-Site1L3Out-Site2L3 ModeActive/StandbyL3 ModeActive/StandbyCompute leaf always applies the PBR policySite1Site2Compute leaf always applies the PBR policyEPGExtEPGWebC10.10.10.1010.10.10.11PBR policy always applied on the same compute leaf where it
109、was applied for inbound trafficEnsures the same service node is selected for both legs of the flowDifferent L3Outs can be used for inbound and outbound directions of the same flowL3Out-Site2112BRKDCN-2980North-South CommunicationOutbound Traffic 2023 Cisco and/or its affiliates.All rights reserved.C
110、isco Public#CiscoLiveEPGWebEPGAppL3 ModeActive/StandbyInter SiteNetworkSite1Site2ConsumerProviderEPGAppEPGWebC113BRKDCN-2980 EPGs can be locally defined or stretched across sites and can be part of the same VRF or in different VRFs(and/or Tenants)PBR policy is always applied only on the leaf switch
111、where the ProviderProvider endpoint is connectedThe Provider leaf always redirects traffic to a local service nodeProvider leaf always applies the PBR policy(and learns consumer EP info)L3 ModeActive/StandbyEast-West CommunicationConsumer to Provider FlowConsumer leaf does not apply the PBR policyEP
112、-AppO-UTEP S2 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveEPGWebEPGAppL3 ModeActive/StandbyInter SiteNetworkSite1Site2ConsumerProviderEPGAppEPGWebC114BRKDCN-2980L3 ModeActive/StandbyEast-West CommunicationProvider to Consumer Return FlowConsumer leaf does not apply the
113、 PBR policyEP-AppO-UTEP S2Provider leaf always applies the PBR policy EPGs can be locally defined or stretched across sites and can be part of the same VRF or in different VRFs(and/or Tenants)PBR policy is always applied only on the leaf switch where the Provider endpoint is connectedThe Provider le
114、af always redirects traffic to a local service node 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveEPGWebEPGAppL3 ModeActive/StandbyInter SiteNetworkSite1Site2ConsumerProviderEPGAppEPGWebC115BRKDCN-2980 The Provider leaf must always apply the PBR policy,even if it hasnt l
115、earned the EP endpoint yet Mandates to specify the IP prefix under the consumer EPG covering all the endpoints part of that Mandates to specify the IP prefix under the consumer EPG covering all the endpoints part of that EPG(this configuration is enforced on NDO)EPG(this configuration is enforced on
116、 NDO)Becomes challenging when multiple EPGs are part of the same BD(“application centric”deployment model),use of/32 prefixes possible from ACI release 6.0(3F)L3 ModeActive/StandbyEast-West CommunicationWhat if the Communication is Initiated by the Provider?Provider leaf must always be able to apply
117、 the PBR policy,even if it hasnt learned the consumer EPs info yetEPG-App Class-ID information statically configured on the provider leaf nodeEPG-App Class-ID information statically configured on the provider leaf node2Define an IP prefix for the EPG covering all the endpoints in that EPG1New PBR Su
118、pported Use Cases 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveACI Multi-Site and PBR EnhancementsFuture Supported Use CasesMany-to-OneSupport for one service node only(if the Provider is the Ext-EPG)or two service nodes(if the Provider is a regular EPG)Intra-VRF onlyTr
119、affic redirected only through the service node on the providers siteWorks for both“network centric”and“app centric”designsAny-to-AnySupport only for single service node insertionDistributed deployment model(traffic is redirected via both local and remote service node)Works for both“network centric”a
120、nd“app centric”designsTransit Intersite L3OutRedirect intersite transit routing traffic flows Traffic is redirected via both local and remote service nodeSupport only for single service node insertionIntra-VRF and inter-VRFBRKDCN-2980117NDO 4.2(1)ACI 6.0(3F)2023 Cisco and/or its affiliates.All right
121、s reserved.Cisco Public#CiscoLiveACI Multi-Pod White Paperhttp:/ Multi-Pod Configuration Paperhttps:/ Multi-Pod and Service Node Integration White Paperhttps:/ Multi-Site White Paperhttps:/ Multi-Site Deployment Guide for ACI Fabricshttps:/ Multi-Site and Service Node Integration White Paperhttps:/
122、Multi-Site Training Sessionshttps:/ Multi-Site Where to Go for More InformationBRKDCN-2980138 2023 Cisco and/or its affiliates.All rights reserved.Cisco Public#CiscoLiveFill out your session surveys!Attendees who fill out a minimum of four session surveys and the overall event survey will get Cisco
123、Live-branded socks(while supplies last)!These points help you get on the leaderboard and increase your chances of winning daily and grand prizesAttendees will also earn 100 points in the Cisco Live Challenge for every survey completed.BRKDCN-2980139 2023 Cisco and/or its affiliates.All rights reserv
124、ed.Cisco PublicContinue your educationVisit the Cisco Showcase for related demosBook your one-on-oneMeet the Engineer meetingAttend the interactive education with DevNet,Capture the Flag,and Walk-in LabsVisit the On-Demand Library for more sessions at www.CiscoL you#CiscoLive 2023 Cisco and/or its a
125、ffiliates.All rights reserved.Cisco Public#CiscoLive142Gamify your Cisco Live experience!Get points Get points for attending this session!for attending this session!Open the Cisco Events App.Click on Cisco Live Challenge in the side menu.Click on View Your Badges at the top.Click the+at the bottom of the screen and scan the QR code:How:1234142 2023 Cisco and/or its affiliates.All rights reserved.Cisco PublicBRKDCN-2980#CiscoLive