Skill 4.2: Design and implement multi-site or hybrid network connectivity
The term hybrid cloud is used in many ways across the IT industry. Typically, hybrid cloud means to connect your local datacenters to the cloud and run workloads in both locations. This section covers connecting VNets to other VNets, as well as options for connecting your on-premises network to the Azure cloud. There are a range of connection types that can be leveraged for the many scenarios that you as a cloud administrator could face on the exam.
Choose the appropriate solution between ExpressRoute, Site-to-Site and Point-to-Site
When connecting your on-premises network to a VNet, there are three options that provide connectivity for various use cases. These include Point-to-Site virtual private network (VPN), Site-to-Site VPN, and Azure ExpressRoute.
Point-to-Site (P2S) virtual private network (VPN)
This connection type is between a single PC connected to your network and Azure VPN Gateway running over the internet. Sometimes referred to as a “VPN Tunnel,” this on-demand connection is initiated by the user and secured by using a certificate. The connection uses the SSTP protocol on port 443 to provide encrypted communication over the internet between the PC and the VNet. The P2S connection is very easy to setup because it requires little or no change to your existing network. The latency for a point-to-site VPN is unpredictable because the traffic traverses the internet.
P2S connections are useful for remote employees or those that only want to establish connectivity when they need it and can disconnect from the Azure VNet when they are finished with their tasks. In Figure 4-31, depicts an example of developers or testers that only need to connect to the Azure VNet when they are writing code or performing tests on their applications.
FIGURE 4-31 Remote Developers and Tester connecting to Azure VNet using P2S
Site-to-Site (S2S) VPN
S2S connections are durable methods for building cross-premises and hybrid configurations. S2S connections are established between your VPN on-premises device and an Azure VPN Gateway. This connection type enables any on-premises devices you authorize to access VMs and services that are running in an Azure VNet. The connection is known as an IPSec VPN that provides encrypted network traffic crossing over the internet between your on-premises VPN device and the Azure VPN Gateway. The secure encryption method used for these VPN tunnels is IKEv2. Just as with P2S VPNs, the latency for S2S VPNs is unpredictable because the network connection is an internet connection. The on-premises VPN device is required to have a static Public IP address assigned to it and it cannot be located behind a NAT. An example of a S2S VPN is shown in Figure 4-32.
FIGURE 4-32 S2S VPN connection between Azure and On-Premises
There is a variation of this S2S network where you create more than one VPN connection from your VPN Gateway typically connecting to multiple on-premises sites. This is known as a Multi-Site S2S connection. When working with multiple connections, you must use a route-based VPN type. Because each VNet can only have one VPN Gateway, all connections through the gateway share the available bandwidth. In Figure 4-33, you see an example of a network with three sites and two VNets in different Azure regions.
FIGURE 4-33 Multi-Site S2S Network with three locations and two Azure VNets
S2S VPNs should be used for connecting to Azure on a semi-permanent or permanent basis based on your plans for the cloud. These connections are always on and reliable (only as much as your internet connection though). If there are issues with your internet provider or on the public internet infrastructure, your VPN tunnel could go down or run slowly from time to time. Mission critical workloads running in Azure in a hybrid-cloud configuration should use ExpressRoute with a S2S as a backup.
ExpressRoute lets you connect your on-premises networks into the Microsoft cloud over a private connection hosted by a Microsoft ExpressRoute provider. With ExpressRoute, you can establish connections to Microsoft cloud services, such as Microsoft Azure, Office 365, and Dynamics 365.
ExpressRoute is a secure and reliable private connection. Network traffic does not egress to the internet. The latency for an ExpressRoute circuit is predictable because traffic stays on your provider’s network and never touches the internet.
Connectivity can be from a Multiprotocol Label Switching (MPLS), any-to-any IPVPN network, a point-to-point ethernet network, or a virtual cross-connection through a connectivity provider at a co-location facility. Figure 4-34 shows the options for connecting to ExpressRoute.
FIGURE 4-34 Examples of ExpressRoute Circuits
ExpressRoute is only available in certain cities throughout the world, so it is important to check with a provider to determine its availability.
An ExpressRoute connection does not use a VPN Gateway, although it does use a Virtual Network gateway as part of its required configuration. In an ExpressRoute connection, the Virtual Network gateway is configured with the gateway type ‘ExpressRoute,’ rather than ‘VPN’
To connect to Azure using ExpressRoute, set up and manage the routing or contract with a service provider to manage the connection. ExpressRoute uses the BGP routing protocol to facilitate the connections between the routers on the network. There is support for the use of private and public Autonomous System Numbers (ASN), and public or private IPs depending upon which part of the Microsoft cloud that you are connecting to with ExpressRoute.
Each ExpressRoute circuit has two connections to two Microsoft edge routers from your network edge. Microsoft requires dual BGP connections from your edge to each Microsoft edge router. You can choose not to deploy redundant devices or ethernet circuits at your end; however, connectivity providers use redundant devices to ensure that your connections are handed off to Microsoft in a redundant manner. Figure 4-35 shows a redundant connectivity configuration.
FIGURE 4-35 Multiple Cities Connected to ExpressRoute in Two Azure Regions
As seen in Figure 4-36, ExpressRoute has three routing domains: Azure Public, Azure Private, and Microsoft. They are configured as an active-active load sharing configuration. Azure Services are assigned categories based on what they provide and have different IP Address schemes within the Azure regions.
Azure Private Peering Azure compute services, namely virtual machines (IaaS) and cloud services (PaaS), that are deployed within a Virtual Network can be connected through the private peering domain. The private peering domain is a trusted extension of your core network into Microsoft Azure.
Azure Public Peering Services such as Azure Storage, SQL databases, and websites are offered on Public IP addresses. You can privately connect to services hosted on Public IP addresses, including VIPs of your cloud services, through the public peering routing domain.
Microsoft Peering ExpressRoute provides private network connectivity to Microsoft cloud services. Software as a Service offerings, like Office 365 and Dynamics 365, were created to be accessed securely and reliably via the internet.
FIGURE 4-36 ExpressRoute Routing Domains
ExpressRoute can be purchased in different speeds and in one of two modes: Metered or Unlimited.
Metered All inbound data transfer is free of charge, and all outbound data transfer is charged based on a pre-determined rate. Users are also charged a fixed monthly port fee (based on high availability dual ports).
Unlimited All inbound and outbound data transfer is free of charge. Users are charged a single fixed monthly port fee (based on high availability dual ports).
There is also a premium add-on that can be enabled if your network is global enterprise in nature. The following features are added to your ExpressRoute circuit when the premium add-on is enabled:
Increased routing table limit from 4000 routes to 10,000 routes for private peering.
More than 10 VNets can be connected to the ExpressRoute circuit.
Connectivity to Office 365 and Dynamics 365.
Global connectivity over the Microsoft core network. You can now link a VNet in one geopolitical region with an ExpressRoute circuit in another region.
Choose the appropriate gateway
A VPN Gateway is a type of Virtual Network gateway that sends encrypted traffic over a public connection to an on-premises location. You can also use VPN Gateways to send encrypted traffic between Azure VNets over the Azure Backbone network. To send encrypted network traffic between your Azure VNET and an on-premises datacenter, you must create a VPN Gateway for your VNet.
Each VNet can only have one VPN Gateway. However, you can create multiple connections to the same VPN Gateway. An example of this is a multi-site connection configuration. When you create multiple connections to the same VPN Gateway, all VPN tunnels, including Point-to-Site VPNs, share the bandwidth that is available for the gateway.
A Virtual Network gateway is composed of two or more VMs that are deployed to a specific subnet called the GatewaySubnet. The VMs that are in the GatewaySubnet are created upon creation of the VPN Gateway. VPN Gateway VMs are configured to contain routing tables and gateway services specific to that VPN Gateway. You can’t directly configure the VMs that are part of the VPN Gateway, and you should never deploy additional resources to the GatewaySubnet.
When you create a VNet gateway using the gateway type vpn, it creates a specific type of VPN Gateway that encrypts traffic. The Gateway SKU that you select determines how powerful the provisioned VMs will be to handle the amount of aggregate throughput (network traffic for all devices sending and received through). It is important to note the Basic VPN Gateway is only recommended for very small networks with only a few users. The VPN options are captured in Table 4-8.
TABLE 4-8 VPN Gateway options
|S2S/VNet-to-VNet Tunnels (Maximum)
|P2S Connections (Maximum)
|Aggregate Throughput Benchmark
Identify support devices and software VPN solutions
To facilitate a Site-to-Site (S2S), cross-premises VPN connection using a VPN Gateway, a VPN device is required to configured on-premises. These S2S connections can be used to create a hybrid solution, or whenever you want to secure connections between your on-premises networks and your VNets.
In partnership with device vendors, Microsoft has a validated a set of standard VPN devices proven to be compatible. These devices range from routers and firewalls to software that can be run on Linux or Windows servers. All of the devices in the device guide should work with the Azure VPN Gateways. If you have a device that is not listed, there is guidance from Microsoft on the standards that are supported and by using this information, it is possible to connect to the Azure network.
Identify network prerequisites
Connecting to the Azure cloud does require proper planning, and there are some very important steps to look at prior to configuring VNets and hybrid connections to Azure.
Here are some considerations that you should start with when determining how to connect your network to Azure:
IP address spaces It is critical that your IP address spaces don’t overlap with existing networks or those that you are planning for in the future. This is complicated even further when considering that you are most likely building more than one VNet in Azure. It is best to use an offline IP Address Management (IPAM), tool to ensure that all your networks are accounted for before you build anything.
Public IP addresses & AS numbers If you plan on using S2S or ExpressRoute connections to Azure, it is important to have public registered IP addresses. If you are using ExpressRoute or BGP with S2S connections having a registered AS number for your company is the proper way to configure the network.
VPN devices Making sure that you have an approved VPN device is the best bet for creating VPN connections to Azure. You should test these devices by creating a VPN to a test network to ensure that it works properly as a part of your Azure pilot project.
Estimated network throughput It is important to understand the amount of network traffic that goes from your site to Azure over the VPN Gateway. You need this information to select the VPN Gateway SKU to provision. This is also useful for deciding on the size of the ExpressRoute circuit if you chose to go with this type of configuration.
Subscriptions It is also important to consider the number of subscriptions that you have and if they have the same Azure AD Tenet associated with them. This impacts your network design because there are limits to the various configurations and the number of possible connections, depending upon how you configured the subscriptions for your company.
Implement Virtual Network peering service chaining
VNet peering allows you to connect two Azure Resource Manager (ARM), VNets located in the same region together without complex integration. The peered VNets must have non-overlapping IP address spaces. The address spaces in either VNet cannot be added to or deleted from a VNet once a VNet is peered with another VNet.
Once they are peered, the VNets appear as one network and the VM in the peered VNets can communicate with each other directly. The traffic between VMs is routed through the Microsoft backbone network in the same way that traffic is routed between VM in the same VNet through private IP addresses.
You can peer VNets that exist in two different subscriptions, if a privileged user of both subscriptions authorizes the peering, and the subscriptions are associated to the same Azure Active Directory tenant. There is also a limit of 50 VNet peerings per subscription.
Figure 4-37 shows two VNets in the North Central region peered together. Notice their IP address spaces do not overlap as this would make the peering impossible.
FIGURE 4-37 VNet peering between two networks in the North Central Region
Network Traffic flowing through VNet peerings is private, as it never touches any gateways or the internet. This connection is low-latency and high-bandwidth between resources in different VNets. The same speed considerations for VMs on the local VNet apply (based on the VM size). Once Vnet peering is enabled, VMs on each VNet will have the ability to use resources in one VNet from another.
It is important to understand that VNet peering is between two Virtual Networks. There is no derived transitive relationship across the peerings. If you peer VNetA to VNetB and then peer VNetB with VNetC, VNetA is not peered to VNetC. Figure 4-38 shows you a functional architecture of this and describes how VNetA is not peered to VNetC.
FIGURE 4-38 Vnet Peerings do not have a transitive relationship
You can configure user-defined routes that point to VMs in peered VNets as the “next hop,” IP address to enable service chaining. Service chaining enables you to direct traffic from one VNet to a virtual appliance in a peered Virtual Network through user-defined routes. Figure 4-39 provides a view of a network where service chaining is implemented.
FIGURE 4-39 Service chaining allows for the use of common services across VNet Peerings
Configure Virtual Network and Multi-Site Virtual Networks
You can connect VNets to each other, enabling resources connected to either VNet to communicate with each other across VNets. You can use either or both of the following options to connect VNets to each other:
Peering Enables resources connected to different Azure VNets within the same Azure region to communicate with each other. The bandwidth and latency across the VNets is the same as if the resources were connected to the same VNet.
VNet-to-VNet connection Enables resources connected to different Azure VNets within the same, or different Azure regions. Unlike peering, bandwidth is limited between VNets because traffic must flow through an Azure VPN Gateway.
Using VNet-to-VNet connections allows for connecting Azure regions to each other via the Microsoft backbone. This configuration allows for always on connections between the Azure datacenters.
Peering and VNet-to-VNet connections can be used together to enable all networks to see each other. Peering would be setup between one or more VNets within the same Azure regions and then one of these networks would have a VNet-to-VNet VPN setup. The Allow Gateway Transit must be turned on for these packets to be leveraged.
Creating a VNet-to-VNet connection across Azure Regions using a VPN Gateway using the Azure Portal
Figure 4-40 shows a diagram of what you should create. VNETA and VNETB are peered in the North Central Azure regions. Another VNet, VNETC, should be deployed to the North Europe region and then connected to VNETB via a S2S VPN.
The following steps capture the basic process of connecting these networks together:
Create gateway subnets on each VNETB and VNETC
Provision virtual network gateways on VNETB and VNETC
Create connections between the two networks
Configure VNETA to VNET B peering to Use Remote Gateway
Configure VNETB to VNETA peering to Allow Gateway Transit
FIGURE 4-40 Multi-Region VNet deployment using Peering in and VPN Gateway over the Microsoft Backbone
Create GatewaySubnets on each VNETB and VNETC
Using the portal, navigate to each of the VNets, and click the subnets link under settings. From the subnets blade select the +Gateway subnet and assign an address space using a /28 CIDR, as seen in Figure 4-41 and Figure 4-42.
FIGURE 4-41 Adding a GatewaySubnet to VNETB
FIGURE 4-42 Adding a GatewaySubnet to VNETC
Provision Virtual Network Gateways on VNETB and VNETC
Complete the following information for VNETB:
Gateway type VPN
VPN Type Route-based
Virtual Network VNETB
First IP Configuration Create New, VNETBGW-PUBIP
Location North Central US
FIGURE 4-43 Creating the Azure VPN Gateway for VNETB
Complete the following information for VNETC:
Gateway type VPN
VPN Type Route-based
Virtual Network VNETC
First IP Configuration Create New, VNETCGW-PUBIP
Location North Europe US
FIGURE 4-44 Creating the Azure VPN Gateway for VNETC
Create Connections between the two Networks
After you provision the VPN Gateways, create two connections in Azure to bring up the VPN tunnel. To create this object, open VNETB in the Azure portal, as shown in Figure 4-45. Under Settings, locate Connections and click it to open. When the Connections blade opens, click +Add.
Complete the VNETB Add connection blade by using the following inputs:
Second Virtual Network Gatewa VNETCGW
Shared Key A1B2C3D4E5 (any unique value matching on both sides)
FIGURE 4-45 Creating the Connection between VNETB to VNETC
This process needs to be repeated for VNETC. Open VNETC in the Azure portal, as shown in Figure 4-46. Under Settings, locate Connections and click it to open. When the Connections blade opens, click +Add.
Complete the VNETC Add connection blade by using the following inputs:
Second Virtual Network Gateway VNETBGW
Shared Key A1B2C3D4E5 (any unique value matching on both sides)
FIGURE 4-46 Creating the Connection between VNETC to VNETB
Configure VNETA to VNETB Peering to Use Remote Gateway
To allow packets from VNETA to cross the VPN Gateway configured on VNETB, reach VNETC, you must make a change to the VNETA-to-VNETB peering by opening VNETA in the portal to locate the peerings link. Then open the VNETA-to-VNETB, and select Use Remote Gateways in the Configuration section, as seen in Figure 4-47.
FIGURE 4-47 Enabling the Use remote gateways option for the VNETA-to-VNETB Peering
Configure VNETB to VNETA Peering to Allow Gateway Transit
To allow packets from VNETA to VNETC you must make a configuration change to the VNETB-to-VNETA peering. To do so, open the VNETB in the portal and locate the peerings blade. Select the VNEB-to-VNETA connection followed by the Allow Gateway Transit option in the Configuration section, as seen in Figure 4-48.
FIGURE 4-48 Enabling the Allow gateway transit option for the VNETB-to-VNETA peering
After the connection objects are created, it might take a few minutes for the connection to come up. Figure 4-49 shows the status of the connection object by looking on the Overview blade of VNETB-VNETC-Conn1.
FIGURE 4-49 Status of the Connection between VNETB and VNETC shown as Connected
Gateways and On-premises Connectivity
Any VNet can have its own VPN Gateway and use it to connect to an on-premises network. When VNets are peered, this doesn’t change. You can also configure VNet- -VNet connections using VPN Gateways. Even the VNet(s) are peered to other VNet(s).
When both options for VNet interconnectivity are configured, the traffic between the VNets flows through the peering configuration (that is, through the Azure backbone).
When VNets are peered in the same region, you can configure the VPN Gateway in the peered VNet as a method to point traffic to an on-premises network. In this case, the VNet using a remote gateway unfortunately cannot have its own gateway. A VNet can have only one gateway. The gateway can either be a local or remote gateway. In Figure 4-50, you see VNetA and VNetB located in the West Europe region peered together with VNetA providing VPN services to both networks. In this configuration, all three networks can reach each other.
FIGURE 4-50 Gateway Transit allows peered VNets to connect with networks that are across a VPN Gateway