Enterprises typically have had few options for connecting remote offices, particularly those in obscure locations or foreign countries. Telecom operators ruled the virtual roost, and IT groups were at the mercy of carriers demanding high prices while delivering questionable quality and support, with nominal penalties for violating service-level agreements.
No longer. The WAN winds are shifting, giving IT groups significant leverage. The global spread of high-speed DSL and all-fiber backbones means Internet performance is good enough for most applications. Although the Internet has always been a relatively inexpensive option vis-à-vis private networks, the price-performance gap is narrowing all the time, while compliance requirements that mandate encryption reduce private data networks' perceived security edge.
Bottom line, when it comes to WANs, IT managers should keep their options open. Today's providers are focused on consumer business and may not be updating their private WAN infrastructures. As this equipment ages and providers cram more Layer 3 VPN connections into older routers, we could face the sporadic multiday outages companies saw in the waning days of frame relay.
Although measures of cost, performance and reliability are tipping the scales from Layer 3 private networking to Internet substitution for many enterprises, those still running legacy applications or those that want to control their own routing will have less flexibility. In these instances, a middle ground, Layer 2 VPNs running over MPLS (Multiprotocol Label Switching), may be the best bet. L2VPN services provide a base on which enterprises can build private, routed networks and also serve where routing isn't necessary. Companies that require only simple connectivity should consider two MPLS-based services: VPLS (virtual private LAN services), which simulate Layer 2 networks, and VLLs (virtual leased lines), which mimic leased lines. Long term, however, all companies should look closely at migrating to Internet-based connectivity.
It's The Net Result That Counts
Expecting the Internet to deliver acceptable performance for corporate applications may sound strange to some, while the security-minded see the Internet as today's Wild West--not a place for sensitive data.
But those of us who've engineered and built global networks know just how little the performance of a well-engineered public Internet link differs from that of a private WAN. As the former chief architect at Cable & Wireless, I tracked the data on packet loss (typically a fraction of a percent) and delay we published on a global basis. In a private study of performance, we showed that our portion of the public Internet performed with end-to-end jitter of less than 50 microseconds 99.99 percent of the time. This study involved multiple testing points, large data transfers and network sampling over a period of months.
Of course, not all Internet backbones are created equal. IT can improve the odds of getting better-than-average connectivity by being mindful of a few facts. First, the strength and the weakness of the Internet is that it's a collection of networks, or autonomous systems (AS). Ultimately, the boundaries between these ASs can cause trouble. When ASs are connected in multiple places, as in a multihomed enterprise or in the peering points between large ISPs, the connections form a redundant, and more reliable, path for traffic. But multiple connections are harder to engineer and require more attention with regards to capacity planning and network management.
Minimizing the number of ASs that traffic must traverse greatly increases the performance of an Internet-substitution WAN (see diagram, at left). Of course, the first AS boundary crossing is from the enterprise to the service provider, and it goes without saying that IT must pay close attention to this link. But though one enterprise may find acceptable performance in a network where most of its locations are attached to a well-engineered ISP backbone with ample peering bandwidth, another may suffer from poor performance because of the ISP backbone, ISP peering or both, and this is where IT should focus.
Paranoia For The Real World
Security is becoming less of a differentiator between Internet and private networks due to advances in VPN technology. Beginning with remote-access concentrators that encrypt connections from host-to-network, vendors increasingly are moving security features into their high-end and even low-end customer premises routers and switches. The lines between the conventional router and firewall are blurring as well. IT now has a choice between a simple rules-based secure VPN, a dynamic route-based VPN or a hybrid of the two. Of course, there are still DDoS attacks and worms on the Internet, but running IPsec tunnels over anonymous connections in remote offices reduces the risk of these attacks.
In the past, running the necessary number of IPsec tunnels (twice the number of connections) to accommodate a larger company's requirements has been a problem for routers, especially the smaller devices in remote offices. The routers may have difficulty managing the state of so many tunnels, or network managers may find it difficult to maintain so many encryption keys. Dynamic Multipoint VPN, a little-known Cisco technology, may address this problem. A DMVPN is a series of encrypted multipoint GRE (Generic Routing Encapsulation) tunnels in a hub-and-spoke layout. A DMVPN minimizes latency by providing a temporary full mesh, without the configuration and state complexity required to maintain a true full mesh; this is especially useful in a VoIP network. DMVPN technology works by anticipating that some spokes may be addressed dynamically, as in the case of consumer-grade Internet connections. At the hub, routers speak the all-but-forgotten Next-Hop Resolution Protocol. When new spokes come up on the network, they know how to reach the hub, establish a secure connection and register with the NHRP route server. When one spoke has traffic to send to another, it queries the NHRP server for the destination spoke's location. The two spokes then establish a direct IPsec connection that survives for the traffic flow or some predefined amount of time, and is then torn down.
Meanwhile, compliance requirements are driving more companies to pay for encryption equipment for even their private networks, eliminating one of the few cost advantages that a private network can offer over the Internet. Although the risk of interception is no greater on an MPLS network than it was on its frame and ATM predecessors, today's regulatory environment is forcing companies to be stricter about how privacy is protected on the WAN. In the case of the federal government, for example, the OMB (Office of Management and Budget) and standards from NIST (National Institute of Standards and Technology) have driven some agencies to adopt strict guidelines for encrypting network traffic.
Win-Win Situation
As companies look to run multiple protocols over their WANs or control their own routing infrastructures, running a WAN over the Internet becomes too complicated or not feasible. Services like Ethernet-based VLAN metro service and VPLS are addressing these issues.
With VPLS, service providers can offer native Ethernet services over the metro and wide area. One small provider that's using this technology to cater to vertical markets is Yipes Enterprise Services.
"VPLS and Ethernet hand-offs allow us to be very flexible and to offer innovative services like extranets and Web-based bandwidth-on-demand," says Keao Caindec, Yipes' chief marketing officer. Yipes uses partnerships to provide service on a local, national and global level.
In a VPLS network, PE (provider edge) routers carry and distribute information about the Ethernet MAC addresses that communicate through them in a bridge module (see diagram, right). The benefits of a VPLS to the enterprise revolve around its simple, familiar configuration: The VPLS network can be a true bridged extension of the LAN. In a more complex case, the VPLS network can serve as an Ethernet backbone for a single enterprise or (as with Yipes) a shared media for an industrywide extranet. Side benefits include less expensive CE (consumer edge) equipment and fewer engineers and technicians needed.
Converging multiple legacy technologies onto one backbone not only reduces complexity, it boosts efficiency by providing greater flexibility in traffic engineering. And a new breed of edge router lets providers host multiple services--including VLL, VPRN, VPLS and more--from the same PE equipment. All this translates into cost savings for enterprises, both in terms of the services they buy and the equipment required to connect to providers.
MPLS has some big advantages for the service provider as well. Because MPLS packets are switched rather than routed (see "At Your Service"), less expensive network equipment can be deployed.
The MPLS layer is often hidden from enterprise IT, and many carriers don't offer native MPLS services to their customers. But that's OK--enterprises still benefit because with MPLS, service providers offer a set of flexible services, often from the same edge equipment. Services might be combined to ease into a transition from a legacy technology. An enterprise might move from frame relay to a VLL and later add a VPRN on the same link. Other services might provide both metro-area and wide-area networking, as is the case with VPLS. Each service will typically provide a gateway to the Internet.
Network managers must allow for the bandwidth into and out of the MPLS cloud at each site, but they no longer have to wrangle the bandwidth needed between sites.
The Downside
The great majority of enterprises that use "MPLS networks" are really using an L3VPN. In these networks, the provider is taking responsibility for both the WAN and for routing over the WAN, which can benefit an enterprise. So why aren't true MPLS-based networks more popular?
First, there's very little interconnection among VPN offerings from different providers, and there are limits to what a single provider's VPN can do. Large telcos, including AT&T and Verizon Business, have built extensive private MPLS networks, but even these big guys can't reach into every part of the globe. What's more--as many enterprises find out the hard way--the cost for a single provider to offer these networks goes up as enterprises move into the far reaches of the world.
Then there's the notion of the priority of transit traffic. As we mentioned, MPLS packets can be marked with differing priorities, often based on the customer's own prioritization of the traffic indicated by DiffServ. However, there are no agreed-on standards for indicating the priority of MPLS traffic between providers, so even if a customer could entice two providers to interconnect, the priority of resulting traffic flows between those networks would not necessarily be the same.
Given the problems of network reach and security, some forward-looking companies are turning either to a different kind of network, from a different kind of provider--virtual network operators (see "VNOs: Smooth Network Operators" below).
VPRN: Layer 3 By Any Other Name
The most popular form of virtual private network, the VPRN, goes by a number of names. Often when we read the term MPLS network, the reference is to a VPRN, but the terms tag switching network, Layer 3 VPN or IP VPN apply too.
VPRNs are a new twist on an old idea--the managed router network--and can take a number of forms. At each enterprise location in a VPRN, a CE router sends routes to the service provider using an open standard routing protocol. The service provider's PE routers then establish MPLS connections from Point A to Point B.
In this scenario, CE routers don't speak MPLS. As with many telecom services, the provider can make the connection from A to B along any path--as long as the result behaves within the bounds of an SLA. Because PE routers carry the enterprise's routes in a VRF (virtual routing and forwarding) instance, and there could be hundreds of enterprise VPRNs attached to a single PE router, the VRF is important to the operation of these networks. PE routers carry and disperse the individual enterprise's VRF table to directly attached corresponding CE routers. PE routers also exchange the VRFs of multiple enterprises; this communication takes place through open standard routing protocols, like BGP.
"VPRNs have become the most popular option for enterprises who look to replace an aging ATM or frame relay infrastructure," says Martin Cappuro, Qwest's director of enterprise products. "One reason is that the technology and its risks are familiar to those who've already invested in a virtual network based on ATM or frame relay." Cappurro should know--he works with Qwest's L3VPN as well as its conventional frame relay and ATM offerings.
Routing configurations in the wide area can be greatly simplified as well; for example, enterprises can insert routes into the provider's routing system. The provider carries those routes to all enterprise locations to redistribute them. An enterprise can use an open standard routing protocol--often RIPv2 or OSPF--enterprisewide.
Service providers like the VPRN concept because VPRNs let them make good use of shared infrastructure. With a VPRN network, a provider can divide the resources within a single PE router and offer a form of managed wide-area routing to multiple enterprises. Service providers can create a VPRN infrastructure over a number of networks, including GRE or L2TPv3 (Layer 2 Tunneling Protocol version 3), but MPLS affords more flexibility to traffic-engineer the transport network that carries enterprise traffic. Service providers also offer enterprise customers a variety of network services based on MPLS, including priority service based on packet marking or express routes.
Layer 2 VPNS: Bridging The Divide
MPLS proponents have long touted L2VPN's ability to create Layer 3 as well as Layer 2 VPNs. Now that L3VPNs are mature technology, a number of providers are turning to L2VPNs to spark the interest of enterprises and to fill a need that the more complex L3VPN simply can't fill, like an industrywide extranet.
MPLS L2VPNs have been possible since the early days of the MPLS standardization process; Luca Martini, an enterprising architect at Level 3 Communications, saw the need for a bridge between the MPLS and legacy networking worlds. The result, commonly called the "Martini Draft," enables MPLS backbone providers to extend links via ATM and other legacy methods.
With this technology, a provider with an MPLS backbone could "converge," or replace, existing legacy backbones with MPLS and save on the operational expense of running multiple legacy and MPLS backbones. In a Martini network, the PE router converts connections from an ATM-attached enterprise to MPLS, makes the changes in encapsulation, and switches the resulting payload over a Layer 2 VPN, or more specifically, a pseudowire.
Pseudowires can carry frame relay, ATM, Ethernet, Sonet or TDM payloads. Verizon Business, which already supports two large VPRN networks, recently announced a Virtual Private Wire Service, according to Mike Marcellin, executive director of product management. The company had previously announced an expansion of its metro Ethernet access to its Private IP/VPRN networks.
Even if an enterprise chooses to connect to the wide area over ATM or frame relay and continues to use existing CE equipment, once it's attached to the MPLS network, it often can add MPLS-based services to the mix. Some government organizations that were running large point-to-point backbones on frame relay, for example, have migrated those networks to carriers that support frame relay over MPLS.
Their main sites remain connected in a point-to-point mesh, but they now can overlay a VPRN on the architecture and benefit from any-to-any traffic flow, especially in remote offices. This can be especially useful as an organization adds technologies that benefit from low-latency transit, like VoIP or IP videoconferencing. It's also helpful for segregating established flows, like mainframe-to-mainframe traffic, from more volatile traffic, like VoIP.
Jeffrey Young is a senior analyst with the Burton Group, specializing in network architecture, Internet networks and backbones, and telecommunication service providers. Write to him at [email protected].
VNOS: Smooth Network Operators
In legacy networks, well-defined levels of service meant an enterprise could purchase a combined network from multiple ATM providers, each specializing in a different geographic area, that were already connected or could easily interconnect. Niche providers, like Equant, grew to fill the needs of ATM networking on a multinational scale.
Not so in the MPLS (Multiprotocol Label Switching) world. Although there's been recent work in the IETF to standardize an NNI (network-to-network interconnect) for MPLS providers, few are making use of the technology. Thus most VPN networks based on MPLS are purchased from a single provider. The same might be said for metro Ethernet or IPsec VPN providers that have very few interconnections.
This situation paves the way for a different kind of network provider, the VNO (virtual network operator). In a world where, increasingly, everything is virtual, why not?
As the name implies, VNOs don't own their telecommunications infrastructures. They lease bandwidth between their own meet points and operate equipment that receives transit traffic from one provider and makes a meaningful translation before forwarding it on. If a VNO customer needs to send traffic from Point A to Point B, the VNO makes sure that the traffic is handled correctly by all transit service providers. VNOs tend to be experts in a broad range of connectivity options; for example, a VNO may lease a metro Ethernet solution in three cities, a VPLS network in three more cities and connections from two nationwide VPRN providers to build a network for one enterprise.
"VNOs could be described as outsourced architecture, engineering and operations groups for the enterprise," says Ciaran Roche, senior consultant for London-based VNO Vanco.
VNOs are large consumers of wholesale telecom services and as such can generate economies of scale; many pass those savings on to their customers. Vanco and other VNOs would seem to be in competition with large operators in that they sell wholesale services from those carriers to enterprises. But the VNOs prefer to look on it as more of a collaborative model. In fact, the strength of a VNO's operation often lies in its carrier relationships.
"Vanco is basically selling the global provider's network at no cost to the global provider" Roche says. Vanco is always the single point of contact for service-level agreement and operational issues.
On the business side, VNOs generally fall in the middle of the road when it comes to cost. Where they excel is TCO over the life of a contract. In the case of Vanco, most contracts are for three- to five-year terms, as with most telcos. The main difference is, Vanco has incentive to improve contract pricing over time.
"Anyone can give away a bit of money at the beginning of a five-year contract to get the business. That's easy," Roche says. Year-over-year, the average drop in carrier cost is around 20 percent, Roche says, and Vanco passes some 70 percent of that savings on to the end user.
VNOs like Vanco make their money in outsourced program and network management. This arrangement may not be suitable for every enterprise, especially those that must maintain control of day-to-day network operations or program management (see the comparison chart at right).
At Your Service
In most "virtual networks" offered by service providers, an underlying MPLS backbone might provide transport for Metro Ethernet, frame relay, ATM and VPRNs. To accomplish this, MPLS lies somewhere between circuit switching and packet switching: It's often described as a "Layer 2 1/2" technology. A closer look at the MPLS packet describes why.]
The label is a simple construct that groups sent packets by their ultimate destinations, without regard for intermediate hops. The MPLS packet journeys along a Label Switched Path (LSP) until its end, where the label can be popped off the packet and the flow continues by normal means (IP forwarding).
MPLS packets are, by design, a hybrid of routing and switching--MPLS is a product of a time when the hardware and memory required to switch ATM cells was far less expensive than the corresponding hardware and memory required to route IP packets. MPLS packets contain one or more ATM-like headers in front of an IP packet of variable length. What's more, MPLS headers can be pushed onto the packet or popped from the packet by an LSR (label-switch router).
