skip to primary navigationskip to content

MPLS virtual private network (VPN) service

The CUDN MPLS virtual private network (MPLS VPN) service allows internal parts of institutional networks to be routed across the CUDN infrastructure, keeping them separate from the global CUDN traffic, except where explicitly interconnected.

At client sites, the MPLS VPN is presented as a regular VLAN, which can be fed from the PoP switch to where it is required. This enables an institution to extend its internal network to other sites without needing to rent additional fibre capacity or operate its own network of routers, whilst benefiting from the redundancy, resilience, capacity and physical reach provided by the existing CUDN infrastructure.

For institutions that have been asked to host part of the internal network of others typically just need to carry this additional VLAN from their PoP to the required hosts.

The MPLS VPN service is provided using MPLS (Multi-Protocol Label Switching) technology on the backbone equipment. The exact operation of MPLS is beyond the scope of this documentation. Unlike with BGP, however, it is usually unnecessary for institutions to understand, configure or support this protocol to use the MPLS VPN service: the CUDN routers provide the translation between regular (non-MPLS) IP routing protocols and MPLS forwarding used on the backbone, although a technical introduction to MPLS VPN is provided, for those interested.

This service only supports layer 3 (routed) connections between sites. For layer 2 (switched) VLAN services between sites, the CUDN provides the inter-site VLAN service.

Important note: this service should not be confused with the CUDN Remote Access VPN service, which is used by end users wishing to connect into the CUDN from elsewhere on the internet. To help reinforce this distinction, the VPN service provided here will be referred to as an 'MPLS VPN'.


Introduction to MPLS VPN

The simplest way to think of an MPLS VPN is as a completely separate set of routers, each with their own routing tables (default routes, connected VLAN interfaces, etc.). These routers are usually set up to route traffic for (part of) the internal network of an institution.

For example, ignoring redundancy arrangements and other details of the CUDN (such as the precise structure of core and distribution routers), consider the following diagram of St. Botolph's College:

MPLS VPN overview


  • is a regular router, outside of the College network and routes traffic across the CUDN global address space, between institutions and out onto the internet; it also routes traffic into St. Botolph's College's network via route.botolphs.
  • route.botolphs is an institutional router that routes traffic between the regular CUDN backbone (via, a local internal client subnet (with host user-1.main.botolphs) as well as to an internal backbone network (via to extend to a hostel site.
  • is a backbone router inside the institutional network: it routes traffic via route.botolphs to the internet (including the global part of the CUDN, outside the institution) and the client subnet at the main institution site; it also routes traffic across the backbone network to a router serving a remote site (
  • is another backbone router inside the institutional network: it is the client subnet router for a hostel site (serving user-2.hostel.botolphs) and also routes traffic bound for the internet and institution main site back across the backbone to

An MPLS VPN allows the two internal backbone routers, and to be physically the same routers used for the CUDN global address space, but have their own private Virtual Routing and Forwarding (VRF) tables. In the example above, and are different logical routers provided by the same physical piece of router hardware, with the two links connecting to route.botolphs being provided as separate VLANs on the same physical link. Each of the logical links have separate IP address ranges.

route.botolphs could also provide a firewall service, filtering traffic between the institutional network (including the MPLS VPN) and the global CUDN via single, central institutional firewall. Indeed, this is one of the primary reasons for using an MPLS VPN: forcing traffic for an institution's sites, distributed around the CUDN, back through a central firewall.

Of course, in reality on the CUDN, these routers are in redundant pairs, with gateway addresses protected by a first hop redundancy protocol. There are also typically PoP switches situated at each site, handling failover of the uplinks to the routers. The institutional network can also be more complicated: the 'outside' and 'inside' links with the CUDN can be provided by separate institutional routers.

The service can easily be scaled, with multiple remote sites being served from the same or separate routers. Typically, when multiple remote sites are added, traffic routes directly between them (over the MPLS VPN), without hubbing back via the main site.


Routing to and from an MPLS VPN (between an institutional network and the 'inside' router provided by the CUDN) can use either static or dynamic routing:

  • Static routing is ideal for simple cases, with a single 'main' site acting as a gateway in and out of the network.
  • Dynamic routing is useful in more complex cases and can support multiple, redundant gateways between the MPLS VPN and the institutional network, potentially at different sites. This configuration requires a good understanding of BGP to implement.

These cases can also be mixed (with some connections into an MPLS VPN being static and some being dynamic), as long as they do not create a conflict between static and dynamic routing information.

Static routing

Where an institution has a single 'main' site, with a single router providing the gateway between the MPLS VPN and the institutional network, a simple static routing configuration is the easiest the configure. Redundancy is limited to that provided by protecting the gateways at each end of the link with a first hop redundancy protocol (which can include the institutional router).

For example, consider the following setup for St. Botolph's College (considering only IPv4; IPv6 is fully supported but will obviously use different addresses on each connection):

MPLS VPN static routing

The ranges in use are as follows:

  • is allocated to St. Botolph's College on the CUDN, this includes:
    • used on the College's main site
    • used for connecting College hostels using the MPLS VPN service, of which only is currently in use
  • An 'outside' /29 link subnet between and route.botolphs (the exact range is unimportant for this example; a /29 is used to support the physical router addresses for a first hop redundancy protocol)
  • An 'inside' /29 link subnet between route.botolphs and

The routing is then configured as follows:

  • On (the global CUDN router), the range used by the College is routed via the 'outside' address of the College router, route.botolphs.
  • route.botolphs has a 'connected' interface serving hosts on the main site (in; the default route ( is set via the CUDN global router ( and the range used by the MPLS VPN for hostels is routed back into the CUDN via, the router inside the MPLS VPN.
  • is configured with a static default route ( via the 'inside' address of route.botolphs.
  • has an interface serving the hosts at the host site (
  • Routing information inside the MPLS VPN, between and, is exchanged using a variety of protocols related to MPLS (including MBGP — Multiprotocol BGP), but this is transparent to the institution.

The above configuration supports an easy expansion of the MPLS VPN service: further sites can be added, using addresses inside the range already routed into the MPLS VPN (for example, To avoid traffic loops, however, some additional blackhole or null routes need to be set up:

  • route.botolphs has a blackhole route for to match ranges which are not (yet) otherwise routed (by longer prefixes) and avoid traffic to unused addresses bouncing between and route.botolphs (as the latter would otherwise match the default route).
  • Similarly, has a blackhole route to to match unused ranges inside the MPLS VPN, rather than having traffic bouncing between it and route.botolphs.

An institution requesting the MPLS VPN service should produce a diagram similar to that above, to explain the desired configuration. This should show the PoP switches and subnets involved.

As a network becomes more complicated, it may be desirable to move to a dynamic routing configuration. An MPLS VPN can initially be set up with static routing and then transitioned to dynamic routing, if needs arise.

Dynamic routing (BGP)

To provide more redundancy on the connection between an MPLS VPN and an institutional network, BGP can be used to allow multiple, independently-routed links, potentially split across different sites. The operation of this is almost identical to one outside of an MPLS VPN, in the CUDN global address space: eBGP peerings are established on the 'inside' connection between an institution's router(s) and the CUDN ingress routers to the MPLS VPN and the routes exchanged. In this case, however, the institutional network is effectively the upstream provider to the MPLS VPN (offering the gateway out onto the CUDN and internet).

The configuration is commonly known as MPLS Inter AS Option A, contrasted with Option B, below.

The use of eBGP on the 'inside' links is usually combined with eBGP peerings outside the MPLS VPN, to provide equivalent redundancy on the 'outside' connections between the institutional routers and the CUDN global address space routers.

For example, consider an expansion of the St. Botolph's College network to include BGP on both the 'outside' and 'inside' connections:

MPLS VPN bgp routing


  • The pairs / route-X.botolphs and / route-Y.botolphs have normal eBGP peerings, outside the MPLS VPN. These, along with an internal St. Botolph's College network, provide a redundant connection between to the global CUDN and the College network. The CUDN routers advertise the CUDN prefixes and default route; the College routers advertise the College prefixes.
  • To provide redundancy linking into the MPLS VPN, the pairs route-X.botolphs / and route-Y.botolphs / also have eBGP peerings. Here, the roles are reversed: the College routers provide the upstream connectivity, advertising the College and CUDN prefixes and default route; the CUDN routers inside the MPLS VPN advertise either a summary prefix (covering all ranges used inside the MPLS VPN) or the prefixes of individual sites.
  • The institution must run the appropriate iBGP peerings and IGP (Interior Gateway Protocol) to handle redundancy across their internal network.

Remote (client) sites can be connected via a VLAN to a regular CUDN PoP switch and routed directly by the CUDN routers, as with a static configuration. The example above shows a St. Botolph's College user hosted at Mincington College: the St. Botolph's College user is placed on the "St. Botolph's College users in Mincington College" client VLAN and will be routed back across the MPLS VPN, through to the St. Botolph's College internal network and then out onto the global CUDN.

The following should be noted about the 'inside' peering:

  • The AS number of the CUDN routers inside the MPLS VPN will be the same as that in the global space: 64602.
  • There will be separate /30 'inside' link subnets from the 'outside' link subnets (in the above example, there will be four link subnets: / route-X.botolphs and / route-Y.botolphs, outside the MPLS VPN, and route-X.botolphs / and route-Y.botolphs / inside the MPLS VPN).
  • As with a static configuration, the institutional network can be more complicated than shown: the 'inside' peering does not need to be with the same institutional routers as the 'outside' peering.
  • Although the AS number is the same, the policies an institution should implement on the peering will be different:
    • The institutional network will need to advertise routes to their internal network (probably by redistributing their blackhole routes) as well as to the CUDN prefixes and default route. This is because the MPLS VPN will need to use the institutional network to reach external routes.
    • The institutional network will need to accept routes over the peering for sites inside the MPLS VPN.
  • If an institution wishes to re-advertise routes learnt from the global CUDN into the MPLS VPN, the eBGP peerings into the MPLS VPN will need to be configured to accept a route with their own ASN (AS Number) as part of the path (since they would otherwise reject it). The institution should make it clear this is required.
  • To maintain maximum flexibility and ease of management, the filtering of routes into and out of the MPLS VPN may not be as strict as a global CUDN peering: as the network is contained within an institution, it is the responsibility of that institution to ensure that the correct routes are accepted and advertised.
  • As with any BGP peering, it is recommended that route summaries be used where possible, to minimise the number of routes exchanged.

As with a static configuration, an institution requesting the MPLS VPN service must produce a diagram showing the routers involved, along with the subnets and client sites.

Dynamic routing (BGP with MPLS)

As an alternative to the above, regular BGP configuration, the CUDN supports direct MPLS forwarding between ASs, avoiding the need to create the separate internal BGP peerings and link subnets.  Instead, the existing "outside" BGP peerings are used to exchange MPLS VPN routing information between the CUDN and the institution and labeled packets exchanged between them.

This configuration is known as MPLS Inter AS Option B (the above configuration is known as Option A) and requires that the institutional network border routers support MPLS.

The configuration involves:


  • The institution should select some MPLS Route Distinguishers (RDs) using the type 1 format, with the IP address part being a CUDN-routed address belonging to the institution and an assigned number of their choice.
  • The institution will be advised of the appropriate Route Targets (RTs) to use for the VPN routes.
  • The existing external BGP peering with the CUDN will have the VPNv4 (and VPNv6, if IPv6 is used) address-families added to it.
  • The MTU used across the links with the CUDN will be increased appropriately, to accommodate the MPLS labels.
  • MPLS forwarding will be enabled across the existing external links to the institutional network.


This configuration is relatively advanced and the institution will need to ensure they have a good understanding of the required configuration and protocols.  It is generally only desirable when an institution has a large number of MPLS VPNs as it simplifies the configuration of these, avoid multiple internal link subnets.  For simple configurations, the internal BGP peering mechanism is probably more appropriate.

Presentation at a site

The way an MPLS VPN is presented on a particular connection varies according to the method used to connect the CUDN at that site, namely whether a PoP switch or a directly-routed link is present.

These methods can be mixed within a particular MPLS VPN. For example, a main site may use routed links but the remote sites may use VLANs fed through a PoP.

Connection via a PoP

If a site has a PoP switch, an MPLS VPN connection is presented as a single VLAN, with the two CUDN routers serving that PoP being set up in a redundant pair using a first hop redundancy protocol. The institution will see the VLAN as any other and can feed it through their network as required.

This technique is particularly useful when one institution wishes to have part of their internal network fed into another, allowing the two networks to be kept logically separate. This requires that the hosting institution merely add an additional VLAN to their internal network, similar to a voice client or wireless access point management network, and feed it through to where it is required by the client institution. The responsibility for the traffic on the hosted network is the responsibility of that institution (rather than the hosting institution), in terms of charging, IP allocation and CERT responsibilities.

As with any VLAN fed to the PoP, the institution should confirm how they would like it presented (which ports and how: tagged or untagged; edge ports or trunk ports).

It is also possible to have an MPLS VPN presented in the same manner as a directly-routed link, though multiple VLANs (one per inter-router link) on the PoP. This is the same as when BGP is used through a PoP.

Connection via directly-routed links

If an institution does not have a PoP (and thus uses a dynamic routed connection with BGP for their global connection), routed VLANs with first hop redundancy cannot be provided across the point-to-point links between the institutional border routers and the CUDN routers. In this case, it is necessary to interface with the MPLS VPN through eBGP peerings.

However, the 'inside' and 'outside' links can be presented as different VLANs on the same physical connection. In the dynamic routing configuration above, the / route-X.botolphs and route-X.botolphs / links would use the same physical fibre; the other link ( / route-Y.botolphs / would use another.

Protocol support

The MPLS VPN service supports a subset of protocols available in the CUDN's global address space:

  • IPv4 — unicast and multicast
  • IPv6 — unicast only

Support for each protocol must be explicitly enabled: institutions will need to state which are required.

Address space

The address space used inside the MPLS VPN is that of the internal network of an institution. This is likely to involve one or more of:

  • public (globally-routable) addresses (e.g. 131.111.x.x, or 2001:630:21x:...),
  • CUDN-wide private addresses (e.g. 172.16.x.x) which will be SNATed (Source Network Address Translated) at the border of the CUDN, before exiting onto Janet / the internet, and
  • institution-private addresses (e.g. 192.168.x.x)

Although institution-private addresses can be routed around inside the MPLS VPN, they cannot be routed directly onto the global portion of the CUDN (through an institutional router), without first being translated (with SNAT) into either CUDN-wide or public addresses (for internet-bound traffic, it is best to use public addresses to avoid double NAT).

The CUDN itself cannot provide this translation service: this must be implemented by an institution (most likely at their 'main' site). If outbound external connectivity (either to the public internet, or just to the CUDN) is required from an MPLS VPN, it is recommended to use CUDN-wide private addresses for hosts on the internal network: this has the advantage of simplifying the networking at the institution (by eliminating the need for the institution to run an SNAT service, along with associated logging) and allowing the real addresses of hosts on an internal institutional network to be seen, making tracing easier (and allowing individual internal hosts to be blocked or examined).

The institution is free to advise UIS as to which addresses/subnets are to be routed where, as required by their internal networking policy. However, for administrative reasons, the CUDN cannot currently route ranges overlapping public and CUDN-wide private ranges in a manner contradictory to the CUDN global allocation policy. For example, an institution cannot request that the range of another institution is routed internally (e.g. St. Botolph's College cannot ask for a CUDN-wide private subnet of the Department of Small Studies to be routed to one of their hostels, unless it really is part of the Department of Small Studies). Institution-private addresses can be routed without restriction.

Router addresses

When routing client subnets, the addresses chosen for the routers must typically follow one of the CUDN edge schemes.

For IPv4, the recommended scheme is 'top' (whereby the router address is 'top - 1' [one less than the broadcast address] and the two addresses below this are used for the routers' physical addresses, to managing the first hop redundancy protocol).

For IPv6, a fixed router address is normally provided at subnet::1, as per a global CUDN connection. However, most hosts will discover the router through ICMPv6 RA (Router Advertisement).


Registration of the addresses used by hosts inside an MPLS VPN depends on the type(s) of addresses in use:

  • For CUDN global addresses, the hosts can be registered in the IP Register database, as normal. The subnets and CUDN router addresses will be automatically registered, when the MPLS VPN is set up.
  • If institution private addresses are used, it is the responsibility of the institution to register and reserve the addresses used by the CUDN routers in their own local registry. 

When registered in the DNS, the names for the CUDN routers inside the MPLS VPN will typically have an additional institutional element added below the 'net', to indicate the institution to which the MPLS VPN applies.

For example, in the St. Botolph's College example above, the client subnet is fed to a hostel. Assuming this is presented to the hosting institution (Mincington College) as VLAN 788, and the CUDN routers providing the gateway are route-B (the primary) and route-C (the secondary), the addresses will be registered in IP Register as follows:

  • — (note the extra 'botolphs' under 'net')
  • —
  • — (the gateway address protected by first hop redundancy, which should be used by clients)
  • 172.27.89.{193-219} — available for an institution to register client hosts (probably directly in the; whether an the site is indicated in the hostname through an extra layer in the DNS path, or with a prefix, is up to the institution)


At the present time, only IPv4 multicast is supported inside an MPLS VPN.  IPv6 multicast may be supported in the future, when technology and equipment allow.

Multicast inside an MPLS VPN requires careful configuration to support it in a scalable manner. Institutions requesting multicast must discuss with UIS Networks how they plan to use it so it can be configured correctly. Details required include:

  • the locations and numbers of sources and members
  • the number and bandwidth of simultaneously active groups
  • the frequency and times when multicast will be used

A less optimal configuration will still work, but may not give best performance, or be the most efficient, in terms of the resources used on network links and router processing.

Configuration of the 'outside' and 'inside' connections to the CUDN are different:

  • On the outside link to the CUDN, the normal inter-domain configuration is required, comprising PIM-SM (Protocol Independent Multicast – Sparse Mode), BGP RPF (Reverse Path Forwarding) routes and MSDP. The institution must also provide its own PIM-SM RP (Rendezvous Point).
  • However, the whole of the inside of the institutional network (comprising both the institutional network and the MPLS VPN) is a single PIM domain: a PIM-SM RP should be configured inside the institutional network and advertised to the CUDN routers across the 'inside' connection using PIM BSR (BootStrap Router) protocol; this RP will likely be the same one as set up for the outside connection.

This is illustrated by the following diagram:

MPLS VPN multicast

If an institution only requires that multicast works internally (and not be connected to the global CUDN), only the 'inside' part of the configuration needs to be set up.

Helper services (DHCP and directed broadcast forwarding)

The DHCP relay and directed broadcast (typically to support Wake-on-LAN) services are available on subnets routed inside an MPLS VPN, just as they are on subnets in the global address space. The packets will be relayed within the MPLS VPN (and back through the institutional router(s) onto the global address space, if appropriate).


Traceroutes across an MPLS VPN provided by the CUDN will not expose the individual backbone routers traversed to reach each site; only the ingress CUDN router into the MPLS VPN will be shown.

For example, consider the following traceroute from the global address space to inside an MPLS VPN for the University Library:

kikuchiyo:~ rcf34$ traceroute
traceroute to (, 64 hops max, 52 byte packets
1 ( 0.971 ms 0.744 ms 3.159 ms
2 ( 3.520 ms 13.186 ms 1.174 ms
3 ( 0.458 ms 0.368 ms 0.364 ms
4 ( 0.512 ms 0.455 ms 0.452 ms
5 ( 0.385 ms 0.394 ms 0.427 ms
6 ( 0.621 ms 0.607 ms 0.596 ms
7 ( 1.681 ms 1.638 ms 1.660 ms

The various hops are:

  • Hop 1 is the first hop institutional router of the traceroute source.
  • Hops 2 to 4 are traversing the CUDN backbone global routing space to get to the border router into the University Library.
  • Hop 5 is the University Library border router / firewall.
  • Hop 6 is the ingress router, feeding into the MPLS VPN provided the CUDN (note the domain ending '' to indicate the University Library MPLS VPN hosted on the CUDN).
  • Hop 7 is the traceroute target device itself.

Note that the intermediate CUDN backbone routers inside the MPLS VPN are not shown between hops 6 and 7 (in particular, the CUDN router serving the target device).

In the reverse direction (from inside the MPLS VPN to the global address space), only the ingress router at the remote site is shown:


Tracing route to []
over a maximum of 30 hops:
1    <1 ms    <1 ms    <1 ms []
2    <1 ms    <1 ms    <1 ms []
3     1 ms     1 ms     1 ms []
4     1 ms     1 ms     1 ms []
5     1 ms     1 ms     1 ms []
6     1 ms     2 ms     4 ms []
7     1 ms     1 ms     1 ms []

Trace complete.

In this direction:

  • Hop 1 is the first hop CUDN router at the remote site inside the MPLS VPN.
  • Hop 2 is the University Library router / firewall which connects to the inside of the MPLS VPN.
  • Hop 3 is the CUDN router linked to the outside of the University Library router / firewall in the global routing space.
  • Hops 4 and 5 are the CUDN backbone global routing space.
  • Hop 6 is the institutional router at the target end.
  • Hop 7 is the host itself.

Again, note that the intermediate CUDN backbone routers inside the MPLS VPN are not shown between hops 1 and 2.

Security of traffic

It should be noted that, unlike a remote access VPN, traffic across an MPLS VPN is not implicitly secured with encryption or authentication, beyond the separation from global traffic provided by the operation of the protocol itself.

Institutions requiring additional security should use IPSec, SSL or other techniques for encrypting and/or authenticating data. Whilst every care is taken to keep the MPLS VPN traffic private, it is possible a configuration error or operational problem may cause traffic to leak into or out of an MPLS VPN onto the global network.

The CUDN and UIS Networks are not responsible for any issues that occur due to such problems.

Management of routers

Although the CUDN routers inside an MPLS VPN are effectively providing part of an institution's internal network, they are still managed by UIS Networks: no special access to query or reconfigure them is provided, as per normal, global CUDN routers. This includes SSH/TELNET and SNMP.

At some point, it is hoped to offer some visibility into the network, such as a Looking Glass to query routes, or an API to query ARP tables.


The MPLS VPN service is charged-for, on a per-site basis, to the "home" institution (the one requesting the private network; not necessarily the one where the VLAN is presented).  A "site" is considered to be either: 

  • A redundantly-routed VLAN presented on a PoP (either the institution's home PoP, or another PoP), or
  • A pair (or part, thereof) of eBGP peerings inside the private network — typically two are required into a single AS.

For example, in the diagram shown in the introduction, St Botolph's College would pay for two site charges: one for the private link inside their network (between route.botolphs and and one for the VLAN presented on the PoP serving the remote hostel (from

An institution may request that the main data VLAN presented on a PoP is inside an existing MPLS VPN (e.g. if the PoP is installed for that institution and is an extension of their internal network) for no extra charge.  However, an unused additional VLAN included in a standard PoP connection cannot be used in lieu of an MPLS VPN per-site charge (e.g. if an institution is only using a single VLAN on its home PoP, the [as yet unused] second one cannot be used for this). 

Technical introduction to MPLS

Note: This section is intended as a basic introduction to how MPLS operates — it is not necessary to understand this to utilise the MPLS VPN service described here. For a more complete technical introduction, please see a guide such as MPLS Fundamentals by Luc De Ghein, published by Cisco Press.

MPLS (Multi Protocol Label Switching) works by prefixing packets traversing the provider network (in this case, the CUDN) with one or more labels. These labels are used in place of the normal forwarding methods based on MAC address (switching) and IP address (routing).

As a packet enters the MPLS VPN from a customer (institutional) network, the ingress router of the provider network prefixes the labels required for the packet to reach the required egress router. On the egress router, these labels are stripped off, restoring the original packet for forwarding on to its destination, back into the customer network at another site. The ingress and egress routers are known as Provider Edge (PE) routers.

The labels are arranged such that only the PE routers need to know about the MPLS VPN itself: the routers in between need only understand the label stating which router is the egress router, not what the traffic being carried actually is (or even what MPLS VPN it is part of). The routers in the middle of the network are known as the Provider (P) routers.

This arrangement simplifies deployment, as only the PE routers need to be explicitly configured with the details of an MPLS VPN: the P routers only need to know enough to carry traffic between one PE router and another. This enables MPLS to scale well, as the P routers do not need to have knowledge of the routing tables of the customer networks.

Indeed, it is possible for the P routers to route traffic other than that which they are normally capable of forwarding (for example, IPX). This is the Multi Protocol nature of MPLS: only the PE routers need to understand the traffic being forwarded and how to add and remove the labels as it enters and exits the MPLS network.

A series of protocols are used to communicate the meaning of the labels between routers. In particular, MBGP (Multiprotocol BGP) is used to distribute the labels used to identify the different VPNs in use on the network; PE routers can translate between regular (non MPLS VPN) routes and MPLS VPN routes, when relaying them between the customer routers and other PE routers.

Last updated: 8th February 2017


If you have any enquiries regarding UIS network services, or other University network topics, please send an email to: