The University Computing Service (UCS) is gradually enabling its services with support for the Internet Protocol version (IPv6), the next generation of TCP/IP, the network communications protocol which is used to route traffic across the global internet. The version used predominantly on the internet, which most people are familiar with, is version 4.
This document explains some of the key differences between the two protocols and how IPv6 will be implemented on the Cambridge University Data Network (CUDN).
- Introduction to IPv6
- Support for IPv6 on the CUDN
- University allocation
- Institution allocation
- Host allocation
- Service addresses
- Default gateway / router addresses
- Mixing public IPv6 and private IPv4
- Registering host addresses in IP Register
- Enabling IPv6 in an institution
This section is intended as a basic introduction to IPv6 for the purpose of understanding the decisions taken when designing the deployment of a new protocol on the CUDN. More comprehensive information on IPv6 in general is available elsewhere and should be consulted by IT staff, prior to deployment.
It is important to remember that IPv4 and IPv6 are technically completely separate protocols with no way to intercommunicate between them. More details on this issue are given below.
The most visible difference between IPv6 and IPv4 is the larger addresses: IPv4 addresses are 32 bits long and written in 'dotted quad' format with four decimal numbers - e.g. 220.127.116.11. IPv6 uses 128 bit addresses, written in 8 groups of 16 bit blocks (each written as 4 hexadecimal digits), separated by colons - e.g. 2001:0630:0212:0012:0000:0000:000d:0001.
When writing addresses, within each block of 4 hex digits the leading zeroes may be omitted. In addition, once per address (and no more) a contiguous run of one or more blocks which are all ':0000:' may be replaced by a double colon. These shorthands are the usual way in which IPv6 addresses are presented for display in software and network equipment. For example, the above address will most commonly be written as 2001:630:212:12::d:1.
Netmasks are written in the same 'mask length' format as IPv4, with '/length' after the address. For example, a /126 subnet has 126 bits of network address, leaving 2 bits for host/network address (bottom being the network anycast address, although this is not required; there is no top address for a directed broadcast).
The format for writing IPv6 addresses is defined in RFC5952 — examples here will be written in this format except where emphasis is needed (for example, sometimes a leading '0' is used in blocks of 4 digits to illustrate the use of four bits of space).
Edge subnets (which PCs, printers, etc. connect to, as opposed to subnets used to link routers together) are typically all /64 in size, for various reasons. A /64 subnet leaves 64 bits for the host address which is large enough for any practical single layer 2 domain (e.g. a VLAN).
Because of this, an IPv6 subnet will usually never need enlarging due to an increased number of hosts being connected (unlike an IPv4 subnet where, on the CUDN, multinetting is typically used to provide additional addresses). As such, it is more important to think of the number of subnets required, rather than the number of addresses on any individual subnet.
IPv6 differs radically from IPv4 in how a host obtains an address to talk to the network. There are four main ways in which this can be done:
- The most common mechanism, up until recently, is called EUI-64 and uses features built into ICMPv6 (Internet Control Message Protocol, which provides things such as PING in IPv4 and IPv6 but has been significantly extended in IPv6 for other purposes). These allow a host to obtain information about the subnet (address and netmask) to which it's attached and construct a unique address for itself, based on its MAC address. As such, it is not necessary to run a DHCP server, nor statically configure an address, in order for it to talk to the network.
- Becoming increasingly common (the default beginning with Windows XP and Mac OS X 10.7) are privacy addresses (RFC4941) whereby the client selects a random address for itself (albeit one valid for the subnet to which it is attached) and changes it every few days, or when it's disconnected and reconnected to the network. This system was invented to avoid a particular device being tracked across the global internet, based on an IPv6 address constructed from its MAC address resulting in a common local part on all client networks. This system does not require any particular support beyond that for EUI-64, above.
- In addition to these automatic mechanisms, hosts can still be statically configured with an address. This is typically used when a particular host or service needs a known address so it can be registered in the DNS and located. The IPv6 protocol mandates that hosts support multiple addresses on a single network interface, so it is common to register one address per service rather than per server; the service addresses can then be moved between physical boxes. Note that EUI-64 addresses should not be moved between boxes as they technically belong to the Ethernet interface which owns a particular MAC address - when setting up a service, a fixed address should be chosen.
- There is also a DHCPv6 protocol but support is not currently pervasive and there is no requirement by clients to use it; it also requires a DHCPv6 server. A common use for this is to obtain additional information beyond the basic IP address (such as the DNS servers, PXE boot servers, etc.).
As well as the above mechanisms for determining a host address, the ICMPv6 protocol allows the router to supply the address of the default gateway router, avoiding the need to manually configure this, or use a separate protocol such as DHCP to discover it. However, some operating systems disable router discovery when a static address is configured - in these situations, a default gateway must be configured manually.
The end result of all this is that end hosts will typically decide which address they're going to use and can get onto the network without explicit configuration, as opposed to the network administrator allocating and configuring addresses, before a host can talk to the network. This obviously raises issues of traceability and accounting. These changes were introduced deliberately, to minimise the overhead of connecting to an IPv6 network.
Various configuration options can be set on networks to 'break' the autoconfiguring nature of IPv6 (in particular, suppressing router responses to requests for available subnet prefixes and router addresses). However, this tends to go against the 'spirit' of IPv6 and is often a futile exercise which will present problems later. The University Computing Service recommends that IPv6 is embraced in full and the differences in behaviour are accepted, rather the attempts made to disable them and bend things into a manner more like IPv4.
IPv6 has an equivalent of 'private addresses' - called Unique Local Addresses (ULAs) but their use is intended to be slightly different from private addresses in IPv4. Because the address space in IPv6 is so much greater than IPv4, and any particular organisation should have sufficient globally-routable addresses for every device to have such as an address, there should be no need for Network Address Translation (NAT) and the array of associated protocols to allow it to support protocols with embedded IP addresses (e.g. protocol fix-up for FTP) or needing to open inbound connections (e.g. UPnP, NAT-PMP).
NAT is not recommended as a security feature, even under IPv4, but a regular firewall or access control lists used, if traffic is to be filtered. Also, given the increasingly diverse range of locations from which users wish to access services, making them directly accessible from the internet, in a secure way, is strongly recommended, where possible.
ULAs should only be used where a device really has no need to be connected to or from the internet, or vice versa. Which devices/networks this applies to can be difficult to determine. For example, the past few years of IPv4 has seen the introduction of printers obtaining firmware and accepting print jobs across the internet, network-enabled security cameras and other devices. If unsure, and renumbering will be difficult later, putting devices on globally-routable addresses and using appropriate access controls on routers and firewalls (which can be relaxed later) is probably the best solution.
There are no plans to deploy ULAs on the CUDN at present. This policy can be reviewed if a need arises.
IPv6 was designed as a completely new protocol from IPv4 and not as a backwards-compatible upgrade: no attempts were made to allow the two protocols to intercommunicate directly (i.e. you cannot use an IPv6-only device to connect to an IPv4-only device, or vice-versa).
However, a number of protocols (Toredo, 6to4, NAT64, etc.) have been developed which allow some degree of interoperability. Some of these must be explicitly supported on each host, whilst some allow a special protocol-level proxy router to be placed somewhere on a network (often not even on the local network but somewhere else on the internet) and used to allow hosts to intercommunicate. Most of these protocols have been designed as an interim measure, to get IPv6 where it would not normally be able to reach, because of legacy firewalls and routers, for example.
In some cases, support for these protocols comes already enabled on user devices and can spring into life, if a suitable proxy router is discovered on the internet. This can result in them being used, even without a system or network administrator being aware of it, perhaps bypassing firewall policies or resulting in weird routes, via the internet, to reach even local CUDN destinations. Whilst the use of some of these protocols can be blocked, often the easiest way to prevent their use is to enable native support for IPv6 on the local network.
Because of this, and other reasons, experience has shown that many of these protocols generate more problems than they solve. The long-term goal should be for native IPv6 support with a view to turning off IPv4 and so their use is generally discouraged, in favour of native IPv6.
Whilst many of these protocols will work fine on the CUDN, it is not expected that explicit local support will be given for them (in terms of either resolving issues with getting them working, or providing the required proxy services themselves).
In the interim, it is expected that hosts run in a dual-stack environment, supporting both IPv4 and IPv6 natively. When looking up the IP address of a host in the DNS (e.g. "www.cam.ac.uk"), if both IPv4 and IPv6 addresses are available, and the client host has IPv6 connectivity, it will typically try to connect to IPv6 first, before falling back on IPv4, if that fails. It is expected this situation will exist for many years, while IPv4 is gradually deprecated.
Support for IPv6 for on the CUDN is largely equivalent to that for IPv4:
- Address ranges can be allocated and routed as normal
- Hosts and routers can be registered in the IP Register database and are available in the DNS
- Redundant first hop routing is provided and provides the same failover times
- BGP is supported for redundant connections
- Multicast is available, including connectivity to the global internet
- Tracking and logging of connections
- MPLS VPNs can carry IPv6 unicast traffic
There are a number of areas where IPv6 support is currently NOT available, either due to technical limitations or services not yet being developed. These include:
- MPLS VPNs do not yet support IPv6 multicast
- The CUDN/Janet traffic accounting system ignores IPv6 traffic (although it is logged)
- The University Wireless Service does not yet provide IPv6 connectivity
These limitations are expected to be addressed as resources and technical solutions allow.
The CUDN has a single public IPv6 /44 address block - 2001:630:210::/44, giving the addresses 2001:630:21P:QRST:..., with 20 bits for the subnet number, i.e. 1 million subnets of /64 in size. The block is part of the Janet allocation (2001:630::/32) - academic, research and other institutions connected solely to Janet typically all have addresses in this block.
The CUDN block was allocated by RIPE on behalf of the university and associated institutions, including colleges and research units connected solely to the CUDN. It is NOT expected additional space will need to be required: institutions will be assumed to get their IPv6 allocated from the CUDN block and not receive separate allocations outside of this range, regardless of whether they had those in IPv4. If an institution feels this to be inappropriate, they should discuss this with Network Support at the University Computing Service.
The space is divided up into a series of /48 blocks as follows:
|0||Loopbacks and /126 intra-backbone and backbone-institutional links|
|2||/56s for institutional networks|
|3||... further /56s for institutional networks (as required)|
|b||Services for external visitors|
|c||University-wide services (e.g. wireless, telephones, etc.)|
Where the use is marked with a dash (-), it is reserved for future use.
As the CUDN block is part of the Janet range it is Provider-Aggregatable (PA) - as far as the internet outside Janet is concerned, the CUDN is not seen independently in terms of routing: a single /32 prefix is carried on the internet backbone for all the education and research institutions inside Janet and parts cannot be broken off and routed differently. Conversely, the CUDN's IPv4 space is a mix of PA and PI (Provider-Independent - where address space is NOT part of a larger block allocated to an ISP). This limitation is one of the main reasons an institution may require address space outside Janet's /32 (and, thus, the CUDN's /44).
Upon requesting its first block of IPv6 address space (by contacting IP Register, as with IPv4), a single /56 prefix from the above /44 will be reserved, giving 256 subnets for internal use. Institutions may decide how this address space is to be organised, typically into a series of /64s. In the case of the simplest institutions, only requiring a single subnet, this is likely to just be the first subnet within the /56 block, i.e.:
|2001:0630:021||P||:||Q||R||0||0||One and only subnet for Department of Small Studies|
If and when an institution with a single subnet requires additional subnets, these will be allocated from the /56 block reserved for the institution.
More complex sites may have multiple internal subnets and wish to structure their organisation of this space. For example, colleges may wish to separate student residential subnets from administrative and servers connections - they can adopt a more complex strategy that takes advantage of boundaries that are maskable for routing and access control reasons. For example - St Botolph's College, receiving the allocation 2001:630:212:300::/56, may decide to subnet as follows:
|2||0||Administrative staff - city centre site|
|1||Administrative staff - west Cambridge site|
|8||0||Student bedrooms - first court|
|1||Student bedrooms - second court|
|2||Student bedrooms - third court|
|e||0||eduroam network - city centre site|
|1||eduroam network - west Cambridge site|
|f||0||Building management - city centre site|
|1||Building management - west Cambridge site|
|4||CCTV - city centre site|
|5||CCTV - west Cambridge site|
Note that, despite being reserved an entire /56 and, possibly, having all or part of the block being routed to their network, institutions must still request the creation of individual /64 subnets through IP Register, before they are put into use.
It should be noted that allocations are made to institutions and not sites - an institution with multiple sites will typically not be allocated a separate block for each site, but will have portions of the /56 allocated to the institution routed to each site, as requested by the institution.
Some large institutions may have internal or affiliated units or groups hosted on their network which have a high level of complexity or autonomy. It may be desirable that separate /56 blocks are allocated for use by those groups, distinct from the institution hosting them. These may include:
- A hosted service which sits outside (or above) the hosting institution, and may move elsewhere in future or should be distinct from the institution itself. An example might be school-level services (e.g. website, fileservers, Active Directory infrastructure, etc.) that are hosted in one or more of the constituent departments making up the school.
- A group which is expected to be broken off from the hosting institution at some point in the near future. A separate block will allow a clean split and avoid fragmentation of the institutional blocks.
- A group with sufficient size, complexity or autonomy where a separate block is required to give them sufficient scope for organising the address space.
- An institution providing connectivity or support to another (e.g. contracted-out IT provision) - a separate block should be allocated to the hosted institution (or the routing transferred to the hosting institution).
- Temporary usage (which may extend to several weeks or months) and it would be beneficial to the hosting institution that the address space was distinct from their own.
In all cases, the hosted group or unit should have sufficient complexity that renumbering will present a problem - for example, a single school web server on an IP address in one of its departments would not present much problem, if it had to change its address to be hosted in another department in the same school. In general, renumbering in IPv6 is a much simpler process than in IPv4 due to mechanisms built in to the protocol to support it (often without needing to reconfigure client machines at all).
These situations are dealt with on a case-by-case basis and institutions should contact IP Register, if they have a query.
The basic rules for users and network administrators on machines and users using the CUDN are the same for IPv6 as IPv4. Principally:
- They must be allowed to do so.
- Any particular access must be traceable back to the machine and user which made it.
On the CUDN, it is a requirement that hosts and their IP addresses are registered in the IP Register database for three main reasons. Some of these reasons are no longer applicable in IPv6:
- To mark an address as "used" (and avoid it being allocated to something else).
The large amount of IPv6 address space available on a single subnet, and the mechanisms a client has to select a unique address for itself without assistance, means that individual addresses do not typically need marking in this way.
- To record the end user, system administrator, equipment details and location of the device.
While the need to do this for a particular device remain, it is difficult to register a static address due to the uncontrollable and dynamic way in which some of the host address configuration mechanisms in IPv6 operate.
- To register the hostname and IP address in the DNS.
DNS entries are primarily registered to allow clients to locate the address of a server. Whilst reverse DNS is sometimes used to identify a client, perhaps as a form of authentication/authorisation, this is not especially secure and advised against. Registration of IP addresses and hostnames is considered 'good form' in IPv4 but is probably impractical in IPv6, for the dynamic address reasons.
Given the above three reasons, in IPv6, the policy on the CUDN has changed:
- Hosts with manually-selected static addresses should be registered.
Typically these will be machines providing services (e.g. a web site, IMAP mailserver, desktop workstation contactable by SSH) and a DNS name needs to be registered to provide a way to locate the host. In addition, if a static address is chosen for other purposes (e.g. to have a fixed address for outbound connection sources), it is a good idea to register it in the database to mark that address as used.
- It is not necessary to register every host in IP Register, unless a DNS entry is required.
Pure client machines typically do not need registering in the database. When making outbound connections on to the CUDN and public internet, the address will be seen as unregistered and have no reverse DNS; likewise, the host will have no forward lookup. Hosts using the EUI-64 mechanism could be registered, if the local network administrators desire, but it would not be mandatory.
It is very important to note that, even though it is not necessary to register pure client devices, Janet rules still require us to know which device (and user) was using which address(es) at which time (and vice versa). This may be accomplished by performing network level logging and/or authentication. However, if no such logging is in place, it may be necessary to force every client machine to disable privacy extensions and force a static EUI-64 address, then register those addresses in IP Register.
Obviously, if host names are not registered in the IP Register database, care should be taken to avoid a duplicate of any particular name being used (if this matters - there won't be a clash in the DNS, but might be in local subnet service location, for example). As such, a local registry of hostnames may need to be maintained or, alternatively, boxes in IP Register could be registered with no addresses.
When allocating a fixed address for a service (or server) for registration in the DNS, largely any scheme can be selected by the institution (the only important thing is NOT to set the 7th bit in the first byte of the host portion, i.e. <network>:0200:~ should be clear, as this bit means the address is 'universally unique' which a manually selected address will not be). However, as a suggestion, the University Computing Service's own scheme is presented here.
The UCS splits elements (separated by colons) of the host portion of an address to give a 'service identifier' and 'instance':
|Network portion (/64)||Host portion|
The service identifiers are largely unimportant but serve to reserve a /112 (i.e. ~65,000 addresses) for a particular service or area. They can just be allocated sequentially, or chosen to represent something memorable - perhaps referring to the common TCP or UDP port number used by a particular service: for example '80' for websites, '25' for email. Alternatively, something making a word or acronym could be used: 'd' for DNS servers, 'aaa' for RADIUS ("Authentication / Authorisation / Accounting"), '109' (hexadecimal representation of "log") for logging servers, etc.
The instance numbers are then typically allocated sequentially, sometimes with test servers receiving a high number (e.g. 'FFFx').
Some examples are shown below:
|2001:630:212:8::d:0||First recursive DNS server||'d' for DNS; '0' = first server|
|2001:630:212:12::d:1||Second recursive DNS server||...; '1' = second server|
|2001:630:212:8::e:0||First email server||'e' = email|
|2001:630:212:8::80:0||First web server||'80' = TCP port 80 (for HTTP)|
|2001:630:212:8::aaa:0||First RADIUS server||
'aaa' = Authentication/Authorisation/Accounting
As stated above, hosts can discover the router address using ICMPv6 and this does not usually need to configured. However, routers will generally need an address in the same subnet for operational purposes.
On the CUDN, an edge subnet is typically connected to two routers. These routers will have physical addresses in the lowest 65,536 addresses (i.e. ...::0 to ::ffff), as well as ...::1 as a First Hop Redundancy Protocol-protected address. This is effectively the network having a service identifier of '0'. For example, if a host on the 2001:630:212:300::/64 subnet requires the default router address to be manually configured (perhaps because entering a static address for a service disables router discovery via ICMPv6), it should use 2001:630:212:300::1.
What happens inside institutional networks is a matter for the institution.
As stated above, there are no current plans to use ULAs (Unique Local Addresses - the equivalent of "private addresses" in IPv4) on the the CUDN. However, the majority of hosts on the CUDN are using private addresses merely as a method to avoid using a public IPv4 address for a host which does not need to be directly contacted from the internet.
As such, client machines using CUDN-wide private IPv4 addresses will gain public IPv6 addresses, when IPv6 is enabled for them. This gives rise to a situation where a host has an address with a '.inst.private.cam.ac.uk' name for its IPv4 address and a '.inst.cam.ac.uk' name for its public IPv6 address.
In this situation, it is recommended that the host is given a name in the public space (e.g. 'inst.cam.ac.uk', rather than 'inst.private.cam.ac.uk'). It may be that the public IPv6 name is not registered in the DNS, if the machine is a pure client; if the name is registered, a vbox object may be more appropriate to indicate one of the addresses is hosted on the same box as the other.
The IP Register database does not consider two registrations with the same name, one in the public and one the private namespace, as being a clash. Care must be taken if this has happened and they refer to different boxes (this situation is probably undesirable, anyway).
Much of the IP Register interface is now enabled for IPv6. IPv6 addresses for hosts can be added using the box_ops page; services (as secondary addresses on a host) are perhaps better represented using vbox_ops.
- On /64 host subnets where the CUDN routers provide the default gateway, the bottom /112 addresses (...:0:0000 to ...:0:ffff) are reserved for the CUDN router addresses. The remainder (i.e. ...:0000:0000:0001:0000 to ...:ffff:ffff:ffff:ffff) is available to the institution to allocate to hosts/services.
- Point-to-point link subnets with the CUDN are typically /126s with the CUDN end at ...:1 and the institutional end at ...:2.
- Subnets routed internally by an institution will typically be set up the same as the above, except that the gateway / upstream end will be available to the institution to register the router address(es) themselves.
Due to the relatively new status of IPv6 and changes to the above policy, there are several subnets which were allocated prior the above arrangements. As such, they may use a different organisation (such as a /64 link subnet to the CUDN). These can be reorganised to conform to the above arrangements, if requested.
In principle, enabling IPv6 on an institutional network is simply a matter of adding an IPv6 subnet prefix to the router interface. Most hosts will already be IPv6-ready, pick up an address within a few seconds and begin talking IPv6, when the service they're trying to reach is available on IPv6.
However, care must be taken before switching this on as many of the pitfalls outlined above may occur. These include:
- Hosts may begin using unpredictable addresses.
Some of the mechanisms clients may use to select an IPv6 address result in unknown and dynamically changing addresses, leaving an inability to trace a device or user.
- IPv6 may bypass firewalls and other access control mechanisms.
IPv6 is a different protocol at the ethernet level - some firewalls may block all unknown types of ethernet traffic; others may allow it to flow through unimpeded. As such, care must be taken to ensure hosts are appropriately secured over IPv6.
- A different-looking address will be seen in logs, etc.
Similar to the previous item, requests will start to be seen from IPv6 addresses - these may break log processing scripts or be allowed/denied access to services secured by IP address.
The CUDN logs IPv6 traffic for auditing and traffic charging purposes in the same way as it does with IPv4. However, this often just narrows a particular address down to a specific institutional network - if privacy extensions are in use, it may not be possible to track an individual host; this is the responsibility of the institution itself.
The University Computing Service encourages institutions to begin preparing for IPv6. Depending on how an institutional network is connected to the CUDN, various options are available to allow testing of IPv6 in a controlled fashion. It is strongly recommended that IPv6 functionality is included as a mandatory item on all future equipment specifications.
Last updated: September 2014