Route Servers

Overview

Normally on a peering exchange, all connected parties will establish bilateral peering relationships with each other member port connected to the exchange. As the number of connected parties increases, it becomes increasingly more difficult to manage peering relationships with members of the exchange. A typical peering exchange full-mesh eBGP configuration might look something similar to the diagram on the left hand side.

IXP full mesh peering relationships		IXP route server peering relationships

The full-mesh BGP session relationship scenario requires that each BGP speaker configure and manage BGP sessions to every other BGP speaker on the exchange. In this example, a full-mesh setup requires 7 BGP sessions per member router, and this increases every time a new member connects to the exchange.

However, by using a route server for all peering relationships, the number of BGP sessions per router stays at two: one for each route server. Clearly this is a more sustainable way of maintaining IXP peering relationships with a large number of participants.

Should I use this service?

The INEX route server cluster is aimed at:

• small to medium sized members of the exchange who don’t have the time or resources to
  aggressively manage their peering relationships
• larger members of the exchange who have an open peering policy, but where it may not
  be worth their while managing peering relationships with smaller members of the exchange.

As a rule of thumb: If you do not have any good reason not to use the route server cluster, you should probably use it.

The service is designed to be reliable. It operates on two physical servers, each located in a different data centre. The service is available on all INEX networks (public peering lans #1 and #2 and INEX Cork) on both IPv4 and IPv6. Each server runs a separate routing daemon per VLAN and per protocol. Should a single BGP server die for some unlikely reason, no other BGP server is likely to be affected. If one of the physical servers becomes unavailable, the second server will continue to provide BGP connectivity.

INEX has also implemented inbound prefix filtering on its route-server cluster. This uses both RPKI and internet routing registry data from the various IRR databases to allow connected members announce only the address prefixes which they have registered publicly. If your prefix has a valid RPKI ROA, it will pass. If the RPKI ROA check result is unknown (you have not set up a ROA), we fall back to the IRRDB test.

INEX uses BIRD (Bird Internet Routing Demon) running on Ubuntu LTS 18.04 for its route server cluster. BIRD is widely used at internet exchanges for route server clusters, and has been found to be reliable in production.

How do I use the service?

If enabled, the route servers are set up to accept BGP connections from your router. Once this has been done, you will need to configure a BGP peering session to the correct internet address. The IP addresses of the route servers are listed as follows:

For Cisco routers, you will need something like the following bgp configuration:

router bgp 64496
  no bgp enforce-first-as
  !
  ! Route server #1
  !
  neighbor 185.6.36.8 remote-as 43760
  neighbor 185.6.36.8 description INEX Route Server
  !
  address-family ipv4
    neighbor 185.6.36.8 password 
    neighbor 185.6.36.8 activate
    neighbor 185.6.36.8 prefix-list pl-announce-to-inex out
  !
  ! Route server #2
  !
  neighbor 185.6.36.9 remote-as 43760
  neighbor 185.6.36.9 description INEX Route Server
  !
  address-family ipv4
    neighbor 185.6.36.9 password s00persekr1t
    neighbor 185.6.36.9 activate
    neighbor 185.6.36.9 prefix-list pl-announce-to-inex out

You should also use route-maps to control outgoing prefix announcements to allow only the prefixes which you intend to announce.

Note that the route server system depends on information in either RPKI or the various IRR databases (typically the RIPE IRR database for INEX members). If you have not published either a valid ROAs or correct route:/route6: objects in this database, your prefix announcements will be ignored by the route server and your peers will not route their traffic to you via the exchange. This is checked by INEX Operations during provisioning and guidance and assistance is provided as required.

Community based prefix filtering

The INEX route server system also provides well known communities to allow members to control the distribution of their prefixes. These communities are defined as follows:

0:peer-as

43760:peer-as

0:43760

43760:43760

So, for example, to instruct the route server to distribute a particular prefix only to AS64111 and AS64222, the prefix should be tagged with communities: 0:43760, 43760:64111 and 43760:64222.

Alternatively, to announce a prefix to all INEX members, excluding AS64333, the prefix should be tagged with community 0:64333.

INEX’s route server clusters support BGP large community prefix distribution control on all peering LANs, using the following policy:

43760:0:peer-as

43760:1:peer-as

43760:0:0

43760:1:0

INEX’s route server clusters also support BGP large community AS path prepending control on all peering LANs, using the following policy:

43760:101:peer-as

43760:102:peer-as

43760:103:peer-as

If your router supports large communities, you should use these over standard 16-bit communities as a large number of INEX members now have a 32-bit ASN. You should not mix standard 16-bit communities and large communities – please choose one or the other.

Looking Glasses

You can find INEX’s looking glasses for all route server (and route collector) instances here: https://www.inex.ie/ixp/lg.

If the route server is filtering a prefix, it will show in the looking glass with a warning symbol in the route listings under State/PfxRcd. You can click on Details next to this to see why a prefix is filtered.

Filtering Policy

INEX’s Route Server filtering policy is defined in the source code for IXP Manager on github. This is a summary of what it does.

1. Drop small prefixes – longer than /24 for ipv4 and longer than /48 for ipv6.
2. Drop all well-known martians and bogons.
3. Ensure that there is at least 1 ASN and less than 64 ASNs in the AS path.
4. Ensure that the peer AS is the same as the first AS in the AS path.
5. Drop any prefix where the next-hop IP address is not the same as the peer IP address. This prevents prefix hijacking.
6. Drop any prefix with a transit network ASN in the AS path.
7. Ensure that origin AS is in set of ASNs from the client’s IRRDB AS-SET.
8. If the prefix is evaluated as RPKI valid, accept.
9. If the prefix is evaluated as RPKI invalid, drop.
10. If the prefix is evaluated as RPKI unknown, revert to standard IRRDB prefix filtering.

RFC1997 Pass-Through

RFC1997 defines some well-known communities including NO_EXPORT. INEX’s route servers do not interpret these well-known communities but passes them through.