ExaBGP – BGP routing with health checks

ExaBGP is an excellent tool for playing around with BGP. It is feature rich, has an API and gives you the option to run scripts which can announce/withdraw routes.

In my environment I use ExaBGP for certain services to ensure high availability (and load balancing using equal cost multipath, ECMP). To do this I wrote a simple Perl script – it takes a configuration file in which you define a list of services. Each service can have multiple IP’s assigned to it. The health check for each service can be anything you want, as long as it returns an exit code of 0 on success and anything else for failure.

As an example, I use this to load balance my DNS servers. I announce the same prefix via multiple data centers (anycast). I run ExaBGP on each DNS server which does its own health checks and announces the IP’s used for DNS. IPSEC tunnels run between each DC and these DNS routes are also announced over those tunnels. With this setup it ensures the following:

– High availability – The health checks verify the service is working with an expected response. If something is wrong the routes are withdrawn in a few seconds.
– Load balancing – Load balancing is handled with equal cost multipath. You can also setup tiers of servers by using different metrics. No load balancers to worry about, its all handled by the router.
– Scalability – If I need more capacity I just add a new DNS server to the cluster. Once BGP peering is setup, ECMP takes care of the rest.
– Fail over – As I announce the same prefixes in multiple data centers which in turn are announced to each other over tunnels between them, if the service is not available in one data center BGP will take care of it and route to the next available data center.

The health check configuration file is automatically checked for changes, if there is a change it is validated and made live automatically.

A copy of the script with further information can be found on GitHub: https://github.com/sysadminblog/exabgp-healthcheck

Posted in Linux, Networking and tagged , .

Leave a Reply

Your email address will not be published. Required fields are marked *