BGP Dynamic Routing and Neutron Ryan Tidwell - HP Jaume Devesa - Midokura Vikram Choudhary - Huawei
Overview • Routing Cloud Network Traffic • Neutron BGP Dynamic Routing Service Overview • Applications of BGP Dynamic Routing with Neutron • Future Work • Q&A
Solutions for Routing Cloud Traffic • Neutron networks are typically stub networks with default route and host routes statically defined for outbound flows by Neutron • Outbound gateway IP determined by reading ‘gateway_ip’ from the external network subnet entity • Next hop for inbound flows must be communicated to infrastructure routers, but how? • Options • Static routing • Dynamic routing
Static Routing of Cloud Traffic • Static routing requires manual configuration of next-hops for each tenant network prefix or floating IP host route / prefix in upstream infrastructure routers • Floating IP’s either must be confined to single L2 network, or host routes must be configured manually • Operator intervention required each time a router is created or deleted • Prefixes don’t move between Neutron routers easily
Dynamic Routing • Operator configures routing protocol amongst infrastructure routers • Make Neutron insert routes into the routing protocol on subnet, router, and floating IP CRUD • Isolate Neutron L3 agent from these changes, so use Neutron as a BGP route server and peer it with infrastructure routers • Neutron will advertise routes to peers, but does not learn from peers
Why BGP? • Separation between data plane and control plane • Work with different AS • Minimal topology to manage
Applications of Neutron BGP Dynamic Routing • Routed Model for Floating IP’s • Unbind the floating range from the L2 network (see routed network segments) • As floating IP’s become unbound from the L2 network, we can advertise a host route for a floating IP as it moves across different L2 network segments. • Directly Routable IPv4/IPv6 Tenant Networks • Use BGP to advertise tenant prefixes for direct routing without floating IP’s or statically routing tenant prefixes • DVR • Enable north-south DVR by advertising host routes with the compute node as the next-hop • Presents some scaling challenges (large number of host routes), is route aggregation possible and would it help?
Future Applications of Neutron BGP Dynamic Routing • Routed Network Segments • https://review.openstack.org/#/c/225384/ • This spec is to support operators who want to be able to attach intances to the network using an L3 domain as the identifier instead of an L2 network i.e. the traditional Neutron “network” • Offers a way of using Neutron to model L3 networks decoupled from the L2 segments they span • Assigning a floating IP wouldn’t require a Neutron router. Use BGP to advertise the floating IP • L3/BGP VPN (Potential Future Application) • Advertise route distinguisher for an address scope to PE routers • Not within scope for Mitaka, but is a potential enhancement • Advertise Floating Range through a Neutron defined Gateway Router
Routed Model for Floating Range spanned in multiple L2 domains
Directly Routable Tenant Networks with Address Scopes IPv6 networks don’ t need to be natted Some small providers don’t want to use Floating IPs Address Scopes will allow to define L3 routed domains instead of forcing NAT on tenant routers (public access to tenant networks) Subnet Pools allow to create non-overlapping Subnets Address Scopes will group non-overlapping Subnet Pools. BGP will automatically advertise new created subnets External Gateway is the BGP peer
Routable Tenant Networks with Address Scopes
Advertise the Floating Range With previous examples, cloud admin has to configure the Gateway Router to advertise Floating Range to ISP or other Enterprise Routers If Gateway Router belonged to Neutron model, we could associate the External Network to BGP speaker and advertise it. Some Neutron SDN controllers (like MidoNet) can implement the Dynamic Routing extension and advertise the Floating Range(s)
Advertise Floating Range
DVR With BGP • Each instance IP is advertised as a host route with the compute node IP on the external network as the next- hop • External network does not need to consume a routable prefix. It can be treated as if it were a link-local prefix. • Large numbers of host routes won’t necessarily scale nicely. Is there a creative way to host aggregate routes?
Sample Deployment (simple) • Entire cloud is treated as a single autonomous system • Operator network runs in a separate autonomous system • eBGP peering with operator network
Sample Deployment (advanced) • External network for each rack • Each rack is treated as an autonomous system • Infrastructure routers redistribute Neutron routes learned by BGP into an IGP • A single address scope can be shared across racks
Potential MPLS/BGP VPN Application
Why MPLS/BGP VPN - Omni presence of MPLS technology. - Almost all the backbone routers understands MPLS. - QoS guarantee - Easier to manage - Scales reasonably
L3 VPN Support • While not the target use case, L3VPN is important to consider. • At the moment this effort is focused primarily on building the mechanism by which Neutron can “speak” BGP and advertise routers to neighbors • In future cycles we will be looking to add L3VPN support and see how similar work in this arena can be discussed, combined and move forward.
Future Work • L3 VPN • BGP-MPLS for tenant-only address scopes • OSPF and IS-IS are very different protocols from BGP, and we don’t think we can leverage any work done on BGP. • Route policing support