A little bit about me • Cloud.Com Founding Engineer • Software architect for CloudPlatform • Responsible for overal architecture, performance, and scalability • Committer and PPMC member • BS from UC Berkeley and MS from Stanford
Outline • Difference between automation, orchestration, and provisioning • Networking Example • Anatomy of a plugin • Myth busting
Automation, Orchestration, & Provisioning
What is the difference? • The key is the data center abstraction layer – Virtual Machine, Template, Nic, IP Address, Volume, Network, Rules, Snapshot • Orchestration orchestrates within this abstraction layer • Provisioning manifests concepts in abstraction layer on physical resources • Automation orchestrates above the data center abstraction layer to bring greater functionality
Examples • Orchestration – VM deploy, Volume creation, Network creation, Network rules propagation • Provisioning – Starting a VM on a hypervisor – The actual movement of a volume from one storage pool to another • Automation – HA – Putting a physical resource into maintenance mode
Enabling Innovation • CloudStack Orchestration must not define the innovation. – Partners define their own APIs. – Partners and CloudStack can work together on unified APIs through design process on Apache. • Differentiate between orchestration and provisioning. – CloudStack only orchestrates. – Provisioning is always pushed to the partner. • Clearly defined data center abstraction layer. – Changes in this layer are broadcasted to partners. • Utilize CloudStack’s orchestration to deploy and auto-scale partners’ technologies.
CloudStack Terminology (End User) • Network – A single concept to encapsulate multiple network technologies to simplify representation to the end user. – Each Network always carries its Network Traffic Type. – CloudStack DOESN’T understand how to provision this conceptual network on to the physical network. • NetworkService – L2-L7 network services that partners have written to operate within a Network. – Currently defined: Load Balancing, Port Forwarding, Firewal , Gateway, DNS, DHCP, Static NAT, VPN, Source NAT, User Data. • NetworkOffering – A packaging of the NetworkServices provided to the end user on a particular Network. – NetworkOfferings are put together by cloud service provider.
CloudStack Terminology (Provider) • Network Traffic Type – Traffic types are mapped to the underlying physical network by the cloud service provider. – Traffic type is not the same as network (Guest traffic type can actually be carried on multiple networks) – Currently defined: Public, Guest, Storage (Backup really), Management • NetworkServiceProvider – Plugin that understands how to provide one or more NetworkServices by using VPX or physical resource. • PhysicalNetwork – Actual wiring of the data center.
CloudStack Terminology (Partner) • NetworkGuru – Plugin that understands the network isolation technology, mac addressing scheme, and IP addressing scheme deployed and how to map Network Traffic Types to the underlying physical network. – CloudStack passes Network to NetworkGuru to “implement” before the network is needed by a virtual machine. – CloudStack asks the NetworkGuru to issue ip, mac, and isolation to a virtual machine before it starts. – CloudStack informs the NetworkGuru when a virtual machine stops so it can collect resources. – When all virtual machines in a Network are stopped, CloudStack garbage-collects the Network by asking the NetworkGuru to shutdown the network. – CloudStack provides a default implementation for VLAN based isolation technology. • NetworkElement – Interface that specifies the events CloudStack signals to the NetworkServiceProviders when a Network needs to be “implemented” and shutdown and when a virtual machine joins and leaves a Network.
“Architect” Model • The builder offers multiple blueprints for the owner to build the house. • Owner chooses on a blueprint and then adds on with additional enhancements such as hardwood floors, granite counter tops, etc. • General contractor builds to the blueprint by orchestrating between different sub- contractors to build different parts of the blueprint. • There are two general category of contractors. – Rough-in sub-contractors who take care of plumbing, electricity, framing, foundation. – Finish sub-contractors who put in flooring, kitchen cabinets etc. • Each sub-contractor is responsible for only their work but looks over the entire blueprint to make sure their work can actually be done. – E.g. A lighting plan may conflict or needs to change depending on the framing plan. • General contractor is responsible for sequencing the sub-contractors to make sure everything the sub-contractor is dependent on is ready when the sub-contractor arrives to do his work. • Every change requires a the blueprint to be republished so every sub-contractor can make their appropriate changes.
Comparison Building a house Building a network • Owner • End user • Builder • Cloud Service Provider • General Contractor • CloudStack • Rough-in Sub-Contractors • NetworkGurus • Finish Sub-Contractors • NetworkServiceProviders • Blueprint • Abstraction Layer • Cabinets, Flooring, Counter • Tops, etc NetworkServices
Architectural Principles • CloudStack clearly defines the difference between orchestration and provisioning. – Orchestration the ordering of what needs to happen in CloudStack’s abstraction layer. – Provisioning is the actual work performed at the resource. • CloudStack clearly defines the difference between network definition and network services. – Network definition is handled by NetworkGuru. – Network services is handled by NetworkServiceProvider. • CloudStack broadcasts changes in the network every time NetworkServices and virtual machines changes in the Network. • CloudStack al ows the Cloud Service Provider to setup the appropriate mappings between virtual concepts such as Network and Network Traffic Type to the underlying physical network.
Sequence Flow for VM Creation Engine End User Security User VM VirtualMac Network Network Job Storage Mgr Rest API Checkers Mgr hine Mgr Mgr Guru Scheduling Deploy VM ACL Checks Allocate Entity in CS Allocate VM Allocate NIC Allocate IP Allocate Volume Schedules Deploy Job Returns with job id, VM id Query Job Result Returns with job status
Sequence Flow for VM Deployment User VM VirtualMac Deploymen Network Storage Network Network Server Template Job Threads Services API t Mgr hine Mgr Mgr Mgr Guru Element Resources Mgr Planner Start VM Start User VM Start VM Get a Deployment Plan (Host and StoragePool) Prepare Nics Reserve resources for Nic Notify that Nic is about to be started in network Agent Calls Prepare Volumes Prepare template on Primary Storage Agent Calls Agent Start VM Call Stores job result
Anatomy of a plugin
CloudStack WebServices API
OAM&P API End User API AWS API Pluggable Service API Engine Accounts Business Logic CloudStack Plugins Security Manager A anage r anage r pdate anage r H anage r anage r ent Planner Events Resourc e M Rules M U M M Capacity M Manager ent Planner rok Service Provider eploym Usage CloudStack Orchestration rok Service Provider Manager Adapters etw etw eploym Domain Network Guru Manager ispersing D Network anager Element First Fit D Account achine ork plate etScaler N ser D Deployment N U Manager anager anager anager anager M etw Virtual Router N N M Tem M Snapshot M Planner Limits Virtual M Hypervisor Storage M Guru Manager Framework Agent Manager Cluster Manager Data Access Layer
Anatomy of a Plugin • Server Component: – Deployed on management server Rest API – Can implement multiple Plugin APIs to affect its feature Implementation – Can expose its own API through Pluggable Service so Plugin API administrators can configure the Data Access Layer plugin – Can access the database • ServerResource: S S e e rrv v e e rrR R e e ssou ou rrcce e - Optional. Required if Plugin needs to be co- - Optional. Required if Plugin needs to be co- located with the resource – Deployed co-located with the located with the resource - Implements translation layer to talk to resource - Implements translation layer to talk to resource - Communicates with server component via JSON physical resource - Communicates with server component via JSON – Cannot access the database
Anatomy of a Plugin • Can be two jars: server component to be deployed on management server and an optional ServerResource component to be deployed co-located with the resource • Server component can implement multiple Plugin APIs to affect its feature • Can expose its own API through Pluggable Service so administrators can configure the plugin • As an example, OVS plugin actual y implements both NetworkGuru and NetworkElement
Plugin Interfaces Available • NetworkGuru – Implements various physical network technologies and ip address • NetworkElement – Facilitate network services on network elements to support a VM (i.e. DNS, DHCP, LB, VPN, Port Forwarding, etc) • DeploymentPlanner – Different algorithms to place a VM and volumes. • Investigator – Ways to find out if a host is down or VM is down. • Fencer – Ways to fence off a VM if the state is unknown • UserAuthenticator – Methods of authenticating a user • SecurityChecker – ACL access • HostAl ocator – Provides different ways to allocate host • StoragePoolAl ocator – Provides different ways to allocate volumes
Adding a Plugin to CloudStack • Components are configured though components.xml • Supports DAO, Manager, and Adapter patterns
Extending CloudStack Networking 2. prepare (Network, Nic, DeployDestination, VmInfo) 1. prepare (part of start vm) Net e w t o w rk Net e wo w rk kEle E men e t n Plu P gga g ble b Se S rvic v e ic Manage ag r e Device Configuration MyDn D s n Dev e ice ic Ser e Admin API (CRUD) vice ic Dns n Service ic 3. addDnsRecord(ip, fqdn) MyDnsDeviceMa Demonstrates one way to MyD y nsElement MyDnsDeviceM MySQL na n ge g r e inform an external DNS server when an instance starts. Ag A e g nt n Man a ag 4.Enqueue AddDnsRecord er Que u ue Classes shaded blue form a plugin / service bundle to integrate an external DNS My M Dns n De D v e i v ce c R e e R s server. Clients of the Serve v rRe R sourc r e c ourc r e c instance can then use DNS names to access the 5.API cal to Dns Device instance.
Let’s do some myth busting
Myth 1 • Network is Layer 2 • Network is Layer 3 • Network confines how I implement my physical network.
Busted • No. Network is just a concept to represent where virtual machines can access each other. – One Network to rule them all, One Network to define them, One Network to bring them all and in the cloud bind them. • What defines a network to be layer 2 or layer 3 is the NetworkGuru.
Myth 2 • I must convert my service to run inside CloudStack’s plugin. • CloudStack controls the deployment of my service.
Busted • CloudStack’s plugin interfaces specify the provisioning events and data associated with the events. • Plugin implementations can be client stubs that cal out to your actual service upon receiving these events. • If you so desired, plugin implementations can also expose APIs to configure the plugins.
Myth 3 • My service cannot scale independently from CloudStack. • My service’s availability is tied together with CloudStack’s availability.
Busted • CloudStack does not need to know how a service scales. – CloudStack does not manage instances of a service. – CloudStack does not manage instances of the same plugin. Plugin code is always a singleton.