Cloud services power the apps that are becoming backbone of modern society. The workload of clou...
Cloud services power the apps that are becoming backbone of modern society. The workload of cloud APIs is typically driven by external customers and can fluctuate dramatically minute-by-minute. Rapid spikes in load can result in request failures as load increases beyond backend capacity and the size of web worker pools. This talk explores the use of asynchronous frameworks like python Twisted and gevent to implement services that can dynamically keep socket connections open and increase request latency in order to avoid request failures. We explore how that architectural approach helps Twilio provides high-availability Voice and SMS APIs.