Obama For America on AWS Younjin Jeong Solutions Architect
What am I talking about today? What was OFA? Why is this relevant? • Who did it? • What did they build?
How did they do that? • Technologies and Tradeoffs • Services vs. Software
What did they learn from building something so big?
Full Disclosure I work for AWS AWS does not endorse political candidates
Yes, I talk too much
So here’s the Idea ~30th biggest E-commerce operation, globally ~200 distinct new applications, many mobile Hundreds of new, untested analytical approaches Processing hundreds of TB of data on thousands of servers Spikes of hundreds of thousands of concurrent users
FUN FUN FUN
NOT FUN NOT FUN a few constraints… NOT FUN
~30th biggest E-commerce operation, globally ~200 distinct applications, many mobile Hundreds of new, untested analytical approaches Processing hundreds of TB of data on thousands of servers Spikes of hundreds of thousands of concurrent users
Critically compressed budget Less than a year to execute Volunteer and near-volunteer development team Core systems will be used for a single critical day Constitutionally-mandated completion date
CHALLENGE ACCEPTED !
Built by guys and gals like these: Obama For America
Business as usual.. …for a technology startup
Election Day – OFA Headquarters
So they built it all, and it worked
The old approach, even from Amazon
The old approach.. Might have some problems..
Cloud Computing Benefits No Up-Front Pay Only for Low Cost Capital Expense What You Use Self-Service Easily Scale Improve Agility & Infrastructure Up and Down Time-to-Market Deploy
Sites Communications Ad Targeting Ops Tools Analytics Apps Micro-targeting Micro-listening Reporting Registrations Volunteer Coordination Etc, etc, etc.
Technology Choice Expected Tradeoff Polyglot Development More Complex Ops
Cloud Hosting Less Infra Control,
performance Diverse, App-centered More Complex Ops, Databases Fragility, Data Corruption
SOA, queue-based system Dev Complexity, slower integrations system performance
Technology Choice Expected Tradeoff Upside Polyglot More Complex Build as little as Development Ops possible, rev-1 faster, reuse dev skills
Cloud Hosting Less Infra Control, Scale, Speed, Cost
Diverse, App- More Complex Heterogeneous centered Databases Ops, Fragility, Resilience, right tools for the job
SOA, queue-based Dev Complexity, Scalability, system integrations slower system serviceability,
performance operational flexibility, and
substantially faster in aggregate
No time to waste
This applies to lots of services! ELB ElastiCache RDS CloudSearch Route53 You can mostly do S3 these on your own… CloudFront DynamoDB But do you have extra: focus, expertise, time, research, money, risk-tolerance, staff, dedication to innovate, operations coverage, scalability in design...
Looks pretty simple. Inserts 7.5m records in DynamoDB, in 8 minutes
One thing that is difficult to prepare for…
They had this built for the previous 3 months, all on the East Coast.
They had this built for the previous 3 months, all on the East Coast. AWS + Puppet + Netflix Asgard + We built this CloudOpt + DevOps = part in 9 hours
to be safe. Cross-Continent Fault- Tolerance On-Demand