#MongoDBWorld High Performance MongoDB on Amazon EC2 Sandeep Parikh Partner Technical Solutions, MongoDB Kyle Lichtenberg Solutions Architect, Amazon Web Services
Wait, isn’t there a whitepaper on this? • Much has changed since March 2013! – (and an update is in the works!)
New EC2 Instances Types • I2 released in December 2013 – Next generation High I/O Instance Family – 4-32 vCPUs – 30-244 GiB of Memory – 800-6,400 GB of SSD Storage (up to 315,000+ IOPS) • R3 released in April 2014 – Next Generation Memory Optimized Instance Family – 2-32 vCPUs – 15-244 GiB of Memory
Latest Generation Instance Features • Enhanced Networking (SR-IOV) – Improved packet per second (PPS) performance – Lower inter-instance latencies – Very low network jitter • SSDs – No mechanical seeking! – Higher I/O than magnetic • Updated Hardware Architecture – Ivy Bridge Intel Xeon E5-2670 v2 – Higher sustained memory bandwidth (up to 63,000 MB/s)
AWS Block Storage Review • Amazon Elastic Block Store (“EBS”) – Network attached – Durable – Snapshot backups to S3 • Amazon EBS Provisioned IOPS (“PIOPS”) – Standard EBS + definable, consistent performance • Instance Storage (“Ephemeral Storage”) – Lowest latency – Included in instance cost – Volatile!
Storage Considerations EBS EBS PIOPS Instance Store ~100 IOPS Magnetic Up to 30 IOPS/GB Up to 365,000 3 IOPS/GB General (max 4,000/volume) IOPS/instance Purpose (SSD) Up to 1TB/Volume Up to 1TB/Volume Up to 6.4TB Linear performance Linear performance Fewer IOPS on increases up to 12 increases up to 12 smaller instance sizes volumes on 8xlarge volumes on 8xlarge instances instances Max 365,000 Max 36,000 Max 48,000 IOPS/Instance IOPS/Instance IOPS/Instance No built-in backup Snapshots! Snapshots! functionality
Storage Considerations • When using EBS – Use EBS Optimized Instances! – Use PIOPS for consistent performance – Use snapshots for backups • When using Instance Storage – Replicate! Replicate! Replicate! – Use a replica with EBS volumes for backups
Cost Considerations Assuming a performance target of 140,000 IOPS: All EBS All EBS PIOPS Hybrid Instance Store 47x 1TB General 35x 134GB @ 4K 4x 800GB SSD Purpose (SSD) IOPS 2x i2.2xlarge 6x r3.2xlarge 4x 1TB General (EBS Optimized) 6 r3.2xlarge Purpose (SSD) (EBS Optimized) 1x i2.2xlarge (EBS Optimized) Volumes: $4,700.00 Volumes: $586.25 Volumes: $400.00 Compute: $3,294.00 EBS IOPS: $9,100.00 Compute: $3780.78 Total: Compute: $3,294.00 Total: $7,994.00/mo Total: $4,180.78/mo All costs are estimations an$ d 1 are2 b ,3 ase 9 d o 4 n p.2 ubli5 sh / e m d prio cing as of 6/18/2014.
Cost is an important performance metric! • Reserved Instances – Pricing model designed for steady state workloads – Pay a small upfront fee for a reduced hourly rate – Light, Medium, and Heavy utilization – 1 and 3 year terms
Pricing Models • On-Demand – One-time fee: $0.00 – Monthly Cost: $4,180.78 Hybrid Instance – Annual Cost: $50,169.36 Store 4x 800GB SSD • 1 YR Heavy Reserved 2x i2.2xlarge – One-time fee: $10,920 4x 1TB General SSD – Monthly Cost: $1,119.58 1x i2.2xlarge – Annual Cost: $13,434.96 ($24,354.96) (EBS Optimized) • 3 YR Heavy Reserved Volumes: $400.00 – One-time fee: $16,440.00 – Monthly Cost: $965.86 – Annual Cost: $11,590.32 ($17,070.32) All costs are estimations and are based on published pricing as of 6/18/2014.
Functional Deployment Patterns • Proper instance type and storage selections play a huge part in price and performance. • Not all storage architectures are created equal, but neither are all environments!
High Performance MongoDB
MongoDB Storage Considerations Instance EBS Volumes Storage Ephemeral Persistent Set number of volumes Multiple volumes per per instance instance RAID with LVM RAID with LVM or MD Snapshots via LVM Snapshots
Storage Differences PIOPS EBS Instance SSDs Highest durability Highest performance Requires EBS-optimized instances Available on new generation instances Higher price per IOP Included in instance pricing
Development, Testing Single Node Replica Set App App MongoDB MongoDB Primary MongoDB Secondar y MongoDB Secondar y
Production, High Performance Replica Set Sharding App App App App mongos mongos mongos MongoDB MongoDB MongoDB MongoDB Primary Primary Primary Primary MongoDB MongoDB MongoDB MongoDB Secondar Secondar Secondar Secondar y y y y MongoDB MongoDB MongoDB MongoDB Secondar Secondar Secondar Secondar y y y y
Deployment Strategies: SSDs • SSD deployments – Replica Sets MongoDB Primary and – MMS Backup MongoDB MongoDB • High performance Secondar Secondar y y • Highly available • Continuous backup MMS Backup Agent
MongoDB Management Service: Backup Fully-ma m naged, Custo t m m agent e -based, snapshot Point in-tim t e im con c tin t uous scheduling and recov e ery back c up k ret e en e tio t n Per Pe fo f rma rm nce Res Re tores require e Encrypt p ed e data t impa m ct ct simi m lar to t 2-factor tra t nsfe f r 2-fact Seco c ndary auth ut ent e i nt cati t on
Deployment Strategies: Hybrid Mixe i d d S S S D S /EB /E S S dep
l dep o l y o me m nt n Key ey is s Sec S on o d n a d ry r Pow Po er w f er u f l u ,l h i h gh-b h an a d n width d av a o v i o ding n g with th PIO P PS P S instance replication EBS EB instance replication lag a ex. Aggr gg ega e t ga e e ex. r3.8xlarge multipl p e e Mo M nito t r wi w th with 12 vo v lumes me for MM M S M PIOPS EBS max max IOPS volumes
Measuring Performance MMS System mongoperf Monitoring stats tools MongoDB D metric i s c io i sta t t Mo M ngo g DB D ops p /second Vi V s i ualiz li atio a ns mpstat Da D shboards Se S t thread a s, file fi Siz i e, sar Al A e l rts sa cac a hing