IT'S NOT ALL ABOUT YOUR CODE! Time spent in your code? Overal percentage?
NETWORKING! IT'S MOSTLY ALL ABOUT NETWORKING What happens when you type an address in your browser? The TCP/IP Stack: Data Link, Network, Transport, Application (also Physical, Session, Presentation) This is cal ed the OSI model.
TCP: HOW DATA FLOWS FROM A TO B A node sends "packets" Packets must be of a certain size (~1500 bytes, 64k max) Only a certain amount of packets are al owed "in flight" Packets are guaranteed delivery, in-order (different than UDP) Starts with a Three-Way handshake. Syn -> Syn Ack -> Ack
HOW LONG DOES THIS TAKE? NYC to SFC: ~2530mi, RTT in vacuum: 27ms, RTT in fiber: 41.04ms IP and Routes add delay. (Check out Ping and Traceroute) Remember the three way handshake? Latency. Bandwidth may not matter Connections are scarce, precious resources! (Why we have connection pooling)
HOW DOES HTTP FIT IN? Application level protocol Defines how requests and responses sit on top of TCP Http 1.1: Keep-alive, pipelining Browsers 6-8 connections per server Think of 1mb of data split 1, 10, 100 ways
Most client-server optimization is about tuning for network latency and managing TCP connections. Networking isn't just about a browser and a server. Intra- and inter- data center connections are important! Optimize for throughput: keep data smal , reduce latency, reduce connections.
WEB SERVERS! Web servers like nginx and apache handle http requests. Application servers run programs which produce dynamic content. Think Thin, Unicorn, Puma, Node, etc. Sometimes it's a blurry line. Depends on architecture.
THREADED VS. EVENTED How connections are mapped to requests How threads are mapped to requests Memory usage and context switching How does your OS help?
ALL ABOUT ASYNC Non-blocking io is essential in al cases EventMachine, Node, Twisted: Event loops Heroku's big problem with Rails
A TYPICAL SCENARIO LOAD BALANCER -> WEBSERVER -> APP SERVER Load Balancer Distributes Requests Web Servers (nginx/haproxy/varnish) handles connections, static content Application Servers or Process Pools respond to requests
SO WHAT DO WEB SERVERS ACTUALLY DO? Very good at handling http connections Parse http requests Add filters like gzip, authentication Buffering, Caching Reverse-Proxy support
DIFFERENCES WITH APPLICATION SERVERS (AND RUNTIMES!) Al ow you to program dynamic content Thin, Unicorn, Puma, Passenger MRI, Rubinus, JRuby Node vs. Rails vs. Django vs. Whatever?
RACK A common interface between Ruby web servers and web frameworks. Provides simple http primitives Supports middleware Can mix Rails and Sinatra endpoints
DATABASES! They're beasts! Sql - ACID NoSql - BASE Both have their own set of problems.
TUNING DATABASES In both cases, it's about indexes. Learn to find slow queries. Avoid n+1 queries (go for eager loading). Optimize for read/write patterns.
SCALING DATABASES How do you add more nodes? Again, understand read/write patterns. Understand latency.
POLYGOT DATA LANDSCAPE Postgres, Mysql Cassandra, Riak, HBase MongoDb Redis Memcache
CACHING! Html Caching Fragment Caching Object Caching How easy is it to expire your cache?
HTML CACHING Uses http headers ETags, Cache-Control, Expires Anonymous vs. Authenticated Users How are you expiring cache?
FRAGMENT CACHING Stil Html! More surgical ESI make tools like Varnish quite powerful
OBJECT CACHING Most common Most flexible Easy db integration Memcache for reads Redis for structured data Compress (CPU cycles are there) You're just denormalizing data
QUEUES! THEY'LL BE YOUR BEST FRIEND. Very Async Great for handling spikes Offload low-priority requests Great for elastic workers Resque, IronWorker, Amazon SQS
PAAS VS. IAAS How do you want to run your apps? What control do you want or need? Other AWS Services: ElasticCache, R53/ELB, ASG, Search, CloudFormation, OpWorks, Multi-AZ, Multi-Region Cloud Services: Most tech is offered as SaaS NewRelic, Loggly
GOOGLE'S SPDY One persistent SSL connection True Duplex, no waiting Better window scaling Compressed headers (cookies, etc) Content Prioritization Server Push
EXAMPLES How would you design a news site? How would you design an e-commerce site? How would you design a social site?
PROBLEM The server request/response time is very long.
PROBLEM The server request/response time is very fast, but when more load increases, times increase dramatical y.
PROBLEM A page takes a long time to load, but the server time is very fast.
PROBLEM We are sending images to a bunch of customers. Sending images is very slow, but nothing is maxed out. The problem increases when we try and send more images.