Changes between Initial Version and Version 1 of db/mtgs/aws2014


Ignore:
Timestamp:
04/10/2014 11:23:59 AM (5 years ago)
Author:
danielw
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • db/mtgs/aws2014

    v1 v1  
     1 
     2== Talk: "Big Data and High Performance Computing Solutions in the AWS" (marketing perspective) == 
     3https://www.youtube.com/watch?v=naCUrtd6XJE 
     4 
     5 
     6== Talk: Scaling to 10 million users == 
     7Fun Fact: In 2014, the amount of capacity added daily on AWS exceeds the capacity to operate Amazon’s $7billion business in 2014. 
     8 
     9=== Stages of growing: === 
     10* Initial: single machine (e.g., EC2 instance) running web+db, with DNS routed by AWS Route53. 
     11 
     12* Users>100: Split db off into a db service instance (e.g., using Amazon Relational Data Store (RDS)), or another vm instance dedicated to the db. 
     13 
     14* Users>1000: Replicate machine and db instance. Now we have two pairs of web+app and db. Let the dbs coordinate using replication. Balance between them using Elastic Load Balancer. Place each pair in different “availability zones” so that a data center failure in one of the zones affects only one web+app and db pair–your system is still functional in the other zone. 
     15 
     16* Users>10k: Add even more pairs. Consider more availability zones. 
     17 
     18* Beyond: Move static content to S3 and serve using cloudfront for edge CDN service. Use elastic cache and/or dynamodb to cache state and reduce traffic to db instances. 
     19 
     20Amazon autoscaling can use metrics from cloudwatch to make decisions to add more web nodes or db instances as load changes. 
     21 
     22Use service-oriented architecture: this facilitates making your components stateless, replaceable, and scalable. Don’t let components talk directly to each other–use indirection so that either side of the communication can fail without affecting the other. 
     23 
     24Overall advice: split processing into pieces that are loosely-coupled and stateless as is possible.  
     25 
     26== Other notes: == 
     27Spatial libs for AWS CloudSearch and DynamoDB seem interesting. They 
     28are focused on really low latencies (10-100ms). Cloud Search has those 
     29latencies when doing lat/lon area searches.