AWS - What is Elastic Compute Cloud (EC2) ?
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud.
EC2 is divided into various "families" or instance types. EC2 instances are designed to host different workloads, such as database, web or application services. The customer has root access to the instance, which can be spun up or down via the AWS portal or programmatically with an API call. Instances can be provisioned very quickly and are designed to scale both vertically and horizontally .
EC2 has a lot of flexibility with networking and integration with other AWS services. Each EC2 is deployed in a Virtual Private Network (VPC). This is a virtual network defined by the user and allows have complete control the virtual networking environment, including selection of an IP address range, creation of subnets, and configuration of route tables and network gateways.
There are 3 main options when purchasing an EC2 instance, On Demand, Reserved and Spot.
- On Demand - You pay a fixed rate by the hour with no commitment or up-front payment.
- Reserved - The capacity is reserved for 1 or 3 year terms via an upfront cost. This provids a greater discount over On Demand.
- Spot - Unused EC2 capacity is auctioned, allowing you to bid for capacity between certain times. This provides very low compute prices over On Demand and Reserved. One small point to note, if your instance is terminated by AWS due to the spot price you will not be charged for the partial hour. However should you terminate the instance, you will be charged for the hour.
EC2 provides a range of instance types, each optimized to deal with different workloads. They are,
- I2 and D2 - Storage Optimized
- R3 and X1 - Memory Optimized instances
- T2 – Burstable instances
- M3 and M4 - General Purpose instances
- C3 and C4 - Compute Optimized instances
- G2 - GPU instances
EBS (Elastic Block Storage)
Amazon's Block Storage (EBS) offering allows to you to create block-level storage volumes and assign these to your Amazon EC2 instances. These volumes are automatically replicated within an Availability Zone to ensure resilience in the event of component failure.
There are three volume types: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic, each providing varying levels of performance. These are shown below,
- General Purpose SSD (GP2) - Offers 99.999% availability. Provides 3 IOPS per GB with up to 10,000 IOPS. With the ability to burst to 3000 IOPS for short periods for volumes under 1Gb.
- Provisioned IOPS SSD (IO1) - Designed for I/O intensive applications, providing 10,0000 IOPS plus.
- Standard - Provides the lowest storage cost.
- Throughput Optimized HDD (ST1) - Provides better performance/throughput over Magnetic for applications requiring up to 500MB/s.
- Cold HDD (SC1) - Provides better performance/throughput over Magnetic for applications requiring up to 250MB/s.
Volume snapshots can be created across AWS regions allowing you to backup and restore your data as needed.
EFS (Elastic File System)
Amazons Elastic File System (EFS) is a file storage service for your EC2 instances. Allowing you to, unlike EBS, mount a single volume to multiple instances. Additionally EFS provides,
- NFSv4 support
- Billing on used storage only
- Ability to scale up to petabytes
- Support for thousands of concurrent NFS connects
- Data storage across multiple AZ within a region
- Block Based storage
- Read after write consistency
Volume vs Snapshots
A volume is a virtual hard disk that exists on EBS. Where as a snapshot is an incremental copy (i.e blocks that have changed since last snapshot) of a volume from a given point in time. Snapshots are stored on S3 and are encrypted automatically. Due to this volumes restored from snapshots are encrypted automatically.
NOTE When taking a snapshot of the root device volume always stop the instance attached to the root device volume. This is because AWS will, by default, will stop the instance when this function is performed.
- Create volume within UI.
- Attach volume to instance with UI.
- Get the name of the newly created volume via lsblk.
- Check it has no filesystem on it via file -s /dev/<volume_name>.
- Create and format the filesystem on the volume via mkfs -t ext4 /dev/<volume_name>
- Mount the filesystem via the commands mkdir /fileserver && mount /dev/<volume_name> /fileserver
RAID 5 is not recommended on EBS by AWS, due to it being good for reads but bad for writes. However RAID 0/1/10 should be considered instead. Additionally to provide greater IOPS multiple EBS volumes can be RAIDED together.
Application Consistent Snapshots
When taking a snapshot the data within the cache is excluded. This does not cause an issue for single volumes but for RAID arrays can cause interdependence to the array. There are 3 different methods to ensure the application stops writing to the disk, prior to performing a snapshot. They are Freezing the FS, unmounting the RAID Array or performing a shutdown of the instance.
An Amazon Machine Image (AMI) is a master image for the creation of EC2 instances. AMIs can be thought of as templates that consist of the operating system and other additional software required to deliver a service or a portion of it.
Metadata for each instances can be called via a REST API. To return a list of variables that can be used issue a HTTP GET to http://169.254.169.254/latest/meta-data/.
Example : To obtain the public IP of the instance you can use the public-ipv4 variable, like so HTTP GET http://169.254.169.254/latest/meta-data/public-ipv4
A placement group refers to the assignment of EC2 instances within a Availability Zone in order to ensure low-latency and also provide 10Gb network connectivity. This recommended for applications that require low latency, high network throughput.
However, there are some caveats with placement groups that you should be aware of,
- Cannot span multiple AZ's.
- The placement group name must be unique within the account.
- Only certain instance types can be launched within the placement group.
- AWS recommend that the same type/family are launched within a placement group.
- Placement groups cannot be merged.
- You cannot move existing instances into a placement group.