My AWS certification notes part 1 from A Cloud Guru material

AWS Global Infrastructure

• Region – Geographical location
• Availability Zone – Data center, 2 or more in region
• Edge Location – Cashing, Cloud Front, CDN- Content Delivery Network

Compute

• EC2 – vm
• EC2 container service
• Elastic BeanStalk
• Lambda
• Lightsail – Virtual private service, fixed IP address
• Batch – Batch computing in cloud

Storage

• S3 – simple storage service, object based stored, bucket
• EFS – Elastic File System – Network based storage
• Glacier – Data Archival
• Snowball – bring large data into datacenter
• Storage Gateway – Virtual appliance, replicates date to S3

Database

• RDS – Relational Db service, mysql, postgress, oracle
• DynamoDB – Non relational
• Elasticache – caching data
• Red Shift – Data Warehousing

Migration

• AWS Migration Hub – track the migration
• Application Discovery service – discover the dependencies. Eg: sharepoint or dependency with DB
• Database migration service
• Server migration service
• Snowball

Networking & Content Delivery*

• VPC – virtual privacy cloud. Configure your own private cloud
• CloudFront – CDN, stores your image/video closer to your AZ, edgelocation
• Route53 – DNS service
• API Gateway –
• Direct Connect – direct line from hdq to amazon cloud

Developer Tools

• CodeStar – colabrating code
• CodeCommit – store code. Version controller
• Code Build
• Code Deploy
• CodePipeline
• X-Ray – debug
• Cloud9 – IDE, browser

Management Tools*

• Cloud watch –
• Cloudformation –
• CloudTrail – log the changes
• Config – monitor the configuration, visual
• OpsWorks
• Service Catalog
• Systems Manager – patch all the EC2
• Trusted Advisor – advice security, risks, capacity
• Managed services

Media Sevices

• Elastic Transcoder – resize video to android /ios

Machine Learning

• Sagemaker –
• Comprehend
• Deeplens – physical hardware – camara
• Lex
• Machine learning
• Polly – text to voice
• Rekognition – upload image / video, will tell you what it is
• Amazon translate – translator
• Amazon Transcribe – speech recognition – speech to text

Analytics

• Athena
• EMR – big data solution. To process large data
• Cloudsearch
• ElasticSearchService
• Kinesis – bigdata, ingesting large amount of data in cloud
• Kinesis Video Streams
• Quicksight – BI tool
• Data Pipleline – moving the data bw to amazon services
• Glue – ETL

Security & Identity & Compliance

• IAM – Identify Access Management
• Cognito – Auth service, gives temp access to AWS
• GuardDuty – malicious activity
• Inspector – governance tool, check the vulnerability
• Macie – scans S3 for PII data
• Certificate Manager – manage SSL
• CloudHSM – Hardware security Module – stores encryption keys
• Directory Service – integrating Active directory
• WAF – Web Application Firewall – prevent XSS, Sql injection
• Shield – prevent Ddos attacks
• Artifact – compliance report, PCI report

Mobile Service

• Mobile Hub – management console for mobile apps,
• PinPoint – push notifications
• AWS Appsync – automatically updates the data (offline).
• Device Farm
• Mobile Analytics

AR / VR

• Sumerian – code name

Application Integration*

• Step functions
• Amazon MQ
• SNS – Notification service
• SQS – Decoupling
• SWF – Simple Workflow Service

Customer Engagement

• Amazon Connect
• SES – Simple Email Service

Business Productivity

• Alexa for Business
• Chime – Video conference like xoom
• Work Docs
• WorkMail – like office 365

Desktop and App Streaming

• Workspaces
• AppStream 2.0 like citrix

IOT

• IOT device management
• Amazon FreeRtos – OS for microcontroller
• GreenGrass

Game Development

• GameLift

IAM

• Identity Access Management
• Its Global
• Allows to manage users and their level of access to AWS console
• Users
• Groups – collection of users under one set of permissions
• Role
• Policy – document defines one or more permissions
• Managed policies
  • Created and administrated by AWS
  • Can be attached to multiple users, roles, groups with aws or different aws accounts
  • You cannot change the permissions
  • Recommended
• Customer Managed policies
  • Policy that you create
  • Can be attached to multiple users, roles, groups, but only with in your own aws account
• Inline policies
  • Policy embedded with user, role or group
  • When you delete, user, role or group, inline policy also will be deleted
• Users – Programmatic, console
• Programmatic – Access Key Id & Secret Access Key
• Console – User & Password
• For region specific settings, add conditions as part of policy
• For new users by default no permissions
• https://.signin.aws.amazon.com/console – IAM users signin link
• Power user – Access to all the AWS resources except management of groups,users with in IAM

STS – Security Token Service

• Grants users limited & temp access to AWS resources
• Federation – SAML, AD, Single sign on
• Federation Mobile – FB, amazon, google or openId providers to login
• Cross account access – one aws account to access another resource
• Federation – Combining list of users from one domain with list of users in another domain. Like from IAM to AD or FB
• Identity Broker – service that allows you take an identity from point A and join (federate) it with point B, you have to develop, it will not come out of the box
• Identity Store – AD, FB, Google
• Identity – user of service like FB
• Identity Broker always authenticate with federation (AD, LDAP, FB, Google) first then STS
• Temp Security Credentials
• Access Key ID
• Secret access key
• Security token
• GetFederationToken
• Returns the set of temp security credentials for federated user
• you must call GetFederationToken operation using the long term security credentials of IAM user. not IAM role

Active Directory Federation

• AssumeRoleWithSAML API
• Always authenticated with AD first then AWS
• SAML – Security Assertion Markup language

Web Identity Federation

• Authenticate using Google, FB, LinkedIn
• AssumeRoleWithWebIdentity API
• ARN – Amazon Resource Name

Cognito

• Web identity federation service
• User first authenticated with web identity provider (FB). Received auth token, exchanged for temp. aws credentials allowing them to assume an IAM role
• Provide signup / signin for your apps
• Access for guest users
• Acts identity broker between your app and web ID providers (FB or Google)
• Recommended for mobile apps runs on aws
• Uses Push synchronization – to push updates and synchronize user data across multiple devices
• SNS is used to silent push notification to all the devices associated with the given user identity
• User pools – A user pool is a user directory in Amazon Cognito. With a user pool, your users can sign in to your web or mobile app through Amazon Cognito. Your users can also sign in through social identity providers like Facebook or Amazon, and through SAML identity providers.

EC2

• Elastic Compute Cloud
• Allowing only to pay capacity you used
• OnDemand – Pay a fixed rate by hour or second, low cost, flexible, short time, unpredictable workload, first time, no upfront payment, no long-term commitment
• Linux by second, windows are by hour
• Reserved – Reserve 1 year to 3 year terms, steady state, predictable usage, and reserved capacity, standard RI, convertible RI, and Scheduled RI. (RI – Reserved Instances)
• Spot – bid price, flexible start and end times
• If you terminate the instance, you have pay for the partial hour
• If amazon terminate the instance (bid range), you don’t have to pay
• Dedicated – Physical, use existing software licenses.
• Public IP or Elastic IP address
• Evaluation logic – default deny
• For penetration test – request to amazon and get the approval
• Test allowed (after approval) only for EC2 and RDS
• Vulnerability and penetration test to Other resources are prohibited
• If the instance is terminated, you can find the reasons under ‘State transition reason’ label
• By default all accounts are limited to 5 elastic ip addresses per region
• Xen – underlying hypervisor
• Auto scaling group of spot instaces in primary and Auto scaling group of OnDemand instances in secondary is cost effective

Instance Families

• D2 – Density – File servers / Dataware / Hadoop
• R4 – RAM – Memory optimized Apps/DBs
• M5 – main choice General – app servers
• C5 – CPU – CPU intensive Apps / DNs
• G3 – Graphics – Video Encoding, 3D
• I3 – IOPS, high speed storage – NoSQL, DB, DataWare
• F1 – FPGA – Field Programmable Gate Array – Hardware Acceleration
• T2 – cheap general purpose – web servers, small DB’s
• P3 – Graphics (pics) – Machine Learning, Bit Coin
• X1 – Extreme Memory – SAP HANA, Apache spark
• H1 – High disk throughput – HDFS
• DIRTMCGFPX – dirty Melbourne ground (old)
• DR Mc GIFT PX – 2017 acronym
• FIGHT DR Mc PX
• F – FPGA
• I – IOPS
• G – Graphics
• H – High Disk Throughput
• T – Cheap general purpose (Think T2 micro)
• D – Density
• R – RAM
• M – Main choice for general purpose
• C – Compute
• P – Graphics (think pics)
• X – Extreme memory

EBS

• Elastic Block Store – allows to create to storage volumes and attach with EC2 instance
• Virtual disk
• Think of disk in the cloud attached to EC2
• Automatically replicated in AZ
• IOPS
• General Purpose SSD – GP2 – upto 10000 IOPS
• boot volumes
• low latency apps
• dev and test environments
• Provisional IOPS SSD (IO1) – more than 10000 IOPS – designed for I/O intensive applications,
• large relational or nosql DB, Mongo, Cassandra, Mysql, oracle etc
• Can provision upto 20000 IOPS per volumn
• IOPS – input output operation per second
• Throughput Optimized HDD (ST1) – magnetic not SSD
• Big Data
• Frequently accessed workloads
• Data warehouse
• Cant boot volume
• Minimum volumn is 500 GB
• Cold HDD (SC1)
• Lowest cost
• Less Frequently accessed workloads
• File server
• Cant boot volume
• Magnetic (standard)
• Bootable
• HDD
• Cheap
• Data accessed Infrequently
• Cannot mount 1 EBS volume to multiple EC2 instances – instead us EFS
• Can transfer reserved instance from one AZ to another
• Snapshot life cycle policy – for backups
• EBS optimized instance provides additional, dedicated capacity for EBS I/O
• Encryption availible only in certain instance types
• To improve performance
• EBS optimized instances
• Modern linux Kernal
• RAID 0 to maximize utilization of instance resources

Instance Store

• Called as Ephemeral storage
• Volume cannot be stopped. If the underlying host fails you will lose the data. But EBS backed instance can be stopped. You will not lose the data if the instance stopped.
• Cannot be attached or detached to other instances
• We can reboot both. Will not lose the data ??
• By default root volume will be deleted on termination. But EBS we can have option
• Usecase
• boot volumes
• transactional and no sql Db
• datawarehosuing
• ETL
• Cannot attach instance store volumes once the EC2 is launched
• few EC2 types doesnt support instace store volumes

EC2 Lab

• AMI – Amazon Machine Instance (when you create AMI, registerImage is the final process)
• AMI can be shared to other AWS account
• AMI can be changed to public
• If you make AMI, it will not be immediately available across all regions
• AMI’s can be only be shared within region. For other regions do copy
• Instance type – t2 micro
• Default VPC
• One subnet always will be in one AZ. Same subnet range will not be shared across AZ.
• Termination protection – by default its Off
• Monitoring – cloud watch
• Tenancy – dedicated
• Advanced Details – user input, Boot instructions (install PHP SDK, apache)
• Boot instructions (install PHP SDK, apache) – User Data section
• Storage – Root, Boot volume
• By default Root volume can’t be encrypted. Either bit locker, or encrypt when creating AMI or API.
• By Default Root EBS volume will be deleted on termination. Option can be changed
• Tags
• Security Group, SSH, SSL, HTTP.
• Source – Custom, Anywhere, My IP. Anywhere means open to world
• Key Pair – Pem file
• Same private key can be used for multiple EC2
• Status check –System Status check, Instance Status check
• Monitoring, Cloud watch – basic monitoring, CPU. Disk,Nw
• Standard monitoring – 5 mins
• Private key should be protected from read/write from other users.
• Unprotected private key file – do chmod 400
• Unix
• Change the permission chmod 400 *.pem
• ssh ec2-user@ -i
• sudo yum update -y -> to update security packages, -y to force update
• yum install httpd –y –> install apache
• cd /var/www/html
• nano – text editor
• nano index.html
• Add something
• Ctrl X to exit and save
• service httpd start
• chkconfig httpd on – starts automatically with boot
• Windows
• Putty Keygen to convert pem to ppk Security Group
• Its virtual firewall
• State Full
• 1 instance can have multiple security groups
• 1 security group can be assigned to multiple instances.
• Rules will be applied immediately. Restarting the instance not required.
• Inbound rule will automatically add outbound – State full
• In VPC, Network Access control list, State less, we need to add both inbound and outbound. State less
• Can’t deny the traffic. Everything denied by default. Add rule to allow traffic.
• Can’t block specific IP address, instead use Network access control
• All inbound traffic are blocked by default
• All Outbound traffic are allowed by default

EBS Volumn

• Its virtual hard disk
• Snapshot – point in time copy of volume.
• Volumns exists on EBS
• Snapshots exists in S3.
• Snapshots are incremental. This means only the blocks changed since last snapshot will be moved to S3
• First snapshot will takes time
• Snapshots encrypted automatically
• Volumes restored from encrypted snapshots also encrypted automatically
• You can share snapshots, only if unencrypted. This can be shared to other AWS accounts or public
• EC2 and EBS Volumes has to be in same AZ. Always
• Root volume – snapshot
• Modify volume – no downtime
• EBS volume can be changed on the fly. Including Size and Storage type
• From SSD, can’t change to HDD, but HDD can be changed to both SSD and HDD
• Standard (Magnetic) – you can’t modify the volume, others can
• To create / move volume to another AZ, create snapshot of root volume
• From snapshot, create volume, image, copy
• With Copy, you can move it to another region
• TO create snapshot for root, you should stop the instance. however you can create the snapshot when instance is running
• From Snapshot, both volume and image can be created
• AMI are regional, AMI can be launched where it’s stored.
• AMI can be copied to another region, using console, cli and aws EC2 api’s
• Snapshot of RAID array – take an application consistent snapshot. Stop the application and flush all cashes to disk. (Freeze the file system 0r unmount the RAID array or stop EC2 before snapshot) – stop any kind of IO
• you cant delete the snapshot of EBS volume that is used as root device of a registered AMI

Encrypt EBS Volume

• While creating volume, option to encrypt. Volumes can be attached or detached to running EC2 (Volume and EC2 should be in same AZ)
• Volume from encrypted snapshot will be automatically encrypted
• Volume from non-encrypted snapshot will be not be encrypted
• Can create encrypted snapshot from non-encrypted snapshot by creating encrypted copy of the non-encrypted snapshot
• Root volume will not be encrypted by default
• To encrypt root volume, 2 ways
• 1. OS level like bit locker
• 2. Create snapshot, Copy, while copying to do encrypt, from encrypted snapshot, create image and launch Image. Launching will not supported in free tier
• Encryption – Master Key – Default – aws/ebs
• lsblk – List the block devices
• file –s {/dev/xvdf} – to view the filesystem
• create file system before mount
• create file system – mkfs –t ext4 /dev/xvdf
• to mount go to root and – mount {/dev/xvdf} /filesystem (create filesystem dir in root)
• to unmount – umount –d {/dev/xvdf}
• You cannot encrypt the existing volume

RDS

• Relational Database Service
• SQLServer
• Oracle
• MySQL server
• PostgreSQL
• Amazon Aurora
• MariaDB
• Elastic Cache – cache the frequently accessed data
• Supports Memcached, Redis (open source caching engines)
• RDS us OLTP
• RedShift – OLAP
• RDS will always give you DNS. No ip address
• Assign security group. Important to access RDS from EC2
• RDS backup – 1. Automatic backup 2. DB Snapshot
• Automatic Backup – by default, point in time backup in a regular interval, can recover to any point within retention period, retention period is 1 to 35 days. It will take full daily snapshot and will store transactions throughout the day. So when we recover, aws choose most recent daily backup and apply transactions relevant to that day.
• Backup data stored in S3. You will free storage space in S3 equal to your DB size
• During backup window, there will be some latency
• Snapshot – Manual, user initiated. They are stored even after RDS is deleted. (in Automatic, backups will be deleted in RDS is deleted)
• When you restore it will be new RDS instance with new RDS endpoint. (both auto and snapshot)
• Encryption – all RDS supports encryption ??
• Encryption can be done at the time RDS creation. also ensure underlying instance type supports DB encryption
• Encryption at rest is not availible for DB instances running on SQL server express editionF
• Encryption is done using KMS – AWS Key Management Service
• Once RDS is encrypted, data from snapshot or back up also will be encrypted.
• We can’t encrypt existing RDS instance. we need take snapshot, copy, encrypt and restore
• Multi AZ-RDS – it allows you have an exact copy of your DB in another AZ. AWS does the replication for you. in case of DR, aws automatically failover to backup RDS.
• RDS endpoint is always a DNS name
• Multi AZ is only for DR not for performance
• Multi AZ is synchronous
• For performance go for read replica
• Read replica – it allows you to have read only copy of production DB
• Read replica – Asynchronous
• Read Replica – must have auto backup turned on, can have upto 5 read replicas, can have read replica’s of read replica (latency), each replica will have own endpoint, , can be promoted has own DB but this will break replication, can have read replica in second region.
• Can set read replica’s as multi AZ for mysql and mariaDB. PostgreSql is not supported yet.
• Read replica can be encrypted
• Read replica – might return stale data due the replication lag. cannot accept write queries
• RDS cannot be paused or stopped. Take the snapshot for future use and terminate
• Supports
• Auto backup
• Auto software patching
• Auto failure detection and recovery
• Scaling is not automated. User has to do some clicks
• When creating RDS, user must specify, multi AZ or not
• you cant RDP or SSH into RDS instance
• no charge to data transfer to replicas
• RDS reserved instances are available for multi AZ deployments
• Multi AZ provides high availability across AZ. Not across the region
• To improve performance
• Read Replica
• Elasticache
• Shards
• Multi AZ – synchronous replication
• Read Replica – axsynchronous replication

Aurora

• will run only in AWS infrastructure
• MySQL compatible
• 5 times better performance then MYSql
• storage autoscaling – 10 GB to start with
• 2 copies of your data in each AZ, with minimum of 3 AZ. so 6 copies of your data
• Storage is self healing. data blocks and disks are continuously scanned for errors and repaired automatically
• designed to transparently handle the loss of upto 2 copies of data without affecting database write availability and upto 3 copies without affecting read availability
• 2 types of replica
• Aurora replicas (currently 15) – in case if primary aurora is down automatically failover to replica
• MySql read replicas (currently 5) – will not fail over
• automatically stores the data in DB cluster across mutiple AZ in a single region. so for Aurora, multi AZ is not required
• For aurora, For DR, – create read replica in another region

Elastic Cache

• Elastic Cache – cache the frequently accessed data
• Good if your app / db is read heavy work loads – social networking, gaming
• Supports Memcached, Redis (open source caching engines)
• Memcached – Memory object caching system, No persistence, can grow or shrink simila to Ec2 auto scaling, individual nodes are expandable, automatic node replacement and auto discovery. Multi-threaded, no multi AZ capability
• Memcached usecases – caching is primary goal or offload your DB or simple cashing model as possible or planning to run large cache nodes and require multithreaded performance with utilization of multiple cores or you want to ability to scale you cache horizontally as you grow then use Memcached.
• Redis – In memory key value store, supports data structures like sorted set and List. It supports master slave replication and Multi AZ. So it’s used to achieve cross AZ redundancy.
• Redis – replication and persistence features
• Elastic Cache manages redis as a relational database, stateful, supports failover
• Redis usecases – more advance datatypes as lists, hashsets, sets or sorting and ranking datasets in memory like leaderboards or persistence of keystore is important or pub sub capabilities or you want to run in multiple AZ with failover then go with Redis.
• Caching strategies
• Lazy loading
• Write through
• Lazy loading – loads the data into cache only when necessary. Only at the time its requested
• Cache hit – returns the data
• Cache miss or expired – returns null – so your application needs to get the data form DB and update to elastic cache
• Write through – adds or update to the cache whenever data is written to the database
• Data In the cache is never stale

• aws
• certification
• Cheet-Sheet