S3

S3 use cases

• Backup and storage
• Disaster Recovery
• Archive
• Hybrid Cloud storage
• Application hosting
• Media hosting
• Data lakes & big data analytics
• Software delivery
• Static website

 

S3 bucket

• 모든 리전과 모든 계정 통틀어서 글로벌리 유닉한 네임을 가져아 함
• 버킷은 리전 레벨에서 정의되고, 생성됨

Naming convention

• No uppercase, No underscore
• 3-63 characters long
• Not an IP
• Must start with lowercase letter or number

 

S3 Objects

• object(파일)은 키를 가짐
• 키는 full path
  • s3://my-bucket/my_file.txt
  • s3://my-bucket/my_folder1/another_folder/my_file.txt

• 키는 prefix + object name
  • s3://my-bucket/my_folder1/another_folder/my_file.txt 

• 버킷 내에는 디렉토리 개념은 없음(UI에서는 그렇게 보이지만)
• 오브젝 맥스 사이즈 5TB(5000GB)
• 5GB보다 큰 사이즈를 업로드할 경우 멀티파트 업로드 사용(100MB 넘는 경우 멀티파트 업로드 권장됨)
• Metadata (list of text key / value pairs – system or user metadata)
  시스템이나 사용자에 의해 설정되어 파일에 관한 요소나 메타데이터를 나타낼 수 있음
• Tags (Unicode key / value pair – up to 10) – useful for security / lifecycle
•  Version ID (if versioning is enabled)

 

 

S3 Security

• user-based
  • IAM Policies  : IAM 유저에게 어떤 API가 허용 될 수 있는지 정의함

 

• Resource-Based
  • 버킷 정책
    • Object Access Control List - 더 정교하게
    • Bucket Access Control List  - less common

 

• IAM principal S3 에 접근가능한 경우
  • user IAM permission ALLOW OR resource policy ALLOW AND there's no explicit DENY

 

• Encryption : s3 오브젝트들을 인크립션 키로 암호화 가능

 

S3 Bucket Policies

•  JSON based policies
  • Resources: buckets and objects
  • Effect: Allow / Deny
  • Actions: Set of API to Allow or Deny
  • Principal:The account or user to apply the policy to
•  Use S3 bucket for policy to:
  • Grant public access to the bucket
  • Force objects to be encrypted at upload
  • Grant access to another account (Cross Account)

 

• 예) public access - use bucket policy

 

• 예) user access to s3 - IAM permission

 

• 예) EC2 instance access - use IAM Roles

• 예) Cross-Account Access - use Bucket Policy

 

 

Block public access

기업 데이터 유출을 막기 위한 옵션

 

 

Static Website Hosting

URL은 리전에 따라 다음 2가지 형식 중 하나임

http://bucket-name.s3-website-aws-region.amazonaws.com 
http://bucket-name.s3-website.aws-region.amazonaws.com

 

Versioning

• 파일의 버전관리를 할 수 있음
• 의도치 않은 삭제에 대해 복구 할 수 있고 이전 버전으로 롤백이 쉬움
• 버전 활성화시 이 전 버저닝 안되고 있던 파일들의 버전은 null
• 버저닝 중지시 이전 버전들 파일은 삭제되지 않음

 

S3 복제

• 복제하려면 소스와 타겟 버킷의 버저닝을 활성화해야 함!!
• CRR : Cross Region Replication
• SRR : Same Region Replication
• 버킷은 서로 다른 AWS 계정일 수 있음
• 복제는 비동기
• 적합한 IAM 권한을 S3에 줘야 함
• After you enable Replication, only new objects are replicated
• Optionally, you can replicate existing objects using S3 Batch Replication
  • Replicates existing objects and objects that failed replication

 

•  For DELETE operations
  • Can replicate delete markers from source to target (optional setting)
  • Deletions with a version ID are not replicated (to avoid malicious deletes)

 

• There is no “chaining” of replication
  • If bucket 1 has replication into bucket 2, which has replication into bucket 3
  • Then objects created in bucket 1 are not replicated to bucket 3

 

S3 Storage Classes

• Amazon S3 Standard - General Purpose
• Amazon S3 Standard-Infrequent Access (IA)

• Amazon S3 One Zone-Infrequent Access
• Amazon S3 Glacier Instant Retrieval
• Amazon S3 Glacier Flexible Retrieval
• Amazon S3 Glacier Deep Archive
• Amazon S3 Intelligent Tiering

• Can move between classes manually or using S3 Lifecycle configurations

• Durability(내구성):

• High durability (99.999999999%, 11 9’s) of objects across multiple AZ
• If you store 10,000,000(천만개) objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000 years(만년에 한번 유실)
• Same for all storage classes

 

• Availability(가용성):

• Measures how readily available a service is
• Varies depending on storage class
• Example: S3 standard has 99.99% availability = not available 53 minutes a year

 

S3 Standard

• 99.99% Availability
• Used for frequently accessed data
• Low latency(지연시간 짧음) and high throughput(처리량이 높음)
• Sustain 2 concurrent facility failures(두 개의 기능 장애를 동시에 버틸 수 있음)
• Use Cases: Big Data analytics, mobile & gaming applications, content distribution 

 

• Amazon S3 Standard-Infrequent Access (IA)
  •  For data that is less frequently accessed, but requires rapid access when needed
  • Lower cost than S3 Standard 
    
  • Amazon S3 Standard-Infrequent Access (S3 Standard-IA) 
    
    • 99.9% Availability
    • Use cases: Disaster Recovery, backups
  • Amazon S3 One Zone-Infrequent Access (S3 One Zone-IA)  
    • High durability (99.999999999%) in a single AZ; data lost when AZ is destroyed
    • 99.5% Availability
    • Use Cases: Storing secondary backup copies of on-premises data, or data you can recreate

 

 

 

• Amazon S3 Glacier Storage Classes
  •  Low-cost object storage meant for archiving / backup)
  • Pricing: price for storage + object retrieval cost
  • Amazon S3 Glacier Instant Retrieval
    • Millisecond retrieval, great for data accessed once a quarter
    • Minimum storage duration of 90 days
  • Amazon S3 Glacier Flexible Retrieval (formerly Amazon S3 Glacier):
    • Expedited (1 to 5 minutes), Standard (3 to 5 hours), Bulk (5 to 12 hours) – free
    • Minimum storage duration of 90 days
  • Amazon S3 Glacier Deep Archive – for long term storage:
    • Standard (12 hours), Bulk (48 hours)
    • Minimum storage duration of 180 days

 

 

 

 

 

 S3 Intelligent-Tiering

• Small monthly monitoring and auto-tiering fee
• Moves objects automatically between Access Tiers based on usage 
• There are no retrieval charges in S3 Intelligent-Tiering
• Frequent Access tier (automatic): default tier
• Infrequent Access tier (automatic): objects not accessed for 30 days
• Archive Instant Access tier (automatic): objects not accessed for 90 days 
• Archive Access tier (optional): configurable from 90 days to 700+ days 
• Deep Archive Access tier (optional): config. from 180 days to 700+ days

 

 

S3 Storage Classes Comparison