OBJECT STORAGE / 03
Amazon S3
Durable object storage for any data
THE BIG PICTURE
An object's journey
- UploadFiles · backups · datasets
- Bucket + keyAn address for each object
- Storage classMatch your access pattern
- LifecycleTransition or expire objects
An object lives under a key in a bucket. Storage classes and lifecycle rules help match cost to access patterns.
01Objects, keys and uploads
- General-purpose bucket names are unique within an AWS partition; buckets are regional
- A PUT to an existing key replaces the whole object; bytes are never edited in place
- Maximum object size is 48.8 TiB; a single PUT request accepts up to 5 GiB
- Above 5 GiB use multipart: up to 10,000 parts, 5 MiB to 5 GiB; the last may be smaller
- Changing user metadata means copying the object onto itself; tags update in place
02Prefixes and request rates
- The namespace is flat; console folders are key prefixes, not real directories
- Per prefix: at least 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD per second
- Request capacity is per prefix, so spreading keys over prefixes raises throughput
- ListObjectsV2 returns at most 1,000 keys per call; follow the continuation token
- Reads after PUT, overwrite or DELETE are strongly consistent, with no stale-read window
03Storage classes
- Standard is the default: frequent access, multi-AZ, no minimum duration or retrieval fee
- Intelligent-Tiering suits unknown access patterns; it charges a per-object monitoring fee
- Standard-IA: infrequent reads; 30-day minimum, 128 KB minimum billed size, retrieval fee
- Glacier Instant and Flexible Retrieval have 90-day minimums; Deep Archive has 180 days
- One Zone classes keep data in one AZ, so suit re-creatable data; all target 11 nines
04Lifecycle and versioning
- Versioning keeps overwritten and deleted data; a plain DELETE adds a delete marker
- Versioning can be suspended but never removed; noncurrent versions are billed
- MFA Delete is enabled by the root user and needs MFA to permanently delete versions
- Lifecycle rules move objects to other storage classes or expire them after set days
- Lifecycle can expire noncurrent versions and abort stale incomplete multipart uploads
05Replication
- CRR copies to a bucket in another Region; SRR copies within the same Region
- Source and destination both need versioning, and an IAM role grants S3 access
- Only objects written after the rule exists replicate; use Batch Replication for old ones
- Permanent deletes of versions do not replicate; delete markers can be replicated
- Replication Time Control adds a 15-minute target and metrics for an extra charge
06Encryption
- All new objects are encrypted at rest with SSE-S3 by default; no setup is needed
- SSE-KMS uses a KMS key you choose; each KMS request is billed and quota-limited
- S3 Bucket Keys cut the number of KMS requests for SSE-KMS objects
- SSE-C needs your key on each request; new buckets generally block SSE-C writes by default
- Deny requests where aws:SecureTransport is false to enforce TLS in transit
07Access control
- Block Public Access is on by default for new buckets; account settings override buckets
- ACLs are disabled by default; grant access with IAM and bucket policies instead
- Cross-account access needs an allow in the caller's IAM policy and the bucket policy
- An explicit deny in any applicable policy overrides every allow
- Presigned URLs carry the signer's access and expire; SigV4 allows up to 7 days
08Static website hosting
- Enabled per bucket; S3 serves an index document and an error document
- The website endpoint is HTTP only; HTTPS needs CloudFront in front of the bucket
- Website endpoints require public reads; private CloudFront OAC uses the S3 REST endpoint
- Routing rules can redirect paths or hosts; the index document serves folder-style paths
09Event notifications
- Object events can go to SNS, SQS, Lambda or Amazon EventBridge
- EventBridge supports richer filters, multiple targets, archive and replay
- Legacy notifications filter only on key prefix and suffix
- Delivery is at least once; consumers must tolerate duplicates
- Events may arrive out of order; compare the sequencer field per key
10Logs, metrics and analytics
- Server access logs are best-effort and written to a separate target bucket
- CloudTrail data events record object-level API calls and are billed per event
- Daily storage metrics are free; request metrics are opt-in and billed per metric
- S3 Inventory writes scheduled object lists with metadata as CSV, ORC or Parquet
- Storage Lens reports usage and activity across accounts, Regions and buckets
11Pricing model
- Storage is billed per GB-month, with the rate set by storage class
- Requests are billed by type; PUT, COPY, POST and LIST cost more than GET
- IA and Glacier classes add a per-GB retrieval charge on read
- Data transfer out to the internet is billed per GB; transfer in is free
- Minimum-duration charges, lifecycle transitions and KMS requests are billed separately
12Common pitfalls
- Incomplete multipart uploads keep their parts billed until aborted; add a rule
- Noncurrent versions stay billed until a lifecycle rule expires them
- 11 nines covers hardware loss, not accidental deletes or overwrites; keep versions
- Bucket names with dots cause TLS certificate errors on virtual-hosted HTTPS; avoid dots
- Deleting a bucket needs it empty, including every version of every object
Go to the source
Use AWS documentation for current limits, availability, and pricing.