What do a photo backup service, a machine-learning pipeline, and a company’s nightly database dump have in common? They probably all live in object storage. It is the quiet workhorse behind a huge share of modern cloud data, yet many people who use it every day could not explain how it differs from a plain old folder on a disk. This tutorial walks through the model from the ground up, shows the operations you will actually type, and points out the mistakes that trip up almost everyone the first time. (Cloud)
What Object Storage Actually Is
Traditional storage comes in two familiar shapes. Block storage presents a raw device that a filesystem or database formats and manages. File storage presents a hierarchy of directories and files with names, paths, and permissions. Object storage throws out the hierarchy. Instead, it manages data as discrete objects, and each object is made of three parts:
| Part | What it holds | Example |
|---|---|---|
| Data | The actual bytes (a photo, a log file, a video) | 4 MB JPEG |
| Metadata | System and custom key-value pairs | content-type: image/jpeg, owner: marketing |
| Key | A unique identifier inside a container | campaigns/2026/hero-shot.jpg |
Those containers are called buckets (or containers), and the namespace inside a bucket is flat. That last point confuses people constantly. A key like `campaigns/2026/hero-shot.jpg` looks like a path, but the slashes are just characters in a name. There are no real directories. The console and command-line tools fake the folder view for your convenience, which is exactly why beginners assume behavior that the storage layer does not actually provide. A useful encyclopedia overview of the object storage model explains the history and the contrast with block and file systems in more depth than a tutorial can.
The model became mainstream when Amazon Simple Storage Service launched in 2006 and turned it into a pay-as-you-go cloud service. Today the S3 API is the closest thing the industry has to a lingua franca: many independent object stores, including open-source ones, speak an S3-compatible dialect, so the operations you learn transfer almost everywhere. (What is Amazon S3?)
Setting Up Your First Bucket
Before any object exists, you need a bucket. The exact clicks vary by provider, but the concepts do not.
- Pick a region close to the systems that will read and write the data most often.
- Choose a globally unique name. Bucket names are DNS-style: lowercase letters, numbers, and hyphens.
- Decide on versioning. Turning it on keeps every overwrite as a separate version, which saves you from accidental deletions and ransomware overwrites alike.
- Set default encryption so new objects are protected without anyone remembering to tick a box.
- Add a lifecycle rule early, even a simple one that moves objects to cheaper tiers after 30 days or deletes incomplete multipart uploads after a week.
That last step is the one people skip. Incomplete multipart uploads are invisible in most listings but still bill you for the parts that landed. Cleaning them up automatically is one of the highest-value habits in object storage.
If you run your own infrastructure rather than rent it, the open-source Ceph RADOS Gateway documentation describes how to expose an S3-compatible endpoint on your hardware. The commands below work against it unchanged.
Core Operations: The Verbs You Will Actually Use
Almost everything in an object-storage workflow reduces to a handful of operations. In the S3 REST API, they map cleanly onto HTTP verbs: PUT uploads, GET retrieves, DELETE removes, and HEAD fetches metadata without the body. Buckets and objects are addressed either by path style or virtual-hosted style.
Here is the shape of a typical session with the AWS CLI, which is the most common teaching tool:
- Create a bucket: `aws s3 mb s3://my-unique-bucket-name`
- Upload a file: `aws s3 cp report.pdf s3://my-unique-bucket-name/reports/`
- List contents: `aws s3 ls s3://my-unique-bucket-name/reports/`
- Download: `aws s3 cp s3://my-unique-bucket-name/reports/report.pdf .`
- Delete: `aws s3 rm s3://my-unique-bucket-name/reports/report.pdf`
Two details matter more than the syntax. First, uploading the same key twice overwrites the object unless versioning is on. There is no “file already exists, overwrite?” prompt. Second, objects are effectively immutable. You do not edit an object in place; you replace it with a new version or a new key. That immutability is a feature, not a limitation, because it lets the storage layer spread data across many disks and machines without locking.
For large files, use multipart upload. The client splits the file into parts, uploads them in parallel, and the service stitches them together. It is faster, resumable, and required above certain size thresholds on most providers. If a transfer dies halfway, you resume rather than restart.
Consistency, Durability, and What the Numbers Mean
Two guarantees get quoted constantly, and both deserve a skeptical read.
Consistency describes whether a read immediately reflects a recent write. For years, S3 offered eventual consistency for overwrites and deletes, which meant a freshly updated object might briefly return stale data. That changed in December 2020, when S3 moved to strong read-after-write consistency for new uploads, overwrites, and deletes. Do not generalize that to every object store. Providers differ, and some S3-compatible systems still expose weaker guarantees for listings or overwrites. Check the documentation for the store you actually use.
Durability describes the odds that stored data survives the year without loss. S3 Standard is designed for 99.999999999 percent durability, often called eleven nines. That is a design target, not an audited guarantee, and different providers quote different figures. Durability is also not availability: a service can keep your data perfectly safe while being briefly unreachable. Plan for both, and keep an independent copy of anything you cannot afford to lose.
Mistakes That Catch Almost Everyone
A few pitfalls show up in nearly every first project.
- Treating keys as folders. Deleting a “folder” in the console is really a bulk delete of every key with that prefix. It is not atomic, and a partial failure leaves a mess.
- Making buckets public to “just test something.” Public buckets are a recurring cause of data leaks. Use presigned URLs with short expiry instead.
- Forgetting that listing is paginated. A naive list call returns a page, not everything. Loop until the continuation token runs out.
- Ignoring egress costs. Uploads are usually cheap or free; downloads and cross-region transfers are where bills grow. Design so compute sits near the data.
- Skipping lifecycle rules. Storage grows silently. Without rules, old versions and orphaned parts pile up forever.
- Assuming POSIX semantics. Object stores do not support file locking, appending to a file, or renaming a directory cheaply. Renaming one key is a copy plus a delete.
If you want to keep up with how these platforms evolve, the news section at 9481.org tracks cloud and infrastructure developments worth watching. (News)
When Object Storage Is the Right Tool
Object storage shines for unstructured data at scale: backups, media libraries, log archives, static website assets, data lakes, and model training sets. It scales horizontally almost without limit, costs far less per terabyte than block storage, and its HTTP interface means anything that can make a web request can use it.
It is a poor fit for a relational database’s data files, a shared filesystem that many servers mount, or any workload doing small random writes with tight latency needs. Those still belong on block or file storage. A practical rule: if your access pattern is “write once, read many times, address by name,” object storage wins. If it is “read and write small pieces constantly, address by offset,” it does not. (Blog)
Two features deserve a place in your design from day one. Versioning protects against overwrites and deletions. Object lock, sometimes called WORM for write-once-read-many, prevents anyone, including an administrator, from deleting objects for a set retention period. Together they form the backbone of most ransomware-resilient backup designs. The official S3 object lock documentation explains retention modes and legal holds in detail. The broader cloud category at 9481.org covers related infrastructure topics if you want to go deeper.
A Minimal Working Example
Here is a complete first exercise that touches every core concept. It assumes the AWS CLI is installed and configured.
- Create a bucket with versioning enabled.
- Upload a small text file.
- Upload the same key again with different content.
- List object versions and confirm two versions exist.
- Retrieve the older version by specifying its version ID.
- Add a lifecycle rule that expires noncurrent versions after 30 days.
- Delete the object, then restore it by removing the delete marker.
Step seven is the one that teaches the most. In a versioned bucket, a delete does not remove data; it places a delete marker on top. Remove the marker and the object reappears. Once that clicks, versioning stops feeling like magic and starts feeling like a safety net you can reason about.
For readers who prefer to learn from primary technical literature rather than vendor guides, the USENIX File and Storage Technologies conference proceedings publish peer-reviewed storage systems research, including work on erasure coding and object store design. It is dense, but it explains why these systems are built the way they are.
Conclusion
Object storage rewards a small amount of up-front understanding. Learn the three-part object model, accept that keys are names rather than paths, turn on versioning and lifecycle rules before you need them, and treat durability claims as design targets rather than promises. Do that, and the S3-style API becomes a dependable, boring tool you reach for without thinking. Skip it, and you will eventually discover your “folders” were never folders and your old versions were never kept. The gap between those two outcomes is about an afternoon of practice.
Frequently Asked Questions
What is the difference between object storage and a normal file system?
A file system organizes data in a hierarchy of directories with real paths, supports in-place edits and appends, and offers file locking. Object storage keeps data in a flat namespace where each object has a unique key, its bytes, and metadata. Objects are replaced rather than edited, and there are no true directories, only key prefixes that tools display as folders.
Do I need to use Amazon S3 to learn object storage?
No. The S3 API is widely implemented, so most tutorials teach S3 operations because those skills transfer to other providers and to open-source stores. You can practice locally against an S3-compatible server or a self-hosted S3 endpoint, then move to a cloud provider later with minimal changes.
What does eleven nines of durability actually mean?
It means the storage system is designed so that, in a given year, the expected probability of losing an object is about 0.000000001 percent. It is an engineering target based on replication and error-correction design, not an audited guarantee, and it says nothing about whether the service will be reachable at any given moment.
Can I edit a file stored as an object?
Not in place. You upload a new version or overwrite the key with new content. For small edits this means downloading, changing, and re-uploading. For large files, multipart upload lets you replace individual parts, but the result is still a new object version rather than an in-place modification.
Why did my storage bill grow when I deleted files?
Two common reasons. First, in a versioned bucket, deleting an object only adds a delete marker, and all prior versions still bill. Second, incomplete multipart uploads leave parts behind that most listings do not show. Lifecycle rules that expire noncurrent versions and abort incomplete uploads solve both.
Is object storage suitable for hosting a database?
Generally no. Databases expect low-latency random reads and writes at the block level, plus file locking and append semantics that object stores do not provide. Object storage is better suited to backups, archives, media, static assets, and analytics data lakes. Some analytical engines do read directly from object storage, but they are built specifically for that access pattern.
