MinIO Upload: Choosing the Right Approach for Each Situation
Writing code to upload files to MinIO isn't hard. Three lines of SDK or one CLI command gets the job done. The real issue is choosing the right approach. Sticking with a server-relay architecture for browser uploads doubles your traffic; uploading a 2 GB file over an unstable network connection gets you nowhere; using multipart upload without a cleanup policy quietly eats your disk.
Three variables determine which upload method to use: file size, whether the client must go through a server, and network stability. Based on these axes, here's how to decide between a simple PUT, multipart, and presigned URLs.
Summary of the Three Approaches
- SDK (
fput_object/put_object): Connects directly from server-side to MinIO. Automatically selects between a simple PUT and multipart based on file size. - Presigned URL: The server issues a signed URL, and the browser or mobile client PUTs directly to MinIO using that URL. Bypasses server traffic entirely.
- mc CLI: For batch uploads and folder sync. Primarily used in CI/CD pipelines.
All three use the S3-compatible API internally, so connecting to MinIO with the AWS SDK also works.
Simple PUT vs. Multipart: The 5 MiB Threshold
When you call fput_object or put_object from the MinIO Python SDK, the SDK inspects the file size and automatically chooses between a simple PUT and multipart. The default threshold is 5 MiB (= 5 × 1,048,576 bytes).
fput_object takes a local file path; put_object takes a file object (stream) along with a length.
from minio import Minio
client = Minio(
"minio.example.com:9000",
access_key="YOUR-ACCESS-KEY",
secret_key="YOUR-SECRET-KEY",
secure=True,
)
# Upload by file path — SDK decides whether to use multipart
result = client.fput_object(
"my-bucket",
"data/report.csv",
"/local/path/report.csv",
content_type="text/csv",
)
print(result.object_name, result.etag)
When uploading directly from a stream source (an in-memory buffer, an HTTP response body, etc.), use put_object.
import io
data = b"hello, minio"
client.put_object(
"my-bucket",
"greet.txt",
io.BytesIO(data),
length=len(data),
content_type="text/plain",
)
For multipart uploads, you can tune part size with the part_size parameter. The default of 0 lets the SDK calculate the optimal size based on the file. On unreliable networks, smaller parts are preferable — only the failed part needs to be retransmitted, not the entire file.
result = client.fput_object(
"my-bucket",
"large-video.mp4",
"/local/large-video.mp4",
part_size=10 * 1024 * 1024, # 10MiB
)
content_type is the field people most often forget. Omitting it causes MinIO to store the object as application/octet-stream, and browsers won't handle images or PDFs correctly on download. Always specify it for image and document files.
Disabling SSL verification (by passing a urllib3 instance via the http_client parameter) is only acceptable in internal test environments. For production without a CA-signed certificate, register a self-signed certificate in the client's trust store instead.
The hidden cost of multipart is that any parts already uploaded will linger in storage if the upload is interrupted. This is covered in the last section.
Presigned URL: Delegating Upload Authority to the Client
Routing file uploads through your server doubles the traffic: client → server → MinIO. A 100 MB file consumes 100 MB of both server inbound and outbound bandwidth. Presigned URLs shorten this path to client → MinIO.
The server generates a signed URL and hands it to the client. The client sends an HTTP PUT request directly to that URL. MinIO validates the signature and stores the file. The file content never touches the server.
from datetime import timedelta
# Server side: issue a presigned URL valid for 1 hour
url = client.presigned_put_object(
"my-bucket",
"uploads/user-avatar.png",
expires=timedelta(hours=1),
)
# Pass this URL to the client
The default expiry is 7 days, which is also the maximum. The SigV4 signing spec caps expiry at 604,800 seconds (7 days), and MinIO follows the same limit. For upload use cases, keep URL lifetimes short — if the client uploads as soon as it receives the URL, even 1 hour is plenty.
The client-side PUT looks like this:
// Browser: upload directly to MinIO using a presigned URL
const file = document.querySelector('input[type=file]').files[0];
await fetch(presignedUrl, {
method: 'PUT',
body: file,
headers: { 'Content-Type': file.type },
});
There is one constraint. A single presigned URL is capped at 5 GiB, the S3 single-PUT limit. Beyond that, you need multipart presigned URLs — issuing a URL per part, uploading each part, then sending a separate Complete request. The added complexity is substantial, so in practice it's better to keep simple presigned PUTs for files under 5 GiB and handle anything larger with the server-side SDK.
Common Mistakes When Writing Python SDK Code
A frequent source of confusion is mixing up fput_object and put_object. The distinction is simple: use fput_object for local files, put_object for streams. When passing a stream to put_object, setting length to -1 makes MinIO use chunked transfer encoding, which breaks in some proxy environments. Whenever you know the exact length, always specify it explicitly.
To track upload progress, pass a callback to the progress parameter. You can use the progress hook from urllib3.request.RequestMethods that the SDK exposes, or build a wrapper around tqdm.
from minio.progress import Progress
result = client.fput_object(
"my-bucket",
"large-file.zip",
"/local/large-file.zip",
progress=Progress(),
)
The Progress() class is built into minio-py and prints a progress bar to the terminal.
mc CLI: Batch Uploads and Automation
The mc CLI is well-suited for folder sync and pipeline automation. Register an alias first.
mc alias set myminio http://minio.example.com:9000 ACCESS-KEY SECRET-KEY
# Upload a single file
mc cp ./report.csv myminio/my-bucket/data/report.csv
# Folder sync — removes files from the bucket that no longer exist in the source
mc mirror --overwrite --remove ./local-dir/ myminio/my-bucket/target/
The --remove flag deletes objects from the bucket that are absent from the source directory. If the local directory is empty, this can wipe the entire bucket, so always verify the source path explicitly in production.
For credential injection in CI/CD pipelines, the MC_HOST_{alias} environment variable handles everything in one place.
# GitHub Actions example
- name: Upload artifacts
env:
MC_HOST_myminio: http://${{ secrets.MINIO_USER }}:${{ secrets.MINIO_PASSWORD }}@minio.example.com:9000
run: mc cp --recursive ./dist/ myminio/releases/${{ github.sha }}/
This combines alias registration and authentication into a single line, and credentials are never directly exposed in logs.
Cleaning Up Incomplete Multipart Parts
When a multipart upload is interrupted, the parts already transmitted remain in the bucket. They are invisible to end users because they were never assembled into a complete object, but they still consume storage. In high-upload-traffic services, leaving this unaddressed causes disk space to shrink for no apparent reason.
To list incomplete parts in a bucket:
mc find myminio/my-bucket --incomplete
Automatic cleanup is configured through a lifecycle policy. An AbortIncompleteMultipartUpload rule tells MinIO to automatically delete incomplete parts after a specified number of days.
<LifecycleConfiguration>
<Rule>
<ID>abort-incomplete-multipart</ID>
<Status>Enabled</Status>
<AbortIncompleteMultipartUpload>
<DaysAfterInitiation>7</DaysAfterInitiation>
</AbortIncompleteMultipartUpload>
</Rule>
</LifecycleConfiguration>
This policy should be set at bucket creation time. Deciding to use multipart upload and configuring a cleanup policy are a package deal — by the time you try to add the policy later, parts have already accumulated.