Should You Zip? Sending Thousands of Small Files vs One Archive

2026년 8월 7일

2 min read

One archive almost always wins — but not for the reason most people assume. It is per-file overhead, not compression, that decides.

Archive them. Sending 50,000 small files individually is dramatically slower than sending one archive of the same total size, and the reason is per-file overhead rather than compression.

Where the time actually goes

Every file in a transfer costs a fixed amount of work independent of its size: a request to start it, a response, a database row, a filesystem entry, and the same again on the recipient's side when they save it. Call it a fraction of a second. Multiply by 50,000 and the overhead alone runs into hours — before a single meaningful byte has moved.

One 20 GB archive pays that cost once.

Compression is a secondary benefit

Whether the archive is smaller depends entirely on the content. Text, CSV, SQL, and source code compress by 70–90%. JPEG, PNG, MP4, and anything already .gz compress by essentially nothing, and spending CPU time trying is wasted effort.

For already-compressed content, use a stored archive — zip -0 or plain tar with no compression. You still get the overhead win without the pointless CPU cost.

The other advantages

Structure survives. Directory layout is preserved, which matters when the recipient needs to reproduce a result.
Atomicity. One archive either arrives or does not. Fifty thousand files can arrive 99% complete, and finding the missing 1% is miserable.
One checksum. Verifying integrity is a single command instead of a manifest.
Resume works properly. A chunked upload of one large file resumes cleanly; a half-finished batch of thousands is harder to reason about.

When not to archive

If the recipient only needs a handful of the files, or will pick through them selectively, sending them separately is kinder. And if the total is small — a few dozen files — the overhead is not worth thinking about.

Practical recipes

# Text-heavy data — compress
tar -czf dataset.tar.gz dataset/

# Already-compressed media — bundle without compressing
tar -cf footage.tar footage/

# Cross-platform, recipient on Windows
zip -0 -r footage.zip footage/