1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
|
# MinIO: It Works, But It Hates Me
*By someone who survived a 150,000-file sync and lived to tell the tale.*
---
MinIO is fast. It's lightweight. It's compatible with Amazon S3. It’s everything you want in a self-hosted object storage system.
Until you try to **use it like a filesystem**.
Then it becomes the most temperamental, moody, selectively mute piece of software you've ever met.
---
## What I Was Trying to Do
All I wanted was to migrate ~40GB of Mastodon media from local disk into a MinIO bucket. Nothing fancy. Just a clean `rclone sync` and a pat on the back.
---
## What Actually Happened
- **Load average spiked to 33**
- `find` froze
- `rclone size` hung
- `zfs snapshot` stalled so long I thought the server died
- The MinIO **UI lied to my face** about how much data was present (5GB when `rclone` said 22GB)
- Directory paths that looked like files. Files that were secretly directories. I saw `.meta` and `.part.1` in my dreams.
---
## The Root Problem
MinIO is **not** a filesystem.
It's a flat key-value object store that's just *pretending* to be a folder tree. And when you throw 150,000+ nested objects at it — especially from a tool like `rclone` — all the lies unravel.
It keeps going, but only if:
- You feed it one file at a time
- You don’t ask it questions (`rclone ls`, `rclone size`, `find`, etc.)
- You don’t use the UI expecting it to reflect reality
---
## The Fixes That Kept Me Sane
- Switched from `rclone ls` to `rclone size` with `--json` (when it worked)
- Cleaned up thousands of broken `.meta`/`.part.*` directories using a targeted script
- Paused `rclone` mid-sync with `kill -STOP` to get snapshots to complete
- Used `du -sh` instead of `find` to track usage
- Lowered `rclone` concurrency with `--transfers=4 --checkers=4`
- Drank water. A lot of it.
---
## The Moral of the Story
If you're going to use MinIO for massive sync jobs, treat it like:
- A **delicate black box** with fast internals but fragile mood
- Something that **prefers to be written to, not inspected**
- An S3 clone with boundary issues
---
## Final Thought
MinIO *does* work. It's powerful. It’s fast. But it also absolutely hates being watched while it works.
And you won't realize how much until you're 100,000 files deep, snapshot frozen, and `rclone` is telling you you're doing great — while the UI smirks and says you're at 5 gigs.
MinIO: It works.
But it hates you.
---
**Filed under:** `disaster recovery`, `object storage`, `sync trauma`, `zfs`, `rclone`, `why me`
|
