This was supposed to be boring.
A classic hardware case: one faulty M.2 disk, a RAID-1 mirror, ESXi booting happily from a virtual disk. The kind of thing you expect to fix between two coffees.
Spoiler: it wasn’t boring.
The server was configured with two M.2 drives in RAID-1, hosting the ESXi boot volume. Simple mirror. One disk fails, the other survives, life goes on.
One of the drives—drive 0—was reported as faulty. The plan was straightforward:
Replace drive 0 → RAID rebuild starts → data is copied from drive 1 → done.
That’s how RAID-1 is supposed to work.
What Actually Happened
The technician replaced drive 0.
And… nothing rebuilt.
Instead:
- ESXi stopped booting
- The RAID configuration disappeared
- Both disks were shown as “Unconfigured Good”
No virtual disk. No mirror. No boot device.
At this point, we weren’t just dealing with a boot issue anymore.
Why This Was a Big Deal
This server wasn’t alone.
It was part of a 2-node vSAN cluster. If this node stayed down, the blast radius would grow fast:
- vSAN objects at risk
- Potential cluster rebuild
- Data evacuation
- Reconfiguration hell
In a 2-node setup, you don’t have much margin for error. One node limping is uncomfortable. One node gone is a crisis.
(And yes—rebuilding the vSAN cluster would absolutely have been on the table.)
The Second Attempt (and the “Wait… what?” Moment)
We brought the technician back onsite.
This time, we tried something different:
- Removed both disks
- Inserted only the known good disk (drive 1) into slot 0
- Powered on the server
And then—magic.
The server booted.
ESXi came up like nothing had ever happened. Same configs. Same host. Same data.
The Recovery
Once we confirmed the system was stable:
- We inserted the new/spare disk into slot 1
- RAID-1 rebuild started automatically
- Data synced cleanly
No vSAN rebuild.
No reinstall.
No data loss.
Just a very stressed admin finally exhaling.
What Did I Learn?
RAID controllers can be surprisingly literal.
Replacing “drive 0” doesn’t always mean “replace the broken half of the mirror”. Sometimes it means “break the mirror’s identity entirely.”
When things go sideways:
- Don’t panic
- Isolate the last known good disk
- Let the system prove it still boots
- Then rebuild around it