I finally caught a drive failure before it nuked the RAID array

Last week I was swapping out a hot spare in a client's server out near Denver and heard the tech on the phone next to me say "SMART looks clean, just re-seat it." That got me thinking, I've seen three separate RAID rebuilds die in the past 2 years because maintenance logs only showed pass/fail and nobody checked the raw reallocated sector counts. One drive had 412 reallocated sectors before it dropped out and took two weekends to recover. My takeaway is to check the raw values in smartctl, not just the health status line, and log the trend over time. A jump of even 20 sectors in a month means that drive is on borrowed time. How many of you actually pull raw SMART data on every drive during routine checks, or are you just trusting the green checkmark too?

2 comments

2 Comments

jamiew5318d ago

Blew my mind that 412 bad sectors got ignored because someone just trusted the green light.

james_campbell1218d ago

Cringed when I read that. How does a drive sit there with over 400 bad sectors and the software just shrugs it off because the light is green? That's like ignoring a check engine light because the car still starts. You'd think after the first handful of bad sectors, the thing would at least throw a warning flag, not just keep trucking along pretending everything is fine. Makes you wonder how many other "healthy" drives out there are hiding the same problem. I'd be checking my own drives tonight if I were anyone reading that thread.