Linux Server/NVR Rebuild

[wct097]

I have a little Linux box that I use for a Ubiquiti NVR, Unifi controller, and NAS (samba). It's an old Dell Optiplex 390 with a core i3 that I bought as an employee purchase from my old employer. Jammed a bunch of old/spare drives into it and configured them with software RAID1.

It's been running for a while and I could use more space for the NVR and samba share, and one of the drives in the boot raid array has failed. I'm going to replace the aging case/cooler fans, add a WD Red 4tb NAS drive, and replace the failed 250gb raid member with a 240gb SSD. I'm also going to pull some of the smaller drives and add a drive cage so that I can mount the drives rather than having a couple just laying around loose.

The goal here will be to replace things without losing data and having to reinstall the OS and apps. I think I'll need to shrink the root volume a tad, then specify that the HDD is "write-mostly" in an effort to make the SSD effectively the primary read drive in the array.

Pics to come.

 

[BrandonM7]

Sounds like the kind of thing that will either go perfectly smoothly or end in total loss of everything.

 

[wct097]

Yeah. I'm prepared either way. Would just be kinda cool to see it go right.

This particular case has two 5.25" bays, one populated with a DVD drive. I have the drive unplugged so I can use that SATA port for a HDD. I also have a cheap SATA card so that I could just shove the thing full of old drives.

Specs are as follows:
i3-2120
8gb RAM
/dev/md0 (RAID1) 2x Seagate 3.5" 250gb (one failed, surprise surprise)
/dev/md1 (RAID1) 1x Seagate 3.5" 500gb, 1x WD 2.5" 500gb
/dev/md2 (RAID1) 2x Seagate 2.5" 320gb

I ordered this:

Amazon.com: EverCool Dual 5.25 in. Drive Bay to Triple 3.5 in. HDD Cooling Box: Gateway

Buy EverCool Dual 5.25 in. Drive Bay to Triple 3.5 in. HDD Cooling Box: Fans & Cooling - Amazon.com ✓ FREE DELIVERY possible on eligible purchases

www.amazon.com

The machine has two 3.5" bays in the bottom that house the 2x 250GB Seagate, the rest are haphazardly bolted to the 5.25" frame and the floppy drive frame (no external bay), and the 2x 320s are laying loose.

End-state for this weekend, I'd like to have it as follows:

/dev/md0 (RAID1) 1x Seagate 3.5" 250gb HDD, 1x 240gb SSD - both mounted in the lower drive bays.
/dev/md1 (RAID1) 1x WD 2.5" 500gb HDD mounted with the 240gb SSD in the lower bay, 1x 500gb Seagate HDD mounted in the new 5.25" bay
/dev/md2 - deleted
New WD 4tb 3.5" HDD - standalone with two partitions, 1tb for cameras, 3tb for files

Eventually I'll probably ditch the 500gb Seagate and add another couple WD Red drives in the 5.25" bay and make a RAID 5 array out of it, but for what I'm using it for, the single 4tb will suffice for a while. When I build my next workstation (possibly Ryzen 3000 based this summer), I'm likely going to go with dual NVMe SSDs in a 1tb boot/ 2tb data arrangement. The goal being to be silent, so I'll avoid installing spindle drives for bulk storage. Instead I'll put additional drives in this machine to serve for bulk storage.

I realize this is boring to most people, but I'm a geek and like playing with this stuff, even if it's a cheap home server and not a real professional grade piece of hardware.

 

[wct097]

Well, looks like I'm going to be rebuilding from scratch unless I can get grub installed on the secondary RAID1 volume to make this turd boot.

Not sure exactly what is going on here. Apparently /dev/sda and /dev/sdb were members of the RAID1 array. /dev/sda failed, and on reboot the machine wouldn't boot. I think it has to do with GRUB not being installed on both drives and the primary drive with it installed being bad.... though I still get a GRUB menu. I've managed to boot with a rescue CD and mount the RAID array and can see the data. Attempting to clone that raid array to a bare drive and see if I can then boot from the cloned drive. I imagine that if I'm successful, I'll have to boot into rescue again to disable mdadm raid arrays and change the fstab.

What a pain in the ass! Thanks Seagate! Two of the 5 in this box are bad. Typical. I'm trying to recover to the 500gb WD now.

 

[wct097]

Well crap. Looks like a full rebuild. I can get the system to boot to a login screen but the keyboard doesn't respond and the network interface doesn't come up. I've spent way too much time screwing around with it and am just going to start over from scratch. New build will be:

240gb SSD - boot/root partition
500gb WD 2.5" & 500gb Seagate 3.5" in RAID1 - redundant file storage
4TB WD Red 3.5" - standalone file storage and video storage

Parts should be here today, but it'll be Sunday or Monday before I get this done.

When I build my new workstation this summer, I'll probably pick up some more NAS drives. Depending on pricing, I could add two 4tb and convert to RAID5, or buy 3 larger ones and ditch the 500gb drives. 5x3.5" or 2x2.5" and 4x3.5" drives is about the most I can really fit in this box.