The Tacoma server (which houses everything but podcasts) was down for about four hours today.
From my administration blog:
Looks like I had a strange cascading failure today that began with a switch freaking out, which took the router offline and caused the main virtualization server’s ethernet driver to crap the bed.
We’re back now.
However, the virtualization server took it’s time and hiccuped a bit because of LVM inconsistencies and volumes that aren’t there. Now that the system is back online, I’m going to focus on getting that taken care of, and making sure the last few sets of backups are still good. This might mean things are intermittently slow as I throw commands at LVM.