Power outage

Sun, July 23, 2006 - 1:00 AM

Okay, there was a huge power outage in Monrovia, which is where the computers that I operate are located. In theory, my computer room is protected by a 500kva generator with a 48 hour supply of fuel, and two 400kva UPS's. One of those UPS's was just installed.

All of my computers (250 of them) except for one critical component are running off the older UPS. We got a second one because we maxed out the KVA on the first. It was just installed last month.

The new UPS is supporting the disk systems (SAN) for our merchandising application which runs the entire company. It houses a 1terrabyte database.

The UPS suporting that database failed at 7:30pm. This is a very bad thing.

It's now 12:30pm and we finally got the SAN running again. The disks remounted and the file integrity checks were fine. Now the DBA is taking over to verify and correct, if necessary, Oracle.

This was complicated by the use of what's called a write-behind disk cache. This means every time data is written to the disk, it is actually stored in memory for a few seconds. The oracle database is essentially corrupt because of this.

The DBA rolled back the database to a few minutes before the failure to eliminate this corruption. Unfortunately, this means we will be down for a few hours while this is performed.

---

Comments

---

Jocelyn Hall (Sun, July 23, 2006 - 8:55 AM)

You don't ever really sleep, do you? :)

Lee R. Green (Sun, July 23, 2006 - 9:14 AM)

You don't ever really sleep, do you? :)

I think he's cloned himself. How else can he go to movies and shows while simutainiously rebuilding a massive network?

Elizabeth Mahina (Sun, July 23, 2006 - 11:28 AM)

wow that's crazy! YOu are like a computer super hero!

Richard Lowe Jr (Mon, July 24, 2006 - 4:31 PM)

What is that strange word you are using "sleep"? What does that mean? ;-)

Actually, I get by fine on five hours of sleep each night. If I sleep more, I get cranky. If I sleep less, I get stupid. Five hours is perfect.

I have lots more time than many people partially because I don't watch television. I have a TV with a DvD player and a VCR, but I don't have it hooked to cable or an antenna. And no video games, newspapers or internet news. These things are all time wasters... so instead of watching television, I get my butt outside and do something, or read a book, write an article or write a comment in my blog... ;-)

---