This Is What Happens When You Like Your Job Too Much

You end up spending 14hours there.

I'm on the streetcar home, blogging on my laptop because I figure that when I get home, I won't be able to do much other than crash. We had a big server move today, we took all of our remaining servers off of the old system and put them behind my shiny new firewall. There were however, a few hitches:

• The old network used a weird system of public and private IPs on the same machines, so despite the fact that a box was behind a firewall, it still had *both* a 192.168.7.x IP as well as a routeable one. We were moving to my system which uses only unrouteable IPs so all the network configs had to be rewritten.
• The old IPs were very different from the new ones (192.168.7.x vs 192.168.0.x) so all of the machines had to be re-mapped, and, because a huge portion of the software was written by people who have a love-affair with IPs instead of names, I had to go into all the code for all the running software and re-map those IPs as well.
• MySQL replication was being stupid and didn't start up properly so we had to re-create all 40GB of data on the slave server and restart... that's a lot of data, even over the LAN, it took a long-ass time.
• I made a number of mistakes in the process that cost me a good deal of time, not the least of which was the occasional forgotten NFS share that was connected to the wrong subnet when another server tried to talk to it... not good.

Needless to say, it was not an easy night. I'm pretty worn out, but the boss was cool and said I could leave early tomorrow which works out nicely since Linuxworld is running then ;-) I have to admit though, as rough as tonight has been, it was kinda fun -- all this work, all those addresses, ports etc. and I actually knew what was going on. Pretty neat stuff ;-)