From Indecent to Immoral
I just discovered that my company does ad campaigns for the U.S. Navy.
Busted DNS
We've been having considerable issues at work over the move of a particular DNS record on Monday. I had to take the domain over with little or no warning and as a result something appears to have gone awry. All this week, the boss and I have been wracking our brains trying to figure out what exactly is broken (if anything) and I thought I would post the progress here.
Ted, as the Elder Geek on my viewership list, if you have anything helpful to contribute, PLEASE comment or email me.
We have 2 DNS servers:
cohen.somedomain.com (aa.bb.cc.189) brazilian.somedomain.com (xx.yy.zz.12)
Cohen is the master, Brazilian the slave, but both are visible to the world:
dns-1.somedomain.com -> xx.yy.zz.12 -> Brazilian dns-2.somedomain.com -> aa.bb.cc.189 -> Cohen
Given that Brazilian is a slave of Cohen, the records for someotherdomain.com are the same. Here is what we have for that domain:
$TTL 14400
@ IN SOA dns-1.somedomain.com. root.someotherdomain.com. (
2006050301 ; serial
7200
3600
1209600
86400 )
someotherdomain.com. 14400 IN NS dns-1.somedomain.com.
someotherdomain.com. 14400 IN NS dns-2.somedomain.com.
localhost 14400 IN A 127.0.0.1
someotherdomain.com. 14400 IN A xx.yy.zz.43
dns-1 14400 IN A aa.bb.cc.189
dns-2 14400 IN A xx.yy.zz.12
dns-3 14400 IN A xx.yy.zz.12
dns-4 14400 IN A xx.yy.zz.12
dns-5 14400 IN A xx.yy.zz.12
; local
cohen 14400 IN A aa.bb.cc.189
; atlanta
brazilian 14400 IN A xx.yy.zz.12
entropy 14400 IN A xx.yy.zz.43
; cnames ---------------------------------------------------------------------
smtp 14400 IN CNAME cohen
www 14400 IN CNAME entropy
ad 14400 IN CNAME some.other.domain.thats.not.ours1.
as 14400 IN CNAME some.other.domain.thats.not.ours2.
content 14400 IN CNAME some.other.domain.thats.not.ours3.
; mx -------------------------------------------------------------------------
someotherdomain.com. IN MX 0 cohen.someotherdomain.com.
someotherdomain.com. IN MX 10 brazilian.someotherdomain.com.
Now you might see a problem with the dns-1 A records listed there. They were created because I had to take over this domain from someone who was running it independent of any other domain and was asked to instead link it to the somedomain.com's NS records as I have above. Nonetheless, there still appears to be requests for dns-#.someotherdomain.com out there so I created these A records as a stand-in. Please let me know if you feel this to be a Bad Idea.
I am aware of the fact that a CNAME to a record outside of the zone is considered Bad Form and likely even illegal, but since we had to point those domains to these other hosts, I know of no other way to do it.
Also, I have reservations as to the content of the 2nd line in the file. As this version was adapted from an example on another server I'd like to know if it's alright to have an SOA record for dns-1.somedomain.com in the someotherdomain.com zone file.
Lastly, Reverse-DNS for our subclass has been delegated to us as well. So, instead of our ISP managing reverse lookups, I've had to set that up on Cohen and slave it out to Brazilian (note that Cohen is on a different network).
Here are the contents of the reverse lookup file:
$ORIGIN 2-62.zz.yy.xx.in-addr.arpa.
$TTL 86400
@ IN SOA cohen.somedomain.com. root.somedomain.com. (
2006062705 ; serial
21600 ; refresh after 6 hours
3600 ; retry after 1 hour
604800 ; expire after 1 week
86400 ) ; minimum TTL of 1 day
IN NS dns-1.somedomain.com.
IN NS dns-2.somedomain.com.
2 IN PTR dallaire.somedomain.com.
12 IN PTR brazilian.somedomain.com.
13 IN PTR ethiopian.somedomain.com.
14 IN PTR survivor.somedomain.com.
15 IN PTR tsing-tao.somedomain.com.
16 IN PTR kenyan.somedomain.com.
22 IN PTR absinthe.somedomain.com.
23 IN PTR absolut.somedomain.com.
24 IN PTR bailey.somedomain.com.
25 IN PTR espresso.somedomain.com.
26 IN PTR laurier.somedomain.com.
27 IN PTR margarita.somedomain.com.
28 IN PTR martini.somedomain.com.
29 IN PTR mcclung.somedomain.com.
30 IN PTR packeteer.somedomain.com.
42 IN PTR anomaly.somedomain.com.
43 IN PTR entropy.somedomain.com.
I took take over this domain on Monday, but the servers hosting DNS for this domain had been offline since Friday (oops). When I brought up the domain on my own servers, there were the usual hiccups that could have been caught by some, but the experimental period was short.
Now the problem: We have two issues, one more pressing than the other, but they may be related.
A significant percentage (>5%, <30%) of sites running ad-code using this domain (as.someotherdomain.com) have been complaining of dead images. Instructions from our end asking them to flush their DNS have been met with "I did, but it's still broken"
One site administrator was quick enough to try out DNSReport.com and found this for as.someotherdomain.com:
A timeout occurred getting the NS records from your nameservers! None of your nameservers responded fast enough. They are probably down or unreachable. I can't continue since your nameservers aren't responding. If you have a Watchguard Firebox, it's due to a bug in their DNS Proxy, which must be disabled.
However, when I looked into this, I re-ran the report using only someotherdomain.com and everything checked out:
I'd very much like to know if this is indeed a problem or if I'm worrying about nothing.
From some Windows machines, the following command returns some very odd responses when querying Brazilian for information on any domain it controls:
nslookup someotherdomain.com xx.yy.zz.12 (root) nameserver = E.ROOT-SERVERS.NET (root) nameserver = F.ROOT-SERVERS.NET (root) nameserver = G.ROOT-SERVERS.NET (root) nameserver = H.ROOT-SERVERS.NET (root) nameserver = I.ROOT-SERVERS.NET (root) nameserver = J.ROOT-SERVERS.NET (root) nameserver = K.ROOT-SERVERS.NET (root) nameserver = L.ROOT-SERVERS.NET (root) nameserver = M.ROOT-SERVERS.NET (root) nameserver = A.ROOT-SERVERS.NET (root) nameserver = B.ROOT-SERVERS.NET (root) nameserver = C.ROOT-SERVERS.NET (root) nameserver = D.ROOT-SERVERS.NET *** Can't find server name for address xx.yy.zz.12: No information Server: UnKnown Address: xx.yy.zz.12 Name: someotherdomain.com Address: xx.yy.zz.43
Here's hoping it "fixes itself" somehow by tomorrow...
Burnt
It occurs to me at this point that using one-word titles for all of my blog posts will inevitably lead to duplicate post titles... a phenomenon I've managed to avoid thus far. Nonetheless, this is the perfect word for how I feel right now.
The past few weeks have been... difficult at best. Activism, girlfriend, promotion, x-girlfriend, raise, more responsibility, more activism, two (pro-bono) websites for two different family members, late-night meetings and little or no sleep. Hell, I haven't made dinner (save for one token evening) for over a month!
I was doing alright though. Sure dinner was kinda expensive to swing, but I was getting it done -- until this week. Ever since I've taken over the all of the office networking stuff I've felt completely overrun. It's too much. Not only that, but I honestly don't know what I'm doing in many cases. Often I find that I've setup a whole system of networking that "just works" but not completely right because no one ever showed me how to do it the Right Way. Programs I installed are spewing packets to places I know they shouldn't and things as important as DNS just don't seem to plug into the world right... ever.
It's starting to really wear me down.
This Is What Happens When You Like Your Job Too Much
You end up spending 14hours there.
I'm on the streetcar home, blogging on my laptop because I figure that when I get home, I won't be able to do much other than crash. We had a big server move today, we took all of our remaining servers off of the old system and put them behind my shiny new firewall. There were however, a few hitches:
- The old network used a weird system of public and private IPs on the same machines, so despite the fact that a box was behind a firewall, it still had *both* a 192.168.7.x IP as well as a routeable one. We were moving to my system which uses only unrouteable IPs so all the network configs had to be rewritten.
- The old IPs were very different from the new ones (192.168.7.x vs 192.168.0.x) so all of the machines had to be re-mapped, and, because a huge portion of the software was written by people who have a love-affair with IPs instead of names, I had to go into all the code for all the running software and re-map those IPs as well.
- MySQL replication was being stupid and didn't start up properly so we had to re-create all 40GB of data on the slave server and restart... that's a lot of data, even over the LAN, it took a long-ass time.
- I made a number of mistakes in the process that cost me a good deal of time, not the least of which was the occasional forgotten NFS share that was connected to the wrong subnet when another server tried to talk to it... not good.
Needless to say, it was not an easy night. I'm pretty worn out, but the boss was cool and said I could leave early tomorrow which works out nicely since Linuxworld is running then ;-) I have to admit though, as rough as tonight has been, it was kinda fun -- all this work, all those addresses, ports etc. and I actually knew what was going on. Pretty neat stuff ;-)
The Greater Good
I've been thinking about my job a lot lately. I think it's because I'm beginning to feel too comfortable.
This is how I figure it works (bear with me here).
There are a wide array of, for lack of a better term, Bad Jobs in the world. These jobs are Bad in the sense that they not only provide no real benefit to the world, but often, they work against the Common Good.
Some brazen examples of this include Shell Oil, Sony, Microsoft, or Nestle etc. All "Bad" companies in the sense that they hurt society and/or the planet in the pursuit of profit, but in truth, the majority of companies in the world are not so much in the business of Doing Good, as they are in that of making making money.
I've become more at odds with this whole concept because of what I've been doing in my off hours. From 10 - 6 I work for an online advertising company, but lately I've been spending a lot of my energies helping out with the Toronto Public Space Committee, an organisation which, at it's core, stands for the concept that Public Space is for the public, and not for the advertisers. In other words, I work for the enemy by day, and by night, I moonlight as someone who gives a damn.
Now this post is not meant to start an argument about the virtues of advertising. No one is ever going to convince me that a 10-metre high ipod billboard is "good for society" -- it's not, end of story. No, this is about something one of my teachers back at VFS told me: "Now you know everything I can teach you about web development" he said, "Use your powers for Good, not Evil".
What am I doing?
More and more I feel as though my involvement with the TPSC has less to do with the fact that I want to help, and more to do with me trying to somehow offset the work I'm doing during the day.
The world does not need more ads telling us to buy things we don't need. The world needs more faith, fewer guns, patience, and an honest attempt at slowing down. Can I honestly say that I contribute to any of these things in my forty hours each week?
But this all comes back to Good jobs and Bad Jobs. The world has an abundance of vacancies in Bad jobs because the majority of people in the world want to do Right and will often jump at the chance of supporting people doing Good Things. What's worse, the more detrimental to the world a company is, the more money it tends to have and so they can afford to hire more people. All this makes it very hard to find Good Work.
I need to start looking for a Good job. I just got another raise, and here I am, complaining about my job... I guess that means that there's still hope for me.
TheDarkHours.com DVD Release Today
One of the contracts I've been working on lately was released today, TheDarkHours.com is a site to fasciliate the release of the freaky-as-hell movie by the same name.
I wrote the backend code and worked with the client to setup new digs for the old site as their previous hosting provider sucked bigtime. I also did some limited Photoshop for the current site (as you can see, this is not my strong suit). Regardless, the site's ready and online... I just hope they pay me this time or I'm going to be very bitter.
Office Christmas Party 2005
I never used to be on time, but since I moved to Toronto, it seems that I'm either early or right on time for everything... AND EVERYONE ELSE ISN'T. I suppose it's karmic.
This year the venue (The Savoy) had tables for us to sit around -- a nice change from last years where we were standing pretty much all the time.
When the bosses were handing out presents, half of us crowded along the bar.
It's 1:30am as I write this. I got home about a half hour ago and when this is done I'm very much going to bed. The night was fun, but my ears hurt from the pounding music and my throat is sore from all the scream-conversation.
Is it just me or did that sentence make me sound like a really old man?
Anyway, I got to the party late and no one had arrived yet. I spent the first 15min or so just taking pictures of the empty room and experimenting with the lightlevels on my camera. When Charlie and the others arrived though, things started to pick up ;-)
Most everyone had brought their significant other, Colin came with his wife Maggie and Markus brought along a pretty blond girl named Julie I believe. There were, however a number of us who came stag and proceeded to flirt shamelessly with the various lady-folk. And how did I spend most of my night? Talking to a pretty girl of course ;-)
The bosses went all out for the night, buying red wine & champagne as well as funding an open bar (Markus insisted that I do a shot of liquid cocaine with him, while Istvan had me drink some nasty fruity thing.) Half way through the evening, Marc and Sam brought out presents for everyone: I got a gift certificate for futureshop and another one for indigo -- very cool, Colin got one for H&M -- not so cool for him, but I bet Maggie will appreciate it ;-)
I wish my pictures had come out better, but low-light shots are always so hard with a digital camera. These are some of the better ones.
And for those interested, here's a link to last year's party.
markus
about a week ago, markus posted a comment on my blog with the express purpose of finding out who's googling his name. you see, when someone comes to my blog by way of a search engine, i can see that search request, and when the request is for someone in the office, i let them know in case they care.
aileen had been boasting to markus that she'd been googled two or three times already and how his name never came up once (since his name wasn't on the site) so he posted his name in the comments and anxiously waited.
not more than a week later, his first hit. someone was googling for markus templer. and then today, another hit, this time, for pictures of markus templer. after letting him know about this most recent development though, one of his sales coworkers got an idea:
"wanna play a little joke on markus?" james asked. "wait a little while and tell him that you saw another google search for his name, this time for nude pictures of markus templer".
i really must say, my performance was exemplary. i totally had him, but i couldn't believe how good i was at the lying. we still haven't told him, i might just leave it up to james to break the news... either that, or he'll have to read this post.
at the very least, this post should improve his google rank ;-)
edit: As it turns out, there really are nude pictures of markus templer on the internet. just do a google image search and you find this. and yes, i asked. it really is him.
now it's all gonna be my fault
exciting time ahead for me. my boss has begun my slow but deliberate move from software development (php/mysql) to systems and network administration. in laymen's terms this means i'm going to write less and less website stuff and do more and more maintenance and management of the servers and the network itself.
today is a big day though. last night, a few of us got together and took the whole office off the old network and put it on my new one, behind my gateway running the firewall rules i setup over time (see my software page soon for a copy of the script). what's more, i'm managing the dns as well so now if anything (other than email) breaks or just doesn't work right, i get to fix it. the responsibility is cool, but i'm not sure that i like the new level of stress accompanied with it.
the dark hours
i've been developing the website for this movie over the past few months and thought i'd take this moment to plug the film so it can make a lot of money and make me look good ;-)
suzanne cheriton (red eye media), brett lamb (freelance art director and graphic designer) and i put this thing together and after running into problem after problem the site is finally ready to launch. i did all the backend code (php & mysql including some fun voodoo with apache's mod_rewrite), brett did all the graphic work and suzanne organised the whole damn thing. it's got trailers, actor bios, production notes and a news section with dynamic content (complete with rss). it's even got an invite to a free, private advance screening if you hurry ;-)
anyway, i haven't even seen the movie yet. i figured that since the trailer was so creepy, i should see it in the theatre first with everyone else. it opens this friday:
| toronto | paramount toronto (cineplex) |
| vancouver | granville vancouver (empire) |
| montreal | amc forum montreal (amc) |
| calgary | uptown screen calgary (indie) |
| edmonton | city centre edmonton (empire) |
| halifax | bayer's lake halifax (empire) |