Now I’ve had issues with the site going down before, but these were weird. My traffic has actually started tapering off when Dreamhosts DNS load balancer went south. So though the servers where up, at times you couldn’t get to them (depending on your DNS cache). According the Dreamhost the issue was:
The problems regarding dns and our load balancer have been resolved. It was a combination of problems that caused the outage. The main problem was that the default udp settings on the load balancer did not have any timeout values set (BAD!!!). This was causing any udp connections that didn’t complete to fill up connection tables on the load balancer causing its CPU to spike and stop serving new requests. We sincerely apologize for this outage and are working on new plans that would prevent a similar outage in the future.
So they fixed stuff on their end, but just in case I’ve setup a secondary DNS with David Ulevitch’s (of OpenDNS) Free EveryDNS Service. So now there are now 7 name servers with the info. So technically there shouldn’t be this problem again.
Tags: Hosting, Interesting, Networking, Site Update
Leave a Reply