Today slut-o-meter.com got posted on Digg.com. It took a couple of hours to get upgraded to the front page, but once it did, mysql on my server couldn’t handle it. Adam Smith quickly jumped in and limited the type of statistics that were being reported and offloaded a few static parts of the site to another server but it was not enough. Eventually all of the system’s RAM as well as its SWAP were completely full and after traveling to HE.net to check it out, I found the server’s console with the image seen at the bottom of this post. It was not possible to login via SSH or from the console itself so my server lost its 319 days of uptime. Because I had to reboot the server to fix these memory issues, I also upgraded to the current legacy kernel once it rebooted to take care of any potential security flaws. It now appears to be working just fine, but the load has also gone down quite a bit. In the morning when I get up, I’ll be installing some Linux OS on my Dell SC420 to place at the colo as a backup system. I’ll also configure this box to do secondary DNS, and be a box usable for offloading sites in the event of a flash flood like today.
