Fed-X
12-07-2002, 06:41 PM
As I am sure (and was notified by) many of you noticed, PTO was up and down throughout the Thanksgiving holidays. I am surprised I have any hair left after all the issues that crept up at one of the worst possible times.
For anyone that is interested, the datacenter that hosts our dedicated server (call it PTO) was hit with Denial of Service "hacker" attacks. During this event, it somehow triggered a fault in the appliance administrator and became very unstable. I was literally having to log in and turn the site back on every time it went down. How often did it go down anyway? Whenever it felt like it, for little or no reason at all. Completely random.. Which made it a REAL pain in my rear to keep up with. I didn't know the site went down until I tried to log on, or someone contacted me about it. After that, it was a couple mouse clicks and everything was back up and working fine, until it felt like going nutz once again.
I want to thank Darin_S, who I contract with to due server admin work. I had him on the phone or via instant messaging quite a bit while he was also out of town for the holidays. The sad thing about the entire event is that we still don't know exactly what was causing it. After much research, patching, etc., I've heard that there is a "bug" in the "Ensim" control panel software running on our server. I have also heard that our community's traffic is to blame. BS! Other web communities running our software have over 1000 concurrent members on at any given time, and runs just fine.
In the end, I got in touch with a programmer that deals with issues such as these and had him go in and build a custom fix. It seems to be working fine now, but I am just a little ticked off about it happening in the first place.
After MUCH though (and financial review) I have decided to move servers to a higher end facility, with better software, etc.
It's going to cost more but I don't care. I don't want to see PTO down for more than 30 minutes a year, and then only when I flip a switch to upgrade things, not when some anonymous error shuts us down.
The new server is what is called a "managed" server. There are people in the facility 24-7-365 who I can call and yell at, if need be. The hosting datacenter we are currently at is called an un-managed facility. They have people there but they only will fix a hardware problem, or physically reset the server if needed. Other than that, it's up to us if there are any problems. After this Thanksgiving holiday fiasco, I know having a completely unmanaged server isn't the best thing out there. I yell, they don't hear or care. :(
This new server also has an AWESOME interface for anyone I offer web hosting to. It's very easy to manage and use.
I have already signed off on the new server lease and the "box" (server) is being built this coming week. I am ending my lease with our current provider and we should transfer everything over to the new server around the 12th or 13th of this month. If all goes correctly, PTO should be offline for no more than 15-20 minutes. Maybe less?
I'll be sending out more info prior to the move.
Again, my apologies for anyone who had trouble reaching PTO over the Thanksgiving holidays. It's my intent to ensure this does not happen again. :)
Take care,
David
For anyone that is interested, the datacenter that hosts our dedicated server (call it PTO) was hit with Denial of Service "hacker" attacks. During this event, it somehow triggered a fault in the appliance administrator and became very unstable. I was literally having to log in and turn the site back on every time it went down. How often did it go down anyway? Whenever it felt like it, for little or no reason at all. Completely random.. Which made it a REAL pain in my rear to keep up with. I didn't know the site went down until I tried to log on, or someone contacted me about it. After that, it was a couple mouse clicks and everything was back up and working fine, until it felt like going nutz once again.
I want to thank Darin_S, who I contract with to due server admin work. I had him on the phone or via instant messaging quite a bit while he was also out of town for the holidays. The sad thing about the entire event is that we still don't know exactly what was causing it. After much research, patching, etc., I've heard that there is a "bug" in the "Ensim" control panel software running on our server. I have also heard that our community's traffic is to blame. BS! Other web communities running our software have over 1000 concurrent members on at any given time, and runs just fine.
In the end, I got in touch with a programmer that deals with issues such as these and had him go in and build a custom fix. It seems to be working fine now, but I am just a little ticked off about it happening in the first place.
After MUCH though (and financial review) I have decided to move servers to a higher end facility, with better software, etc.
It's going to cost more but I don't care. I don't want to see PTO down for more than 30 minutes a year, and then only when I flip a switch to upgrade things, not when some anonymous error shuts us down.
The new server is what is called a "managed" server. There are people in the facility 24-7-365 who I can call and yell at, if need be. The hosting datacenter we are currently at is called an un-managed facility. They have people there but they only will fix a hardware problem, or physically reset the server if needed. Other than that, it's up to us if there are any problems. After this Thanksgiving holiday fiasco, I know having a completely unmanaged server isn't the best thing out there. I yell, they don't hear or care. :(
This new server also has an AWESOME interface for anyone I offer web hosting to. It's very easy to manage and use.
I have already signed off on the new server lease and the "box" (server) is being built this coming week. I am ending my lease with our current provider and we should transfer everything over to the new server around the 12th or 13th of this month. If all goes correctly, PTO should be offline for no more than 15-20 minutes. Maybe less?
I'll be sending out more info prior to the move.
Again, my apologies for anyone who had trouble reaching PTO over the Thanksgiving holidays. It's my intent to ensure this does not happen again. :)
Take care,
David