Page 1 of 1
May 4-5 Outage Report an operational bullitin
#1
Posted 05 May 2010 - 09:15 AM
At approximately May 4 19:30 EDT, The datacenter facility the DoujinStyle server is operated from experienced a hailstorm that repeatedly interrupted the facility's power, the backup power systems at the facility were unable to keep up with this issue, the first outage corrupted the table containing the posts within the database for the DoujjinStyle forum. Multiple recovery attempts were attempted, however, due to the outages, none succeeded at that point. This issue continued, at which point it was decided the server was to be shut down as to avoid additional data corruption.
During the midnight hour EDT (May 5), the storm issue had ended, at which point the server was restarted and data recovery attempts were made, most of which were inefficiently slow and did not produce the desired results, it was made general consensus that a backup would have to be utilized and that data loss would occur.
At May 5 04:00 EDT, A "last ditch" effort was made, as to reconfigure the MySQL server to operate on a configuration that would permit it to utilize as much disk space as the recovery process required to complete the task.
At May 5 04:11 EDT, The recovery process had completed successfully with no data loss, table was noted to have 1,486,636 rows post-recovery, additional checks were made to other tables of the database, which minor abnormalities were corrected.
At May 5 04:30 EDT, The abnormalities had been identified and corrected, the database had passed all tests successfully. At this point, the forum software was re-activated and released to user use.
During the midnight hour EDT (May 5), the storm issue had ended, at which point the server was restarted and data recovery attempts were made, most of which were inefficiently slow and did not produce the desired results, it was made general consensus that a backup would have to be utilized and that data loss would occur.
At May 5 04:00 EDT, A "last ditch" effort was made, as to reconfigure the MySQL server to operate on a configuration that would permit it to utilize as much disk space as the recovery process required to complete the task.
At May 5 04:11 EDT, The recovery process had completed successfully with no data loss, table was noted to have 1,486,636 rows post-recovery, additional checks were made to other tables of the database, which minor abnormalities were corrected.
At May 5 04:30 EDT, The abnormalities had been identified and corrected, the database had passed all tests successfully. At this point, the forum software was re-activated and released to user use.
#2
Posted 05 May 2010 - 09:16 AM
Thanks for all of your hard work, Kradorex. We honestly really appreciate everything that you and Keiro both do here at DoujinStyle. You guys help keep this place up and running along with Holy, we would not be here without your work.

made by: rainbow_rena
Normal Theme Song: Pink Floyd - Comfortably Numb
Combat Theme Song: You're Not Alone (Final Fantasy IX)
#4
Posted 05 May 2010 - 04:46 PM
Well, at least IRC was still up. Man, that was a blast last night lol
Great to see that the server is up again though. Hopefully no more freak hailstorms come your way, 'cause I think the one that hit you guys started here in California first lol
Great to see that the server is up again though. Hopefully no more freak hailstorms come your way, 'cause I think the one that hit you guys started here in California first lol
Click Me :D
Spoiler
#7
Posted 05 May 2010 - 07:36 PM
Eyndyne, on 05 May 2010 - 12:46 PM, said:
Well, at least IRC was still up. Man, that was a blast last night lol
Great to see that the server is up again though. Hopefully no more freak hailstorms come your way, 'cause I think the one that hit you guys started here in California first lol
Great to see that the server is up again though. Hopefully no more freak hailstorms come your way, 'cause I think the one that hit you guys started here in California first lol
Yeah, IRC was badass, even though I did find it a bit TOO active.. 2 messages every second!

ReIncarnate, on 05 May 2010 - 10:51 PM, said:
Oh that delightful character, the space needle fellow! The bolded text is coming back to me now.
KOishi went to OSU.
I live in New Ohio.
Johnny Knoxville || Alphonzo
#8
Posted 07 May 2010 - 06:17 PM
Thanks a lot for all your hard work. I'm always a bit worried when I see the board offline but finally great to hear things going well
Thank you again for keeping up this excellent community
Thank you again for keeping up this excellent community
#9
Posted 07 May 2010 - 11:14 PM
I promised to Holy that I would not let this server be taken down, except for things out of my control, (like weather and DNS outages... stuff like that.)
So far, I'm holding to my promise quite well.
So far, I'm holding to my promise quite well.
Keiro D.
Systems and Network Administrator, EidolonHost.com CEO

A shattered Warrior... waiting for peace.
Systems and Network Administrator, EidolonHost.com CEO

A shattered Warrior... waiting for peace.
#10
Posted 09 May 2010 - 12:24 PM
Keiro, on 08 May 2010 - 07:14 AM, said:
I promised to Holy that I would not let this server be taken down, except for things out of my control, (like weather and DNS outages... stuff like that.)
So far, I'm holding to my promise quite well.
So far, I'm holding to my promise quite well.
Blame thy weather, I say.
Yes, DS is certainly holding up very well.

Page 1 of 1

Sign In
Register
Help


MultiQuote











