We are experiencing a major job delay due to a database outage that happened this weekend. I am looking into the extent of the issue, but it may be a several days to get things back to normal. It does not appear all data was lost for gameplay, and we hope to back populate as jobs catch up and we come up with a decent recovery plan. Although there is a chance blogs, challenges and related tasks will not be filled in for the missed days.
Will post more when I have it.
UPDATE 13:51 EST - 2010-09-17: Got the database recovered and running good again, had to do some work on the job server as well. Plan is to do the bulk of recovery work late Friday and Saturday night, into Sunday. This weekend we should have new blogs going out by the end, and things pretty back to normal if all goes well. Still looking into recovering blogs and data for the missing days.
UPDATE 00:15 EST - 2010-09-20: ARGH edition. Things didn't go as planned this weekend. Ran into some issues not related to the database. So things do seem a bit cursed trying to get things back to normal, the systems aren't exactly cooperating. The good news is, the pulls seem to be getting data each night, just everything else refuses to play nice. I am tied up all day tomorrow launching another product, but will have an update for you all Tuesday. Recovery is in sight, just not sure how far out yet, and how perfect it is.
UPDATE 08:12 EST - 2010-09-23: Blogs have been published for a day or two now, and they should be back to the normal schedule. Still working on cleaning up everything else, and looking into back populating blog possibly, however the time required might make it difficult
TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451611469e20133f43ac9ae970b
Listed below are links to weblogs that reference 360voice Job outage:
Comments
Thanks for the update!!!!
Posted by:Frostbite |September 14, 2010 at 11:07 PM
The update on the front page is appreciated.
Posted by:splitwig75 |September 15, 2010 at 07:59 AM
good luck guys....
Posted by:fastharry |September 15, 2010 at 10:47 PM
Does this outage have anything to do with Halo: Reach?
Posted by:Dr Technique |September 16, 2010 at 12:57 AM
Sorry to hear that this is happening. Thanks for keeping us in the loop.
Posted by:Tristanp |September 16, 2010 at 12:44 PM
Any further update yet?
Posted by:Regelos |September 17, 2010 at 01:39 PM
Thank you we appreciate the update!
Posted by:LausDomini |September 17, 2010 at 04:27 PM
Good luck with getting everything back up and running again. Hopefully all of the recovery goes smoothly, I'm curious to see how many people have been playing Reach.
Posted by:ypod |September 17, 2010 at 05:21 PM
Thanks Steve... Appreciate all of the work you guys still put into this site!
Posted by:Fshguy |September 17, 2010 at 09:13 PM
thanks for sorting this out hopefully everything goes to plan thanks for telling us about.
Posted by:blue mist unit6 |September 18, 2010 at 06:15 AMget trapper on it...
Posted by:fastharry |September 20, 2010 at 04:10 PMSo, Tuesday has come and gone again. Over here it's already Thursday! Not that I am sitting on the edge of my seat, but if you make a promise, you should try and keep it. Even if it is just a one-liner with "still working on it".
Hope you get it fixed and thank you for putting in the work. For what it is worth, I do appreciate this site.
Posted by:Jos |September 23, 2010 at 03:21 AM
yeah big d.
Posted by:justin charles |September 30, 2010 at 02:01 AM
Any word on the missed days? I'd really like to have them since as I am reading this correctly, they weren't lost, they just weren't posted. But I can understand if that's impossible to do.
Posted by:Dorian Grave |September 30, 2010 at 03:58 AMIs this ever going to be fixed?
I'm stuck in a crappy auto challenge that won't finish and so i can't joing a challenge that i actually want to.
I even deleted my account and rejoined and i'm still stuck in the auto challenge and update would be nice even if it just removing auto challenges that should have expired.
Do you not care about the site, a broken site and no update for weeks isn't very good.
Posted by:Shite Auto Challenge |October 05, 2010 at 04:48 PMI hear you - I don't particularly mind if the blogs aren't backfilled for that downtime. I'd just like my current autochallenge (due to end in -24 days time) to stop.
Posted by:Dale |October 08, 2010 at 12:02 PMFUCK YOU 360voice
When True achievements was down it was updated on a regular basis and work was carried it out to fix it.
You haven't given any update since September.
Go Fuck Yourselves
Posted by:Shite Auto Challenge |October 08, 2010 at 01:51 PM
Appreciate you keeping us up to date on this...
Posted by:free xbox 360 |October 08, 2010 at 05:50 PM
Ok, so this site is officially dead.
Posted by:D Lincoln |October 09, 2010 at 12:22 PMWhat is the deal guys.....I am not going to start dropping FBombs but I can see from your twitters that not ALL of you are totally busy.
PLEASE RESET THE CHALLENGES....
You have had long enough to fix the situation
This is only a preview. Your comment has not yet been posted.
\ABS\Auto Blog Samurai\data\Save Money with discounts on PS3, Xbox 360, Wii\xboxvoice\spinner.gif)
Your comment could not be posted. Error type: Your comment has been posted. Post another comment The letters and numbers you entered did not match the image. Please try again. As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.
Having trouble reading this image? View an alternate.
Your Information
(Name and email address are required. Email address will not be displayed with the comment.)
Name is required to post a comment
Please enter a valid email address
Invalid URL
Logo action
The GoodsMake your Xbox TalkThe original blogject
Forums
Email us!RSS Readers
Subscribe to email alertsCategoriesCommunityContestsE3EventsFeaturesMember of the MomentNewsPodcastPressReviewsRecent Posts360voice Job outageDonate to Gamers Outreach, win the coolest Xbox ever?Saw this and thought of PSonicE3 2010 - Forum is open!Most Popular Blogs for May 2010Most Popular Blogs for April 2010Remembering Angela Peterson, Gamer Chick - UPDATEMember of the Moment: PSonicMost Popular Blogs for March 2010Are you going to PAX East 2010? See you there!ArchivesSeptember 2010August 2010June 2010May 2010April 2010March 2010February 2010January 2010December 2009November 2009More...
No comments:
Post a Comment