Emergency Downtime 10th/11th July 2012

All site related news and updates posted here
Forum rules
New topics created by admin and moderators only please.
Feel free to reply to any unlocked topics.

Emergency Downtime 10th/11th July 2012

Postby Wedgy » Tue Jul 10, 2012 9:17 pm

IA maybe down between:
23:00 - 08:00 BST Tuesday 10th - Wed 11th July

For anyone interested, had the following message from our server company:



Yesterday we upgraded the capacity on our server storage system. Unfortunately this has resulted in an unexpected degradation of performance while the storage RAID controller re-stripes data over the new disks.

To ensure data integrity the storage has essentially gone into a type of “safe mode”. It has turned off write buffering and it's SSD caching, both of which degrade performance, especially on disk writes. The time it takes to re-stripe the data is dependant on how many other things are competing to access the data at the same time.

We have been talking to the manufacturers and working hard to try and speed up the the re-striping. Unfortunately, no measures we've been able to take while the system is online have helped. We have therefore decided to take the server platform offline this evening in an attempt to get this resolved as quickly as possible. This will allow the storage system to re-stripe whilst no data is still being written to the disks and to use the full system resources. We hope this will speed up the process but at this stage do not know by how much.

We will be performing this emergency maintenance between:

23:00 - 08:00 BST Tuesday 10th - Wed 11th July

All services will experience an outage during this window. We're really sorry for any inconvenience caused and will follow up with more detail about why this happened and what we're going to do to make sure it doesn't happen again.
"Afterall, I am a kind Admin, a caring Admin, I listen to the people"
Image

Help promote us! Look HERE
User avatar
Wedgy
Site Admin
Site Admin
 
Posts: 7020
Joined: Mon Sep 29, 2008 2:09 pm
Location: Wiltshire
Also Known As: Wedginald

Re: Emergency Downtime 10th/11th July 2012

Postby Wedgy » Wed Jul 11, 2012 6:27 am

And we're back!

Looks like we were down all night :(
"Afterall, I am a kind Admin, a caring Admin, I listen to the people"
Image

Help promote us! Look HERE
User avatar
Wedgy
Site Admin
Site Admin
 
Posts: 7020
Joined: Mon Sep 29, 2008 2:09 pm
Location: Wiltshire
Also Known As: Wedginald

Re: Emergency Downtime 10th/11th July 2012

Postby Xplumberlives » Wed Jul 11, 2012 2:32 pm

Thanks Matt, I didn't notice a thing! ;)
"All modern aircraft have 4 dimensions: span, length, height and politics.
TSR-2 simply got the first 3 right. ”
— Sir Sydney Camm
User avatar
Xplumberlives
Boeing B-17 Flying Fortress
Boeing B-17 Flying Fortress
 
Posts: 41159
Joined: Wed Oct 01, 2008 9:27 pm
Location: Close to a former Harrier Airfield
Also Known As: ROCKETMAN

Re: Emergency Downtime 10th/11th July 2012

Postby Wedgy » Thu Jul 12, 2012 6:55 am

Guess you all noticed the 2nd time around. :(

Apologies, they emailed me yesterday morning to say the downtime still hadnt fixed the issue and they needed a further 12hrs from 8pm last night - well I didnt get to pick up that email until post 8pm last night due to work getting in the way, so too late to tell you all. Apologies again for that.

Hopefully will hear from them today to see if its all finally resolved!
"Afterall, I am a kind Admin, a caring Admin, I listen to the people"
Image

Help promote us! Look HERE
User avatar
Wedgy
Site Admin
Site Admin
 
Posts: 7020
Joined: Mon Sep 29, 2008 2:09 pm
Location: Wiltshire
Also Known As: Wedginald

Re: Emergency Downtime 10th/11th July 2012

Postby Jigsaw » Thu Jul 12, 2012 7:17 am

Not your fault mate. I did notice though :((

;)
God created man because a vibrator can't mow the lawn!

Image
User avatar
Jigsaw
Boeing 747
Boeing 747
 
Posts: 6455
Joined: Thu Feb 05, 2009 1:58 pm
Location: Scunthorpe

Re: Emergency Downtime 10th/11th July 2012

Postby Mayfly » Fri Jul 13, 2012 8:16 pm

I didn't, I was away with no internet - so couldn't have been a better time :D
In memory of a very dear friend - Mike Pearson

Very fond memories of Robbie Gilvary - DTs 1st Vulcan Captain who taught DT all he knew.
User avatar
Mayfly
Moderator
Moderator
 
Posts: 29684
Joined: Mon Sep 29, 2008 7:11 pm
Location: Bomber County

Re: Emergency Downtime 10th/11th July 2012

Postby Dan4th » Fri Jul 13, 2012 9:11 pm

HA!

I'll bet they called you up and asked
permission :p

D.
I like to do my planning retroactively!
User avatar
Dan4th
Boeing B-52 Stratofortress
Boeing B-52 Stratofortress
 
Posts: 14732
Joined: Thu Nov 06, 2008 9:42 pm
Location: Bangor, Maine, USA
Also Known As: Token Murkin


Return to News and Updates

Who is online

Users browsing this forum: No registered users and 0 guests