Service Status
current status: no known issues
last updated: 27/3/08, 12:01 (UK local time)
To let us know about any issue relating to desalo-hosted sites or mail, please:
- call 0845 09 555 25 during normal UK working hours,
or (at any time):
- SMS +447906812921
- email
For non-emergency technical support, or for sales enquiries, please:
- call 01896 820161 or 0845 09 555 25 during normal UK working hours,
or
- email
last known service issues:
web and email access issue for some desalo customers on Sat 22/3/08 and
Tues 25/3/08 - resolved
Some of our US-hosted customers were affected by scheduled maintenance to carry out a large-scale server move and systems upgrade overnight in the US. Planned downtime was 12 hrs, and most sites were up and running again within 8 hrs, but there have been knock-on effects. Engineers are aware of these and have been working to clear up any issues.
We noticed that four of our customers' sites were down for between 20 minutes
and, intermittently, around 2 hours between 20:52 and 22:45 (UK local time)
on Tuesday, 25/3/08. Again, reassuringly, engineers at the data centre housing
the affected cluster were aware of the fact and working on the problem before
we even noticed it.
Update, 27/3/08, 09:45: Two sites were extremely sluggish for around four hours last night, but all seem to be performing optimally now, and have been since around 23:00 GMT, 26/03/08.
The issues appear to be resolved, but we'll naturally keep an even closer eye on things, just to make sure.
Update, 27/3/08, 12:01: The two sites that were sluggish last night have been fine in all of this morning's tests. The adverse effects of last Saturday's move appear to have now been resolved. We use an external, independent monitoring service that tests the availability of sites every ten minutes, and it has not reported any problems since 25/3/08. Speed tests are also back to normal, but please let us know if you notice any issues with your own site.
web and email access issue for some desalo customers on Tue 24th July 2007 - resolved since late Tue 24th/early Wed 25th July 2007
Here's a statement from the company hosting the affected sites (who had nothing to do with the affected data centre, but were hit because the transatlantic carrier was in the centre - arrggh!):
====
The problem between here and the UK (and many overseas locations) seems
to have stemmed from an outage with a large service provider called
"Global Crossing". I believe there was a major outage with this
provider, and it affected pretty much anyone who's [sic] path between
themselves and our datacenters went through Global Crossing...
====
Global Crossing can in turn quite justifiably claim that they were not to blame, as the outage was down to backup generators not kicking in at the data centre in San Francisco that they are housed in. Such data centres are supposed to cope with anything the world can throw at them, but this time their backups failed. Some huge sites were affected.
Slashdot had fun with the incident, especially as the date centre concerned had that very day issued a press release announcing more than 2 years without any downtime: http://hardware.slashdot.org/article.pl?sid=07/07/24/2210255
Again, our apologies to any of our customers who found that they were affected by the outage.
posted by Chris at 00:50
earlier announcement: 10:52, Wed 25th July, 2007
web and email access issue for some desalo customers since morning of Tue 24th July - now resolved
This issue seems to have been sorted out. The three sites that I found were affected yesterday are now working again.
Please let me know if you notice any continuing problems, and thanks again for your patience.
posted by Chris at 10:52
earlier announcement: 21:03, Tue 24th July, 2007
web and email access issue for some desalo customers since morning of Tue 24th July
This issue is still going on for some users, unfortunately.
It's not a server issue (though it still looks like a DNS problem to me, as it's consistently affecting the same domain names). On the same servers there are a) sites that are fully accessible from the UK, b) sites that are accessible from Germany or other parts of Europe, but not the UK and c) sites that are accessible from neither the UK nor other parts of Europe that I've been able to hop through from when testing.
Naturally, I'll keep watching out for news and will keep you posted.
Many thanks for your patience.
posted by Chris at 21:03
earlier announcement: 15:55, Tue 24th July, 2007
There's discussion elsewhere on the web as to whether it's a routing issue or a DNS issue, but whichever it is, the fact remains that a few people in the UK are not seeing their sites and are not able to get their mail via POP3 (i.e. they can use mail2web.com to access their mailboxes, but not a program such as Outlook, Thunderbird or Apple's Mail).
In contrast to our earlier announcements, it now looks as though it's not the whole of Europe that's unable to access the rather small number of affected sites, but perhaps only the UK (though this is far from clear). What is clear is that many (perhaps all) of the sites that people in the UK find it impossible to access can be accessed from other parts of Europe. There have been confirmations of such findings from Finland and Germany.
If you'd like to test whether your own site is available from Germany, try this proxy:
http://anonymouse.org/
It's free and quick, and you don't need to register.
If you are a desalo customer and you'd like to talk any of this over, please feel free to call me in the UK on 01896 820255 or 01896 820161 at any time up to 23:00 tonight (24 July 2007) and from 10:00 tomorrow morning if things haven't been fixed by then.
We'll update you on this page as soon as things appear to have been resolved, or if we receive more information. Oh, and just to be clear, this is not something we can do anything about ourselves. Nor is it something our immediate suppliers can do anything about. All we can do at the moment is pass reports up the line, monitor the situation and report back to you here as soon as we find out more.
Many thanks for your patience.
posted by Chris at 15:55
earlier announcement: 14:52, Tue 24th July, 2007
Not all of our sites using US servers are affected. Please let us know if yours is and we'll look into it and pass reports up the line to technical support staff for them to pass on to Global Crossing. Might be due to a corrupted DNS registry as it does NOT appear to be server dependent.
We'll update you on this page as soon as things appear to have been resolved, or if we receive more information.
earlier announcement: 14:00, Tue 24th July, 2007 (edited at 14:52 to improve factual accuracy as more information became available)
Some customers are finding that their sites and emails are currently (afternoon of July 24th 2007) unavailable.
There have been reports that this is due to a transatlantic link run by a connection provider called Global Crossing (http://www.globalcrossing.com/) going down and a data centre run by Dreamhost thus becoming unavailable for European visitors (though not the rest of the world).
This means that a) you may not be able to see your site from the UK at the moment, b) you may not be able to access mail from mailboxes associated with affected sites via conventional means (though may still access them via mail2web.com or similar services) c) that many people will still be able to see your sites, even if we in the UK can't see them just now, and will still be able to send you mail.
Dreamhost have contacted other UK customers to let them know that they are aware of the issue, but that it is, unfortunately, outside their control.
I'm not aware of any announcement as yet from Global Crossing, or if any re-routing work is underway, but can only assume that it is.
Apologies if this incident is affecting you and your sites/mail. One upside is that you should not lose any mail as a result. In our tests, we've still been able to send mail to affected domains and have been able to access mail via mail2web.com (or gmail for sites that forward their mail to gmail backups), just not via POP3. That means that mail is still arriving, but that it's currently inaccessible via the usual means. That being so, it should be there waiting for you once you do get access again if you find you can't access your mail at this very moment. We'll keep an eye on things and will let you know (here on this page) as soon as we know any more.
There are currently no other issues that we are aware of.
Please email or call us on 01896 820255 if you have specific problems
to report, or would like help with
support issues.
Earlier Issues, resolved since ~16:30,
Wed 27th June:
Due to the severe recent flooding
in Sheffield and Leeds, some customers
that we provide UK web and email hosting
for may have experienced technical
difficulties on Wed 27th June.
Sorry if this caused you
any problems.
Unfortunately, the event was beyond
our control (and beyond the control
of our suppliers), but we'll keep you
posted on this page if we hear
of further developments.
At the moment
(20:33, 27/6/07) things seem to be
back to normal.
Here are status reports from Donhost/Pipex,
who provide the affected web and email
data centres:
|
Current Status
These issues may have had some impact on your service recently, but they are now resolved. We apologise for any inconvenience caused.
Past Status Reports
| Time |
12:58 Wed 27 Jun 2007 |
| Services Affected |
All servers |
| Details |
We are currently experiencing severe network
connectivity issues in our Leeds datacentre, which is affecting
many services for which we sincerely apologise. Engineers
are attending and we are treating this with the highest
urgency to get services restored as soon as possible...
...We shall update here as soon as we have any further information. Thank you
for bearing with us during this time. |
| Update |
13:28 Wed 27 Jun:
The engineers working on this issue have advised that our primary fibre link has been damaged by the severe flooding which has taken place in Sheffield over the last few days.
As of Tuesday of this week the failover link has been in use, however a line break occured at approximately 12pm Wednesday 27 June, resulting in sub-optimal routing for all Pipex customers with services terminating in our Manchester and Leeds data centres, as the third standby line is heavily loaded network traffic.
Presently engineers are working on bringing additional capacity to the final standby line while the issues relating to the Primary and Secondary are addressed.
Unfortunately, the cause of these issues are beyond the control of Donhost or Pipex, the causes for these issues namely being extreme weather conditions in this region for the Primary link, and human error caused by third party workmen on the Secondary line. We apologise for any inconvenience caused by this matter and once again assure you that our engineers are working on rectifying the problem as soon as they are able.
15:05 Wed 27 Jun:
Engineers have now routed connections to our network through back-stored 'spare' bandwidth while the issues with our primary and secondary network links are corrected. This has resulted in significant performance improvement for all of our customers. Work is ongoing to bring the network back to full capacity.
15:51 Wed 27 Jun:
Connectivity for our network has now been re-routed through spare Pipex fibre in London.
The primary network link is well on the way to being repaired in the Sheffield 'Point of Presence' and will be brought back online later this afternoon.
The location of secondary network link fault has now been identified by engineers and will be repaired fully in due course. |
|
NEXT »: Services |