News

The Reefuge

Administrator
Jul 9, 2011
4,152
1,929
The Reefuge Downtime
Hello everyone,

IMPORTANT: IF YOU REGISTERED AFTER THE 30TH OF JUNE YOUR ACCOUNT MAY HAVE BEEN DELETED. PLEASE REGISTER AGAIN IF YOUR ACCOUNT HAS BEEN DELETED.

Glad to be back. Wow, what a crazy 24 hours.

tl;dr - Hard Drive died, complete server reinstall.

As you all know by now The Reefuge has been offline for the past 24 hours. As some of you may know, The Reefuge is hosted on a dedicated server. It does not use shared hosting or VPS.

Yesterday, there was a problem with MySQL. Whilst this problem was being investigated, the entire server went offline and was not reachable via the internet. A support ticket was sent to the host and it was later determined that the entire hard drive had failed. We tried to salvage the data from the hard drive but it was so far gone that this was deemed almost impossible. The hard drive was thrown away.

Luckily we have a small, secondary server that is used for backing up LavaPixel Networks (The Reefuge Site). The latest backup was made on the 30th of June. We used this backup to restore all of the data onto the server. Unfortunately this means that anything created after the 30th of June is gone, including new user registrations, posts, etc. However, this could have been a whole lot worse. Should the backup have not been implemented in the first place, The Reefuge could have lost a whole lot more than what we did.

Currently data is still being restored onto the server, so if the server seems a bit laggy it is because it is still being worked on. All sites hosted by LavaPixel Networks are now fully operational again.

We are looking into a new way of setting up the server later this year to prevent something like this happening again. Currently, we are looking into RAID 1. In order to do this, we will need to change hard drives (which can be done fast and easily). However, this is costly.

We apologise for the inconvinience caused to all of our members.

Please also note that the server is still being worked on. Expect small intervals of downtime over the next few hours.

Thanks.
 

alan

Member
Jan 26, 2012
212
51
Hobart
you mean to say that the server you are running this on is NOT running R1? Maybe you should look at moving it to a virtual host rather than a hardware host not running raid arrays.
 

The Reefuge

Administrator
Jul 9, 2011
4,152
1,929
Hello @alan,

The server is going to switch to RAID 1 soon. It will continue to have its off-site backups in addition to this.

Thanks.
 

alan

Member
Jan 26, 2012
212
51
Hobart
If you need a new VM, let me know, i can hook you up with our DC guys. raid 60 arrays.
 

The Reefuge

Administrator
Jul 9, 2011
4,152
1,929
Hello @alan,

Thanks for the offer. For the time being we are okay. The server is back to normal and currently I am looking into adding RAID1, this should prevent a repeat of the other day.
Thanks.