More than 3.6 million web servers representing 464,000 domain names went offline in a few minutes following the fire that broke out on Wednesday night, March 10, at our hosting company OVH, including French government sites. Be assured: your Cozy domains were not affected by the fire and no data was lost.
Our infrastructure withstood the fire despite the very rapid loss of 43 of our servers and in keeping with our strategy of transparency, we decided to detail you the impact of this incident on our business and how we were able to cope with it.
The French web paralyzed
The websites and services of more than 12,000 French companies that had chosen to have their data hosted by OVH were taken offline following the disaster.
This is notably the case for :
- the digital workspace (ENT) of the Île-de-France region
- the Centre Pompidou
- the data.gouv.fr website
- the Simplified Procedures website
- the CityScoot application
- Coinhouse, a specialist in cryptocurrency transactions.
And the list of "burnt" machines is still long.
Why did your Cozy not be impacted by this fireplace?
At Cozy Cloud, our business is the protection of your data. Your data is hosted in geographically separated datacenters - Gravelines, Roubaix and Strasbourg, managed by OVH which guarantees for example that a "catastrophe" on one site will not affect the data hosted on the other sites. Each of your data is replicated 3 times in redundant systems, spread over several sites. Our infrastructures are also sized so that the loss of servers does not disturb the service.
Following this exceptional accident, 43 servers were destroyed out of a total of more than 200 and despite this, there was no noticeable impact on the service provided to our users, nor was any data lost.
What was our action plan?
First of all, nothing! The configuration and automation in place responded perfectly and isolated the affected components to maintain data and service despite everything.
Then, as soon as we became aware of the fire shortly after midnight, our team established the following action plan:
- Identify losses and prioritize actions to replace machines
- Immediately order replacement machines from other OVH datacenters to restore redundancy and our scalability and performance capabilities
- Re-installation of machines with priority given to restoring the redundancies most affected by the accident. Our configuration automation allowed us to perform these re-installations in only a few minutes per machine and thus minimize the time when our redundancy was reduced
- 4. Real-time communication to our various partners of the impact on their production and specific action plans
All of these actions performed by our team took place in less than 36 hours of intervention after the disaster was announced.
What is the total impact to you, the Cozy users?
This incident has not impacted the availability of your Cozy and its data. However, action is required on the part of Cozy Banks users to resume synchronization of their new bank data.
"Keep calm: your data is safe"
Making data backups is essential today but it also requires technical skills. Our mission is to do it for you in order to facilitate your already complex digital life.
Testimony of an user who chose to save his data in a Cozy.
I had my laptop broken into and stolen. But thanks to Cozy, I didn't lose any data, and I have access to everything from my phone! I was able to print invoices partly recovered by the connectors, and other documents, from my Cozy on my phone. Anyway, I just wanted to tell you that I am very happy to be part of your users!
Congratulations to all our users who have already chosen Cozy!
Faced with the challenges of digital sovereignty, and in order to keep our promise, we have always chosen to work with French and more European partners.
The reason is simple: by interacting with them on a daily basis, we have control over our infrastructure and therefore your data. This makes it easier to act quickly and effectively in these extreme cases.
We therefore reaffirm our confidence in OVH and give them all our support to overcome this ordeal.
Thanks to our team
We would like to thank all of the system administrators who have been able to work with us, since the beginning of the Cozy adventure, to set up an architecture that is sufficiently robust, redundant and resilient. It is because of their work that this incident has not impacted your data or access to your Cozy today.
This article is available also in French.
- Article Journal du Net : Incendie chez OVH : le plan de reprise se précise
- Article de Numerama : Incendie chez OVH, l'hébergeur précise ce qui est récupérable et ce qui est perdu
- Communication officielle OVH par Octave Klaba