Outage in our Ashburn facility

Incident Report for Hostek

Postmortem

Affected Services:
Customers hosted in our Ashburn location

Date and reported time of issue:
16/March/2024 16:03UTC (11:03CST) > 20:28UTC (15:28CST)

Root Cause:
Hardware fault affecting network switches.

Contributing Factors:
Switches are configured in a redundant failover mode.
The secondary switch of this pair failed to take on the active role.
We, alongside our hardware vendor are investigating why this occurred.

Resolution and Recovery
Issue resolved as of 16th March 2024 @ 20:28UTC (15:28CST).

Future Corrective Actions
A maintenance window will be put in place in the coming days.
During this maintenance window we will be we will be implementing and improving additional features to ensure the additional reliability and redundancy of our network infrastructure

Posted Mar 18, 2024 - 15:40 UTC

Resolved

Our network team has identified and resolved the issue affecting services in Ashburn.
A detailed report will be made available as soon as possible and we will continue to closely monitor the services.

Posted Mar 16, 2024 - 20:32 UTC

Update

Our networks team is continuing to work with our upstream provider to resolve the underlying issue.

Posted Mar 16, 2024 - 18:53 UTC

Update

Our networks team are continuing to investigate the underlying cause of this issue alongside our upstream provider.
Further updates will be provided.

Posted Mar 16, 2024 - 18:00 UTC

Investigating

We are currently working with our providers in this data center to restore service as soon as possible. We are investigating at this point with no eta at this time.

Posted Mar 16, 2024 - 16:38 UTC

This incident affected: Network Infrastructure (ASH Region).