Manchester Network
Incident Report for Simwood
Resolved
After further investigation it appears that a customer in Manchester with a pair of redundant ports activated both in response to the LONAP issue. This introduced a loop which subsequently shut down multiple ports on our network, affecting access from some other networks to our old network stack in Manchester.

We have put measures in place to ensure this cannot reoccur in the future. We believe this affected a small volume of traffic to the Manchester site. The Slough and London locations were both unaffected.

Please accept our apologies for any inconvenience caused.
Posted 11 months ago. Oct 18, 2017 - 13:54 UTC
Identified
We are still investigating this issue, but it co-incided with an outage on the LONAP peering exchange. When this was identified we took down our peering sessions with LONAP but were still experiencing packet loss with connections to our Manchester site from some other networks.

LONAP provided an RFO indicating that there was some member traffic looping back at HEX which caused a forwarding database instability, this has been corrected as of 1237 (BST)

We are still investigating the underlying cause of the other issue we experienced after taking down the session to LONAP, and will provide an update when we have further information.

At present, the Manchester site is operating as normal. Please accept our apologies for any inconvenience this caused.
Posted 11 months ago. Oct 18, 2017 - 13:12 UTC
Investigating
We are aware of an issue affecting the Manchester network resulting in intermittent connectivity issues. We're investigating this as a priority and will provide more information as soon as it becomes available.

Services in London and Slough are unaffected.
Posted 11 months ago. Oct 18, 2017 - 11:34 UTC