The Open Telekom Cloud(OTC) had a broad network outage on 2017-11-08 from 13:44 CET
until 14:41 CET. The OTC customers were informed about it an hour after
the event and some details on what exactly happened were sent out in the
morning of 2017-11-10.
The Open Telekom Cloud_ (OTC) allows customers to associate elastic IPs
(floating IPs) with vNICs (ports) on Virtual Machines (VMs) to expose
them to the internet and support in- and outgoing connections.
The Open Telekom Cloud (OTC) consists
of several environments; the environment that customers normally get to
see is the production environment. There is also a staging environment
that can be divided into several compartments and is used to test and
validate fixes; it is commonly referred to as PreProduction environment,
or short PreProd.
The Open Telekom Cloud (OTC) offers a
number of services to customer VMs that these can use without needing
to have internet access.
While many of these are preconfigured in the public images from the OTC
ImageFactory and thus will
automatically be used, customers with private images will need to
configure their images (and networks) to use these explicitly. Customers
creating subnets via the API will also need to configure the name server
(DNS) themselves.
During this time, the Management/Control Plane (Web Interface and API)
will be unavailable several times for a number of minutes. The User/Data
Plane (Virtual Resources) will be unreachable for a few minutes as well.
In addition, due to the nature of the changes, the Virtual Machines
(ECS) will not be live-migrated during this round of updates; instead
all VMs will be terminated and restarted a few minutes later (cold
migration). The User Plane changes will be performed in AZ1 (eu-de-01)
before changes are started in AZ2 (eu-de-02), so cloud native
applications should not see a service disruption.