2022-01-12 Temporary drop of some services

Description

On 2022-01-11 21:53 (UTC), a secondary, non-production server has stopped functioning. This server is non-production and not monitored. It should have minimal to no effect on the client services.

However, it turned out, that production client services were depending on it. We are investigating why there was such dependence. 

Unfortunately, the automatic watchdogs did not raise alarms, since the main production clusters were not affected. Because of this, the 24x7 team did not get notification and did nothing until the morning.

Details

Some ERP.net services have not functioned:

(Your local time may vary)

Aftermath

We will remove the improper dependence and will try to avoid further dropouts for such reasons.