CI/CD - Hosted runners on Linux
Google Compute Engine, Digital Ocean
October 26, 2018 14:10 UTC
[Resolved] The root cause of CI Queue performance problems was resolved and the system is back now to normal capacity. We will continue to monitor it, but CI jobs should be processed without additional slowness. For more information see gitlab.com/gitlab-com/gl-infra/infrastructure/issues/5375. October 24, 2018 19:33 UTC
[Monitoring] The load on CI Queues went down and the system is operable again since last 2 hours. We've also prepared and are validating the fix for the issue. We're still monitoring the system until the issue will be finally resolved.
October 24, 2018 12:59 UTC
[Identified] We've identified the root cause of CI queues performance problem. We're now working on a fix.
October 23, 2018 20:35 UTC
[Monitoring] At this moment the performance problems of CI Queues settled down. We'll continue to monitoring it as well as investigating to find out the root cause of the issue. We're sorry for any inconveniences.
October 23, 2018 13:13 UTC
[Investigating] CI queuing times are much bigger than usual. It looks like there is a big slowdown of auto-scaled machines creation events. We're investigating the problem.
