Active Issue

Updated a few seconds ago

Back to current status

Status History

Filter: packages.gitlab.com (Clear)



January 2025

Packages on GitLab.com (packages.gitlab.com) become unavailable sporadically with 502 errors.

January 10, 2025 03:54 UTC

Incident Status

Degraded Performance


Components

packages.gitlab.com


Locations

AWS




January 10, 2025 03:54 UTC
[Resolved] The fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com) has completed successfully. We are now marking this incident as resolved. More about the incident and its progress can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 10, 2025 02:32 UTC
[Monitoring] The fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com) is ongoing. It appears to be progressing as expected. More about the incident and its progress can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 21:55 UTC
[Monitoring] The fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com) is ongoing. We expect it to take about 20 hours, and will post updates every 4 hours. More about the incident and its progress can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 19:53 UTC
[Monitoring] No updates at this time. The fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com) is still processing. More about the incident and its progress can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 18:13 UTC
[Monitoring] The fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com) has been implemented, and is processing now. More about the incident and its progress can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 16:30 UTC
[Identified] We're still working on implementing the fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its progress can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 15:54 UTC
[Identified] We're still working on implementing the fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its progress can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 15:22 UTC
[Identified] We're still working on implementing the fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its updates can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 14:49 UTC
[Identified] We're still working on implementing the fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its updates can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 14:17 UTC
[Identified] We're still working on implementing the fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its updates can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 13:43 UTC
[Identified] We're working on implementing the fix for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its updates can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 13:09 UTC
[Identified] We've identified the root cause for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com) and working on a fix. More about the incident and its updates can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 12:48 UTC
[Investigating] We're still investigating the root cause for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its updates can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 12:33 UTC
[Investigating] We're still investigating the root cause for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its updates can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 12:17 UTC
[Investigating] We're still investigating the root cause for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its updates can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 12:02 UTC
[Investigating] We're still investigating the root cause for the sporadic unavailability of packages on GitLab.com (packages.gitlab.com). More about the incident and its updates can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

January 9, 2025 11:45 UTC
[Investigating] We are currently investigating an incident that is causing packages on GitLab.com (packages.gitlab.com) to become unavailable sporadically with 502 errors. More about the incident can be read here: gitlab.com/gitlab-com/gl-infra/production/-/issues/19091

Accessing gitlab.com results in 500 errors

January 6, 2025 16:42 UTC

Incident Status

Partial Service Disruption


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners on macOS, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, docs.gitlab.com, Canary, Product Analytics - Configurator (Beta), Product Analytics - Collector (Beta), Product Analytics - Background processing (Beta), Product Analytics - Queries (Beta), GitLab Duo


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




January 6, 2025 16:42 UTC
[Resolved] We've identified the cause of the errors and the errors have returned to normal. The incident is now resolved. More information on the incident can be found in this issue: gitlab.com/gitlab-com/gl-infra/production/-/issues/19079

January 6, 2025 15:57 UTC
[Monitoring] We've identified the cause of the errors, and the errors have returned to normal. The incident is now mitigated, but we're continuing to monitor for any further impact. More information on the incident can be found in this issue: gitlab.com/gitlab-com/gl-infra/production/-/issues/19079

January 6, 2025 15:27 UTC
[Monitoring] We're continuing to monitor for further errors related to the incident. We see further decreases in the errors.

January 6, 2025 15:17 UTC
[Monitoring] We saw elevated 500 errors on gitlab.com which have then decreased. We're continuing to monitor for further errors

November 2024

Update the CA of certificates for number of hostnames

November 7, 2024 01:55 UTC

Description

To maximize device compatibility going forward, GitLab will update the CA of certificates used by a number of hostnames to Google Trust Services. For full details please refer to the CR issue : gitlab.com/gitlab-com/gl-infra/production/-/issues/18625


Components

packages.gitlab.com


Locations

AWS


Schedule

November 6, 2024 00:00 - November 7, 2024 00:00 UTC



November 7, 2024 01:55 UTC
[Update] GitLab.com planned maintenance for updating certificate CAs has completed. There were no reported problems during the monitoring period in the maintenance window. Thank you for your patience.

November 6, 2024 23:44 UTC
[Update] Maintenance started as of November 6, 2024 00:00 and is expected to end November 7, 2024 00:00 UTC

November 6, 2024 02:48 UTC
[Update] Maintenance is underway to update the CA of certificates used by a number of hostnames to Google Trust Services. No customer impact is expected from this change. We'll continue to monitor until the end of the maintenance window. For full details please refer to the CR issue : gitlab.com/gitlab-com/gl-infra/production/-/issues/18625

November 4, 2024 00:39 UTC
[Update] Due to the ongoing production change lock this maintenance has been rescheduled, to begin 2024-11-06 00:00 to 2024-11-07 00:00 UTC We apologize for the inconvenience; this update has been reflected on the infrastructure issue as well: gitlab.com/gitlab-com/gl-infra/production/-/issues/18655

October 30, 2024 14:01 UTC
[Update] Due to circumstances beyond our control, the maintenance has been rescheduled from the original timeframe of 2024-10-31 - 2024-11-01 to 2024-11-04 - 2024-11-05. We apologize for the inconvenience; this update has been reflected on the infrastructure issue as well: gitlab.com/gitlab-com/gl-infra/production/-/issues/18655

September 2024

GitLab.com Connectivity Issues

September 3, 2024 05:34 UTC

Incident Status

Service Disruption


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners on macOS, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, docs.gitlab.com, Canary, Product Analytics - Configurator (Beta), Product Analytics - Collector (Beta), Product Analytics - Background processing (Beta), Product Analytics - Queries (Beta), GitLab Duo


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




September 3, 2024 05:34 UTC
[Resolved] All GitLab services have been restored. We are actively monitoring the system, and everything appears normal at this time. For more details, please follow the issue at: gitlab.com/gitlab-com/gl-infra/production/-/issues/18490

September 3, 2024 05:19 UTC
[Monitoring] We've identified and addressed the root cause of the issue. All GitLab services have been restored, and we are actively monitoring the system. Follow the issue for more details : gitlab.com/gitlab-com/gl-infra/production/-/issues/18490

September 3, 2024 05:09 UTC
[Investigating] GitLab.com is currently experiencing connectivity issues due to a high error rate. Our infrastructure team is investigating. Issue: gitlab.com/gitlab-com/gl-infra/production/-/issues/18490

August 2024

Some requests timing out

August 23, 2024 03:59 UTC

Incident Status

Operational


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners on macOS, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, docs.gitlab.com, Canary, Product Analytics - Configurator (Beta), Product Analytics - Collector (Beta), Product Analytics - Background processing (Beta), Product Analytics - Queries (Beta), GitLab Duo


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




August 23, 2024 03:59 UTC
[Resolved] The issue has been resolved and all functionality on GitLab.com is nominal. Please see gitlab.com/gitlab-com/gl-infra/production/-/issues/18435 for more details.

August 23, 2024 01:53 UTC
[Monitoring] The issue has been identified and the incident resolved. We will continue monitoring for a while.

August 23, 2024 01:27 UTC
[Identified] We have identified an underlying cause and are working to mitigate it. See gitlab.com/gitlab-com/gl-infra/production/-/issues/18435 for more details.

August 23, 2024 00:34 UTC
[Investigating] We have noticed that there are delays in web UI updates for example, MRs not showing up after submission, We are continuing to investigate - please see gitlab.com/gitlab-com/gl-infra/production/-/issues/18435 for more details.

August 23, 2024 00:27 UTC
[Investigating] Some requests are slow or timing out due to DB saturation. We are investigating. Please see [this issue](gitlab.com/gitlab-com/gl-infra/production/-/issues/18435) for further details.

July 2024

"Package Manifest is not coherent" errors

July 11, 2024 20:26 UTC

Incident Status

Partial Service Disruption


Components

packages.gitlab.com


Locations

AWS




July 11, 2024 20:26 UTC
[Resolved] The two MR's that were opened to address this incident are deployed and live on GitLab.com. This incident is now resolved. Some users may still see the error due to npm auto-corrections and we are adding documentation on how to handle it. See gitlab.com/gitlab-com/gl-infra/production/-/issues/18253 for more details.

July 10, 2024 19:44 UTC
[Identified] Some users are receiving the error message "Package Manifest is not coherent" when attempting to publish NPM packages. A second MR has been opened to address another underlying bug that was discovered while addressing the root cause. Once this MR is merged and deployed, the incident will be resolved. Affected users can subscribe to the issue for updates here: gitlab.com/gitlab-com/gl-infra/production/-/issues/18253

July 10, 2024 14:41 UTC
[Identified] Some users are receiving the error message "Package Manifest is not coherent" when attempting to publish NPM packages. We have identified the cause, and are working on a fix. Possible workarounds have been added to the incident - gitlab.com/gitlab-com/gl-infra/production/-/issues/18253#note_1992957468

July 9, 2024 23:23 UTC
[Identified] Some users are receiving the error message "Package Manifest is not coherent" when attempting to publish NPM packages. The cause is known, the fix has been merged and will be deployed to production in the next deployment. Affected users can subscribe to the issue for updates here: gitlab.com/gitlab-com/gl-infra/production/-/issues/18253

GitLab.com is unavailable

July 11, 2024 18:28 UTC

GitLab.com is unavailableService Disruption

Incident Status

Service Disruption


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners on macOS, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, docs.gitlab.com, Canary, Product Analytics - Configurator (Beta), Product Analytics - Collector (Beta), Product Analytics - Background processing (Beta), Product Analytics - Queries (Beta), GitLab Duo


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




July 11, 2024 18:28 UTC
[Resolved] This incident is now resolved and no further errors are being seen on GitLab.com. Our investigation points to a disk performance issue. You can see further details and corrective actions in gitlab.com/gitlab-com/gl-infra/production/-/issues/18269.

July 11, 2024 15:51 UTC
[Investigating] Our investigations are ongoing. We will post our next update when we have more information. Please follow the incident issue for updates here: gitlab.com/gitlab-com/gl-infra/production/-/issues/18269

July 11, 2024 15:28 UTC
[Investigating] No material updates to report. We're continuing to investigate the issue. Please follow the incident issue for updates here: gitlab.com/gitlab-com/gl-infra/production/-/issues/18269

July 11, 2024 15:00 UTC
[Investigating] We are seeing improved service availability and investigations continue for this problem. Please follow the incident issue for updates here: gitlab.com/gitlab-com/gl-infra/production/-/issues/18269

July 11, 2024 14:43 UTC
[Investigating] We are seeing improved service availability and investigations continue for this problem. We are planning to make the incident issue publicly available as soon as possible.

July 11, 2024 14:29 UTC
[Investigating] We are investigating a problem with gitlab.com being unavailable for some users. Our investigations continue and we will post further status updates shortly.

July 11, 2024 14:18 UTC
[Investigating] We are investigating a problem with gitlab.com being unavailable for some users. We will post further status updates shortly.

Intermit 500 errors on packages.gitlab.com

July 5, 2024 22:09 UTC

Incident Status

Operational


Components

packages.gitlab.com


Locations

AWS




July 5, 2024 22:09 UTC
[Resolved] Service has stabilized after the implemented fix. Access should be fully restored to packages.gitlab.com site. Marking as resolved.

July 5, 2024 21:24 UTC
[Monitoring] We have implemented a possible fix. Service has been restored to packages.gitlab.com . Users should no longer see a 500 error at login. We will continue to monitor packages.gitlab.com to make sure it's stable before we mark it as resolved.

July 5, 2024 20:37 UTC
[Investigating] packages.gitlab.com is experiencing intermit 500 errors. We are currently in the process of investigating the cause at this time. This will not allow new downloads of our GitLab product. We will update once we have more information or at the hour.

July 5, 2024 19:51 UTC
[Investigating] packages.gitlab.com is experiencing intermit 500 errors. We are currently in the process of investigating the cause at this time. This will not allow new downloads of our GitLab product. We will update once we have more information or at the hour.

July 5, 2024 18:43 UTC
[Investigating] packages.gitlab.com is experiencing intermit 500 errors. We are currently in the process of investigating the cause at this time. This will not allow new downloads of our GitLab product.

July 5, 2024 18:39 UTC
[Investigating] packages.gitlab.com is experiencing intermit 500 errors. We are currently in the process of investigating the cause at this time. This will not allow new downloads of our GitLab product.

Users unable to download NuGet packages

July 2, 2024 15:11 UTC

Incident Status

Degraded Performance


Components

packages.gitlab.com


Locations

AWS




July 2, 2024 15:11 UTC
[Resolved] This incident was resolved by rolling back a feature flag. We've got confirmation from users that packages are being downloaded succesfully again. More details here: gitlab.com/gitlab-com/gl-infra/production/-/issues/18228

July 2, 2024 14:45 UTC
[Investigating] We have received reports from users unable to download NuGet packages on GitLab.com. We're investigating the issue at the moment. More details here: gitlab.com/gitlab-com/gl-infra/production/-/issues/18228

August 2023

Degraded performance of packages.gitlab.com

August 31, 2023 19:11 UTC

Incident Status

Degraded Performance


Components

packages.gitlab.com


Locations

AWS




August 31, 2023 19:11 UTC
[Resolved] There has been no further sign of performance degradation on packages.gitlab.com since traffic returned to normal. Thanks for your patience! See gitlab.com/gitlab-com/gl-infra/production/-/issues/16281 for details.

August 31, 2023 18:34 UTC
[Monitoring] Performance on packages.gitlab.com is returning to normal. Our engineers have identified the cause and are monitoring. See gitlab.com/gitlab-com/gl-infra/production/-/issues/16281

August 31, 2023 18:22 UTC
[Identified] We're seeing degraded performance on packages.gitlab.com due to increased traffic. Some customers may be experiencing intermittent 5xx responses or timeouts. See gitlab.com/gitlab-com/gl-infra/production/-/issues/16281 for more details.

June 2023

Slow requests on GitLab.com

June 7, 2023 13:32 UTC

Slow requests on GitLab.comPartial Service Disruption

Incident Status

Partial Service Disruption


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, docs.gitlab.com, Canary


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




June 7, 2023 13:32 UTC
[Resolved] We have identified the root cause, have seen recovery and are therefore resolving this incident. More information in: gitlab.com/gitlab-com/gl-infra/production/-/issues/14839

June 7, 2023 13:21 UTC
[Investigating] Customers are experiencing slow requests on GitLab.com. More information in: gitlab.com/gitlab-com/gl-infra/production/-/issues/14839

May 2023

Hardware upgrade for the GitLab.com datastore

May 27, 2023 04:06 UTC

Description

In 2 weeks, as part of a planned maintenance window, we will perform a hardware upgrade for the GitLab.com datastore. Users may experience temporary 50X errors for a brief period during this window. See gitlab.com/gitlab-com/gl-infra/production/-/issues/10694


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, docs.gitlab.com, Canary


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS


Schedule

May 27, 2023 03:00 - May 27, 2023 04:00 UTC



May 27, 2023 04:06 UTC
[Update] GitLab.com's database layer maintenance is complete now, and we're fully back up and running. We'll be monitoring the platform to ensure all systems are functioning correctly. Thank you for your patience.

May 27, 2023 03:41 UTC
[Update] The database switchover is now complete, we expect all SQL statements to be routed to the new nodes. The site is back up and we're continuing to verify that all systems are functioning correctly. Thank you for your patience.

May 27, 2023 03:00 UTC
[Update] The database maintenance is starting now. The database Primary node switchover should happen at any moment in the next hour and during the switchover users might see 50X errors for a very brief span of time. For details see gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

May 27, 2023 01:51 UTC
[Update] We will be conducting a maintenance database activity in an hour, from 03:00 to 04:00 UTC. Users might see 50X errors for a very brief span of time during this window. For details see gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

May 26, 2023 02:27 UTC
[Update] We will be conducting a maintenance database activity Tomorrow, 2023-05-27, from 03:00 to 04:00 UTC. Users might see 50X errors for a very brief span of time during this window. For details see gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

May 24, 2023 02:14 UTC
[Update] Correction: We will be conducting a maintenance database activity this Saturday, 2023-05-27. from 03:00 to 04:00 UTC. Users might see 50X errors for a very brief span of time during this window. For details see gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

May 24, 2023 02:05 UTC
[Update] We will be conducting a maintenance database activity this Saturday, 2023-05-20, from 03:00 to 04:00 UTC. Users might see 50X errors for a very brief span of time during this window. For details see gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

May 20, 2023 03:58 UTC
[Update] The maintenance is rescheduled to next Saturday, 2023-05-27, from 03:00 to 04:00 UTC. There is currently a long database migration running to fix an incident (gitlab.com/gitlab-com/gl-infra/production/-/issues/14468) that we can't risk to interrupt. We apologise for any inconvenience.

May 20, 2023 03:07 UTC
[Update] The scheduled maintenance will be not be starting at 03:00 UTC, and has been delayed. More details soon to follow.

May 20, 2023 02:00 UTC
[Update] We will be conducting a maintenance database activity in an hour, from 03:00 to 04:00 UTC. Users might see 50X errors for a very brief span of time during this window. For details see gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

May 20, 2023 01:00 UTC
[Update] We will be conducting a maintenance database activity in 2 hours, from 03:00 to 04:00 UTC. Users might see 50X errors for a very brief span of time during this window. For details see gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

May 19, 2023 02:00 UTC
[Update] We will be conducting a maintenance database activity tomorrow, 2023-05-20, from 03:00 to 04:00 UTC. Users might see 50X errors for a very brief span of time during this window. For details see gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

May 17, 2023 02:00 UTC
[Update] We will be conducting a maintenance database activity this Saturday, 2023-05-20, from 03:00 to 04:00 UTC. Users might see 50X errors for a very brief span of time during this window. For details see gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

May 12, 2023 02:00 UTC
[Update] Next week, as part of a planned maintenance window on 2023-05-20 from 03:00 to 04:00 UTC, We will perform a hardware upgrade for the GitLab.com datastore. Users may experience temporary 50X errors for a brief period during this window. See gitlab.com/gitlab-com/gl-infra/production/-/issues/10694

March 2023

packages.gitlab.com is unavailable

March 31, 2023 08:56 UTC

Incident Status

Service Disruption


Components

packages.gitlab.com


Locations

AWS




March 31, 2023 08:56 UTC
[Resolved] We are no longer seeing errors from packages.gitlab.com. We will set this as resolved now. We are continuing investigations for the root cause and will post further updates in gitlab.com/gitlab-com/gl-infra/production/-/issues/8635

March 31, 2023 07:51 UTC
[Monitoring] We are no longer seeing errors from packages.gitlab.com. We are still investigating the cause of the issue. We will post further updates as more information is available in the next couple of hours. Details: gitlab.com/gitlab-com/gl-infra/production/-/issues/8635

March 31, 2023 06:16 UTC
[Monitoring] Access to packages.gitlab.com is restored. While the service has been restored, our team is continuing to investigate the root cause of the issue. We will move this to monitoring state for now. Details: gitlab.com/gitlab-com/gl-infra/production/-/issues/8635

March 31, 2023 05:48 UTC
[Investigating] We do not have any material updates to report. However, we want to assure you that our team is continuing to investigate the issue. We will send another update in the next hour. Details: gitlab.com/gitlab-com/gl-infra/production/-/issues/8635

March 31, 2023 05:27 UTC
[Investigating] We are continuing to investigate the cause of the problem on packages.gitlab.com and will post further updates as more information is available. Details: gitlab.com/gitlab-com/gl-infra/production/-/issues/8635

March 31, 2023 05:03 UTC
[Investigating] We are currently experiencing an issue on packages.gitlab.com. This service is currently not available, and our team is currently looking into this. Kindly refer to this link for more details: gitlab.com/gitlab-com/gl-infra/production/-/issues/8635

Reliability improvements on packages.gitlab.com

March 18, 2023 13:00 UTC

Description

We are making a series of improvements to the packages.gitlab.com infrastructure starting with the database requiring service downtime.


Components

packages.gitlab.com


Locations

AWS


Schedule

March 18, 2023 11:30 - March 18, 2023 13:00 UTC



March 18, 2023 13:00 UTC
[Update] Scheduled maintenance is complete.

March 18, 2023 11:30 UTC
[Update] Scheduled maintenance is starting.

January 2023

High Error Rate on Select GitLab.com

January 5, 2023 11:24 UTC

Incident Status

Service Disruption


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, Canary


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




January 5, 2023 11:24 UTC
[Resolved] Our Engineers identified the root cause and addressed it with a fix. The incident is now mitigated and users are expected to work with MRs without encountering any errors. For details, check gitlab.com/gitlab-com/gl-infra/production/-/issues/8201

January 5, 2023 09:18 UTC
[Monitoring] Services continue to recover and customers reported successful logins though SAML. However, some users are still facing errors with MRs. Please check the issue for the workaround: gitlab.com/gitlab-com/gl-infra/production/-/issues/8201#note_1228554821

January 5, 2023 08:45 UTC
[Monitoring] The error rates are decreasing, but users might still experience errors when authenticating through SAML and working with MRs. Investigation is still ongoing. For details, check gitlab.com/gitlab-com/gl-infra/production/-/issues/8201

January 5, 2023 08:24 UTC
[Monitoring] We are seeing the error rate decreasing and the services recovering. Our Engineers will keep monitoring the systems. For more details, please check gitlab.com/gitlab-com/gl-infra/production/-/issues/8201

January 5, 2023 08:07 UTC
[Investigating] We are currently seeing a high rate of errors on GitLab.com. Our infrastructure team is currently investigating. More information can be found here: gitlab.com/gitlab-com/gl-infra/production/-/issues/8201

August 2022

GitLab.com slow to respond

August 26, 2022 06:21 UTC

GitLab.com slow to respondDegraded Performance

Incident Status

Degraded Performance


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, Canary


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




August 26, 2022 06:21 UTC
[Resolved] We are no longer seeing connectivity issues for GitLab.com after a period of monitoring. We therefore believe that this incident has been resolved. Details in : gitlab.com/gitlab-com/gl-infra/production/-/issues/7656

August 26, 2022 05:08 UTC
[Monitoring] All the services seems to have recovered. We will continue to monitor the performance and provide the updates here. More information on the root cause can be found here: gitlab.com/gitlab-com/gl-infra/production/-/issues/7656

August 26, 2022 04:57 UTC
[Investigating] We are still investigating the root cause of this issue. More details here: gitlab.com/gitlab-com/gl-infra/production/-/issues/7656

August 26, 2022 04:55 UTC
[Investigating] We are noticing some slow response issues with GitLab.com.

June 2022

Spike in error rate on gitlab.com

June 23, 2022 07:19 UTC

Incident Status

Operational


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, Canary


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




June 23, 2022 07:19 UTC
[Resolved] Error rates have returned to normal levels. Follow-up discussion and actions are being discussed here: gitlab.com/gitlab-com/gl-infra/production/-/issues/7310

June 23, 2022 06:46 UTC
[Monitoring] We are investigating a brief spike in errors that lasted about 20 minutes, we are monitoring the recovery and will put more detail in gitlab.com/gitlab-com/gl-infra/production/-/issues/7310

Connectivity issues for GitLab.com

June 21, 2022 08:43 UTC

Incident Status

Service Disruption


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, Canary


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




June 21, 2022 08:43 UTC
[Resolved] We are no longer seeing connectivity issues for GitLab.com after a period of monitoring. We therefore believe that this incident has been resolved. Details in: gitlab.com/gitlab-com/gl-infra/production/-/issues/7288

June 21, 2022 08:00 UTC
[Monitoring] Services seem to be back to normal, and we continue monitoring. Details in: gitlab.com/gitlab-com/gl-infra/production/-/issues/7288

June 21, 2022 07:45 UTC
[Identified] We are seeing services slowly recovering and continue monitoring issues with our CDN provider.

June 21, 2022 07:22 UTC
[Identified] No material updates at this point. We are continuing to monitor issues with our CDN provider

June 21, 2022 07:02 UTC
[Identified] We are continuing to monitor connectivity issues from some regions due to connectivity problems from our CDN provider.

June 21, 2022 06:52 UTC
[Investigating] We are investigating connectivity issues for GitLab.com, this is affecting all services.

Gitlab.com is having slowness issues

June 15, 2022 23:28 UTC

Incident Status

Degraded Performance


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, Canary


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




June 15, 2022 23:28 UTC
[Resolved] We are no longer seeing performance issues on GitLab.com after a period of monitoring. We believe the incident has now been resolved. More details can be found here: gitlab.com/gitlab-com/gl-infra/production/-/issues/7258

June 15, 2022 23:12 UTC
[Identified] Our teams had identified the root cause of the slowness and are discussing preventive actions for it. Performance is back to normal.

June 15, 2022 22:53 UTC
[Investigating] Our teams detected a performance degradation for all components for GitLab.com, it appears this degradation is now stabilizing back to normal performance. We are still investigating.

June 15, 2022 22:36 UTC
[Investigating] Requests to GitLab.com are seeing elevated error rates and slowness, we are investigating.

April 2022

Elevated Error Rate on GitLab.com

April 28, 2022 07:37 UTC

Incident Status

Service Disruption


Components

Website, API, Git Operations, Container Registry, GitLab Pages, CI/CD - Hosted runners on Linux, CI/CD - Hosted runners on Windows, CI/CD - Hosted runners for GitLab community contributions, SAML SSO - GitLab SaaS, Background Processing, GitLab Customers Portal, Support Services, packages.gitlab.com, version.gitlab.com, forum.gitlab.com, Canary


Locations

Google Compute Engine, Digital Ocean, Zendesk, AWS




April 28, 2022 07:37 UTC
[Resolved] We transitioned this incident to resolved. Check the issue for more details: gitlab.com/gitlab-com/gl-infra/production/-/issues/6933

April 28, 2022 07:23 UTC
[Monitoring] We have identified the cause to this incident and do not anticipate any additional impacts at this time. We are continuing to monitor services as well as complete additional investigation. More details here: gitlab.com/gitlab-com/gl-infra/production/-/issues/6933

April 28, 2022 06:34 UTC
[Investigating] We are still investigating the root cause. The error rates on affected systems recovering, and we are monitoring to ensure that the issue does not recur.

April 28, 2022 06:12 UTC
[Investigating] We're investigating increased error rates across GitLab-com services, some users might be experiencing 500 errors intermittently, more info can be found in here: gitlab.com/gitlab-com/gl-infra/production/-/issues/6933





Back to current status