Recurring Tests are delayed
Incident Report for Snyk
Resolved
Recurring Tests continue to run as normal and we have observed no further issues.

The Snyk Incident Response has been stood down.
Posted Aug 27, 2024 - 08:37 UTC
Update
The missed weekly Recurring Test from the 17th of August will be executed as part of the normal run scheduled tomorrow.

The root cause was identified, and monitoring will remain ongoing to confirm no further issues during tomorrow's schedule.
Posted Aug 23, 2024 - 14:16 UTC
Update
Our Engineers have reported that the re-run of the failed weekly (17th August) Recurring Test has taken longer than expected and has, therefore, been aborted to prevent negatively impacting today's Recurring Tests.

We will provide a further update tomorrow on whether we will attempt to re-run the missed test.
Posted Aug 22, 2024 - 15:28 UTC
Update
The weekly Recurring Test scheduled for the 17th of August that failed to execute will be initiated shortly by our Engineers. This will ensure that all recurring tests are up to date and on schedule.
Posted Aug 22, 2024 - 13:01 UTC
Update
Our Engineers have confirmed that Recurring Tests continue to execute as expected. At the moment we are unsure if the missed Recurring Tests from Saturday the 17th of August will be executed before the next scheduled run (24th of August).

The Issues API has experienced no further data latency affecting responses.

We will continue to monitor the progress of our Recurring Tests and provide further updates in due course.
Posted Aug 21, 2024 - 10:44 UTC
Update
To help mitigate the data latency experienced with Issues API responses our Engineers reduced the Recurring Test rates throughout the weekend. This activity proved successful in completely resolving the data latency.

This affected our scheduled Recurring Tests for Saturday (17th of August), which were not executed. These will be manually executed at a later date (to be confirmed). All other Recurring Tests are running as scheduled.
Posted Aug 19, 2024 - 08:14 UTC
Update
Our Engineers have implemented a fix to reduce the data latency in the Issues API responses. Monitoring is underway to ensure the success of the applied fix.

Recurring tests continue to execute as expected.
Posted Aug 16, 2024 - 14:45 UTC
Update
Our Engineers have observed no further issues with recurring tests and are confident our customers should experience no further impact related to their recurring tests.

We are still observing some data latency affecting our Issues API responses, we're continuing to monitor this closely.
Posted Aug 16, 2024 - 08:01 UTC
Update
Monitoring remains ongoing, however, our Engineers are seeing a significant improvement regarding the data latency affecting Issues API responses. This is now only impacting a small percentage of projects.
Posted Aug 15, 2024 - 16:33 UTC
Monitoring
Our Engineers have implemented a fix for our recurring tests and are now monitoring to ensure all tests execute as expected.

We are still, however, observing data latency affecting Issues API responses.

We will share a further update once our monitoring period has been completed.
Posted Aug 15, 2024 - 16:04 UTC
Identified
Our Engineers have identified a potential fix which is currently under review. We will provide a further update shortly.
Posted Aug 15, 2024 - 13:57 UTC
Update
Our Engineers have confirmed that as a result of the recurring test delays we are observing data latency, which is affecting Issues API responses.
Posted Aug 15, 2024 - 08:32 UTC
Update
Customers can manually retest as manual testing is unaffected.
Posted Aug 15, 2024 - 07:55 UTC
Investigating
Our Engineers have identified that our recurring tests are delayed. This is affecting weekly and daily recurring test schedules.

Some customers may observe that their scheduled recurring tests have not executed as expected or are delayed.

The Snyk Incident Response has been invoked and our Engineers are investigating the issue.
Posted Aug 15, 2024 - 07:54 UTC
This incident affected: SNYK-US-01 (app.snyk.io) (Snyk Code, Snyk Container, Snyk IaC, Snyk Open Source).