Rownd - Notice history

Platform experiencing degraded performance

API - Operational

100% - uptime
Apr 2025 · 100.0%May · 99.96%Jun · 99.78%
Apr 2025
May 2025
Jun 2025

Embedded UI components - Operational

100% - uptime
Apr 2025 · 100.0%May · 100.0%Jun · 100.0%
Apr 2025
May 2025
Jun 2025

Platform - Degraded performance

100% - uptime
Apr 2025 · 100.0%May · 100.0%Jun · 100.0%
Apr 2025
May 2025
Jun 2025

Website - Operational

100% - uptime
Apr 2025 · 100.0%May · 100.0%Jun · 100.0%
Apr 2025
May 2025
Jun 2025

Notice history

Jun 2025

API instability
  • Resolved
    Resolved

    Our monitoring has shown that this incident has been resolved, as do the status pages for Cloudflare and Google Cloud. Thanks for your patience.

  • Update
    Update

    The issue affecting Rownd API services was caused by this Google Cloud outage (https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW), which caused major disruption to the Cloudflare services, such as Workers KV (https://www.cloudflarestatus.com/incidents/25r9t0vz99rp). These services are recovering, and as such, Rownd services are also returning to normal. We expect to see these intermittent disruptions dissipate over the next half hour.

  • Monitoring
    Monitoring

    Cloudflare has identified the issue and made repairs. There may continue to be some intermittent failures as services are restored, but we are seeing conditions return to normal. We will continue monitoring the situation until it is fully resolved.

  • Update
    Update

    Cloudflare is now reporting outages across a broad swath of services, but has not yet identified a root cause. We are continuing to monitor the situation. You can find more information on https://cloudflarestatus.com

  • Identified
    Identified

    One of our core vendors (Cloudflare) appears to be experiencing an outage affecting multiple services. Things seem to be working slowly or intermittently. We're continuing to dig into the root cause and will post additional updates soon.

  • Investigating
    Investigating

    We are investigating issues with some Rownd APIs returning 500 errors or responding slowly. We will update this ticket as more information becomes available.

May 2025

Intermittent API failures
  • Resolved
    Resolved

    During our monitoring, we noticed a significant drop in traffic to some of our backend services.

    Upon further investigation, we discovered that the internal TLS trust chain had broken due to the renewal of our private intermediate CA. This caused leaf certificates that depended on the previous intermediate CA cert to lose their trusted status, and so traffic to various internal services was affected. The result is that users may have experienced issues signing in and some platform accounts may have seemed like they were missing data.

    Rest assured that no data was lost during this incident.

    Moving forward, we will implement improved monitoring and processes that ensure the expiration of an internal root or intermediate CA is a) known well in advance, and b) automatically triggers the necessary processes that will refresh leaf certificates such that they tie back to the updated cert chain. This should prevent further incidents like this one.

    We understand how important authentication is to every product, and we are committed to ensuring that our customers can depend on Rownd as a service with near 100% uptime.

    We apologize for this incident and we will do better. If you have any further questions or concerns, please don't hesitate to reach out. We're happy to answer any inquiry.

  • Monitoring
    Monitoring

    Some of our load balancers failed to retrieve an updated certificate during our normal automated refresh cycle. We manually recycled the load balancing system and new certificates were retrieved successfully.

    We are monitoring this situation to ensure that the system is not encountering other certificate-related issues and we are implementing processes to provide better visibility and awareness when such an issue arises in the future. We are also working on improvements to our infrastructure that will prevent this type of problem from occurring in the future.

  • Investigating
    Investigating

    The Rownd API is returning 503 errors approximately 25%-30% of the time. We are investigating this issue.

Apr 2025 to Jun 2025

Next