All systems operational
We're not aware of any issues affecting our systems.
(Viewing historical data)
How uptime is calculated
Uptime = (Total time - Contractual downtime) / Total time
"Contractual downtime" is the period during which Fabriq is unavailable as commonly defined in our commercial contracts. It includes any periods of partial or full outages, excluding when Fabriq's liability is waived (e.g. unavailability due to natural disasters or third parties outside Fabriq's control). It does not include periods of degraded performance or scheduled maintenance.
The application was rolled back to a working revision.
The login page prompted for e-mail and password for all visitors and no one could input valid credentials.
Service has been restored. The cause of the issue relates to a DNS and routing mis-configuration when implementing custom domains for customers.
All HTTP requests to most subdomains of Fabriq just return the URL that was hit instead of their normal page or payload.
We have identified the origin of the increase in the database load. It is related to the configuration of extra fields on a specific customers. We instructed our operations team to stop changing the configuration while we fix the root cause. The application and API are responsive again.
The database load is becoming considerable again, despite doubling the size of the database instance. This is affecting the API and application responsiveness.
We restarted the server instances after upgrading the database. The API and application are responsive again.
The database load has increased again. We stopped the server instances and started an immediate increase of the database size.
The database's load has reduced. The API and application are responsive again.
The latency of the API increased significantly as the database is under considerable load.
The higher load on the database seems to have been caused by sharing an announcement to a team while the announcement already belongs to a large number of teams. This causes a high number of messaging events to be generated. We solved the issue by optimizing the number of messaging events that are built. We are also scheduling an increase of the database's size, to preemptively address any potential future issues of the same nature.
The API is performant again, after the database has cleared enough queries. The performance degradation lasted a maximum of 33 seconds. We are investigating the cause.
Our API is seeing a steady number of requests which are making the database's CPU clock up to 100%. This is degrading the performance. Currently investigating.
We re-provisioned the three instances that were becoming unresponsive and provisioned a fourth one. Service has recovered, but we still do not know the root cause of the incident. We are continuing the investigation.
Our monitoring tool flagged our API as timing out. In turn, this made the application unavailable. We are investigating.
The certificates have been reissued. The API and application are now available again.
The team has been made aware of the incident and is investigating.
The API is unavailable because of expired SSL certificates. As a result, the application became unavailable as well.