Investigating issue regarding processing of background jobs

Incident Report for ConvertKit

Resolved

A failsafe trigger was enabled when our background instances ran out of filesystem space. We expanded the filesystem and turned the failsafe off while we catch up with background jobs.

We have since turned the failsafe back on and everything should be back to normal.

We are running a post mortem in the morning to make sure we have better safeguard so we can catch this kind of issues before it happens next time.

Posted Aug 09, 2018 - 17:38 PDT

Monitoring

We have found the problem and applied a fix. We are monitoring our background jobs.

Posted Aug 09, 2018 - 17:04 PDT

Investigating

We are currently investigating this issue.

Posted Aug 09, 2018 - 16:47 PDT

This incident affected: API, Application, and Email sending.