Impact

Emails are being rejected after we send them because they are failing DMARC. Users have also reported that DKIM signing is failing on emails they are sending.

Timeline

• 2026/01/28 09:39 GMT+0: First report of DKIM issues on emails sent from our servers.
• 2026/01/28 15:50 GMT+0: Support requested from higher level staff members (administrative access to system).
• 2026/01/28 16:57 GMT+0: A fix was executed, changing the keys to match.

Technical Details

DKIM uses a pair of cryptographic keys (one public, one private) to validate the authenticity of outgoing emails. We sign each email with the private key, and recipients use the public key to verify that your email hasn’t been tampered with in transit. We rotate these keys regularly to: 1) maintain security, and 2) ensure DKIM remains a delivery-time authentication mechanism rather than a permanent proof of sending.

Unfortunately when the most recent change was enacted around the 27th of this month, an error was encountered setting the new public record on Route53, which is AWS’s implementation of DNS management. This unfortunately failed silently and did not generate a warning for our staff.

Scott helped us determine the issue, and once he identified it, he also helped us resolve it by updating our DNS to match the database. Scott set up a resilient system to manage this, and unfortunately we were hit with the one issue that he has seen in 6+ years of it working reliably. We want to thank him for being so quick to help in the issue.

Tests run after the changes were made live by both staff and customers show that the new records are successful, and being queried correctly by services like Gmail. This means that your emails aren’t going to be rejected.

Takeaways

In light of this issue, to help ensure that there is more resilience against this in the future, we will be working on systems to actively check once the changes have been made, to ensure that this isn’t a problem again.

We will be working on an early alert system that will activate after we rotate keys. One alert will notify us that the DKIM changes are being made, and a second alert to let us know if it was successful. This will allow us to confirm the public key is properly set before we start signing emails after we rotate our keys.