Users may not able to access binders and contracts shared to them
Incident Report for Zefort
Postmortem

Summary

We encountered an issue where duplicate or overlapping permission entries in our database were causing complexity and performance concerns. To improve efficiency, we performed a cleanup to remove these duplicates. However, due to a logic error in the update process, some valid permission entries were mistakenly removed. Despite our testing, we did not catch this issue before deployment.

Impact: As a result, many users temporarily lost access to some content they should have had access to. No data was lost, only the ability to view or manage certain content was affected.

Resolution

Once the issue was identified, we restored affected permissions from backups. Our team conducted a thorough review, including going through logs, to ensure all permissions were correctly reinstated. We also took steps to ensure that contracts received via email during the incident, which may have been affected by missing permissions, were properly ingested.

Next Steps

To prevent similar incidents in the future, we are:

  • Enhancing automated tests to better detect unintended data modifications.
  • Improving our review processes for database migrations and cleanup operations.

We sincerely apologize for any inconvenience this may have caused. We remain committed to improving our processes to provide a more reliable service.

If you have any concerns or need assistance, please contact our support team.

Posted Feb 13, 2025 - 09:12 UTC

Resolved
This incident has been resolved.
Posted Feb 05, 2025 - 12:45 UTC
Monitoring
Access to all users has been restored. We are still monitoring the situation closely.
Posted Feb 05, 2025 - 10:35 UTC
Update
A fix is being deployed and users should start seeing their access to binders and contracts restored.
Posted Feb 05, 2025 - 09:10 UTC
Update
We have identified the issue and are working on a fix to restore user access to binders and contracts.
Posted Feb 05, 2025 - 07:42 UTC
Update
We are continuing to investigate this issue.
Posted Feb 05, 2025 - 07:22 UTC
Investigating
We are currently investigating an issue with binder visibility. Users may not have access to all binders and contracts that they have been given permission to.
Posted Feb 05, 2025 - 07:21 UTC
This incident affected: my.zefort.com.