My problem with the "release-branch" approach is mostly that your prod releases end up being big-bang multi-feature releases. Any bug in one feature can delay the entire release. More changes usually means more room for things to go wrong. It's also harder to debug issues when they do happen, and they will happen eventually, regardless of how thorough your test cycle is. Your developers may also have lost context if they time between writing the code and deploying it is so long (eg if this process forces you to monthly or quarterly releases).
> Once the release branch reaches the appropriate level of stability, it should be gradually rolled out, zone by zone, to production. Zones should have some soak time before further zones are touched. The response to production anomalies should be a roll back to the previous version.
I 100% agree with you on this. Canary releases, followed by gradual rollouts with the option to roll back are ideal.
> Do this by enabling features and changes via configuration rather than via binary changes.
If your CI/CD pipeline is fast enough that you can deploy on each commit to master (multiple times daily), then you can roll forward in the rare situations that rolling back isn't an option. Feature flag frameworks certainly have some useful features, but they add a layer of complexity to your deployments which feels like a band-aid to help get around slow deployments. Totally valid for eg an iOS app that takes 2 weeks for Apple to review, but if you're deploying to the web where you can push new releases whenever you want, I don't think the added complexity is worth it.
I think your thorough, managed release approach is absolutely appropriate when the consequences of failure are life-threatening, or heavy revenue losses or brand damage. But the worst case scenario for most of the software most of us work on are closer to "1% of users could potentially see an error message during a 5-60s window before we detect the increased error rate and roll back the canary release"
> Once the release branch reaches the appropriate level of stability, it should be gradually rolled out, zone by zone, to production. Zones should have some soak time before further zones are touched. The response to production anomalies should be a roll back to the previous version.
I 100% agree with you on this. Canary releases, followed by gradual rollouts with the option to roll back are ideal.
> Do this by enabling features and changes via configuration rather than via binary changes.
If your CI/CD pipeline is fast enough that you can deploy on each commit to master (multiple times daily), then you can roll forward in the rare situations that rolling back isn't an option. Feature flag frameworks certainly have some useful features, but they add a layer of complexity to your deployments which feels like a band-aid to help get around slow deployments. Totally valid for eg an iOS app that takes 2 weeks for Apple to review, but if you're deploying to the web where you can push new releases whenever you want, I don't think the added complexity is worth it.
I think your thorough, managed release approach is absolutely appropriate when the consequences of failure are life-threatening, or heavy revenue losses or brand damage. But the worst case scenario for most of the software most of us work on are closer to "1% of users could potentially see an error message during a 5-60s window before we detect the increased error rate and roll back the canary release"