> Since there's no way to roll back server upgrades,
Apart from the obvious comment about snapshots (at the volume or machine level), really most of the time your problems are because of a specific package.
And while most package managers don't support downgrades per se, you can just remove the offending package and install the old one. Nine times out of ten that would give you more time to fix the problem.
That said, you need to test before you go to production, no matter how trivial the patch may seem. Staged rollouts, a separate environment, or both.
And that's also the elephant in the room in Ted's rant. Sure, if we didn't have to test anything, we could update every 6 months. But we have to, and we can't.
Apart from the obvious comment about snapshots (at the volume or machine level), really most of the time your problems are because of a specific package.
And while most package managers don't support downgrades per se, you can just remove the offending package and install the old one. Nine times out of ten that would give you more time to fix the problem.
That said, you need to test before you go to production, no matter how trivial the patch may seem. Staged rollouts, a separate environment, or both.
And that's also the elephant in the room in Ted's rant. Sure, if we didn't have to test anything, we could update every 6 months. But we have to, and we can't.