This is a process that some of Google followed as of 10 years ago. I cannot tell you whether internal pressures have caused groups to not follow it or they are measuring the wrong thing.

I know from experience that A/B testing can make it hard to have a coherent product design (Amazon's page is a good example of that). But it is still much, much better than the usual alternative.