The fragmentation problem for users is that they must be conscious of the deviations of the particular handset/carrier/version/store fragment they are buying in to or potentially suffer the consequences of their ignorance.

That is, the Android phone you buy may be a very different experience than the Android phone I buy, and that creates a lot of opportunity for dissatisfaction internal to the platform which simply would not be there with a strong first-party stance on how things ought to be done.