That isn't how elections work. You can't assume that all voters in non-swing states would never change their vote.
You could say the same thing about a popular vote where a candidate wins by 10 votes - "oh, our entire system is decided by 10 voters? how unfair"
For the US election, votes in other states resulted in a situation where only 5 were swing states. If those voters in non-swing states voted differently in enough numbers, then there wouldn't be just 5 swing states.
Look at any of the elections in the past 40-50 years and you can clearly see that which states are "swing states" and which aren't changes over time.
Something like 130,000 voters voting the other way would have changed the election.
However unlike in 2016, a strong plurality voted Trump - 2 million more than voted for Harris. Had Harris won the popular vote that argument might have meant something.
Not sure how true this is and I guess it is moot at this point (except as far as thinking how to ensure all legal votes are counted in the future):
https://www.gregpalast.com/trump-lost-vote-suppression-won/
"Trump lost. That is, if all legal voters were allowed to vote, if all legal ballots were counted, Trump would have lost the states of Wisconsin, Michigan, Pennsylvania and Georgia. Vice-President Kamala Harris would have won the Presidency with 286 electoral votes. And, if not for the mass purge of voters of color, if not for the mass disqualification of provisional and mail-in ballots, if not for the new mass “vigilante” challenges in swing states, Harris would have gained at least another 3,565,000 votes, topping Trump’s official popular vote tally by 1.2 million."
Which makes it insane that such massive disruption can happen as a result. When the result is balanced on a knife edge, the outcome ought to reflect that, instead of swinging dramatically one way or the other. I don’t know how you design a system like that, but this is nuts.
A similar number removed citizenship from all Brits in 2016.
A few hundred votes slipped the US election in 2000 and caused the invasion of Iraq.
You tackle this by having societal norms and strong institutions. The internet broke that. The concentration of wealth broke that. The unprecedented algorithmic manipulation broke that.
Brexit was especially wild. At least the US has the excuse that the law says we do it this way. To make such a drastic change based on a simple majority in a nonbinding referendum is really out there.
The US was the first modern democracy, since then we’ve learned how to make better ones (proportional representation parliamentary systems), but the US system is just stable enough to keep limping along.