Some philanthropic donations are kinda prestige-for-cash deals. My employer, Oxford University, accepted a large chunk of change from Wafic Saïd to endow their business school. If they hadn't taken it, he'd certainly still have the money. But we wouldn't have lowered our own reputation by associating ourselves with a... not spotless businessman.
If it's done anonymously, then sure: you're quite right. But if it isn't then there is, in fact, a price. And it's not always a great deal.
> But that money is going to exist whether you accept it or not
But by accepting the donation you may be signaling that you're ok with how the donor made their money. By turning it down, you may also have a PR opportunity to spread a positive message.