There's a trivial way to not re-add data that was removed: don't do it without user opt-in, whom admittedly you have access to ask at the moment of data collection. If you don't have the ability to ask users to opt in, you probably shouldn't be collecting the data anyway, with very few exceptions like criminal records.

edit for clarity: by criminal records, I mean for the official management of them, not for scraping their content.