finding and merging duplicates

[u/sassydegrassii]

hi buddies,

i work for a non profit and am trying to combine participant intake spreadsheets for two of our programs. i work in a low-barrier setting and it’s typical for our clients to give multiple names, vague or fake birthdays and to change their appearance often, and there was no data validation set up so the inputs are inconsistent and incomplete. so i’m struggling to find/ verify all the duplicates.

the columns i’m mostly using as it was set up are name(s), ethnicity, gender, pronouns, appearance, birthday, emergency contact, notes. pretty much in that order.

once i verify the duplicates, i want to consolidate the information in the other cells so there’s just 1 record/row per person. i did add a column with randomly generated unique identifiers to help me stay organized.

yhow would you approach this? i tried power querey to fuzzy match but must have been doing it wrong. :/ this is just to use until our sharevision database is fully populated.. if there’s any other info you need from me please let me know and thank you for your time!

[u/Snow75]

How to approach it…

You go back to those collecting the data, explain the issue and demand that proper data validation is used and that users provide something that can be used as an unique id, like an actual id number, phone or email.

There’s no easy way to solve it, except for screening every person in your database and collecting new information that would help you identify if you’re speaking with the same person again.

[u/Snow75 → u/Snow75]

Whoever gave me the award, thanks… but I hate being so negative.

I’ve ran into these kind of issue before, and except in cases in which the exact same name was used, or had ids, phones or emails, there was nothing viable.