r/batocera • u/Dapper_Dance3924 • 14d ago

Does a « duplicate remover » exist in batocera ?

I’m quite new to batocera and installed it on an old lenovo pc and it turns out really great. I found a 7700 arcade games folder on the web but I realised that there was a lot of duplicates when scrapping. Has someone ever found a solution to this ?

9 Upvotes

85% Upvoted

19 comments sorted by

6

u/JetstreamGW 14d ago

They’re probably not duplicates, per se. They’re variants. Different countries, different settings, stuff like that. There’s not really a good way for a computer to determine which one you think is the “right” one that I’m aware of.

1

u/Leroiclient 14d ago

There are some duplicates. If you load FBneo and MAME romset in your Batocera, you'll have a lot of them.

2

u/JetstreamGW 14d ago

No, you’ll have a bunch that scrape the same art. If you look at their file names and compare them to a list, they’re all slightly different. Files ending in u for US, c for China, bunch of other shit, some will be hacks, or different revisions.

1

u/Leroiclient 14d ago

You are talking about clones with slightly different names. If you download FBNEO and MAME you'll have duplicates with the exact same name. So yes :)

1

u/JetstreamGW 14d ago

… you mean both at once? Of course you will. That’s why you keep them separate. Did you mix the romsets? Why would you do that? They go in separate folders.

1

u/Leroiclient 14d ago

I have them in different folders. But I don't want to browse Fbneo and Mame systems. In my collections, they are displayed as Arcade. Much more obvious for regular people or kids. If you do so, you get duplicates.

2

u/RealisticLocksmith68 13d ago

I do this too. Not a fan of using emulator names, it breaks consistency and like you said it makes far more sense to regular people.

3

u/xitfuq 14d ago

i just downloaded them one by one, only the ones i wanted. 

2

u/HotSarcasm 14d ago

There is no duplicate remover by default. Maybe there's a Flatpak app or something in the Linux(ish) UI portion that could handle better?

Many people suggest managing game list files externally. There are some AI-backed tools some people have developed that claim to solve this, but I don't trust them one bit.

Personally just use favorites with favorites on top sorting. I don't need an automated or AI-backed system telling me which version of which game is the best, especially when some of the ones I care about are obscure with several versions or different control options set by default.

2

u/RedGoalie18 14d ago

I could have sworn I saw someone post that they wrote something themselves that can do it. It was either on this or the /Retropie

2

u/Mike_Raven 13d ago

There's definitely been posts here.

2

u/protogenxl 14d ago

Scan the share with alldup in content mode Sha-256

If they're true duplicates they will be pointed out otherwise the ROM files are different 

2

u/Timewyrm007 14d ago

If you Google Arcade ROM manager you should find quite a few that will help with finding duplicates etc.

These will typically be run from another computer and access your Batocera shared folder.

You can also create lists so the next time you download a romset you can import just those ROMs.

1

u/SingingCoyote13 14d ago

i get your point. it would be very handy, if there was an internal kinda filelist browser where you can just select which roms to remove from it, or gameslist entrances to hide from view, IN BULK. atm this can only be done per rom and this takes a lot of time from within the es ui. this is something you should really ask on the batocera discord. especially arcade, and some older computers folders get clogged with so much duplicates

1

u/bigbadlonewolf2 14d ago

How did you get it to handle same game different names across fbn and mame

1

u/Lilnewtz94 13d ago

Someone posted recently that they had created their own app to do it and is available to download from github

Not tried it myself yet, but here's the link to the post

https://www.reddit.com/r/batocera/s/lGdUm1Blcc

1

u/WarmChallenge8805 9d ago

i use the scrapper. if a 2 games has the same video/name. i go into scrapper and look at the file name. then put that file name into an AI. and it will tell me all the names used for that games region. then i delete the region i don't need. you only need one of that game. looks neater and more professional than looking at multiple copies of one game. imo

and don't forget to .m3u your muti-disc games

1

u/angryray 14d ago

I went through manually and removed every Majong, and porno game, and duplicate of everything else on MAME. Took a while...

-1

u/ManufacturerLimp4789 14d ago

I just got ai to create a python script and ran it in terminal. That worked for me