Good software for tidying up huge image backup mess?

Hey folks,

so I was just listening to episode 221 of DLN, and I got motivated to tidy up the mess that is my backup solution. I have most of my stuff backed up using Syncthing between my PC and home server, and have been considering backing the whole thing up to BackBlaze as well, but none of it is properly ordered and it’s all in a mess.

The photos in particular are in an awful disarray, with around 150GB in 20 different folders full of old WhatsApp pictures from years ago, severeal different folders with camera pictures from 3 phones, pictures that were on an old Laptop etc. Rather than sorting it out, I’ve just added to the mess over the years by backing things up in device specific folders and just dumping my pictures in there. On top of that, I’m pretty sure there’s a whole bunch of duplicate photos in there that are in several different folders because I had copied them to different devices in the past, which I then backed up.

I now want to sort it all into appropriate folders, and was thinking of doing it thematically, e.g. by holidays, pet pictures, nature and the like.

What would you guys recommend for tackling this? I think I’ll first need something that maybe checks hashes or image contents through some algorithms and deletes duplicates, and then something that makes it easy go through photos and move them to a specific folder. I could slowly manually go through it with two instances of my file manager side by side and just cutting and pasting into respective folders, but is there something more efficient?

Has anyone been in a similar situation and found something that works well?

Not as many pictures in as many places, but I had a similar issue. fdupes does great at looking for duplicates. That link shows you how it could work, before you go about destroying any duplicates as well, which helped me learn how to use it in a non-destructive manner (no sense in having backups if you accidentally delete everything from them!).

The recent episode of DLN Extend had Wendy talking about Rapid Photo Downloader. I do not have experience with this, but now that I have de-duplicated most of my pictures, the next thing I want to do is reorganize them into something meaningful, or at least more cohesive naming structure than various phones/cameras over the year has done.

DigiKam is able to detect duplicates and filter up to how much percent different it is.