Jump to content
Chaos.Lady

Best way to find/organise duplicate files across multiple disks

Recommended Posts

Hey guys, my storage is getting ridiculous and I'm positive that quite a bit of it is multiple copies of crap.

 

Do you know of any GOOD way of searching out duplicate or versioned copies of files across multiple drives and external storage? I'm happy to even buy a program (within reasonable limits) if it does what I need and works well.

 

I will want to ensure that the files are the same or versioned, and then decide on an action for them (move or delete or whatever). I'm also trying to clean up and consolidate files in a reasonably organised way - so consolidate photos and music and stuff that has gotten scattered across drives as well as cleaning out the dupes.

 

There are heaps of crapware out there, but I'm looking for something decent.

 

Suggestions? Recommendations?

Share this post


Link to post
Share on other sites

Just searching the term "find duplicate files" turns up some seemingly useful results.

 

DupeGuru sounds interesting, there's even music and photo oriented versions which do fuzzy matching so can find matches despite different bitrates, photo resize, rotate or small edits.

 

Looks like many have free versions though some are crippled. I'd just hit up the original source for a few (avoid the likes of Cnet which usually bundle Junkware installers), then run through Malware and Virus scan before installing.

 

Obviously you'd want one that allows review before action but I think most/all do it that way.

Share this post


Link to post
Share on other sites

best thing ive ever used is an old version of "Easy Duplicate Finder", but i am unsure if the current incarnation is still free and awesome (http://www.easyduplicatefinder.com)

 

mine is a portable version that came from http://portableapps.com, and the one they include now seems different ( http://portableapps.com/apps/utilities/duplicate-files-finder-portable)

Share this post


Link to post
Share on other sites

I'm using CloneSpy atm. Does what I need it to, and t's free.

Share this post


Link to post
Share on other sites

Coolio guys.

 

Thanks! I'm downloading a few of them to play with this weekend. I'm gonna be working across nearly 10 terrabytes of data - so it's gonna take me a while to scan. :)

 

Will let you know how I go.

Share this post


Link to post
Share on other sites

Yup. I've converted all my previously internal drives to externals (with cases) and currently have 6 of them hanging off the PC. Including the 3tB in the PC now + 3tB + 1tB that was in my previous rig + additional smaller older drives I'm bumping into 10tB mark now.

 

Gah. Consolidation sucks.

Share this post


Link to post
Share on other sites

^

 

Get two 8TB archive drives. Use one as a live drive and the other as a backup, cold or on another PC. Keep USB disks for data you can't send over the network, frisbeemark, etc : p

Share this post


Link to post
Share on other sites

^

 

Get two 8TB archive drives. Use one as a live drive and the other as a backup, cold or on another PC. Keep USB disks for data you can't send over the network, frisbeemark, etc : p

 

keep in mind they store files very differently to a normal drive. do NOT use them in a RAID or NAS device. these are to be directly HBA connected and used as such.

I'm sure firmware will fix it in time, but for now, use as a normal HDD.

Share this post


Link to post
Share on other sites

They'd be pretty horrible raid drives. They're great as JBODs.

 

Even in JBOD-ish storage solutions, they fall over.

BTRFS with Parity = data loss

 

Normal Ext4\NTFS\ReFS\HFS+ disk only; for now.

As i said i'm sure firmware will sort it out.

Typical seagates though......

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×