Sign In:     

Forum: General Discussion

Topic: A programme that will find duplicate files in different folders - Page: 1

This part of topic is old and might contain outdated or incorrect information

Anyone know of a programme that will find duplicate files in different folders?
There are some audio & video files that i have that I have mistakenly put in more than one folder, and it will take forever to go through all the folders manually to find the duplicates so wondering if there is a programme that will do this?
 
Posted Thu 10 Sep 09 @ 8:04 am
Simply do a google search "Duplicate file finder".
 
abeetech
 
Be careful how you use any duplicate finder utiliy. I had a bad experience once... Once it found duplicates, it then asked how I want to handle it. I chose to automatically remove duplicates. Bad mistake... I have about 20,000 karaoke songs that are not zipped and had the same file name as I buy my songs from various studios such as Sound Choice and top hits monthly for example. My club customers sometimes ask me for a certain brand for a specific song as they are use to that version at another club and they like it better so I end up with duplicates of many song titles. When I get my songs i get rid of the disc number and only keep the song title and artist name so VDJ looks organized. Up until 2 years ago i always opened the zip files so i would have the 2 files in a folder(cdg &mp3). Imagine, I had the same song by various studios in different folders.... here's what happened. It removed a cdg file from 1 folder and removed an mp3 from another folder so when I searched for a song in VDJ I may have found the mp3 but it lost the cdg file so there were no words on the screen. This happened to zip files as well. I lost one or more of the other versions in the cleanup. Good thing I have 2 backups of everything. I have spent years adding to my collection and it only took 1 hour to screw up. So be careful....
 
 
I need a program that will find duplicate files by selected fields of the tag or delimited fields of the name. All of the duplicate finding programs use the file size and crc which is fine for finding EXACT duplicates or they use the FULL file name....again good for EXACT duplicates. What I need is a program that will find a human's definition of a duplicate...I need to find Usher-Yeah by promo only, screenplay, rockamerica, etc. or Usher-Yeah from the promo only hot series, the express series and the best of classics 2004 series and let me decide which one I want to keep.

See the problem is the file size and crc will not be the same between the rip from the hot series and the express series so the program won't find it as a duplicate. Since I include the series name in the title and tag one when I rip the file the names will be like USHER_YEAH_PO HOT.mpg and the other will be USHER_YEAH_PO EXPRESS.mpg so it won't read those as duplicates either as the entire file name is not the same (same with the tag).

Is there a program that will let you select which tag fields it will ONLY look at in deciding a duplicate or not (i.e. read ONLY the artist and title field and ignore the rest of the tag or title). That way I can just search for USHER-YEAH and every one from every source will show up regardless of size or where I get it from.
 
What operating system are you using? If its vista it's search function is great.
 
drzinc wrote :
What operating system are you using? If its vista it's search function is great.


only if you know what you are searching for. I am LAZY, I paid money for a computer....I want it to search for my duplicates, give me a listing of them and then let me decide what I want to do wigh each of them. With the power of computers and all the wonderful things they can do like VDJ, etc. this program does not seem as though it is too much to ask a computer to do....but as always I am probably the only one who wants it to do what I want it ti do so there is no market for anyone to write such a thing.
 
Before removing any duplicates. Place all your gig music on a separate drive so any errors can be limited. I have yet to find one which is 100% reliable as all have to be used with caution. I find MP3 renamer a good tool as it also finds duplicates
 
dj-e-lectric wrote :

I want it to search for my duplicates, give me a listing of them and then let me decide what I want to do wigh each of them. With the power of computers and all the wonderful things they can do like VDJ, etc. this program does not seem as though it is too much to ask a computer to do....but as always I am probably the only one who wants it to do what I want it ti do so there is no market for anyone to write such a thing.



I'm with ya here, and there IS a solution.

eMusic tag editor is the one I use for both EDITING and RENAMING tags/tracks, but also finding and removing duplicates. It has a complete "duplicate finder" function. (I just spent a few hours recently doing this very exercise) And you can define what to search for - tags, filenames, combination of both/each. It's pretty good.

It's by www.abyssaudio.com but: ***CAUTION, it appears their site has been hacked and has some trojans in it*** (I've had it for years so don't need to get to their site.) I forget how much $$ but it's pretty cheap, like under $30.
 
chriso710 wrote :
Before removing any duplicates. Place all your gig music on a separate drive so any errors can be limited. I have yet to find one which is 100% reliable as all have to be used with caution. I find MP3 renamer a good tool as it also finds duplicates


Tried that one, but again it only finds duplicates based on the FULL file name or tag so it won't find duplicates from different services if you have the service name in the file name or tag.

The closest I have found is tag & rename can sort by any field on the tag (progressive on 2 fields) and than manually go through the list looking for ones that are "back to back" in the listing, but with 12,000 mp3s and over 20,000 videos I don't really have the time or ambition to do it manually (hell what's another tb or 2 at $100 a pop per drive).
 
dj-e-lectric wrote :
The closest I have found is tag & rename can sort by any field on the tag (progressive on 2 fields) and than manually go through the list looking for ones that are "back to back" in the listing, but with 12,000 mp3s and over 20,000 videos I don't really have the time or ambition to do it manually (hell what's another tb or 2 at $100 a pop per drive).


You have to try eMusic Tag editor. Google it and find many download sites for 15 day free trials.

http://www.softpedia.com/get/Multimedia/Audio/Tag-Editors/eMusic-Tag-Editor.shtml

http://www.brothersoft.com/emusic-tag-editor-71185.html

The "duplicate finder" will let you scan and find duplicates based on tag, combination of tags, and/or filename. Last run i did was set on "Title" only and 5 minutes later it shows a complete list if all the tracks with dup titles. Then you can easily choose whatever action you want (in my case, i deleted my "lesser quality" (in kbs) versions. My db is 21,000 tracks. - couple of hours of cleaning got rid of a few hundred junk tracks.

Of course, if your tags and/or file names are hosed, then this (or any other solution) won't work that great and it will take you longer.
 
 
You can try Clone Remover. It will help you find duplicate images (photos, pictures). Search by contents – files may have different names, but similar content. You can also use filters to search for duplicates of a certain type only. Try it.
http://www.moleskinsoft.com
 
 
I highly recommend Audio Dedupe. It will actually listen to all your music and find duplicates even if they have different names or don't have tags. It's very good. Only $30.
 
I highly reccomend the software DONT REMOVE THE DUPLICATES and its free too.

Why should you ? The HDs are so cheap. You will certainly find some good reasons too. Example. A song might be in a collection like UK Top, the same song could be in a Artist's Album and so on... Why should be just 1? Think about it.
 
djdad wrote :
I highly reccomend the software DONT REMOVE THE DUPLICATES and its free too.

Why should you ? The HDs are so cheap. You will certainly find some good reasons too. Example. A song might be in a collection like UK Top, the same song could be in a Artist's Album and so on... Why should be just 1? Think about it.


LOL !!!
 
Very funny DJ Dad :)
 
DJ DAD has a valid point.

I have several Sinatra albums that would get ruined by a dupe checker. Each compilation has value (Jobim, Sands, Basie, etc).

Also, you might have the original track from the 50s and reissue (simulated stereo that sounds bad) and the dupe checker might delete the former.

BUT..... I get several subscription CDs and having those dupes removed (exact tracks and mixes) saves several GB.

Like always, be careful and keep backups.
 
49%