Sign In:     

Forum: General Discussion

Topic: Remove Duplicates? - Page: 1

This part of topic is old and might contain outdated or incorrect information

I am only playing / using music videos (no mp3's etc.) - I convert all my music DVDs to individual tracks (VOB's) as soon as I purchase them. Currently I have about 5000 music videos in separate directories, named the same as the original source DVDs. Some of the same tracks (VOBs) appear on more than one source DVD and therefore in more than one directory as well. I often make use of the auto dj feature to play background music and then add tracks from various albums into a playlist. Usually there are duplicates, but the remove duplicates function in the playlist does not seem to work - I have even tried it with only two tracks in the playlist, named the same, but added from two different source directories. I don't know whether it will work with audio only files or other video formats, but with VOBs it ignores me if I select the option "remove duplicates".

Any help?
 
Posted Wed 02 Dec 09 @ 8:30 am
I believe it only removes duplicates of the link to the same physical file, not just videos with the same name. I have had to manually delete duplicates from various sources on all my files and I have over 40,000 videos! Needless to say I am still working on it.
 
Its a never ending job!!removing dupes!!compilations..greatest hits..single packs ,whole albums..
duplicates come soo easy what do u do??no programs perfect for removing them!! u will have songs missing from sum albums so be careful and good luck
 
Hi dj-e-lectric

What format do you use in your video files?

Joey....
 
Hi abeukes

You'll have to find dup's outside VDJ by comparing folder content and then a good tool to use is Total commander here's the link: http://www.ghisler.com/

Good luck

Joey....
 
+1 for Total Commander, very useful tool
 
 
Hrm... I have written a tool that uses some fairly sophisticated comparisons to scan directory trees for files with similar names.

The program then presents the user with groups of files which may be the same name. You can then view the file (in Media Player Classic), remove the file to another directory, or flag it as a not duplicate file so it doesn't ask about it again. This is pretty handy for doing this, although it's still a ton of work. Just 100 times easier than looking manually.

It's written in Perl, runs from the command line and NOT very user friendly, but it gets the job done... I guess if someone is desperate (and is fairly good with the command line and programming stuff) they could have a go at it.

I could possibly make it into an executable as I've done with my program that labels VOBS ripped from a DVD... That would make it much much easier to start using it.
 
Supacon wrote :
Hrm... I have written a tool that uses some fairly sophisticated comparisons to scan directory trees for files with similar names.

The program then presents the user with groups of files which may be the same name. You can then view the file (in Media Player Classic), remove the file to another directory, or flag it as a not duplicate file so it doesn't ask about it again. This is pretty handy for doing this, although it's still a ton of work. Just 100 times easier than looking manually.

It's written in Perl, runs from the command line and NOT very user friendly, but it gets the job done... I guess if someone is desperate (and is fairly good with the command line and programming stuff) they could have a go at it.

I could possibly make it into an executable as I've done with my program that labels VOBS ripped from a DVD... That would make it much much easier to start using it.


Hi Supacon

DVOBS riiped from a DVD, do you mean like tagging the vobs file so file information can be recognized by VDJ?

Thanks

Joey...
 
Yeah, I wrote a program when I started using DVDs that basically does the following:

You rip the DVD with DVD Decrypter spit by chapter.
You enter into excel a list of the artists and titles of each song into separate fields... you then copy them into a text file so that the artsts and titles are delimited by a tab (default if you copy from a spreadsheet).

Then you save that info to a text file called artiststitles.txt in the directory where the VOBs are.

When you run my program while in that directory, it will rename the files in accordance with the contents of the txt. If you have separate intro tracks, the program can merge them into the main file, or leave them separated and label it as an intro. It determines this by the size of the file.

It's been a handy tool for me, but it's not the most slick user interface ever, I'll admit. I can send it to anyone if they are interested.

Note that this is completely different from my program that removes duplicates... and it does NOT automatically figure out what your songs are; The best bet is go to on the manufacturer's website and copy the tracklist from them.
 
Hi Supacon

My main consern is to be able to find a way to get all my .vob file info into VDJ including the genre field, if this is possible then I'm very much interested in your program.
I have alot of files ripped and named already but only the covers import and have no time to manually enter them (six fields per file) into VDJ.

This would be a tremendous help to me.

Thank You

Joey....
 
JoeyKJ wrote :
Hi dj-e-lectric

What format do you use in your video files?

Joey....


Some of my older Videos were ripped with the h.264 codec as .mp4 and the quality is the best in my opinion; however, sony vegas would not read them properly and leave blank frames when editing so now I rip everything as vob (mpg2) because that is the format sony vegas uses natively for editing. After editing I save them as mp4 with the h.264 codec.
 
Supacon wrote :
Yeah, I wrote a program when I started using DVDs that basically does the following:

You rip the DVD with DVD Decrypter spit by chapter.
You enter into excel a list of the artists and titles of each song into separate fields... you then copy them into a text file so that the artsts and titles are delimited by a tab (default if you copy from a spreadsheet).

Then you save that info to a text file called artiststitles.txt in the directory where the VOBs are.

When you run my program while in that directory, it will rename the files in accordance with the contents of the txt. If you have separate intro tracks, the program can merge them into the main file, or leave them separated and label it as an intro. It determines this by the size of the file.

It's been a handy tool for me, but it's not the most slick user interface ever, I'll admit. I can send it to anyone if they are interested.

Note that this is completely different from my program that removes duplicates... and it does NOT automatically figure out what your songs are; The best bet is go to on the manufacturer's website and copy the tracklist from them.


there is a commercially available program called "Flash Renamer" which will do much the same thing but it can use any text file you assign to any rip folder you assign. I actually did a macro in excel that will take the promo only dvd database listing, automatically do all the "preperation" I need on the DVD listing (put the information in the order I need it, delete all the punctuation some programs may read as delimiters, delete the columns I don't use, etc.) and export it as a text file. I then copy and paste the text file into flash renamer, open the folder for the rips for that dvd, hit rename and it is done.....just be sure you rename any rips track numbers 1-9 with the leading zero (i.e 1 becomes 01 otherwise the listing will be track 1,10,11,12....2,20,21 instead of 1,2,3,....) I hate software that only alphabetizes or sequences by one character at a time instead of analyzing the entire string.
 
OK so you have no tagging problems, does VDJ reconize file info on all your fields?

I'm having a problem with my vob files only covers are imported.

Joey...
 
Hi dj-e-lectric

Thanks for that info and I already have Flash Renamer I don't know it could be done this way.

I'm using Decrypter to rip my DVD's, then name the files.

Is there a particular way to also add album, year, genre, comment.
What are the steps to paste the text into F/R.

Thank You

Joey....
 
Here's a query for a problem I have lately. I still use mp3's and with 35000 am sure there is some dupes. What proggie or method is best for removing them
 
JoeyKJ wrote :
OK so you have no tagging problems, does VDJ reconize file info on all your fields?

I'm having a problem with my vob files only covers are imported.

Joey...


VDJ does not support tags on mpeg2 or vob video files. It only recognizes tags on .mp4 video files.....why I have no idea and I don't bother to ask why or when because those are two questions vdj staff won't ever answer.
 
isyahtiger87 wrote :
Here's a query for a problem I have lately. I still use mp3's and with 35000 am sure there is some dupes. What proggie or method is best for removing them


we would all like to know that. I have not found a good automated program for removing duplicates...that's what this thread actually started out searching for.
 
Hi abeukes

You'll have to find dup's outside VDJ by comparing folder content and then a good tool to use is Total commander here's the link: http://www.ghisler.com/

Good luck

Joey....


Hi Joey,

I don't want to remove duplicates on my HDDs, only in the current playlist. I have already written an application which can search and remove dups and another rename tool, where you split the DVDs into separete vobs (DVD decrypter - split to chapters) and then you copy the tracklisting from the web (online DVD shops). The tool can remove tracknumbers, trim spaces around and between words, remove illegal characters (\/:*?"<>|), replace and/or add text, apply titlecase and re-format the track titles (artist - title/ title - artist etc.) as you prefer before it renames the vobs.
 
Hi abeukes

So I take it that you want to edit out dup's within VDJ playList, well I think the list are text files and you would need a text editing program that can find dup titles and artist, (Not just words)

Not sure of one that specifically has that capability, maybe someone can help.


Joey....
 
69%