Roy/SAC Intro Development

When I decided to pack my collections of scene art logos and pixel art fonts to make them public for others to download, I got the idea to create a nice little intro that comes with the packages. Following an ancient tradition in the scene, an intro does serve the purpose to announce the release and to relay greetings and messages to fellow sceners. The intro (or Cracktro), which is short for "Crack Intro") pre-dates the text NFO files by many years. In fact, NFO files are something that was developed pretty late in the game and was only extensively used for releases on the PC.

NFO files were pretty much a novelty on machines like the Commodore 64 and also not common for the use in releases on the Commodore Amiga. I created a NFO file and file_id.diz for my collections releases of course.

Anyhow, I created an Intro with the help of the OSDM (OldSkool Demo Maker) intro engine developed by the fellow German scener with the name Peace/Testaware and released my collections with that new intro of mine.

As always, I was not 100% happy with the results and started tweaking, changing and expanding things. Here are the results of that tweaking and one intermediate version that will probably never be used for a release ever. The current version will most likely be used for any update to the current packages or for some new collections (I have a lot more stuff floating around here than just logos and fonts hehe).

Current Version 4.5 of the Intro

Code: Peace/Testaware
Design, Graphics, Concept & Direction: Roy/SAC
Music: Mantronix/Razor 1911

• Watch on Vimeo
• Watch on YouTube
• Download Video Version in AVI Format (263 MB)
• Download Win32 Executable (370 KB)

Previous Versions of the Intro

The credits for the previous versions of the intro are slightly different. I used a different intro tune there.

Code: Peace/Testaware
Graphics, Design and Concept: Roy/SAC
Music: track "Cognition" by Jozz/TRSI

Version 1	Version 3
Watch on Vimeo Watch on YouTube Download Video in AVI Format (63 MB) Download Win32 Executables	Watch on Vimeo Watch on YouTube Download Video in AVI Format (54 MB) Download Win32 Executables
Note: You can always download the AVI version of the intro at VIMEO.com. Look to the lover right of the detail page for each of my videos there.

I thought it was a nice idea to include greetings to most of the guys that I knew and dealt with in the scene and still remember, so I started creating a list to collect all the handles of those folks and included them in the second scroller (the smaller copper-bars scroller at the bottom). Most folks are from Germany, especially Berlin, where I grew up, but it also includes some folks from other countries in Europe, Israel, Canada and the United States.

Long List of Personal Greeting (Alphabetically)

Ace , Acen , Ados , Akim , Alf/ACC , Alf/Melmac , Alien/Nostromo , Alphatron , Amblin , Amok , Angel Death , Antibody , Argon Factor , Asphyx , Axess , Bacchus , Bad Boy , Beatmaster , Ben Garrett/Defacto2 , Bionic , Black Spyrit , Blacklord , Blaster , Brian O Neil , Brockhaus , Bug Lord , Busy Bee , Buttcher (Berlin) , Bymn , Caihai , Camel Joe , Capone , CC Catch , CeeJay , Censored , Cercyon , Chester , chub DrUid , Coast , Cosmic , Count Zero (Berlin) , Crackfield , Creator of Hell , Crypton , Cyber Brain , Cyric/DOD , Cyz , DAC , Dalezy , Darkside , Datacrime , Deathbringer , Deathwalker , Deleter , Deryll , Dipswitch , Dr.Acid , Dr.Lazy , Dragnet , Dream Design , Dusty , Elvin , Elvin Knox , Elwood , Exterminator , Fan Fan Latulipe , Fatman , Feldmann , Ferrex , FFC , Firebird (Berlin) , Fli7e , Fox , Frank Borally , Frog , Fulcrum , Garfield , Gee , Giovanni , Glink , Goonie , Grap , Gregory , Grymmjack , Guiver/EXS , Gumbo , Hagman , Hetero , Iceman , Iceman/WOF , iD MUD , Idefix , Idiana , Intruder , Jagor , Jaset , Jason Scott/Textfiles.com , Jester Radics , Jkowall , Johnny Cyberpunk , Kaethe , KC Kid , Killer , Kobold , Kral , Krazy Nomad , Larry ILG , LBM , Leecher JTC , Lemming , Logic , Lord MX , Lord Scarlett/SixteenColors.net , LTD/BLH , Mach One , Mad Danger , Mad Mac , Madmax (Berlin) , Madmax/CPI , Magician , Magnum , Marky , Matador , Matt , Maverick , Max/DOM , McLoud , Mefis , Menion Leah , Mitfit , Monday , Monster , Mozart , Mr.Axxess , Mr.Diesel , Mr.Lightning , Mr.Mad/NRG , Mr.Mixx , Mr.Rox , Mr.Swapper , Mr.Twister , Neon , Neophyte , O-Dog , Oliver Stone , OSHO , Overdoze , Palladin , Paragon , Paso , Pennywize , Pilot , Pitbull/RZR , Pitty , Poldi , Proton , Psychofox , Puschel , Quickmix , Rad Man/ACID , Radiocity , Rainer/AFL , Rainer/SAC , Raiser , Ramses , Ranx , Ratso , Rave , Rescue 911 , Retaliator , Rhett Jones , Ripwave , Roach , Rowdy , Royal Knight , Salty , Sextronix , Shamen , Shockwave , Shot , Skin , Sky , Slash , SLE , Snoop , Snoopy , Sonic , Spacerat , Sparky , Speedlock , Spoon , Sporky , Squizzy , SSC , Starx , STB , Stone , Stonehedge , Stryker , Suicide , Svenzzon , Synec , Term , Termi , The Barrier , The Driver , The Outlaws , The Punisher , The Shark , The Syndicate , Thunder , Thunderhawk , Tic , Toast , Toony , Toth , Toxic Trancer , Trans/BF , Trinitron , Trixter , Tronics , Trouble , Twin! , Twinbit , Twister , UCI , Ufonaut , Ultimate , Ultimate Warrior , Uridius , Urmel , Vax , VFast , Vincent , Virago , Werner , Wessi , Whiteheat EXLC , Wilkins , Wing , Wizard , Xan , Xeek , Xerxes , XLR8OR , XOX , Z80 , Zerovision , Zippoid , Znake , Zyrix

If you have any comments or feedback to the intros, let me know and post a comment below.

Thanks and Cheers!

Carsten aka Roy/SAC

Microsoft Internet Explorer Search Providers Import and Export

Microsoft Internet Explorer 7 and 8 and also Mozilla Fire Fox come with a feature called "Custom Search Providers" or Search Scopes. It allows you to add web sites and search engines to your search box to search them directly, if you need to. For example could you add Amazon.com search to your search providers and search the Amazon.com site directly from your search box, if you are looking for a book, CD or DVD and use Amazon.com as primary online shop for this kind of purchases.

You can get a selection of custom search providers at this Microsoft web site. There you can find a number of options for popular web sites and other search engines. Also available there is the option to create your own custom search provider for your favorite web site. This is possible, as long as the site provides a site search option and passes the search parameters, such as the search term in the URL.

If you have a web site yourself and want to offer your visitors the option to add your site search to their web browser, then you can do that as well. I did this for my small project SQLHunt.com, a MS SQL Server resources meta-search engine based on Google Custom Search Engine. See the link at the bottom of the homepage that reads "Widgets and Gadgets". One of the options there is to add the SQLHunt.com search to your web browser.

The search providers page for Internet Explorer on the Microsoft web site.

All nice and good and I made myself good use of this feature. I have by now probably 20+ custom search providers added to my search box. Some via the Microsoft web site, others custom or via the "add" option at a web site directly. There is unfortunately no easy way to copy your search providers to another computer or to back it up and re-install, if you do a re-install of your current system.

The search providers (or search scopes) are stored by Microsoft in the Windows System Registry. You can find your current search scopes and export them via the system tool RegEdit.exe to a file at

HKEY_Current_User\Software\Microsoft\Internet Explorer\SearchScopes

For each search provider exists a sub-key, in case that you only want to export specific search providers and not all of them. You can then take the exported file (with the extension .REG), copy it to the other machine and double click on it. You should be prompted to acknowledge that you want to import the file into the system registry. Do that and your search provider settings will be imported.

Here is a simple BATCH script to export and import your search provider settings.

SearchScope.bat Code

   1:  @echo off

   2:  CLS

   3:  if "%1"=="" (CALL :SELOPT) ELSE SET OPT=%1 

   4:  if "%2"=="" Call :GETFP   

   5:   

   6:  IF %OPT%==IMPORT Call :DoImport

   7:  IF %OPT%==EXPORT Call :DoExport

   8:  Echo Finished!

   9:  Echo.

  10:  pause

  11:  GOTO :EOF

  12:   

  13:  :SELOPT

  14:  echo Microsoft Custom Search Provider Import/Export Tool

  15:  echo by Carsten Cumbrowski aka Roy/SaC at http://www.roysac.com/blog

  16:  echo.

  17:  SET /p OTmp=Enter (I) to Import, (E) to Export or (A) to Abort: 

  18:  IF "%OTmp%"=="I" SET OPT=IMPORT&SET OK=1

  19:  IF "%OTmp%"=="i" SET OPT=IMPORT&SET OK=1

  20:  IF "%OTmp%"=="E" SET OPT=EXPORT&SET OK=1

  21:  IF "%OTmp%"=="e" SET OPT=EXPORT&SET OK=1

  22:  If NOT "%OK%"=="1" EXIT

  23:  GOTO :EOF

  24:   

  25:  :GETFP

  26:  echo.

  27:  echo Please Specify the name and path for the file to %OPT% 

  28:  Echo Enter "a' to abort.

  29:  echo. 

  30:  SET /P FNAME=File Name: 

  31:  if %OPT%==IMPORT (

  32:    IF "%FNAME%"=="a" EXIT

  33:    IF NOT EXIST "%FNAME%" Call :GETFP

  34:  )

  35:  GOTO :EOF

  36:   

  37:  :DoImport

  38:  REG Import %FNAME%>NUL 2>&1

  39:  Echo Search Scopes Imported

  40:  Goto :EOF

  41:   

  42:  :DoExport

  43:  REG Export "HKCU\Software\Microsoft\Internet Explorer\SearchScopes" %FNAME%>Nul 2>&1

  44:  Echo Search Scopes Exported to %FNAME%

  45:  GOTO :EOF

Here is a more complex example script. It reads the settings from the HKEY_USER branch rather than the HKEY_CURRENT_USER one. The script looks up the SID for the current user and exports the registry settings, replacing the SID with a place holder to be substituted when imported again elsewhere.

This was more of a "proof of concept", since you do not have to go through all this length as the script above illustrates. However, this batch could be modified to export the settings from other users on the same machine. It was not easy to accomplish and very tricky to do. Just for a test, look at the code and tell honestly, if you understand every statement used there. :)

The only thing you would have to do, is replacing %%USERPROFILE%% with the path to the users home directory (e.G. "C:\Documents and Settings\UserName"

SearchScopesImportExport.bat Code

   1:  @Echo off

   2:  cls

   3:  SETLOCAL ENABLEEXTENSIONS

   4:  SETLOCAL ENABLEDELAYEDEXPANSION

   5:  if "%*"=="" Goto :ShowUsage

   6:  if "%~2"=="" Goto :ShowUsage

   7:   

   8:  SET RootKey="HKLM\software\Microsoft\Windows NT\CurrentVersion\ProfileList"

   9:  Set Val=ProfileImagePath

  10:   

  11:  Call SET CurProf=%%USERPROFILE%%

  12:  SET CurSID=""

  13:   

  14:  Echo Determining Current User's SID ...

  15:  Call :GETSID %RootKey%

  16:  Echo The current users SID is %CurSID%

  17:  Call Set RegTmp=%Temp%\%CurSID%_SearchScopes.reg

  18:   

  19:  IF "%1"=="export" Call :ExportScopes "%~2"

  20:  IF "%1"=="import" Call :ImportScopes "%~2"

  21:   

  22:  Echo.

  23:  echo Done Processing

  24:  echo.

  25:  pause

  26:  echo.

  27:  Goto :EOF

  28:   

  29:   

  30:  :GETSID

  31:  FOR /F "tokens=7* delims=\" %%i IN ('REG QUERY "%~1"') DO (

  32:    FOR /F "tokens=2*" %%a in ('REG QUERY "%~1\%%~i" /v %Val% ^|FINDSTR %Val%') DO (

  33:       CALL SET ldclientdir=%%b

  34:       if "%CurProf%"=="!ldclientdir!" (

  35:         Set CurSID=%%~i

  36:       )

  37:    )

  38:  )

  39:  GOTO :EOF

  40:   

  41:  :ExportScopes

  42:  echo Exporting Search Scope Settings ...

  43:  REG Export "HKU\%CurSID%\Software\Microsoft\Internet Explorer\SearchScopes" %RegTmp%>Nul 2>&1

  44:  Echo Replacing SID with place-holder CURRENTUSERSID ...

  45:  Call :SandR %RegTmp% "%~1" %%CurSID%% CURRENTUSERSID

  46:  goto :eof

  47:   

  48:  :SandR

  49:  ::1 = input file

  50:  ::2 = output file

  51:  ::3 = search string

  52:  ::4 = replacement value

  53:  if EXIST "%~2" DEL /Q "%~2"

  54:  for /f "tokens=1,* delims=]" %%a in ('"type "%~1"|find /n /v """') do (

  55:     CALL SET _result=%%b

  56:     if NOT "%%b"=="" (

  57:       CALL SET _fchar=%%_result:~0,1%%

  58:       if "!_fchar!"=="[" (

  59:         set "line=%%b"

  60:         if defined line (

  61:            call set "line=echo.%%line:%3=%4%%"

  62:            for /f "delims=" %%X in ('"echo."%%line%%""') do %%~X>>"%~2"

  63:         )   ELSE echo.>>"%~2"

  64:       ) else (

  65:           echo %%b>>"%~2"

  66:       )

  67:     ) else (

  68:         echo.>>"%~2"

  69:     )

  70:  )

  71:  goto :eof

  72:   

  73:  :ImportScopes

  74:  if NOT EXIST "%~1" GOTO :ShowUsage

  75:  echo Preparing Registry Data for Import ...

  76:  echo Replacing place-holder CURRENTUSERSID with SID ...

  77:  if EXIST %RegTmp% Del %RegTmp%

  78:  Call :SandR "%~1" %RegTmp% CURRENTUSERSID %%CurSID%%

  79:  Echo Importing Search Scopes ...

  80:  if EXIST %RegTmp% REG Import %RegTmp%>NUL 2>&1

  81:  goto :eof

  82:   

  83:   

  84:  :ShowUsage

  85:  cls

  86:  echo.

  87:  echo USAGE

  88:  echo   Script.bat import^|export IMPORTFILE.REG^|OUTPUTFILE.REG

  89:  echo.

  90:  echo Example Export of current Search Scopes to a file 

  91:  echo with the name MySearchScopes to the local directory C:\Data

  92:  echo.

  93:  echo   Script.bat export c:\Data\MySearchScopes.reg

  94:  echo.

  95:  Echo Example Import of Search Scopes from file generated via this script

  96:  Echo.

  97:  Echo  script.bat import c:\import\MySearchScopes.reg

  98:  echo.

  99:  pause

 100:  echo.

That's it. I hope that you will find this little tool and information useful.

Cheers!

Carsten aka Roy/SAC

In Case You Forgot Adobe Photoshop at Home, don’t worry!

In the case that you forgot Adobe Photoshop at home and need to make some last minute tweaks of some nice and cool graphics for your blog or MySpace page, while at a friends house, who only has MS Paint installed on his Computer, then you don’t have to decide between no image or crappy MS Paint tweaked image anymore.

There is now a nice third option available. All that your friend needs is a web browser and an Internet connection… and who doesn’t have this nowadays. Finding a home with black and white television is probably easier than one with a computer without Internet, right?

Anyhow. There have been in the past already multiple attempts to provide a practical photo and image editor on the World Wide Web. Yeah, you could tweak photographs a little bit, remove some red eyes, flip the image, crop it and adjust the colors and contrast or brightness, but that was about it. Pixlr has something for that as well, called Photo Express, where you can do some last minute tweaks to a photo before you put it up on Flickr or Photobucket.

But if you had to do a little bit more tweaking to an image, you still required a desktop application installed on your computer. Pixlr provides a neat alternative to that. Their image editor is more than just a nice toy where you can draw squares and lines like in MS Paint. The Pixlr image editor supports Layers and has some features like its big cousins, the Adobe Photoshop’s and alike. The interface also matches closely the look and feel of Photoshop, which will make designers who live and die with PS, feel like home instantly.

I actually felt so much at home that I got stuck when I tried to use the right click context menu or keyboard short-cuts that I am used to. The editor is Flash based and not Ajax like Google Docs and thus not supports the right mouse button (I wonder how this works on a Macintosh :) ).

The first thing you do when you go to the editor on the web is to select if you want to start a new image, load a picture from your computer or directly from a web site (e.g. Flickr). Supported are the standard formats JPEG, PNG and GIF.

I can only suggest to jump right into it and play around with all the features available yourself. That’s what I did and it worked out pretty well. If you don’t think that it feels right or could work for you, you just leave and never come back. That’s the beauty of a web based (and free) application. There is nothing to uninstall or something like that, because you never installed anything to begin with.

Here is a screen shot of my experiments with the editor.

Here is a screen shot of the light weight Photo Express application for last minute photo tweaks. Notice the photograph? It’s the only photograph with that many SAC members in it. It was taken at the BDN Party 3 in Berlin, Germany (where I won the “Fast ANSI” graphics compo and a T-Shirt as a prize hehe).

A quick note for web masters and web developers. Pixlr provides a web API that you can easily integrate the image editor into your own web application. The API has some nice features that let you control, if the user can save the image on his computer or not, if he can change the title, or if you specify it for the user, the source (e.g. an image template that you automatically load or just the image that the user uploaded on your web site and wants to make some tweaks to it, without doing it on his own computer and re-upload the modified image again. The possible implementations are probably somewhat limited, but in those few cases where it makes sense, having the option to provide a pretty good editor versus none or just a crappy one comes in very handy.

So go and check it out. It’s free, no trial, no premium versus basic free features or stuff like that (at least I didn’t see anything like that, when I checked out their web site and products).

Cheers!

Carsten aka Roy/SAC

MS DOS Commands, Wild Cards, Input/Output Redirection and Variables

While I was looking for ways to make my life easier by automating things, I rediscovered the MS DOS batch features of the latest MS DOS release and even more extended version of MS DOS for the early Microsoft Windows 32 bit Operating Systems, like Windows 95, Windows CE and Windows 98 (dubbed MS DOS 7).

You can accomplish a lot of things right from the DOS prompt without needing to program fancy Windows Applications or something like that.

Many things where I thought the use of “real” programming languages like Visual Basic or VBScript would be necessary, can be accomplished by using sequences of MS DOS commands in a Batch File (.BAT).

BATCH commands are supported by any Microsoft Windows Operation System right out of the box. You do not have to install anything, not mess around with permissions nor do you require Administrator rights and permission to write, edit or execute batch scripts.

The Basics for Newbies

To learn about the available commands and functions that you can use in BATCH files, open a MS DOS window first. To do that, click on “Start”, then “Run”, type “cmd” and then press the ENTER key on your keyboard.

In the MS DOS window type “help” and then press ENTER. Returned will be a list of commands with a brief description of their purpose right next to each of them. Type “help COMMAND”, where COMMAND stands for any command that was listed by “help”, to get a detailed documentation and description of the individual DOS command. There is actually a BATCH file that was written by Rob Van der Woude, which generates a HTML document from those “help” commands, something like a documentation or reference, if you will. You can download the script source here. Save it as AllHelp.bat and then execute it. When it is done, you should have a new file with the name “allhelp.htm” in the same folder as the batch file itself. You can open the HTML file with any web browser.

That covers the basics. The rest of this post is more advanced and for people who are familiar with the basic MS DOS batch features and syntax.

MS DOS (and Windows) Wild Cards

I will explain the wildcards characters used in MS DOS and also Windows for File and Folder Filtering in commands like the "DIR" command and other file based operations. It causes sometimes for confusion and even confused me that I decided to look it up and also conducted some tests to verify the claims made by various people.

There are two (2) wild card characters that can be used for the file system commands in MS DOS and Windows; the question mark character (?) and the asterix or star character (*).

• ? Matches any single (1) character. It also matches no (0) characters (none), if it is being used in the leading or trailing position of the filter.
• * Matches matches any and no characters, regardless of its position in the filter

Here are some examples to illustrate the subtle nuances between each of the two wildcards.

abc*f   matches abcdf, abcdef and abcf
abc*    matches abc, abcd and abcde
*def    matches def, cdef and abcdef

abc?ef  matches abcdef but NOT abcef
abc?    matches abc and abcd but NOT abcde
?def    matches def and cdef but NOT abcdef (and also NOT bcdef)
??def   matches def, cdef and bcdef but NOT abcdef

File System Commands Input and Output Routing

Placeholders used:

• COMMAND stands for the command line command string, which could be anything from the "DIR *.*" command to "ECHO Hi"
• SRC stands for Source
• DEST stands for Destination
  
• STDIN (0) stands for Standard Input or channel 0
• STDOUT (1)  stands for Standard Output or channel 1
• STDERR (2) stands for Errors or channel 2

STDIN and STDOUT can be various different things, such as:

• NUL = Nothing (usually used for suppressing any output to anywhere)
• CON = Console or Screen
• PRN = Printer (Only works, if a DOS printer is configured on LPT1)
• AUX = Auxiliary
• COM1 ... COM9 = Serial Ports 1 to 9
• LPT1 ... LPT9 = Parallel Ports 1 to 9
• FileName = a File Name, File Path/Location & File Name
• Command = another Command

The character “<” (less than) stands for Input, the character “>” (greater than) stands for Output (with overwrite) and 2x “>” (greater than)  = “>>” also stands for Output, but without overwrite (appending).

COMMAND>DEST redirects the Standard Output of COMMAND to DEST (Overwriting Previous Outputs)
COMMAND>>DEST redirects the Standard Output of COMMAND to DEST (Not Overwriting Previous Outputs)
COMMAND<SRC feeds COMMAND via the Standard Input from SRC

If you redirect the output to a File for example, you maybe noticed that sometimes not everything is being redirected into the file and some output still occurs on the screen. The reason for that is that you only redirect the STDOUT channel, but not the STDERR channel, which some commands and applications actually use for their error messages output. If you want to redirect the STDERR output to the same output channel specified as DEST use the following syntax:

COMMAND>DEST 2>&0

COMMAND output ">" redirect to DEST (STDOUT), "space", 2, which stands for STDERR and ">" redirect to “&0”, which stands for STDOUT

Variables in MS DOS Batch

The basics are clear to most folks, like setting a new environment variable with Set VARNAME = VALUE, returning the current value of VARNAME via SET VARNAME or using the VALUE of VARNAME via %VARNAME% in the batch script code. Also the use of the runtime generated variables %1 … %X for the command line parameters that were passed to the batch script are basic BATCH script knowledge.

As you know, parameter values that include the space (blank) character, must be enclosed in double-quotes (“) to avoid that the BATCH script will interpret the individual segments before and after a space as separate parameters. For example BATCH.BAT C:\Documents and Settings would result in three (3) parameters in the Batch Script, like:

%1 = C:\Documents
%2 = and
%3 = Settings

You have to enter BATCH.bat “C:\Documents and Settings” instead to have the path only appear as one (1) parameter in the batch, which is accessible via %1. The problem is that %1 will also include the double-quotes, which are not really part of the value that you want to pass. In many cases this does not matter or is even good, if you invoke file commands for example, where the parameter should again be enclosed in double-quotes to work properly.

To remove the enclosing (“) (double-quotes) from a parameter, add the character “~” (tilde) after the “%” (percent) sign. For example: %~1

This does not work for regular environment variables that were created via the “SET” command. The following would not work and error out, if you try to execute it:

SET VAR=”VALUE”
ECHO %~VALUE%

Other characters that require that a parameter value will be enclosed in double-quotes are: &()[]{}^=;!'+,`~

Did you know that %0 also exists? %0 stands for the  command  itself. In the examples above, %0 would return BATCH.bat. Note: Calling the script via the command BATCH without the extension “.bat” is also valid, if no other executable with the same base name exists in the same directory or path (e.g. a BATCH.exe or BATCH.com). If the batch file is executed this way, %0 also would not include the extension and simply return the value BATCH.

The SET Command

Typical use would be this: Set Counter = 0  where the value of “Counter” would then be accessible in the batch script (and beyond!) via %Counter%

Using the parameter /A indicates that the variable has a numeric value and should also be treated as such. This is important if you want to do arithmetic operations with a variable such as: set /A Counter+=1

This is especially powerful in combination with the “Delayed Environment Variables Expansion” feature enabled.
See more about this option further down below.

set /P VARNAME=Prompt_String

Displays “Prompt_Sting”, then waits for user input (in DOS, not a Windows Input Box) and stores the entered value in the variable VARNAME once the user finished his input by pressing the ENTER key.

Arithmetic Operators for the SET Command

()                  - grouping
! ~ -               - unary operators
* / %               - arithmetic operators
+ -                 - arithmetic operators
<< >>               - logical shift
&                   - bitwise and
^                   - bitwise exclusive or
|                   - bitwise or
= *= /= %= += -=    - assignment
  &= ^= |= <<= >>=
,                   - expression separator

String Substitutions

%PATH:str1=str2%  = Expands PATH and substitutes any occurrences of str1 with str2

String Offsets

%PATH:~10,5% = Expands Path and use 5 characters of the value from position 11 (offset) only

Left & Right String Functions

%PATH:~0,-2% = Expands PATH and uses all but the last 2 characters of the value

Other system environment variables

Other variables than %PATH%, which I used in the previous examples, which are good to know, but also require that EXTENSIONS were enabled via the SETLOCAL ENABLEEXTENSIONS command or via cmd.exe parameter.

• %CD% = Path to current directory
• %DATE% = Current Date
• %TIME% = Current Time
• %RANDOM% = Expands to a random number between 0 and 32767
• %ERRORLEVEL% = Returns the current ErrorLevel value
• %CMDEXTVERSION% = Version of the current command processor extensions
• %CMDCMDLINE% = original command line that invoked the command processor

Delayed Environment Variables Expansion

Always disabled by default, but may be enabled/disabled via the /V command line switch to CMD.EXE or via the command “setlocal enabledelayedexpansion” at run-time.  Disabled means that variables will be expanded at the time they are parsed, which is bad, if the value of the variable is supposed to change within the batch script itself. Here is a nice example script to illustrate the different behavior.

   1: set VAR=before

   2: if "%VAR%" == "before" (

   3:     set VAR=after

   4:     if "%VAR%" == "after" @echo If you see this, it worked

   5: )

If you run this, the script should return nothing. The second IF statement never can become true, even though you have the “SET” command right before it that should have changed the variable’s value to make the IF statement true. Why? Because %VAR% was already set to “before” and because of that expands to “before” throughout the entire script. Now add at the very top the line “setlocal enabledelayedexpansion” and run the script again. Now you should get the text “If you see this, it worked” back as a result.

Here is another example of the implication of the “setlocal enabledelayedexpansion”  statement.

   1: set LIST=

   2: for %%i in (*) do set LIST=!LIST! %%i

   3: echo %LIST%

This script will not return the list of all files in the current directory. It will only show one file name of the directory. It would work the same way as if you would have made the FOR command statement to:

for %%i in (*) do set LIST=%%i

Add “setlocal enabledelayedexpansion”  at the top of the script again and run it once more.
Another typical use where enabling delayed expansion is crucial, are Counters and other mathematical operations and manipulation of values during the execution of a BATCH script.

The SHIFT Command

The SHIFT Command is probably difficult to get its head around it. The applications are versatile. Shift basically shifts the sequence of parameters to the left. Parameter 2 becomes Parameter 1, Parameter 1 disappears and Parameter 2 would be empty, unless there is another Parameter 3. This is hard to grasp, so here is a short sample batch that illustrates this behavior.

   1: @echo off

   2: Call :testfn 1 2

   3: goto :EOF 

   4: :testfn

   5: echo 1 = %1, 2 = %2

   6: echo shift

   7: shift

   8: echo 1 = %1, 2 = %2

Results:

 1 = 1, 2 = 2
 shift
 1 = 2, 2 =

There are other variables like the ones starting with 2x “%” like %%a or variables that are enclosed in “!” instead of “%”, like !VARNAME! instead of %VARNAME%, but that would require me to make this post much longer than it already is. Check the help for the “FOR”, “SET” and “CMD” commands to learn more about them.

Additional Resources to BATCH Scripting

• Rob Van der Woude Batch Files Page by Rob Van der Woude
  
• Batch File Examples at DosTip.com
  
• Windows XP Command Line Syntax by Simon Sheppard at SS64.com
  
• DOS Batch Lessons at Allenware.com
  
• Batch Reference at Allenware.com
  
• MS DOS Fasthelp at MS Technet
  
• Collection of BATCH file tools at Bookcase.com

I hope that you find this post of mine useful. Use the comment section below for comments, feedback, suggestions and expressions of appreciation :). Thank you.

Cheers!

Carsten aka Roy/SAC

A Guide to Video & Audio for Microsoft Windows

What is this? Online video is booming and more dads become hobby “Spielbergs” every year. Dropping prices and increasing power and capability of video recording, playing and transmitting equipment is also doing his part to aid this trend.

Chances are good that it happened to you that you received a media file, audio or video from somewhere or someone that you wanted to play back on your computer, but couldn’t. Your friend opened it without problems, but you are getting a cryptic and technical error message instead, with no real solution offered to solve the problem.

If you started doing editing of videos and published your creations on CD, DVD, online or elsewhere, the problems encountered related to compatibility issues of your video and audio data that you are working on, probably increased exponentially compared to the time when you just consumed media.

That was at least what happened to me.

I found myself spending more and more time on solving those issues and less and less time on actually editing and publishing videos. I had to learn more about the subject than I ever wanted to, but I was kind of forced to deal with it anyway. I collected a lot of resources and experiences over the past months and even years and sat down one day to work on a consolidated article, post, guide or whatever you want to call it. Well, here is the result of this work that I actually enjoy: writing (and talking hehe).

I hope that this article and the provided information and resources will help you to solve some of the issues faster than I did and to get a general understanding of the subject in a much shorter period of time. I found many great resources, but none that is like mine. The resources that I found were often very detailed and very much technical, talking about my problems, but providing answers that I did not understand and asked me to do decisions for things where I did not understand the consequences of each choice that I could pick from. I pretty much had to rely on instinct, luck and trial and error, which is not a great way of fixing Windows issues efficiently and absolute.

I try to be more generic and only in a few cases specific, where I think I have to, but in the hope that I already provided enough general information that you got a pretty good idea of what the details mean once we get to them.

Feel free to use the comments to provide feedback, suggestions, praise or ask questions.

Thanks and Cheers!

Carsten aka Roy/SAC

The Containers - Media Files

A video container contains the video images, the audio and any additional features and content for the presentation of what normal people call “a video”. The additional stuff is often not noticed or believed by many, not part of the video files themselves.

Those things include “closed captions” or “sub titles”, “chapter indexes”, “Menu’s” like DVD menus and “Metadata” like “Genre” tags, “Description”, “Copyright”, “Publisher”, “Author”, “Ratings” etc.

The containers are represented as video files with specific file extensions that helps (more the) users and (less so) the computer to be able to identify the different containers, used for a particular “video”.

 

Container	Name	Parent	Company/Consortium
3GP/3GP2	3rd Generation Mobile	MP4	3GPP/Mpeg
AVI	Audio Video Interleave	AVI	Microsoft
ASF	Advanced Streaming/System Format		Microsoft
DIVX	DivX Media Format		DivX
DV	Digital Video		Sony, JVC, Panasonic
DVR-MS	Microsoft Digital Video Recording		Microsoft
EVO	Enhanced VOB	VOB	Mpeg
FLV	Flash Video		Apple
M2TS/MTS	MPEG-2 Transport Stream 192 B		Mpeg
MCF	BSD/GNU Container		BSD/GNU
MKV/MKA	Matroska Video/Audio		Public Domain
MP4	MPEG-4		Mpeg
MPG/MPEG	MPEG Program Stream		Mpeg
MOV/QT	Apple QuickTime Movie		Apple
NUT	NUT Project/GPL		GPL
OGM/OGG	Ogg Media File		Xiph.org
PS	MPEG-2 Program Stream	MP4	Mpeg
RAT	ratDVD (based on VOB)	VOB	GPL
PVR/DVR	MPEG-2 Transport Stream 188 B, DVR (Digital Video Recorder) PVR (Personal Video Recorder)		Mpeg
RIFF	Resource Interchange File Format (AVI)	AVI	Microsoft/IBM
RM/RM VB	Real Media - Variable Bitrate		Real Networks
TS/TP/TRP	MPEG-2 Transport Stream 188 B	MP4	Mpeg
VOB	Video Object		Mpeg
WMA	Windows Media Audio	ASF	Microsoft
WMV	Windows Media Video	ASF	Microsoft

Container	Video Codecs	Audio Codecs
3GP/3GP2	H.263, H.264, MPEG-4	AMR-NB/WB, (HE-)AAC
AVI	MPEG-4, MJPEG, DV, Cinepak, Indeo	MP3, MP2, AD(PCM), AC3
ASF	Windows Media Video, VC-1, MS MPEG4 v3	Windows Media Audio
DIVX	DivX	MP3, AC3, PCM
DV	DV	PCM
DVR-MS	MPEG-2	MP2, AC3
EVO	H.264, VC-1, MPEG-2	(E)AC3, DTS(HD), PCM
FLV	H.263, H.264, VP6	MP3, AAC, ADPCM
M2TS/MTS	H.264, VC-1, MPEG-2	(E)AC3, DTS(HD), PCM
MCF	various	various
MKV/MKA	H.264, MPEG-4	MP3, AC3
MP4	MPEG-4, H.264	AAC
MPG/MPEG	MPEG-1, MPEG-2	MP2
MOV/QT	H.264, MPEG-4, MPEG-1, MJPEG, Sorenson Video	MP3, AC3, PCM
NUT	Anything	Anything
PS	MPEG-2	MP2, AC3, DTS, PCM
OGM/OGG	Ogg Theora, Xvid	Ogg Vorbis, MP3, AC3
PVR/VDR	MPEG-2, H.264	MP2, AC3
RAT (ratDVD)	XEB	AC3, DTS, PCM, MP2
RIFF	ANI, MPEG-4, MJPEG, DV, Cinepak, Indeo	WAV
RM/RM VB	Real Video	Real Audio, AAC
TS/TP/TRP	MPEG-2, H.264	MP2, AC3
VOB	MPEG-2	AC3, DTS, PCM, MP2
WMA	-	Windows Media Audio
WMV	Windows Media Video, VC-1	Windows Media Audio

Pretty much all containers support directly or via support tools variable audio bit-rates (VBR), variable video frame-rates (VFR) and b-frames.

FLV, MPG/MPEG (and TS) do not support chapters, those containers also do not support subtitles (or only very poorly via workarounds)

Wikipedia – Comparison of Container Formats

The Components

File Sources

Now I mentioned already that a container holds all the pieces together that we call a “video”. Now we have to start taking them apart and enter the dark side of video format land (slowly).

A file source is a special software component that is able to open one or multiple of the mentioned video containers and get them ready to be processed. It can read the header information of the container and check if it makes sense and what criteria the next components would have to meet to be able to process this media file. You already learned that many containers can handle multiple video and also multiple audio encoding formats. 

Let me use a modified version of a classic saying: "Not all .AVI files are made equal". There is usually not just ONE component that can handle ALL of the video formats an AVI container supports. Also, the container contains VIDEO and AUDIO data, which are two entirely different things (technically) that will have to be processed separately. They must be separated, split from one another, which is also referred to as demuxing.

Muxer, Demuxer (Splitter)

Let’s start it easy and separate just the Audio, Video.

Several of the media containers also support other types of data that are related to the video and/or audio content. Also more than one instance of each media type is possible. A good example is the audio selection on a DVD. Most DVD movies offer a choice of spoken languages and sound quality, such as Stereo, Surround Sound and Dolby Digital. Also available are typically a set of different languages for the sub titles or closed captions for the hearing impaired folks out there.

Captions/Sub Titles, Meta Data and Menu’s are still not unimportant, but less of a problem in my opinion. So I decided to move them to the end. Hey, that does not make me intolerant towards people who are impaired in their hearing. I am a collector junkie myself and love Meta Data.

The file source component that I mentioned before is often part of a splitter/demuxer, which does make sense. A file source already has to know the format of the container. It's from there only one more logical step to take the parts of the media content in the container and pass them over to the right next component to process it. 

The reversed process of the splitting of the content in a media container (demuxing) is the combining of the various parts that need to go into a container and wrap them together to generate a single file that has all the video and audio data etc. in it. This process is being called "muxing" and the component that does this is referred to as a Muxer. The muxer receives the data from the counter part where the demuxer passes the different data streams to.One gets the audio to deal with, the other one the video and maybe another one to take care of the closed captions. The Demuxer passes the data to a "Decoder", while the "Muxer" would receive its data from an "Encoder". An Encoder/Decoder or both is referred to as a Codec.

Codec

The word codec is short for compressor/decompressor but also referred to as encoder or decoder.

Codecs are in many cases not bound to a specific container, which does not make things easier and only more confusing, especially, if you consider the amount of Codecs and their variations that exist. A codec can be a hardware or software component, but in our case are only the software Codecs of importance.

That component can compress and/or decompress video using a particular compression algorithm.

Since all those Codecs and containers must be defined somewhere that the computer system and software can find out what it needs in order to provide/play back all the elements that make up your “video”. You probably noticed all the “aka” and “/” usages for the same or very similar thing. It’s a mess!

The codec tweaking tool GSpot Version 2.7 knows about 749 different FourCC video Codecs and 247 different audio compression Codecs. That gives you an idea about the scope of the problem. This does not even include any special filters, splitters, file sources; renderers and all that other stuff that is part of the system, playing a role in the process of bringing up the video from a file on your screen and the audio out of your speakers.

FourCC

To help the computer a little bit with making sense out of things, a system was devised that is called FourCC or 4CC, which means literally “four character codes”. The folks from the video game development company Electronic Arts introduced what is considered the father of the FourCC concept, called IFF (Standard for Interchange Format Files) in 1985 for the Commodore Amiga line of computers (“EA IFF 85”).

A simplified definition for 4cc could be: Every media component format will be assigned a unique 4 character code, which will be included in the media data itself, that apps and systems can look at a specific place in the media data to find out what format the data is in. It can then simply check, if it has the information for the format that is labeled with that 4 character code or not. If it does not, it will return an error and tell the system or user that it requires the specifications for that format to understand the structure and layout of the data in order to be able to process them properly (e.g. show images from your last vacation instead of a bunch of colored dots that look more like works by Escher).

A good list with Codecs, their FourCC designation, company name and website URL is available at FOURCC.ORG. 

The words “Filter” and “Codec” are unfortunately often used beyond their actual meaning, which only adds to the whole confusion even more. A “DirectShow Filter” might be a Codec, a file source, a splitter, a muxer, a renderer or maybe actually just a filter. The term Codec might be used in the same fashion, but usually implies that the actual encoding and/or decoding piece, is at least part of the software package. For example, if you download the DivX Codec from the DivX website, you get more than just the encoder or decoder, even more than the stuff that is needed by Windows to encode and create or decode and render videos using the DivX video compression algorithm. It comes with players, a converter, statistics and configuration tools along with it.

Why Do We Need All This in the First Place?

Video data are nice, but also have a flaw: they are huge and eat up a lot of space, so much space that it is even with today’s low hard disk cost and higher available bandwidth not feasible to work with raw video data all the time. Codecs reduce the size of video and audio in a flick significantly. There are a number of different algorithms out there and used for the compression, each with their own pros and cons. Some compress higher with less loss of quality, but suck up a lot more resources and demand much more powerful hardware to decode the flicks again for presentation that not all hard and software is able to support them yet, compared to other Codecs, that might not be as good with compression and loss of quality, but are faster and require much less computing power to do something with the crunched data files.

But it is also true that competition, patents and commercial interests play a significant part in this game as well. It does not have to be that messy as it is today.

Most popular Codecs on the Internet

(based on Codecs Database)

I will not mention every of the over 1000 or so video and audio codecs that exist. I will reduce it to one a few that are popular online and offline and cover well over 90% of the entire spectrum. 

Some Video Codecs

MPEG-1/MPEG-2

The video codecs used for standard DVD Video and VCD/SVCD. Typical file formats have the extensions: .VOB

or .MPG/.MPEG.

MPEG-4 Part 2

Most .AVI  files on the internet are using one of the following two video codecs, the commercial DivX video codec (4CC codes: DIVX, DX50, DIV3, DIV4 and DIV5)  and the open source Xvid codec (4CC: XVID, sometimes also xvid or Xvid, which should not but unfortunately does matter some times, if a software component is case-sensitive. I bitched about that elsewhere already).

Xvid is able to process many of the different versions of DivX, except for the really early ones

MPEG-4 AVC (Advanced Video Coding)

.MP4 files and newer Apple QuickTime .MOV files (4CC: AVC1). You can rename a MOV file to MP4 for example, if a program causes trouble when you want to load a MOV file with it and it might does the trick for you.  Typical are H.264 format based codecs, like the open source X.264 codec  (4CC: X264) or the commercial Dicas H.264 codec (4CC: DAVC)

QuickTime Movies

.MOV (also .QT) files from Apple. The latest versions use the H.264 format, but Apple used in the past a propriatory format (4CC: SV10) and then versions of the Sorenson Video Quantizer  video codec (4CC: SVQ1 or SVQ3), with it's version 3 being already a variation of the H.264 format to make it the next logical step  for Apple and it's QuickTime product to use a standard video format and not continue with its own.

Flash Video

Probably mostly because of the popularity of YouTube who uses Flash Video for showing movies on its social video sharing platform and the general popularity of using Flash for video on the Web, .FLV transformed from exotic video format to a format most internet users are familiar with.

Flash 7 uses the Sorenson Spark video codec (4CC: SPRK) and Flash 8 the On2 VP6 codec (4CC: VP60)

Windows Media

Microsoft developed their own video and audio codecs, which are known as Windows Media Video and Windows Media Audio. They are used in the media containers WMV and WMA and ASF for streaming. 

Real Media

Like Microsoft, Real Network, the giant content portal on the Internet, developed their own video and audio codecs (4CCs: RV20, RV30, RV40). The .RM (real video) and .RA (real audio) file format is still the primary format for Real Networks Real Player. Real Video became popular because it was one of the first video formats that could be streamed over the Internet.

Some Audio Codecs

MPEG-1 Layer 3

Simply known as MP3 is the most known and used audio format today. Part of the credits for making it as popular as it is today would probably have to go to the former file sharing peer-to-peer network Napster and of course to Apple iTunes and their iPod portable MP3 audio player. It was developed by the Frauenhofer Institute in Germany and also made head-line new with it's patent disputes and lawsuits in the early 1990s.

(AD)PCM

Raw WAV files are usually encoded in PCM format. 

AC3

Better known as Dolby Digital (5.1). The popular sound format used on many DVD movies for high quality surround sound at home and at movie theaters. 

MPEG Audio (AAC)

AAC stands for Advanced Audio Coding and was meant to be the successor of MP3. It is today in wide spread use, for example in iPhones, iPods (and iTunes now) or the Sony Playstation 3 game console.

DTS (AC)

You probably know DTS from the sound options of many DVDs. DTS is a common alternative audio format, competing with Dolby Digital. DTS stands for Digital Theatre System -  Coherent Acoustics if spelled out. Similar to Dolby Digital 5.1, but some say that it is slightly better that 5.1. Mh. :) 

I mentioned Windows Media Audio already. 

Filters

There are multiple meaning of the word "filter" in video land. In DirectShow (see further down below), every component is called a "Filter", independently from it's function and purpose. So a DirectShow Filter could be a File Source, a Splitter, Encoder/Decoder (Codec), Renderer or actually a Filter in the true sense of the word.

A Filter in traditional terms is a component that alters the video or audio stream. Classic filters that you probably used yourself every time when you edited a video, sound or picture, without knowing that you used one. PhotoShop users will smile now, because they should have a pretty good idea of what is coming now.

If you resize a video for example, or change it's contrast, brightness, saturation, hue, color encoding, number of colors, or turn color video images to gray-scaled ones, deformations, slicers, blenders, mixers, volume adjustments, normalizers etc. etc. All those functions are performed by Filters. Something goes in, gets manipulated and then comes out and hopefully enhanced on the original output as we wanted it to do.

Renderers

Renders are responsible for the output of the video and/or audio stream data to a device that is capable of displaying or playing back the content, such as the video card that then renders the images on to your computer screen or display or to the sound card, which feeds your side speakers or your receiver and then out of your high-end speaker system that you can listen to the sounds and voices of the audio or video file.

Streaming

Streaming is only a special form of rendering that requires that the video data are rendered fast enough to allow the play back of the audio or video in real time, without the need to download the whole video or audio before I play it back. You might think that this is the normal way of doing it, but if you ever converted a video to a DVD or a DVD movie (e.g. MPEG-2/PCM) to a MP4 video file (e.g. H.264/AC3) for example then you probably know that this process takes still longer than the video material itself is long. A 90 minutes movie takes usually more than 90 minutes to re-code/convert from one media type/format to another. The conversion of a video file to HD DVD (such as Blu Ray) can take many hours or days, depending on the hardware equipment used for the conversion.

That is not an option for streaming, especially if the video is streamed while the content is being generated/recorded, such as a Live Event Broad Cast. You don't want to wait to see today’s ball game tomorrow, because it takes a day to recode the video images that you can watch it on your computer or TV, right?

Not all video formats can be streamed and the ones that can, need to be transmitted to the receiver in an optimized way to ensure that the video stream is constant to guarantee fluid video images and flawless sound. For the transmission of the video and audio data are different streaming protocols used, like RTSP, PNM, MMS  or HTTP.

Components Wrap Up

Now you have all those components and elements in the process and all these different methods and formats where non of them just by itself is anything but easy to do. It is already hard enough to get all those things work together in unison and as a whole that the least thing you would want to do is adding another layer of complexity to it and makes things even more complex than they already are, right? Yeah, sure, common sense tells you that this is probably a good idea, but since when was common sense applied to the real world? Think about it!

DirectShow (DS) - Filters, Video for Windows (VfW)

Now Microsoft thought that it could make things even messier… Aehm, I mean improve on them. But I will repeat what they say themselves in the introduction to their “Microsoft DirectShow 9.0 System Overview” at MSDN.Microsoft.com.

The Challenge of Multimedia

Working with multimedia presents several major challenges:

Multimedia streams contain large amounts of data, which must be processed very quickly.

Audio and video must be synchronized so that it starts and stops at the same time, and plays at the same rate.

Data can come from many sources, including local files, computer networks, television broadcasts, and video cameras.

Data comes in a variety of formats, such as Audio-Video Interleaved (AVI), Advanced Streaming Format (ASF), Motion Picture Experts Group (MPEG), and Digital Video (DV).

The programmer does not know in advance what hardware devices will be present on the end-user's system.

The DirectShow Solution

DirectShow is designed to address each of these challenges. Its main design goal is to simplify the task of creating digital media applications on the Windows® platform, by isolating applications from the complexities of data transports, hardware differences, and synchronization.

To achieve the throughput needed to stream video and audio, DirectShow uses DirectDraw® and DirectSound® whenever possible. These technologies render data efficiently to the user's sound and graphics cards. DirectShow synchronizes playback by encapsulating media data in time-stamped samples. To handle the variety of sources, formats, and hardware devices that are possible, DirectShow uses a modular architecture, in which the application mixes and matches different software components called filters.

DirectShow provides filters that support capture and tuning devices based on the Windows Driver Model (WDM), as well as filters that support legacy Video for Windows (VfW) capture cards, and codecs written for the Audio Compression Manager (ACM) and Video Compression Manager (VCM) interfaces.

The following diagram shows the relationship between an application, the DirectShow components, and some of the hardware and software components that DirectShow supports.

The “simple” concept of DirectShow by Microsoft illustrated

The DirectShow API media-streaming architecture was introduced with the DirectX SDK for 32Bit Windows operating systems. If you remember Windows 95 and 98 and played some games on it, then you will remember that DirectX was always required and the minimum version needed to run the game was even shipped with the setup CD-Rom’s for the games and part of the setup process.

The DirectX SDK kind of died as a package with versions etc. The last DirectX version released was a DirectX 9.0 update from February 2005. DirectShow was moved to the Windows SDK. Windows 2003 Server and Vista are the first systems where only the Windows SDK is used and no DirectX anymore.

DirectX was the successor of Video for Windows (VFW), which was introduced in 16-bit Windows (Windows 3.0, 3.1 and Windows for Workgroups). For backwards compatibility reasons to be able to run 16-bit apps under 32-bit Windows, do the 32-bit OS versions of Microsoft support VFV.

But, they did not leave VFV completely alone. Starting with DirectX, many of the VFV features were suppressed, causing incompatibility issues for some 16-bit video applications, if used in Windows 32-bit. They launch and kind of work, but at the same time not really, at least not as expected and most of the time wanted.

Getting your computer to process any type of video with the application that uses DirectShow or VFV does not make things work for applications, which uses the other model. This can get funny and made me scratch my head more than once. Phenomena like player X can play back a video in format Y after you installed an add-on (probably a Codec), but player Z remains unable to do anything with that video.

One uses VFW and the other DS. Both are separate frameworks and each must be supported specifically by any component that is part of your video experience. Some applications support both, but most use either one or the other only.

Because of this mess, do some applications do not use either of those systems and use their own system to process video data. Those applications are limited to what they come with and support themselves. Some allow third parties to create plug-ins to support additional Codecs and file formats. Nullsoft’s Winamp Player would be a popular example for this approach.

DirectShow is like Lego with all the same great attributes and possibilities and virtually the same flaws as well, because of these great possibilities. If you ever owned Lego and played with it, chances are that you owned more than one set. You probably also mixed the parts of different sets and did not keep them separate. With the time became it harder to get original sets together again, because the manual got lost and you had to start guessing which part goes were and was part of a specific set or not. Also parts got lost some times. You had to improvise, and substitute things, which was only possible to a limited amount, because you had to make sure that the parts fit.

The process is virtually the same with DirectShow. A set would be a video file for example that you want to play back with a DirectShow enabled video player, such as Windows Media Player. The components of the set are actually mini-sub-sets itself, like a set in Lego that was part of a theme, e.g. modern city or the knights theme. Imagine having other players coming and using your sets and/or bring some new sets or even modify some of the sets that you already built, because they did not like the way you did it.

Those other players can be any video related software (big app to small tool) that is using DirectShow. Even a small and not very important tool that you want to use for splitting a video into multiple pieces for example, might messes things up or cause the system do things different that it used to before. Stuff is most of the time changed without warning.

I do not have an answer to the problem, that you cannot sit down, make everything work the way you like and then keep it forever. I can only say that if you got everything to work how you need it, be careful what software you install in the future and leave settings related to video and audio alone.

And to make thinks even more interesting for the future, Microsoft decided to introduce yet another system to the mix with their upcoming operation system version dubbed Windows 7 with the bright and shiny name:

Windows Media Foundation. "It will make everything better, promise". It sure will. The only question remains is for whom it does. Mhh?!

Video Tools

Splitter/Joiner

There are also video and audio file splitter tools that do what their name implies. They let you slice up a video or audio file into multiple pieces (splitter) or combine multiple video or audio files into a single file (joiner). You cannot simply cut a video or audio file into pieces or copy different media files together (like with the binary copy command in DOS).

Media files have a header, data at the beginning of the file, which are crucial that media players are able to play them. A missing or corrupt header can make the entire media file unusable und unplayable. If you simply cut a file in half, then you would end up with one file that has now incorrect information about the media file and a second one with no information at all. Individual video frames also have multiple components and a header. If it is a compressed video file (what typically all video files that are used by regular users are), then it is even more complicated, because some frames are needed to render a bunch of other frames, so called key frames.

Cutting a file into pieces without considering all this factors would be like going to a butcher and have him cut the next best piece of a cow when you order X pounds of steak. No pretty picture, but I hope that you get the idea.

Cutting a media file into pieces is generally easier than joining them together (as with almost everything else in life, taking things apart is easier than putting them back together). As long as the Cutter tool knows the file format structure and knows where to cut and to rebuild the file header, things will be fine. If they are not recoding the slices in the process, cutting is also done pretty fast.

Joining files is tougher, unless you want to re-join the pieces that you just created with your cutter. This is rarely the case though. If you only want to slice a media file up for storage and transportation, because the media file is too large, then you should not use a media cutting tool in the first place. Breaking it up with your backup software or with tools like ZIP, RAR, ACE, HJSplit or MasterSplitter will do just fine.

Simpler joiner tools are only able to combine media files that have exactly the same format. When I say exactly, then I mean more than just having the same file extension. The to-be-joined files must be compressed with the same video and audio Codecs and also settings that determine the video and audio quality, such as Bit Rate, Frame Rate, Sampling Rate and Resolution have to be identical. If this is the case, the joiner only has to re-create the information and not to re-encode any audio or video data.

More sophisticated joiners are a bit more flexible and will make adjustments to the source media in order to be able to join them. Some even re-encode the whole video and audio and work like converter, compositing and rendering tool all-in-one. If the joiner has to re-encode anything, then the process can take considerably more time, but it provides the most flexibility regarding the video and audio sources that can be combined together, even allowing the joining of media in entirely different formats.

To join .AVI files that are in the same format, you can use the free tool VirtualDub. You can also use free compositing tools like Windows Moviemaker (also free), or professional software, like TechSmith Camtasia Studio, Adobe Premiere, Pinnacle Avid, Ulead Video Studio (now Corel) etc., but there you have to re-encode the results every time, regardless if it would be necessary or not.

There are many commercial tools available. I can recommend the Splitter and Joiner tools from Boilsoft, because they are very universal and support various video file formats, including .AVI, .WMV, .ASF, .RM and .MPG. They cost about $50 together and you can test them free for a few days before you have to decide whether you want to buy them or not.

The Video Joiner by ImToo is more flexible and re-codes the source video files to the format that you select and only cost $19, with a free trial period to test it yourself.

Resources

The drops of water on the hot stones in the video & audio encoding and decoding desert. Some drops might vaporize away without quenching your thirst, while others might provide just the little bit of water that you need to survive another day in this desert. :)

Codec Packs

It became clear to geeks and people with a great deal of knowledge and understanding of the subject matter that this is not something that you can expect from a normal user to deal with. A large number of people would probably not understand it and an even larger number of people do not even want to understand it to begin with. I belong to the second group that is not a part of the intersecting set with the first group. I just want to watch and edit a movie, that’s it.

The market failed, the competition lead to incompatibility issues and conflicts and the need to spend the equivalent of a college education in time and resources in order to be able to talk the talk and blame it on the other party.

Ordinary people stepped up and developed tools and build packages, most of the time absolutely free, that other people can download and install, to make many of the things work that usually do not, or at least make the problems smaller and easier to deal with.

They are called Codec Packs and are a collection of containers, Codecs, filters and tools that are pre-configured to work together and as many as possible components compatible to each other.

The problem is that Codec Packs, if not done right, can help making things worse. Some packs also think that doing their own thing, ignoring any other standards and best practices, creating their own little island is the solution to the problem. This lead to hate and love that you will get exposed to, if you are doing some of your own research on this subject. Some despise them, some hate them, others love them and calling them saviors. 

To sum it all up, Codec Packs might not a quick and general applicable solution to a wide spread problem as they tried to went out to become. From what I heard from "experts", other geeks and based on some experiences that I made myself with those packs, there are only a few that I would recommend. 

Some of the biggest and popular Codec Packs include:

• K-Lite Codec Pack (KLCP) – provides different packs with different configurations for the different types of users out there. They also have pages with name, brief description and download link for circa 300 video and audio Codecs, filters, splitters, Muxers and De-Muxers etc.
• Combined Community Codec Pack (CCCP) is the pack that is praised the most (or hated the least) by people who call themselves expert in this subject. I was okay with it, but personally preferred K-Lite.

Other Codec Packs
Please use with Caution and check first, if they offer everything that you need, before you install them.

• Windows Essential Codec Pack (W.E.C.P.) 
• Cole2k Media - Codec Pack
• Gordian Knot Codec Pack
• MediaCoder Pack
• X-Codec Pack

Special Players and Codecs

• QuickTime Alternative
• RealPlayer Alternative
• nLite Windows Installation Customizer
• WMLite, TTPack and QTLite for light-weight versions of Windows Media Player, Apple QuickTime, Real Networks Real Player, FFdshow, Ogg and AC3
• Media Player Classic (MPC) and VobSub
• DirectShow FilterPack (DSFP)

I don't use codec packs today anymore. I install and deal with codecs one-by-one. This is probably the best way to go in the long ground, once you are familiar with the various available options out there and know what you need for your own purposes. The needs of video creators and consumers are not alway the same. I am both and experienced this first hand myself.

Info and Help Resources to Audio & Video

• http://www.doom9.org/
• http://www.videohelp.com/
• http://www.free-codecs.com/
• http://www.codecguide.com/

Advanced: Tools you got to have

Only for the geeks out there, tools to tweak and troubleshoot codec and filter related issues and tools to fix them as well.

• Media Info Lite
• Graph Edit
• GraphEditPlus  (commercial software, $60)
• GSpot Codec Appliance
• AVICodec 
• RadLight Filter Manager
• DirectShow Filter Manager
• Codec Tweak Tool (part of the K-Lite Codec Pack) 
• Sherlock - The Codec Detective
• WinXP Decoder Check  (alternative download)
• VirtualDubMod
  - DirectShow Input Driver for VirtualDub
  - VirtualDub MPEG2 
• MP3Tag
• abcAVI
• aviUtl

Codec Lists and Databases

• Codec Library
• Codecs Database
• Codecs.com
• abcAVI Codecs List
• Codecs Wiki at Multimedia.cx
• FourCC.org

Acronyms and Definitions

Source: http://www.fourcc.org/

Acronym	Definition
ACM	"ACM" files, also known as "Audio Converter Modules", are audio codec drivers installed on your system which export methods that can be performed on a particular audio stream. This includes converting audio from one type of stream to another. An example of this is perhaps an MP3 ACM which could convert MP3 audio to PCM audio.
ATSC	Advanced Television Standards Committee. The technical group that defined the high definition TV standard for US terrestrial transmission.
AVC	Advanced Video Codec. Otherwise known as MPEG4, part 10 this is the codec that most of the worlds broadcasters are moving to for HD transmissions.
AVI	Audio Video Interleave - a Windows file format used to store movies.
Codec	CODEC stands for COmpressor/DECompressor - a software or hardware component that can compress and/or decompress video using a particular compression algorithm. Though CODECS are generally thought of in the context of video and audio, they are not limited to this scope.
FOURCC	Four Character Code - an 8 digit (or 4 ASCII character) value used to identify the pixel format or compression standard (codec) used to store images or video files. FOURCCs are also used for a similar purpose in some audio applications.
HD	High Definition. This refers to a video picture size higher than SD (see below) and typically one of the set of resolutions defined by ATSC such as 1080i (1920x1080), 720p (1280x720) or 480p (720x480).
JPEG	Joint Photographic Expert Group. The standards group that defined the hugely successful JPEG still image codec.
MPEG	Motion Pictures Expert Group. The clever people who brought you the entire ISO standard Codecs that are used in TV broadcast today.
NTSC	National Television Standards Committee. The group which defined the 480 (visible) line video standard used for analog and pre-HD digital TV transmissions in the USA.
PCM	PCM is Pulse Code Modulation. This is basically an audio format that is pretty much "raw" audio. This is generally the format that audio hardware directly interacts with. Though some hardware can directly play other formats, generally the software must convert any audio stream to PCM and then attempt to play it. An analog audio stream generally looks like a wave, with peaks and valleys that are called its amplitude. These are generally sent to the audio hardware and are then digitized. The conversion basically samples this data stream at a given frequency such as 8000Hz, etc. This sampling will generally measure the voltage passing through so many times a second and generate a value based upon this. Remember, Im not an audio engineer so this is just a simplistic way of thinking about this. The PCM format, for example, I believe the base line is either 127 or 128 (in 8 bits per sample). This is the middle, silence. Anything going below is a low tone and above is a higher tone. If you take a bunch of these values and play them at a certain speed, they will make a sound.
RGB	Red Green Blue - a method of describing colors commonly used in PC graphics.
SD	Standard Definition. The size of a typical video image for a "legacy" TV system. In the US, this will typically describe an image with 480 lines (and one of a number of widths, 720 being the maximum, 480, 640 and 704 being other common choices). In Europe (and other areas which use PAL TV standards), SD refers to an image with 576 lines of resolution.
YCrCb	Color format typically used in video processing. A color is defined in terms of a luminance (Y, brightness) value and two "color differences" or chrominance values (Cr and Cb). One correspondent indicates that Y, R and B likely refer to primary colors yellow, red and blue and "C" indicates a color difference.
YIQ	YIQ is another color format and another way of defining a color and is commonly used in NTSC video systems as far as I can remember.
YUV	YUV also describes a luminance/chrominance color model. This is frequently used interchangeably with YCrCb though, technically, they are different.

Some Handy BATCH and VB Scripts for Organizing Files

Here are four scripts and batch files that come in handy sometimes when you organize or clean-up files and folders on your hard disk. You can use the provided scripts as you see fit. They are provided “as is” and are not supported. Also be aware that I will not take any responsibility for any damage or harm that the scripts might cause. You use them at your own risk. You are free to use, copy, modify and delete them (Freeware).

Move Files to Sub Folders Alphabetically

The simple batch script creates the sub folders A, B, C, D … Z and 0-9 in the current directory where the batch is executed and moves all files in that same directory to the corresponding sub folder.

For example, all files starting with “a” or “A”, like “Anna.zip” or “achim.gif”, will be moved to the sub folder “A”. All files that do not start with a letter from the English alphabet, such as numbers (“0", “1” … “9”), special characters like “!”, “_”, “$” etc. and foreign characters like “ä” or “é” are moved to the sub folder “0-9”

I wrote in march a post about organizing file collections on your computer and provided also some scripts to help you sort and catalog files. I found a much simpler way of doing the same stuff as the batch script “!organize.bat” and the VBScript script “!folders.vbs”. It does not replace the script “!dirlist.vbs”, which creates the file index. If you want to use “!dirlist.vbs” with this new and shorter script to move files into sub folders by alphabet, simply add the following line of code after line 7 (after “Move * 0-9” and before “goto :EOF”):

wscript !dirlist.vbs

This batch also does one thing slightly different. It moves to the sub folder “0-9” not just files that start with a number, but also all other files that do not start with a letter from the English alphabet.

MoveToAlphabetSubFolders.bat

   1: @echo off

   2:  

   3: for %%f in (a b c d e f g h i j k l m n o p q r s t u v w x y z 0-9) do Call :MoveFiles "%%f"

   4: goto :EOF

   5:  

   6: :MoveRest

   7: Move * 0-9

   8: goto :EOF

   9:  

  10: :MoveFiles

  11: if NOT EXIST %1 MD %1

  12: if %1==0-9 Goto :MoveRest

  13: for %%f in (%1*) do move "%%f" %1

Remove Empty Folders and Sub Folders

This little batch file scans the directory that was provided as a parameter and all its sub-directories for empty folders and deletes them, including the provided folder itself, if it is empty as well (or became empty after all empty sub-folders were removed).

RemoveEmptySubFolders.bat

   1: @echo off

   2: set Folder="%~1"

   3: if %Folder%=="" @echo Syntax RemoveEmptySubFolders Folder&goto :EOF

   4: if not exist %Folder% @echo Syntax RemoveEmptySubFolders Folder - %Folder% not found.&goto :EOF

   5: setlocal

   6: :: REMOVE EMPTY SUBFOLDERS

   7: for /f "tokens=*" %%A in ('dir /ad /s /b %Folder% ^|Sort /Reverse') do (

   8:  RmDir "%%A" 2>NUL

   9: )

  10: :: REMOVE FOLDER, IF EMPTY

  11: RmDir %Folder% 2>NUL

  12: endlocal

Move Files to Folder and Auto Rename Duplicates

This tool requires Windows Script.

This little VBScript moves all files from the source folder, including all files from it’s sub folders to the specified target folder and renames files, if a file with the same name already exists at the destination directory. The rename also works, if you have several files with the same name (such as a file with the same name in several sub directories of the source folder). The renamed file is named FILENAME[X}.EXT, where X starts with 1 and increments by 1 if already another file with the same name exists. For example

1. Source Dir\file.zip
2. Source Dir\Sub Folder1\file.zip
3. Source Dir\Sub Folder2\file.zip

would be moved and renamed to “Target Dir” like this

1. Target Dir\file.zip
2. Target Dir\file[1].zip
3. Target Dir\file[2]/zip

If you specify the target directory to be the same as the source directory, the script will automatically ignore the files in the source directory root and only process the files in the source directories sub folders. This feature is quite nice, if you want to consolidate all files in a directory tree in the root directory of the tree itself.

If you run the script without any parameters, you will be asked to first specify the location of the source directory (it will show as default value the path to the current directory where the script was executed) and then the target directory (it will show as default value to parent directory of the entered source directory).

Execution of the script without any parameters:

Wscript.exe MoveFilesAutoRenDup.vbs

You can also provide the source and also the destination directories as parameters to the script and you won’t be prompted to enter one anymore.

Execution of the Script with source and destination directory provided as command line parameters:

Wscript.exe MoveFilesAutoRenDup.vbs X:\SOURCEFOLDER X:\DESTINATIONFOLDER

MoveFilesAutoRenDup.vbs

   1: Dim FSO: Set FSO = CreateObject("Scripting.FileSystemObject")

   2: Dim arguments: Set arguments = Wscript.arguments

   3: Const ForAppending = 8, ForReading = 1, ForWriting = 2

   4: Dim iFilesCount: iFilesCount  = 0

   5: Dim  sSrc, sDest, DC, sExt, sBase, oFOld

   6: Dim sTo, sFrom

   7:  

   8: if arguments.unnamed.count = 0 then

   9:   sFrom = "N*A"

  10:   Do While sFrom <> "" and FSO.FolderExists(sFrom) = false

  11:   sFrom = InputBox("Move All Files and Auto Rename " & _ 

  12:           Dupes","Enter Source Directory" & _

  13:           ": ", fso.GetAbsolutePathName("."))

  14:   Loop

  15:   if sFrom = "" then

  16:      WScript.Quit(1)

  17:   End if

  18: else

  19:   sFrom = arguments.unnamed.Item(0)

  20: end if

  21: if not FSO.FolderExists(sFROM) then

  22:    WScript.Quit(2)

  23: end if

  24:  

  25: if arguments.unnamed.count = 1 then

  26:   sTo="N*A" 

  27:   Do While sTo <> "" and FSO.FolderExists(sTo) = false

  28:   sTo = InputBox("Move All Files and Auto Rename Dupes","Source Directory: " & _ 

  29:         vbcrlf & sFrom & vbcrlf & "Enter Destination Directory" & _

  30:         ": ", FSO.GetParentFolderName(sFrom))

  31:   Loop

  32:   if sTo = "" then

  33:      WScript.Quit(1)

  34:   end if

  35: else

  36:   sTo = arguments.unnamed.Item(1)

  37: end if

  38: if not FSO.FolderExists(sTo) then

  39:    WScript.Quit(2)

  40: end if

  41:  

  42: Set oFold = FSO.GetFolder(sTo)

  43: if lcase(sTo) <> lcase(sFrom) then

  44:   ShowFiles oFold.Path

  45: end if

  46: ShowSubFolders oFold

  47:  

  48: Wscript.echo iFilesCount  & " files moved"

  49: '========================================

  50: Sub ShowSubFolders(Folder)

  51:   Dim SubFolder

  52:   For Each Subfolder in Folder.SubFolders

  53:     ShowFiles Subfolder.Path

  54:     ShowSubFolders Subfolder

  55:   Next

  56: End Sub

  57: '----------------------------------------

  58: Sub ShowFiles(sPath)

  59:   Dim oFile, oFolder

  60:   iFolderCount = iFolderCount + 1

  61:   Set oFolder = FSO.GetFolder(sPath)

  62:   iFilesCount = iFilesCount + oFolder.Files.count

  63:   For each oFile in oFolder.Files

  64:     sDest = FSO.BuildPath(sTo, oFile.Name)

  65:     sExt = FSO.GetExtensionName(sDest)

  66:     sBase = left(oFile.Name,len(oFile.name)-len(sExt)-1)

  67:     DC = 1

  68:     Do While FSO.FileExists(sDest) = true

  69:       DC = DC + 1

  70:       sDest = FSO.BuildPath(sTo, sBase & "[" & DC & "]." &  sExt)

  71:     Loop

  72:     sSrc = FSO.GetAbsolutePathName(FSO.BuildPath(sPath,oFile.Name))

  73:     FSO.MoveFile sSrc, sDest

  74:   Next

  75: End Sub

  76:  

  77:  

Determine the Short Path (8.3 DOS Name) of the Current Directory

This tool requires Windows Script.

If you still use some old 16bit or DOS applications that do not support the long file names of FAT32 and NTFS, then you know that in order for those tools to work, the short 8.3 DOS Name of directories and files must be used with the application. The dos command “DIR /X” provides that information, but only for the current directory. If you need to know the short name of the entire path to a sub directory a couple levels down from the drive root, this approach becomes pretty cumbersome.

Here is a little script that helps with that. You can execute it from the command prompt at the directory where you want to know the short name by typing:

wscript.exe c:\tools\shortpath.vbs

I assume that the script was saved in the directory c:\tools, if you saved it somewhere else, change the command accordingly. 

You can also add it to Windows Explorer as a shell extension like in this example below. How to install the script as a shell extension in Windows XP is explained in the last paragraph of this post.

Example for the Long Path in Windows 32 bit:

C:\Documents and Settings\Administrator\Local Settings\Application Data\Microsoft

Short 8.3 DOS Path Equivalent (on my computer, might differs from yours)

C:\DOCUME~1\ADMINI~1\LOCALS~1\APPLIC~1\MICROS~1

The script returns the short path in an input box that you can easily copy and paste it from there into your application or document.

Here is the source code of the script.

shortpath.vbs

   1: Dim oFso: Set ofso = CreateObject("Scripting.FileSystemObject")

   2: Dim arguments: Set arguments = Wscript.arguments

   3:  

   4: if arguments.unnamed.count = 0 then

   5:   Path = ofso.GetAbsolutePathName(".")

   6: else

   7:   Path = arguments.unnamed.Item(0)

   8: end if

   9:  

  10: Set tmpf = ofso.GetFolder(Path)

  11: PathShort = tmpf.ShortPath

  12: Set tmpf = Nothing

  13:  

  14: a = InputBox("8 Character DOS Short Path","Path:", PathShort)

  15: Set oFso = Nothing

Using the Tools from the Command Prompt

You can run all of the provided scripts and batch files from the DOS command prompt. To open a command prompt on your computer, click on “Start” and then on “Run”. Type “cmd” and then press “Enter”. If you copied the batch files to a location that is included in the Windows %Path% environment variable, such as the Windows system directory (usually c:\Windows) or the System32 directory (usually c:\Windows\System32) then you do not have to provide the path to the BATCH files in order to be able to execute them. You can simply type the name of batch file and Windows will find it automatically and execute it.

Make sure that you are in the right directory when you call the batch script. If you open a command prompt the way I described it above, you are usually located at your users home directory (usually c:\documents and settings\USERNAME, where USERNAME is the name of the current Windows user). You would have to navigate to the right directory using the “CD” command (type “help cd” and then press “enter” to open the help documentation for the “CD” command)

To execute a VBScript script from the command prompt, you have enter first “wscript.exe”, followed by the path and the file name of the script, even if the .VBS file is located in a directory that is included in the %Path% environment on your computer. “wscript.exe” is a tool, which is located in the Windows System32 directory (usually c:\Windows\System32) and processes the VBScript code. Unlike with .BAT files, Windows is not capable of executing VBScript code directly. If you get an error that “wscript.exe” was not found, then you have not Windows Script installed on your system (it is usually installed by default). You can download and re-install the Windows Script components from the Microsoft web site here.

To open quickly a command prompt in the folder that you selected in Windows Explorer, add a the “Command Prompt Here” Explorer Extension to your System Registry. If you do not know how to do it manually with the Regedit.exe tool from Windows, copy the code below and save it to a file with the name “CmdHere.reg”.

CmdHere.reg

   1: Windows Registry Editor Version 5.00

   2:  

   3: [HKEY_CLASSES_ROOT\Directory\shell\Command &Promt Here]

   4:  

   5: [HKEY_CLASSES_ROOT\Directory\shell\Command &Promt Here\command]

   6: @="cmd.exe /k cd \"%1\""

Double-click on the file and Windows should ask you if are sure that you want to import the contents of the file CmdHere.reg into the system registry. Acknowledge with “OK” and the extension will be installed and ready to use.

Add Tools to Windows Explorer Context Menu for Folders

The Paragraph before already showed you how to create a Windows Explorer shell extension for the Folders context menu, but let me show you some specific examples for the tools that I provided in the first segment of this blog post.

I assume that you copied all the scripts and batch files to a folder called c:\tools. If you want to use a different folder or file name for the scripts, simply adjust the .REG file samples below. Notice the two back slashes (“\”) instead of one? The second one is necessary to escape the “\” of the path name, that the script will not interpret it, but take it as a literal string value. If you forget to change a single back-slash to a double-back-slash, an error will occur if you try to execute/import it. You do not have to specify the path to the batch script at all, if you copied it to a  directory that is included in the %PATH% environment of Windows, such as the Windows directory and the System32 directory of the system.

You can also create one .REG file to install all the scripts and batch files to the Explorer context menu. Make sure that the line “Windows Registry Editor Version 5.00” only appears once in the file at the very top and not multiple times, because that would cause errors, if you try to import it to your registry database.

Here is an example for the registration of the MoveToAlphabetSubFolders.bat

The “Move to A-Z” that you see in the code below is the Label for the context menu. If you want it to say something different, replace the two instances in the code with your label text.

The other script “RemoveEmptySubFolders.bat” requires as parameter the folder name and location to process.
To execute the batch script for the current folder selected in the context menu, add the parameter "%1” to the command. Line 6 of the .REG file below would look like this. Notice the “\” before the (“), like for the “\” of the path is it necessary to “unescape” the double-quotes character to prevent interpretation of the character.

@="C:\\tools\\RemoveEmptySubFolders.bat \"%1\""

MoveToAlphabetSubFoldersInst.reg

   1: Windows Registry Editor Version 5.00

   2:  

   3: [HKEY_CLASSES_ROOT\Directory\shell\Move to A-Z]

   4:  

   5: [HKEY_CLASSES_ROOT\Directory\shell\Move to A-Z\command]

   6: @="c:\\tools\\MoveToAlphabetSubFolders.bat"

Here is the example for one of the VBScript tools. Notice that the command starts with wscript.exe followed by the location and name of the script. This is necessary, because Windows does not execute VBS scripts automatically like it does with BAT (Batch) scripts.

ShortPathInst.reg

   1: Windows Registry Editor Version 5.00

   2:  

   3: [HKEY_CLASSES_ROOT\Directory\shell\Short Path]

   4:  

   5: [HKEY_CLASSES_ROOT\Directory\shell\Short Path\command]

   6: @="wscript.exe c:\\tools\\shortpath.vbs"

You can download all BATCH files, VBScript files and REG files from this post as one ZIP compressed archive:

Download FileToolsCodeAndRegFiles.zip

Also included in the archive are the .REG files: MoveFilesAutoRenDupInst.reg and RemoveEmptySubFoldersInst.reg to install the other two scripts to the Windows Explorer context menu; the .REG file InstallAll.reg to register all 4 scripts and the “Command Prompt Here” extension to Windows Explorer and UnInstallAll.reg to remove all 5 extension from Windows Explorer.

Cheers!

Carsten aka Roy/SAC

De-Dupe Files Script Tool for Windows 32bit V2.0beta

I know, it’s less than a month that I introduced the De-Dupe script Version 1.1. See my original blog post where I introduced the script here. I used the script myself quite a lot and implemented improvements and additional features to it as a consequence of it.

 

I won’t post the whole source code in this post this time though, because it got significantly bigger and would cause problems with loading the post in your web browser. Well, the code is still included in the install package of the script/tool and 42KB zipped is not very much to download, isn’t it.

 

Download De-Dupe V2.0 Beta ( Roy-DeDupeScript20b.zip)

The background story and general file structure remained the same, but I made a lot other changes, especially “under the hood”.  The most important improvement is the PERFORMANCE. Where V1.1 was taking several minutes to process, V2.0 only takes seconds instead. De-Duping folders with thousands and not just hundreds of files is now NOT A PROBLEM for this script anymore. Here are the details:

Name of the Software: DeDupe Files Script Tool
Author: Carsten Cumbrowski
Version 2.0 beta
License: Freeware
Date: April 2009

Visit http://www.cumbrowski.com/ for resources to web and database development and internet marketing. There you can also find the contact page with various means to get in touch with the author of this tool.

The script detects duplicate files within a directory.

Duplicate files are files that have the same MD5 Check Sum value.
Two DIFFERENT/NON IDENTICAL files having the same MD5 Check Sum is not impossible, but highly unlikely.
This allows the script to detect duplicate files regardless of their file name or other characteristics, such as "date created" or "date modified".

The tool scans all files within a directory. It does not include files in sub directories of the processed folder.

Multi-Threading

The Script now supports Multi-Threading for the MD5 Checksum determination. That was the bottle-neck of the previous version of the script. The default is set to 50 threads, but it can be changed in the settings or on the fly via the command line switch /threads:NN where NN should be a number greater than or equal 1. I don't know the maximum value here, because it depends on the machine where the script runs on.

Be careful and only increase it in small steps to increase performance even more. The thread count starts actually with 0, which means, that if you set /threads:49 (default) then you actually get 50 threads.

Dupe Actions

There are 5 different Actions you can choose from to tell the script what to do, if it finds a duplicate file

1. rename dupe to aFile1_EXT_bFile2[DEDUPED].EXT where aFile1 is the original file followed by "_" and its EXT(ension), "_"     bFile2 is the original base file name of the Dupe followed by [DEDUPED] and the dupe files original .EXT(ension)
2. Rename dupes as in 1, but MOVE to a new sub folder "[Deduped]" of the path being processed
3. (DEFAULT ACTION) Don't rename dupes, just MOVE to a new sub folder "[Deduped]"
4. delete dupes (gone for good, unless you enabled "Recycled Bin" to be able to recover deleted files
5. Create sub folder at specified location (/cdb:BACKUPPATH) with name "yyyy-mm-dd_hh-mm-ss_FolderName",  create index file !Index.txt with archive location and name and original locations of files, separated by "|"

The Default DeDupe Action can be overwritten via the command line option
/action:[1...5] , e.g. /action:3  for the default Action

Examples:

Dupe Actions

Dupe Action (1)
If a duplicate file is found, it will be renamed by by appending the original file name as prefix with an '_' as separator, which is also used to replace the "." that indicates the file extension of the original file name (other "." in the file name itself remain). At the end of the file name is the string [DEDUPED] added.

For Example aFile1.EXT and bFile2.EXT are identical. After the script was executed, one of the two files will remain as it is and the other one is being renamed. Which file will be considered the "original" is determined by which file was found first. The script sorts the files by name first, before it dedupes them.

In this example bFile1.EXT would be considered the original and bFile2.EXT will be renamed to aFile1_EXT_bFile2[DEDUPED].EXT. This makes dupes appear right after the original, if you sort the directory by file name. To be able to filter the dupes to copy/move them away or to delete them, use the copy, move or del command in MS DOS. For example "DEL *[DEDUPED].*" would delete all duplicate files found and renamed by the script.

Dupe Action (2)
If you want the dupes renamed as in Action 1, but would like to have them moved away from the source directory, choose Dupe Action 2. Dupes are still being renamed as in (1), but the script moves the dupes to a sub directory called "[DeDuped]" within the processed folder.

Dupe Action (3)
If you just want the dupes moved away from the source folder, but keep the original file names, use this Dupe Action (which is the default action btw)  You will find the duplicate files all in the subfolder "[DeDuped]" in the processed folder.

Dupe Action (4)
If you simply want to get rid of the dupes and delete them, use this Action

Dupe Action (5)
A variation of Action 3. The Difference is that the dupes are not moved to a sub folder below the processed file folder, but to a central dupe archive folder, where a new sub directory is being created with the date and time of the DeDupe processing and the Folder Name that was DeDuped.

The default location for that centralized backup folder is "C:\[DEDUPE_BACKUP]", but that can be changed. Either via the registry settings or on the fly via command line option:

/cdb:BACKUPPATH

Log Files

The script creates one file by default and the second one optional in the processed directory:

"!DeDupe-FileList.txt" - (optional feature) a list of all files in the directory and their MD5 Check Sum Values (tab separated)

"!DeDupeLog.txt" - (enabled by default) a processing logfile where you can find the list of dupes that were detected, their old & new file name and the corresponding original file

If you do not want any of the files to be created, change the options for "WriteFileList" and "WriteDeDupeLog" to "0" in the beginning of the code of "DedupeFilesInFolder2.vbs" ; alternatively use the command line options /log:[0/1] and /list:[0/1] to turn the creation of the list and/or log on/off. You can also specify a different file name for the file list and the DeDupe log, but you cannot change the path.

/list:0 or /list:1
/listfile:FileListName.Ext

/log:0 or /log:1
/logfile:LogFileName.Exe

You can also suppress all dialogs via the command line option /quite:[0/1]. /quite:1 would disable the progress dialog, results message and all error messages.

/quite:0 or /quite:1

Note, the script returns error levels for batch processing regardless of the "quiet" settings.
The ErrorLevel codes are:

0 = Script Ran Successful
1 = Script Ran, but there were no files to process
2 = The script was aborted (only relevant if progress dialog is on)
4 = Script Error (md5sum.exe not or processing path not found)

Important. Version 2 of the Script enforces execution with CSCRIPT.EXE it re-launches itself, if it is executed with WSCRIPT.EXE. It is doing exactly that on purpose, if executed via the Shell Extension. If you are using the Quiet option and want to get the correct ErrorLevel back, you must execute the script with CSCRIPT.EXE from your application!

Installation/De-Installation

Use the provided Batch Scripts "DeDupeInstall.bat" and "DeDupeUnInstall.bat" to install or un-install the De-Dupe Shell Extension.

Installation

Double click on the Batch Script File "DeDupeInstall.bat"
That's it.

Notes:
The install batch file copies md5sum.exe and DedupeFilesInFolder2.vbs into your System32 directory under your windows installation directory and Imports the registry file "DedupeInstall.reg" into your systems registry database. It creates entries under the Registry Key: HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Directory\shell\

Non of the files in the installation directory will be needed anymore to run the script itself. You will need them only to uninstall the tool or to re-install it again, if necessary.

Uninstallation

Double click on the Batch Script File "DeDupeUnInstall.bat"
That's it.

Notes:
The Un-Install batch file deletes the two files from your System32 directory and utilizes the registry file DedupeUnInstall.reg to remove the entries for the script from your systems registry database. If you want to continue to use the tool md5sum.exe and only want to disable the shell extension, either simply double click on the file DedupeUnInstall.reg without executing the uninstall batch file (the script DedupeFilesInFolder2.vbs
will remain in your System32 folder though) or you can copy the tool back into your system folder manually after you ran the uninstall batch file.

Script Settings in the System Registry

The Install Script automatically creates the default settings for the the script execution in the Windows Registry.
If you did not use the install script and run the DeDupe script for the first time, the script will create the missing registry entries based on the default values specified in the script code itself. Those settings are specifically of importance for the use of the DeDupe shell extension in Windows Explorer.

The settings in the registry are set for each Windows User separately.
To find and modify the settings, open the Registry Editor that comes with Windows. (Start / Run, enter: Regedit, press "enter")

Navigate to HKEY_CURRENT_USER \ Software \ DeDupe2 \ Parameters

Value Name               Type        Default Value          Command Line Param Equivalent
------------------------------------------------------------------------------------------
CentralDupeBackupFolder  REG_SZ      C:\[DEDUPE_BACKUP]            /cdb:PATH
DeDupeLogFName           REG_SZ      !DeDupeLog.txt                /logfile:FILENAME
DupeAction               REG_DWORD   3                             /action:N
FileListFName            REG_SZ      !DeDupe-FileList.txt          /listfile:FILENAME
MaxForks                 REG_DWORD   31 (HEX) or 49 (Decimal)      /threads:NN
Quiet                    REG_DWORD   0                             /quite:N
WriteDeDupeLog           REG_DWORD   1                             /log:N
WriteFileList            REG_DWORD   0                             /list:N

Upgrade from Previous DeDupe Versions

You might noticed that the main script has the name "DedupeFilesInFolder2.vbs". The previous version of the script has the name "DedupeFilesInFolder.vbs"

The install script also creates a different shell extension with the name "DeDupe2". If you installed version 1.x of the script and then use the install script for version 2, both scripts will be installed on your machine. You could continue to use them in parallel, if you want to, but I would not recommend it to the average user. I suggest to run the uninstall script of the previous version first and then the install script of the new version.

Note: If you run the uninstall script of the previous version AFTER you installed version 2 of the script, version 2 will no longer function properly because the uninstall script of the previous version also removes the 3rd party tool "md5sum.exe" from the System Directory. You either have to copy that tool back to the windows system directory manually or run the installation script for Version 2 once more. Doing that will overwrite any settings in the registry, which you might have changed already.

About the Software

The DeDupe Windows Explorer Shell Extension Script Tool is written in VBScript and is executed by the system tool WScript.exe. The DeDupe script (DedupeFilesInFolder2.vbs) uses a small support tool that it requires to work properly.

"md5sum.exe" is a small command line tool that return the MD5 Check Sum value for a file.
It can also validate MD5 check sums, which is a feature that is not used by the DeDupe script.
You can find out more information about it at http://etree.org/md5com.html
Md5Sum was written by bruce@gridpoint.com

Legal Stuff/Copyright and Disclaimer

The 3rd party tool that come with the DeDupe script is freeware and can be used and copied by anybody without the need of a license or to pay a fee.  Since I did not write that tool, I cannot take any responsibility for any issues that they might cause by it, via my script or without out.

This DeDupe script is also freeware and can be used, copied and modified for free,

Important Disclaimer!

The author, of this software accepts no responsibility for damages resulting from the use of this product and
makes no warranty or representation, either express or implied, including but not limited to, any implied warranty of merchantability or fitness for a particular purpose.

This software is provided "AS IS", and you, its user, assume all risks when using it.

Change Log

V1.1

• MD5Sum Determination Issue Resolved for file names with spaces in it
• Sorting by File Name Issue Resolved, now the "original" is really the first one sorted by name
• Progress dialog implemented to show status
• Quiet option implemented to suppress all dialogs
• Return of ErrorLevels implemented for batch scripts that call the script
• Rename logic changed, [DEDUPED] added to the renamed file in addition to existing logic
• touch.exe tool removed. It did not work reliable, period
• File List output with file names and their MD5 checksums implemented
• Log File output implemented
• command line parameters introduced to suppress file list and log file creation as well as to enable/disable "quiet" mode
• general code clean up

V2.0

• MD5Sum Determination now separate Step using Multi-Threading for increased Performance
• New DupeAction: 2, 3, 4 and 5 implemented
  1. = Rename dupes as in 1, but MOVE to a new sub folder "[Deduped]" of the path being processed
  2. = Don't rename dupes, just MOVE to a new sub folder "[Deduped]"
  3. = delete dupes
  4. = Create sub folder at specified location with name "yyyy-mm-dd_hh-mm-ss_FolderName", create index file "!Index.txt"   with archive location and name and original locations of files, separated by "|"
• Script now Enforces CSCRIPT.EXE (call from Shell Extension still uses WScript, because if I run it with Cscript from there a stupid DOS Shell Window is visible and open all the time)
• Message Output changed to use IE because of CSCRIPT execution in batch mode (which suppresses Wscript.echo)
• Settings now Saved in Registry, Manual overwrite via command line parameters is still possible. Use of Defaults in Code vs. Registry is also an option.

I hope that you will find this script useful. Please let me know your opinion, suggestions, feedback and recommendations for improvements via the comments section down below.

Cheers!

Carsten aka Roy/SAC

Some Google Tools & Services and Other Annoyances

You know how it is… the small annoyances are the ones that bother us the most, especially if you have to deal with the same thing all the time on an almost daily basis. I am going to show you a few of those annoyances that drive me nuts at the moment.

List of Annoyances Covered by this Post

• Annoyance #1 – Google YouTube Tag Subscriptions
• Annoyance #2 – Google YouTube Comments Notification
• Annoyance #3 – Google YouTube Friend Invites Processing
• Annoyance #4 – Google Chrome Pop-Up Blocker
• Annoyance #5 - 4CC Codec Value Case Sensitivity by VirtualDubMod

If you know me, then you know that I usually not simply complain about something, but also provide suggestions and recommendations for solutions to the problem. Maybe you had one or the other issue as well. In that case it would show you that you are not the only person with this issue. If you don’t, then you know about them now. I do this public so maybe we are lucky that somebody reads this, who is actually in the position to do something about one or more of the problems illustrated or somebody reads this who knows somebody who SHOULD read it, because that other person is one of the people who could solve one or more of those problems.

All but one are related to Google products or services and all but one of those related to Google are about their video sharing site YouTube, which I use pretty much daily … rather extensively (having four different accounts there myself, go figure).

Annoyance #1 – Google YouTube Tag Subscriptions

Let me start light with something that I only encountered recently. I never had that issue in the past that it would have been an annoyance, but as you see, things can change over time.

You have the ability in YouTube to subscribe to video updates of other YouTube accounts/channels (I have 4 of those, CirqueDuSoleilGuru would be the most popular one I guess, SACReleases is the second most popular and the remaining two are TurnbeutelvergesserB and CumbrowskiCom) . You also have the ability to subscribe to a certain keyword phrase or word, to get notified, if somebody uploads a video to the site and uses this phrase or word to tag the video. This can be a very effective way to discover video content about things you are interested in and may be not would have found out about yourself otherwise… by coincidence maybe, if you are  lucky.

It works well, as long as the keyword or phrase is unique and highly descriptive of what you are looking for. If the phrase is ambiguous to begin with, it’s already a problem from the get go, but it also happens that previously unique keywords get “poisoned”. Such poisoning happened to the phrase “Circus” to which I subscribed to.

Everything was alright until Britney Spears decided that she had to release a new single with the title “Circus”. Guess what kind of videos I get now for the term “Circus”… right.., Britney Spears videos “ohne Ende”, but that’s not what I am caring about. I am interested in the ancient performing arts involving breathtaking acrobatics and maybe some mindboggling magic and/or amusing clowns.

It would be nice, if I could at least remove the unwanted videos rather quickly. There is no option today. In order to get rid of them in my Subscription/Alert “Inbox” I have to start watching each video first.. one by one. That is really annoying.  In addition to a way to remove videos that I can identify as “not interested in” without watching quickly from the video listing, would it be helpful to be able to configure negative keywords or stop words and/or using advanced search query parameters like AND & OR & NOT to filter results and/or exclude known ambiguous versions of the same keyword phrase,

Annoyance #2 – Google YouTube Comments Notification

Getting a notification about any comment posted to any of your uploaded videos is great, especially if you open up and let people comment and publish those comments without moderating them first (YouTube’s Spam detection routines are actually doing a great job).

Notifications are not only available at your YouTube account, but you also can enable email notification to your primary email address as well.

The problem with this notifications is that you cannot really do much with them as you would probably want to in the one or other case. You can select the notifications and delete them from your inbox (which does not do anything to the actual comment itself). The link to “see all comments” takes you to the same page where the comment that you are looking at can actually be found, if you are lucky. It won’t take you to the comment though and often to a page where this particular comment can nowhere be found. Getting to the actual comment for options usually involves the use of the on-page keyword search (CTRL-F) and clicking of one or more links to get to the page with the comment at all.

There you will finally find all the things that you might want to do with the comment, such as reply to it, mark as spam, remove it all together and maybe even block the whole user, if he frequently posts comments that you manually delete for some reason time and again.

Would it be so hard to make those things already available from the notification message in your inbox? Is it? C’mon, you must be kidding me! :)

Annoyance #3 – Google YouTube Friend Invites Processing

Especially one of my YouTube accounts gets friend invites all the time. Hey, who doesn’t want to be popular and befriended by everybody, right? Well, yeah, but only if that other person really means it and not just tries to befriend you for pure selfish reasons (aka Spam and Attention Whoring).

It takes more time than it should to find out what the real intentions of that other user are, because you don’t have much information about any person who decides to click the “Add to Friends” button on your channel home page. Information that would help with pre-processing the majority of the requests right from the friend invites list screen.

Is the other user a subscriber to your channel? If he is, chances are much better that this person actually cares about what I am doing on YouTube. Did the person add any of my videos to his favorites or any other playlist?

Did the user ever comment on any of my videos, send a video response or personal message? Just a simple YES/NO would be enough. Counts would be even better, but I don’t want to get too demanding here.

Also helpful is the info how many friends that user already has and where he is from. Yes, you can get those two information if you click on the name and visit his channel homepage, but that takes time too and if you do that and press the back button, you won’t get back were you left off, not even to the same section, which makes things twice as bad and even more time consuming.

In my example below you can see how it could look like, if those suggestions would be implemented. Guess which friend request I’d ignore right from that screen without even checking further? Right, the fourth one. Not a subscriber, never fav’ed or bookmarked any of my stuff, never posted a comment or contacted me, but has more friends than I know real people. That user does not care about me and only wants to promote himself. He clicked on “Add as Friend” to as many people he could, maybe even in an automated fashion via a spam tool or script.

Enough of YouTube. There is certainly more than those things that I pointed out, but those are the things that I hate to deal with on a nearly day to day basis. It’s getting old and not necessary really.

Annoyance #4 – Google Chrome Pop-Up Blocker

It is already sad enough that tools like AI Roboform and others do not (cannot?) support the somewhat new Google Chrome web browser. It’s fast, I can attest to that. Yeah, faster than IE (dough) and also faster than FireFox (especially of fully loaded with plug-ins). Google added some tools and features themselves to the browser, such as the Pop-Up blocker, but the bad thing is that this feature fell victim to the wide spread Google syndrome of “smart” (or not) automation with little to no ways for humans to interfere and control the behavior of the feature.

Google’s engineers are often too self-confident that they are able to create a model or algorithm that is capable of predicting the users intentions, needs and goals in every case and at every time, under any possible real life condition and situation. I don’t want to downplay the intelligence of any Google engineer or something like that, but sorry, they should have learned by now that this is simply impossible to do. You can make a good guess and be right most of the time, but you will never be able to get it 100% right every single time.

If you combine this with the disability to manually overwrite unwanted and miss-guessed behavior, you got yourself a classic example of an annoyance on your hands. If you had to use a site that generates legitimate pop-ups that are needed to use that web site properly and each and every one of those pop-ups gets blocked by the browser, requiring additional steps to get to the pop-up window and worse, have close to that action also the option to block all pop-ups entirely and irreversible without the ability to manually get it back, than you will understand what I am so annoyed about.

I hope that my little illustration below shows what I am mean by that. I also added my suggestions to solve those problems to that illustration as well. The simple and short answer is: WHITE LIST option please! Thanks.

Annoyance #5 - 4CC Codec Value Case Sensitivity by VirtualDubMod

Update: I found a solution for this annoyance, but I am not really happy about it. Anyway, the problem goes away, if you install FFDSHOW, an open source video decoding/codecs/filters  package/utility(ies). You can download the latest version of FFDSHOW at AfterDawn.com. I wish I would understand what FFDSHOW does to make it work to be able to fix this problem without the need to install the whole FFDSHOW package. :( 

Okay, Google is now of the hook for the time being. Others create annoyances as well. It’s not an exclusive right that Google has bunked for their products and services. VirtualDub is a nice little (and entirely free) tool for the manipulation and conversation of video files. It has many filters and plug-ins to be able to read various video source formats and to enhance or manipulate those videos. At the end you always get out an AVI file, but you have can have a wide variety of AUDIO and VIDEO encoder options that you can use. AVI is only a very broad and widely supported container, unlike some other video formats, such as Adobe Flash (FLV) or Apple Quick Time (MOV) that are limited in which video and/or audio encoding algorithm can be used.

A popular video codec that I prefer to use for my AVI files is XVID, which is a freeware Mpeg-4 based video codec, like the popular commercial codec Div/X, but IMO better (compression). The codec that was used for the audio and video in an AVI file is noted in the header section of the file that other tools, such as video players like Windows Media Player are able to open that video file and to use the proper decoders to render the video and audio data on to your screen. I use more than one video converter in addition to VirtualDub, such as MP4Converter by 4Media for most video sources and Moyea FLV Converter for Flash Video (FLV) to AVI conversions only. I wrote a special post about the subject of video tools and converters last year, if you are interested in this subject and would like to learn more about it.

Those converters are capable of generating AVI files using the XVID video codec. For whatever reason, those tools write the name (4CC Code) of the codec in lower-case letters into the AVI file header.

For another unknown reason, VirtualDub is case-sensitive when it comes to codec settings in the video file, although “xvid”, “XVID” or “XVid” or “Xvid” always mean the same thing… The 4CC specifications are not case-sensitive. “xvid” and “XVID” therefore cannot be two different video codec variations that could be meant.

VirtualDub does not care. If the codec name in the AVI is “xvid” instead of “XVID”, VirtualDub plays stupid and claims that it does not support the used codec. BS. Of course it does. If I use another free tool to manually change/set the 4CC codec name in the AVI header and change “xvid” to “XVID”, VirtualDub reads and processes that video file without any problems. The video data themselves remained unchanged, only the info in the header was changed. This is a pain in the butt, especially if I want to do batch processing, because I have not found a tool yet that lets me change the 4CC value in an AVI file in a batch process, e.g. via command line options or something like that. I also don’t know how to get the other tools to write XVID in upper-case instead of lower-case into the AVI file headers that they produce.

I did manage somehow on my previous system (before I re-installed everything from scratch) to convince VirtualDub that it can read XVID and also xvid files to then generate new AVI files with XVID used in the header.

I cannot remember how I did that. I tried a number of things with no success so far. If you happen to know the answer to this, please let me know in the comments below. I’d really appreciate it. Until then, why this BS to begin with? Is it so hard for the folks from the VirtualDub open source project to make the tool non-case sensitive when it comes to the 4CC value in the headers of AVI video files? Gee!

Enough Annoyances! I don’t think that anybody in his right mind would have read this whole post, but as I said at the beginning of this post, I felt like having to do this, in the secret hope that somebody will read it, who is in the position to remedy one or more of the mentioned annoyances.

If you had similar or the same problems I had, feel free to tell me and anybody else who reads this post about it. You and me are not alone. Trust me on this one!

Cheers!

Carsten aka Roy/SAC

Jason Scott from Textfiles.com Text-urized

I wrote a while back already a post about the online Image to Text (ASCII) converter tool at Photo2Text.com.

Back then I used Matt Cutts from Google as my Guinea-Pig to illustrate it’s capabilities. This is not the purpose of this blog post though. Shortly after I did my post, I played around with it a bit more. I experimented with a photograph of Jason Scott Sadofsky from Textfiles.com (who also did the DVD documentary “BBS – The Documentary”, which I highly recommend, if you are interested in the subject of Bulletin Board Systems)

I tweaked the source photo of him to get better results and saved them on my hard drive and then forgot about them until now. I stumbled across them by accident and thought that I should not hide them, or at least make them accessible for Jason himself.

I took the results from the converter and tweaked them a bit more in Photoshop, adding some color and stuff like that. You can see below the stages of my changes. Click on the thumbnail images for the larger original sized version of it.

Original 7-Bit ASCII as Image.  Download the original text ASCII file	Colors added	Final picture with background added and name tag

This is the final piece and I hope that Jason (and you too) will like it.

I also created an ANSI version of the photograph with the tool. It is only using the number characters 0-9, but a very small font. The result is good, but it looks to little like an ASCII to me, because of the font size.

This ANSI was actually created as a HTML document. You can see the original HTML/Text version here.

If you are interested in the subject of image to ASCII art conversion, I also suggest to check out my post where I reviewed a number of online and desktop image to text converter tools.

I also wrote a post about video to text conversion via the vlan video player with special conversion plug-in.

If the subject of ASCII and ANSI text art is new to you and you would like to learn more about this old school art form that actually pre-dates the computer, check out my ASCII Art Academy for more resources and information.

Semi-related: Did you see my post about ASCII/RTTY art nudes versus their original Centerfold Photographs?
I still need help with matching up more of the ASCIIs with their source photographs. You can help.

Cheers!

Carsten aka Roy/SAC

De-Dupe Files Script Tool for Windows 32bit

You can skip the background story about the why, when and how I decided to write this little File De-Duping script, if you are not interested in it and jump right the section about how the script works, what it does, where to download it, how to install and de-install it and the source code of it as well. The tool is Freeware, but any donation (money or goods) and/or simply a “Thank You” (e.g. via the comments section at the end of this post) are appreciated nevertheless.

Important Node: I updated the script and this post, because of some bugs that I found and issues with the 3rd party tool “touch.exe”. I had to remove it and come up with another solution for the problem that it solved. I also added some nice stuff, so it was more than just bug fixing :).

The Background Story

Who did not have the problem yet to have hundreds of any type of files, text files, documents, spread sheets, images, videos and others in one directory with the a high chance that you have duplicates, identical files that only have a different file name. I often have to deal with duplicate images that I downloaded from the internet.

I cannot always remember, if I downloaded a particular image already or not and go by the slogan, archive / save / back up first, sort later, because there will be nothing to sort, if you don’t save a copy and go back to the place where you found them at a later time, to learn that the stuff isn’t there anymore for any reason. Maybe even the whole web site is gone the way of the Dodo.

The original file names are often useless, either to generic, like image1.jpg, logo.gif or simply 1.jpg, 2.jpg etc. or they are long and cryptic without any meaning, like: 3104458219_cc0dfd3980_o_d.jpg or something like that. So you end up giving the files your own name and thus make it highly likely that you download the same file at a different time again and give it a different file name than you did the first time around. Voila, duplicate.

I looked at a number of tools and options and found some that were decent, but all of them had always something (or didn’t have something) to make things that should be semi automatic to a manual time consuming ordeal. What I wanted is something that finds duplicates, does not delete them straight out or have me right then go over each dupe found to make a decision about it right away. I also wanted to be able to verify that the dupes found are really identical so it must be easy to know quickly which file is a duplicate of another file.

Non of the solutions that I tried delivered on all those aspects so de-duping of files was painstaking and time consuming, if I did it or I simply would not do it at all and carry a bunch of dupe garbage around .

The idea for this simple script of mine came when I did the sorting and inventory of the SAC art pack releases back in December last year and January this year. I uploaded files, like MODs that I converted to Mpeg-1 Audio Layer 3, short MP3, and other files to my file sharing account at Mediafire.com. I learned that Mediafire.com has a de-duper function build into their system that prevents user from uploading the same file twice to their account. You can upload it a second time, but you will get the message that the file already exists and that your new upload was being deleted. I got that message and first thought that it was an error on Mediafire’s part.

What dupes? There were no dupes, I thought. Yeah, there were some SID Adlib music files that had the same size and all, but there where like a dozen of them, all with a different name and even different prefix, indicating that the files were created by different musicians. Also the dates were different. Being close to send an email to Mediafire customer support, I decided to listen to the songs that could be potential duplicates to the file that was rejected (Mediafire didn’t tell me the file name or shared URL of the original file). And there it was, the same tune with a different file name and different file dates by the same artist.

I was curious about how Mediafire noticed that the files are identical and did some research. They are using the MD5 Check Sum value of a file. Chances are astronomical that two files that are not byte by byte identical will have the same MD5 check sum value. That’s a smart, fast and easy way to find dupes and the idea of writing a de-dupe script that does exactly what I want it to do was born.

This is enough of a background story I believe. Let’s get busy with the Script itself.

How the Script Works

The script detects duplicate files within a directory. Duplicate files are files that have the same MD5 Check Sum value.  Two DIFFERENT/NON IDENTICAL files having the same MD5 Check Sum is not impossible, but highly unlikely. This allows the script to detect duplicate files regardless of their file name or other characteristics, such as "date created" or "date modified".

The tool scans all files within a directory. It does not include files in sub directories of the processed folder.

If a duplicate file is found, it will be renamed by by appending the original file name as prefix with an _ as separator, which is also used to replace the "." that indicates the file extension of the original file name (other "." in the file name itself remain). At the end of the file name is the string [DEDUPED] added.

Example

For Example aFile1.EXT and bFile2.EXT are identical. After the script was executed, one of the two files will remain as it is and the other one is being renamed. Which file will be considered the "original" is determined by which file was found first. The script sorts the files by name first, before it de-dupes them.

In this example bFile1.EXT would be considered the original and bFile2.EXT will be renamed to aFile1_EXT_bFile2[DEDUPED].EXT. This makes dupes appear right after the original, if you sort the directory by file name. To be able to filter the dupes to copy/move them away or to delete them, use the copy, move or del command in MS DOS. For example "DEL *[DEDUPED].*" would delete all duplicate files found and renamed by the script.

More New Stuff

The script creates two files by default in the processed directory:

• "!DeDupe-FileList.txt" - a list of all files in the directory and their MD5 Check Sum Values (tab separated)
• "!DeDupeLog.txt" - a processing log file where you can find the list of dupes that were detected, their old & new file name and the corresponding original file

If you do not want any of the files to be created, change the options for "WriteFileList" and "WriteDeDupeLog" to "0" in the beginning of the code of "DedupeFilesInFolder.vbs". Alternatively use the command line options:
”/log:[0/1]“ and “/list:[0/1]“ to turn the creation of the list and/or log on/off.

You can also suppress all dialogs via the command line option “/quite:[0/1]”.
“/quite:1” would disable the progress dialog, results message and all error messages.

Note, the script returns error levels for batch processing regardless of the "quiet" settings.
The Error Level codes are:

0 = Script Ran Successful
1 = Script Ran, but there were no files to process
2 = The script was aborted (only relevant if progress dialog is on)
4 = Script Error (md5sum.exe not or processing path not found)

Also new, a nice progress dialog using MS Internet Explorer and an extended results message box. Here are some screen shots of the new and updated windows.

 

Installation/De-Installation - Download

Download this small 36 KB ZIP file with the name Roy-DeDupeScript11b.zip and extract the archive to a folder on your local hard drive. The ZIP archive contains the following EIGHT files:

DedupeFilesInFolder.vbs

DeDupeInstall.bat

DedupeInstall.reg

DeDupeUnInstall.bat

DedupeUnInstall.reg

file_id.diz

md5sum.exe

Readme.txt

File_ID.diz and Readme.txt are simply text files, md5sum.exe is a 3rd party command line utility that was developed by somebody else and is being needed for my script. The two .BAT files and two .REG are only needed for the installation and de-installation of the tool. The .VBS file is the main tool script written in Visual Basic Script (VBScript).

Use the provided Batch Scripts "DeDupeInstall.bat" and "DeDupeUnInstall.bat" to install or un-install the De-Dupe Shell Extension.

Installation

Double click on the Batch Script File "DeDupeInstall.bat"
Thats it.

Notes:
The install batch file copies md5sum.exe and DedupeFilesInFolder.vbs into your System32 directory under your windows installation directory  and Imports the registry file "DedupeInstall.reg" into your systems registry database. It creates entries under the Registry Key:

HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Directory\shell\

Non of the files in the installation directory will be needed anymore to run the script itself. You will need them only to uninstall the tool or to re-install it again, if necessary.

De-installation

Double click on the Batch Script File "DeDupeUnInstall.bat"
The Un-Install batch file deletes the three files from your System32 directory and utilizes the registry file DedupeUnInstall.reg to remove the entries for the script from your systems registry database. If you want to continue to use md5sum.exe and only want to disable the shell extension, either simply double click on the file DedupeUnInstall.reg without executing the uninstall batch file (the script DedupeFilesInFolder.vbs will remain in your System32 folder though) or you can copy the tool back into your system folder manually after you ran the uninstall batch file.

About the Software

The De-Dupe Windows Explorer Shell Extension Script Tool is written in VBScript and is executed by the system tool WScript.exe. The De-Dupe script (DedupeFilesInFolder.vbs) uses a small support tool that it requires to work properly.

"md5sum.exe" is a small command line tool that return the MD5 Check Sum value for a file.
It can also validate MD5 check sums, which is a feature that is not used by the De-Dupe script.
You can find out more information about it at http://etree.org/md5com.html
Md5Sum was written by bruce@gridpoint.com

Legal Stuff/Copyright and Disclaimer

The 3rd party tool that come with the De-Dupe script is freeware and can be used and copied by anybody without the need of a license or to pay a fee. Since I did not write that tool, I cannot take any responsibility for any issues that they might cause by it, via my script or without out.

This De-Dupe script is also freeware and can be used, copied and modified for free,

Important Disclaimer!

The author, of this software accepts no responsibility for damages resulting from the use of this product and
makes no warranty or representation, either express or implied, including but not limited to, any implied warranty of merchantability or fitness for a particular purpose.

This software is provided "AS IS", and you, its user, assume all risks when using it.

Depending if I have the time and the urge to extend on the script, new features might get added to this tool in the future. I could envision additional configuration options and alternative options for what to do with duplicate files that were found by the script. Since you are free to do modifications to the script yourself and improve on it, I would appreciate, if you would let me know and send me the enhanced version of yours, if you decide to take matters into your own hands. :)

Change Log

V1.1

MD5Sum Determination Issue Resolved for file names with spaces in it

Sorting by File Name Issue Resolved, now the "original" is really the first one sorted by name

Progress dialog implemented to show status

Quiet option implemented to suppress all dialogs

Return of Error Levels implemented for batch scripts that call the script

Rename logic changed, [DEDUPED] added to the renamed file in addition to existing logic

touch.exe tool removed. It did not work reliable, period

File List output with file names and their MD5 checksums implemented

Log File output implemented

command line parameters introduced to suppress file list and log file creation as well as to enable/disable "quiet" mode

general code clean up

Source Code

Here is the Source Code of the script. After that is also the code of the install and uninstall batch files and registry key values and settings.

Again, you can download the whole code and the 3rd part command line tool md5sum.exe in a single ZIP archive called Roy-DeDupeScript11b. You may need a ZIP extracting utility, although Windows XP and later should be able to open the file without the need to install additional software. However, if this does not work for any reasons, either download the commercial program WinZip at this web site or download and install the open source Zip and other archiver’s processing tool called PeaZip.

DedupeFilesInFolder.vbs Source Code

   1: =======================================================================

   2: Parameters you might want to change

   3: =======================================================================

   4: Specifies the action to take if dupes are found, used by DupeHandling

   5: Values other than 1 are not supported/implemented yet

   6: Dim DupeAction: DupeAction = 1

   7:  

   8: If you want to suppress the progress dialog, the results message popup 

   9: and Error Messages set bQuiet = true

  10: Note, the script returns error levels for batch processing regardless

  11: of the bQuiet settings. The ErrorLevel codes are:

  12: 0 = Script Ran Successful

  13: 1 = Script Ran, but there were no files to process

  14: 2 = The script was aborted (only relevant if progress dialog is on)

  15: 4 = Script Error (md5sum.exe not or processing path not found) 

  16: Dim bQuiet: bQuiet = false

  17:  

  18: LOGFILES

  19: Set to 1 to generate file list with name and md5 sum, set to 0 to disable

  20: Dim WriteFileList: WriteFileList = 1

  21: File name for the file list. File is saved in processing path folder

  22: Dim FileListFName: FileListFName = "!DeDupe-FileList.txt"

  23:  

  24: Write a log file with all dupes that were processed

  25: Dim WriteDeDupeLog: WriteDeDupeLog = 1

  26: File name for the file list. File is saved in processing path folder 

  27: Dim DeDupeLogFName: DeDupeLogFName = "!DeDupeLog.txt"

  28:  

  29: =======================================================================

  30: Dont touch the stuff below this line, unless you know what 

  31: you are doing.

  32: =======================================================================

  33: Dim oFso: set oFso = Wscript.createobject("scripting.fileSystemObject")

  34: Dim oFolder, oFiles, oFile, oLogFile

  35: Dim iCounter: iCounter = 0 File Counter

  36: Dim sFolderPath Work Folder Path (Current folder)

  37: Dim arguments: Set arguments = Wscript.arguments

  38: Dim md5sumPath,sErr, sMsg, sMD5CS

  39: Dim MyArray() Array with MD5 Checksum and File Names

  40: Dim iDupCnt: iDupCnt = 0  Counter for Dupes

  41: Dim iDupErr: iDupErr = 0  Error Count for Dupe Action

  42: Dim iMD5SumErr: iMD5SumErr = 0 Error Count for MD5 Check Sum Calc

  43: Dim iFilesCnt: iFilesCnt = 0 Number of files proc 

  44: Dim iFilesProc:iFilesProc = 0 Number of files processed (iFilesCnt - iMD5SumErr)

  45: Dim iDupProc Dupe Processed Count (iDupCnt - iDupErr) 

  46:  

  47: global const and vars for Statusbar

  48: Const conBarSpeed = 80

  49: Const conForcedTimeOut = 900000

  50: Dim objIE

  51: Dim objProgressBar

  52: Dim objTextLine1

  53: Dim objTextLine2

  54: Dim objQuitFlag

  55:  

  56: Dim bAbort: bAbort = false

  57:  

  58: System Constants

  59: Const SYSTEM_FOLDER = 1, TEMP_FOLDER = 2

  60: Const ForAppending = 8

  61: Const ForReading = 1

  62: Const ForWriting = 2

  63:  

  64: ========================================================================

  65: Initialization of Work Environment

  66:  

  67: if arguments.Named.Exists("quiet") then

  68:  

  69:   if arguments.Named.Item("quiet") = 1 then

  70:     bQuiet = true

  71:   end if

  72:  

  73:   if arguments.Named.Item("quiet") = 0 then

  74:     bQuiet = false

  75:   end if

  76:  

  77: end if

  78:  

  79: if arguments.Named.Exists("list") then

  80:  

  81:   if arguments.Named.Item("list") = 0 or arguments.Named.Item("list") = 1 then

  82:     WriteFileList = arguments.Named.Item("list")

  83:   end if

  84:  

  85: end if

  86:  

  87: if arguments.Named.Exists("log") then

  88:  

  89:   if arguments.Named.Item("log") = 0 or arguments.Named.Item("log") = 1 then

  90:     WriteDeDupeLog = arguments.Named.Item("log")

  91:   end if

  92:  

  93: end if

  94:  

  95: Check for command line paramater passed

  96:  

  97: if arguments.unnamed.count = 0 then

  98:   Set Path to Current Path 

  99:   sFolderPath = ofso.GetAbsolutePathName(".")

 100: else

 101:   Set Path to folder that was passed as argument for the script call

 102:   sFolderPath = arguments.unnamed(0)

 103: end if

 104:  

 105: Make sure that 3rd party tools md5sum.exe and touch.exe are either in 

 106: the System32 directory or the current path (I dont check the whole Path Env)

 107: md5sumPath   = oFso.BuildPath(oFso.GetSpecialFolder(SYSTEM_FOLDER), "md5sum.exe")

 108:  

 109: if not oFso.FileExists(md5sumPath) then

 110:   md5sumPath = oFso.BuildPath(oFso.GetAbsolutePathName("."), "md5sum.exe")

 111:  

 112:   if not oFso.FileExists(md5sumPath) then

 113:     sErr     = sErr & "Md5sum.exe not found in " & oFso.GetSpecialFolder(SYSTEM_FOLDER) & _

 114:     " nor " & oFso.GetAbsolutePathName(".") & vbcrlf & vbcrlf

 115:   end if

 116:  

 117: end if

 118:  

 119: Make sure that the folder (especially the ones passed as Param) exists

 120:  

 121: if not oFso.FolderExists(sFolderPath) then

 122:   sErr = sErr & "Processing Folder: " & sFolderPath & _

 123:   " does not exist." & vbcrlf & vbcrlf

 124: end if

 125:  

 126: If something is not right, show error and abort the script

 127:  

 128: if sErr <> "" then

 129:   if bQuiet = false then Wscript.echo sErr

 130:   CleanUpAndQuit 4

 131: end if

 132:  

 133: Dim sLogOutput: sLogOutput = oFso.BuildPath(sFolderPath,DeDupeLogFName)

 134:  

 135: Okay.. Lets get started

 136: ------------------------------------------------------------------------

 137:  

 138: Set oFolder = oFso.GetFolder(sFolderPath)

 139: Set oFiles  = oFolder.Files

 140: iFilesCnt   = oFiles.count

 141:  

 142: if iFilesCnt > 0 then

 143:   ReDim MyArray(oFiles.count,3)

 144:   Build 2 Dimensional Array with CheckSum of 

 145:   Filename & File Name itself for all files in 

 146:   current directory. Looking like this 

 147:   (x = dimention 2 and y = dimention 1) 

 148:   the 3rd column is MD5 +[]+ lower case file name for sorting purposes

 149:   43a52d14577de0299146aa9f8f0c062f, file1.ext, 43a52d14577de0299146aa9f8f0c062f[]file1.ext 

 150:   0052d12577de56567546aa9f8f0c0af3, file2.ext, 0052d12577de56567546aa9f8f0c0af3[]file2.ext

 151:  

 152:   if bQuiet = false then

 153:     Launch Status Bar

 154:     StartIE "De-Dupeing Files in " & sFolderPath

 155:     SetLine1 "Step 1/4: Reading Files and MD5 Check Sums. Path:" & sFolderPath

 156:   end if

 157:  

 158:   For each oFile in oFiles

 159:     iCounter = iCounter + 1

 160:     sMD5CS   = GetMd5Sum(oFile.name)

 161:     MyArray(iCounter - 1,0) = sMD5CS

 162:     MyArray(iCounter - 1,1) = oFile.name

 163:     MyArray(iCounter - 1,2) = sMD5CS & "[]" & lcase(oFile.name)

 164:     Check if Abort Button was pressed

 165:  

 166:     if bQuiet = false then

 167:  

 168:       If IsQuit() = True Then

 169:         bAbort = true

 170:         Exit For

 171:       End If

 172:  

 173:       Set Status Bar Value   

 174:       SetLine2 "Files Processed: " & CStr(iCounter) & " of " & cstr(iFilesCnt)

 175:     end if

 176:  

 177:   Next

 178:  

 179: end if

 180:  

 181: iCounter = iCounter - 1

 182:  

 183: if bAbort = true and bQuiet = false then

 184:   Close Status Bar

 185:   CloseIE

 186: end if

 187:  

 188: if iCounter >= 0 and bAbort = false then

 189:  

 190:   if bAbort = false then

 191:  

 192:     if bQuiet = false then

 193:       Set Status Bar Value     

 194:       SetLine1 "Step 2/4: Sort Files"

 195:       SetLine2 "Processing " & cstr(iCounter - 1) & " Files"

 196:     end if

 197:  

 198:     Sort the Array by File Name

 199:     Call QuickSort(MyArray,0,ubound(MyArray,1),2)

 200:  

 201:     if bQuiet = false then

 202:       Check if Abort Button was pressed

 203:  

 204:       If IsQuit() = True Then

 205:         bAbort = true

 206:       End If

 207:  

 208:     end if

 209:  

 210:   end if

 211:  

 212:   if WriteFileList = 1 then

 213:     Write File List out into Text File

 214:  

 215:     if bQuiet = false then

 216:       Set Status Bar Value

 217:       SetLine1 "Step 3/4: Writing File List"

 218:       SetLine2 oFso.BuildPath(sFolderPath,FileListFName)

 219:     end if

 220:  

 221:     Call WriteFile(MyArray)

 222:     Check if Abort Button was pressed

 223:  

 224:     if bQuiet = false then

 225:  

 226:       If IsQuit() = True Then

 227:         bAbort = true

 228:       End If

 229:  

 230:     end if

 231:  

 232:   end if

 233:  

 234:   if bAbort = false then

 235:  

 236:     if bQuiet = false then

 237:       Set Status Bar Value

 238:       SetLine1 "Step 4/4: Detect and Process Duplicates"

 239:       SetLine2 ""

 240:     end if

 241:  

 242:     Detect Duplicates     

 243:     Call FindDupes(MyArray)

 244:  

 245:     Wrapping up

 246:     iDupProc = iDupCnt - iDupErr

 247:  

 248:     if bQuiet = false then

 249:       Close Status Bar

 250:       CloseIE

 251:     end if

 252:  

 253:     sMsg = "Number of Files Found: " & iFilesCnt & vbcrlf & _

 254:     "Number of MD5 Sum Errors: " & iMD5SumErr & vbcrlf & _

 255:     "Number of Files Processed: " & iFilesProc & vbcrlf & _

 256:     "------------------------------------" & vbcrlf & _

 257:     "Number of Dupes Found: " & iDupCnt & vbcrlf & _

 258:     "Number of Dupe Processing Errors: " & iDupErr & vbcrlf & _

 259:     "Number of Dupes Processed: " & iDupProc & vbcrlf

 260:  

 261:     ErrorLogWrite "Number of Files Found: " & iFilesCnt

 262:     ErrorLogWrite "Number of MD5 Sum Errors: " & iMD5SumErr

 263:     ErrorLogWrite "Number of Files Processed: " & iFilesProc

 264:     ErrorLogWrite "Number of Dupes Found: " & iDupCnt

 265:     ErrorLogWrite "Number of Dupe Processing Errors: " & iDupErr

 266:     ErrorLogWrite "Number of Dupes Processed: " & iDupProc

 267:  

 268:     if WriteFileList = 1 then

 269:       sMsg = sMsg & vbcrlf & "List of Files Generated at:" & vbcrlf & _

 270:       oFso.BuildPath(sFolderPath,FileListFName) & vbcrlf

 271:       ErrorLogWrite "List of Files Generated at: " & _

 272:       oFso.BuildPath(sFolderPath,FileListFName)

 273:     end if

 274:  

 275:     if WriteDeDupeLog = 1 then

 276:       sMsg = sMsg & vbcrlf & "Log File Generated at: " & vbcrlf & sLogOutput

 277:     end if

 278:  

 279:     if bQuiet = false then

 280:       WScript.echo sMSg

 281:     end if

 282:  

 283:   else

 284:  

 285:     if bQuiet = false then

 286:       Close Status Bar

 287:       CloseIE

 288:     end if

 289:  

 290:   end if

 291:  

 292: else

 293:  

 294:   if bAbort = false then

 295:     No Files Found to dedupe

 296:  

 297:     if bQuiet = false then

 298:       Wscript.echo "No Files to de-dupe found in " & sFolderPath

 299:     end if

 300:  

 301:     CleanUpAndQuit 1

 302:   end if

 303:  

 304: end if

 305:  

 306: if bAbort = true then

 307:   Aborted Message

 308:  

 309:   if bQuiet = false then

 310:     Wscript.echo "The De-Dupe Script Was abborted."

 311:   end if

 312:  

 313:   CleanUpAndQuit 2

 314: end if

 315:  

 316: CleanUpAndQuit 0

 317:  

 318: ==============================================================================

 319: Function GetMd5Sum(ByVal strFile)

 320:    Declare the FileSystemObject object constants and variables.

 321:   Dim objTS, strTempFile, strCmdLine, objRE

 322:  

 323:   With oFso

 324:      Construct a temporary filename.

 325:     Do

 326:     strTempFile = .BuildPath(.GetSpecialFolder(TEMP_FOLDER), "!" & .GetTempName)

 327:   Loop While .FileExists(strTempFile)

 328:  

 329:   Use cmd.exe to construct a command that will execute md5sum.exe

 330:   strCmdLine = .BuildPath(.GetSpecialFolder(SYSTEM_FOLDER), "cmd.exe") _

 331:    & " /c " & md5sumPath & "  """ & strFile & """>" & strTempFile

 332:  

 333: End With

 334:  

 335:  Execute the command in a hidden window. Wait for the command

 336:  to complete before continuing.

 337: CreateObject("WScript.Shell").Run strCmdLine, 0, True

 338:  

 339:  Open the temporary file.

 340: s         = ""

 341: On Error Resume Next

 342: Set objTS = oFso.OpenTextFile(strTempFile, 1)

 343: s         = objTS.ReadAll

 344: On Error Goto 0

 345:  

 346: check that it didnt fail and has the checksum

 347:  

 348: if trim(s) <> "" and instr(s," *") > 0 then

 349:   GetMD5Sum  = left(s,instr(s," *") - 1)

 350:   iFilesProc = iFilesProc + 1

 351: else

 352:   Error... not good

 353:   iMD5SumErr = iMD5SumErr + 1

 354:   GetMD5Sum  = ""

 355: end if

 356:  

 357: objTS.Close

 358: oFso.DeleteFile strTempFile

 359: End Function

 360:  

 361: ==================================================================================

 362: Array Sort Functions

 363: Sub SwapRows(ary,row1,row2)

 364: == This proc swaps two rows of an array 

 365: Dim x,tempvar

 366:  

 367: For x = 0 to Ubound(ary,2)

 368:   tempvar     = ary(row1,x)

 369:   ary(row1,x) = ary(row2,x)

 370:   ary(row2,x) = tempvar

 371: Next

 372:  

 373: End Sub  SwapRows

 374: Sub QuickSort(vec,loBound,hiBound,SortField)

 375: ==--------------------------------------------------------==

 376: == Sort a 2 dimensional array on SortField        ==

 377: ==                            ==

 378: == This procedure is adapted from the algorithm given in: ==

 379: ==  ~ Data Abstractions & Structures using C++ by ~   ==

 380: ==  ~ Mark Headington and David Riley, pg. 586  ~   ==

 381: == Quicksort is the fastest array sorting routine for   ==

 382: == unordered arrays.  Its big O is  n log n         ==

 383: ==                            ==

 384: == Parameters:                      ==

 385: == vec     - array to be sorted             ==

 386: == SortField - The field to sort on (2nd dimension value) ==

 387: == loBound and hiBound are simply the upper and lower   ==

 388: ==   bounds of the arrays 1st dimension.  Its probably  ==

 389: ==   easiest to use the LBound and UBound functions to  ==

 390: ==   set these.                       ==

 391: ==--------------------------------------------------------==

 392: Dim pivot(),loSwap,hiSwap,temp,counter

 393: Redim pivot (Ubound(vec,2))

 394:  

 395: == Two items to sort

 396:  

 397: if hiBound - loBound = 1 then

 398:  

 399:   if vec(loBound,SortField) > vec(hiBound,SortField) _

 400:     then Call SwapRows(vec,hiBound,loBound)

 401:   End If

 402:  

 403:   == Three or more items to sort

 404:  

 405:   For counter = 0 to Ubound(vec,2)

 406:     pivot(counter)       = vec(int((loBound + hiBound) / 2),counter)

 407:     vec(int((loBound + hiBound) / 2),counter) = vec(loBound,counter)

 408:     vec(loBound,counter) = pivot(counter)

 409:   Next

 410:  

 411:   loSwap = loBound + 1

 412:   hiSwap = hiBound

 413:  

 414:   do

 415:   == Find the right loSwap

 416:   while loSwap < hiSwap and vec(loSwap,SortField) <= pivot(SortField)

 417:   loSwap = loSwap + 1

 418:   wend

 419:   == Find the right hiSwap

 420:   while vec(hiSwap,SortField) > pivot(SortField)

 421:   hiSwap = hiSwap - 1

 422:   wend

 423:   == Swap values if loSwap is less then hiSwap

 424:   if loSwap < hiSwap then Call SwapRows(vec,loSwap,hiSwap)

 425:  

 426: loop while loSwap < hiSwap

 427:  

 428: For counter = 0 to Ubound(vec,2)

 429:   vec(loBound,counter) = vec(hiSwap,counter)

 430:   vec(hiSwap,counter)  = pivot(counter)

 431: Next

 432:  

 433: == Recursively call function .. the beauty of Quicksort

 434: == 2 or more items in first section

 435: if loBound < (hiSwap - 1) then Call QuickSort(vec,loBound,hiSwap - 1,SortField)

 436: == 2 or more items in second section

 437: if hiSwap + 1 < hibound then Call QuickSort(vec,hiSwap + 1,hiBound,SortField)

 438:  

 439: End Sub  QuickSort

 440: Sub PrintArray(vec,lo,hi,mark)

 441: ==-----------------------------------------==

 442: == Print out an array from the lo bound  ==

 443: ==  to the hi bound.  Highlight the column ==

 444: ==  whose number matches parm mark     ==

 445: ==-----------------------------------------==

 446: Dim i,j

 447: sRes = ""

 448:  

 449: For i = lo to hi

 450:  

 451:   For j = 0 to Ubound(vec,2)

 452:     sRes = sRes & vec(i,j) & vbTab & vbTab

 453:   Next

 454:  

 455:   sRes   = sRes & vbcrlf

 456: Next

 457:  

 458: wscript.echo sRes

 459: End Sub

 460:  

 461: ===================================================================================

 462: Actual De-Duper Functions

 463: Sub FindDupes(Arr)

 464: Dim a, b, s, iCnt, sOrg

 465: sKey = ""

 466: iCnt = Ubound(Arr,1)

 467:  

 468: For a = 0 to iCnt

 469:  

 470:   s = trim(Arr(a,0))

 471:  

 472:   if s <> "" then

 473:  

 474:     if sKey = "" then

 475:       First CheckSum Value in Array, Set Key, dont check further 

 476:       sKey = s

 477:       sOrg = Arr(a,1)

 478:     else

 479:       CheckSum from previous file set, check if identical

 480:  

 481:       if s = sKey then

 482:         Dupe

 483:         DupeHandling s,Key, Arr(a,1), sOrg

 484:       else

 485:         Set key to Checksum of new file, because it is different

 486:         sKey = s

 487:         sOrg = Arr(a,1)

 488:       end if

 489:  

 490:     end if

 491:  

 492:   end if

 493:  

 494:   if bQuiet = false then

 495:     Set Status Bar Value   

 496:     SetLine2 "Files Processed: " & CStr(a + 1) & " of " & cstr(iCnt + 1)

 497:  

 498:     Check if Abort Button was pressed

 499:  

 500:     If IsQuit() = True Then

 501:       Exit For

 502:       bAbort = true

 503:     End If

 504:  

 505:   end if

 506:  

 507: Next

 508:  

 509: End Sub

 510:  

 511: Sub DupeHandling(MD5dupe, MD5Org, FNameDupe, FNameOrg)

 512: Here is where You decide what to do with the found duplicate

 513: You could for example perform additional checks 

 514: beyond the MD5 Checksum also

 515: Dim sSrc, sOrg, sOrgExt, sOrgBase, sDest, sDestName, sDestExt, sDestBase

 516: Increase Dupe Counter

 517: iDupCnt = iDupCnt + 1

 518:  

 519: Determine the action to take

 520:  

 521: Select Case DupeAction

 522:   Case 1

 523:     Rename Dupe by appending Original File name as prefix with 

 524:     an _ as separator. Also the extension of the original file

 525:     Full Path of Dupe File 

 526:     sSrc    = oFso.BuildPath(sFolderPath,FNameDupe)

 527:     FUll Path of Org File

 528:     sOrg    = oFso.BuildPath(sFolderPath,FNameOrg)

 529:     Get Extension of Org File

 530:     sOrgExt = oFso.GetExtensionName(sOrg)

 531:  

 532:     Get Base File Name of Org File without Extension

 533:     sOrgBase = left(FNameOrg,    InStrRev(FNameOrg, "." & sOrgExt, - 1,1) - 1)

 534:  

 535:     Build New File name/path for Dupe Path\OrgBase_OrgExt_DupeBase.DupeExt

 536:     sDestExt  = oFso.GetExtensionName(FNameDupe)

 537:     sDestBase = left(FNameDupe,    InStrRev(FNameDupe, "." & sDestExt, - 1,1) - 1)

 538:     sDestName = sOrgBase & "_" & sOrgExt & "_" & sDestBase & "[DEDUPED]" & "." & sDestExt

 539:     sDest     = oFso.BuildPath(sFolderPath, sDestName)

 540:  

 541:     Move

 542:  

 543:     if oFso.FileExists(sDest) then

 544:       New File already exist, cannot rename dupe, Increase Dupe Processing Error Count

 545:       iDupErr = iDupErr + 1

 546:       ErrorLogWrite "Rename Failed! Org: " & FNameOrg & ", Dupe Src: , " & _

 547:       FNameDupe & ", Dest: " & sDestName

 548:     Else

 549:       oFso.MoveFile sSrc, sDest

 550:       ErrorLogWrite "Dupe Processed! Org: " & FNameOrg & ", Dupe Src: , " & _

 551:       FNameDupe & ", Dest: " & sDestName

 552:     End if

 553:  

 554:   Case Else

 555:     Not implemented yet

 556: End Select

 557:  

 558: End Sub

 559:  

 560: ==============================================================

 561: Support Funtions

 562: Sub WriteFile(arr)

 563: Write List of Files with their MD5 Sums to a Text file

 564: Dim a loop count

 565:  

 566: Dim f: f = oFso.BuildPath(sFolderPath,FileListFName)

 567: Check if an old Listings File Already Exists and Delete it

 568:  

 569: if oFso.FileExists(f) then

 570:   oFso.DeleteFile f, true

 571: end if

 572:  

 573: Dim oF: Set oF = oFso.OpenTextFile(f, ForAppending, true, - 2)

 574:  

 575: File Name + TAB + MD5 Sum of File

 576:  

 577: For a = 0 to Ubound(arr,1)

 578:   oF.writeline trim(arr(a,1)) & vbtab & trim(arr(a,0))

 579: Next

 580:  

 581: oF.Close

 582: Set oF = Nothing

 583: End Sub

 584:  

 585: Function ErrorLogWrite(sErrLogMsg)

 586:  

 587: Dim bOpenLog: bOpenLog = false

 588: Dim sFullErrMsg

 589:  

 590: if WriteDeDupeLog = 1 then

 591:  

 592:   if not isObject(oLogfile) then

 593:     set oLogfile = nothing

 594:   end if

 595:  

 596:   if not (oLogfile is nothing) then

 597:   else

 598:     bOpenLog = true

 599:   end if

 600:  

 601:   if  bOpenLog = true then

 602:     Set oLogfile = oFSO.OpenTextFile(sLogOutput, ForWriting, True, - 2)

 603:     ErrorLogWrite("----------------------------------------------")

 604:     ErrorLogWrite("New DeDupe Batch Started")

 605:     ErrorLogWrite("Work Path: " & sFolderPath)

 606:     ErrorLogWrite("-----------------------------------------------")

 607:   end if

 608:  

 609:   sFullErrMsg = LogDateFormat(now) & chr(9) & sErrLogMsg

 610:  

 611:   oLogFile.Writeline sFullErrMsg

 612:  

 613: end if

 614:  

 615: end function

 616:  

 617: function LogDateFormat(dSourceDate)

 618: Const sLogDtNumbers = "0000"

 619: Dim sLgDtYYYY, sLgDtMM, sLgDtDD, sLgDtHH, sLgDtNN, sLgDtSS

 620:  

 621: sLgDtYYYY     = right(sLogDtNumbers & year(dSourceDate),4)

 622: sLgDtMM       = right(sLogDtNumbers & month(dSourceDate),2)

 623: sLgDtDD       = right(sLogDtNumbers & day(dSourceDate),2)

 624: sLgDtHH       = right(sLogDtNumbers & hour(dSourceDate),2)

 625: sLgDtNN       = right(sLogDtNumbers & minute(dSourceDate),2)

 626: sLgDtSS       = right(sLogDtNumbers & second(dSourceDate),2)

 627: LogDateFormat = sLgDtYYYY & "-" & sLgDtMM & "-" & sLgDtDD & _

 628: " " & sLgDtHH & ":" & sLgDtNN & ":" & sLgDtSS

 629: End Function

 630:  

 631: =================================================================

 632: Progress Bar Code

 633:  

 634: --------------------------------------------------------

 635:  Function   StartIE

 636:  Abstract   Launch IE Dialog Box and Progress bar

 637:  Parameters Titel of the box

 638: --------------------------------------------------------

 639:  

 640: Private Sub StartIE(strTitel)

 641: Dim objDocument

 642: Dim objWshShell

 643:  

 644: Set objIE        = CreateObject("InternetExplorer.Application")

 645: objIE.height     = 230

 646: objIE.width      = 400

 647: objIE.menubar    = False

 648: objIE.toolbar    = false

 649: objIE.statusbar  = false

 650: objIE.addressbar = false

 651: objIE.resizable  = False

 652: objIE.navigate ("about:blank")

 653:  

 654:  wait till ie is loaded

 655: While (objIE.busy)

 656: wend

 657:  

 658: set objDocument = objIE.document

 659:  setup the dialog box  

 660: WriteHtmlToDialog objDocument, strTitel

 661:  

 662:  with ie/html loaded, define assorted objects...

 663: set objTextLine1       = objIE.document.all("txtMilestone")

 664: set objTextLine2       = objIE.document.all("txtRemarks")

 665: Set objProgressBar     = objIE.document.all("pbText")

 666: set objQuitFlag        = objIE.document.Secret.pubFlag

 667:  

 668: objTextLine1.innerTEXT = ""

 669: objTextLine2.innerTEXT = ""

 670:  

 671:  objIE.document.body.innerHTML = "Building Document..."

 672:  + "<br>load time= " + n

 673: objIE.visible = True

 674:  

 675:  set focus to ie 

 676: Set objWSHShell = WScript.CreateObject("WScript.Shell")

 677: objWshShell.AppActivate("Microsoft Internet Explorer")

 678: End Sub

 679:  

 680: --------------------------------------------------------

 681:  Function CloseIE

 682:  Abstract Close the IE Browser Windows

 683: --------------------------------------------------------

 684:  

 685: Private Function CloseIE()

 686: On Error Resume Next

 687: objIE.quit

 688: End Function

 689:  

 690: --------------------------------------------------------

 691:  Function   SetLine1

 692:  Abstract   Set Text Line in the Progress Bar Dialog Box

 693:  Parameters Progress Text

 694: --------------------------------------------------------

 695:  

 696: Private sub SetLine1(sNewText)

 697: On Error Resume Next

 698: objTextLine1.innerTEXT = sNewText

 699: End Sub

 700:  

 701: --------------------------------------------------------

 702:  Function   SetLine2

 703:  Abstract   Set Text Line in the Progress Bar Dialog Box

 704:  Parameters Progress Text

 705: --------------------------------------------------------

 706:  

 707: Private sub SetLine2(sNewText)

 708: On Error Resume Next

 709: objTextLine2.innerTEXT = sNewText

 710: End Sub

 711:  

 712: --------------------------------------------------------

 713:  Function   IsQuit

 714:  Abstract   Checks if the quit button was pressed

 715:  Parameters Progress Text

 716: --------------------------------------------------------

 717:  

 718: Private function IsQuit()

 719: On Error Resume Next

 720: IsQuit   = False

 721:  

 722: If objQuitFlag.Value = "quit" Then

 723:   IsQuit = True

 724: End If

 725:  

 726: End function

 727:  

 728: --------------------------------------------------------

 729:  Function   WriteHtmlToDialog

 730:  Abstract   Set HTML Text for the IE Dialog box

 731:  Parameters IE Document Object, Title Text

 732: --------------------------------------------------------

 733:  

 734: Private Sub WriteHtmlToDialog(objDocument, strTitel)

 735: objDocument.Open

 736: objDocument.Writeln "<title>" & strTitel & "</title> "

 737: objDocument.Writeln "<style>"

 738: objDocument.Writeln " BODY {background: Silver} BODY { overflow:hidden }"

 739: objDocument.Writeln " P.txtStyle {color: Black; font-family: Arial; " _

 740:  & " font-size: 10pt; font-weight: normal; margin-left: 10px; " _

 741:  & " width: 340px } "

 742: objDocument.Writeln " input.pbStyle {color: Navy; font-family: Wingdings; " _

 743:  & " font-size: 10pt; background: Silver; height: 20px; " _

 744:  & " width: 340px } "

 745: objDocument.Writeln "</style>"

 746: objDocument.Writeln "<div id=""objProgress"" class=""Outer""></div>"

 747:  write out text lines... 

 748: objDocument.Writeln "<P id=txtMilestone class=txtStyle style=margin-left: 10px> </P>"

 749: objDocument.Writeln "<P id=txtRemarks class=txtStyle style=margin-left: 10px ></P>"

 750: objDocument.Writeln "<CENTER>"

 751:  write progbar

 752: objDocument.Writeln "<input type=text id=pbText class=pbStyle value= >"

 753: objDocument.Writeln "<br><br>"  space down a little

 754:  write cancel button...

 755: objDocument.Writeln "<input type=button value=Cancel " _

 756:  & " onclick=SetReturnFlag(""quit"") >"

 757: objDocument.Writeln "</CENTER>"

 758:  write hidden object...

 759: objDocument.Writeln "<form name=secret >" _

 760:  & " <input type=hidden name=pubFlag value=run >" _

 761:  & "</form>"

 762: objDocument.Writeln "<SCRIPT LANGUAGE=VBScript >"

 763:  write "local script" to handle cmdCancel_Click event...

 764: objDocument.Writeln "Sub SetReturnFlag(sFlag)"

 765: objDocument.Writeln " secret.pubFlag.Value = sFlag"

 766: objDocument.Writeln " txtMileStone.style.color = ""Red"" "

 767: objDocument.Writeln " txtRemarks.style.color = ""Red"" "

 768: objDocument.Writeln "End Sub"

 769:  progress bar

 770: objDocument.Writeln "Function PctComplete(nPct)"

 771: objDocument.Writeln "pbText.Value = String(nPct,"" "") & String(4,""n"")"

 772: objDocument.Writeln "End Function"

 773:  calc progress bar and direction

 774: objDocument.Writeln "Sub UpdateProgress()"

 775: objDocument.Writeln "Dim intStep"

 776: objDocument.Writeln "Dim intDirection"

 777: objDocument.Writeln "If (IsNull(objProgress.getAttribute(""Step"")) = True) Then"

 778: objDocument.Writeln "intStep = 0"

 779: objDocument.Writeln "Else"

 780: objDocument.Writeln "intStep = objProgress.Step"

 781: objDocument.Writeln "End If"

 782: objDocument.Writeln "if (IsNull(objProgress.GetAttribute(""Direction""))=True) Then"

 783: objDocument.Writeln "intDirection = 0"

 784: objDocument.Writeln "Else"

 785: objDocument.Writeln "intDirection = objProgress.Direction"

 786: objDocument.Writeln "End If"

 787: objDocument.Writeln "if intDirection=0 then"

 788: objDocument.Writeln "intStep = intStep + 1"

 789: objDocument.Writeln "else"

 790: objDocument.Writeln "intStep = intStep - 1"

 791: objDocument.Writeln "end if"

 792: objDocument.Writeln "Call PctComplete(intStep)"

 793: objDocument.Writeln "if intStep>=23 then"

 794: objDocument.Writeln "intDirection=1"

 795: objDocument.Writeln "end if"

 796: objDocument.Writeln "if intStep<=0 then"

 797: objDocument.Writeln "intDirection=0"

 798: objDocument.Writeln "end if"

 799: objDocument.Writeln "objProgress.SetAttribute ""Step"", intStep"

 800: objDocument.Writeln "objProgress.SetAttribute ""Direction"", intDirection"

 801: objDocument.Writeln "Window.setTimeout GetRef(""UpdateProgress""), " & conBarSpeed

 802: objDocument.Writeln "End Sub"

 803:  timeout function

 804: objDocument.Writeln "Sub DialogHardTimeout()"

 805:   objDocument.Writeln "SetReturnFlag(""quit"")"

 806: objDocument.Writeln "End sub"

 807: objDocument.Writeln "Sub Window_OnLoad()"

 808: objDocument.Writeln "theleft = (screen.availWidth - document.body.clientWidth) / 2"

 809: objDocument.Writeln "thetop = (screen.availHeight - document.body.clientHeight) / 2"

 810: objDocument.Writeln "window.moveTo theleft,thetop"

 811: objDocument.Writeln "Window.setTimeout GetRef(""UpdateProgress""), " & conBarSpeed

 812: objDocument.Writeln "Window.setTimeout GetRef(""DialogHardTimeout""), " & conForcedTimeOut

 813: objDocument.Writeln "End Sub"

 814: objDocument.Writeln "</SCRIPT>"

 815: objDocument.Close

 816: End Sub

 817:  

 818: Sub CleanUpAndQuit(RetCode)

 819: House Cleaning

 820:  

 821: if not isObject(oLogfile) then

 822:   set oLogfile = nothing

 823: end if

 824:  

 825: if not (oLogfile is nothing) then

 826: else

 827:   oLogFile.Close

 828:   set oLogfile = nothing

 829: end if

 830:  

 831: Set oFso = Nothing

 832: WScript.Quit(RetCode)

 833:  

 834: End Sub

 

DeDupeInstall.bat

   1: @echo off

   2: Cls

   3: echo.

   4: echo  Installing De-dupe Shell Extension and Its Support Tools

   5: echo =============================================================

   6: echo.

   7:  

   8: echo   1. Copy md5sum.exe to %SystemRoot%\system32\

   9: copy md5sum.exe %SystemRoot%\system32\md5sum.exe

  10: echo.

  11:  

  12: echo   2. Copy DedupeFilesInFolder.vbs to %SystemRoot%\system32\

  13: copy DedupeFilesInFolder.vbs %SystemRoot%\system32\DedupeFilesInFolder.vbs

  14: echo.

  15:  

  16: echo   3.Register "DeDupe" Directory Shell Extension

  17: regedit /s DedupeInstall.reg

  18: echo.

  19:  

  20: echo   4. done... DeDupe Shell Extension Installed Successfully

  21: echo.

  22: echo.

  23: pause

DeDupeInstall.reg

Note the default registry value in HEX. This was necessary to be able to add the value:
“wscript.exe %SystemRoot%\system32\DedupeFilesInFolder.vbs”  to the registry. Notice the %SystemRoot% environment variable? That needs to be expanded by windows, when you activate the shell extension. If you would create the value as Reg_SZ string, it would not work.

It needs to be an expandable string (because it will be expanded at runtime), REG_Expand_SZ. Expandable string values can only be created by the registry import, if you provide the value of a string in hexadecimal format. This tells Windows to create REG_EXPAND_SZ value instead of the basic REG_SZ.After each character follows the hex value 00 and at the end two more 00 are added as well. Why this is necessary, I have no idea, but this is how you have to do it.

   1: Windows Registry Editor Version 5.00

   2:  

   3: [HKEY_CLASSES_ROOT\Directory\shell\DeDupe]

   4: @="DeDupe"

   5:  

   6: [HKEY_CLASSES_ROOT\Directory\shell\DeDupe\command]

   7: @=hex(2):77,00,73,00,63,00,72,00,69,00,70,00,74,00,2E,00,65,00,78,00,65,00,20,\

   8:   00,25,00,53,00,79,00,73,00,74,00,65,00,6D,00,52,00,6F,00,6F,00,74,00,25,00,\

   9:   5C,00,73,00,79,00,73,00,74,00,65,00,6D,00,33,00,32,00,5C,00,44,00,65,00,64,\

  10:   00,75,00,70,00,65,00,46,00,69,00,6C,00,65,00,73,00,49,00,6E,00,46,00,6F,00,\

  11:   6C,00,64,00,65,00,72,00,2E,00,76,00,62,00,73,00,00,00

DeDupeUnInstall.bat

   1: @echo off

   2: CLS

   3: echo.

   4: echo  Un-Installing De-dupe Shell Extension and Its Support Tools

   5: echo =============================================================

   6: echo.

   7:  

   8: echo   1. Delete md5sum.exe from %SystemRoot%\system32\

   9: if EXIST %SystemRoot%\system32\md5sum.exe del /Q /F "%SystemRoot%\system32\md5sum.exe"

  10: echo.

  11:  

  12: echo   2. Delete DedupeFilesInFolder.vbs from %SystemRoot%\system32\

  13: if EXIST %SystemRoot%\system32\DedupeFilesInFolder.vbs del /Q /F 

  14:          "%SystemRoot%\system32\DedupeFilesInFolder.vbs"

  15: echo.

  16:  

  17: echo   3. Unregister "DeDupe" Directory Shell Extension 

  18: regedit /s DedupeUnInstall.reg

  19: echo.

  20:  

  21: echo   4. done... DeDupe was Uninstalled Successfully

  22: echo.

  23: echo.

  24: pause

DedupeUnInstall.reg

   1: Windows Registry Editor Version 5.00

   2:  

   3: [-HKEY_CLASSES_ROOT\Directory\shell\DeDupe]

   4: @="DeDupe"

This is another tool created by Carsten Cumbrowski aka Roy/SAC and I hope that you will find this one helpful as well. For comments, praise or complaints, please use the comments section of the blog below. Thanks.

Cheers!

Carsten aka Roy/SAC

How to Package a Scene Release?

Scene releases, particular the pirate or warez scene, publishes their releases in a certain way, which is done pretty much consistent and is done so for many years.

Back to ASCII Art Academy

File Format

The final release format is ZIP. This became the de-facto standard sometimes during the early 1990s. Back then programs were usually not that large and provided via floppy disks. Each floppy disk was usually compressed into one PkZip archive. If the program used multiple floppy disks, then the ZIP file names were numbered.

With the arrival of CD-ROM, release sizes increased. Also, a CD-ROM does not fit on a floppy disk and often the individual files on the CD-ROM were also too large to fit on a single floppy disk. ZIP did not offer the capability to break up archives into multiple files, with each file having a pre-set size to make sure that it fits on to the archive medium that you wanted to use (1.44 MB floppy disks for the most part).

To solve this problem, release groups utilized other packers like ARJ, RAR and later also ACE that were capable of splitting and then packed the individual ARJ/RAR/ACE archive files once more with ZIP.

Full CD-ROMS (ISO images) were not released at first, due to limitations in available bandwidth and HD sizes, but even the CD-RIP releases grew more and more in size over time. When a release had 10 disks, X was often used for the 10th disk to save precious characters in the file name, but it wasn't for long that number of disks for releases would be way bigger than 10 disks. 30, 40 and 50+ disks for a single release became more and more the norm than being the exception.

While release sizes grew, also the 3.5" floppy disks started to become more and more of a novelty and release groups talked with each other and agreed to allow the use of 2.88 MB size volumes for each "disk" or file of the release.

It was not for long that 2.88 MB would also not cut it anymore and the limitation of volume or file size was abolished entirely. Releases today are still split into volumes of different sizes (usually between 5 and 100 MB), but for other, practical, reasons. If for example the release would be 1 GB in size and be released as one file and you or the server where you are downloading the release from do not support "resume" in case the transfer gets interrupted, then you would probably get very mad, if the download of that 1 GB file breaks up after 990 MB, especially if you did this on dial-up or slow DSL, where it takes many hours or days to download that much data.

You would have to start the download all over again, even though only 10 MB are missing. There are also still a number of different storage media available and used that might have a smaller capacity than the full size of a release.

The practical reasons diminish more and more, but release group will probably continue splitting up releases into multiple files. This would be more out of tradition than anything else, but hey, if you have a major release, then you don't want to put it out there in one file only. Single file releases are usually only used for cracks/keygens only, trainers, patches, product updates, dox/manuals and small applications/tools. A release spanning multiple files usually indicates a full release of some sort.

File Names

Also in the early 1990s it became practice that the release filename starts with a file prefix to identify the release group that published the release. Old MS DOS file name restrictions caused the prefix used by groups to be no more than the first 2, 3 to max. 4 characters of the file name. If you had a release that spans multiple disks, with the DOS limit of 8 characters for the file name, only 3-5 characters remained to somewhat identify the name of the actual release itself. 

Some example of file name prefixes used by release groups:

cls = Class
gns = Genesis
pdx = Paradox
rzr = Razor 1911
tdu = TDU Jam
trsi = Tristar & Red Sector Inc.
x- = X-Force

NFO and File_ID.DIZ

The ZIP release file (and not the archive inside the ZIP) usually also contains two small text files. I won't say much  about the purpose of the file FILE_ID.DIZ. If you do not know what it is and what it was/is used for, see this older blog post of mine to learn more about it.

The NFO file name in the archive is usually called like the name of the group and in rare cases like the ZIP release file, with the file extension .NFO to indicate that it is an NFO file. NFO stands for Info or Information. MS Dos file name restrictions limited the length of a file extension to max. 3 characters length.

At the top of the NFO file is usually a logo with the name of the release group using ASCII art. The logo is usually made up of "block ASCII" characters in MS DOS, which some editors and text file viewers under MS Windows and other operating systems cannot display properly my default. The users only sees some garbage characters instead and unable to make out what those characters try to represent. For how to look at NFO files under Windows, see my article about the underground text art styles.

The NFO file contains information about the release itself, instructions for how to install it, information about the use and purpose of the software/application/program (something were release groups typically do a poor job or not include anything at all), latest news about the release group, greetings to other groups and people in the scene, a member list that shows who is currently part of the group and who is not and a  list of "sites" or "boards", in the pre-Internet past Bulletin Board Systems (BBS) and then FTP Servers on the Internet. Contact information might or might not be included in the NFO file as well, such as Email, a web address or an Internet Relay Chat (IRC) channel where members of the group can be found.

Cracktros, Installers, KeyGens and Cracks

In many cases are those the only files in the release, but depending on the release itself and the group that released it, other files might be included as well. Software Key Generators (KeyGens) and Cracks.

KeyGens

KeyGens are simple programs to generate valid software registration information that you enter within the software itself to unlock any limitation that the unregistered version of the tool has. Those information are typically provided to users who actually paid for a license of the software. KeyGens provide the same, but without paying anything.

Cracks

Cracks are either a program that modifies the installed program to remove any copy protection, or modified files of the original software that the user has to replace after installing the original program on his machine (with the protection still being intact).

Installers

Installers were especially popular during the time when stripped down versions of CD-ROM releases were published by release groups. Those CD-RIPS did not include any original install or setup program that was part of the original software release. Also, archive tools like RAR, ARJ and ACE are not part of the operating system and many users do not have it installed on their computers. To this day, many users do not know and never heard about WinRAR or WinACE, let alone installed it on their system.

The Installer is a tool written by the release group to extract the split archives to the program files folder and might also adds any registry entries required by the software to run, if it was needed.

The installer interface never uses the standard Windows Installer SDK that you are used to from installing "normal" software. The scene installers are completely custom build, often using a flashy and colorful designed user interface with a logo of the group and more noticeable with background music, which can be surprising, if you never used a scene installer before.

Cracktros

Cracktro stands for Crack-Intro, the introduction for a cracked piece of software. A Cracktro does not serve any practical purpose that is required in order to distribute, install and use the release by the warez release group.

The sole purpose of a Cracktro is to showcase the release group itself. You can find a large number of video captures of crack intros at my YouTube channel "SACReleases".

The only reference to the release in a Cracktro is one or two lines of some sort of text within the Cracktro with the name and maybe credits for the supplier, cracker and/or packager. In some cases the Cracktro is used to transmit special messages and information to other people in the scene, but for the most part a cracktros purpose is purely promotional.

A cool Cracktro can be watched and enjoyed independently from the release itself. The Cracktro concept and idea dates back to the earliest widely used home computers like the Commodore 64, Atari 800 or Apple II.

Not surprisingly, cracktros evolved to larger productions, called demos, which spun off to become a huge scene by itself without having ties to the warez scene anymore. To learn more about the history of the demo scene, check out this article of mine.

Beyond All This

That's pretty much all that you can find in scene release files. Wait, there is more. It is more the exception today than it is the norm, but that used to be the other way around in the past.

Release ZIP archives can contain text files and executables (.EXE or .COM files) that are not related to the released software nor the release group itself. Those extra files in the ZIP are advertisements for Bulletin Board Systems, FTP Servers, Trading Groups or individuals that got in contact with the ZIP file before you downloaded it to your machine. BBS and FTP servers used to add to every file uploaded to them such an advertisement for themselves. By looking at those ads, you could tell, which board was among the best and fastest around, because if an ad for that board was in a release file when you downloaded it, then it meant that it passed through that other board before it got to where you downloaded it from.

Preparing a Scene Release

Preparing a scene release without any special tools or scripts is a pain in the neck. You first have to pack the to be released software itself with RAR (which is typically used today) and create multiple volumes, if the release is larger.

After that, you have to use WinZIP to pack each of the volumes ones more. You also have to include the NFO file and file_ID.diz in each of the ZIP archives as well.

If you have a release that spans more than a handful volumes, this job becomes somewhat tedious and time consuming, even if you are using the command line options of the packers and not their graphical interface.

Many release groups developed internally tools and scripts to prepare their releases. Depending on the sophistication of the group and the needs, release preparation tools can become quite a piece of software by itself, beyond just doing the final packaging of the release, including easy to perform updates of the content of the NFO and File_ID.DIZ files, text changes in the Cracktro and ability to recompile it with those changes included etc.

Well, I don't have any release preparation tools that I could or would give out to anybody, but I do have a script that does make the packaging at the end of the release preparation process very easy and automated.

Release Creation Script

Here is a MS DOS Batch Script called !ProcREL.bat. You can download the full script here (you have to rename the downloaded file from !ProcREL.bat.txt back to !ProcREL.bat) Script fixed and extended on March 13, 2009. See notes at the and of the post, after the code.

You require the tools WinZIP and WinRAR to be able to use this script. In the case of WinZIP do you require to have the separate command line version of the tool (WZZIP.EXE). In case of WinRAR, the command line version RAR.EXE is usually included in the general release version of the tool.

You have to make small changes to the script in any case. You can change some other settings there as well, if you like to, but that would not be necessary to make it work in general.

If you installed WinZIP and WinRAR to their typical default location on your C-Drive, changes to the lines 7 and 8 of the batch script won't be necessary. Line 9 however has to be changed in any case, unless your NFO file has the name NFOFILE.NFO, which I don't think to be the case. If you just provide the name of the file, make sure that the NFO file is located in the same directory where you execute the script itself. You could also provide the full path to the file and the file name, if you want to. Line 10 only needs to be changed, if you want to include a File_ID.DIZ file from a different location than the location where you run the batch script from.

The script only has to parameters which are both required. The first parameter is the base name of the file name for the release itself, without any extension. File numbers will be added by the script to the file name automatically. File numbers are always 2 characters, starting at 01 and supporting up to 99 files max.

The second parameter is the specification of the content that make up the release itself. This can be the name and path to an individual file or using the DOS Wildcards *, ? etc. to include multiple files. If your release contains sub folders that also have to be included, a change to line 27 of the script will be necessary.

Example:

!ProcREL.bat PRE-RELN C:\RELEASE\*.*

The script would create 98 MB RAR volumes with the Name PRE-RELNXX.RAR from all files in the folder C:\RELEASE, where XX stands for 01 to 99, then create a ZIP file PRE-RELNXX.ZIP for each RAR volume where it also includes the files FILE_ID.DIZ and the specified .NFO file  to each of the ZIP archives.

You have to add the switch "-r"  to the call of RAR.EXE to include sub-folders. Also, the current setting is to create volumes of 98 MB in size, which is pretty much the maximum used by release groups today. You can change that size in line 27 as well, just change the number for "-v98078k" to -vXXk where XX stands for the desired volume size in Kilobytes. If you forget the "k" at the end, the number will be interpreted as bytes instead, creating volumes that would be a bit too small I guess.

Here are the steps the script does perform:

1. Calls RAR to create volumes from your input file or files.
   
   
2. RAR automatically creates file with the name BASENAME.partXX.rar. I do not like the ".part"  in the name and prefer BASENAMEXX.RAR instead, where XX is a number between 01 and 99. So I rename all the files created by step 1. Note: The batch script is case sensitive. Make sure that the extension generated by RAR is ".rar" all lower-case. I then rename it to .RAR (all upper-case)
   
   
3. Generate a list with all RAR files in the directory and create a ZIP file with the name BASENAMEXX.RAR.ZIP where the RAR archive is added to. Then I also add the NFO and FILE_ID.Diz to each of the new ZIP Files
   
   
4. Rename the Files BASENAMEXX.RAR.ZIP to BASENAMEXX.ZIP
   
   
5. Clean up (delete) temporary files that were created by the batch script.

   1:  @ECHO OFF

   2:  CLS

   3:   

   4:  REM =================================================

   5:  REM Please Change Name and Path for the Following Variables

   6:  REM =================================================

   7:  SET ZipExe=C:\PROGRA~1\WinZip\WZZIP.EXE

   8:  SET RARExe=C:\PROGRA~1\WinRAR\RAR.EXE

   9:  SET NFOFile=RoORS.NFO

  10:  SET FILEID=File_ID.DIZ

  11:   

  12:  REM =================================================

  13:   

  14:  IF NOT EXIST %NFOFile% goto NONFO

  15:  IF NOT EXIST %FILEID% goto NODIZ

  16:  IF "%1"=="" goto USAGEINFO

  17:  IF "%2"=="" goto USAGEINFO

  18:  set /a zipf=0

  19:  IF "%4"=="ABORT" (

  20:    FOR %%Z in (%1??.ZIP) DO (

  21:      GOTO ZIPSFOUND

  22:    )

  23:  ) ELSE (

  24:    Del /Q "%1??.zip"

  25:    Echo Delete "%1??.zip"

  26:  )

  27:   

  28:  Echo Delete "%1.part??.rar" (they should not be there though)

  29:  DEL /Q "%1.part??.rar"

  30:   

  31:  REM =================================================

  32:  REM use "rn" instead of "a" to move files into the archive

  33:  REM -v98078k creates volumes of a bit less than 100 MB 

  34:  REM to fit on a ZIP-100 disk

  35:  REM -m5 sets the compression level. 5 is maximum, 

  36:  REM you can also set it to 0,1 .. 4 which is faster

  37:   

  38:  Echo Create RAR Archives %1.partXX.RAR for Data Selection %2

  39:  %RARExe% a -v98078k -m5 "%1" "%2"

  40:   

  41:   

  42:  REM =================================================

  43:  Echo Rename %1.partXX.RAR to %1xx.RAR

  44:  set /a pos=0

  45:  For /f %%x in ('dir /on /b "%1.part*.rar"') do (

  46:   set /a pos+=1

  47:  )

  48:  if %pos%==0 goto NORAR

  49:  For /L %%n in (1,1,%pos%) do (

  50:    if %%n leq 9 (call :RenRAR %1 0%%n ) else (call :RenRAR %1 %%n )

  51:  )

  52:   

  53:  Echo.

  54:  Echo =================================================

  55:  Echo %pos% RAR volumes were created!

  56:  Echo Please make now the necessary adjustments to your

  57:  Echo NFO file and File_ID.DIZ files,

  58:  Echo When you are done with it and ready to create the 

  59:  Echo ZIP release files....

  60:  echo.

  61:  pause

  62:   

  63:   

  64:  REM =================================================

  65:  Echo Build List of RAR Archives in Folder...

  66:  dir /B /A:-D *.RAR >!RARFiles.txt

  67:   

  68:  REM =================================================

  69:  Echo Process RAR Archives...

  70:  FOR /F "delims=|" %%i IN (!RARFiles.txt) DO (

  71:     Echo Processing %%i

  72:     IF EXIST "%%i.ZIP" del "%%i.ZIP"

  73:     %ZipExe% -a "%%i.ZIP" "%%i"

  74:     IF EXIST "%NFOFile%" %ZipExe% -a "%%i.ZIP" "%NFOFile%"

  75:     IF EXIST "%FILEID%" %ZipExe% -a "%%i.ZIP" "%FILEID%"

  76:     IF NOT "%3" == "KEEPRAR" (DEL /Q "%%i")

  77:  )

  78:  REM =================================================

  79:  Echo Build List of TEMP ZIP Files...

  80:  Dir /b *.RAR.ZIP>!ZIPfiles.txt

  81:  REM =================================================

  82:  Echo Rename ZIP Files...

  83:  REM =================================================

  84:  FOR /F "delims=." %%i IN (!ZIPfiles.txt) DO (

  85:     Echo Rename %%i.RAR.ZIP to %%i.ZIP

  86:     IF EXIST "%%i.ZIP" del "%%i.ZIP"

  87:     REN "%%i.RAR.ZIP" "%%i.ZIP"

  88:  )

  89:  REM =================================================

  90:  Echo Deleting TEMP Files...

  91:  IF EXIST !ZIPfiles.txt Del /Q !ZIPfiles.txt

  92:  IF EXIST !RARfiles.txt Del /Q !RARfiles.txt

  93:   

  94:  ECHO =================================================

  95:  ECHO Done!

  96:  ECHO =================================================

  97:  Echo.

  98:  Pause

  99:  goto END

 100:   

 101:  REM =================================================

 102:  :USAGEINFO

 103:  Echo.

 104:  Echo Usage:

 105:  Echo !ProcRel.bat BASEFILENAME InputData (KEEPRAR/DELRAR) (ABORT/OVERWRITE)

 106:  Echo.

 107:  Echo The parameter pairs #3: KEEPRAR / DELRAR and #4: ABORT/OVERWRITE are optional.

 108:  Echo If you want to toggle parameter #4, parameter #3 must be specified as well 

 109:  Echo The default behavior is DELRAR, which means that the RAR archives will be deleted, 

 110:  Echo once they were copied into the release ZIP archive file and OVERWRITE, which means that

 111:  Echo the script deletes any ZIP files that exist in the directory with the same name 

 112:  Echo.

 113:  Echo If you want to keep the copy of the RAR archive in addition to the release ZIP's or if you 

 114:  Echo want the script to abort, if it detects an existing ZIP file with the same name 

 115:  Echo as the script intends to create then you have to set the parameters,

 116:  Echo.

 117:  Echo They are also case sensitive! If they are misspelled, the default behavior is triggered.

 118:  Echo Example:

 119:  Echo !ProcRel.bat X-RELN C:\RELN\*.*

 120:  Echo.

 121:  Echo !ProcRel.bat X-RELN C:\RELN\*.* KEEPRAR ABORT 

 122:  goto END

 123:   

 124:  REM =================================================

 125:  :NORAR

 126:  echo.

 127:  Echo Error!

 128:  echo Error! No RAR Archives with the file names

 129:  echo %1.partXX.RAR

 130:  echo were Created!

 131:  goto END

 132:   

 133:  REM =================================================

 134:  :NONFO

 135:  echo.

 136:  Echo Error!

 137:  echo NFO-File %NFOFile% not found!

 138:  goto END

 139:   

 140:  REM =================================================

 141:  :NODIZ

 142:  echo.

 143:  Echo Error!

 144:  echo File_ID.DIZ file at %FILEID% not found!

 145:  goto END

 146:   

 147:  REM =================================================

 148:  :ZIPSFOUND

 149:  echo.

 150:  Echo Error! (ABORT Parameter specified)

 151:  Echo Current directory contains already one or more ZIP files 

 152:  Echo with the name: %1??.ZIP 

 153:  goto END

 154:   

 155:  REM =================================================

 156:  :RenRAR

 157:   IF EXIST "%1.part%2.rar" ren "%1.part%2.rar" "%1%2.RAR"

 158:   echo ren "%1.part%2.rar" "%1%2.RAR"

 159:   

 160:  REM =================================================

 161:  :END

 162:  REM Finished!

 163:  Echo.

 164:  Goto :eof

 165:   

I hope that you find this little BATCH script useful.

NOTE March 13, 2009:

I found a bug in my script, which I fixed in my blog post and in the version for download. While I was fixing the bug I also found another small issue and also thought about some options to give you control over a few behaviors of the batch. 

The first addition is the PAUSE of the script after it created the RAR archive volumes. It will tell you how many RAR files were created that you can make the necessary modifications to your prepared File_ID.diz and NFO file.

Once you did that, simply press enter in the DOS window and the script will continue.

I also added two optional parameters to the batch. The first one lets you specify that you would like to keep the copy of the RAR archive volumes in addition to the ZIP files where the volumes are included, instead of deleting them once zipped. The second parameter lets you specify what you want the batch to do, if it detects ZIP archives with the same file name as the script intends to create. The default is delete and re-create, but if you prefer that the script aborts, you can now specifty it via a command parameter.

Back to ASCII Art Academy

Cheers!

Carsten aka Roy/SAC

Roy/SAC Homepage

Convert Video to ASCII Text Art

Uh, third post in 2 days... that surprises even me. Hehe

It was by coincidence actually, but somebody at Mahalo Answers made me aware of the feature of the free VLC (VideoLan) video player to convert video images to color ASCII art (some would call this ANSI hehe) in real-time.

I thought that this is a cool feature, just by hearing about it. I downloaded the player and was unable to find the option for the ASCII art output, so I asked the guy at Mahalo Answers about it and luckily for me, he answered this question as well (a bit delayed, which made me drop the ball on it as well).

The feature is buried deeply within the configuration options of the player and not that easy to find.

To help you with this problem, see my step by step guide to enable the feature in the VideoLan player. Every step is illustrated with a screen shot, so that there should not be any excuses for why you could not set it up yourself properly. Start with getting the latest version of the VideoLan video player at VideoLan.org (note: the latest version does not have this feature anymore. See note at the end of the post for details). The player is available across platform, over a dozen different Linux and Unix distributions and versions. Binaries for Windows, Macintosh Mac OS X, BeOS and Syllable are also available by the way. It's released under the GNU License.

Step by Step Guide to Enable the Color ASCII Art Output Module Configuration in the VLC Video Player

1. Start the VLC/VideoLan media player and open any supported video file.

2. Click on "Settings" in the top menu navigation and select "Preferences"

3. Audio is selected by default at the left. Check out the bottom-right where you can find a check box labeled "Advanced Options", which should be unchecked (default). Check that box to enable the Advanced Options.

4. Back to the navigation to the left. Click on the plus sign next to "Video" in the preferences box to open the configuration tree node for the "Video" options.

5. Click on the "Output Modules" node

6. Change the "default" in the drop down options for video output module to "Color ASCII Art Video output"

7. Click "Save" in the lower left of the window

8. Stop the video that you opened, if it isn't stopped already or if you stopped it after step 1.

9. Press "Play" to re-start the video again with the new video output module

Here are some results of me playing with that option for some videos of mine to give you an idea of how the real-time converted video images look in text mode with colors. I am not sure yet, if the module sticks to the 16 pre-set colors defined for PC MS DOS ANSI, which would be really cool. It may uses other colors instead. I have to do some more tests to find that out.

 
Playing Around with the VLC Color ANSI Art Video Output Module from Carsten Cumbrowski on Vimeo.

You can download this video in AVI format at Mediafile.com.

The player itself is worth checking out in general, beyond the ASCII output feature, which is kind of a "hidden treasure" within the features set of this open source video player. It supports virtually all major video formats, including Mpeg, AVI, QuickTime MOV/MP4, Windows Media WMV/ASF, Macromedia/Adobe Flash (FLV), the open source Matroska, Real Media (RealPlayer), OGG, FLAC and even Midi and raw DV video, WAV, MP3 audio etc.

It can play back streaming video UDP, TCP and DCCP/RTP Unicast or Multicast, HTTP/FTP and MMS in addition to traditional DVD video and audio, Audio CD, Video CD and SVCD, DVB from Satellite, Digital TV and cable television. A lot of stuff for NO BUCK at all. You cannot beat free, or can you?!

Also check out my previous posts about ASCII and ANSI Text Animation:

• ASCII plus Animation is ASCIImation
• What is ANSI Animation or ANSImation?
• Beck ASCIImation and a Walking Man 7-Bit ASCII
• my ASCIImations and ANSI Animations Video Play List at YouTube
• Review of Image to Text (ASCII) Converters

Cheers!

Carsten aka Roy/SAC

Update: I just learned that the current version of VLC (Version 0.9.8a, 16 MB) does not have the ASCII output feature anymore. I have VLC Player Version 0.8.6f Janus (wxWidget interface) from last year, where the feature is still there. You can download this older version here from my web site: vlc-0.8.6f-win32.zip (9.19 MB). I don't know why they would remove such a nice feature from their player. Mhhh.

File Archiving - Collection Management and Organization

I am a collector! More correct would be to say that I hate to throw stuff and that I try to accumulate any data I can get my hands on about any subject of great interest to me.

This must be a trait that I inherited from my dad, who drives my mom nuts with his collector habits.

My Dad's Collecting Habit

The problem that my dad has is the fact that he collects only physical stuff, such as Coins (legal tender coins only), Stamps (West German and American only), Post cards from cities and towns with crest of that town on it (a shi*tload of towns in Europe have their own crest and are only happy to print them on post cards of their city, much to the dismay of my mother), post cards with only a single picture, which must be a true aerial photograph (no picture from a tall building, mountain or anything like that), Locomotives and box cars in model format, manufactured by the former East German company "TT-Eisenbahnen" (shut down a couple years after the wall fell, but still alive and supported by fans and model train owners).

I think that is it... I hope that I did not miss any. As you can see, always very specific subjects, but then more than one subject, which created a slight inventory problem for my parents, which was only resolved when my sister and me moved out into our own places as soon as possible (I moved out with 19, but my parents paid for the basic rent hehe).

My Own Collection Habits from the Past

I used to collect stamps as a kid (only "Space" and "Astronomy" motives) and for a brief period of time "crown caps" where the product or company name was printed on the metal cap. This was rarely done in East Germany, but typical in the Western countries. I abandoned this collection around the time when the wall came down. Go figure. The only physical thing that I collected since 1982 until today are the print issues of the oldest and most popular (of only 3 or 4 or so) East German comic magazine called "Mosaik", which was published monthly since December 1955.

The other physical stuff is not really collecting, because if there is not much to collect, then I don't call it a real collection, even if I have everything (or most) of it. Into this category would be my Cirque du Soleil DVDs and CDs and my VNV Nation stuff and support as a fan of the band.

Digital Collections

However, my collector habit manifested itself in its full extend on the computer where the stuff that you collect is purely digital in nature and thus only uses very little storage space (compared to physical collections). It started when I did the Warez stuff and even more so, when I ran my own BBS. I automatically came across (and got a copy) the most recent software that mattered in those days (worthless junk remains worthless junk and is not worth stashing  up anywhere, even if it is digital junk). I never threw away or deleted any software that I had a copy (or original of), even if the software was too old for "trading" in Warez boards and I also never used the software myself ever. I had it, that was all that mattered.

My software collection was unfortunately destroyed by the German Police, who took away any data storage media that was not an original and they could find during the raid of my apartment (because of my BBS "Closed Society") back in spring 1997. That was the part that hurt me the most... they destroyed everything... it was not just stored somewhere else to still exist as a whole, but inaccessible to me.

Anyhow, things changed with the Internet. A lot of old and forgotten stuff by mainstream culture can now be found online. Not only software like games, but other things that were created as well. I was interested in computer art for example, specifically ANSI and ASCII text art and pixel art. I created stuff myself. I was also interested in the demoscene, that produced and still produces tons of new demos, created by folks like me for the pure sake of creating them and showing them off.

The copyright situation for most of this old stuff is unclear, not specified or simply not enforced by anybody who could make claims of the intellectual property created just for fun, without commercial background. In other cases is the content still officially "copyright protected", but the owner is long out of business or lost any interest in his own property, because he thinks that it does not have any commercial value anymore. I wrote about those things in the past already.

I have a bunch of collections of various kinds. Art, Pixel Art Fonts, Pixel Art Logos, Music, Software.... or using a  simple word that by the end of the day categorizes them all... ...Files.

I am not as bad as Jason Scott from textfiles.com who's collecting habits and urges dwarf mine in comparison.

Keeping Stuff Organized

Organizing these vast amount of items, which are in most cases ridiculous small in size for today's standards, is a pain in the neck and time consuming. I frequently stumble across collections created by others to the same subject that is of interest to me and sorting stuff out, is usually not that easy. Most of those collections are simply a dump of hundreds or thousands of files, each item packed with ZIP, RAR, LHA etc. and then packed together ones more to a multi-part RAR archive or ISO DVD/CD image.

The stuff I usually do with each of those "dumps" is the same, so I wrote myself some small scripts to make things easier for me and to speed up the archiving and sorting process.

The collection that I download are usually just dumps of many files all lumped into a single directory ones you unpacked the RAR archive or ISO image. I always break those up and sort them into individual sub directories by the first letter of the title of each item. The file names are in almost every case the title of the software, demo production, image, mod file, PDF document, text file etc. So I always have to create 27 directories (A-Z plus one for all titles that start with a number, titled 0-9) and then move the files to their respected directories.

An index of all the files is usually also missing to my script is generating one for me. A clean and simple index, which you could not achieve without manually tweaking it, with the standard DIR command in MS DOS.

My Free Helper Scripts

You can download the source scripts packed into a single ZIP archive: !archive_scripts_src_roysac.zip

This batch file creates sub directories "0-9", "A", "B" ... "Z", If they do not already exist (script: !folders.vbs) and then moves all files in the current directory to their corresponding sub folders (part of batch: !organize.bat). After the files were moved, an index file is created (script "!dirlist.vbs") with the name "!" + the current folder name + ".txt". The index file lists all files in the previously created directories. It ignores any other sub folder that might exists in the same dir.

All this is done by three scripts. You only have to run the main one. It calls the other two when needed. The script does not have or require any parameters or configuration options. Just copy the three files into the folder where you dumped all the files and run "!organize.bat".

!organize.bat Script

@echo off

REM      !organize.bat 

REM      Batch File by Carsten Cumbrowski, March 2009

REM       -----------------------------------------------------------------------------------

REM      Required support files used by this script:

REM       - !folders.vbs

REM       - !dirlist.vbs

REM      PURPOSE

REM       -----------------------------------------------------------------------------------

REM      This batch file creates sub directories "0-9", "A", "B" ... "Z"

REM      If they do not already exist and then copies all files in the 

REM      current directory to their corresponding sub folders

REM      (Yeah, that's why all of the script files start with a "!" in the file name)

REM      After the files were moved, an index file is created

REM      with the name "!" + the current folder name + ".txt"

REM      The index file lists all files in the previously created directories

REM      Note: the script ignores other sub directories that might

REM      exist in the current folder, also files that don't start with

REM      0-9 or A-Z (a-z) in the name are ignored

REM       -----------------------------------------------------------------------------------

REM      Copyright: NONE, Use it! Improve it! Share it!

cls

REM -----------------------------------------------------------------------------

REM Create Sub Directories 0-9, A, B ... Z If they not exist

REM -----------------------------------------------------------------------------

echo Creating Sub Directories 0-9, A, B ... Z ...

wscript !folders.vbs

REM -------------------------------------------------------------------

REM Move files into their respective sub directories

REM -------------------------------------------------------------------

echo Moving Files to Sub Directories ...

move 0*.* 0-9\

move 1*.* 0-9\

move 2*.* 0-9\

move 3*.* 0-9\

move 4*.* 0-9\

move 5*.* 0-9\

move 6*.* 0-9\

move 7*.* 0-9\

move 8*.* 0-9\

move 9*.* 0-9\

move a*.* a\

move b*.* b\

move c*.* c\

move d*.* d\

move e*.* e\

move f*.* f\

move g*.* g\

move h*.* h\

move i*.* i\

move j*.* j\

move k*.* k\

move l*.* l\

move m*.* m\

move n*.* n\

move o*.* o\

move p*.* p\

move q*.* q\

move r*.* r\

move s*.* s\

move t*.* t\

move u*.* u\

move v*.* v\

move w*.* w\

move x*.* x\

move y*.* y\

move z*.* z\

REM -------------------------------------------------------------------

REM Create !<FolderName>.txt Index File 

REM -------------------------------------------------------------------

echo Create !<FolderName>.txt Index File ...

wscript !dirlist.vbs

REM -------------------------------------------------------------------

REM Done!

REM -------------------------------------------------------------------

echo done! 

echo.

pause

!folders.vbs Script

'---------------------------------------------------------------

'Script Name: !folders.vbs

'---------------------------------------------------------------

'VBScript by Carsten Cumbrowski, written in March 2009

'---------------------------------------------------------------

'Purpose: Creates Sub Folders  "0-9", "A","B" ... "Z" 

'         in the current folder, if they do not exist already

'         The Script was created for the use with collection

'         of scripts and batch files for archiving and fileing

'         of software and file collections.

'---------------------------------------------------------------

'Parameters: No Parameters Required or Supported

'            Simply run the script from the DOS command prompt

'            in Windows by typing "wscript !folders.vbs" 

'---------------------------------------------------------------

'Copyright: NONE, F**k that shit! Use it! Improve it! Share it!

'---------------------------------------------------------------

Dim oFso, a

Set oFso = CreateObject("Scripting.FileSystemObject")

if NOT oFso.FolderExists("0-9") then

  oFso.CreateFolder("0-9")

end if

For a = 65 to 90

  if NOT oFso.FolderExists(chr(a)) then

    oFso.CreateFolder(chr(a))

  end if

Next

Set oFso = Nothing

!dirlist.vbs Script

'---------------------------------------------------------------

'Script Name: !dirlist.vbs

'---------------------------------------------------------------

'VBScript by Carsten Cumbrowski, written in March 2009

'---------------------------------------------------------------

'Purpose: Generates a .txt file with the name "!" + name of 

'         current folder. The text file contains a list of all

'         files in the sub directories with the names "0-9",

'         "A","B" ... "Z" and totals for the number of files

'         and size in bytes of each sub folder and in total.

'         The Script was created for the use with collection

'         of scripts and batch files for archiving and fileing

'         of software and file collections.

'---------------------------------------------------------------

'Parameters: No Parameters Required or Supported

'            Simply run the script from the DOS command prompt

'            in Windows by typing "wscript !dirlist.vbs" 

'---------------------------------------------------------------

'Copyright: NONE, F**k that shit! Use it! Improve it! Share it!

'---------------------------------------------------------------

'Declaration of Constants

Const ForAppending = 8

Const SeparatorLength = 70

'Declaration and Initialization of Work Variables and Objects at Runtime

Dim oFso: Set oFso= Wscript.createobject("scripting.fileSystemObject")

Dim sFolderPath: sFolderPath = oFso.getAbsolutePathName("")

Dim oFolder: Set oFolder = oFso.GetFolder(sFolderPath)

Dim sFilePath: sFilePath = oFso.GetAbsolutePathName("!" & oFolder.Name & ".txt")

'Declaration of Un-Initialized Work Variables

Dim oSubFolder, aFiles

'Declaration of Work Variables with Initial Values

Dim iFolderFileCount: iFolderFileCount = 0

Dim iFolderByteCount: iFolderByteCount = 0

Dim iFileCount: iFileCount = 0

Dim iByteCount: iByteCount = 0

Dim sFileList: sFileList = ""

'Check if an old Listings File Already Exists and Delete it

if oFso.FileExists(sFilePath) then

   oFso.DeleteFile sFilePath, true

end if

Dim oFile: Set oFile = oFso.OpenTextFile(sFilePath, ForAppending, true, -2)

oFile.WriteLine ""

oFile.WriteLine "Content of: " & oFolder.Name

oFile.WriteLine replace(space(SeparatorLength)," ","=")

oFile.WriteLine ""

'Process Sub Folders of Current Work Folder

For each oSubFolder in oFolder.SubFolders

   if oSubFolder.Name = "0-9" or _

      ( len(oSubFolder.Name) = 1 and asc(left(oSubFolder.Name,1))>= 65 and _

      asc(left(oSubFolder.Name,1))<= 90 ) then

   'Only process Sub Folders "0-9", "A","B" ..."Z" and ignore the rest

    Set oFolderFiles = oSubFolder.Files

      'Get Files in Sub Folder and Build List

    For each oFolderFile in oFolderFiles

         'oFso.GetFileName

       sFileList = sFileList & oFolderFile.Name & vbcrlf

         iFolderFileCount  = iFolderFileCount + 1

         iFolderByteCount = iFolderByteCount + oFolderFile.Size

       iFileCount = iFileCount + 1

         iByteCount = iByteCount + oFolderFile.Size

    Next

     oFile.WriteLine replace(space(SeparatorLength)," ","-")

     oFile.WriteLine "Folder: " &  oSubFolder.Name

     oFile.WriteLine replace(space(SeparatorLength)," ","-")

     oFile.WriteLine "Total: " &  formatnumber(iFolderFileCount,0) & " files"

     oFile.WriteLine "Total Size: " &  formatnumber(iFolderByteCount,0) & " bytes" & _

                     " (" & formatnumber(iFolderByteCount/1024,0) & " KB / " & _

                     formatnumber(iFolderByteCount/1024000,0) & " MB)" 

     oFile.WriteLine replace(space(SeparatorLength)," ","-")

     oFile.Write sFileList

     oFile.WriteLine ""

     sFileList = ""

     iFolderFileCount = 0

     iFolderByteCount = 0

   end if

Next

oFile.WriteLine replace(space(SeparatorLength)," ","=")

oFile.WriteLine oFolder.Name

oFile.WriteLine "Total: " &  formatnumber(iFileCount,0) & " files"

oFile.WriteLine "Total Size: " &  formatnumber(iByteCount,0) & " bytes" & _

                " (" & formatnumber(iByteCount/1024,0) & " KB / " & _

                formatnumber(iByteCount/1024000,0) & " MB)" 

oFile.WriteLine ""

oFile.Close

Set oFile = Nothing

Set oFolder = Nothing

Set oFso = Nothing

I hope that you might also find use for those little but helpful scripts. If not, I hope that you enjoyed my little personal trick back in time about my collecting habits. If you are a collector like me, then you can probably relate to those things without any problems. If you are not, then you might got a nice laugh out of it instead. hehe.

Cheers!

Carsten aka Roy/SAC 

Convert Word 2007 Documents and Upload to Google Docs

It started with a script by Michael Suodenjoki called "docx2pdf.vbs" that lets you convert individual Word 2007 documents to PDF. It is possible to use it to process a batch of files, such as all documents in a specific directory, by using (programming) another sophisticated MS DOS batch script, but that is a pain in the neck itself. I ran into issues, because I had spaces in the file names of the documents that I wanted to convert and I thought to myself that it is probably easier and safer to change the VBScript itself instead of writing another script in a different language.

By the way, for the conversion to PDF format with Microsoft Word 2007 is it necessary that you installed next to Microsoft Office 2007 of course, also the "Microsoft Save as PDF" Add-In for 2007 Office, which is available for free download at the Microsoft.com web site here.

Extending the Docx2Pdf Conversion Script

So I changed the script to allow the processing of a whole folder instead of just one file. While I was doing that, I realized that I need  a filter to make sure that I only process  documents that Word 2007 can load. While I was thinking about a solution about that, I realized that a)  Word 2007 supports more than just Word 2007 .DOCX documents and b) that the Google Docs API only supports Word 97 .DOC documents and not PDF, like the manual uploader in their web interface. Converting my Word 2007 documents to be able to get them up on Google Docs was actually the cause to get me started with all this.

Google Docs Limitations

The Google Docs web uploader supports Adobe PDF and Word 97 DOC format for documents, but you have to upload documents one by one manually. That sucks big time and I hope that Google will do something about that soon, but in the mean time there is not much we can do about it. In order to get several documents up on Google Docs quickly, you have to use their API or the create document via Email or Email with Word 97 document attachment features. Adobe PDF is supposed to work with their GData API for Google Docs, but I never got those Python scripts to work. I was able to connect to Google, but not to transfer... anyway.

Word 2007 supports the output of documents in numerous formats, including Word 97, HTML, RTS, TXT and PDF, if you installed the free Microsoft PDF Add-on. So I added the option to specify the output format.

Then I was thinking of automation of uploads to Google Docs and realized that I needed an additional filter to avoid that I will ending up sending the same documents to Google over and over again. Thus I added some date filter options based on the creation date and date when the document was last modified.

Last but not least I had to add the option to send the converted documents (that were converted to .DOC format) to Google Docs. The upload via email was the way to go and my script was complete. Here is what came out of all of this.

Save Word 2007 Documents in a different Format and more

Usage: cscript /nologo doc2pdf.vbs [<input-file>] [/dir:<input-dir>] 
[/date:created/modified] [/datefrom:mm-dd-yyyyy] [/dateto:mm-dd-yyyy] [/days:nn]
[/f:<output-format>] [/o:<output-file>] [/googledocs:<email>] 

Basic Options:

/help  
Specifies that this usage/help information should be displayed.

/verbose
Specifies that detailed information about the processing should be displayed.

Parameters (optional):

/dir:<input-dir>
Directory Batch Processing. Use <input-file> argument as filter option (e.g. *.DOCX or *.* for all)

Date Filter Options

/date:created/modified
Date filter option (only for batch processing). Specify if the date parameters should apply to either the
creation date of the file or the date when it was last modified.

/datefrom:mm-dd-yyy
/datetto:mm-dd-yyy
From and To Date options for that work together with the /date option. If /dateto is not provided, then the script will automatically assume today. Alternative to a specific date range is it possible to provide the following parameter

/days:nn
"nn" stands for a numeric value from 0 to whatever and are the number of days from today, to determine the from date. The value 1 for example would mean yesterday, or files that were created/changed since yesterday until today. 7 would be a week and so on.

Output Options Parameters

/f:<format>
Specifies the output format values: DOCX, DOCM, DOTX, DOTM, XML, PDF (default), XPS, TXT, DOS, ASC, HTML, HTM, RTS, DOC, WEB

/o:<file>
Optionally specification of output file, which only works for single file conversions and not batch processing.

Send to Google Docs via Email Option

/googledocs:<email>

Your personal email to upload documents, which looks like: longstring@prod.writely.com
You can find that email, if you login to your Google Docs account and go to the "Upload" screen.

Note: Google Docs only supports the import of documents in DOC file format (Office 97).
PDF files are not supported. The file size is also limited to 500KB per document.

Examples:

1. Convert all Word 2007 documents (.docx) that were modified on or since December 31, 2008 and are located in C:\DOCS to PDF and show detailed information about the processing.

cscript /nologo docx2pdf2.vbs *.docx /f:PDF /dir:C:\DOCS /date:modified /datefrom:12-31-2008 /verbose

2. Convert all Word 2007 documents (.docx) that where created in the last 7 days and are located in
C:\MY DOCUMENTS to Word 97 (.DOC) and send them to Google Docs.

cscript /nologo docx2pdf2.vbs *.docx /f:DOC /dir:"C:\MY DOCUMENTS" /date:created /days:7 
 /googledocs:mysecretemail@prod.writely.com 

3. Convert a single Word 2007 Document C:\DOCS\mydocument.docx to PDF and save it as
C:\MY DOCUMENTS\mynewdocument.pdf

cscript /nologo docx2pdf2.vbs "C:\DOCS\mydocument.docx" /o:"C:\MY DOCUMENTS\mynewdocument.pdf" 

4. Convert C:\DOCS\mydocument.docx to HTML and save it in the same folder

cscript /nologo docx2pdf2.vbs "C:\DOCS\mydocument.docx" /f:HTML 

Notes and Considerations

The script must be called with CSCRIPT.EXE instead of WSCRIPT.EXE, because it makes use of the STDOUT option of VBSCRIPT. WSCRIPT.EXE is used by default, if you start a .VBS Visual Script document (e.g. double click or call from a .BAT file).  Its not the end of the world, if you dont, because the script will re-launch itself 
using CSCRIPT, if you execute it with WSCRIPT anyway. However that might cause other  issues, depending how you are using this script.  For example the redirection of the output into a text file wont work, because 
the script opens a new DOS session. I used the proper call of the script in my examples above. The "/nologo" parameter is not required by the way. I only use it to suppress the Microsoft messages and screens.

There are different ways to send out emails via a script that runs on a users computer, each with their own problems and advantages. I implemented two options into my script. By default is Microsofts CDO used to send out the emails to Google Docs, but I also added the option to use Outlook MAPI instead.

Using MAPI instead of CDO

Pretty near the beginning of the script is a line that reads:

Const EmailSendMethod = 2 1 = MAPI (Outlook), 2 = CDO

To use MAPI instead of CDO, simply change this line to

Const EmailSendMethod = 1 1 = MAPI (Outlook), 2 = CDO

I wanted to go with MAPI first, but the biggest problems that I had were those stupid security messages that popped up every time that I wanted to send one email to Google. Since I do not want to sit in front of the computer to acknowledge dozens or more emails/documents to be sent, I changed the script to use CDO instead.

Here is what I got when I used MAPI. First was this message, which made perfect sense to me, although I missed the option to say "always allow access for this application" that it does not come up in the future anymore.

I know that from Skype and it works perfect there, but nothing like it in Microsoft Outlook.

 

Anyhow, I set the option to "Allow access for 10 minutes", which should have been enough to process a bunch of documents and send them out to Google, but it wasnt to be like that. Instead I got a pop-up every time I created a new Email object. The Message window has the "Allow" button disabled for several seconds, before you can approve the email to be created and nothing is done until you pressed "Allow", over and over again.

If you know how I can tell Outlook 2007 to trust my script that resides locally on the users machine and thus be able to use MAPI to generate email messages automatically, please let me know. I know that you can get something like a "Trusted Publisher Certificate", but hey, I am not a software company who is selling a commercial plug-in to people, not even a compiled program at all. Its plain and pure code that is executed and processed by Windows on the fly, without the need to compile it first.

CDO Requirements and Troubleshooting

CDO uses the build-in SMTP server feature that comes with Windows XP (except XP Home). It is not installed by default. You can install SMTP by going to "Start\Control Panel\Add or Remove Programs". There select on the left the Tab "Add/Remove Windows Components". Scroll down to "Internet Information Services (IIS)", highlight it and press the "Details" button. There you have to enable "SMTP Service", which will automatically also enable "Common Files" and "World Wide Web Service", if it was not installed already on your computer either. Press "Okay" to close the "Details" screen and then "Next". Follow the steps on the screen. You might require your original Windows Installation Disk, so have it handy before you get started with this.

The script will error out, if you do not have SMTP installed on your machine yet!

The nice thing about it is that you dont even require to have an email client installed and running (or ready to run) in order to send out an email from your machine. That is a draw-back as well, because you will dont have a reference or copy of the emails that you sent out. They wont appear in  the "Sent" folder of MS Outlook, Outlook Express, Live Mail or whatever email client you might be using.

If you had SMTP installed on your computer already, but not running (like I did), then the script will run without any errors. It will also say that it sent out the emails to Google, but that is not entirely correct. It created it, yes, but sending them is an entirely different story. If the SMTP service is stopped, no email is being sent. Windows dumps the prepared emails into the "Pickup" folder of "IIS", which is located by default at "C:\Inetpub\mailroot\Pickup", unless you configured it to be somewhere else.

Make sure that the SMTP Service is running. To check that and to start the service manually, go to your computers "Administrative Tools" and start "Services" (Administrative Tools are located in the "Control Panel", like the "Add or Remove Programs" tool). Look for the service with the name "Simple Mail Transfer Protocol (SMTP)". See the image below for illustration.

Double-Click on it and press "Start", if the status of the service is anything else than "Started". If it fails to start, then it could be that services where it depends on are not started as well. You can find out by selecting the "Dependencies" tab. Make sure that the services listed there are running as well.

See on the left. It shows that the "Event Log" and "IIS Admin" services need to run that the SMTP service can be started as well.

Once the SMTP service is running, you are ready to rock and roll. If you ran the script and then found out that the service was not running, emails stuck in the "Pickup" folder will not be processed automatically once SMTP is running. Another email must be triggered that SMTP will pick them up and add them to its processing "Queue" folder for sending and then finally send it out to Google.

 

SMTP Service Security (for CDO)

Having the SMTP Service running will upset some anti-virus or anti-spyware programs, because it can become an issue, if it is not secured, detected by hackers and then abused to relay spam message via your computer to other people on the Internet.

You dont to invite trouble, so go and change the security settings that nobody else can use your mail service, except you.

Open the IIS manager, which is located under "Administrative Tools" (like the "Services" tool). There you will find the item "Default SMTP Virtual Service" entry on the left.

Highlight it, right-click with your mouse and select "Properties". Then select the "Access" tab and click on the "Authentication" button.

It is probably set to "Anonymous Access". Change that to "Integrated Windows Authentication".

Users that need permissions to use the service can be added via the "Security" tab as needed.

This will take of things and keep your SMTP service on your machine secure.

Email Delivery Issues with Google Docs

I use the secret email address for your Google Docs account not only as the TO address of the email, but also as the sender address as well. This has a nice little side effect to it.

Sometimes there are issues with Google Docs and it is unable to process a document that it received via attachment of an email for whatever reason.

If the email delivery fails, a system notification is being sent by the receiving mail server to the sender of the email, your Google Docs upload email address. Since the email upload feature also supports sending a whole text email to be converted to a document, this is what Google Docs is doing with the Error emails that it receives about the original uploads that failed.

If you open up such Error Email document, you will be able to find out, which document (file name) that was sent by the script was not uploaded and processed properly. Just send it again with another batch or upload it manually via the web interface, whatever is easier for you.

 

The Script and the Source Code Download

Here is the source code that you can see how the various features were implemented (and to use it for your own tools and scripts hehe). Dont type it off the screen or copy and paste it into a document, if you want to use my script as it is. You can download the full source of it here as a text file. It is called docx2pdf2.vbs.txt. Rename it to remove the .TXT and you are ready to rock and roll.

Thats it! Have Fun!

Carsten aka Roy/SAC

   1:  

   2:   DOC2PDF2.VBS Microsoft Scripting Host Script (Requires Version 5.6 or newer)

   3:   --------------------------------------------------------------------------------

   4:   Author: Carsten Cumbrowski 

   5:   Created: December 2008 and February 2009

   6:   

   7:   Based on a script by  Michael Suodenjoki

   8:   This script can create a PDF file from a Word document provided youre using

   9:   Word 2007 and have the Office Add-in: Save As PDF installed.

  10:   It can also save the documents in other formats that are supported by MS Word 2007

  11:   The script allows batch processing of multiple files (e.g. a directory) and also the 

  12:   sending via email to Google Docs (output format DOC, Office 97 required) 

  13:  

  14:   Script must be called with CSCRIPT instead of WSCRIPT

  15:   Its not the end of the world, if you dont, because the script will relaunch itself 

  16:   using CSCRIPT, if you execute it with WSCRIPT anyway. However that might cause other

  17:   issues, depending how you are using this script

  18:   For example the redirection of the output into a text file wont work, because 

  19:   the script opens a new DOS session

  20:   

  21:  ForceCscript true

  22:   Constants

  23:   

  24:  Const iGoogleDocsSizeLimit = 5120000

  25:  Const iGoogleDocsReqFormat = 0

  26:   

  27:  Const WdDoNotSaveChanges = 0

  28:  Const EmailSendMethod = 2   1 = MAPI (Outlook), 2 = CDO

  29:   

  30:   see WdSaveFormat enumeration constants: 

  31:   http://msdn2.microsoft.com/en-us/library/bb238158.aspx

  32:  Const wdFormatPDF = 17   PDF format. 

  33:  Const wdFormatXPS = 18   XPS format. 

  34:  Const wdFormatDocument97=0  Word97 Format

  35:  Const wdFormatDocumentDefault = 16 Word Default (.DOCX in Word2007)

  36:   

  37:  Const wdFormatWebArchive = 9 Web Archive

  38:   

  39:  Const wdFormatTemplate = 1 Word 97 Template

  40:   

  41:  Const wdFormatRTF = 6 RTF

  42:   

  43:  Const wdFormatHTML = 8 Std HTML

  44:  Const wdFormatFilteredHTML = 10 Filtered HTML

  45:   

  46:  Const wdFormatText = 2 Windows Text

  47:  Const wdFormatTextLineBreaks = 3 Windows Text with line-breaks preserved

  48:  Const wdFormatEncodedText = 7 Encoded Text (Unicode)

  49:  Const wdFormatUnicodeText = 7

  50:  Const wdFormatDOSTextLineBreaks = 5 DOS Text w Line-Breaks

  51:  Const wdFormatDOSText = 4 DOS

  52:   

  53:   

  54:  Const wdFormatXML = 11 XML

  55:  Const wdFormatXMLDocument = 12 XML DOc   .docx

  56:  Const wdFormatXMLDocumentMacroEnabled = 13 XML Macros Enabled  .docm

  57:  Const wdFormatXMLTemplate = 14 XML Template .dotx

  58:  Const wdFormatXMLTemplateMacroEnabled = 15 XML Template Macros Enabled  .dotm

  59:   

  60:  Const TEXTMSG = 0

  61:  Const HTMLMSG = 1

  62:   

  63:   Global variables

  64:  Dim arguments, bShowDebug, sOutFormat, iwdFormat, sOutExt, sFolder

  65:  Dim owdo  As Word.Application

  66:  Dim owdocs  As Word.Documents

  67:  Dim ofso, bDateFilter, sDateFilterType, sDateFrom, sDateTo, iDays

  68:  Dim bVerbose,output, sGoogleEmail

  69:  Dim ol, ns

  70:   

  71:  Set arguments = WScript.Arguments

  72:  sOutFormat = "PDF"

  73:  bDateFilter = false

  74:   

  75:  sDateFilterType = ""

  76:  sDateFrom = ""

  77:  sDateTo = ""

  78:  sGoogleEmail = ""

  79:   

  80:   

  81:  --------------------------------------------------------------------------------------------

  82:  Function DetermineOutFormat(sOut)

  83:     Dim iOut

  84:     Select Case ucase(sOUt)

  85:      Case ""

  86:          iOut = wdFormatPDF 

  87:          sOutExt = "pdf"

  88:    Case "PDF"

  89:          iOut = wdFormatPDF 

  90:    Case "XPS"

  91:          iOut = wdFormatXPS 

  92:    Case "DOC"

  93:          iOut = wdFormatDocument97

  94:      Case "DOT"

  95:          iOUt = wdFormatTemplate 

  96:    Case "HTML"

  97:          iOut = wdFormatHTML 

  98:    Case "HTM"

  99:          iOut = wdFormatFilteredHTML 

 100:    Case "DOCX"

 101:          iOut = wdFormatXMLDocument 

 102:    Case "DOCM"

 103:          iOut = wdFormatXMLDocumentMacroEnabled 

 104:    Case "DOTX"

 105:          iOut = wdFormatXMLTemplate 

 106:    Case "DOTM"

 107:          iOUt = wdFormatXMLTemplateMacroEnabled 

 108:    Case "TXT"

 109:          iOut = wdFormatTextLineBreaks 

 110:    Case "XML"

 111:      iOUt = wdFormatXML 

 112:    Case "ASC"

 113:          iOUt = wdFormatDOSTextLineBreaks 

 114:    Case "DOS"

 115:          iOUt = wdFormatDOSText 

 116:          sOutExt = "txt"

 117:      Case "RTF"

 118:          iOut =     wdFormatRTF     

 119:      Case "WEB"

 120:          iOUt = wdFormatWebArchive 

 121:      Case else

 122:         iOut = -1

 123:    End Select

 124:    DetermineOutFormat = iOut

 125:   

 126:  End Function

 127:   

 128:   

 129:  ---------------------------------------------------------------------------------------------

 130:   ECHOUSAGE

 131:   Outputs the usage information.

 132:  

 133:  Function EchoUsage()

 134:    If arguments.Count=0 Or arguments.Named.Exists("help") Or _
            arguments.Named.Exists("h") or iwdFormat=-1 Then

 135:     output.WriteLine "Save Word 2007 Compatible Document(s) in a different Format and more"

 136:     output.WriteLine ""

 137:     output.WriteLine "Usage: cscript /nologo doc2pdf.vbs [<input-file>] [/dir:<input-dir>] " & _ 
                           "[/date:created/modified] [/datefrom:mm-dd-yyyyy] " & _
                           "[/dateto:mm-dd-yyyy] [/days:nn] [/f:<output-format>] " & _
                           "[/o:<output-file>] [/googledocs:<email>]"

 138:     output.WriteLine  ""

 139:     output.WriteLine  "Available Options:"

 140:     output.WriteLine  "=================="

 141:     output.WriteLine  "/help - Specifies that this usage/help information should be displayed."

 142:     output.WriteLine  "/verbose - Specifies that detailed information about the " & _
                            "processing should be displayed."

 143:     output.WriteLine  ""

 144:     output.WriteLine  "Parameters (optional):"

 145:     output.WriteLine  "======================"

 146:     output.WriteLine  "/dir:<input-dir>"

 147:     output.WriteLine  "    Directory Batch Processing. Use <input-file> argument as " & _
                                "filter option (e.g. *.DOCX or *.* for all)"

 148:     output.WriteLine  ""

 149:     output.WriteLine  "Date Filter Options"

 150:     output.WriteLine  "--------------------"

 151:     output.WriteLine "/date:created/modified - Date filter option"

 152:     output.WriteLine "/datefrom:mm-dd-yyy - From Date for date-created or date-modfied filter"

 153:     output.WriteLine "/datetto:mm-dd-yyy - To Date for date-created or date-modfied filter"

 154:     output.WriteLine " ...or .."

 155:     output.WriteLine "/days:nn - created/changed within the last nn days. "

 156:     output.WriteLine ""

 157:     output.WriteLine "Output Options Parameters"

 158:     output.WriteLine "-------------------------"

 159:     output.WriteLine  " /f:<format> Specifies the output format values:"     

 160:     output.WriteLine  "    DOCX, DOCM, DOTX, DOTM, XML, PDF (default), XPS, TXT," & _
                            " DOS, ASC, HTML, HTM, RTS, DOC, WEB" 

 161:     output.WriteLine  " /o:<file> Optionally specification of output file."

 162:     output.WriteLine ""

 163:     output.WriteLine "Send to Google Docs via Email Option"

 164:     output.WriteLine "------------------------------------"

 165:     output.WriteLine "Note: Google Docs only supports the import of documents in " & _
                             "DOC file format (Office 97)."

 166:     output.WriteLine "PDF files are not supported. The file size is also limited to " & _
                             "500KB per document."

 167:     output.WriteLine ""

 168:     output.WriteLine "/googledocs:<email> - your personal email to upload documents," & _

                             " which looks like: longstring@prod.writely.com "

 169:     output.WriteLine ""      

 170:     output.WriteLine  "Examples:"

 171:     output.WriteLine  "========="

 172:     output.WriteLine "1. Convert all Word 2007 documents (.docx) that were modified " & _
                             "on or since December 31, 2008 and are located in C:\DOCS " & _
                             "to PDF and show detailed information about the processing."

 173:     output.WriteLine ""

 174:     output.WriteLine "cscript /nologo docx2pdf2.vbs *.docx /f:PDF /dir:C:\DOCS " & _
                           "/date:modified /datefrom:12-31-2008 /verbose"

 175:     output.WriteLine ""

 176:     output.WriteLine "2. Convert all Word 2007 documents (.docx) that where created " & _
                           "in the last 7 days and are located in C:\MY DOCUMENTS to " & _
                           "Word 97 (.DOC) and send them to Google Docs. "

 177:     output.WriteLine ""

 178:     output.WriteLine "cscript /nologo docx2pdf2.vbs *.docx /f:DOC /dir:""C:\MY DOCUMENTS"" " & _
                           "/date:created /days:7 /googledocs:mysecretemail@prod.writely.com"

 179:     output.WriteLine ""

 180:     output.WriteLine "3. Convert a single Word 2007 Document C:\DOCS\mydocument.docx " & _
                             "to PDF and save it as C:\MY DOCUMENTS\mynewdocument.pdf"

 181:    output.WriteLine ""

 182:     output.WriteLine "cscript /nologo docx2pdf2.vbs ""C:\DOCS\mydocument.docx"" " & _
                             "/o:""C:\MY DOCUMENTS\mynewdocument.pdf"""

 183:     output.WriteLine ""

 184:     output.WriteLine "4. Convert C:\DOCS\mydocument.docx to HTML and save it in " & _
                             "the same folder"

 185:     output.WriteLine ""

 186:     output.WriteLine "cscript /nologo docx2pdf2.vbs ""C:\DOCS\mydocument.docx"" /f:HTML"

 187:     output.WriteLine ""                         

 188:    End If 

 189:  End Function

 190:   

 191:  -------------------------------------------------------------------------------------------

 192:   CHECKARGS

 193:   Makes some preliminary checks of the arguments.

 194:   Quits the application is any problem is found.

 195:  

 196:  Function CheckArgs()

 197:     Check that <doc-file> is specified

 198:    If arguments.Unnamed.Count <> 1 Then

 199:      output.WriteLine "Error: Obligatory <doc-file> parameter missing!"

 200:      WScript.Quit 1

 201:    End If

 202:   

 203:  if arguments.Named.Exists("f") then

 204:        sOutFormat = arguments.Named.Item("f")

 205:  end if

 206:   

 207:  if arguments.Named.Exists("googledocs") then

 208:    sGoogleEmail =  arguments.Named.Item("googledocs")

 209:  end if    

 210:  if sGoogleEmail <> "" then

 211:     sOutFormat = "DOC"

 212:  end if

 213:   

 214:  sOutExt = sOutFormat

 215:  iwdFormat = DetermineOutFormat(sOutFormat)

 216:   

 217:  if arguments.Named.Exists("date") then

 218:   

 219:      if arguments.Named.Exists("days") then

 220:         if isNumeric(arguments.Named.Item("days")) then

 221:            iDays = int(abs(arguments.Named.Item("days")))

 222:            sDateFrom = formatdatetime(dateadd("d",iDays*-1,now),vbshortdate)

 223:            sDateTo = formatdatetime(now,vbshortdate)

 224:         end if

 225:       else

 226:          if arguments.Named.Exists("datefrom") then

 227:            if isDate(arguments.Named.Item("datefrom")) then

 228:              sDateFrom = formatdatetime(arguments.Named.Item("datefrom"),vbshortdate)

 229:            end if

 230:          end if

 231:           if arguments.Named.Exists("dateto") then

 232:             if isDate(arguments.Named.Item("dateto")) then

 233:                sDateTo = formatdatetime(arguments.Named.Item("dateto"),vbshortdate)

 234:             end if

 235:           else

 236:               sDateTo = formatdatetime(now,vbshortdate)

 237:           end if

 238:           end if

 239:           

 240:       if arguments.Named.Item("date") = "created" then

 241:          sDateFilterType = "c"

 242:       end if

 243:       if arguments.Named.Item("date") = "modified" then

 244:          sDateFilterType = "m"

 245:       end if

 246:      

 247:       if sDateFilterType <> "" and sDateFrom <> "" and sDateTo <> "" then

 248:          bDateFilter = true

 249:                  if bVerbose = true then

 250:                     output.WriteLine "Date Filter on! From Date: " & sDateFrom & _
                          ", To Date: " & sDateTo & ", Filter: " & sDateFilterType

 251:                  end if

 252:       end if

 253:   

 254:  end if

 255:   

 256:   

 257:   

 258:   

 259:  End Function

 260:   

 261:  ------------------------------------------------------------------------------------------

 262:   DOC2PDF

 263:   Converts a Word document to PDF using Word 2007.

 264:   Input:

 265:   sDocFile - Full path to Word document.

 266:   sPDFFile - Optional full path to output file.

 267:  

 268:   If not specified the output PDF file

 269:   will be the same as the sDocFile except

 270:   file extension will be .pdf.

 271:  

 272:  Function DOC2PDF( sDocFile, sPDFFile )

 273:   

 274:   

 275:    Dim owdoc  As Word.Document

 276:    Dim sPrevPrinter  As String

 277:   

 278:   

 279:    if sFolder = "" then

 280:        sDocFile = ofso.GetAbsolutePathName(sDocFile)

 281:          sTmpFolder = ofso.GetParentFolderName(sDocFile)

 282:    else

 283:      sDocFile = sDocFile

 284:      sTmpFolder = sFolder

 285:    end if

 286:   

 287:    If Len(sPDFFile)=0 Then

 288:      sPDFFile = ofso.GetBaseName(sDocFile) + "." + lcase(sOutExt)

 289:    End If

 290:   

 291:    If Len(ofso.GetParentFolderName(sPDFFile))=0 Then

 292:      sPDFFile = sTmpFolder + "\" + sPDFFile

 293:    End If

 294:   

 295:   

 296:     Enable this line if you want to disable autoexecute macros

 297:     owdo.WordBasic.DisableAutoMacros

 298:       if bVerbose = true then

 299:           output.WriteLine "Converting: " & sDocFile 

 300:       end if                

 301:     Open the Word document

 302:    Set owdoc = owdocs.Open(sDocFile)

 303:   

 304:     Let Word document save as PDF

 305:     - for documentation of SaveAs() method,

 306:       see http://msdn2.microsoft.com/en-us/library/bb221597.aspx 

 307:    owdoc.SaveAs sPDFFile, iwdFormat 

 308:   

 309:    owdoc.Close WdDoNotSaveChanges

 310:   

 311:       if bVerbose = true then

 312:           output.WriteLine "Saved As: " & sPDFFile & "(Format: " & iwdFormat & ")"

 313:       end if                

 314:   

 315:       if sGoogleEmail <> "" then

 316:          SendEmailToGoogleDocs sGoogleEmail, sPDFFile

 317:       end if

 318:  End Function

 319:   

 320:  ------------------------------------------------------------------------------------------

 321:   Returns an array with the file names that match Path.

 322:   The Path string may contain the wildcard characters "*"

 323:   and "?" in the file name component. The same rules apply

 324:   as with the MSDOS DIR command.

 325:   If Path is a directory, the contents of this directory is listed.

 326:   If Path is empty, the current directory is listed.

 327:   Author: Christian dHeureuse (www.source-code.biz)

 328:   

 329:  Public  Function ListDir (ByVal Path)

 330:     Dim fso: Set fso = CreateObject("Scripting.FileSystemObject")

 331:     If Path = "" then Path = "*.*"

 332:     Dim Parent, Filter

 333:     if fso.FolderExists(Path) then       Path is a directory

 334:        Parent = Path

 335:        Filter = "*"

 336:       Else

 337:        Parent = fso.GetParentFolderName(Path)

 338:        If Parent = "" Then If Right(Path,1) = ":" Then Parent = Path: Else Parent = "."

 339:        Filter = fso.GetFileName(Path)

 340:        If Filter = "" Then Filter = "*"

 341:        End If

 342:     ReDim a(10)

 343:     Dim n: n = 0

 344:     Dim Folder: Set Folder = fso.GetFolder(Parent)

 345:     Dim Files: Set Files = Folder.Files

 346:     Dim File

 347:     For Each File In Files

 348:   

 349:          bDateFiltered = false

 350:        if bDateFilter = true then

 351:             if sDateFilterType = "c" then

 352:               dDate = File.DateCreated 

 353:             end if            

 354:             if sDateFilterType = "m" then

 355:               dDate = File.DateLastModified

 356:             end if

 357:                       if datediff("d",dDate,sDateFrom) >= 0  or _
                               datediff("d",dDate,sDateTo) < 0 then

 358:                         bDateFiltered = true

 359:                           if bVerbose = true then

 360:                               output.WriteLine File.Name & " skipped (Date Filter), (dc: " & _
                                    File.DateCreated & ", dm: " & File.DateLastModified & ")" 

 361:                           end if

 362:                       end if      

 363:              end if

 364:        if  bDateFiltered = false then

 365:            If CompareFileName(File.Name,Filter) Then

 366:                If n > UBound(a) Then ReDim Preserve a(n*2)

 367:                a(n) = File.Path

 368:                n = n + 1

 369:                    else

 370:                           if bVerbose = true then

 371:                               output.WriteLine File.Name & _
                                    " skipped (File Path Filter Argument)" 

 372:                           end if                    

 373:            End If

 374:              end if

 375:        Next

 376:     ReDim Preserve a(n-1)

 377:     ListDir = a

 378:     End Function

 379:  -----------------------------------------------------------------------------------------

 380:  Private Function CompareFileName (ByVal Name, ByVal Filter)  (recursive)

 381:     CompareFileName = False

 382:     Dim np, fp: np = 1: fp = 1

 383:     Do

 384:        If fp > Len(Filter) Then CompareFileName = np > len(name): Exit Function

 385:        If Mid(Filter,fp) = ".*" Then     special case: ".*" at end of filter

 386:           If np > Len(Name) Then CompareFileName = True: Exit Function

 387:           End If

 388:        If Mid(Filter,fp) = "." Then      special case: "." at end of filter

 389:           CompareFileName = np > Len(Name): Exit Function

 390:           End If

 391:        Dim fc: fc = Mid(Filter,fp,1): fp = fp + 1

 392:        Select Case fc

 393:           Case "*"

 394:              CompareFileName = CompareFileName2(name,np,filter,fp)

 395:              Exit Function

 396:           Case "?"

 397:              If np <= Len(Name) And Mid(Name,np,1) <> "." Then np = np + 1

 398:           Case Else

 399:              If np > Len(Name) Then Exit Function

 400:              Dim nc: nc = Mid(Name,np,1): np = np + 1

 401:              If Strcomp(fc,nc,vbTextCompare)<>0 Then Exit Function

 402:           End Select

 403:        Loop

 404:     End Function

 405:  -------------------------------------------------------------------------------------------

 406:  Private Function CompareFileName2 (ByVal Name, ByVal np0, ByVal Filter, ByVal fp0)

 407:     Dim fp: fp = fp0

 408:     Dim fc2

 409:     Do                                   skip over "*" and "?" characters in filter

 410:        If fp > Len(Filter) Then CompareFileName2 = True: Exit Function

 411:        fc2 = Mid(Filter,fp,1): fp = fp + 1

 412:        If fc2 <> "*" And fc2 <> "?" Then Exit Do

 413:        Loop

 414:     If fc2 = "." Then

 415:        If Mid(Filter,fp) = "*" Then      special case: ".*" at end of filter

 416:           CompareFileName2 = True: Exit Function

 417:           End If

 418:        If fp > Len(Filter) Then          special case: "." at end of filter

 419:           CompareFileName2 = InStr(np0,Name,".") = 0: Exit Function

 420:           End If

 421:        End If

 422:     Dim np

 423:     For np = np0 To Len(Name)

 424:        Dim nc: nc = Mid(Name,np,1)

 425:        If StrComp(fc2,nc,vbTextCompare)=0 Then

 426:           If CompareFileName(Mid(Name,np+1),Mid(Filter,fp)) Then

 427:              CompareFileName2 = True: Exit Function

 428:              End If

 429:           End If

 430:        Next

 431:     CompareFileName2 = False

 432:     End Function

 433:  -------------------------------------------------------------------------------------------

 434:       

 435:  Sub SendEmailToGoogleDocs(sToEmail, sFileAttachment)

 436:   

 437:  Dim ToAddress

 438:  Dim MessageSubject

 439:  Dim MessageBody

 440:  Dim MessageAttachment

 441:  Dim newMail

 442:   

 443:   

 444:   

 445:  Set oCheckFile = oFso.GetFile(sFileAttachment)