PSXMinimise Standard This article is a work in progress (see notes at the end) So it was late one night and I couldn’t sleep, which is nothing abnormal for me and I got thinking. Is there any way I can improve the compression ratio of my collection of PS1 games, or was 7zip with LZMA the de-facto and best option? The truth it turns out is a little more complicated. Finding The ‘Best’ Compression I’ve tried numerous compression formats in the search for the best one for PlayStation games. I needed a test game to find the best archive solution for the game data. I would tackle the audio content of the discs later. I grabbed my backup copy of Oddworld: Abe’s Oddysee (one of my favourite games) and tried all the regular formats that PeaZip could handle in their “Best” modes. Compression FormatOptionsCompression TimeResulting FilesizeNone–685,461 KB7ZipUltra2 minutes403,753 KBARC (FreeArc)94 minutes289,820 KBBrotli935 minutes417,106 KBBZip2Ultra16 minutes512,282 KBGZipUltra7 minutes514,581 KBXZUltra3 minutes403,755 KBZIPUltra6 minutes514,582 KBZstd193 minutes447,022 KBZPAQUltra34 minutes409,185 KB Up until I saw the FreeArc compression sizes, it would seem that 7zip ultra compression was the best for PS1 data. At first, I thought there’d been some kind of error and the archive couldn’t have been as small as it became. However, after some checking I finally saw that the result was indeed accurate and it did give an excellent compression ratio of PlayStation 1 game data. Sub-channel Data A few years ago, I used to visit the EmuParadise website, and I recall that they used to compress their bin files with a tool called ECM. Armed with this vague piece of information I started searching the web for information about it. I managed to find the ECM Tools package (on archive.org) written in 2002 by Neill Corlett that is used to encode and decode files. These tools either add or remove sub-channel error correction data, or “Error Code Modelling” on a CD image. Removing this information will save some data from an image file and it can be added back in such a way that the file is an identical binary. FilenameOriginal SizeSize (after ECM Encoding)Destruction Derby 2 (Europe) (Track 01).bin71,603 KB64,141 KBDino Crisis (Europe) (Track 1).bin378,702 KB340,398 KBDriver (Europe).bin729,961 KB651,308 KBGrand Theft Auto (Europe) (Track 01).bin91,673 KB81,870 KBMediEvil (Europe) (Track 1).bin516,356 KB478,157 KBOddworld – Abe’s Oddysee (Europe).bin685,461 KB612,203 KBTomb Raider II (Europe) (Track 01).bin224,530 KB218,150 KB So it seems it is rather worth processing the ECM data, with all the games I’ve tested so far you get some kind of saving and with others you get a substantial saving. Audio Compression To compress the audio portions of the PS1 games, I decided to use FLAC, the free lossless audio encoder. I used this because the input files you pass to FLAC when decoded will be exactly the same as the original files. In a typical PS1 game, Track 01.bin is usually the game data, and if there are any other tracks (Track 02.bin, etc) they are CDDA files, which you can compress using FLAC if you tell it how these raw files can be processed. flac -8 -V --force-raw-format --endian=little --channels=2 --bps=16 --sample-rate=44100 --sign=signed *.bin An explanation of these options is that -8 sets the maximum level of FLAC compression and -V makes FLAC verify the files it has written after they have been created. The –force-raw-format argument instructs the command line version of FLAC to treat the input files as raw data. The –endian=little setting instructs FLAC of the byte order of the files. –channels=2 indicates that the audio stream is stereo. –bps=16 informs that the files are 16bit, –sample-rate=44100 is the standard audio sampling rate for a compact disc and finally –sign=signed tells FLAC that it’s a signed audio file. I am indebted to Cacovsky for the vital information on how raw CD audio can be converted to FLAC. Putting It All Together I had a few design goals for anything I created. It needed to rely on mostly open tools so that the techniques used are operating system agnostic and could fairly easily be ported to any other system and to use nothing that has encumbering license terms. Compression speed isn’t much of a factor as these files will really only be compressed once, however decompression speed shouldn’t be prohibitively slow. The methods used needed to be binary compatible so that the games were exactly the same after decompressing as they would be beforehand and I also wanted a system to verify that the files were indeed exactly the same as the originals. From the results of compression earlier, FreeArc was definitely the obvious choice, given that it had a great improvement in compression ratios of the other archive formats and it didn’t take significantly longer to compress the data. Firstly I removed the ECM data from the first track and then processed the audio data from the other tracks so that it’s filesize is reduced using FLAC, the entire game can be packaged up in a .arc file. Before doing this however, I needed a way to make sure that my compressed files would be the same as the originals when running the decompression process. On Linux there is a utility called sha256sum which will calculate a hash which can be compared against a file later to ensure that no changes have been made. On Windows there is a built in hash checker, however it doesn’t seem to be able to compare from a list of checksums in a single file and verify the data is intact. I am again indebted to Dave Benham who maintains HASHSUM.BAT at the DOSTips Forum. He created a script that is compatible with sha256sum but for Windows that works (mostly) in batch script. The Scripts So after a few sleepless nights, I’ve put together some utilities for automatically compressing and decompressing the games. find . -maxdepth 1 -type d -exec bash -c "cd \"{}\" && sha256sum * > \"uncompressed.sha256\"" \; @echo off rem ------------------------------ rem PSXMinimise Compression Script rem Version 0.17 rem Created by jcx rem https://jcx.life/psxminimise rem ------------------------------ IF EXIST "*(Track 01).bin" ( ecm *"(Track 01).bin" del *"(Track 01).bin" ) ELSE ( IF EXIST "*(Track 1).bin" ( ecm *"(Track 1).bin" del *"(Track 1).bin" ) ELSE ( ecm *".bin" GOTO compress ) ) :audiocomp flac -8 -V --force-raw-format --endian=little --channels=2 --bps=16 --sample-rate=44100 --sign=signed *.bin del *.bin :compress for %%I in (.) do set CurrDirName=%%~nxI freearc a -m9 -s "%CurrDirName%.arc" *.bin.ecm *.flac *.cue *.sha256 IF EXIST "*.bin.ecm" ( del *.bin.ecm ) IF EXIST "*.flac" ( del *.flac ) IF EXIST "*.cue" ( del *.cue ) IF EXIST "*.sha256" ( del *.sha256 ) IF EXIST "*.bin" ( del *.bin ) :end My compression script is a bit more ugly than the unpacker, since I imagine people will unpack the games more often than they will pack them. @echo off rem ------------------------------ rem PSXMinimise Decompression Script rem Version 0.37 rem Created by jcx rem https://jcx.life/psxminimise rem ------------------------------ color 5F echo --------------------------------------------------------- echo ____ ______ ____ __ _ _ _ echo ^| _ \/ ___\ \/ / \/ (_)_ __ (_)_ __ ___ (_)___ ___ echo ^| ^|_) \___ \\ /^| ^|\/^| ^| ^| '_ \^| ^| '_ ` _ \^| / __^|/ _ \ echo ^| __/ ___) / \^| ^| ^| ^| ^| ^| ^| ^| ^| ^| ^| ^| ^| ^| \__ \ __/ echo ^|_^| ^|____/_/\_\_^| ^|_^|_^|_^| ^|_^|_^|_^| ^|_^| ^|_^|_^|___/\___^| echo. echo --------------------------------------------------------- echo Created by jcx at https://jcx.life/psxminimise echo Version 0.37 echo --------------------------------------------------------- timeout /T 5 echo --------------------------------------------------------- echo Unpacking .ARC echo --------------------------------------------------------- freearc e *.arc echo --------------------------------------------------------- echo Re-adding Error Code Modulation to Track 1 echo --------------------------------------------------------- IF EXIST "*(Track 01).bin.ecm" ( unecm *"(Track 01).bin.ecm" del *"(Track 01).bin.ecm" goto audioextract ) ELSE ( IF EXIST "*(Track 1).bin.ecm" ( unecm *"(Track 1).bin.ecm" del *"(Track 1).bin.ecm" goto audioextract ) ELSE ( unecm *".bin.ecm" GOTO verify ) ) :audioextract echo --------------------------------------------------------- echo Uncompressing CD audio tracks echo --------------------------------------------------------- flac -d --force-raw-format --endian little --sign=signed *.flac rename *.raw *.bin del *.flac echo. goto verify :verify echo --------------------------------------------------------- echo Verifying Checksums echo --------------------------------------------------------- echo. IF EXIST "*.bin.ecm" ( del *.bin.ecm ) IF EXIST "*.flac" ( del *.flac ) call hashsum.bat /c /nh /ns uncompressed.sha256 IF %ERRORLEVEL% EQU 0 GOTO success GOTO error :success rem cls IF EXIST "*.arc" ( del *.arc ) color 2F echo --------------------------------------------------------- echo ____ ______ ____ __ _ _ _ echo ^| _ \/ ___\ \/ / \/ (_)_ __ (_)_ __ ___ (_)___ ___ echo ^| ^|_) \___ \\ /^| ^|\/^| ^| ^| '_ \^| ^| '_ ` _ \^| / __^|/ _ \ echo ^| __/ ___) / \^| ^| ^| ^| ^| ^| ^| ^| ^| ^| ^| ^| ^| ^| \__ \ __/ echo ^|_^| ^|____/_/\_\_^| ^|_^|_^|_^| ^|_^|_^|_^| ^|_^| ^|_^|_^|___/\___^| echo. echo --------------------------------------------------------- echo Created by jcx at https://jcx.life/psxminimise echo Version 0.37 echo --------------------------------------------------------- echo Unpacking has been completed successfully. echo. echo. timeout /T 5 GOTO END :error color 4F echo --------------------------------------------------------- echo An error %ERRORLEVEL% occured. echo --------------------------------------------------------- pause GOTO END :end Conclusion GameOriginal Filesize7zip Filesize7z PercentagePSXMinimisePSXMinimise PercentrageDestruction Derby 2620 MB533 MB85 %393 MB63 %Dino Crisis405 MB202 MB49 %170 MB41. %Driver712 MB357 MB50 %298 MB41 %Grand Theft Auto714 MB612 MB85 %452 MB63 %MediEvil535 MB322 MB60 %283 MB52 %Oddworld: Abe’s Oddysee669 MB387 MB57 %241 MB36 %Tomb Raider II708 MB460 MB64 %317 MB44 %WipEout 2097681 MB566 MB83 %413 MB60 %Average630.5 MB429 MB67.22 %320 MB50.60 %Average Saving–200 MB32.77 %309 MB49.40 % You could get smaller game files if you removed some of the games content (a process known as “Ripping” using something like PocketISO to remove audio tracks and FMV sequences from games, however I wanted the games to be intact to their original release, and I think this has done quite a good job. Note: This article isn’t completely finished, when writing I got rather tired and haven’t finished it off yet! Downloads and more to come soon! ~ jcx
