Stone Oakvalley's Amiga Music Collection (SOAMC=)
The automated Commodore Amiga Music recorded to MP3. Project covers 193 different Amiga Music File Formats!


                    
More music:  
   



T H E   R E A L   D E A L   I N   -   A U T H E N T I C  -   C O M M O D O R E   A M I G A   M U S I C  !




SOAMC= ENDGAME: Extraction and Decrunching progress
in All about SOAMC= | Thursday, May 10, 2018 | 10:01


The following list shows all the files we are identifying, extracting and decrunch (with XFDecrunch) PRIOR to actual module/music scanning. They are divided into groups. This process was started in Apil 2018 and will take several months.


-----------------------------------------------------------------------------------------------------------------------------
The origin of files located in my Amiga collection is from my most recent compiled SOTDS database that has not not been released yet online in SOTDS format, as the software "SOTDS_SEARCHER.EXE" and "SOTDS_CONSTRUCTOR.EXE" is currently in development.

Since my .DAT files are already in text format, I'm currently using other tools to filter out file signatures for ADF, ADZ, DMS, LHA etc.

The current statistics of what we are processing:
Rows: 28230191 (28.23mill)
Total Words: 197611337 (197.61mill)
Locations: 788144 (0.79mill)
Original Database: 2769955320 bytes (2769.96 MB)

Rows = actual single files. Used in our scanning.
Total Words = Only applicable for SOTDS Searcher (how many words that are unique). Not used in the scanning procedure.
Locations = Sources that are disk-images, archives, cd-images as listed below and used in our scanning.

Statistics of disk-images, archives, cd-images - based on SOTDS database:
Note: Items listed below includes disk-images and archives located INSIDE cd-images as well and referenced as "SOURCE/LOCATION" in my SOTDS database.
ADF = 209331 items
ADZ = 1118 items
DMS = 45040 items
LHA = 371150+ items (not determined 100% yet due to crash in SOTDS_Searcher.exe)
LZH = 42270 items
LZX = 15806 items
ZIP = 13687 items
RAR = 64 items
Z = 23 items
GZ = 3597 items
TAR = 297 items
Plain Files = N/A

BIN (CD-image) = 24 items
IMG (CD-image) = 29 items
ISO (CD-image) = 345 items
MDF (CD-image) = 4 items
NRG (CD-image) = 10 items
Plain files inside CD-images = N/A


Now on with the progress and filters:

For Executable files identified by 000003f300000000 - first 8 bytes
Inside Plain ADF: 22 April 2018 to 10 May 2018 - DONE!
Inside Plain ADZ: None to process - DONE!
Inside Plain DMS: 10 May 2018 - DONE!
Inside Plain LHA: 12 May 2018 - 20 May 2018 - DONE!
Inside Plain LZH: 11 May 2018 - 12 May 2018 - DONE!
Inside Plain LZX: 10 May 2018 - 11 May 2018 - DONE!
Inside Plain ZIP: 20 May 2018 - 21 May 2018 - DONE!
Inside Plain RAR: None to process - DONE!
Inside Plain Z: None to process - DONE!
Inside Plain GZ: None to process - DONE!
Inside Plain TGZ: None to process - DONE!
Inside Plain TAR: None to process - DONE!

Plain Files (will ignore all extensions above): 01 Jul 2018 - 02 Jul 2018. DONE!

CD-Images are references to BIN, IMG, ISO, MDF, NRG.

Inside Plain ADF located inside CD-images: 04 July 2018 - 06 July 2018. DONE!
Inside Plain ADZ located inside CD-images: 06 July 2018 - In progress.
Inside Plain DMS located inside CD-images: 11 July 2018 - 13 July 2018. DONE!
Inside Plain LHA located inside CD-images: 13 July 2018 - In progress.
Inside Plain LZH located inside CD-images: xx July 2018 - In progress.
Inside Plain LZX located inside CD-images: xx July 2018 - In progress.
Inside Plain ZIP located inside CD-images: xx July 2018 - In progress.
Inside Plain RAR located inside CD-images: None to process. DONE!
Inside Plain Z located inside CD-images: None to process. DONE!
Inside Plain GZ located inside CD-images: xx July 2018 - In progress.
Inside Plain TGZ located inside CD-images: None to process. DONE!
Inside Plain TAR located inside CD-images: None to process. DONE!

Plain Files located inside CD-images (will ignore all extensions listed above): Pending


We have selected the most common and easily known headers used for typically datafiles as they do not start with the executable header. Naturally, we cannot list all kinds of crunchers here in this section as it would take months to try and find every compresser ever used on Amiga Files, sort and tag them like this. We have simply chosen these to try and cut down the processing at least a little bit :-)

Even though until now we have stored every cruncher detected on executables, see the bottom link at this page.

So, after these 4 specifc and known headers is located, decrunched and put into scanning queue, we will simply add all those other files that does not match either the executable signature or the ones listed below and simply run ALL OF THEM through XFD inside WinUAE in order to decrunch all files. Naturally, we are speaking of millions of files and we have no idea how long that process is gonna take. We would probably have to look at several months of processing in several instances of WinUAE running 24/7 :-) More updates when we know them, of course!!

For PowerPacker (PP20) datafiles identified by 50503230090a - first 6 bytes
Inside Plain ADF: 10 Jun 2018 - 23 Jun 2018. DONE!
Inside Plain ADZ: None to process. DONE!
Inside Plain DMS: 10 Jun 2018 - 23 Jun 2018. DONE!
Inside Plain LHA: 10 Jun 2018 - 23 Jun 2018. DONE!
Inside Plain LZH: 10 Jun 2018- 23 Jun 2018. DONE!
Inside Plain LZX: 10 Jun 2018- 23 Jun 2018. DONE!
Inside Plain ZIP: 10 Jun 2018- 23 Jun 2018. DONE!
Inside Plain RAR: None to process. DONE!
Inside Plain Z: None to process. DONE!
Inside Plain GZ: None to process. DONE!
Inside Plain TGZ: None to process. DONE!
Inside Plain TAR: None to process. DONE!

CD-Images are references to BIN, IMG, ISO, MDF, NRG.

Inside Plain ADF located inside CD-images: Pending scan.
Inside Plain ADZ located inside CD-images: Pending scan.
Inside Plain DMS located inside CD-images: Pending scan.
Inside Plain LHA located inside CD-images: Pending scan.
Inside Plain LZH located inside CD-images: Pending scan.
Inside Plain LZX located inside CD-images: Pending scan.
Inside Plain ZIP located inside CD-images: Pending scan.
Inside Plain RAR located inside CD-images: Pending scan.
Inside Plain Z located inside CD-images: Pending scan.
Inside Plain GZ located inside CD-images: Pending scan.
Inside Plain TGZ located inside CD-images: None to process. DONE!
Inside Plain TAR located inside CD-images: Pending scan.


For XPK (XPKF) Crunched datafiles identified by 58504b46 - first 4 bytes
Inside Plain ADF: 26 Jun 2018 - In progress.
Inside Plain ADZ: 26 Jun 2018 - In progress.
Inside Plain DMS: 26 Jun 2018 - In progress.
Inside Plain LHA: 26 Jun 2018 - In progress.
Inside Plain LZH: 26 Jun 2018 - In progress.
Inside Plain LZX: 26 Jun 2018 - In progress.
Inside Plain ZIP: 26 Jun 2018 - In progress.
Inside Plain RAR: None to Process. DONE!
Inside Plain Z:None to Process. DONE!
Inside Plain GZ: None to Process. DONE!
Inside Plain TGZ: None to Process. DONE!
Inside Plain TAR: None to Process. DONE!

CD-Images are references to BIN, IMG, ISO, MDF, NRG.

Inside Plain ADF located inside CD-images: Pending scan.
Inside Plain ADZ located inside CD-images: Pending scan.
Inside Plain DMS located inside CD-images: Pending scan.
Inside Plain LHA located inside CD-images: Pending scan.
Inside Plain LZH located inside CD-images: Pending scan.
Inside Plain LZX located inside CD-images: Pending scan.
Inside Plain ZIP located inside CD-images: Pending scan.
Inside Plain RAR located inside CD-images: Pending scan.
Inside Plain Z located inside CD-images: Pending scan.
Inside Plain GZ located inside CD-images: Pending scan.
Inside Plain TGZ located inside CD-images: None to process. DONE!
Inside Plain TAR located inside CD-images: None to process. DONE!


For CrunchMania (CrM) Crunched datafiles identified by 43724d - first 3 bytes
Inside Plain ADF: 23 Jun 2018 - 26 Jun 2018. DONE!
Inside Plain ADZ: None to process. DONE!
Inside Plain DMS: 23 Jun 2018 - 26 Jun 2018. DONE!
Inside Plain LHA: 23 Jun 2018 - 26 Jun 2018. DONE!
Inside Plain LZH: None to process. DONE!
Inside Plain LZX: 23 Jun 2018 - 26 Jun 2018. DONE!
Inside Plain ZIP: 23 Jun 2018 - 26 Jun 2018. DONE
Inside Plain RAR: None to process. DONE!
Inside Plain Z: None to process. DONE!
Inside Plain GZ: None to process. DONE!
Inside Plain TGZ: None to Process. DONE!
Inside Plain TAR: None to process. DONE!

CD-Images are references to BIN, IMG, ISO, MDF, NRG.

Inside Plain ADF located inside CD-images: Pending scan.
Inside Plain ADZ located inside CD-images: Pending scan.
Inside Plain DMS located inside CD-images: Pending scan.
Inside Plain LHA located inside CD-images: Pending scan.
Inside Plain LZH located inside CD-images: Pending scan.
Inside Plain LZX located inside CD-images: Pending scan.
Inside Plain ZIP located inside CD-images: Pending scan.
Inside Plain RAR located inside CD-images: Pending scan.
Inside Plain Z located inside CD-images: Pending scan.
Inside Plain GZ located inside CD-images: Pending scan.
Inside Plain TGZ located inside CD-images: None to process. DONE!
Inside Plain TAR located inside CD-images: None to process. DONE!


For StoneCracker (S40) Crunched datafiles identified by 533430 - first 3 bytes
Inside Plain ADF: 26 Jun 2018 - 26 Jun 2018. DONE!
Inside Plain ADZ: None to process. DONE!
Inside Plain DMS: 26 Jun 2018 - 26 Jun 2018. DONE!
Inside Plain LHA: 26 Jun 2018 - 26 Jun 2018. DONE!
Inside Plain LZH: None to process. DONE!
Inside Plain LZX: 26 Jun 2018 - 26 Jun 2018. DONE!
Inside Plain ZIP: 26 Jun 2018 - 26 Jun 2018. DONE
Inside Plain RAR: None to process. DONE!
Inside Plain Z: None to process. DONE!
Inside Plain GZ: None to process. DONE!
Inside Plain TGZ: None to Process. DONE!
Inside Plain TAR: None to process. DONE!

CD-Images are references to BIN, IMG, ISO, MDF, NRG.

Inside Plain ADF located inside CD-images: Pending scan.
Inside Plain ADZ located inside CD-images: Pending scan.
Inside Plain DMS located inside CD-images: Pending scan.
Inside Plain LHA located inside CD-images: Pending scan.
Inside Plain LZH located inside CD-images: Pending scan.
Inside Plain LZX located inside CD-images: Pending scan.
Inside Plain ZIP located inside CD-images: Pending scan.
Inside Plain RAR located inside CD-images: Pending scan.
Inside Plain Z located inside CD-images: Pending scan.
Inside Plain GZ located inside CD-images: Pending scan.
Inside Plain TGZ located inside CD-images: None to process. DONE!
Inside Plain TAR located inside CD-images: None to process. DONE!


After all of this has been processed as listed above, we can start on the actual music/module indentifying and ripping through all decrunched executables and non-executables - which will take additional months.


Please review these related article links:
Click to open urlSOAMC= "END GAME" - A quick followup to the new add-on recording proje...


------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Posted by: Old-schooler, Stone Oakvalley | Publisher: Crazy Multi Talent, Stone Oakvalley
Last revised: July 15, 2018 - 01:16 | Page views: 224


Website Design by post@stone-oakvalley-studios.com - Copyright © 2018 www.stone-oakvalley-studios.com