We have been silent for almost 2-3 months - and with perfect reasons to why :-)
During the last months a lot of processing was already done and custom tools programmed and beta tested. We have discovered several new ideas, problems and most importantly SOLUTIONS to this Guinness World Records related project :-)
We have decided to split the project into 3 PHASES:
Extract and structure/list ALL Amiga oriented files from every disk-image, archive, cd-images, plain files, data files etc. (and everything inside cd-images), group them by various filters by using my SOTDS AmigaFileDatabase just recently updated to 31 million entries, then start to decrunch them!
This included also to filter out NON-DOS disks and scan these also with XFDSCAN to extract and detect compressed files hidden inside NON-DOS disks. This process was just recently in September finalized and we ended up with around 800000 files from 372541 disk-images (ADF/ADZ/DMS etc.) that was detected, decrunched and put into the actual music-ripping queue of PHASE 3.
The total files retrieved from PHASE 1 is yet unknown, but should probably be over 30 million files!
A side-project of PHASE 1 is to locate and find EVERY possible XFD Slave and XPKxxxx.LIBRARY that XFDDECRUNCH and XFDSCAN can benefit from during our automated decrunchng process. This project will be finalized around 25th of September in which the results will be uploaded to our site (see another post), but also used in our automation environment from that time.
This phase will make another 2nd pass of decrunching of all files found in PHASE 1. As you know, some files might be double-crunched (or we loose some files along the way as we finetuned our tools, or due to missing decrunchers XFD, XPK which was located later) and we do not want to miss that chance of loosing some hidden music files while we have a automated decrunching environment already set up.
Another processing will be performed after this and that is to re-organize and group all files we have decrunched/extracted into directories of 1000 pcs pr. each to allow WinUAE/Amiga setup to process it smoothly. We expect this to be finalized at the very end of 2018 in which PHASE 3 below will be instantly started.
When PHASE 1 and 2 was performed as best as possible, we are ready with the actual automated ripping and scanning of music files by the RIPPERS we had already informed about on our website. Imagine the amount of files that will be pushed through 4 ripper procedures (ExoticRipper, EagleRipper, Pro-Wizard-Amiga, and Pro-Wizard-PC). Its an easy calculation of 4. So, 25.6 million files multiply that with 4 = 102million files to be pushed through the system automatically by using multiple instances of WinUAE probably for way over 6 months or more 24/7, and you see why we call this project a Guinness World Records related project, its just simply BEYOND INSANE!
The last bit we're all waiting for :-)
As we have done quite a research to achieve this insane project several lists and actual file compilations can come out of this which can be relased for the Amiga fans worldwide as a nice little bonus gift beside just a gadzillion of real Amiga music files and its MP3/FLAC recordings done on real hardware, such as:
1: Every possible EaglePlayer/DeliTracker player engine that we could dig out.
2: Names for all the different compressor techniques used.
3: A complete list and file compilation of EVERY XFD-slave ever in existence that we could dig out.
4: A complete list and file compilation of EVERY XPK library ever in existence that we could dig out.
and finally, what about we released the ENTIRE decrunched archive of at least 30million files to help others scan for something else than just music? The benefits that this project has spent months, maybe even a year could potentionally set loose a nice way for other Amiga-fans to do additional work from hereon. We do not yet have the complete size in bytes and amount of files ready yet, but its gonna be a MOTHERLOAD of data!