| |
Bacchus
Registered: Jan 2002
Posts: 156 |
Replacing games loader ...
OK, this on cracking but still highly related to coding.
Most games I fiddled with over the year call the loader using a parameter that was the index of the file. Using the same parameter as index for my IFFL or converting it to a two byte string starting the file name has worked well for me in most cases.
I am now facing two games that are not distributed yet (old, but no scene version is out) where there is a lot of data stored directly on the disk and the game then loads it using direct track and sector. Think of it like action adventures. The game loads strings or other really small things by loading T/S and then exctacting the needed part. It's hence not really "files" most of it and there are so many that a file per string is plainly not within reach.
I can think of a few approaches;
1) Keep it as data on disk. Allocate the sectors used and then store the files on the unallocated sectors. You can't compress it - it does take a full disk side any way you look at it. It does work, looks rather neat but cannot be counted as a firstie.
2) Make a big chunk of the data to a file and push it to a REU the first thing you do. The game become ever so much more playable and fast. And the file can be compressed efficiently. You do need a REU (or simply enable it in your emulator or Ultimate Cart) but it's also still not counted as a firstie.
3) Make a big file which you then need to scan as the original 256 byte sectors are now 254, so a sector that was a full page is by necessity spread over two sectors in a file based option. I guess you can also compress the sectors individually and think of the sectors as files in an IFFL. One IFFL file equals a sector. This is an ugly bitch but could be counted as a firstie.
Any other thought on this technical challenge? I must admit I am growing fond of the REU option, and the firstie restriction is the only thing that holds me back. The Tink games we just released had been perfect in REU version. Would have saved SO much work, loading would have been near instant and it would have been a release of two neat files.
Am I missing any options or can someone provide some lateral thinking, that opens up new options by finding approaches I have missed? |
|
| |
Maxlide
Registered: Apr 2003
Posts: 31 |
When the things you need to load are primary text than I would go for an IFFL firstie. RLE pack the text or convert the text from 8-bit text to 7-bit or 6-bit text and there you go. |
| |
Martin Piper
Registered: Nov 2007
Posts: 722 |
Compress each block, store all the data without gaps and have an index.
Then when you want a block, decompress it. |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
I quite like Martin's plan. You could do quite a specialised cruncher for single blocks, too, as offsets are never more than 256 bytes.
Alternately, break the data into tracks rather than blocks, and decrunch a track at a time for better ratio.
How does it know how many bytes to read each time? And can you extract all the possible start locations/chunks, or is that impossible to find without playing through every possible game response? |
| |
Martin Piper
Registered: Nov 2007
Posts: 722 |
A Huffman algorithm would work quite well. The tree can be optimised generally across all the data and stored in code. Then the blocks can be compressed with that tree. Resulting in consistently smaller blocks. |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
I'm always itching to see the data when I read posts like this, or at least a sample thereof. So many different potential strategies depending on the characteristics. |
| |
Bacchus
Registered: Jan 2002
Posts: 156 |
@maxlide: Compression is sort of second here. Key is the question of accessing 256 byte chunks.
Data can be text and gfx and the games extracts the stuff it needs from the sector it just loaded.
@martin: This is very much IFFL. If I think of sectors as files, then the IFFL strategy is only viable if the tables can fit in the RAM I can find for tables. If I use drive RAM, very few IFFL routines will support more than 128 files/sectors (scanner will generate track/sector/offset table). An using multiple IFFL files is a REAL nuance (you need RAM for the scanner and you need to do rescanning). So this is ONLY an option for cases where you can isolate less than 128 "files". But if so, it's for sure valid ...
@chris: well, merging the sectors to tracks means that when I want to load a sector in a higher sector of a track, then I need to start loading+depacking, and then just ditch the data. If compressed, I never know the offset. That is subject to the compression. Compression makes it so that you cannot compute the offset - you need to scan and store a table and you need RAM for the table. |
| |
MagerValp
Registered: Dec 2001
Posts: 1078 |
This is exactly the problem I faced with Ultima IV Remastered where there are 688 raw sectors spread out over three disk sides. Source and docs on GitHub: https://github.com/MagerValp/u4remastered/blob/master/docs/ULoa..
TL;DR: Each sector individually crunched and then packed into IFFL archives that are scanned on startup. Track/Sector/Offset saved under I/O. |
| |
Knight Rider
Registered: Mar 2005
Posts: 131 |
Did you consider an EasyFlash release ? Much wider target audience than REU
Red Storm Rising for example also uses track/sector loading too. This was simply changed to X/Y (tr/se) file base loading and then build as CRT image.
Open the .CRT in a hex editor (from $e1f), and you will even see this
01 00
01 01
01 02
01 03
01 04
01 05
01 06
|
| |
MagerValp
Registered: Dec 2001
Posts: 1078 |
Alternate strategy that was considered: Pack up four sectors into a 1024 byte file that is then crunched. When loading you need a temporary 1024 byte unpack buffer, but it reduces the number of "files" to a quarter. You also get slightly better compression ratio. |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
Quoting Bacchus
@chris: well, merging the sectors to tracks means that when I want to load a sector in a higher sector of a track, then I need to start loading+depacking, and then just ditch the data. If compressed, I never know the offset. That is subject to the compression. COmpression makes it so that you cannot compute the offset - you need to scan and store a table.
Oh I'm aware of that - but I've no idea of how much free space or free CPU you have. For some games decrunching a track into a buffer would be affordable, for others it would not. Then there are compromises, like the 1k buffer MagerValp suggested above.
-cjam |
... 36 posts hidden. Click here to view all posts....
|
| Previous - 1 | 2 | 3 | 4 | 5 - Next |
