| |
Bacchus
Registered: Jan 2002
Posts: 156 |
Replacing games loader ...
OK, this on cracking but still highly related to coding.
Most games I fiddled with over the year call the loader using a parameter that was the index of the file. Using the same parameter as index for my IFFL or converting it to a two byte string starting the file name has worked well for me in most cases.
I am now facing two games that are not distributed yet (old, but no scene version is out) where there is a lot of data stored directly on the disk and the game then loads it using direct track and sector. Think of it like action adventures. The game loads strings or other really small things by loading T/S and then exctacting the needed part. It's hence not really "files" most of it and there are so many that a file per string is plainly not within reach.
I can think of a few approaches;
1) Keep it as data on disk. Allocate the sectors used and then store the files on the unallocated sectors. You can't compress it - it does take a full disk side any way you look at it. It does work, looks rather neat but cannot be counted as a firstie.
2) Make a big chunk of the data to a file and push it to a REU the first thing you do. The game become ever so much more playable and fast. And the file can be compressed efficiently. You do need a REU (or simply enable it in your emulator or Ultimate Cart) but it's also still not counted as a firstie.
3) Make a big file which you then need to scan as the original 256 byte sectors are now 254, so a sector that was a full page is by necessity spread over two sectors in a file based option. I guess you can also compress the sectors individually and think of the sectors as files in an IFFL. One IFFL file equals a sector. This is an ugly bitch but could be counted as a firstie.
Any other thought on this technical challenge? I must admit I am growing fond of the REU option, and the firstie restriction is the only thing that holds me back. The Tink games we just released had been perfect in REU version. Would have saved SO much work, loading would have been near instant and it would have been a release of two neat files.
Am I missing any options or can someone provide some lateral thinking, that opens up new options by finding approaches I have missed? |
|
... 36 posts hidden. Click here to view all posts....
|
| |
MagerValp
Registered: Dec 2001
Posts: 1078 |
Quoting Bacchus400 files means 1200 bytes tables. Track, sector and offset.
Strictly speaking you need less than 10 bits to store the track and sector, if you can spare a few bytes for a small div routine. Even so, that's 900 bytes for the tables, which is a lot.
I'm really liking ChristopherJam's idea of using 1k clusters and three bytes of offsets before the first compressed sector. You still need to read the full compressed cluster, but you don't need more than a single 256 byte buffer in RAM since you can skip the data you don't want. Performance wise I don't think there would be a large hit from using 1k clusters, since it's a relatively small time compared to drive seeking and decrunching the data. |
| |
Bacchus
Registered: Jan 2002
Posts: 156 |
Arhgh. Big reply didn't stick. Recreation of the core parts: :-(
@tim I hear you - our front end guy got snowed over and can't do it now. It's still our ambition to do it...
@jackasser
# Files vis direct T/S
If data is stored in T/S format, you have a number of advantages: You have the full 256 bytes and the data is where you left it. Using files, data "moves around". Where on the disk the data file ends up is different per disk depending in the order you copy the files. Add a little note to a disk before you copy your game then the T/S used is totally different than if you copy it directly to the disk without the note. Also, interleaving is different between drive kernals. ProfessionalDOS doesn't use 10. I also think Jiffy goes for less. Again - with files, you can make zero assumptions on where data is.
You hence need to scan the IFFL file and make tables, as there is no way to programatically tell where the stuff is. That luxury is reserved to the static T/S environment. You typically latch the scanner before the intro so it can do it's job before while you watch the intro.
# Available memory
Your loader + depacker takes memory. If you're lucky, it fits where the old loader was. Otherwise you need to find RAM in the game which is most often a real challenge.
In the ideal IFFL case, the loader and tables fit in drive ram. No need to fiddle with computer ram. Works fine for scenarios where the 2k of drive ram is good for tables, code, sector buffer and all else you need there. Typically you hit a brick wall at around 128 files depending on how compact the loader is. @MagerValp migrated the tables to the computer in his Uload. Works fine if you have the RAM in the computer.
It all boils down to tradeoffs, and my main question was around the scenario of 256+ files and quite restricted memory.
Clustering files gives you the benefit of smaller tables at the cost of slightly deteriorated average loading times (small cost!) |
| |
Perplex
Registered: Feb 2009
Posts: 255 |
When the files are stored on disk, tracks and sectors are determined by some kind of logic, not chosen randomly. If you implement this same logic in native code, all you need is table with sizes for each file, and you can calculate t/s and offset from that, right? |
| |
Count Zero
Registered: Jan 2003
Posts: 1932 |
No.
That "logic" is too easily influenced by file copiers, modified drive ROMs (speeders), etc. |
| |
Perplex
Registered: Feb 2009
Posts: 255 |
Won't those break your tables whether they are precalculated or not? |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
Quote: Won't those break your tables whether they are precalculated or not?
Not if you build them by scanning the file when the game first loads, as Bacchus mentioned above.
Interesting challenge, actually - read all the track & sector links in a file in as little time as possible and store a compact representation in drive ram that can be used for fast random seeking… Probably safe to assume there's usually only a new track every ~20 blocks. |
| |
Bacchus
Registered: Jan 2002
Posts: 156 |
So, scanning is needed.
Storing tables is needed.
And preserving memory is also needed. |
| |
Martin Piper
Registered: Nov 2007
Posts: 722 |
It's these little puzzles that keep programming fun. :) |
| |
Maxlide
Registered: Apr 2003
Posts: 31 |
Quote: It's these little puzzles that keep programming fun. :)
...or let you bite into your table :) |
| |
chatGPZ
Registered: Dec 2001
Posts: 11386 |
Quote:Strictly speaking you need less than 10 bits to store the track and sector, if you can spare a few bytes for a small div routine. Even so, that's 900 bytes for the tables, which is a lot.
when you are dealing with 400 chunks of 256 bytes you dont need to put the offset into the table (you can calculate it easily and fast) |
| Previous - 1 | 2 | 3 | 4 | 5 - Next |
