| |
Shadow
Account closed
Registered: Apr 2002
Posts: 355 |
Load first decrunch later or stream decrunch?
Work on my first attempt on an IRQ-loading demo continues. I am using Dreamload, and everything is working great.
However, some parts tend to be rather big and eat both diskspace and load time.
Obvious solution would of course be to compress the data.
As I see it, there should be two options.
1. Load entire compressed part, then decompress.
2. Load byte-by-byte and stream-decompress
At the moment I am leaning towards solution 1. To get this working, the unpacking must allow for overlapping between packed and unpacked data, since I don't have space for both obviously. But I guess a smart decompressor works 'in reverse' so to speak, so overlapping should not be a problem as long as unpacked data is larger than packed...
I have looked at Exomizer, and it seems like it does things that way, and the decompressor code is fairly compact, so it could be a way to go.
Option 2 I have not looked into as much. Dreamload does support a callback on byte-for-byte basis, so it should be possible I guess.
So, I ask all veterans in this field - how is it usually done? Any tips and general good ideas? |
|
... 59 posts hidden. Click here to view all posts....
|
| |
Ninja
Registered: Jan 2002
Posts: 411 |
Loading and depacking 50 blocks in 11 seconds sounds indeed quite slow. Is there something happening on the screen while loading? What interleave did you use? |
| |
Shadow
Account closed
Registered: Apr 2002
Posts: 355 |
I have a 2x music player running, as well as a little something else. So raster IRQ kicks in 4 times per frame. Not sure exactly how much CPU time I'm using though, but I'm guessing less than $20 lines total.
HCL: Are you crazy man, the test data is one of the effect from my new demo, you would be able to rip it, release it right now and get all the glory!!! :D
Just kidding, I'd be glad to provide the file if you want to do some performance testing. However, as I actually do have some stuff running during loading, so any time comparisons would be a bit unfair. Though your solution is probably much faster anyway, no doubt.
-edit- I'm creating the disk with 'makedisk.exe' and I have set the interleave at 10. Not sure why, I think it was recommended in one of my earlier threads where I asked advice about IRQ-loading! :)
|
| |
HCL
Registered: Feb 2003
Posts: 728 |
"interleave at 10".. hmm, funny choice. Well, in that case an integrated "loader and decruncher at the same time" would probably help you alot. This technique is very forgiving when choosing non-optimal sector interleave, since all otherwise wasted time is used for goodie goodie decrunch :).
Never mind, i still think your stuff runs better than many other of todays demos. Only the fact that you're asking shows that you have some clues :). ..and sure, i can wait with the benchmarking until you've released the demo :). |
| |
Shadow
Account closed
Registered: Apr 2002
Posts: 355 |
I have converted all parts now to 'load-then-deexomize' and saved a whole bunch of space! The previous build of the demo had 131 blocks free, the new has 432. Guess I need to code more parts! :D |
| |
Ninja
Registered: Jan 2002
Posts: 411 |
Yeah, Exomizer can be frustrating :D
Try lowering the interleave, that may help a bit (Yeah, I hear you, guys: "Depack while waiting!" "Load the sectors as they come!"...) |
| |
HCL
Registered: Feb 2003
Posts: 728 |
Agree, i give up my propaganda ;). It sounds like Interleave 6 would work fine in this case. If you have different loading parts with different CPU-load, perhaps 8 is more safe. At 10 you must have really really much shit going on to miss the next sector. |
| |
Shadow
Account closed
Registered: Apr 2002
Posts: 355 |
I tested with interleave 6 instead of 10, but then the load time for the largest part (53 blocks now) went from 5.7 to 13.9 seconds...?!?
-edit- I am not suggesting that HCL is giving bad advice, more likely it is my stuff that is weird :) |
| |
Frantic
Registered: Mar 2003
Posts: 1648 |
Thou shalt never suggest that HCL is giving bad advice. |
| |
HCL
Registered: Feb 2003
Posts: 728 |
@Shadow: Didn't you say your shit needs less than $20 rasterlines!? If that's true, it's really weird :). |
| |
Danzig
Registered: Jun 2002
Posts: 440 |
"shit wasting $20 rasterlines" is a pretty huge "sausage" by the way *puke* just imagine, a screen wide and 32 pixels high bruchwurst... |
| Previous - 1 | 2 | 3 | 4 | 5 | 6 | 7 - Next |
