I'll just go for the streaming exomizer decompressor then.

Btw., there'll be two buffers then: One ringbuffer for the compressed data, which decouples fetching blocks from the drive from actually reading them into the decompressor's input stream (with my loader, they come out of order as they pass the r/w head - for speed reasons, of course), and another ringbuffer for the decompressed data which will then be used for the LZ77 back references.

I'll just implement both and then see, after all there are a lot of non-stream decompressors to choose from already in my loader, and more options are always better.

What do you mean by "multiple sector buffers"? The decompressor just calls a get-next-byte routine, and will be fed the data in the correct order.

And yes, my plan is to have a sector buffer the size of two tracks, so the loader can happily read the next track while the stream feeder goes through the current track's blocks and eventually waits when a block is still missing. At this point i don't see any tricky situations to worry about except for deciding if/when to reject a block to be put in place of a yet-unread one - in favour of a possible next block with less overall overhead.

The decompression look-up buffer is easily resizeable and has no constraints regarding the other buffer's size. I don't see a connection to the reading speed though, since the other buffer is already there for that and hands out the next input stream bytes as soon as possible.

Thanks for the source. I guess the vanilla byteboozer decompressor is easier to change for my purposes, though. :)

Both buffers are resizeable, and the track buffer is not actually a track buffer. The blocks are put there in the correct order, and a reader just needs to increment its pointer linearly, wait if there's a gap, and wrap around if the end of the buffer has been reached. I will do some tests anyway to see how large that buffer should be.

Decompression in advance is something that the application can do if it chooses to. The loader itself does decompression on demand and could easily report the amount of continuous input stream bytes remaining in the buffer, so the application can choose whether to have them decompressed now or later.

About race conditions, does your solution work with interrupts? Because otherwise you'd have none of those, i guess. My loader normally doesn't use interrupts, unless you enable the non-blocking API which keeps polling and loading blocks in the background while the main thread does whatever it is supposed to.

I don't know.. This sounds completely fukkedup. But i suppose it's genious in some way :P. However, as long as all this doesn't render a demo in one or another way, it will stay fukkedup in my world ;).

Here is an article Antitrack wrote for the Domination magazine (later published in Recollection).