| |
Krill
Registered: Apr 2002
Posts: 2844 |
Release id #214940 : TSCrunch
Quoting tonysavon"TSCrunch is an optimal, byte-aligned, LZ+RLE hybrid encoder, designed to maximize decoding speed on NMOS 6502 and derived CPUs, while achieving decent compression ratio (for a bytecruncher, that is). It crunches as well as other popular bytecrunchers, while being considerably faster at decrunching."
[...]
TSCrunch is a bytepacker, with 2byte RLE, 1-2byte LZ tokens and a 512 bytes search window. In this "space" it provides the optimal solution to the puzzle. Exomizer is s different beast, being a bit-cruncher. According to these specs, i'd expect it to fall somewhere into the 60% cluster in the graph below, from https://codebase64.org/doku.php?id=base:compression_benchmarks.
If it is made for in-memory decompression of data, can it also work with prefix data for better compression, i.e., back-referencing to data already in memory (either a static dictionary or, e.g., the previous level)?
And i can't quite follow what "bit-cruncher" vs "byte-cruncher" means. :)
IIRC, Exomizer works on byte-aligned source data as well, also with LZ and RLE.
It produces a bit-stream on the control symbols, though, which is interleaved with byte-aligned literals.
A "bit-cruncher" might maybe be something LZMA- or ANS-like (think ALZ64, or Shrinkler on Amiga), but not Exomizer.
 |
|
... 31 posts hidden. Click here to view all posts....
|
| |
tonysavon
Registered: Apr 2014
Posts: 24 |
Well that would be great and make things much more streamlined on the demo coder side. To offer that as a capability from the loader would be great, but inevitably it would tie you to a specific compression schema, while coders can probably want to experiment on their own with different approaches (and for this having the loader and the codec decoupled is the best option, really). For instance, for petscii animations you don't need codebooks at all and if you have an LZ search window that is longer than 1kb, basically your everyday cruncher becomes the quickest (to implement) video encoder you can think of, with some minor modification (basically matches happen in a circular buffer rather than in plain memory). Je t'aime mon Monstre was done like that (even if it did not stream from disk but from memory). Basically "just" an endless decruncher that would use previous 1000 bytes to decode (or predict, in compression language) the following 1000. Each of them being a video frame.
Je t'aime mon Monstre
But if you had to hardwire a codec I think VQ + delta would be amazing and address a wide range of effects. Also it would not be that difficult to implement on the loader side, it would basically need to loadUncompressed the codecs and then stream the delta. Not difficult for someone who can code fasloaders, but me... I wouldn't know where to start from :-P |
| |
Krill
Registered: Apr 2002
Posts: 2844 |
For all of that, all you need is basically an exposed pollblock call, which would return "try again later" or "block with index #N is ready to download", plus a getblock call which would receive that block and put it to the desired location, no? =)
(Bonus: a skipblock call that would be called if the block is deemed stale, and just discards it without transferring.) |
| |
tonysavon
Registered: Apr 2014
Posts: 24 |
Quote: For all of that, all you need is basically an exposed pollblock call, which would return "try again later" or "block with index #N is ready to download", plus a getblock call which would receive that block and put it to the desired location, no? =)
(Bonus: a skipblock call that would be called if the block is deemed stale, and just discards it without transferring.)
Sounds like a plan!!! :-) |
| |
Krill
Registered: Apr 2002
Posts: 2844 |
Quoting tonysavonSounds like a plan!!! :-) Yeah, well, such a poll-block API existed in earlier versions, but nobody ever really used it. So it was uninvented. Can be re-added any time, of course, should anyone need it. |
| |
tonysavon
Registered: Apr 2014
Posts: 24 |
OK, I think I came up with a simple way to increase compression rate sensibly without affecting the decrunching speed (too much). I just did some preliminary tests and it seems to work nicely, but the decruncher code becomes a bit too large for my liking, at $e8 bytes for the inplace decruncher. I'll hammer it a bit more and see where this takes me, but if throughput stays roughly the same and compression rate increases consistently then it should perform much better in krill's loader. Let's see. thanks for all your help! |
| |
Krill
Registered: Apr 2002
Posts: 2844 |
Quoting tonysavonbut the decruncher code becomes a bit too large for my liking More than $30 bytes more than the earlier version? :)
Because the loader with TS-decruncher is $01d0 bytes currently. |
| |
tonysavon
Registered: Apr 2014
Posts: 24 |
Quote: Quoting tonysavonbut the decruncher code becomes a bit too large for my liking More than $30 bytes more than the earlier version? :)
Because the loader with TS-decruncher is $01d0 bytes currently.
yes, unfortunately. $33 bytes.
Are you using the inplace or not inplace version, btw?
Previous version was
$95 bytes, now $c8 for non-INPLACE decruncher
$b0 bytes, now $ee for the INPLACE decruncher
This is for the versions optimised for speed though, and I can save $10 bytes disabling speed optimisations, but that would be a bit pointless :-)
I must shuffle the code a bit more, but it'll be worth it :-) |
| |
Krill
Registered: Apr 2002
Posts: 2844 |
Using in-place version, of course. =) |
| |
tonysavon
Registered: Apr 2014
Posts: 24 |
Quote: Using in-place version, of course. =)
Figured :-P
Ok, I'll make it work within tomorrow for sure! |
| |
tonysavon
Registered: Apr 2014
Posts: 24 |
Version 1.2 is now available, which adds inplace decrunching and improves a bit on everything.
TSCrunch 1.2 |
| Previous - 1 | 2 | 3 | 4 | 5 - Next |
