| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
Nucrunch 0.1
Continuing from the benchmarks WVL posted in Doynamite 1.x:
I dusted off my unfinished nucrunch in December to pack just enough of the second page of Reutastic to give me some workspace for some precalculations. Pity I didn't schedule enough time to pack the entire demo, else it would have been ~90 blocks instead of 190, but I digress. I've spent bits of the past month cleaning up the code, optimizing the packer (mostly by porting it from python to rust :P), and adding reverse direction support.
It's still no more than a component, with an commandline packer and asm decrunch subroutine, but no tools yet for generating an executable from a single commandline. It does at least now support multiple input segments that are unpacked to their destination addresses, and it's also now useable enough to for me to do some benchmarking.
In short, doynamite's ratio looks pretty unbeatable for anything lz based; my ratio's almost identical despite a somewhat different encoding.
Where I can win though is speed at that ratio; nucrunch is usually ten to twenty percent faster. The one exception in the test corpus is 6.bin, where it's 20% slower; not sure why yet.
I've added the times for pucrunch -ffast below for to complete the comparison. Last two columns are nucrunch, and nucrunch -r (the latter decodes in reverse; should be a more useful component for single filers)
If anyone wants to have a play at this stage, poke me and I'll upload some source. Failing that I'll hold off until I at least have something that can make onefilers without any faffing about with relocating the last couple of pages by hand.
filesizes
# bin rle wvl-f wvl-s tc bb pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1 11008 8020 4529 4151 4329 3383 3711 3265 3225 3230
2 4973 4314 3532 3309 3423 2648 3005 2512 2498 2490
3 3949 3498 2991 2617 2972 2187 2530 2108 2091 2093
4 7016 6456 4242 4085 4225 3681 3924 3617 3622 3614
5 34760 27647 25781 24895 25210 21306 21182 20405 20447 20516
6 31605 12511 11283 10923 11614 9194 9203 8904 8915 8894
7 20392 17295 12108 11285 11445 9627 9789 9289 9140 9144
8 5713 5407 4179 3916 3936 3251 3656 3132 3165 3187
9 8960 7986 6914 6896 6572 5586 6000 5430 5502 5486
filesize in %
# bin rle wvl-f wvl-s tc bb pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1 100 72.9 41.1 37.7 39.3 30.7 33.7 29.7 29.3 29.3
2 100 86.7 71.0 66.5 68.8 53.2 60.4 50.5 50.2 50.1
3 100 88.6 75.7 66.3 75.3 55.4 64.1 53.4 53.0 53.0
4 100 92.0 60.5 58.2 60.2 52.5 55.9 51.6 51.6 51.5
5 100 79.5 74.2 71.6 72.5 61.3 60.9 58.7 58.8 59.0
6 100 39.6 35.7 34.6 36.7 29.1 29.1 28.2 28.2 28.1
7 100 84.8 59.4 55.3 56.1 47.2 48.0 45.6 44.8 44.8
8 100 94.6 73.1 68.5 68.9 56.9 64.0 54.8 55.4 55.8
9 100 89.1 77.2 77.0 73.3 62.3 67.0 60.6 61.4 61.2
number of frames to depack
# bin rle wvl-f wvl-s tc bb pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1 0 11 13 14 15 58 54 27 22 22
2 0 5 7 7 9 38 39 17 14 14
3 0 4 6 6 7 28 31 12 10 10
4 0 8 9 9 10 43 51 20 17 18
5 0 36 39 42 59 300 298 119 104 107
6 0 20 25 25 37 126 152 49 59 59
7 0 22 25 26 32 138 139 60 51 52
8 0 6 8 8 10 43 47 18 16 17
9 0 9 12 12 16 73 81 32 28 29
kilobytes output per second
# bin rle wvl-f wvl-s tc bb pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1 49.0 41.4 38.5 35.9 9.3 10.0 20.0 24.5 24.5
2 48.7 34.8 34.8 27.0 6.4 6.2 14.3 17.4 17.4
3 48.3 32.2 32.2 27.6 6.9 6.2 16.1 19.3 19.3
4 42.9 38.2 38.2 34.3 8.0 6.7 17.2 20.2 19.1
5 47.3 43.6 40.5 28.8 5.7 5.7 14.3 16.4 15.9
6 77.4 61.9 61.9 41.8 12.3 10.2 31.6 26.2 26.2
7 45.4 39.9 38.4 31.2 7.2 7.2 16.6 19.6 19.2
8 46.6 35.0 35.0 28.0 6.5 6.0 15.5 17.5 16.5
9 48.7 36.5 36.5 27.4 6.0 5.4 13.7 15.7 15.1
cycles per byte consumed
# bin rle wvl-f wvl-s tc bb pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1 0 27 56 66 68 337 286 163 134 134
2 0 23 39 42 52 282 255 133 110 111
3 0 22 39 45 46 252 241 112 94 94
4 0 24 42 43 47 230 255 109 92 98
5 0 26 30 33 46 277 277 115 100 103
6 0 31 44 45 63 269 325 108 130 130
7 0 25 41 45 55 282 279 127 110 112
8 0 22 38 40 50 260 253 113 99 105
9 0 22 34 34 48 257 265 116 100 104
decrunch time for nucrunch/rnucrunch relative to doynamite
1: 81.5% (-18.5%) 81.5% (-18.5%)
2: 82.4% (-17.6%) 82.4% (-17.6%)
3: 83.3% (-16.7%) 83.3% (-16.7%)
4: 85.0% (-15.0%) 90.0% (-10.0%)
5: 87.4% (-12.6%) 89.9% (-10.1%)
6: 120.4% ( 20.4%) 120.4% ( 20.4%)
7: 85.0% (-15.0%) 86.7% (-13.3%)
8: 88.9% (-11.1%) 94.4% ( -5.6%)
9: 87.5% (-12.5%) 90.6% ( -9.4%)
|
|
... 95 posts hidden. Click here to view all posts....
|
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
np. Raw file compressed only. (Or at least, that's what I was assuming for rle, wvl, bb & doyna, and it's what I measured for nucrunch, tinycrunch and pucrunch) |
| |
Martin Piper
Registered: Nov 2007
Posts: 726 |
Oh cool. :) I also get similar compression results to yours with my old LZ based code. The circular recent history buffer (updateHistoryBuffer) that stores length and offset helps squeeze more compression at the expense of decompression speed.
Current LZMPi results:
1 3,184
2 2,410
3 1,931
4 3,571
5 20,362
6 8,719
7 9,256
8 3,048
9 5,563
https://github.com/martinpiper/C64Public/tree/master/Compression
https://github.com/martinpiper/C64Public/tree/master/Decompress.. |
| |
MagerValp
Registered: Dec 2001
Posts: 1078 |
For funsies here's the result from LZMV+RLE with a 4k window:
# bin rle wvl-f wvl-s tc bb pu-f doyna nucru rnucr LZMV4k
1 100 72.9 41.1 37.7 39.3 30.7 33.7 29.7 29.3 29.3 38.2
2 100 86.7 71.0 66.5 68.8 53.2 60.4 50.5 50.2 50.1 64.0
3 100 88.6 75.7 66.3 75.3 55.4 64.1 53.4 53.0 53.0 64.6
4 100 92.0 60.5 58.2 60.2 52.5 55.9 51.6 51.6 51.5 61.9
5 100 79.5 74.2 71.6 72.5 61.3 60.9 58.7 58.8 59.0 68.6
6 100 39.6 35.7 34.6 36.7 29.1 29.1 28.2 28.2 28.1 33.6
7 100 84.8 59.4 55.3 56.1 47.2 48.0 45.6 44.8 44.8 54.7
8 100 94.6 73.1 68.5 68.9 56.9 64.0 54.8 55.4 55.8 69.3
9 100 89.1 77.2 77.0 73.3 62.3 67.0 60.6 61.4 61.2 72.6
https://github.com/MagerValp/u4remastered/blob/master/tools/bac..
https://github.com/MagerValp/u4remastered/blob/master/src/u4loa.. |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
OK, got some updated benchmarks. Still no exo or ALZ64, but I've updated ByteBoozer to version 2.0, optimised nucrunch a little, and I've switched to counting cycles for timing BB, Pu & Nu. I've added half a frame to all the other times on the assumption that they were originally all rounded down.
I've also included sizes for LZMPi and ~LZMV4k (thanks guys!), no timings for those yet. I gather bitnax/doynax is a little faster now, too; update yet to come.
filesizes
# bin rle wvl-f wvl-s tc bb2.0 pu-f doyna nucru rnucr LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1 11008 8020 4529 4151 4329 3322 3711 3265 3225 3230 3184 4205
2 4973 4314 3532 3309 3423 2513 3005 2512 2498 2490 2410 3183
3 3949 3498 2991 2617 2972 2098 2530 2108 2091 2093 1931 2551
4 7016 6456 4242 4085 4225 3682 3924 3617 3622 3614 3571 4343
5 34760 27647 25781 24895 25210 20530 21182 20405 20447 20516 20362 23845
6 31605 12511 11283 10923 11614 8998 9203 8904 8915 8896 8719 10619
7 20392 17295 12108 11285 11445 9241 9789 9289 9140 9145 9256 11154
8 5713 5407 4179 3916 3936 3165 3656 3132 3166 3187 3048 3959
9 8960 7986 6914 6896 6572 5491 6000 5430 5502 5487 5563 6505
filesize in %
# bin rle wvl-f wvl-s tc bb2.0 pu-f doyna nucru rnucr LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1 100.0 72.9 41.1 37.7 39.3 30.2 33.7 29.7 29.3 29.3 28.9 38.2
2 100.0 86.7 71.0 66.5 68.8 50.5 60.4 50.5 50.2 50.1 48.5 64.0
3 100.0 88.6 75.7 66.3 75.3 53.1 64.1 53.4 53.0 53.0 48.9 64.6
4 100.0 92.0 60.5 58.2 60.2 52.5 55.9 51.6 51.6 51.5 50.9 61.9
5 100.0 79.5 74.2 71.6 72.5 59.1 60.9 58.7 58.8 59.0 58.6 68.6
6 100.0 39.6 35.7 34.6 36.7 28.5 29.1 28.2 28.2 28.1 27.6 33.6
7 100.0 84.8 59.4 55.3 56.1 45.3 48.0 45.6 44.8 44.8 45.4 54.7
8 100.0 94.6 73.1 68.5 68.9 55.4 64.0 54.8 55.4 55.8 53.4 69.3
9 100.0 89.1 77.2 77.0 73.3 61.3 67.0 60.6 61.4 61.2 62.1 72.6
number of frames to depack
# bin rle wvl-f wvl-s tc bb2.0 pu-f doyna nucru rnucr LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1 0.0 11.5 13.5 14.5 15.5 46.1 54.7 27.5 22.0 21.3
2 0.0 5.5 7.5 7.5 9.5 31.2 39.6 17.5 13.7 13.6
3 0.0 4.5 6.5 6.5 7.5 22.7 31.9 12.5 10.0 10.1
4 0.0 8.5 9.5 9.5 10.5 37.5 52.0 20.5 16.6 17.1
5 0.0 36.5 39.5 42.5 59.5 222.2 298.6 119.5 101.4 100.6
6 0.0 20.5 25.5 25.5 37.5 112.3 152.8 49.5 57.9 56.6
7 0.0 22.5 25.5 26.5 32.5 109.6 139.8 60.5 50.3 50.0
8 0.0 6.5 8.5 8.5 10.5 35.4 47.7 18.5 15.6 15.8
9 0.0 9.5 12.5 12.5 16.5 62.3 81.7 32.5 27.1 27.2
kilobytes output per second
# bin rle wvl-f wvl-s tc bb2.0 pu-f doyna nucru rnucr LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1 46.9 39.9 37.2 34.8 11.7 9.9 19.6 24.5 25.3
2 44.3 32.5 32.5 25.6 7.8 6.1 13.9 17.7 17.9
3 43.0 29.7 29.7 25.8 8.5 6.1 15.5 19.4 19.2
4 40.4 36.2 36.2 32.7 9.2 6.6 16.8 20.6 20.1
5 46.6 43.1 40.0 28.6 7.7 5.7 14.2 16.8 16.9
6 75.5 60.7 60.7 41.3 13.8 10.1 31.3 26.7 27.3
7 44.4 39.1 37.7 30.7 9.1 7.1 16.5 19.8 20.0
8 43.0 32.9 32.9 26.6 7.9 5.9 15.1 18.0 17.7
9 46.2 35.1 35.1 26.6 7.0 5.4 13.5 16.2 16.1
cycles per byte consumed
# bin rle wvl-f wvl-s tc bb2.0 pu-f doyna nucru rnucr LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1 0.0 28.2 58.6 68.7 70.4 272.9 289.7 165.6 134.2 129.4
2 0.0 25.1 41.7 44.6 54.6 244.4 259.3 136.9 108.0 107.5
3 0.0 25.3 42.7 48.8 49.6 212.3 247.9 116.6 93.8 94.6
4 0.0 25.9 44.0 45.7 48.8 200.3 260.4 111.4 90.3 93.1
5 0.0 26.0 30.1 33.6 46.4 212.8 277.1 115.1 97.5 96.4
6 0.0 32.2 44.4 45.9 63.5 245.3 326.3 109.3 127.7 125.1
7 0.0 25.6 41.4 46.2 55.8 233.2 280.7 128.0 108.2 107.4
8 0.0 23.6 40.0 42.7 52.4 219.9 256.4 116.1 96.6 97.2
9 0.0 23.4 35.5 35.6 49.3 223.0 267.7 117.6 96.9 97.4
|
| |
Martin Piper
Registered: Nov 2007
Posts: 726 |
Are you timing with VICE stopwatch to get the cycle times? With the screen off? |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
Chained DC00 timers (based on a routine from Glasnost) and blanked screen for nucrunch
VICE stopwatch for BB & pu, not blanked; which in the light of day seems a tad unfair - forgot that at 3am.
I need to either translate BB or install the assembler HCL uses so I can build it into my testbed instead of just running the produced executable. |
| |
HCL
Registered: Feb 2003
Posts: 728 |
Hmm, compression rate of B2 was something like expected.. but performance was not that impressive, only 15-25% faster than before. I guess i was comparing to original bb, and you probably had the latest optimized decruncher. Never mind, thanx for adding it so quickly!! |
| |
MagerValp
Registered: Dec 2001
Posts: 1078 |
I added exoraw 2.0.9 with default options:
filesizes
# bin rle wvl-f wvl-s tc bb2.0 pu-f doyna nucru rnucr LZMPi LZMV4k exoraw
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1 11008 8020 4529 4151 4329 3322 3711 3265 3225 3230 3184 4205 2988
2 4973 4314 3532 3309 3423 2513 3005 2512 2498 2490 2410 3183 2241
3 3949 3498 2991 2617 2972 2098 2530 2108 2091 2093 1931 2551 1817
4 7016 6456 4242 4085 4225 3682 3924 3617 3622 3614 3571 4343 3454
5 34760 27647 25781 24895 25210 20530 21182 20405 20447 20516 20362 23845 19631
6 31605 12511 11283 10923 11614 8998 9203 8904 8915 8896 8719 10619 8337
7 20392 17295 12108 11285 11445 9241 9789 9289 9140 9145 9256 11154 8751
8 5713 5407 4179 3916 3936 3165 3656 3132 3166 3187 3048 3959 3059
9 8960 7986 6914 6896 6572 5491 6000 5430 5502 5487 5563 6505 5295
filesize in %
# bin rle wvl-f wvl-s tc bb2.0 pu-f doyna nucru rnucr LZMPi LZMV4k exoraw
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1 100.0 72.9 41.1 37.7 39.3 30.2 33.7 29.7 29.3 29.3 28.9 38.2 27.1
2 100.0 86.7 71.0 66.5 68.8 50.5 60.4 50.5 50.2 50.1 48.5 64.0 45.1
3 100.0 88.6 75.7 66.3 75.3 53.1 64.1 53.4 53.0 53.0 48.9 64.6 46.0
4 100.0 92.0 60.5 58.2 60.2 52.5 55.9 51.6 51.6 51.5 50.9 61.9 49.2
5 100.0 79.5 74.2 71.6 72.5 59.1 60.9 58.7 58.8 59.0 58.6 68.6 56.5
6 100.0 39.6 35.7 34.6 36.7 28.5 29.1 28.2 28.2 28.1 27.6 33.6 26.4
7 100.0 84.8 59.4 55.3 56.1 45.3 48.0 45.6 44.8 44.8 45.4 54.7 42.9
8 100.0 94.6 73.1 68.5 68.9 55.4 64.0 54.8 55.4 55.8 53.4 69.3 53.5
9 100.0 89.1 77.2 77.0 73.3 61.3 67.0 60.6 61.4 61.2 62.1 72.6 59.1
For a graph and results sorted by average file size: https://www.icloud.com/numbers/00060NSRmk5sdCbLjtye-UGCA#LZMV_B.. |
| |
Martin Piper
Registered: Nov 2007
Posts: 726 |
Very cool compression with exoraw. Something to strive towards. |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
Impressively small! Any chance of cycle timings for the new additions? |
| Previous - 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 - Next |
