Log inRegister an accountBrowse CSDbHelp & documentationFacts & StatisticsThe forumsAvailable RSS-feeds on CSDbSupport CSDb Commodore 64 Scene Database
You are not logged in - nap
CSDb User Forums


Forums > C64 Coding > Nucrunch 0.1
2016-02-04 10:02
ChristopherJam

Registered: Aug 2004
Posts: 1359
Nucrunch 0.1

Continuing from the benchmarks WVL posted in Doynamite 1.x:

I dusted off my unfinished nucrunch in December to pack just enough of the second page of Reutastic to give me some workspace for some precalculations. Pity I didn't schedule enough time to pack the entire demo, else it would have been ~90 blocks instead of 190, but I digress. I've spent bits of the past month cleaning up the code, optimizing the packer (mostly by porting it from python to rust :P), and adding reverse direction support.

It's still no more than a component, with an commandline packer and asm decrunch subroutine, but no tools yet for generating an executable from a single commandline. It does at least now support multiple input segments that are unpacked to their destination addresses, and it's also now useable enough to for me to do some benchmarking.

In short, doynamite's ratio looks pretty unbeatable for anything lz based; my ratio's almost identical despite a somewhat different encoding.

Where I can win though is speed at that ratio; nucrunch is usually ten to twenty percent faster. The one exception in the test corpus is 6.bin, where it's 20% slower; not sure why yet.

I've added the times for pucrunch -ffast below for to complete the comparison. Last two columns are nucrunch, and nucrunch -r (the latter decodes in reverse; should be a more useful component for single filers)

If anyone wants to have a play at this stage, poke me and I'll upload some source. Failing that I'll hold off until I at least have something that can make onefilers without any faffing about with relocating the last couple of pages by hand.

filesizes
#   bin   rle wvl-f wvl-s    tc    bb  pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1 11008  8020  4529  4151  4329  3383  3711  3265  3225  3230
2  4973  4314  3532  3309  3423  2648  3005  2512  2498  2490
3  3949  3498  2991  2617  2972  2187  2530  2108  2091  2093
4  7016  6456  4242  4085  4225  3681  3924  3617  3622  3614
5 34760 27647 25781 24895 25210 21306 21182 20405 20447 20516
6 31605 12511 11283 10923 11614  9194  9203  8904  8915  8894
7 20392 17295 12108 11285 11445  9627  9789  9289  9140  9144
8  5713  5407  4179  3916  3936  3251  3656  3132  3165  3187
9  8960  7986  6914  6896  6572  5586  6000  5430  5502  5486

filesize in %
#   bin   rle wvl-f wvl-s    tc    bb  pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1   100  72.9  41.1  37.7  39.3  30.7  33.7  29.7  29.3  29.3
2   100  86.7  71.0  66.5  68.8  53.2  60.4  50.5  50.2  50.1
3   100  88.6  75.7  66.3  75.3  55.4  64.1  53.4  53.0  53.0
4   100  92.0  60.5  58.2  60.2  52.5  55.9  51.6  51.6  51.5
5   100  79.5  74.2  71.6  72.5  61.3  60.9  58.7  58.8  59.0
6   100  39.6  35.7  34.6  36.7  29.1  29.1  28.2  28.2  28.1
7   100  84.8  59.4  55.3  56.1  47.2  48.0  45.6  44.8  44.8
8   100  94.6  73.1  68.5  68.9  56.9  64.0  54.8  55.4  55.8
9   100  89.1  77.2  77.0  73.3  62.3  67.0  60.6  61.4  61.2

number of frames to depack
#   bin   rle wvl-f wvl-s    tc    bb  pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1     0    11    13    14    15    58    54    27    22    22
2     0     5     7     7     9    38    39    17    14    14
3     0     4     6     6     7    28    31    12    10    10
4     0     8     9     9    10    43    51    20    17    18
5     0    36    39    42    59   300   298   119   104   107
6     0    20    25    25    37   126   152    49    59    59
7     0    22    25    26    32   138   139    60    51    52
8     0     6     8     8    10    43    47    18    16    17
9     0     9    12    12    16    73    81    32    28    29

kilobytes output per second
#   bin   rle wvl-f wvl-s    tc    bb  pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1        49.0  41.4  38.5  35.9   9.3  10.0  20.0  24.5  24.5
2        48.7  34.8  34.8  27.0   6.4   6.2  14.3  17.4  17.4
3        48.3  32.2  32.2  27.6   6.9   6.2  16.1  19.3  19.3
4        42.9  38.2  38.2  34.3   8.0   6.7  17.2  20.2  19.1
5        47.3  43.6  40.5  28.8   5.7   5.7  14.3  16.4  15.9
6        77.4  61.9  61.9  41.8  12.3  10.2  31.6  26.2  26.2
7        45.4  39.9  38.4  31.2   7.2   7.2  16.6  19.6  19.2
8        46.6  35.0  35.0  28.0   6.5   6.0  15.5  17.5  16.5
9        48.7  36.5  36.5  27.4   6.0   5.4  13.7  15.7  15.1

cycles per byte consumed
#   bin   rle wvl-f wvl-s    tc    bb  pu-f doyna nucru rnucr
- ----- ----- ----- ----- ----- ----- ----- ----- ----- -----
1     0    27    56    66    68   337   286   163   134   134
2     0    23    39    42    52   282   255   133   110   111
3     0    22    39    45    46   252   241   112    94    94
4     0    24    42    43    47   230   255   109    92    98
5     0    26    30    33    46   277   277   115   100   103
6     0    31    44    45    63   269   325   108   130   130
7     0    25    41    45    55   282   279   127   110   112
8     0    22    38    40    50   260   253   113    99   105
9     0    22    34    34    48   257   265   116   100   104

decrunch time for nucrunch/rnucrunch relative to doynamite
1:  81.5% (-18.5%)  81.5% (-18.5%)
2:  82.4% (-17.6%)  82.4% (-17.6%)
3:  83.3% (-16.7%)  83.3% (-16.7%)
4:  85.0% (-15.0%)  90.0% (-10.0%)
5:  87.4% (-12.6%)  89.9% (-10.1%)
6: 120.4% ( 20.4%) 120.4% ( 20.4%)
7:  85.0% (-15.0%)  86.7% (-13.3%)
8:  88.9% (-11.1%)  94.4% ( -5.6%)
9:  87.5% (-12.5%)  90.6% ( -9.4%)
 
... 92 posts hidden. Click here to view all posts....
 
2016-02-07 22:57
Martin Piper

Registered: Nov 2007
Posts: 629
I couldn't find the test files along with the code used to replicate the tests?
2016-02-08 05:55
ChristopherJam

Registered: Aug 2004
Posts: 1359
@Martin, I grabbed the test files from LZWVL (bin.rar)
2016-02-08 09:02
Martin Piper

Registered: Nov 2007
Posts: 629
Quote: @Martin, I grabbed the test files from LZWVL (bin.rar)

Cheers :)
The number of bytes in the top table, is that the raw file compressed only? No self extracting code?
2016-02-08 09:51
ChristopherJam

Registered: Aug 2004
Posts: 1359
np. Raw file compressed only. (Or at least, that's what I was assuming for rle, wvl, bb & doyna, and it's what I measured for nucrunch, tinycrunch and pucrunch)
2016-02-08 11:35
Martin Piper

Registered: Nov 2007
Posts: 629
Oh cool. :) I also get similar compression results to yours with my old LZ based code. The circular recent history buffer (updateHistoryBuffer) that stores length and offset helps squeeze more compression at the expense of decompression speed.

Current LZMPi results:
1 3,184
2 2,410
3 1,931
4 3,571
5 20,362
6 8,719
7 9,256
8 3,048
9 5,563

https://github.com/martinpiper/C64Public/tree/master/Compression
https://github.com/martinpiper/C64Public/tree/master/Decompress..
2016-02-08 13:12
MagerValp

Registered: Dec 2001
Posts: 1055
For funsies here's the result from LZMV+RLE with a 4k window:

#   bin   rle    wvl-f  wvl-s  tc     bb     pu-f   doyna  nucru  rnucr  LZMV4k
1   100   72.9   41.1   37.7   39.3   30.7   33.7   29.7   29.3   29.3   38.2
2   100   86.7   71.0   66.5   68.8   53.2   60.4   50.5   50.2   50.1   64.0
3   100   88.6   75.7   66.3   75.3   55.4   64.1   53.4   53.0   53.0   64.6
4   100   92.0   60.5   58.2   60.2   52.5   55.9   51.6   51.6   51.5   61.9
5   100   79.5   74.2   71.6   72.5   61.3   60.9   58.7   58.8   59.0   68.6
6   100   39.6   35.7   34.6   36.7   29.1   29.1   28.2   28.2   28.1   33.6
7   100   84.8   59.4   55.3   56.1   47.2   48.0   45.6   44.8   44.8   54.7
8   100   94.6   73.1   68.5   68.9   56.9   64.0   54.8   55.4   55.8   69.3
9   100   89.1   77.2   77.0   73.3   62.3   67.0   60.6   61.4   61.2   72.6

https://github.com/MagerValp/u4remastered/blob/master/tools/bac..
https://github.com/MagerValp/u4remastered/blob/master/src/u4loa..
2016-02-08 22:17
ChristopherJam

Registered: Aug 2004
Posts: 1359
OK, got some updated benchmarks. Still no exo or ALZ64, but I've updated ByteBoozer to version 2.0, optimised nucrunch a little, and I've switched to counting cycles for timing BB, Pu & Nu. I've added half a frame to all the other times on the assumption that they were originally all rounded down.

I've also included sizes for LZMPi and ~LZMV4k (thanks guys!), no timings for those yet. I gather bitnax/doynax is a little faster now, too; update yet to come.

filesizes
#    bin    rle  wvl-f  wvl-s     tc  bb2.0   pu-f  doyna  nucru  rnucr  LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1  11008   8020   4529   4151   4329   3322   3711   3265   3225   3230   3184   4205
2   4973   4314   3532   3309   3423   2513   3005   2512   2498   2490   2410   3183
3   3949   3498   2991   2617   2972   2098   2530   2108   2091   2093   1931   2551
4   7016   6456   4242   4085   4225   3682   3924   3617   3622   3614   3571   4343
5  34760  27647  25781  24895  25210  20530  21182  20405  20447  20516  20362  23845
6  31605  12511  11283  10923  11614   8998   9203   8904   8915   8896   8719  10619
7  20392  17295  12108  11285  11445   9241   9789   9289   9140   9145   9256  11154
8   5713   5407   4179   3916   3936   3165   3656   3132   3166   3187   3048   3959
9   8960   7986   6914   6896   6572   5491   6000   5430   5502   5487   5563   6505

filesize in %
#    bin    rle  wvl-f  wvl-s     tc  bb2.0   pu-f  doyna  nucru  rnucr  LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1  100.0   72.9   41.1   37.7   39.3   30.2   33.7   29.7   29.3   29.3   28.9   38.2
2  100.0   86.7   71.0   66.5   68.8   50.5   60.4   50.5   50.2   50.1   48.5   64.0
3  100.0   88.6   75.7   66.3   75.3   53.1   64.1   53.4   53.0   53.0   48.9   64.6
4  100.0   92.0   60.5   58.2   60.2   52.5   55.9   51.6   51.6   51.5   50.9   61.9
5  100.0   79.5   74.2   71.6   72.5   59.1   60.9   58.7   58.8   59.0   58.6   68.6
6  100.0   39.6   35.7   34.6   36.7   28.5   29.1   28.2   28.2   28.1   27.6   33.6
7  100.0   84.8   59.4   55.3   56.1   45.3   48.0   45.6   44.8   44.8   45.4   54.7
8  100.0   94.6   73.1   68.5   68.9   55.4   64.0   54.8   55.4   55.8   53.4   69.3
9  100.0   89.1   77.2   77.0   73.3   61.3   67.0   60.6   61.4   61.2   62.1   72.6

number of frames to depack
#    bin    rle  wvl-f  wvl-s     tc  bb2.0   pu-f  doyna  nucru  rnucr  LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1    0.0   11.5   13.5   14.5   15.5   46.1   54.7   27.5   22.0   21.3
2    0.0    5.5    7.5    7.5    9.5   31.2   39.6   17.5   13.7   13.6
3    0.0    4.5    6.5    6.5    7.5   22.7   31.9   12.5   10.0   10.1
4    0.0    8.5    9.5    9.5   10.5   37.5   52.0   20.5   16.6   17.1
5    0.0   36.5   39.5   42.5   59.5  222.2  298.6  119.5  101.4  100.6
6    0.0   20.5   25.5   25.5   37.5  112.3  152.8   49.5   57.9   56.6
7    0.0   22.5   25.5   26.5   32.5  109.6  139.8   60.5   50.3   50.0
8    0.0    6.5    8.5    8.5   10.5   35.4   47.7   18.5   15.6   15.8
9    0.0    9.5   12.5   12.5   16.5   62.3   81.7   32.5   27.1   27.2

kilobytes output per second
#    bin    rle  wvl-f  wvl-s     tc  bb2.0   pu-f  doyna  nucru  rnucr  LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1          46.9   39.9   37.2   34.8   11.7    9.9   19.6   24.5   25.3
2          44.3   32.5   32.5   25.6    7.8    6.1   13.9   17.7   17.9
3          43.0   29.7   29.7   25.8    8.5    6.1   15.5   19.4   19.2
4          40.4   36.2   36.2   32.7    9.2    6.6   16.8   20.6   20.1
5          46.6   43.1   40.0   28.6    7.7    5.7   14.2   16.8   16.9
6          75.5   60.7   60.7   41.3   13.8   10.1   31.3   26.7   27.3
7          44.4   39.1   37.7   30.7    9.1    7.1   16.5   19.8   20.0
8          43.0   32.9   32.9   26.6    7.9    5.9   15.1   18.0   17.7
9          46.2   35.1   35.1   26.6    7.0    5.4   13.5   16.2   16.1

cycles per byte consumed
#    bin    rle  wvl-f  wvl-s     tc  bb2.0   pu-f  doyna  nucru  rnucr  LZMPi LZMV4k
- ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------ ------
1    0.0   28.2   58.6   68.7   70.4  272.9  289.7  165.6  134.2  129.4
2    0.0   25.1   41.7   44.6   54.6  244.4  259.3  136.9  108.0  107.5
3    0.0   25.3   42.7   48.8   49.6  212.3  247.9  116.6   93.8   94.6
4    0.0   25.9   44.0   45.7   48.8  200.3  260.4  111.4   90.3   93.1
5    0.0   26.0   30.1   33.6   46.4  212.8  277.1  115.1   97.5   96.4
6    0.0   32.2   44.4   45.9   63.5  245.3  326.3  109.3  127.7  125.1
7    0.0   25.6   41.4   46.2   55.8  233.2  280.7  128.0  108.2  107.4
8    0.0   23.6   40.0   42.7   52.4  219.9  256.4  116.1   96.6   97.2
9    0.0   23.4   35.5   35.6   49.3  223.0  267.7  117.6   96.9   97.4
2016-02-09 06:35
Martin Piper

Registered: Nov 2007
Posts: 629
Are you timing with VICE stopwatch to get the cycle times? With the screen off?
2016-02-09 06:49
ChristopherJam

Registered: Aug 2004
Posts: 1359
Chained DC00 timers (based on a routine from Glasnost) and blanked screen for nucrunch

VICE stopwatch for BB & pu, not blanked; which in the light of day seems a tad unfair - forgot that at 3am.

I need to either translate BB or install the assembler HCL uses so I can build it into my testbed instead of just running the produced executable.
2016-02-09 07:02
HCL

Registered: Feb 2003
Posts: 716
Hmm, compression rate of B2 was something like expected.. but performance was not that impressive, only 15-25% faster than before. I guess i was comparing to original bb, and you probably had the latest optimized decruncher. Never mind, thanx for adding it so quickly!!
Previous - 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 - Next
RefreshSubscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search   for   in  
All times are CET.
Search CSDb
Advanced
Users Online
hedning/G★P
Hairdog/BOOM!^Dream
Mike
the mad-zap/Royal Cr..
Brittle/Dentifrice^(?)
kbs/Pht/Lxt
Mojzesh/TGR🇬🇧
anonym/padua
encore
t0m3000/ibex-crew
jmin
Guests online: 366
Top Demos
1 Next Level  (9.8)
2 Mojo  (9.7)
3 Coma Light 13  (9.7)
4 Edge of Disgrace  (9.6)
5 No Bounds  (9.6)
6 Comaland 100%  (9.6)
7 Uncensored  (9.6)
8 The Ghost  (9.6)
9 Wonderland XIV  (9.6)
10 Bromance  (9.6)
Top onefile Demos
1 Party Elk 2  (9.7)
2 Cubic Dream  (9.6)
3 Copper Booze  (9.5)
4 Rainbow Connection  (9.5)
5 TRSAC, Gabber & Pebe..  (9.5)
6 Onscreen 5k  (9.5)
7 Dawnfall V1.1  (9.5)
8 Quadrants  (9.5)
9 Daah, Those Acid Pil..  (9.5)
10 Birth of a Flower  (9.5)
Top Groups
1 Booze Design  (9.3)
2 Nostalgia  (9.3)
3 Oxyron  (9.3)
4 Censor Design  (9.3)
5 Crest  (9.3)
Top Coders
1 Axis  (9.8)
2 Graham  (9.8)
3 Lft  (9.8)
4 Crossbow  (9.8)
5 HCL  (9.8)

Home - Disclaimer
Copyright © No Name 2001-2024
Page generated in: 0.076 sec.