Log inRegister an accountBrowse CSDbHelp & documentationFacts & StatisticsThe forumsAvailable RSS-feeds on CSDbSupport CSDb Commodore 64 Scene Database
You are not logged in 
CSDb User Forums


Forums > CSDb Entries > Release id #154516 : Subsizer 0.5
2017-03-18 19:36
Compyx

Registered: Jan 2005
Posts: 217
Release id #154516 : Subsizer 0.5

Better to take this to the forum.

After adding some printf() statements to track the file open failure with "some demo.prg", the quoting of arguments works, but only sometimes. Which usually indicates a deeper problem somewhere.

Running through valgrind I get same nasty messages:
compyx@asus-p5k:~$ valgrind bin/subsizer "music demo.prg"
==25408== Memcheck, a memory error detector
==25408== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==25408== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==25408== Command: bin/subsizer music\ demo.prg
==25408== 
argv[0] = 'bin/subsizer'
argv[1] = 'music demo.prg'
build matches...
==25408== Warning: set address range perms: large range [0x395f8040, 0x98bd9040) (undefined)
...0.49 s
optimizing matches...
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x10B6F4: cost_enc (in /home/compyx/bin/subsizer)
==25408==    by 0x10C020: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
 5264 (left 70.84%)
0000113235660150,1112,1010122424555667,222233346666789A,132223234566679C
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x4C31CC2: __memcmp_sse4_1 (vg_replace_strmem.c:1094)
==25408==    by 0x10C4B4: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x4C31CFD: __memcmp_sse4_1 (vg_replace_strmem.c:1094)
==25408==    by 0x10C4B4: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x10C4B7: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
 4899 (left 65.93%)
0000113235660150,1112,1010122332456667,222323346666789A,132223234566679C
 4897 (left 65.90%)
0000113235660150,1112,1010122332456667,222323346666789A,132223234566679C
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x4C31CD6: __memcmp_sse4_1 (vg_replace_strmem.c:1094)
==25408==    by 0x10C4B4: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
...2.02 s
==25408== Warning: set address range perms: large range [0x395f8028, 0x98bd9058) (noaccess)
generating output...
...0.02 s
packed 7431 bytes into 4897 bytes
verifed 7431 bytes...ok
==25408== 
==25408== HEAP SUMMARY:
==25408==     in use at exit: 0 bytes in 0 blocks
==25408==   total heap usage: 101 allocs, 101 frees, 1,624,666,687 bytes allocated
==25408== 
==25408== All heap blocks were freed -- no leaks are possible
==25408== 
==25408== For counts of detected and suppressed errors, rerun with: -v
==25408== Use --track-origins=yes to see where uninitialised values come from
==25408== ERROR SUMMARY: 6 errors from 5 contexts (suppressed: 0 from 0)


I tried using --track-origins=yes, but that completely borked my system, that filled my memory and my swap space and made my system completely unresponsive, had to kill valgrind via a tty.


Is there bug a tracker somewhere for this?
 
... 18 posts hidden. Click here to view all posts....
 
2017-03-27 09:21
ChristopherJam

Registered: Aug 2004
Posts: 587
Scatter plot's up at http://www.codebase64.org/doku.php?id=base:compression_benchmar.. :)
2017-03-28 10:56
tlr

Registered: Sep 2003
Posts: 1118
I wanted to replicate some of the results as a baseline but can't really get the same values.

This is what I get for lzwvl-f:
        name       size          left    cycles  frm  spd     cons
        pfp1.bin:  4529 (18)     41.14%  234334 11.9 45.2k/s 51.7c/b
        pfp2.bin:  3532 (14)     71.02%  127774 6.5  37.4k/s 36.2c/b
        pfp3.bin:  2991 (12)     75.74%  95420  4.9  39.8k/s 31.9c/b
        pfp4.bin:  4242 (17)     60.46%  153715 7.8  43.9k/s 36.2c/b
        pfp5.bin:  25781 (102)   74.17%  730916 37.2 45.8k/s 28.4c/b
        pfp6.bin:  11283 (45)    35.70%  644479 32.8 47.2k/s 57.1c/b
        pfp7.bin:  12108 (48)    59.38%  466960 23.8 42.0k/s 38.6c/b
        pfp8.bin:  4179 (17)     73.15%  142155 7.2  38.7k/s 34.0c/b
        pfp9.bin:  6914 (28)     77.17%  226119 11.5 38.1k/s 32.7c/b
The cycle count is slightly off, presumably due to obscure timing model errors in unp64 which I'm using to emulate the depacker. I verified the timing in vice for a few and the error for those was about 1% or so.

Note especially the difference between pfp6.bin and pfp7.bin.

The last value is cycles per byte _consumed_ as in the tables. Is this really a useful measure?
2017-04-06 19:56
tlr

Registered: Sep 2003
Posts: 1118
Teaser:
        name       size          left    cycles  frm  spd     cons
        pfp1.bin:  2961 (12)     26.90%  751339  38.2 14.1k/s 253.7c/b
        pfp2.bin:  2201 (9)      44.26%  432794  22.0 11.1k/s 196.6c/b
        pfp3.bin:  1786 (8)      45.23%  321693  16.4 11.8k/s 180.1c/b
        pfp4.bin:  3438 (14)     49.00%  638705  32.5 10.6k/s 185.8c/b
        pfp5.bin:  19631 (78)    56.48%  3642024 185.3 9.2k/s 185.5c/b
        pfp6.bin:  8407 (34)     26.60%  1864843 94.9 16.3k/s 221.8c/b
        pfp7.bin:  8768 (35)     43.00%  1725244 87.8 11.4k/s 196.8c/b
        pfp8.bin:  3086 (13)     54.02%  520012  26.5 10.6k/s 168.5c/b
        pfp9.bin:  5313 (21)     59.30%  973471  49.5 8.9k/s  183.2c/b

There are many optimizations left to be done. It will use a temporary buffer of slightly less than a page though. This can of course be overwritten between decrunches.
2017-04-08 00:20
ChristopherJam

Registered: Aug 2004
Posts: 587
Ooh, nice. Already considerably faster than Exomizer, which is the only other cruncher in the same ballpark for ratio.

I was thinking cycles per byte consumed might be a useful metric for impedance matching with fastloaders, especially if you're streaming.
2017-04-08 06:32
tlr

Registered: Sep 2003
Posts: 1118
Quoting ChristopherJam
I was thinking cycles per byte consumed might be a useful metric for impedance matching with fastloaders, especially if you're streaming.

Good point!

I'll be away for a week+ now so will be starting with a fresh mind then and see what I can come up with.
2017-04-21 09:42
ChristopherJam

Registered: Aug 2004
Posts: 587
I've updated the scatter plot and tables at codebase.


Article here if that image link fails.
2017-04-21 09:53
j0x

Registered: Mar 2004
Posts: 207
Interesting! Have you considered adding benchmark data from the Card Cruncher or other tools based purely on entropy encoding?
2017-04-21 10:44
ChristopherJam

Registered: Aug 2004
Posts: 587
j0x, submissions are welcome! I just need the compressed sizes and cycle times for decompression for each of the nine files..
2017-04-21 20:24
tlr

Registered: Sep 2003
Posts: 1118
new release: Subsizer 0.6
subsizer 0.6, 2017-04-21
  - improved first pass cost model
  - cleaned up verbose output a bit
  - saved 10 bytes in the dirty sfx decruncher
  - added stand alone decruncher source

Sorry for releasing it right after you updated the tables. ;)
2017-04-22 17:52
tlr

Registered: Sep 2003
Posts: 1118
pfp-stats for 0.6:
                                        duration      outspd    inspd
     file       size  (blks)   left   cycles  frms  k/s   cy/b  cy/b
    ------------------------------------------------------------------
     pfp1.bin:  2961 (12)     26.90%  724217  36.8  14.6  65.8  244.6
     pfp2.bin:  2201 (9)      44.26%  414274  21.1  11.5  83.3  188.2
     pfp3.bin:  1786 (8)      45.23%  308675  15.7  12.3  78.2  172.8
     pfp4.bin:  3438 (14)     49.00%  613097  31.2  11.0  87.4  178.3
     pfp5.bin:  19631 (78)    56.48%  3497850 178.0 9.6   100.6 178.2
     pfp6.bin:  8407 (34)     26.60%  1803984 91.8  16.9  57.1  214.6
     pfp7.bin:  8768 (35)     43.00%  1661112 84.5  11.8  81.5  189.5
     pfp8.bin:  3086 (13)     54.02%  500954  25.5  11.0  87.7  162.3
     pfp9.bin:  5313 (21)     59.30%  934362  47.5  9.2   104.3 175.9
    ------------------------------------------------------------------
Previous - 1 | 2 | 3 - Next
RefreshSubscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search   for   in  
All times are CET.
Search CSDb
Advanced
Users Online
Karmic/HVSC/HF
Guests online: 41
Top Demos
1 Uncensored  (9.7)
2 Edge of Disgrace  (9.7)
3 The Shores of Reflec..  (9.7)
4 Coma Light 13  (9.6)
5 Lunatico  (9.6)
6 Comaland 100%  (9.6)
7 Incoherent Nightmare  (9.5)
8 Wonderland XII  (9.5)
9 Comaland  (9.5)
10 Wonderland XIII  (9.5)
Top onefile Demos
1 Dawnfall V1.1  (9.5)
2 Daah, Those Acid Pil..  (9.5)
3 SidRok  (9.4)
4 Treu Love [reu]  (9.4)
5 Tunnel Vision  (9.3)
6 Dawnfall  (9.3)
7 One-Der  (9.2)
8 Globe 2016 [reu]  (9.2)
9 Hardware Accelerated..  (9.2)
10 Safe VSP  (9.1)
Top Groups
1 Booze Design  (9.4)
2 Oxyron  (9.4)
3 Censor Design  (9.3)
4 Crest  (9.3)
5 Camelot  (9.2)
Top Diskmag Editors
1 Jazzcat  (9.5)
2 Peter  (9.4)
3 Newscopy  (9.4)
4 Remix  (9.3)
5 Tadpole  (9.3)

Home - Disclaimer
Copyright © No Name 2001-2017
Page generated in: 0.196 sec.