Log inRegister an accountBrowse CSDbHelp & documentationFacts & StatisticsThe forumsAvailable RSS-feeds on CSDbSupport CSDb Commodore 64 Scene Database
You are not logged in - nap
CSDb User Forums


Forums > CSDb Entries > Release id #154516 : Subsizer 0.5
2017-03-18 18:36
Compyx

Registered: Jan 2005
Posts: 631
Release id #154516 : Subsizer 0.5

Better to take this to the forum.

After adding some printf() statements to track the file open failure with "some demo.prg", the quoting of arguments works, but only sometimes. Which usually indicates a deeper problem somewhere.

Running through valgrind I get same nasty messages:
compyx@asus-p5k:~$ valgrind bin/subsizer "music demo.prg"
==25408== Memcheck, a memory error detector
==25408== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==25408== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==25408== Command: bin/subsizer music\ demo.prg
==25408== 
argv[0] = 'bin/subsizer'
argv[1] = 'music demo.prg'
build matches...
==25408== Warning: set address range perms: large range [0x395f8040, 0x98bd9040) (undefined)
...0.49 s
optimizing matches...
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x10B6F4: cost_enc (in /home/compyx/bin/subsizer)
==25408==    by 0x10C020: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
 5264 (left 70.84%)
0000113235660150,1112,1010122424555667,222233346666789A,132223234566679C
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x4C31CC2: __memcmp_sse4_1 (vg_replace_strmem.c:1094)
==25408==    by 0x10C4B4: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x4C31CFD: __memcmp_sse4_1 (vg_replace_strmem.c:1094)
==25408==    by 0x10C4B4: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x10C4B7: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
 4899 (left 65.93%)
0000113235660150,1112,1010122332456667,222323346666789A,132223234566679C
 4897 (left 65.90%)
0000113235660150,1112,1010122332456667,222323346666789A,132223234566679C
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x4C31CD6: __memcmp_sse4_1 (vg_replace_strmem.c:1094)
==25408==    by 0x10C4B4: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
...2.02 s
==25408== Warning: set address range perms: large range [0x395f8028, 0x98bd9058) (noaccess)
generating output...
...0.02 s
packed 7431 bytes into 4897 bytes
verifed 7431 bytes...ok
==25408== 
==25408== HEAP SUMMARY:
==25408==     in use at exit: 0 bytes in 0 blocks
==25408==   total heap usage: 101 allocs, 101 frees, 1,624,666,687 bytes allocated
==25408== 
==25408== All heap blocks were freed -- no leaks are possible
==25408== 
==25408== For counts of detected and suppressed errors, rerun with: -v
==25408== Use --track-origins=yes to see where uninitialised values come from
==25408== ERROR SUMMARY: 6 errors from 5 contexts (suppressed: 0 from 0)


I tried using --track-origins=yes, but that completely borked my system, that filled my memory and my swap space and made my system completely unresponsive, had to kill valgrind via a tty.


Is there bug a tracker somewhere for this?
 
... 18 posts hidden. Click here to view all posts....
 
2017-03-18 20:15
tlr

Registered: Sep 2003
Posts: 1701
Interesting! What version of gcc?

I have:
tlr@pinecone:subsizer$ gcc --version
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

tlr@pinecone:subsizer$
2017-03-18 20:23
soci

Registered: Sep 2003
Posts: 472
gcc (Debian 6.3.0-8) 6.3.0 20170221

There are some harmless warnings with it:
fold.c: In function ‘fold’:
fold.c:339:12: warning: unused variable ‘n’ [-Wunused-variable]
     size_t n = low_limit - sa;
            ^
At top level:
fold.c:142:13: warning: ‘dump_rle_entries’ defined but not used [-Wunused-function]
 static void dump_rle_entries(void)
             ^~~~~~~~~~~~~~~~
In file included from fold.c:15:0:
decrunchers.h:277:18: warning: ‘fs_list_header’ defined but not used [-Wunused-variable]
 static FixStruct fs_list_header[] = {
                  ^~~~~~~~~~~~~~
decrunchers.h:270:18: warning: ‘fs_list_tail’ defined but not used [-Wunused-variable]
 static FixStruct fs_list_tail[] = {
                  ^~~~~~~~~~~~
decrunchers.h:263:18: warning: ‘fs_list_decruncher’ defined but not used [-Wunused-variable]
 static FixStruct fs_list_decruncher[] = {
                  ^~~~~~~~~~~~~~~~~~
2017-03-21 18:51
tlr

Registered: Sep 2003
Posts: 1701
update: Subsizer 0.5.1
subsizer 0.5.1, 2017-03-21
  - added LICENSE.txt
  - give error if no end marker can be found.
  - got rid of a few warnings as reported by soci.
  - fixed issue with uninitialized memory as reported by compyx.
  - added decruncher source due to popular demand.  well, only bitbreaker. :)
    (caution: encoding may change between versions)
2017-03-24 04:01
ChristopherJam

Registered: Aug 2004
Posts: 1359
Has anyone tested this on the Pearl for Pigs corpus?

I'm writing a benchmarks page at codebase ( http://www.codebase64.org/doku.php?id=base:compression_benchmar.. ), and would be interesting to see where Subsizer fits in the big picture

I'll add a scatter plot soonish.
2017-03-24 06:35
tlr

Registered: Sep 2003
Posts: 1701
I have these for subsizer 0.5:
    file           size  (blks)   left    gain   time   mem
    -----------------------------------------------------------
        pfp1.bin:  2956 (12)     26.85%  73.15%  0.34s  30.05M
        pfp2.bin:  2205 (9)      44.34%  55.66%  0.10s  27.47M
        pfp3.bin:  1788 (8)      45.28%  54.72%  0.08s  25.08M
        pfp4.bin:  3456 (14)     49.26%  50.74%  0.16s  25.46M
        pfp5.bin:  19519 (77)    56.15%  43.85%  1.23s  74.61M
        pfp6.bin:  8396 (34)     26.57%  73.43%  1.33s  83.25M
        pfp7.bin:  8766 (35)     42.99%  57.01%  0.51s  32.22M
        pfp8.bin:  3063 (13)     53.61%  46.39%  0.15s  24.89M
        pfp9.bin:  5307 (21)     59.23%  40.77%  0.14s  25.58M
    -----------------------------------------------------------
No decruncher speeds as there is no standalone decruncher yet.

Curious question: are we measuring those in cycles without badlines?
2017-03-24 06:44
ChristopherJam

Registered: Aug 2004
Posts: 1359
Quoting tlr
I have these for subsizer 0.5:
…
No decruncher speeds as there is no standalone decruncher yet.

Curious question: are we measuring those in cycles without badlines?


Thanks! Yes, time in cycles with neither badlines nor interrupts, either by using CIA or breakpoints+cyclecounter in VICE.
2017-03-24 07:53
ChristopherJam

Registered: Aug 2004
Posts: 1359
Also, wow - those are tiny!

Even smaller than exomizer. I really need to time those two...
2017-03-24 10:57
Frantic

Registered: Mar 2003
Posts: 1626
Well, if you call yourself The Leader, you need to be second to none!
2017-03-27 07:21
ChristopherJam

Registered: Aug 2004
Posts: 1359
Scatter plot's up at http://www.codebase64.org/doku.php?id=base:compression_benchmar.. :)
2017-03-28 08:56
tlr

Registered: Sep 2003
Posts: 1701
I wanted to replicate some of the results as a baseline but can't really get the same values.

This is what I get for lzwvl-f:
        name       size          left    cycles  frm  spd     cons
        pfp1.bin:  4529 (18)     41.14%  234334 11.9 45.2k/s 51.7c/b
        pfp2.bin:  3532 (14)     71.02%  127774 6.5  37.4k/s 36.2c/b
        pfp3.bin:  2991 (12)     75.74%  95420  4.9  39.8k/s 31.9c/b
        pfp4.bin:  4242 (17)     60.46%  153715 7.8  43.9k/s 36.2c/b
        pfp5.bin:  25781 (102)   74.17%  730916 37.2 45.8k/s 28.4c/b
        pfp6.bin:  11283 (45)    35.70%  644479 32.8 47.2k/s 57.1c/b
        pfp7.bin:  12108 (48)    59.38%  466960 23.8 42.0k/s 38.6c/b
        pfp8.bin:  4179 (17)     73.15%  142155 7.2  38.7k/s 34.0c/b
        pfp9.bin:  6914 (28)     77.17%  226119 11.5 38.1k/s 32.7c/b
The cycle count is slightly off, presumably due to obscure timing model errors in unp64 which I'm using to emulate the depacker. I verified the timing in vice for a few and the error for those was about 1% or so.

Note especially the difference between pfp6.bin and pfp7.bin.

The last value is cycles per byte _consumed_ as in the tables. Is this really a useful measure?
Previous - 1 | 2 | 3 - Next
RefreshSubscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search   for   in  
All times are CET.
Search CSDb
Advanced
Users Online
csabanw
psych
McMeatLoaf
Guests online: 356
Top Demos
1 Next Level  (9.8)
2 Mojo  (9.7)
3 Coma Light 13  (9.7)
4 Edge of Disgrace  (9.6)
5 No Bounds  (9.6)
6 Comaland 100%  (9.6)
7 Uncensored  (9.6)
8 The Ghost  (9.6)
9 Wonderland XIV  (9.6)
10 Bromance  (9.6)
Top onefile Demos
1 Party Elk 2  (9.7)
2 Cubic Dream  (9.6)
3 Copper Booze  (9.5)
4 Rainbow Connection  (9.5)
5 TRSAC, Gabber & Pebe..  (9.5)
6 Onscreen 5k  (9.5)
7 Dawnfall V1.1  (9.5)
8 Quadrants  (9.5)
9 Daah, Those Acid Pil..  (9.5)
10 Birth of a Flower  (9.5)
Top Groups
1 Booze Design  (9.3)
2 Nostalgia  (9.3)
3 Oxyron  (9.3)
4 Censor Design  (9.3)
5 Crest  (9.3)
Top Diskmag Editors
1 Jazzcat  (9.4)
2 Magic  (9.4)
3 hedning  (9.2)
4 Newscopy  (9.1)
5 Elwix  (9.1)

Home - Disclaimer
Copyright © No Name 2001-2024
Page generated in: 0.043 sec.