Log inRegister an accountBrowse CSDbHelp & documentationFacts & StatisticsThe forumsAvailable RSS-feeds on CSDbSupport CSDb Commodore 64 Scene Database
 Welcome to our latest new user AndyAsth ! (Registered 2017-10-16) You are not logged in 
CSDb User Forums


Forums > CSDb Entries > Release id #154516 : Subsizer 0.5
2017-03-18 19:36
Compyx

Registered: Jan 2005
Posts: 271
Release id #154516 : Subsizer 0.5

Better to take this to the forum.

After adding some printf() statements to track the file open failure with "some demo.prg", the quoting of arguments works, but only sometimes. Which usually indicates a deeper problem somewhere.

Running through valgrind I get same nasty messages:
compyx@asus-p5k:~$ valgrind bin/subsizer "music demo.prg"
==25408== Memcheck, a memory error detector
==25408== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==25408== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==25408== Command: bin/subsizer music\ demo.prg
==25408== 
argv[0] = 'bin/subsizer'
argv[1] = 'music demo.prg'
build matches...
==25408== Warning: set address range perms: large range [0x395f8040, 0x98bd9040) (undefined)
...0.49 s
optimizing matches...
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x10B6F4: cost_enc (in /home/compyx/bin/subsizer)
==25408==    by 0x10C020: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
 5264 (left 70.84%)
0000113235660150,1112,1010122424555667,222233346666789A,132223234566679C
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x4C31CC2: __memcmp_sse4_1 (vg_replace_strmem.c:1094)
==25408==    by 0x10C4B4: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x4C31CFD: __memcmp_sse4_1 (vg_replace_strmem.c:1094)
==25408==    by 0x10C4B4: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x10C4B7: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
 4899 (left 65.93%)
0000113235660150,1112,1010122332456667,222323346666789A,132223234566679C
 4897 (left 65.90%)
0000113235660150,1112,1010122332456667,222323346666789A,132223234566679C
==25408== Conditional jump or move depends on uninitialised value(s)
==25408==    at 0x4C31CD6: __memcmp_sse4_1 (vg_replace_strmem.c:1094)
==25408==    by 0x10C4B4: crunch_normal_int.isra.1 (in /home/compyx/bin/subsizer)
==25408==    by 0x109818: main (in /home/compyx/bin/subsizer)
==25408== 
...2.02 s
==25408== Warning: set address range perms: large range [0x395f8028, 0x98bd9058) (noaccess)
generating output...
...0.02 s
packed 7431 bytes into 4897 bytes
verifed 7431 bytes...ok
==25408== 
==25408== HEAP SUMMARY:
==25408==     in use at exit: 0 bytes in 0 blocks
==25408==   total heap usage: 101 allocs, 101 frees, 1,624,666,687 bytes allocated
==25408== 
==25408== All heap blocks were freed -- no leaks are possible
==25408== 
==25408== For counts of detected and suppressed errors, rerun with: -v
==25408== Use --track-origins=yes to see where uninitialised values come from
==25408== ERROR SUMMARY: 6 errors from 5 contexts (suppressed: 0 from 0)


I tried using --track-origins=yes, but that completely borked my system, that filled my memory and my swap space and made my system completely unresponsive, had to kill valgrind via a tty.


Is there bug a tracker somewhere for this?
2017-03-18 20:01
tlr

Registered: Sep 2003
Posts: 1123
Thanks for the report (you and ian)! No bug tracker at the moment but problem noted. There is some parsing of the filename to find ',' and '@' notations for load address and the likes. I guess the problem is there.

Stay away from filenames with spaces in them for the time being.
2017-03-18 20:20
Compyx

Registered: Jan 2005
Posts: 271
The spaces in the file names is a minor issue, I'm more worried about the uninitialized values, and you seem to be allocating huge chunks of memory somehow (might be related).

Happy hacking :)
2017-03-18 20:25
tlr

Registered: Sep 2003
Posts: 1123
Try this:
index 6012af5..55f85ab 100644
--- a/crunch/subsizer/src/crunch_normal.c
+++ b/crunch/subsizer/src/crunch_normal.c
@@ -485,7 +485,7 @@ static int crunch_normal_int(Buffer *sbf, Buffer *dbf, int flags)
     MatchTree *mt = create_matchtree();
     double t1, t2;
     EncodingSet es;
-
+    memset(&es, 0, sizeof(EncodingSet));
 
     msg(MSG_VERBOSE, "build matches...\n");
     t1 = get_time();
2017-03-18 20:36
soci

Registered: Sep 2003
Posts: 363
Sure, that's a good solution for the garbage in structure holes which could affect memcmp at crunch_normal.c:402.

But it also hides the issue that endm is not initialized in optimize_tree like the rest of the fields.
2017-03-18 20:39
tlr

Registered: Sep 2003
Posts: 1123
I know it's ugly, but basically es gets memcpy'd to last_es for comparison so any uninitialized values are compared to themselves. It shouldn't (tm) affect operation.

I'll try to do something nicer in the future though.

Btw, I haven't really been able to reproduce anything special related to spaces in the filename. Any clues?
2017-03-18 20:41
Compyx

Registered: Jan 2005
Posts: 271
That didn't help. Maybe tomorrow when I have more time (hacking on my own stuff and VICE), I'll try to figure out where the uninitialized stuff comes from.
2017-03-18 20:46
tlr

Registered: Sep 2003
Posts: 1123
Ok, thanks. The valgrind errors completely disappeared on my test case here using this. Maybe it depends on the input file... Running Ubuntu 14.04 64-bit on an i5 here. I'd love a reproducible test case. PM if you like.
2017-03-18 20:56
Compyx

Registered: Jan 2005
Posts: 271
I simply used this: http://csdb.dk/release/download.php?id=191285, since it was something I downloaded recently and was small.

The filename stuff happens randomly, when I added some printf()'s to see what was going on, it disappeared for a while, but then came back. So that might indicate stack corruption somewhere.

But so far, that file has given me consistent errors with valgrind. I'm running Debian Stretch 64-bit here, on an old Core Duo box. GCC is 6.3.0, valgrind is 3.12.0.SVN.
2017-03-18 21:03
tlr

Registered: Sep 2003
Posts: 1123
Gaah... Maybe I shouldn't have done as many kludges when coding this. :/

I'm sure there is some stack corruption problem but not easily reproducible here unfortunately. Will clean up some nastyness for the next release.
tlr@pinecone:subsizer$ valgrind subsizer -x "tests/Ninth.prg"
==13935== Memcheck, a memory error detector
==13935== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==13935== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info
==13935== Command: subsizer -x tests/Ninth.prg
==13935== 
read 'tests/Ninth.prg' $0801-$7400.
detected sys: $4000
build matches...
==13935== Warning: set address range perms: large range [0x3a048040, 0x99629040) (undefined)
...6.48 s
optimizing matches...
 7139 (left 25.82%)
1110334466621000,1012,0000001022345657,02112344560678AD,02103434555789AF
 6677 (left 24.15%)
1110334465462410,1012,0000001022345657,02112344560678AD,02103434555789AF
 6682 (left 24.17%)
1110334465462410,1012,0000001022345657,02112344560678AD,02103434555789AF
...6.69 s
==13935== Warning: set address range perms: large range [0x3a048028, 0x99629058) (noaccess)
generating output...
...0.02 s
safe = 2
packed 27647 bytes (109 blocks) into 6974 bytes (28 blocks)
==13935== 
==13935== HEAP SUMMARY:
==13935==     in use at exit: 0 bytes in 0 blocks
==13935==   total heap usage: 100 allocs, 100 frees, 1,624,919,704 bytes allocated
==13935== 
==13935== All heap blocks were freed -- no leaks are possible
==13935== 
==13935== For counts of detected and suppressed errors, rerun with: -v
==13935== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
tlr@pinecone:subsizer$ 
2017-03-18 21:09
soci

Registered: Sep 2003
Posts: 363
Works fine here, also for spaced filenames. Similar system but 32 bit so I had to adjust the following hack otherwise the allocation didn't succeed with valgrind:
safe_malloc(200000000 * sizeof(Match), "matches");
 
... 18 posts hidden. Click here to view all posts....
 
Previous - 1 | 2 | 3 - Next
RefreshSubscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search   for   in  
All times are CET.
Search CSDb
Advanced
Users Online
freℚvibez/Offence
hedning/G★P
tNG
lft
redcrab/G★P
Asthor/Exlusive ON
Cruzer/CML^Oxy^Λrsenic
Flex/Artline Designs
enthusi/ons^PriorArt
Guests online: 185
Top Demos
1 Uncensored  (9.7)
2 Edge of Disgrace  (9.7)
3 Coma Light 13  (9.6)
4 The Shores of Reflec..  (9.6)
5 Lunatico  (9.6)
6 Quad Core 100%  (9.5)
7 Comaland 100%  (9.5)
8 Incoherent Nightmare  (9.5)
9 Wonderland XII  (9.5)
10 Comaland  (9.5)
Top onefile Demos
1 Pandemoniac Part 2 o..  (9.6)
2 Synthesis  (9.6)
3 Dawnfall V1.1  (9.5)
4 Daah, Those Acid Pil..  (9.5)
5 Field Sort  (9.4)
6 Treu Love [reu]  (9.4)
7 Dawnfall  (9.3)
8 KAOS 64  (9.3)
9 Veterans of Style  (9.2)
10 One-Der  (9.2)
Top Groups
1 Booze Design  (9.4)
2 Censor Design  (9.4)
3 Oxyron  (9.4)
4 Crest  (9.3)
5 Finnish Gold  (9.3)
Top Diskmag Editors
1 Jazzcat  (9.5)
2 Peter  (9.4)
3 Newscopy  (9.4)
4 Remix  (9.3)
5 Vengeance  (9.3)

Home - Disclaimer
Copyright © No Name 2001-2017
Page generated in: 0.756 sec.