[CSDb] - User Forums - Release id #117852 : Doynax LZ

You are not logged in - nap

CSDb User Forums

Forums > CSDb Entries > Release id #117852 : Doynax LZ

2013-04-15 16:51

Bitbreaker

Registered: Oct 2002
Posts: 500

Release id #117852 : Doynax LZ

Has anyone else discovered further misbehaviour than that i described in the goofs? I'd add a bunch of features then and release it in a fixed and improved version.

... 16 posts hidden. Click here to view all posts....

2013-04-17 15:40

doynax
Account closed

Registered: Oct 2004
Posts: 212

Quoting enthusi

A bit unrelated,but are there any depackers with minmum of RAM usage?
Stream-decrunch byte-by-byte without large tables in RAM or ZP.
I.e. exomizer hast that nice get_decrunched_byte but uses that 156 Byte table (=large in my context *g*).
Any ideas?

RLE perhaps?

If you've got ROM to spare but no RAM then you can't go wrong with static entropy coding, e.g. Huffman/arithmetic. For static dictionary compression from ROM you might try "Off-Line Dictionary-Based Compression" (Larsson & Moffat.)

2013-04-17 15:45

Bitbreaker

Registered: Oct 2002
Posts: 500

@doynax: i put the stuff to our svn, but would be happy to have the most recent version from you to be sure i based my fixes on that version. Of course things can be switched on and off by ifdefs, but also a command-line option would do. As for the 256 byte literals i found a nice way to branch out Actually i won't get an overrun in the lowbyte, but thereofre i can directly fork out with a bew after the lda lz_scratch before the adding starts. Here it doesn't hurt much it and works well :-) Also started to save some bytes on the decruncher, but it is a hard work :-)

@enthusi: there's decrunchers with smaller size, but usually speed is the tradeoff, as you then have to do a lot of jsrs to common code. So also the doynax decruncher could be smaller, but then slower, i could try a slow but small version though.

2013-04-17 16:43

doynax
Account closed

Registered: Oct 2004
Posts: 212

Quoting Bitbreaker

i put the stuff to our svn, but would be happy to have the most recent version from you to be sure i based my fixes on that version.

Certainly, though I don't think I've changed much since aside from translating it to CA65. IIRC I fixed a bug where the cruncher would crash on zero-byte files.

Quoting Bitbreaker

Of course things can be switched on and off by ifdefs, but also a command-line option would do.

I was thinking more along the lines of the decruncher actually, though I suppose the users would prefer separate files.

Quoting Bitbreaker

As for the 256 byte literals i found a nice way to branch out Actually i won't get an overrun in the lowbyte, but therefore i can directly fork out with a beq after the lda lz_scratch before the adding starts. Here it doesn't hurt much it and works well :-)

Nice :)

Giving it some more thought I think I can do you one better:

	lda lz_scratch
	lda lz_dst+0
;	sec
	sta lz_dst+0
	bcs .upper_rts
	.
	.
	.
.lrun_gotten:
	tay
;	sec
	sbc #$01
	sta lz_scratch

Basically adding the length minus one with carry ensures overflow. Carry is always set before the SBC (due to the preceding branches), and it will also always be set after the literal copying loop as well except for the 256-byte case. However a sector refill must then also have occured, and part of the contract for that function is to raise carry on exit.

Net cost: zero cycles and one byte.

Quoting Bitbreaker

Also started to save some bytes on the decruncher, but it is a hard work :-)

Yeah, as you've probably noticed it was optimized for speed rather than size.

Quoting Bitbreaker

There's decrunchers with smaller size, but usually speed is the tradeoff, as you then have to do a lot of jsrs to common code.

You'll still need a decompression window if some sort though, which would be the main memory hog. I wonder how LZ coders perform with really small windows, many references are rather short after all.

As an aside, has anyone experimented with feeding a "don't care" mask to a compressor? I find myself having plenty of holes in my data, along with plenty of alignment and SMC bytes my code which might benefit.

2013-04-17 17:31

Bitbreaker

Registered: Oct 2002
Posts: 500

Quoting doynax

I was thinking more along the lines of the decruncher actually, though I suppose the users would prefer separate files.

Well, of the user just wants to depack a crucnhed file as a whole, the window stuff and per page rendering is not needed and thus a type bit can be always omitted after a literal run, except when it is of maximum size. In that case we loose the one feature, but gain teh possibility to depack to an arbitrary address, what more often is of good use in a trackmo. In that case, also your proposal with the forced page crossing does not work anymore of course. But well, it is two ways one can go, and the user has to decide which to use. Both can be configured, depending on ones needs. In case it just needs a bit of an ifdef-hell within the .asm part :-)

Quoting doynax

Yeah, as you've probably noticed it was optimized for speed rather than size.

Well, compared to other depacker routines it is still of decent size, and you did a good job in saving bytes while making it fecking fast. I did a diversion of the bongo-cruncher from Mr. Wegi recently + decruncher. It is shorter in size, but your depacker is still 25% faster, though it is way faster than the other well known depackers, and despite shifting in the bits from the stream the usual way. However i did the copy loops faster, by fixing up the get_pointers index beforehand.

Also it would be pretty possible to let the cruncher spit out sfx files as well. Though it was not directly designed for that it is easy to adopt.

2013-04-26 23:38

Krill

Registered: Apr 2002
Posts: 2839

I have just finished adding Doynax LZ to my loader. (Yes, it's FAST! And it seems to be between Exomizer and Pucrunch in pack ratio. New gold standard default demo packer hands down.) I'd like to release it with all packer fixes. When can i expect to see an official Doynax-approved patch? :)

2013-04-27 07:17

Bitbreaker

Registered: Oct 2002
Posts: 500

It turns out, that doynax performs well on small files, and could even perform better per file when tweaking the offsettables. Also the encoding used performs worse on huge files with huge offsets and huge lengths. I made tests with otehr encodings that squeeze out way more bytes on huge files. But well, good that there's furtehr challenges to fight our boredome with :-P

2013-04-27 22:41

chatGPZ

Registered: Dec 2001
Posts: 11114

go go go ! =)

2013-04-28 15:35

Fungus

Registered: Sep 2002
Posts: 616

Yes Yes, bugfixed version puhleeeeeeeeeeeeze =]

2013-04-28 17:19

Bitbreaker

Registered: Oct 2002
Posts: 500

Doynax has all the changes i made and is doing a proper release from that hopefully soon, so i ask for patience :)

2013-05-06 06:47

Bitbreaker

Registered: Oct 2002
Posts: 500

here you go:
Doynamite 1.0