Log inRegister an accountBrowse CSDbHelp & documentationFacts & StatisticsThe forumsAvailable RSS-feeds on CSDbSupport CSDb Commodore 64 Scene Database
You are not logged in - nap
CSDb User Forums


Forums > CSDb Entries > Release id #232862 : BitPickler 1.0.0
2023-05-31 16:55
ChristopherJam

Registered: Aug 2004
Posts: 1356
Release id #232862 : BitPickler 1.0.0

BitPickler is a c64 cruncher optimised for the compression of small demos for 4k competitions and the like. It's very heavy on RAM usage during decompression (by default it uses all of $7c00-$ffff as working memory), but has an excellent compression ratio, and outputs around 800 or so bytes per second.

I'd been playing with some of the components for a while, but basically pulled it together to try and help get Onscreen 5k as small as Thundax and I could manage (while simultaneously being the musician who was one of the reasons we ended up having to work quite hard to get it below 5k and keep it there, nevermind hitting the 4k we'd pondered early in development :) )

At it's core it's a range encoder (tANS with an implicit table) with context mixing, combined with an optional ALZ64 style LZ77 encoder - amusingly, for Onscreen 5k the extra code size for the LZ77 layer almost exactly counterbalanced the savings from the improved ratio. It also searches for an area that benefits from a delta encoding preprocess (so please do group your sine tables etc together into a single block), and can deinterleave instructions from their first and second operands for an initial code block, with the length specified by you as a commandline option.

It is not designed for crunching files bigger than 10kb or so, or for trackmos, or for level crunching. It performs poorly on large empty areas, not so much because it doesn't squash them, but because those areas throw off the predictive encoder despite adaptation. I'm not planning on fixing any of those things.

For most small demos though, it has a significantly better ratio that ALZ64, while decrunching up to 1.5x faster.

Here's how BitPickler 1.0 performs on Artefacts, Immortality, 4kRastersIntro, and a prerelease version of Onscreen 5k,
compared with ALZ64, Exomizer V3.1.0, NuCrunch V1.0.0, TinyCrunch V1.2 and Bitbreaker's port of ZX0 1.5 with Soci's optimisations. (I grabbed the latter from 'somewhere' before the various Dali releases here).




For each test demo, the compressed file size in bytes and decompression time in seconds:
+-----------------------------+  +-----------------------------+
| Onscreen 5k                 |  | Artefacts                   |
+------------+-------+--------+  +------------+-------+--------+
| BitPickler | 4879b |  8.78s |  | BitPickler | 3963b |  5.93s |
| ALZ64      | 5396b | 12.21s |  | ALZ64      | 4024b |  9.21s |
| Exomizer   | 5434b |  0.99s |  | Exomizer(2)| 4095b |  1.93s |
| ZX0        | 5456b |  0.54s |  | ZX0        | 4228b |  0.37s |
| NuCrunch   | 5862b |  0.49s |  | NuCrunch   | 4539b |  0.67s |
| TinyCrunch | 6219b |  0.20s |  | TinyCrunch | 4639b |  0.47s |
+------------+-------+--------+  +------------+-------+--------+
                                 
+-----------------------------+  +-----------------------------+
| Immortality                 |  | 4k Rasters                  |
+------------+-------+--------+  +------------+-------+--------+
| BitPickler |  911b |  2.18s |  | BitPickler | 2363b |  4.43s |
| ALZ64      |  987b |  1.63s |  | ALZ64      | 2438b |  5.14s |
| Exomizer   | 1038b |  0.19s |  | Exomizer   | 2504b |  0.46s |
| ZX0        | 1015b |  0.09s |  | ZX0        | 2545b |  0.23s |
| NuCrunch   | 1269b |  0.10s |  | NuCrunch   | 2788b |  0.23s |
| TinyCrunch | 1221b |  0.07s |  | TinyCrunch | 2868b |  0.11s |
+------------+-------+--------+  +------------+-------+--------+


(note 1, these were with BitPickler's activity indicator enabled - disabling that would save four bytes and improve speed by about 1%)
(note 2, the measurements for Artefacts+exo above used Exo4K-Decruncher, as per the original demo release; it saves 90 bytes compared to exo 3.1.0's usual decruncher, while only running 2.7x slower)
2023-05-31 21:32
Krill

Registered: Apr 2002
Posts: 2800
Pretty great! =)

3963 bytes for repacked Artefacts doesn't leave enough room for the director's cut in 4K, but then i did not expect that anyways.

Will surely use this for upcoming small onefiled demos! :)
2023-06-02 10:56
Frostbyte

Registered: Aug 2003
Posts: 162
Seriously impressive! If I ever get my wannabe-4k intro into a state of nearly-completeness, I'll certainly give BitPickler a go.

As a side note, I'm also very impressed about the speed vs compression ratio of both Exomizer and ZX0. I knew they are good, but didn't expect such solid compression performances in most cases.

Just out of curiosity, what were the starting points, i.e. the uncompressed file sizes?
2023-06-02 16:20
ChristopherJam

Registered: Aug 2004
Posts: 1356
Thanks Frostbyte! I look forward to seeing what you can do with it.

And yes, Exomizer and ZX0 have been seriously pushing the envelope at the midrange. Exo's doubled its decompression speed in the past few years, and its ratio's improved a bit too. Not sure what I'm going to do with my as yet unreleased exo-killer side project now; its a tad slower than Exomizer and ZX0, and the ratio gains aren't as compelling as they once were...

Similarly, Antonio Savona's been doing great work at the high speed/low ratio end of the spectrum - tscrunch is setting a high bar there!

Here are the starting points for the benchmarks above:
+----------------+-------+---------+----------+
| Demo           | Raw   | Pickled | % of Raw |
+----------------+-------+---------+----------+
| Onscreen 5k    | 7423b |  4879b  |   65.7%  |
| Artefacts      | 5224b |  3963b  |   75.9%  |
| 4k Rasters     | 4096b |  2363b  |   57.7%  |
| Immortality    | 2561b |   911b  |   35.6%  |
+----------------+-------+---------+----------+
2023-06-02 17:44
Frostbyte

Registered: Aug 2003
Posts: 162
Thanks CJam! The compression ratios give me hope. ;)
2023-06-06 11:38
Bacchus

Registered: Jan 2002
Posts: 154
Great work. It will make it possible to add even more impossible magic to 4k :-)

/Bacchus
RefreshSubscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search   for   in  
All times are CET.
Search CSDb
Advanced
Users Online
Martin Piper
Guests online: 171
Top Demos
1 Next Level  (9.8)
2 Mojo  (9.7)
3 Coma Light 13  (9.7)
4 Edge of Disgrace  (9.6)
5 No Bounds  (9.6)
6 Comaland 100%  (9.6)
7 Uncensored  (9.6)
8 The Ghost  (9.6)
9 No Sprites  (9.6)
10 Wonderland XIV  (9.6)
Top onefile Demos
1 Party Elk 2  (9.7)
2 Cubic Dream  (9.6)
3 Copper Booze  (9.5)
4 Rainbow Connection  (9.5)
5 TRSAC, Gabber & Pebe..  (9.5)
6 Onscreen 5k  (9.5)
7 Dawnfall V1.1  (9.5)
8 Quadrants  (9.5)
9 Daah, Those Acid Pil..  (9.5)
10 Birth of a Flower  (9.5)
Top Groups
1 Booze Design  (9.3)
2 Nostalgia  (9.3)
3 Oxyron  (9.3)
4 Censor Design  (9.3)
5 Crest  (9.3)
Top Webmasters
1 Slaygon  (9.7)
2 Perff  (9.6)
3 Morpheus  (9.5)
4 Sabbi  (9.5)
5 CreaMD  (9.1)

Home - Disclaimer
Copyright © No Name 2001-2024
Page generated in: 0.052 sec.