[CSDb] - User Forums - Tape loaders using more than two pulse widths for data

You are not logged in - nap

CSDb User Forums

Forums > C64 Coding > Tape loaders using more than two pulse widths for data

2018-12-08 20:16

ChristopherJam

Registered: Aug 2004
Posts: 1409

Tape loaders using more than two pulse widths for data

I’ve been thinking a bit about a novel tape encoding, but it would rely on (among other things) having more than two pulse widths. So far as I can see, none of the old turbo loaders used a long pulse for anything beyond end of byte/end of block - the bitstream itself was just a sequence of short and medium pulses (usually around 200 cycles for short, 300 to 450 for long).

Is there any particular reason none of the popular loaders used (eg) four pulse widths for each bitpair? Even using quite widely separated durations of 210/320/450/600 would then lead to an average time per bit of just 197 cycles, a massive improvement on the 265 cycles per bit you’d get for 210/320.

(The idea I’ve been working on is a bit more complex than that, but if a simple bitpair scheme wouldn’t work for some reason, then the idea I have would need to be scaled back somewhat. Promising that long pulses were used for framing, mind..)

... 23 posts hidden. Click here to view all posts....

2018-12-11 10:00

enthusi

Registered: May 2004
Posts: 677

The 6-pulse loader is NOTHING you'd want for a release.
It was used privately to store own sources/code by Thomas 'back in the day'. Each pulse encoded 3 bits:
%101,%111,%110,%100,%001,%000
Ah, but it used 7 pulses in fact :)
One pulse length encodes 2 bits '01'
I just made a quick histogram (cycles):

While this looks nice and clean, this is how the actual data of basically the best dump looked like:

For a 'public use' loader I strongly suggest to use MUCH larger gaps. The loaders I wrote for the load of tap demo or Jars' Revenge etc are pretty stable. Stability over speed :)
(I just skipped the kernal load header for the histogram, it is based on that data)

2018-12-12 08:40

ChristopherJam

Registered: Aug 2004
Posts: 1409

Wow, so fitting a line to the approximate peaks, I'm getting pulse lengths of

 {167, 217, 268, 318, 369, 419, 470},

so increments of around 50 cycles. That's... brave.

Thanks for the encoding details, simulating how well it performs has revealed a flaw in my calculations. (I was only weighting the symbols by probability that they'd be next, instead of also including a factor for how much output they'd produce).

(also, everywhere I've said pulse width on this page, substitute pulse length; apologies for the ambiguity).

2018-12-12 09:53

Hoogo

Registered: Jun 2002
Posts: 105

...this is how the actual data of basically the best dump looked like...

Do you have any idea what caused these absolutely blurred signals mostly at the end of the blocks?

I guess these were multiple files, and those signals might be some of the original music...

2018-12-12 10:24

enthusi

Registered: May 2004
Posts: 677

Yes, at least the backside had music on it ;-)
But two other things to note. If you press stop/play the starting motor will also produce all sorts of pulse-lengths (at least when read back via c2n) and of course these graphs show TAP bytes, not mere data bytes, so encoded pauses are also shown as pulses. That is the reason for those seemingly spurios pixels at the same horizontal position. Those others I'd bet as well are music.
50 cycles is _WAY_ too short unless you record/play on the very same device without touching anything (and no bad lines, sprites etc obviously).

2018-12-12 12:07

thrust64

Registered: Jun 2006
Posts: 8

That loader is mine. :)

And of course it was only meant for personal use on the same hardware. And hey, it worked for me! :D

I don't remember exactly how I came to 7 pulse lengths. But I did some statistics about bit combinations from some files I had and then some math based on the minimum pulse length differences which seemed feasible. And that resulted into the 7 lengths for optimal performance.

2018-12-12 12:11

thrust64

Registered: Jun 2006
Posts: 8

My target speed was 10k. That also played a role, IIRC.

2018-12-12 12:30

ChristopherJam

Registered: Aug 2004
Posts: 1409

I'm impressed :)

2018-12-12 12:47

enthusi

Registered: May 2004
Posts: 677

Ah good to see you here :)
I, too, was/am impressed, hehe.

2018-12-15 00:39

ChristopherJam

Registered: Aug 2004
Posts: 1409

I've managed to verify my hypothesis that the optimal weights to use for an arithmetic encoding based loader are those that ensure each symbol outputs the same number of bits per cycle.

Approximating the arithmetic encoder with a huffman code only loses one or two percent of efficiency, so that would probably be a somewhat saner path to take.

On the way, I found a marginally faster encoding using thrust64's pulse lengths. Using output words of { 11, 01, 101, 100, 000, 0011, 0010 } results in an average cost per bit of 102.9 cycles, compared
to thrust64's 108.8

Have some pretty graphs, and the script that generated them.

Each graph covers a family of pulse lengths, as given by the equations in the titles. The two lines show optimal cost per output bit, as a function of the number of pulse lengths used for the encoding.
This is purely for the data rate within each word; framing bits and error correction are outside the scope of this investigation, and would in practice add an overhead of 10-20%

There's a table after each graph giving the optimal encoding for the case where seven distinct pulse lengths are used. Note that if the gap is comparable to the shortest pulse length, the
optimal huffman encoding is isomorphic to transition=1, no transition=0, with some extra transitions inserted after long runs of zeros. This changes when the gap is relatively
small compared to the shortest length.

Using Thomas Jentzsch's pulse spacing (167+50*n cycles)
but optimising the encoding for that spacing

+----------------+-----------------+--------------+
| Pulse duration | arithmetic code | huffman code |
+----------------+-----------------+--------------+
| 167 cycles     | 0.31886         | 11           |
| 217 cycles     | 0.22645         | 01           |
| 267 cycles     | 0.16083         | 101          |
| 317 cycles     | 0.11422         | 100          |
| 367 cycles     | 0.08112         | 000          |
| 417 cycles     | 0.05761         | 0011         |
| 467 cycles     | 0.04091         | 0010         |
+----------------+-----------------+--------------+
mean cycles per bit, arithmetic code = 101.3
mean cycles per bit,    huffman code = 102.9

Using AR Turbo's short pulse, Fast Evil's spacing
(150+80*n cycles)

+----------------+-----------------+--------------+
| Pulse duration | arithmetic code | huffman code |
+----------------+-----------------+--------------+
| 150 cycles     | 0.39979         | 0            |
| 230 cycles     | 0.24517         | 10           |
| 310 cycles     | 0.15035         | 110          |
| 390 cycles     | 0.09220         | 1110         |
| 470 cycles     | 0.05654         | 11111        |
| 550 cycles     | 0.03468         | 111101       |
| 630 cycles     | 0.02127         | 111100       |
+----------------+-----------------+--------------+
mean cycles per bit, arithmetic code = 113.4
mean cycles per bit,    huffman code = 116.2

Using AR Turbo's short pulse, Fast Evil's min spacing,
increasing gap for longer pulses (150+80*n**1.1 cycles)

+----------------+-----------------+--------------+
| Pulse duration | arithmetic code | huffman code |
+----------------+-----------------+--------------+
| 150 cycles     | 0.41389         | 0            |
| 230 cycles     | 0.25856         | 10           |
| 320 cycles     | 0.15230         | 110          |
| 416 cycles     | 0.08660         | 1110         |
| 516 cycles     | 0.04810         | 11111        |
| 618 cycles     | 0.02640         | 111101       |
| 724 cycles     | 0.01415         | 111100       |
+----------------+-----------------+--------------+
mean cycles per bit, arithmetic code = 117.9
mean cycles per bit,    huffman code = 119.7

something a bit more conservative (180+ 90*n)

+----------------+-----------------+--------------+
| Pulse duration | arithmetic code | huffman code |
+----------------+-----------------+--------------+
| 180 cycles     | 0.38997         | 0            |
| 270 cycles     | 0.24352         | 10           |
| 360 cycles     | 0.15207         | 110          |
| 450 cycles     | 0.09497         | 1110         |
| 540 cycles     | 0.05930         | 11110        |
| 630 cycles     | 0.03703         | 111111       |
| 720 cycles     | 0.02313         | 111110       |
+----------------+-----------------+--------------+
mean cycles per bit, arithmetic code = 132.5
mean cycles per bit,    huffman code = 136.4

https://jamontoads.herokuapp.com/csdb/201812/compare_encodings...

No, I'm not going to go and write a loader based on this investigation in the near future. But, I hope this is useful or at least interesting to someone else out there.

Many thanks to SLC, Enthusi, thrust64 & tlr for their assistance.

2018-12-16 14:52

thrust64

Registered: Jun 2006
Posts: 8

Cool stuff. I remember that I did some similar research back then.

You limited your research to 7 different pulse lengths. Depending on the shortest pulse and the spacing, this may not be the optimal solution. E.g. for the last, conservative approach, I am pretty sure that a lower number of pulse lengths would give a better result.

Also, what sample data did you base your pattern distribution on? E.g uncompressed data must result into pretty different values depending on the content and very different than compressed one. For the latter, all bits and combinations should have about the same probability.

Previous - 1 | 2 | 3 | 4 - Next

Refresh

Subscribe to this thread: