Log inRegister an accountBrowse CSDbHelp & documentationFacts & StatisticsThe forumsAvailable RSS-feeds on CSDbSupport CSDb Commodore 64 Scene Database
You are not logged in - nap
CSDb User Forums


Forums > C64 Coding > Punter File Transfer Protocol
2015-02-24 23:04
Six

Registered: Apr 2002
Posts: 293
Punter File Transfer Protocol

Reading the doc here: http://cbmfiles.com/genie/geniefiles/TelcomTools/C1-PROTOCOL-DE..

it outlines what seems like a very simple protocol. Seems easy to implement from scratch. To that end, I set up two tcpser instances, two instance of VICE with CCGMS 11, and began a transfer, observing the logging output of TCPSER.

What Steve Punter describes should go like this:
SENDER
[dummy packet 1]---ACK---[filetype packet]---ACK---[First file packet]---ACK---[Second file packet]....
RECEIVER
----------------GOO---S/B-----------------GOO---S/B------------------GOO---S/B-- ------------------GOO...

Where dummy packet 1 would be just the two checksums, 8 for the "next block size", and a block number of 0000, filetype packet is much the same, but with the filetype as the payload and a larger "next block size"

But what I'm observing looks more like this:
SENDER
--------------------------------------------------0x91---ACK---[odd data block]---ACK---SYN---S/B---ACK---[data block 1]
RECEIVER
GOO(over and over again until the transfer starts)----GOO---S/B---------------GOO---S/B---SYN---GOO---S/B----

Where the odd data block has a next block size of 4, and a block number of 0xffff and appears to have the file type in it.

My next step is going to be to try multi-punter on the same setup, then try a few different terms and see if the peculiarities are specific to this implementation of punter on CCGMS 11.

Has anyone else dug into this, or have any insight into the how/why? Is there a comprehensive documentation of this protocol anywhere?
2015-02-25 07:17
Frantic

Registered: Mar 2003
Posts: 1648
The only thing I know about this is that the c*base implementation is available as sourcecode.
2015-02-25 13:27
MagerValp

Registered: Dec 2001
Posts: 1078
Yeah Steve Punter's documentation is a joke. It's inaccurate in several places and upload is as far as I can tell unspecified. I managed to get download working in CGTerm (see punter.c) but I didn't have the patience to figure out how uploads work.
2015-02-25 15:27
Six

Registered: Apr 2002
Posts: 293
MV: Aye, I noticed that. You may have caught my comment regarding as much in the usual place. "now I know why he balked at punter send" ;)

Certainly it can be reverse-engineered. I have looked over the c-base implementation, even started commenting over it and doing a rough convert to c, but only as a guide.

The end goal here is full documentation of the protocol and all of its variants, so that future implementations have a good starting point. Guess I'll keep slogging away at it unless anyone has some docs.
2015-02-25 16:08
morphfrog

Registered: Mar 2002
Posts: 33
Hi
I know Soundemon did a c64 punter implementation for some time ago in skyterm, please ask him , probably can assist.
2015-02-25 20:36
Six

Registered: Apr 2002
Posts: 293
Update:

First, Punter's explanation of the protocol is bunk. It may have applied to an earlier revision for the PET. Hell, he may have just been high as fuck when he wrote it, who knows?

Anyway, my implementation of punter SEND is now functioning up to the last block. (where I continue to work out the final handshaking)

How this REALLY works is as follows:

Sender waits for the receiver GOO.
Sender ACKs that GOO.
Receiver says S/B.
Sender sends an initial packet to indicate coming filetype. It is constructed in the following odd way:
Standard 7 byte header, with a next block size of 4(no idea why), and a block number of 0xffff (again, no idea why). The payload is one byte to indicate file type (1 for PRG, 2 for SEQ or also 3, but don't worry about 3)
The receiver will now GOO (or bad if there's line noise)
Sender ACKs that GOO.
Receiver says S/B.
[Here's wheRE iT gEtS sHittY...]
Sender SYN's that GOO.
Receiver responds with SYN.
Sender S/B's to that SYN.
Receiver will GOO.
Sender ACK's that goo.
Sender S/B to request sending file blocks.

Then we go into the file loop.

Sender Sends a block.
Receiver will GOO or BAD depending if the block was received properly.
Sender either resends until it gets a GOO, aborts, or ACKs the GOO.
Receiver requests another block with S/B

I'm working out the final block handshaking now, but I believe it mirrors the initial handshaking. Once I've got it fully figured out, I'll write up a formal protocol description for reference, as finding any solid documentation on this protocol has been akin to pushing butter up a tiger's ass with a burning stick.
2015-02-25 21:49
Six

Registered: Apr 2002
Posts: 293
Handling the final block.


You're unlikely to run a whole file through in 248-byte sections, there's bound to be some slop at the end. So...

On the next-to-the-last block, make sure to set your NEXT_BLOCK_SIZE to the number of file bytes left + 7 (for the header)

On the last block, set the high-byte of your block number to 0xff, and the NEXT_BLOCK_SIZE to 0x00. When you receive the GOO for the last block, ACK it, then the receiver will S/B. Respond to that with SYN. The receiver will respond with SYN. Respond to that with S/B, and the transfer ends.

Next up, multi-punter, and a cleaned-up description.
2015-02-25 23:01
Frantic

Registered: Mar 2003
Posts: 1648
@Magervalp: Looking forward to an updated version of CGTerm once Six has worked this out completely. ;)
2015-02-26 23:34
Six

Registered: Apr 2002
Posts: 293
Note: There are numerous variations on this protocol. This post covers the version used in CCGMS 11.0 only. It has not thus-far been tested on other versions.

RECEIVER: GOO
SENDER: ACK
RECEIVER: S/B
SENDER: [filetype packet]
RECEIVER: GOO
SENDER: ACK
RECEIVER: S/B
SENDER: SYN
RECEIVER: SYN
SENDER: S/B
RECEIVER: GOO
SENDER: ACK
RECEIVER: S/B
SENDER: [dummy packet]
RECEIVER: GOO
SENDER: ACK

Loop<--------------------------------- 
RECEIVER: S/B                        |
SENDER: [file data packet]           |
RECEIVER: GOO                        |
SENDER: ACK                          |
-------------------------------------|

RECEIVER: S/B
SENDER: [final data packet]
RECEIVER: GOO
SENDER: ACK

RECEIVER: S/B
SENDER: SYN
RECEIVER:SYN
SENDER: S/B

[filetype packet]
Normal packet, one-byte payload, 1 for prg, 2 for seq, next block size of 4, block number of 0xffff

[dummy packet]
Normal packet, no payload, next block size as needed for the sent file, block number of 0x0000

[final packet]
Normal packet, next block size of 0x00, and upper nybble of block number set to 0xff

[file data packet, Normal packet]
bytes 00, 01 : Additive checksum LO/HI
bytes 02, 03 : CRC Checksum LO/HI
byte  04     : Next block size
bytes 05, 06 : block number LO/HI

Checksums are calculated on all bytes from the Next Block Size forward to the end of the packet.  Assuming TempPacket is your byte array of packet data and AdditiveChecksum and CLCChecksum are ushorts, checksum can be generated as follows (thanks to MagerValp for the original on this)

public void Generate_Checksum()
{
    AdditiveChecksum = 0;
    CLCChecksum = 0;
    for (int i = 4; i < TempPacket.Length; i++)
    {
       AdditiveChecksum += TempPacket[i];
       CLCChecksum ^= TempPacket[i];
       CLCChecksum = 
(ushort)((CLCChecksum << 1) | (CLCChecksum >> 15));
     }
}

Validating received packets:
Received packets should pass a checksum test, and be the proper length specified in the preceeding packet.

Handshaking considerations:
You may receive handshakes out-of-sync (KCA, CAK, etc...), your handshaking routines should be able to deal with this.

2015-02-26 23:37
Six

Registered: Apr 2002
Posts: 293
For receivers, dealing with bad packets.

RECEIVER: BAD<---------------|
SENDER: ACK                  |
RECEIVER: S/B                |
SENDER: RESENDS SAME PACKET  |
RECEIVER: (BAD OR GOO)-------|

2015-02-27 11:27
Tao

Registered: Aug 2002
Posts: 115
@Six: I'd be happy if you'd have a look at the C*Base punter implementation; I have a slight recollection of something being wrong with it...
2015-02-27 21:46
Six

Registered: Apr 2002
Posts: 293
Aye, I'm moving on to that next. I'll have to set up C*base on VICE for sniffing.

BTW, talked to Steve Punter himself about this on facebook. (VERY nice guy, btw, and he cleared some things up about the various versions.) He dispelled my suspicion that he was "high as fuck" when he wrote that document... His initial version did, in fact, work as he described. People took that initial version and made a number of variations on it, some of which were incompatible with it and eachother. Eventually a sort-of-standard seems to have been reached. Multi-punter, for instance, he had nothing to do with.

This is both good and bad for my research. It's good, because it means there is a lot of ground and forensics work to do yet, and that will be interesting, but bad because this means I have to root through every old term and BBS I can find so I can work out the various protocol variations in order to build a gold-standard. Looks like this will be a longer project than I originally assumed.
2015-03-01 03:45
Six

Registered: Apr 2002
Posts: 293
Delving into multi-punter. It seems to be the same for each file transfer, but starts in an odd way from the sender. Each file is preceded by the following exchange:

SENDER: 0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09, FILENAME,P,0x0d
SENDER:GOO
RECEIVER: GOO
SENDER: ACK [and so on as a normal xfer.]

After all files are transferred:

SENDER: 0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09, 0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04, 0x0d

No response from the receiver after this.
2015-03-02 13:54
Tao

Registered: Aug 2002
Posts: 115
Quoting Six
Aye, I'm moving on to that next. I'll have to set up C*base on VICE for sniffing.


Well, the source code is available, which might help ever so slightly. I did, however, not write the code, so if I were you I wouldn't bet on getting sane answers about the code :P
2015-03-02 23:10
Six

Registered: Apr 2002
Posts: 293
Do you recall who did write that version?
2015-03-03 08:35
Tao

Registered: Aug 2002
Posts: 115
Quoting Six
Do you recall who did write that version?


Well, it's most probably written by Gunther Birznieks. Possibly by Jerome P. Yoner.

I've made some minor modifications, but that's all.
2015-03-04 19:48
MagerValp

Registered: Dec 2001
Posts: 1078
Wow that is one seriously fucked up protocol. I don't think I could come up with something slower and more error prone if I tried.
2015-03-04 19:59
Burglar

Registered: Dec 2004
Posts: 1101
Quoting MagerValp
Wow that is one seriously fucked up protocol. I don't think I could come up with something slower and more error prone if I tried.
faster than xmodem! at least the xmodem used by ucbbs and cbase. punter would only work if u didnt have too much delay+linenoise, horrible but the fastest available.
2015-03-04 20:34
Count Zero

Registered: Jan 2003
Posts: 1932
also xmodem padded blocks with a crap byte at the end - argl :) - thats not even an alternative to anything!
2015-03-04 22:38
chatGPZ

Registered: Dec 2001
Posts: 11386
Quote:
faster than xmodem!

indeed =)

what i find kindof funny is that in the original punter "docs" it says that the protocol was made especially for bad connections. ugh =)
2015-03-05 00:12
Six

Registered: Apr 2002
Posts: 293
Well, it does do two different types of checksumming, and theoretically has a shifting block size one could use to deal with regular interference. (no-one implements this, at least as far as I have seen)

Multi-punter's the real train wreck. There's not even any facility that I can see to deal with missing the filename/retrying.
2015-06-17 14:36
Didi

Registered: Nov 2011
Posts: 487
Found this today. Said to be sourcecode of Punter Protocol.
http://www.lemon64.com/forum/viewtopic.php?t=56823
2015-06-17 21:50
Six

Registered: Apr 2002
Posts: 293
Aye, it's one of them. Unfortunately doesn't appear to include multi-punter, though I've got that working now as well.
2015-06-18 07:17
JackAsser

Registered: Jun 2002
Posts: 2014
Quote: Aye, it's one of them. Unfortunately doesn't appear to include multi-punter, though I've got that working now as well.

Could you please write a spec that supports both single and multi on CodeBase for future generations? :)
2015-08-13 20:11
morphfrog

Registered: Mar 2002
Posts: 33
As I find this very interesting project , I must ask did you had some time to continue or are it halted right now?
2015-08-14 13:21
Six

Registered: Apr 2002
Posts: 293
It's an ongoing project, but this month my focus is on getting material together for the ECCC get-together on 2015-08-29.

Multi-punter does now work in CBMTerm, but whether or not it would work with a variety of other terminals besides the version of CCGMS it was tested against remains to be seen.

The only reasonable next step is to set up a variety of BBS and terminal software and individually test them until the most common variants of multi-punter are ferretted out and can be handled by my protocol layer. Probably going to be into September on that one.
2018-07-27 20:13
Larry

Registered: Feb 2007
Posts: 26
Did this ever got finished ?
2019-04-27 18:11
morphfrog

Registered: Mar 2002
Posts: 33
Hi
A little bit curious about this project
Very intresting!
2020-12-16 18:46
Six

Registered: Apr 2002
Posts: 293
Thread necromancy. Circled back to this because I need them to work cleanly for the Linux-based BBS. As of right now I can upload/download from/to all of the versions of CCGMS I've tested to the BBS. Does anyone know of any strange variations on Punter I could test/implement?
2020-12-16 18:55
chatGPZ

Registered: Dec 2001
Posts: 11386
I'd also test Handyterm and Novaterm. Also the various incarnations of the C*Base builtin Terminal.
2020-12-16 19:14
morphfrog

Registered: Mar 2002
Posts: 33
Quoting Six
Thread necromancy. Circled back to this because I need them to work cleanly for the Linux-based BBS. As of right now I can upload/download from/to all of the versions of CCGMS I've tested to the BBS. Does anyone know of any strange variations on Punter I could test/implement?


Please make it compatible with multi punter aswell that exists in CCGMS ( dosen't know if it do that in other termimal programs), very nice feature.Please also test the latest versions of CCGMS that alwyz had modified.
2022-03-15 09:43
mist

Registered: Oct 2005
Posts: 5
Here is my own take at documenting the protocol:
https://www.pagetable.com/?p=1663
2022-03-15 11:28
Count Zero

Registered: Jan 2003
Posts: 1932
@mist: very nice! Also https://www.pagetable.com/?p=1641 (comes as a surprise) and https://www.pagetable.com/?p=1656 (nice article) are not way out of scope here and should be mentioned .)
2022-03-15 15:04
chatGPZ

Registered: Dec 2001
Posts: 11386
cool indeed. now someone close the circle and knock up a proper freestanding (and complete) open source implementation :)
2022-03-15 16:00
mist

Registered: Oct 2005
Posts: 5
Does wrapping the original implementation in a minimal 6502 environment and bridging it to stdin/stdout count? ;-)
2022-03-15 16:05
chatGPZ

Registered: Dec 2001
Posts: 11386
urks =D
RefreshSubscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search   for   in  
All times are CET.
Search CSDb
Advanced
Users Online
macx
CyberBrain/NoName
Quetzal/Chrome
Guests online: 68
Top Demos
1 Next Level  (9.7)
2 13:37  (9.7)
3 Mojo  (9.7)
4 Coma Light 13  (9.6)
5 Edge of Disgrace  (9.6)
6 What Is The Matrix 2  (9.6)
7 The Demo Coder  (9.6)
8 Uncensored  (9.6)
9 Comaland 100%  (9.6)
10 Wonderland XIV  (9.6)
Top onefile Demos
1 Layers  (9.6)
2 No Listen  (9.6)
3 Party Elk 2  (9.6)
4 Cubic Dream  (9.6)
5 Copper Booze  (9.6)
6 Rainbow Connection  (9.5)
7 Dawnfall V1.1  (9.5)
8 Onscreen 5k  (9.5)
9 Morph  (9.5)
10 Libertongo  (9.5)
Top Groups
1 Performers  (9.3)
2 Booze Design  (9.3)
3 Oxyron  (9.3)
4 Triad  (9.3)
5 Censor Design  (9.3)
Top Coders
1 Axis  (9.8)
2 Graham  (9.8)
3 Lft  (9.8)
4 Crossbow  (9.8)
5 HCL  (9.8)

Home - Disclaimer
Copyright © No Name 2001-2024
Page generated in: 0.092 sec.