| |
Six
Registered: Apr 2002 Posts: 293 |
Punter File Transfer Protocol
Reading the doc here: http://cbmfiles.com/genie/geniefiles/TelcomTools/C1-PROTOCOL-DE..
it outlines what seems like a very simple protocol. Seems easy to implement from scratch. To that end, I set up two tcpser instances, two instance of VICE with CCGMS 11, and began a transfer, observing the logging output of TCPSER.
What Steve Punter describes should go like this:
SENDER
[dummy packet 1]---ACK---[filetype packet]---ACK---[First file packet]---ACK---[Second file packet]....
RECEIVER
----------------GOO---S/B-----------------GOO---S/B------------------GOO---S/B-- ------------------GOO...
Where dummy packet 1 would be just the two checksums, 8 for the "next block size", and a block number of 0000, filetype packet is much the same, but with the filetype as the payload and a larger "next block size"
But what I'm observing looks more like this:
SENDER
--------------------------------------------------0x91---ACK---[odd data block]---ACK---SYN---S/B---ACK---[data block 1]
RECEIVER
GOO(over and over again until the transfer starts)----GOO---S/B---------------GOO---S/B---SYN---GOO---S/B----
Where the odd data block has a next block size of 4, and a block number of 0xffff and appears to have the file type in it.
My next step is going to be to try multi-punter on the same setup, then try a few different terms and see if the peculiarities are specific to this implementation of punter on CCGMS 11.
Has anyone else dug into this, or have any insight into the how/why? Is there a comprehensive documentation of this protocol anywhere? |
|
| |
Frantic
Registered: Mar 2003 Posts: 1648 |
The only thing I know about this is that the c*base implementation is available as sourcecode. |
| |
MagerValp
Registered: Dec 2001 Posts: 1078 |
Yeah Steve Punter's documentation is a joke. It's inaccurate in several places and upload is as far as I can tell unspecified. I managed to get download working in CGTerm (see punter.c) but I didn't have the patience to figure out how uploads work. |
| |
Six
Registered: Apr 2002 Posts: 293 |
MV: Aye, I noticed that. You may have caught my comment regarding as much in the usual place. "now I know why he balked at punter send" ;)
Certainly it can be reverse-engineered. I have looked over the c-base implementation, even started commenting over it and doing a rough convert to c, but only as a guide.
The end goal here is full documentation of the protocol and all of its variants, so that future implementations have a good starting point. Guess I'll keep slogging away at it unless anyone has some docs. |
| |
morphfrog
Registered: Mar 2002 Posts: 33 |
Hi
I know Soundemon did a c64 punter implementation for some time ago in skyterm, please ask him , probably can assist. |
| |
Six
Registered: Apr 2002 Posts: 293 |
Update:
First, Punter's explanation of the protocol is bunk. It may have applied to an earlier revision for the PET. Hell, he may have just been high as fuck when he wrote it, who knows?
Anyway, my implementation of punter SEND is now functioning up to the last block. (where I continue to work out the final handshaking)
How this REALLY works is as follows:
Sender waits for the receiver GOO.
Sender ACKs that GOO.
Receiver says S/B.
Sender sends an initial packet to indicate coming filetype. It is constructed in the following odd way:
Standard 7 byte header, with a next block size of 4(no idea why), and a block number of 0xffff (again, no idea why). The payload is one byte to indicate file type (1 for PRG, 2 for SEQ or also 3, but don't worry about 3)
The receiver will now GOO (or bad if there's line noise)
Sender ACKs that GOO.
Receiver says S/B.
[Here's wheRE iT gEtS sHittY...]
Sender SYN's that GOO.
Receiver responds with SYN.
Sender S/B's to that SYN.
Receiver will GOO.
Sender ACK's that goo.
Sender S/B to request sending file blocks.
Then we go into the file loop.
Sender Sends a block.
Receiver will GOO or BAD depending if the block was received properly.
Sender either resends until it gets a GOO, aborts, or ACKs the GOO.
Receiver requests another block with S/B
I'm working out the final block handshaking now, but I believe it mirrors the initial handshaking. Once I've got it fully figured out, I'll write up a formal protocol description for reference, as finding any solid documentation on this protocol has been akin to pushing butter up a tiger's ass with a burning stick. |
| |
Six
Registered: Apr 2002 Posts: 293 |
Handling the final block.
You're unlikely to run a whole file through in 248-byte sections, there's bound to be some slop at the end. So...
On the next-to-the-last block, make sure to set your NEXT_BLOCK_SIZE to the number of file bytes left + 7 (for the header)
On the last block, set the high-byte of your block number to 0xff, and the NEXT_BLOCK_SIZE to 0x00. When you receive the GOO for the last block, ACK it, then the receiver will S/B. Respond to that with SYN. The receiver will respond with SYN. Respond to that with S/B, and the transfer ends.
Next up, multi-punter, and a cleaned-up description. |
| |
Frantic
Registered: Mar 2003 Posts: 1648 |
@Magervalp: Looking forward to an updated version of CGTerm once Six has worked this out completely. ;) |
| |
Six
Registered: Apr 2002 Posts: 293 |
Note: There are numerous variations on this protocol. This post covers the version used in CCGMS 11.0 only. It has not thus-far been tested on other versions.
RECEIVER: GOO
SENDER: ACK
RECEIVER: S/B
SENDER: [filetype packet]
RECEIVER: GOO
SENDER: ACK
RECEIVER: S/B
SENDER: SYN
RECEIVER: SYN
SENDER: S/B
RECEIVER: GOO
SENDER: ACK
RECEIVER: S/B
SENDER: [dummy packet]
RECEIVER: GOO
SENDER: ACK
Loop<---------------------------------
RECEIVER: S/B |
SENDER: [file data packet] |
RECEIVER: GOO |
SENDER: ACK |
-------------------------------------|
RECEIVER: S/B
SENDER: [final data packet]
RECEIVER: GOO
SENDER: ACK
RECEIVER: S/B
SENDER: SYN
RECEIVER:SYN
SENDER: S/B
[filetype packet]
Normal packet, one-byte payload, 1 for prg, 2 for seq, next block size of 4, block number of 0xffff
[dummy packet]
Normal packet, no payload, next block size as needed for the sent file, block number of 0x0000
[final packet]
Normal packet, next block size of 0x00, and upper nybble of block number set to 0xff
[file data packet, Normal packet]
bytes 00, 01 : Additive checksum LO/HI
bytes 02, 03 : CRC Checksum LO/HI
byte 04 : Next block size
bytes 05, 06 : block number LO/HI
Checksums are calculated on all bytes from the Next Block Size forward to the end of the packet. Assuming TempPacket is your byte array of packet data and AdditiveChecksum and CLCChecksum are ushorts, checksum can be generated as follows (thanks to MagerValp for the original on this)
public void Generate_Checksum()
{
AdditiveChecksum = 0;
CLCChecksum = 0;
for (int i = 4; i < TempPacket.Length; i++)
{
AdditiveChecksum += TempPacket[i];
CLCChecksum ^= TempPacket[i];
CLCChecksum =
(ushort)((CLCChecksum << 1) | (CLCChecksum >> 15));
}
}
Validating received packets:
Received packets should pass a checksum test, and be the proper length specified in the preceeding packet.
Handshaking considerations:
You may receive handshakes out-of-sync (KCA, CAK, etc...), your handshaking routines should be able to deal with this.
|
| |
Six
Registered: Apr 2002 Posts: 293 |
For receivers, dealing with bad packets.
RECEIVER: BAD<---------------|
SENDER: ACK |
RECEIVER: S/B |
SENDER: RESENDS SAME PACKET |
RECEIVER: (BAD OR GOO)-------|
|
| |
Tao
Registered: Aug 2002 Posts: 115 |
@Six: I'd be happy if you'd have a look at the C*Base punter implementation; I have a slight recollection of something being wrong with it... |
| |
Six
Registered: Apr 2002 Posts: 293 |
Aye, I'm moving on to that next. I'll have to set up C*base on VICE for sniffing.
BTW, talked to Steve Punter himself about this on facebook. (VERY nice guy, btw, and he cleared some things up about the various versions.) He dispelled my suspicion that he was "high as fuck" when he wrote that document... His initial version did, in fact, work as he described. People took that initial version and made a number of variations on it, some of which were incompatible with it and eachother. Eventually a sort-of-standard seems to have been reached. Multi-punter, for instance, he had nothing to do with.
This is both good and bad for my research. It's good, because it means there is a lot of ground and forensics work to do yet, and that will be interesting, but bad because this means I have to root through every old term and BBS I can find so I can work out the various protocol variations in order to build a gold-standard. Looks like this will be a longer project than I originally assumed. |
| |
Six
Registered: Apr 2002 Posts: 293 |
Delving into multi-punter. It seems to be the same for each file transfer, but starts in an odd way from the sender. Each file is preceded by the following exchange:
SENDER: 0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09, FILENAME,P,0x0d
SENDER:GOO
RECEIVER: GOO
SENDER: ACK [and so on as a normal xfer.]
After all files are transferred:
SENDER: 0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09,0x09, 0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04,0x04, 0x0d
No response from the receiver after this. |
| |
Tao
Registered: Aug 2002 Posts: 115 |
Quoting SixAye, I'm moving on to that next. I'll have to set up C*base on VICE for sniffing.
Well, the source code is available, which might help ever so slightly. I did, however, not write the code, so if I were you I wouldn't bet on getting sane answers about the code :P |
| |
Six
Registered: Apr 2002 Posts: 293 |
Do you recall who did write that version? |
| |
Tao
Registered: Aug 2002 Posts: 115 |
Quoting SixDo you recall who did write that version?
Well, it's most probably written by Gunther Birznieks. Possibly by Jerome P. Yoner.
I've made some minor modifications, but that's all. |
| |
MagerValp
Registered: Dec 2001 Posts: 1078 |
Wow that is one seriously fucked up protocol. I don't think I could come up with something slower and more error prone if I tried. |
| |
Burglar
Registered: Dec 2004 Posts: 1101 |
Quoting MagerValpWow that is one seriously fucked up protocol. I don't think I could come up with something slower and more error prone if I tried. faster than xmodem! at least the xmodem used by ucbbs and cbase. punter would only work if u didnt have too much delay+linenoise, horrible but the fastest available. |
| |
Count Zero
Registered: Jan 2003 Posts: 1932 |
also xmodem padded blocks with a crap byte at the end - argl :) - thats not even an alternative to anything! |
| |
chatGPZ
Registered: Dec 2001 Posts: 11386 |
Quote:faster than xmodem!
indeed =)
what i find kindof funny is that in the original punter "docs" it says that the protocol was made especially for bad connections. ugh =) |
| |
Six
Registered: Apr 2002 Posts: 293 |
Well, it does do two different types of checksumming, and theoretically has a shifting block size one could use to deal with regular interference. (no-one implements this, at least as far as I have seen)
Multi-punter's the real train wreck. There's not even any facility that I can see to deal with missing the filename/retrying. |
| |
Didi
Registered: Nov 2011 Posts: 487 |
Found this today. Said to be sourcecode of Punter Protocol.
http://www.lemon64.com/forum/viewtopic.php?t=56823 |
| |
Six
Registered: Apr 2002 Posts: 293 |
Aye, it's one of them. Unfortunately doesn't appear to include multi-punter, though I've got that working now as well. |
| |
JackAsser
Registered: Jun 2002 Posts: 2014 |
Quote: Aye, it's one of them. Unfortunately doesn't appear to include multi-punter, though I've got that working now as well.
Could you please write a spec that supports both single and multi on CodeBase for future generations? :) |
| |
morphfrog
Registered: Mar 2002 Posts: 33 |
As I find this very interesting project , I must ask did you had some time to continue or are it halted right now? |
| |
Six
Registered: Apr 2002 Posts: 293 |
It's an ongoing project, but this month my focus is on getting material together for the ECCC get-together on 2015-08-29.
Multi-punter does now work in CBMTerm, but whether or not it would work with a variety of other terminals besides the version of CCGMS it was tested against remains to be seen.
The only reasonable next step is to set up a variety of BBS and terminal software and individually test them until the most common variants of multi-punter are ferretted out and can be handled by my protocol layer. Probably going to be into September on that one. |
| |
Larry
Registered: Feb 2007 Posts: 26 |
Did this ever got finished ? |
| |
morphfrog
Registered: Mar 2002 Posts: 33 |
Hi
A little bit curious about this project
Very intresting! |
| |
Six
Registered: Apr 2002 Posts: 293 |
Thread necromancy. Circled back to this because I need them to work cleanly for the Linux-based BBS. As of right now I can upload/download from/to all of the versions of CCGMS I've tested to the BBS. Does anyone know of any strange variations on Punter I could test/implement? |
| |
chatGPZ
Registered: Dec 2001 Posts: 11386 |
I'd also test Handyterm and Novaterm. Also the various incarnations of the C*Base builtin Terminal. |
| |
morphfrog
Registered: Mar 2002 Posts: 33 |
Quoting SixThread necromancy. Circled back to this because I need them to work cleanly for the Linux-based BBS. As of right now I can upload/download from/to all of the versions of CCGMS I've tested to the BBS. Does anyone know of any strange variations on Punter I could test/implement?
Please make it compatible with multi punter aswell that exists in CCGMS ( dosen't know if it do that in other termimal programs), very nice feature.Please also test the latest versions of CCGMS that alwyz had modified. |
| |
mist
Registered: Oct 2005 Posts: 5 |
Here is my own take at documenting the protocol:
https://www.pagetable.com/?p=1663 |
| |
Count Zero
Registered: Jan 2003 Posts: 1932 |
@mist: very nice! Also https://www.pagetable.com/?p=1641 (comes as a surprise) and https://www.pagetable.com/?p=1656 (nice article) are not way out of scope here and should be mentioned .) |
| |
chatGPZ
Registered: Dec 2001 Posts: 11386 |
cool indeed. now someone close the circle and knock up a proper freestanding (and complete) open source implementation :) |
| |
mist
Registered: Oct 2005 Posts: 5 |
Does wrapping the original implementation in a minimal 6502 environment and bridging it to stdin/stdout count? ;-) |
| |
chatGPZ
Registered: Dec 2001 Posts: 11386 |
urks =D |