| |
Krill
Registered: Apr 2002
Posts: 2821 |
Release id #197710 : Transwarp v0.64
General Q&A thread, also report problems and error logs here. |
|
| |
iAN CooG
Registered: May 2002
Posts: 3132 |
Black magic. Care to explain what the challenge consists of? I see 2 USR files, trying to add ,U after the filename it gets loaded in loop. Can one load them without altering the diskimage?
Other than that, I guess this is a world record. Cheers. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
The USR files are encrypted. The key is a 40-bit CBM float, passed as 4th parameter to load. I.e.,LOAD"CIPHERED",8,1,π will load the first encrypted file with pi as the key (with previously installed Transwarp).
The challenge for the crackers is finding the key for the other encrypted file. There are only about a million million possibilities, so can be bruteforced. =) |
| |
Burglar
Registered: Dec 2004
Posts: 1031 |
This is truly amazing, it really is loading at the same speed as the fastest iffl-scanners around and these usually don't communicate with the c64 during scan and they only decode the first 2 bytes to capture just the track/sector link.
Not sure if anyone noticed, but loading a transwarp file directly, transparently loads the bootloader first and then the main file. that's a really excellent feature!
so, whats next? obviously the main gripe is that it uses non-standard encoding. a tool that converts regular d64s to transwarp encoded ones should be easy to do, maybe use >35 tracks for the extra space needed. if u want, hit me up on irc, wouldnt mind writing a crossplatform tool like that.
and of course I want a nosound option :) |
| |
Copyfault
Registered: Dec 2001
Posts: 466 |
The screenshots suggest that the effective loading speed depends on the video format, with standard PAL being the fastest.
As I only have a standard PAL C64 I'm not able to do comparisons of all the different speeds on the real deals. If my assumption is correct: what's the reason behind it?
And what iAN said: a brief summary of the "ingredients" you mixed in this uber-tasteful code-cocktail would be absolutely nice :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Burglarand of course I want a nosound option :) Transwarp respects the MSGFLG = $9d setting. That is, when not loading in direct mode, from a running program, there is no sound and no flashing. =)
As for a converter, i guess some script magic with the upcoming cc1541 version would do, no? :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting CopyfaultThe screenshots suggest that the effective loading speed depends on the video format, with standard PAL being the fastest. No, not at all. The "challenge" bit on NTSC in the GIF loads at 50x. The Dawnfall/NTSC bit merely illustrates that 50x is max speed, but not average. (And Dawnfall is one of the few one-filed demos to run flawlessly on NTSC, so...)
Loading speed is mostly depending on track zone. 1-17 is fastest, then it gets progressively slower on the upper track zones, as data on them is less dense.
Quoting CopyfaultAnd what iAN said: a brief summary of the "ingredients" you mixed in this uber-tasteful code-cocktail would be absolutely nice :) I thought iAN was mostly focusing on the challenge, but i can cook up a semblance of a white paper explaining things. =) |
| |
Jammer
Registered: Nov 2002
Posts: 1289 |
In general, is data preconverted that it doesn't have to be performed in runtime in 1541's ram? I'm total layman here, so my apologies :D |
| |
Copyfault
Registered: Dec 2001
Posts: 466 |
Quoting KrillQuoting CopyfaultThe screenshots suggest that the effective loading speed depends on the video format, with standard PAL being the fastest. No, not at all. The "challenge" bit on NTSC in the GIF loads at 50x. The Dawnfall/NTSC bit merely illustrates that 50x is max speed, but not average. (And Dawnfall is one of the few one-filed demos to run flawlessly on NTSC, so...)
Loading speed is mostly depending on track zone. 1-17 is fastest, then it gets progressively slower on the upper track zones, as data on them is less dense. Makes sense! Thanks for the hints, got a bit confused after seeing the sceendumps ;)
Quoting KrillQuoting CopyfaultAnd what iAN said: a brief summary of the "ingredients" you mixed in this uber-tasteful code-cocktail would be absolutely nice :) I thought iAN was mostly focusing on the challenge, but i can cook up a semblance of a white paper explaining things. =) Oops, maybe I misinterpreted what iAN wrote... but somthing white paperish would be extremly awesome. |
| |
iAN CooG
Registered: May 2002
Posts: 3132 |
I was just wondering how to load a usr file, usually you need to add ,U after the filename, I wasn't aware you had to load the standalone loader for these, as there was no instructions whatsoever. Some docs should be added to the release.
All I could determine is that ALL files just link to the same loader on track 18,16 and the bytes after the file name in the directory entries have "TW" plus the actual parameters to load the encoded files. Don't validate the d64s as it's just garbage for normal DOS. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting JammerIn general, is data preconverted that it doesn't have to be performed in runtime in 1541's ram? I'm total layman here, so my apologies :D Data is encoded differently. Plain Commodore GCR encodes 4 bytes raw to 5 bytes on disk, Transwarp encodes 3.5 bytes raw to 5 bytes on disk. (Thus only 223 = $df instead of 256 bytes per block, the 224th byte is the checksum.)
One trick is to encode those 3.5 bytes using a subset of plain Commodore GCR, such that it works just fine with d64 and regular transfer/copy solutions. Part of that trick is that decoding a GCR byte drive-side just takes a table lookup (4 cycles). =) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting iAN CooGSome docs should be added to the release. Yes, should. Didn't bother with those for the first experimental version, hoped the screenshot GIF would give enough information. =)
Quoting iAN CooGAll I could determine is that ALL files just link to the same loader on track 18,16 and the bytes after the file name in the directory entries have "TW" plus the actual parameters to load the encoded files. Don't validate the d64s as it's just garbage for normal DOS. Yes, that's part of the autoload feature Burglar mentioned. Regular (unencrypted) Transwarp files can be loaded simply with LOAD"HI!",8,1 from plain BASIC. =) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Submitted by JackAsser [PM] on 22 November 2020
Really impressive stuff! Any cons other than you can't validate a disk (who cares?!)? Well, GPLv3, depending on your stance. =D
But seriously, maybe the slight format loss, 223 instead of 254 payload bytes per block. And that it has the usual fastload drawbacks, of course. Except it can play dirty beats (with screen off) on oldsid. More cons may occur to me. |
| |
JackAsser
Registered: Jun 2002
Posts: 1987 |
Quote: Submitted by JackAsser [PM] on 22 November 2020
Really impressive stuff! Any cons other than you can't validate a disk (who cares?!)? Well, GPLv3, depending on your stance. =D
But seriously, maybe the slight format loss, 223 instead of 254 payload bytes per block. And that it has the usual fastload drawbacks, of course. Except it can play dirty beats (with screen off) on oldsid. More cons may occur to me.
Ahh yes. Speaking of GPLv3... had idéas to maybe include Transwarp in EotB for really fast loading of saves from disk, however I plan to release it as MIT och other freer licenses. Is GPLv3 compatible with more free licenses or will it taint the whole project?
Anyway, what about a fast saver. I could REALLY use that for EotB. :D Preferably with drum'n'base while saving. Would fit neatly in an RPG setting. ;) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting JackAsserIs GPLv3 compatible with more free licenses or will it taint the whole project? Transwarp is intended to be integrated with cartridges at some point (need to add 2-rev fallback loader for standard format, though). It hooks into the LOAD vector at $0330.
So my stance is that no tainting occurs if your binary uses the KERNAL's LOAD API on Transwarp files while Transwarp happens to be installed.
I.e., if Transwarp is not an integral part of a production and interchangeable with other loaders, a-okay. :)
Quoting JackAsserAnyway, what about a fast saver. I could REALLY use that for EotB. :D Preferably with drum'n'base while saving. Would fit neatly in an RPG setting. ;) Native saver is planned, but i have no idea when it will be done. Submitting the encoder patch to cc1541 repository will happen soonish, though.
Saving will probably be nowhere near as fast as loading, so i guess having the saver be of the arbitrarily interruptible IRQ kind allowing for sprites, music and whatnot would not be a problem. =) |
| |
JackAsser
Registered: Jun 2002
Posts: 1987 |
Quote: Quoting JackAsserIs GPLv3 compatible with more free licenses or will it taint the whole project? Transwarp is intended to be integrated with cartridges at some point (need to add 2-rev fallback loader for standard format, though). It hooks into the LOAD vector at $0330.
So my stance is that no tainting occurs if your binary uses the KERNAL's LOAD API on Transwarp files while Transwarp happens to be installed.
I.e., if Transwarp is not an integral part of a production and interchangeable with other loaders, a-okay. :)
Quoting JackAsserAnyway, what about a fast saver. I could REALLY use that for EotB. :D Preferably with drum'n'base while saving. Would fit neatly in an RPG setting. ;) Native saver is planned, but i have no idea when it will be done. Submitting the encoder patch to cc1541 repository will happen soonish, though.
Saving will probably be nowhere near as fast as loading, so i guess having the saver be of the arbitrarily interruptible IRQ kind allowing for sprites, music and whatnot would not be a problem. =)
Ahh ok! Good to know! EotB uses $ffd5 (load RAM) but by the looks of it, it's just setting the $c3/$c4 destination pointer and then calls ($0330), so easy to fix regardless. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting JackAsserAhh ok! Good to know! EotB uses $ffd5 (load RAM) but by the looks of it, it's just setting the $c3/$c4 destination pointer and then calls ($0330), so easy to fix regardless. I think doing JSR $FFD5 is just what the KERNAL inventors intended. =)
It's in the jump table sitting at the end of the address space. |
| |
MagerValp
Registered: Dec 2001
Posts: 1055 |
I'm absolutely floored, congrats on shipping! I remember us chatting about it 10 years ago, it's amazing that you figured everything out. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting MagerValpI'm absolutely floored, congrats on shipping! I remember us chatting about it 10 years ago, it's amazing that you figured everything out. Thanks! Has it been that long ago already? =) But yeah, i've had a draft in some text file for years, and "due to the current situation" (R)(TM) starting around March this year, i decided to type the stuff into an actual assembly source code. Turned out i had to come up with a few more tricks than envisioned initially, but.. yeah. Worked out fine in the end. :) |
| |
Silver Dream !
Registered: Nov 2005
Posts: 107 |
@krill - kudos! (everyone hats off please) |
| |
Capt'n Obvious
Account closed
Registered: May 2014
Posts: 4 |
Thank you for this incredible feat.
Do you think it can be ported to the VIC-20 ?
Can it be done with the VIAs ? |
| |
jcompton
Registered: Feb 2006
Posts: 70 |
Corner case report--
Hardware: Ultimate 64 running firmware v1.28 / 3.7
NTSC mode
Internally emulated 1541
Transwarp Game disk C
With stock ROMs:
Loading Spike behaves as intended.
With JiffyDOS:
When Spike finishes loading and the ding plays, although the speedometer feedback flashes briefly, the screen clears and goes to a READY. prompt top of screen (as though one has hit RS+RESTORE, etc.)
Spike does RUN just fine, however.
I can replicate this behavior on a cold start.
Can provide a documentary video on request. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Capt'n ObviousThank you for this incredible feat.
Do you think it can be ported to the VIC-20 ?
Can it be done with the VIAs ? Might be possible. But then it's only sensible for expanded VIC-20, no? The 3.5 KB of the stock machine are filled quite swiftly with Soundemon's loader. =) |
| |
jcompton
Registered: Feb 2006
Posts: 70 |
(Additional detail: the load works as intended with Jiffy enabled if I naively use LOAD"*",8 and LOAD"SPIKE",8. The problem only occurs when I use wedge commands.) |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
probably best to check first if it does the same on real hardware :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting jcomptonWith JiffyDOS: Ah, i have tested JiffyDOS compatbility in the early days of development, but forgot to re-check prior to release.
But i have different problems (in VICE): Spike loads and runs just fine, but Transwarp autoboot gets stuck.
Can be worked around by loading Transwarp (or Spike) with ,8 only (not ,8,1), then RUN.
Will contact you for testing the next version, then, but there is no release schedule yet. =) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting jcomptonThe problem only occurs when I use wedge commands.) Ah yes, can reproduce the issue with "%SPIKE" from installed Transwarp. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Groepazprobably best to check first if it does the same on real hardware :) Usually yes, but that kind of stuff (custom ROMs) is almost always a software issue that can be perfectly reproduced and debugged in emulators.
In this case, i had to buffer and restore COMSAV from this list http://unusedino.de/ec64/technical/aay/c64/mapjd.htm to fix the wedge issue and not use a certain drive ROM routine in the boot code. =) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
User Comment
Submitted by GI-Joe [PM] on 23 November 2020
Wow, that's really fast. Unglaublich geiler shice ! what really deserves the name ;) But with the standard file format an x30 loader ? without hardware-gcr decoding or memory expansion in the floppy ? I'm very excited about these releases.
Will this also work with other devices at the same time on the iec bus like i.e. the trackloader by mafiosino ? The 2-rev fallback loader for standard format is planned to work just fine with multiple drives on the bus, just like Transwarp is currently. :) (There is heavy ATN usage, but ATN responders running in the passive drives have been invented: S.T.F.U.)
But how fast it will ultimately be, we'll see. Mafiosino's loader is the current reference there, but i am aiming for a different approach. |
| |
Capt'n Obvious
Account closed
Registered: May 2014
Posts: 4 |
@Krill thank you for the answer, of course I was thinking of VICs whith 40Kb of SRAM or even more via banking. But now you make me think about streaming, even on the 5Kb VIC. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
User Comment
Submitted by Sparta [PM] on 23 November 2020
A jaw dropping, innovative tool from the Loader King! I can't wait to check the code... I am sure it is full of gems. :) Thanks! =) Okay, here's one gem, as we have talked about this stuff a while ago.
Sending bytes over serial.
The traditional method to send over bitpairs is this:$1800 ----C-D- CLK and DATA
___|_|
| __|
vv
$DD00 DC------ go there.Note how both bits are one bit apart in the send register but directly adjacent in the receive register, and they swap order on the way.
Both issues are usually solved by pre-scrambling or using look-up tables to do the scrambling.
This is a bit cumbersome, as it takes cycles and memory for the scrambling, or special pre-scrambled storage formats.
Now here's the trick: using ATNA.$1800 ---DC-0- DATA and CLK
__||
| __|
vv
$DD00 DC------ go there.It's that simple! :D
Note that DATA OUT ($1800 bit 1) must be cleared, and that both methods presented here are performated with ATN being unasserted.
The new method works with ATN asserted (as does the traditional one), but then the usual EOR-bitflip (DATA and CLOCK signals going from drive to C-64 are inverted) is done only on CLK but not on DATA.
Now sending over complete bytes can be done like this:
; value to send is in accu, eor #$ff occurs on receiving side
rol
rol
rol
ldx #ATNA + CLKOUT
[...]; wait here for ready-to-receive signal
sax PB
ror
ror
sax PB
ror
ror
sax PB
ror
ror
sax PBNote how no lookup is required. Also both methods can be mixed for a byte, or the initial 3 RORs be removed, when data is pre-scrambled anyways, such that some bitpairs can be sent faster with just 6 cycles between updates. |
| |
obliterator918
Registered: Aug 2013
Posts: 9 |
I would love to add this to my benchmark data at https://www.obliterator918.com/dtb/ ... but my code must be clobbering your loader. After saving then loading the data stream, it's just 1x. What part of memory do I need to preserve to make sure I don't clobber the loader?
Or, when a crt version comes out, will it work like other fast loaders and I won't have to worry about it?
Good job. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
Right now it does only load files that were preprocessed by the not yet released encoder, and there is no saving - i guess that is your problem? :) |
| |
obliterator918
Registered: Aug 2013
Posts: 9 |
Well yeah, that would do it, haha. I knew the load required specially encoded files, I just incorrectly assumed that saving files while it was loaded would do it. Obviously not, haha. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting obliterator918I just incorrectly assumed that saving files while it was loaded would do it. Obviously not, haha. A native saver will be added in a future version, and the cc1541 Transwarp encoder patch is yet to be finished and submitted.
But for now, you can just send me the files you'd like encoded, and i can send you the corresponding Transwarp images. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting obliterator918Or, when a crt version comes out, will it work like other fast loaders and I won't have to worry about it? The cartridge version is planned to be a replacement for Action Replay fastload + Warp*25. You will be able to both load and save files in standard format or Transwarp format. If it turns out as hoped, loading standard format files will be faster than 25x. =) |
| |
BiGFooT
Registered: Mar 2002
Posts: 31 |
Quote: The USR files are encrypted. The key is a 40-bit CBM float, passed as 4th parameter to load. I.e.,LOAD"CIPHERED",8,1,π will load the first encrypted file with pi as the key (with previously installed Transwarp).
The challenge for the crackers is finding the key for the other encrypted file. There are only about a million million possibilities, so can be bruteforced. =)
Now can I load SMB64 pretty fast, huh? ;) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
BiGFooT wins The Prize! (He just told me the key.) |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
Damn, that was quick :) |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1370 |
Nice work, BiGFooT! |
| |
Count Zero
Registered: Jan 2003
Posts: 1819 |
Congrats. |
| |
BiGFooT
Registered: Mar 2002
Posts: 31 |
Thank you guys.
Is anyone interested in "cracking" details? |
| |
iAN CooG
Registered: May 2002
Posts: 3132 |
bigfoot: of course |
| |
Dano
Registered: Jul 2004
Posts: 226 |
@bigfoot: yeah me, totally! |
| |
BiGFooT
Registered: Mar 2002
Posts: 31 |
Loading "CIPHER" and "CHALLENGE" file without a good parameter respond with
"71,DIR ERROR,00,00" and was the first hint to where to search. Block reading
directory with AR6 monitor (@BR 12 01 1000) and checking the entries showed
an interesting Transwarp structure after the filename, starting after the
"TW" marker. A quick trial with other files in the directory was helpful to
find out which means what. Byte 2&3 contains the load address, 4&5 is the end
andress.
(Example:
LOAD"ALGODANCER",8,8
SEARCHING FOR ALGODANCER
LOADING $0801-$B626: 2.6S, 16.9KB/S, 43X
.:1090 A0 A0 A0 A0 A0 54 57 57
.:1098 14 E2 01 08 26 B6 CB 00
)
It was totally different for the "CIPHERED" file. So I had to figure out the
"encryption" for this structure. It was pretty easy as you can load with the
PI parameter. It loads between $0801 and $517C, so the structure must be
this: ?? ?? 01 08 7C 51 ??.
The xor value was $DA, coming from $DB ^ $01, $D2 ^ $08,… easy enough, right?
Of course, this was a red herring... Finding the xor value of "CHALLENGE" was
just a few seconds ($DF ^ $01, $D6 ^ $08,…) and it was $DE. Based on the
structure it loads from $0801 to $D000. Okay, need to find out the rest.
The value of PI is hardcoded in the rom at $AEA8 as 82 49 0F DA A1.
Hmm… I saw that $DA just a second before! What if we change it to $DE?
LDA #$A8
LDY #$AE
JSR $BBA2
LDA #$DE
STA $64
JMP $BDDD
The result is in the memory at $0100 as 3.14159265. When you try to load
"CHALLENGE" you get the usual "71,DIR ERROR,00,00". Damn. This was the point
where the real work started and Krill’s released source code was so helpful.
Getting a label list for the compiled binary was a huge time saver as like
the various "key" labels to start with. Knowing that there is only two
"drive.key" variable (0,3) present I’ve tried to find out what is happening.
The directory encryption is based on the "key0" variable in
"drive.prepareload" fuction. It’s a plain xor, so what sets it? It’s at the
end of function "prepareload". Dissecting this function was fun, it sets up
the encryption keys and bit scrambling. "key3" came from "GRBPTR" xored with
value $C3. "GRBPTR" was set up in the same function from "floatbuffer", a
few JSR’s after proved that this variable is not a pointer at all.
Function "setscrable" divides the "GRBPTR" by 6 and the remainder used as
the offset of scrambling tables and called 4 times in total. Knowing this
decrypting the first half of the "key" was easy:
$8249 = 33353
33353 % 6 = 5; int(33353 / 6 ) = 5558
5558 % 6 = 2; int(5558 / 6 ) = 926
926 % 6 = 2; int(926 / 6 ) = 154
154 % 6 = 4; int(154 / 6 ) = 25
Based on this the "scrable0..3" offsets is 5,2,2,4, the value for "key3" is
25 ($19), so "key3" = $19 ^ $C3 which is $DA, the xor value of the directory
entry. We need a number for the "CHALLENGE" file $DE xor value, it’s
$DE ^ $C3 = $1D. We can create a new number from this!
((($1D * 6 + 4) * 6 + 2) * 6 + 2) * 6 + 5 = $9689
Asking our beloved breadbox with the previous floating number routine it
answers -2244608. Testing the "CHALLENGE" file with this parameter starts
loading. Now ready for next step…
Inspecting the source code for scramble labels pointed out two functions:
"receivchunk" (s0..2) and "recvblkbuff" (s3) and there’s a "key2" label just
before latter. Nice! Killing "recvblkbuff" at $e4bc, filling the memory and
loading "ALGODANCER" proved the function is not necessary to load and thus
there is no need to care about "key2". What else? "key3" at drive code with
some kind of buffer fizzling… As the recvblkbuff just killed, skip it. Okay,
"key4" and "scramble0..2" awaits. You should ask the question, what is
scramble? Transwarp can swap bits 10,32,54 in any order during transfer.
For example it’s possible to store a byte in bit order of 76105432,
761321054, etc.
It was a mere hour to create a converter to GCR sector, rip out decodetable,
re-implement the drive "readloop" and breadbox "receivchunk" function in PHP.
With this base it’s possible to decode and read the first $c0 byte of any
Transwarp. Dumping track 20 sector 0 showed that it’s the beginning of
"ALGODANCER". Bruteforcing the "CHALLENGE" for "key4" and "scramble0..2"
involved a neat little trick. Knowing that the program loads to basic start
address and occupy almost the whole memory one can assume that the 2nd byte
is $08 (next basic address), 5th is $9e (SYS basic token) and 7th-9th is a
PETSCII number. Just 4 loops (55296 iterations in total) and a compare later
got two hits for "CHALLENGE".
HIT: S0: 5 S1: 5 S2: 1 K4: a3 2b 08 e3 07 9e 32 30 36 39 35 80 2d 1d 1d ...
HIT: S0: 5 S1: 4 S2: 1 K4: ac 24 08 e3 07 9e 32 30 39 36 3a 8f 22 14 14 ...
Checking both result reveals that the 2nd hit is our valid key. Listing the
result showed something interesting.
2019 SMB.64, V1.2
With the help of CSDB, searching 2019 releases with name containing "v1.2" in
it show a mere list of 8 releases. "Super Mario Bros 64 V1.2" was the best
match, digging out the file from the deep proved that it’s the one you’re
looking for. With this information the rest is guessable again as the output
is already known. In about an hour later, implementing the "recvblkbuff" and
"bufferdata" related wheels the next cracker for key2..3 and scramble3 was
ready. It’s a little bit slower with the possible 393216 iterations but the
concepts remained the same. Decode everything whit known information and
matching the unknown with brute force. Surprisingly there was 3 hit with
different "key2" to short match:
HIT: S0: 5 S1: 4 S2: 1 S3: 0 K2: 13 K3: fa K4: ac a9 08 c9 e2 b0 dd ...
HIT: S0: 5 S1: 4 S2: 1 S3: 0 K2: 1f K3: fa K4: ac a9 08 c9 e2 b0 dd ...
HIT: S0: 5 S1: 4 S2: 1 S3: 0 K2: 75 K3: fa K4: ac a9 08 c9 e2 b0 dd ...
Comparing the whole output reveals that the only valid result was the latest.
With every information available we can solve the puzzle:
Scramble 0..3: 5,4,1,0
Key0: $1D
Key2..4: $75, $FA, $AC
((($1D * 6 + 0) * 6 + 1) * 6 + 4) * 6 + 5 = $9311
The floating point number thus the key we’re looking for is
$93,$11,$75,$FA,$AC
Converting this number with our beloved breadbin show the number 297903.834.
Great! Loading "CHALLENGE" with this parameter… is a fail. DAMN!
It was necessary to write a short program with more precision which reveals
the number 297903.83349609. Trimming the parameter by last digits got the
final answer is
297903.83349
The package with sources and the text is downloadable here: http://bigfootinformatika.hu/challenge.zip |
| |
Shine
Registered: Jul 2012
Posts: 327 |
WOW ... better than a good Sherlock Holmes Story!
What a dedication! Thanks a lot ... even i understood only 5% (maybe!) :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Excellent work! =)
Now i wonder, how hard is it now (and would have been before your work) to find a key for a file with an unknown plaintext (which also can't be guessed that early after just finding a few start bytes)?
I knew that picking a somewhat known file with known properties would ease it quite a bit, on top of the many sidechannels to attack, and of course the encryption isn't very strong to start with. But i didn't want to make it too tough as it might have spoiled the fun. :) |
| |
BiGFooT
Registered: Mar 2002
Posts: 31 |
Quote: Excellent work! =)
Now i wonder, how hard is it now (and would have been before your work) to find a key for a file with an unknown plaintext (which also can't be guessed that early after just finding a few start bytes)?
I knew that picking a somewhat known file with known properties would ease it quite a bit, on top of the many sidechannels to attack, and of course the encryption isn't very strong to start with. But i didn't want to make it too tough as it might have spoiled the fun. :)
It's only easy when you know what you want to find.
Plaintext is somewhat easy as you can count and match the characters. A binary however with a different start address and unknown entry point... would be much harder. Still, the first 256*6*6*6 iteration is not that much. For example you can guess the key with bitmap graphics pretty easily just by visualizing the memory and do a few trial and errors. Packed data is the worst to recover.
So yes, in this exact case "cracking" the key was fast, but less possibilities means less necessary resources to do so.
If the basic line got no hint then you're still able to guess it. Comparing with known depackers, or a parser based on the revealed information (validator for basic code, or machine code) as everything have some kind of pattern. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Ok, so who'd be in for a second challenge, then, with a high-entropy plaintext file that is still sensible (and rewarding to decrypt, not just random bitsalad)? :) |
| |
BiGFooT
Registered: Mar 2002
Posts: 31 |
Quote: Ok, so who'd be in for a second challenge, then, with a high-entropy plaintext file that is still sensible (and rewarding to decrypt, not just random bitsalad)? :)
I'm in with https://en.wikipedia.org/wiki/Frequency_analysis ;) |
| |
JackAsser
Registered: Jun 2002
Posts: 1987 |
@bigfoot: thanks!! |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting BiGFooTByte 2&3 contains the load address, 4&5 is the end
andress. Worth noting here, imho, is that Byte 1 contains a CRC8 checksum over the entire file. This is only computed (because slow) and checked when there are block checksum errors with retries, but ultimately a successful file load.
It does happen that funky drives or semi-broken disks load successfully eventually, and this second line of defence decreases the likelihood of false positives (good checksum despite bad data) quite a bit, even with the first line being somewhat more sophisticated than simple byte-wise EOR. :)
Ah, and byte 0 is the file's starting track. :D |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Another trick: JMP ($DD00) for less jitter when reacting to drive signals.
Normally you'd do things like- bit $dd00
bpl/bmi/bvc/bvs - to wait for a state change on the serial bus. This has a jitter of 7 cycles (on PAL, 8 cycles on NTSC/PAL-N because the C-64's CPU is clocked faster than the drive's CPU),
Using JMP ($DD00), this is reduced to 5 (6) cycles of jitter.
$DD01 (the RS232/parallel port bits connected to the user port) needs to be set to output and initialised to the high-byte of a jump table.
So it's trading size for speed, as this jump table is slightly more than $c0 bytes big, due to the incoming bus state bits sitting on bits 7 and 6. (Can of course stuff unrelated stuff into the gaps of this sparse table.)
Using this trick, it is possible to have a minimum of 6 cycles between serial bus signal updates (instead of 8 normally) without requiring further jitter reduction (using the half-variance technique), such that things likeldx #$0a ; CLKOUT + DATOUT
sax $1800; 2 data bits
asl
sax $1800; 2 more data bits sending multiple bitpairs over serial can be done quicker. |
| |
JackAsser
Registered: Jun 2002
Posts: 1987 |
Quote: Another trick: JMP ($DD00) for less jitter when reacting to drive signals.
Normally you'd do things like- bit $dd00
bpl/bmi/bvc/bvs - to wait for a state change on the serial bus. This has a jitter of 7 cycles (on PAL, 8 cycles on NTSC/PAL-N because the C-64's CPU is clocked faster than the drive's CPU),
Using JMP ($DD00), this is reduced to 5 (6) cycles of jitter.
$DD01 (the RS232/parallel port bits connected to the user port) needs to be set to output and initialised to the high-byte of a jump table.
So it's trading size for speed, as this jump table is slightly more than $c0 bytes big, due to the incoming bus state bits sitting on bits 7 and 6. (Can of course stuff unrelated stuff into the gaps of this sparse table.)
Using this trick, it is possible to have a minimum of 6 cycles between serial bus signal updates (instead of 8 normally) without requiring further jitter reduction (using the half-variance technique), such that things likeldx #$0a ; CLKOUT + DATOUT
sax $1800; 2 data bits
asl
sax $1800; 2 more data bits sending multiple bitpairs over serial can be done quicker.
Neat trick. Love it! Also like how you use the word ’stuff’ both as a verb and a noun in the same sentence! ❤️ Almost like the Smurfs smurfing smurfs up. |
| |
Richard
Registered: Dec 2001
Posts: 619 |
Transwarp's speed is amazing. It would probably make a great C64 filebrowser/menu system ;) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
User Comment
Submitted by ThunderBlade [PM] on 27 November 2020
I've waited for this since Heureka Sprint. The 64'er magazine had an article on it concluding with "In theory it should be possible to get even faster than that". Proven now! :) Hmm, but (slightly) faster systems than Heureka Sprint have been existing since before it was invented*. =)
It also seems like some conscious concessions were made for Heureka Sprint, paying with slightly slower speed for standard format compatibility. It uses a 70% encoding, not 75% as the original Vorpal (and its knock-off Action Replay Warp*25) does.
Transwarp does the same. If it were not for the intended d64 compatibility, it could use a slightly denser encoding and thus slightly more speed. But that decision was easy, nobody really wants to transfer weirdo g64 images to real disks. :)
* https://www.lemon64.com/forum/viewtopic.php?t=41482 |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting RichardTranswarp's speed is amazing. It would probably make a great C64 filebrowser/menu system ;) Not before there is a built-in 2-rev fallback fastloader for standard format. And that one will probably be somewhere slightly above half the speed of the custom format. |
| |
Copyfault
Registered: Dec 2001
Posts: 466 |
Quoting KrillAnother trick: JMP ($DD00) for less jitter when reacting to drive signals.
[...] This is so nice!!! And personally I don't care if this requires a jump-table, usually there's enough mem space on c64-side.
Thanks for sharing! |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting KrillOk, so who'd be in for a second challenge, then, with a high-entropy plaintext file that is still sensible (and rewarding to decrypt, not just random bitsalad)? :) Quoting BiGFooTI'm in with https://en.wikipedia.org/wiki/Frequency_analysis ;) New challenge image added, also containing some bonus material. =)
(But i'm really not so sure how much frequency analysis would help with this kind of scrambling.) |
| |
BiGFooT
Registered: Mar 2002
Posts: 31 |
My understanding of "plaintext" was "plain text", so freq. analysis came from that. Anyway, I don't want to play alone. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Another technique: using interrupts to break out of the block read loop without stop condition checks.
In a loader, the loop to read data from disk normally terminates after having collected the block's data.
This is usually done by things like "inx : bne loop" or "tsx : bne loop" when storing block data on the stack using pha.
This requires at least an index register, and 2 cycles to increase it or to retrieve the stack pointer, both then setting the Z flag for loop termination or continuation.
This overhead can be avoided, or rather shifted out of the read loop (which needs to be extra-tight, and every cycle comes at a premium).
Since block read and transfer is tightly coupled in a fastloader, the C-64 can signal breaking out of the read/transfer loop, and it can do so by triggering an interrupt on the drive side via asserting the ATN line.
This causes the drive ROM's interrupt handler to be executed. However, one cannot simply install an interrupt hook or overwrite the IRQ vector, as it is possible on the C-64. (1571 has an interrupt hook, though.)
With some preparations (the disk controller VIA timer needs to have been underrun) the ROM interrupt handler will then check the job code table at $00 for active jobs (job codes $80+).
Having a jump job code ($d0) for $0700 then would execute the block read user code at $0700 to handle the next block rolling by, effectively breaking out of the read loop which was executing before the ATN interrupt was triggered.
Note that the job table is checked backwards, starting with the RAM-less job for $0800, then going down to $0300. So having the code to execute at $0700 spends the least cycles in the ROM interrupt handler. |
| |
MagerValp
Registered: Dec 2001
Posts: 1055 |
Very nice, probably my favorite trick so far. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Submitted by JackAsser [PM] on 2 December 2020
Indeed awesome and now Krill can remove the encryption code becuase symmetric encryption with keys this small is utterly useless. 😂 better use the space for something else imo. But it was a cool challenge though, kudos for making it hard and still fast. Sure, sure. :) And yeah, i did it for the fun of it. But... if i can somehow make the checksum-CRC stuff work with any key, correct or false, it cannot be used to guide bruteforcing (which i totally expected to happen) this ridiculously small key (real search space <40 bits, apparently). :D Wonder how hard that would be... =) |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
nice tricks, tho I wonder a bit why did we have to wait 30 years for these :) I guess we were talking about non gcr irq loaders back in the ancient irc times already. probably the issue here is not many can code loaders. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quote:nice trick, tho I wonder a bit why did we have to wait 30 years for this :) I guess we were talking about the 9th sprite back in the ancient irc times already. probably the issue here is not many can code advanced raster routines. |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
that doesnt stand as many can code advanced raster routines. and next time please dont make it a quote as if I did say that :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Oswaldthat doesnt stand as many can code advanced raster routines. and next time please dont make it a quote as if I did say that :) I was just mimicking any random quote and didn't put an "Oswald" there on purpose. :)
But my point was that "Why wasn't this invented 30 years ago already?" pretty much goes for any kind of demo novelty on this platform, doesn't it? =) (No matter how hard or how few people are dabbling in a particular sub-field.) |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
ok maybe 30 was harsh, tried to understand what you did, could you explain what is ATNA used for normally ? the descriptions (i found) doesnt mean anything to me. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Oswaldok maybe 30 was harsh, tried to understand what you did, could you explain what is ATNA used for normally ? the descriptions (i found) doesnt mean anything to me. ATNA means "Attention Acknowledgement" and controls the hardware's response to incoming ATN ("Attention") signals.
The ATN line is what a bus master (the C-64) asserts when sending commands to bus devices. Any device on the bus would react to an incoming ATN signal by asserting the DATA line. The 1541 cannot control the ATN line, but only react to it.
Now, ATNA controls the ATN acknowledgement polarity.
If ATNA matches ATN (both are 0 or both are 1), nothing happens, and the DATA line is not asserted (but is left to unasserted if no device on the bus is "pulling" it, otherwise it's asserted anyways).
If ATNA does not match ATN, the DATA line is automatically asserted by hardware, signalling to the bus master that at least one device on the bus is responsive (if DATA wasn't asserted before, anyways).
This is also what disturbs ATN-clocked IRQ loader bitpair-transfer protocols (payload on DATA and CLK lines) with more than one device on the bus if no ATN responder code (make ATNA match the incoming ATN signal as quickly as possible in a tight loop) is executing on the passive devices. The DATA line then would be asserted when it's not supposed to be.
HTH =) |
| |
Count Zero
Registered: Jan 2003
Posts: 1819 |
Codebase is awaiting _at least some_ copy/paste work from here - keep it up :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Here's another trick, and it's not only useful for drive coding.
It's an old trick, however. Thanks to Sparta for reminding me of it. =)
Transwarp uses a table to map the sectors on a track to block indices.
With its interleave of 1, this looks quite boring on ordinary file tracks:sectortoindex .byte 0, 1, 2, 3, 4, 5, 6, 7, ..., 20 However, the last track of a file usually has fewer unique blocks than sectors.
It is filled up with redundant blocks in order to load as fast as possible, no matter which sector happens to roll by first after track step.
Consider the last file track having 5 unique blocks. Then the mapping would look like this:sectortoindex .byte 0, 1, 2, 3, 4, 0, 1, 2, ..., 0 Now here's the trick to generate the second mapping from the first, with the number of unique blocks as a parameter: ldx #0
ldy num_unique_blocks
- lda sectortoindex,x
sta sectortoindex,y
inx
iny
cpy #21
bne - This works quite elegantly without having to reset a run variable whenever the threshold of num_unique_blocks is reached.
With some self-modification, it can obviously be optimised to something like- lda ZP0,x
sta ZP1,x
inx
bne - taking just 13 cycles per loop (and fewer when unrolling, too).
That this works is also the reason why both memcpy and memmove calls exist in C-family runtime families. =) |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
I dont get it you start writing the second table from "num_unique_block" and nothing makes X to go back to 0 after counting to 4 with it ? |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
the "trick" is to copy one array into itself after the first N copies, there is a second copy at 0+N, which you can then copy again. no need to reset the variable :) just write down what the array contains after each step, you'll see it then :) |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
ah I thought its a typo not having different labels, as the table is presented twice in the text. cool trick :) |
| |
JackAsser
Registered: Jun 2002
Posts: 1987 |
Needs a scratch buffer of 20 bytes after the table right, to avoid memory corruption? |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
excess len is num_unique_blocks |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting JackAsserNeeds a scratch buffer of 20 bytes after the table right, to avoid memory corruption? No, it stops right at the end of the table, cpy #21. =) |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
fuck yeah, you are quick =D |
| |
JackAsser
Registered: Jun 2002
Posts: 1987 |
Quote: Quoting JackAsserNeeds a scratch buffer of 20 bytes after the table right, to avoid memory corruption? No, it stops right at the end of the table, cpy #21. =)
Duh ofc! sorry for being lame. |
| |
JackAsser
Registered: Jun 2002
Posts: 1987 |
Quote: Duh ofc! sorry for being lame.
I might be lame again (I do have another man cold + cough and fever so bare with me), but when do you reset it to 0..20? That is needed right? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting JackAsserbut when do you reset it to 0..20? That is needed right? Sure. It's done just before loading the first file track, while stepping there. Then left untouched until stepping to the last file track. And get well soon! =) |
| |
JackAsser
Registered: Jun 2002
Posts: 1987 |
Quote: Quoting JackAsserbut when do you reset it to 0..20? That is needed right? Sure. It's done just before loading the first file track, while stepping there. Then left untouched until stepping to the last file track. And get well soon! =)
Phew, I thought I'd lost "it". :D |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
wow, the fastloader boot is ammazing with and without ,1, btw why is $ never fastloaded? some transforming is going on by kernal?
whish I had this loader back in the day it feels like the disk turns into hdd. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Oswaldwhy is $ never fastloaded? Intended fallback to whatever comes from KERNAL (custom or original) or your fastload cartridge.
Didn't want to re-invent all that, as Transwarp is planned to go into cartridge images (Superfluid V0.7) at some point anyways (need to type in suitable fallback fastloader for standard format first, though). |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
ok intended, but why ? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Oswaldok intended, but why ? Because "fastloading" the directory would require extra code.
Normally when loading "$", the drive converts the directory to a BASIC "program" that can be listed, and that one is sent to the computer.
There is not much to speed up except the data transfer, and speeding that up would require uploading some extra piece of code to the drive.
Unless redundantly storing the directory in listable BASIC format with Transwarp encoding next to the normal directory.
Did not want to do any of that.
Just use F3 on Action Replay or equivalent wedge commands with other add-ons. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
I wonder... for the fallback standard-format loader, would there be any interest in support for a parallel connection?
Looking at my notes, it may well be this could turn out to load even faster than Transwarp-encoded files over serial. |
| |
Peacemaker
Registered: Sep 2004
Posts: 243 |
i have no clue about fastloaders or loaders in general. maybe a stupid question. but any chance to get a very fast irqloader version of your loader (great work btw)for our demos? :)
greets |
| |
BiGFooT
Registered: Mar 2002
Posts: 31 |
Quote: I wonder... for the fallback standard-format loader, would there be any interest in support for a parallel connection?
Looking at my notes, it may well be this could turn out to load even faster than Transwarp-encoded files over serial.
Just dumping out my realistic opinion here:
For daily use? Not really. Parallel transfer was a good solution back in time for backup purposes and to speed up the damn slow serial transfer. I don't see the this advantage today. Casual collectors use the standard variants, a few old farts have speed/dolphin machines and drives but I don't see the value of it. A fast, reliable serial fastloader is the best and more than enough for everyone. If there is a way to support the not real "replacements" (*IEC, anything not Ultimate) that would be the best for every user.
On the other point, an even faster experimental tool will be amazed and praised but won't be widely used at all further than trying it out. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting BiGFooTA fast, reliable serial fastloader is the best and more than enough for everyone. Noted. :)
Quoting BiGFooTIf there is a way to support the not real "replacements" (*IEC, anything not Ultimate) that would be the best for every user. Yes, i've been thinking about adding a fallback for SD2IEC and the like, using block-read commands and decoding on the C-64 side.
But then i still don't like the idea of explicit SD2IEC support... would probably have patched the firmware to support Krill's Loader, Repository Version 184 in the meantime otherwise. =) |
| |
Count Zero
Registered: Jan 2003
Posts: 1819 |
I think parallel cables are too few in use and even spread nowadays. Even in the old times not many people had them equipped I'd say.
Of course would be interesting to see speed differences and all but I suppose the work to put into that is not well spend, hm? :) |
| |
BiGFooT
Registered: Mar 2002
Posts: 31 |
Quoting KrillYes, i've been thinking about adding a fallback for SD2IEC and the like, using block-read commands and decoding on the C-64 side.
But then i still don't like the idea of explicit SD2IEC support... would probably have patched the firmware to support Krill's Loader, Repository Version 184 in the meantime otherwise. =)
Don't sit upside-down on a horse... Nope, those devices firmware should support your "damn fast" loader as it was the case with Dreamload. In any other case kernal fallback will always work, no need to support it explicitly. With "support" I've meant that you can aid/help 'em out if necessary.
If it's a must and they don't bother with yours, then there is a dedicated fastloader for those users. Even building a custom AR/RR ROM with different fastloader options would be lovely but no one really care about it. At least I don't. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
join #easyflash on freenode and talk to Unseen i say =) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting BiGFooTEven building a custom AR/RR ROM with different fastloader options would be lovely but no one really care about it. At least I don't. Plan is to add Transwarp including fallback fastloader for standard format (and saver for Transwarp encoding) to tlr's Superfluid V0.7 at some point. Would you care about that? =) |
| |
Burglar
Registered: Dec 2004
Posts: 1031 |
Quoting Krillwould there be any interest in support for a parallel connection? yes! its fairly easy to solder and is supported by vice. main issue would be no 1541u support for that afaik.
im already using a parallel setup by default (with dolphindos), as u know |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Another size-coding trick, but it's quite obvious in hindsight and also not new. =)
Transwarp buffers block data of up to 21 * $1f = $028b bytes per track in drive RAM, which are then transferred while stepping to the next track.
These 21 buffers are scattered in drive RAM (in rather harmless regions so the custom code can return control to ROM after loading without a soft reset).
The buffer addresses are stored in a table of 21 bytes, that is, 16-bit addresses are packed into 8-bit entries.
The table looks like this:>8:0082 c0 11 51 71 91 02 b2 d2 03 23 43 63 83 a3 c3 e3
>8:0092 04 24 44 64 84 The encoded addresses are $00c0, $0110, $0150, etc.
So, an address from that table can be depacked like this:.8:0578 B7 82 LAX $82,Y; read address table entry
.8:057a A2 0F LDX #$0F ; hi-byte mask
[...]
.8:0586 8F 92 05 SAX $0592; buffer address hi-byte
.8:0589 29 F0 AND #$F0 ; lo-byte mask
.8:058b 8D 91 05 STA $0591; buffer address lo-byte Note that the 4:4 granularity is arbitrary. With the 2K RAM of a 1541, it could also have been 5:3 to encode addresses from $0000 to $07f8 in steps of 8. (But for sync time-outs with removed disks etc., the code requires a dummy buffer in unmapped address space somewhere between $0800 and $1000.) |
| |
Copyfault
Registered: Dec 2001
Posts: 466 |
Really nice!
Looking at this code fragment, I'm wondering wether SHX could be used to shave off another cycle (and two bytes)...
ldx $82,y ; read address table entry
lda #$f0 ; lo-byte mask
sax $0691 ; store lo-byte to buffer@$06xx
shx $0692-offset,y ; mask hi-byte with #$07 and store to buffer
Ok, depends on quite some things (first of all, buffer location at $0600 must be allowed and hi-byte may never go over $07, offset=y so it must be known,...), but what I find most interesting: does an SHX used on the drive's cpu reliably have this "&h+1"-masking going on? It has no RDY-line, so should be the case... |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
that calls for some new test programs :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Copyfaultdoes an SHX used on the drive's cpu reliably have this "&h+1"-masking going on? It has no RDY-line, so should be the case... Afaik, it's reliable on all 6502 (and variants with illegals) systems without DMA interference. But yes, what Groepaz said. :)
And of course i optimised only as much as needed. =) (And Y is the buffer number and thus not a constant.) |
| |
Copyfault
Registered: Dec 2001
Posts: 466 |
Yes, y is the buffer no. (and thus not const), you wrote that in your post... but the SHX was too tempting so I dared to share the thought;) |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
transfering while stepping track, awesome idea :) remembers me when I tried to "stream" load vector anim, and it always stopped for track stepping haha. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Oswaldtransfering while stepping track, awesome idea :) remembers me when I tried to "stream" load vector anim, and it always stopped for track stepping haha. I think stepping itself is not to blame for the hiccup. =)
The normal* stepping duration in Transwarp is $90 (144) bycles, that's a fifth (18%) of a revolution, or 4 sectors. So for a regular interleave 4 loader, that means just one missed block. Plus they would use shorter stepping times, usually around $30-$40 bycles. (The $90 setting was determined experimentally, it was the safest setting across all test drives.)
However, the loader you used probably did not load blocks out of order, and then the mean waiting time for a block after trackstep is half a revolution, a tenth of a second, and the worst case double that time.
* With secondary addresses >= 2, the track step times for either or both half-track steps can be increased for drives with particularly long settle times or slow steppers, in order to counter stepping-related loading problems.
Bits 3..1 increase the overall stepping time: +2 = $a0, +4 = $b0, +6 = $c0, ..., +14 = $0100 bycles.
Bits 7..4 increase the time between first and second half-track step: +16, +32 etc. would shift the relation further towards the overall stepping time.
So loading a Transwarp file with ,8,255 would be pretty slow. =)
Having the secondary address take on more meanings than just loading to BASIC start or saved address, i was slightly dismayed to see BiGFoot use ,8,8 out of an old learned habit. =D |
| |
Count Zero
Registered: Jan 2003
Posts: 1819 |
https://codebase64.org/feed.php
Tap tap tap ... oh, thats the sound of my drive sometimes recently - not the frequency of your codebase submissions! :) |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1370 |
load ,8,8 considered harmful!? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting ChristopherJamload ,8,8 considered harmful!?
Nah, can be considered wholesome, but only if you have a rather wonky drive that would give you retries (transient read errors) and slowdowns (missed blocks) after stepping with the default settings. =)
Otherwise just an unnecessary throttle. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
A few words about the encoding scheme.
As is easy to see, it works with plain d64 images, effectively storing 224 bytes of data in a standard Commodore GCR disk block.
Also, attentive readers of the source might have noticed that just a single decoding table of $ae bytes with $69 entries ($45 spare gap bytes) is used.
Here's how it works:
Commodore GCR encodes 4 bit raw nibbles to 5 bits on disk, or 4 bytes raw to 5 bytes on disk.
Enumerating all possible values these 5 bytes can take, we get:11111111 11222222 22223333 33333344 44444444
$60 $60 $79 $60 $60 That's about 6.58 bits of entropy for the 4 outer GCR bytes each, and 6.92 for the middle GCR byte.
However, the possible values the bytes may have are determined by their predecessor (except the first byte).
After "decorrelating" these 5 sets of values, we are left with11111111 11222222 22223333 33333344 44444444
$40 $60 $19 $60 $40
6 6.58 4.64 6.58 6 So we can store 6 + 6 + 4 + 6 + 6 = 28 bits of data without interdependencies of the individual 5 GCR bytes in the same space where Commodore GCR encodes 32 bits.
When combining these reduced sets of possible values for all 5 GCR bytes, we get aheat map:
00: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- 08 08 08 -- 08 08 08 -- 08 08 08
30: -- -- -- -- -- 08 08 08 -- 08 08 08 -- 08 08 08
40: -- -- -- -- -- -- -- -- -- 18 19 19 -- 19 19 18
50: -- -- 13 13 02 1b 1b 1a -- 18 1b 1b 02 1b 1b 0a
60: -- -- -- -- 02 0a 0a 0a -- 18 1b 1b 02 1b 1b 1a
70: -- -- 13 13 02 1b 1b 1a -- 18 1b 1b 02 13 11 --
80: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
90: -- -- 03 03 02 07 03 06 -- 04 03 07 02 07 03 02
a0: -- -- -- -- 02 0e 0a 0e -- 1c 1b 1f 02 1f 1b 1a
b0: -- -- 13 13 02 1f 1b 1e -- 1c 1b 1f 02 1f 19 08
c0: -- -- -- -- -- -- -- -- -- 18 19 19 -- 19 19 18
d0: -- -- 13 13 02 1f 1b 1e -- 1c 1b 1f 02 1f 1b 0a
e0: -- -- -- -- 02 0e 0a 0e -- 0c 0b 0f 02 0f 0b 0a
f0: -- -- 03 03 02 0b 0b 0a -- 08 0a 0a 02 02 -- --
length 0xd9, offset 0x25, used 0x8a, density 63.6%, unique 0x0d
longest gap 0x13, gap space 0x2a The byte values here are just a bitfield to show which of the 5 GCR bytes may take on the index value.
Now, it takes a bit of trial and error for carefully picking 6-bit values to assign to each byte in a table like this, such that:
- each of the 4 outer GCR bytes can reach all 6-bit values from $00 to $3f
- the middle GCR byte can reach all 4-bit values from $00 to $0f.
There are many possible variations to satisfy these constraints, and the final decoding table can also be somewhat smaller than the heat map, as there are more than $40 value bytes to choose from across all 5 GCR bytes. So secondary goals are:
- minimise table size
- maximise gap space to stuff with code and data. |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
still trying to understand this, after 10 minutes I understood what you mean in the first part by bit entropies but still not understanding how your table works. how do you encode data into krill gcr ? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Oswaldhow do you encode data into krill gcr ? There are 5 different encoding tables for the 5 GCR bytes. 4 of them have $40 entries, 1 of them has $10 entries.
These encoding tables map raw 4- or 6-bit values to encoded GCR bytes.
The constraint is that an encoded GCR byte must map to exactly one decoded value for all 5 GCR bytes (in order to use just one single decoding table). Luckily, the values in the decoding table need not be unique, as in the same value may appear in multiple slots.
E.g., a specific GCR byte value may appear for the first GCR byte but is impossible for the fifth, and vice versa. Those two specific GCR values are allowed to decode to the same raw value, however. |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
*processing* |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Not sure where i've lost you. 8-bit GCR value goes into table, 6 bits come out, 1 out of 5 times, 4 bits come out. Repeat. =) |
| |
Oswald
Registered: Apr 2002
Posts: 5017 |
fex: "The constraint is that an encoded GCR byte must map to exactly one decoded value for all 5 GCR bytes" |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Oswaldfex: "The constraint is that an encoded GCR byte must map to exactly one decoded value for all 5 GCR bytes" ldx $1c01 ; read GCR byte
lda decodetable,x; decode Given this one decoding look-up table for all GCR bytes, there is no way to magically map a GCR byte value to more than one decoded value, is there? =) |
| |
WVL
Registered: Mar 2002
Posts: 885 |
Quote: Not sure where i've lost you. 8-bit GCR value goes into table, 6 bits come out, 1 out of 5 times, 4 bits come out. Repeat. =)
How can 6 bits come out if there are only 20 different values in that table? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting WVLHow can 6 bits come out if there are only 20 different values in that table? Do not confuse the heat map with the actual decoding table, which i haven't posted. "The byte values here are just a bitfield to show which of the 5 GCR bytes may take on the index value."
I merely described a recipe of how to generate the mapping. As there is a lot more than one valid solution, i refrained from picking any specific decoding table to show here. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Not a new trick, but a lesson i learned:
It's a good idea to skip any SYNC happening to roll by just when going to wait for a SYNC, like so
skipsync bit $1c00
bpl skipsync
waitsync [...]; SYNC time-out check
bit $1c00
bmi waitsyncThe reason is that custom loaders usually do quite a few things between the SYNC signal and the first data bytes rolling in. As the SYNC signal on standard-format disks is much longer (5 times) than the time between adjacent data bytes coming in from disk (about 26 to 32 cycles depending on track zone/density), this extra time is best not left unused in highly optimised loaders.
Now, when not skipping a current SYNC before waiting and "jumping" right into the middle or even the end of a SYNC, it may happen that there is, in fact, not the time the code requires before data rolls in, and faulty data is read.
This problem is mitigated to some extent by checksumming, but the checksum is rather weak. Faulty data may slip through, resulting in creative errors some time after actual loading.
Chances for this to happen are somewhat higher in an IRQ-loading setup, right after trackstep, and when the motor is spinning up when starting to load a file. |
| |
tlr
Registered: Sep 2003
Posts: 1702 |
Quoting Krill
Not a new trick, but a lesson i learned:
It's a good idea to skip any SYNC happening to roll by just when going to wait for a SYNC, like so
skipsync bit $1c00
bpl skipsync
waitsync [...]; SYNC time-out check
bit $1c00
bmi waitsyncQuote:
Good point! Haven't really thought about that, but might explain a few problems I've encountered during the years. :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting tlrHaven't really thought about that Me neither, until there were some weird transient errors during testing. This patch is one of the things i must backport to the IRQ loader. =) |
| |
Bitbreaker
Registered: Oct 2002
Posts: 498 |
Quoting KrillNot a new trick, but a lesson i learned:
It's a good idea to skip any SYNC happening to roll by just when going to wait for a SYNC, like so
The problems described sound very wellknown to me, and that is finally a good explanation why those effects happen. On the other hand, this might speed up spinup time and such in future. My first thoughts when reading this was measuring the time between two syncs, and one step further measuring the rpm and adapting a few things if too far out of range. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Not a trick, but a caveat that has historically plagued a few loaders (former self of mine also guilty as charged on this one):
Both serial bus lines (DATA and CLK) cannot be expected to switch state at the same time.
Various analogue effects are involved, and even when flipping both bits with a single register write, the other side cannot be expected to see both changes at the same time.
E.g., when waiting for a state change using "wait: BIT $DDOO : b?? wait" and some state is communicated on the other line, $dd00 should be read again.wait: [...] do something while waiting
bit $dd00
bmi wait; wait for DATA to become asserted
bit $dd00
bvs error; just an example |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
From https://csdb.dk/release/?id=198558&show=review User Comment
Submitted by KAL_123 [PM] on 1 January 2021
Nice demo. I freezed it for my SD2IEC with a Retro-Replay, it worked. No need for dirty cartridge freeze tricks, really.
1. Use a cartridge (image)/KERNAL extension with proper saving of RAM below ROM (program code exceeds $A000), such as Action Replay fastload or similar
2. LOAD"*",8,1
3. Once the screen goes blank and starts flashing, press and hold the STOP key
4. After loading has finished, you will be back to BASIC prompt
5. Mount/insert a fresh disk (image)
6. SAVE"LOVECATS",8
You now have a clean standard-encoding copy. Enjoy with SD2IEC et al.! \=D/ |
| |
soci
Registered: Sep 2003
Posts: 473 |
I couldn't just quickly load that release over PCLink to watch it like I normally do for these sort simple stuff.
Instead it had to be written out to a disk and loaded from there. Completely defeats the purpose of it's fast loading feature if it takes more time to get to the point where it actually starts to show something.
Just for fun I tried with my 1541U2 which failed of course. Not that it matters much as I have plenty of disks and drives. Yeah, sure, everyone should probably buy the latest version right?
I have no idea what the authors were smoking. If the release is targeted to emulators or 1541U and such then for single file stuff ram injection is the way to go anyway. But for real hardware these sort of "packaging" does not add any value at all!
It's not your fault Krill of course as it's just a tool and can be used in any way. No offence, it's cool stuff.
But really, people should think before they release to make their stuff more accessible. E.g. how difficult is to upload a standalone prg version as well?! |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting sociInstead it had to be written out to a disk and loaded from there. Completely defeats the purpose of it's fast loading feature if it takes more time to get to the point where it actually starts to show something. That's true for single-consumption fire and forget stuff. For repeated playing on real original hardware, it makes sense to have it packaged like it is.
Quoting sociJust for fun I tried with my 1541U2 which failed of course. Not that it matters much as I have plenty of disks and drives. Yeah, sure, everyone should probably buy the latest version right? Pretty sure just upgrading the firmware to something recent should suffice.
Quoting sociI have no idea what the authors were smoking. If the release is targeted to emulators or 1541U and such then for single file stuff ram injection is the way to go anyway. But for real hardware these sort of "packaging" does not add any value at all! It does for multiple re-use, and on original hardware (plain vanilla stock 1541).
Quoting sociBut really, people should think before they release to make their stuff more accessible. E.g. how difficult is to upload a standalone prg version as well?! I guess they wouldn't be mad at anyone adding that to the entry. =) |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
Quote:Pretty sure just upgrading the firmware to something recent should suffice.
works fine with firmware 3.7 (116) here.
what a strange complaint this is :) |
| |
Digger
Registered: Mar 2005
Posts: 421 |
Could TransWarp technically work with C16 or Plus/4? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting DiggerCould TransWarp technically work with C16 or Plus/4? On 1541, i think it should work*.
On 1551, single-revolution full-density fastloaders have been invented long ago, no?
Would there be interest in a 1541 port for C16?
* You do get the full 1.76 MHz anywhere on screen with display disabled, yeah? I've also just read that you can even get to 2.22 MHz with display disabled. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
A new caveat just rolled in, courtesy of Morphfrog:
If there is a "SYSTEM INCOMPATIBLE" message on install and/or a "?COMPATIBILITY ERROR." (v0.64)/"?COMPAT ERROR." (v0.68) on load, try removing anything connected to the user port. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Won't get around finishing clean cc1541 Transwarp encoder integration for a while.
For the time being, anyone interested in playing around with it or even releasing something with it can drop me a mail to receive patched cc1541 binary with current transwarp.prg and an example shell script, like the Atlantis dudes did with Lovecats not long ago. =) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quote:User Comment
Submitted by Zaphod [PM] on 15 August 2021
Is there some reason this can't be made into a kernal replacement for easyflash after you make the 30x fallback loader? The code would likely not fit in a KERNAL replacement without a counterpart replacement ROM in the drive itself (at which point there would be some more speed improvement options, but no "stock hardware" feature). And i guess the licenses will clash when basing this on the original KERNAL and drive ROMs. A cartridge fastloader would be the way to go.
Quote:Or make Transwarp Ocean Loader (laugh). You'd load the file it would kick in the fast loader, set up the screen, load the music, start the tune, load the color ram, load the picture, flash the borders while the main crunched proggy loads, then stop the border flashing and wait for you to press space in case you want to listen to more music. :) Keep in mind it's still a fastloader, so screen is disabled while loading. You can fastload a start screen, wait for a key press (while optionally playing music or animation), then fastload the main program, though. The showcase images for the upcoming next release will have at least 2 games demonstrating this kind of multiload. (It's also possible to fastload files from the main program, when leaving the resident code and things like the load vector untouched.) |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
Well i'm wondering what an IRQ loader could do. :)
But more useful would be making versions of Lucasfilm stuff that WORK FROM D64.
LOAD":*",8,1
SEARCHING FOR :*
(black screen)
load transwarp.
load Lucasfilm intro, and play it.
load main game.
start.
Requests.
Ballblazer
Koronis Rift NTSC (this might be tricky, dunno if there's enough space to fit transwarp in on the disk)
The Eidolon (with music patch to make it sound like the early version)
Rescue on Fractalus (based off of vorpal version, not off of wacked out Joe Loader version)
the goal is a .d64 version that loads at least as fast as the original disk after it's be parametered, and acts similar to the original. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
You can use Transwarp as a booter for games, which would then proceed using an IRQ loader and continue loading standard-encoding files in the background. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
Then again, in 2021 you can just make an EF version that "loads" even faster :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting GroepazThen again, in 2021 you can just make an EF version that "loads" even faster :) Indeed, games and a flashable cartridge go very well together. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
Those games didn't IRQ load to begin with.
first version of rescue on fractalus used a variant of joe loader that used a different directory track and that sd2iec doesn't recognize. there are cracks of it that works with standard joe loader, but they do not load and save high score properly.
But the real point of this tool is to make d64s that work on a stock 1541 and c64, and load super fast.
Vorpal itself is 10 blocks (VLOADER from the vorpal utility kit), and itself fastloaded with a 1 block autoboot (VBOOT from that same disk).
I see the demo disks cheat and shove 19 or so blocks of transwarp into the directory track, making the loader effectively only take 13 blocks. This would make it difficult to adapt to games that use a lot of files after the main load instead of a track and sector loader.
Installing transwarp to the directory track is not really a solution for the cart. but it needs to have the ability to install transwarp to the disk. Also a way to install a transwarp with a MENU, or to just link it into the first file the way the demo disks do it.
How many blocks (before encoding) can transwarp currently load? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting ZaphodThose games didn't IRQ load to begin with. Neither will Transwarp ever IRQ-load. =)
Quoting ZaphodI see the demo disks cheat and shove 19 or so blocks of transwarp into the directory track, making the loader effectively only take 13 blocks. This would make it difficult to adapt to games that use a lot of files after the main load instead of a track and sector loader. The Transwarp boot file does not have to occupy blocks on the directory track. That is merely an optimisation to have more space for Transwarp files and have an optimised sector layout for faster bootstrapping. There can only be 33 (38) Transwarp files on a disk anyways, as they do not share tracks with other files.
Quoting ZaphodAlso a way to install a transwarp with a MENU, or to just link it into the first file the way the demo disks do it. Any kind of menu program that does not install a fastloader but uses the plain KERNAL load call would work. You'd just put this on the disk first, with Transwarp encoding, and it would load with Transwarp, then load any files from the directory listing using Transwarp (without booting it again, it's still installed in memory) as well. Such a simple menu program could probably even be made in BASIC.
Quoting ZaphodHow many blocks (before encoding) can transwarp currently load? Files can load to anywhere in the $0400..$CFFF range. About 206 standard blocks. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
ahh, so transwarp hides under the kernal.
and of course it looks at the bytes after the filename to determine if it's a transwarp file, and if not (like for directory) banks the kernal back in and JSRs it. I really want to see transwarp koronis rift NTSC version. compressing a bit if if must to fit it all on the disk.
I'm still wondering how the heck it loads ITSELF so fast when you use ,8,1
Archon was well chosen as a game, since it apparently got a vorpal release in europe at sometime.
What is the theoretical max for a "play with sprites and music" or "play with picture and music" loader? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Zaphodahh, so transwarp hides under the kernal. Resident code also resides under IO ($Dxxx), plus some code in lowmem ($02xx and $03xx) with subroutines for memconfig switching, calling ROM routines, etc.
Quoting ZaphodI'm still wondering how the heck it loads ITSELF so fast when you use ,8,1 The first two blocks are a little autostarting fastloader which loads and runs the rest of the file.
Quoting ZaphodWhat is the theoretical max for a "play with sprites and music" or "play with picture and music" loader? Modern IRQ loaders typically make something around 20x tops, but how fast they are depends on how much is going on while loading in the background. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
ahh. and the cart version will be in cart ROM instead of under the kernal.
issue is fitting both transwarp and a vorpal speed standard loader.
Now i'm wondering what a transwarp analog for sd2iec would be. clearly it's possible to make one. I refuse to believe that the 1541 can transmit faster than the sd2iec is capable of. :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Zaphodissue is fitting both transwarp and a vorpal speed standard loader. Modern cartridges have practically unlimited ROM space. Should be no problem to fit multiple fastloaders.
Quoting ZaphodNow i'm wondering what a transwarp analog for sd2iec would be. clearly it's possible to make one. I refuse to believe that the 1541 can transmit faster than the sd2iec is capable of. :) A native SD2IEC fastloader would be bound only by the serial bus limitations. Max transfer speed is somewhere around 32 KB/s, so upwards of 80x. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
Which raises the question of why it isn't around yet.
we can skip the entire gcr encoding issue, and load standard files for sure.
can jiffydos itself be beaten for a screen on loader for sd2iec? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Zaphodcan jiffydos itself be beaten for a screen on loader for sd2iec? That would boil down to the question whether the JiffyDOS protocol is the fastest possible protocol to leave the screen on while loading. I've never looked at it closely, but i'd be surprised if it cannot be improved. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
hmm. it's 2 bits but also allows for run/stop. If that's sacrificed you can probably get a bit faster. but it's on par with state of the art irq loading.
With screen off it really should be possible to do better than 25x, though... |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Zaphodhmm. it's 2 bits but also allows for run/stop. If that's sacrificed you can probably get a bit faster. No need to sacrifice checking the stop key, really. The check takes just a few cycles, and it's sufficient to do that between incoming blocks rather than between bytes.
Quoting Zaphodbut it's on par with state of the art irq loading. Which hints at some optimisation potential. =)
Quoting ZaphodWith screen off it really should be possible to do better than 25x, though... Remains to be seen. Probably needs a few tricks beyond more efficient bus transfers. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
Any progress?
Btw, awesome version number (laugh)
Is there room to add transwarp to koronis rift NTSC version?
It would be nice to have a version that fastcopies and loads fast. :) |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting ZaphodAny progress? Progress with what, exactly? =)
Quoting ZaphodIs there room to add transwarp to koronis rift NTSC version? Dunno. Do you have an image of the original that i could have a look at? But if the game loads data during gameplay, pretty likely there's not enough free space. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
I do have an original.
there are some actual free sectors to work with.
the following sectors are actually confirmed free. (filled with 4b 01 repeating)
2,14 to 2,20
3,20
4,0 to 4,20 (all of track 4)
5,5 to 5,20
18,7 18,8 18,10 18,11 18,13 18,14 18,16 18,17
31,8 to 31,16
35,6 to 35,16
The main program is tracks 7 to part of 17, and is 217 vorpal blocks as near as I can tell. the rest of track 17 is normal dos data. 233-234 transwarp blocks. the completely free track 4 would make up for it if we moved it to start there, and moved the remaining 3 blocks to the end of track 31, assuming a transwarp file can start in the middle of a track.
The lu logo intro is tracks 19 through 22, , and is 69 vorpal blocks as near as I can tell, with the rest of track 22 as normal data. This would be 74-75 transwarp blocks, and you should be able to move the final 12 blocks to the end of track 5.
the lu logo intro might be compressible. the main program will be hard to compress.
others may get freed with the vorpal loader getting replaced, with the protection check patched out, and by compressing some of the vorpal data after conversion to transwarp.
http://eunich.cochems.com/~zaphod/Koronis%20Rift.g64
NTSC version.
The existing non vorpal data will be challenging to relocate. the PAL version is in files, but it doesn't irq load, which breaks immersion.
Ballblazer (also NTSC) would be a much easier conversion, as it doesn't nearly fill the disk, and it's a single load. want it too? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
I have found some images of Koronis Rift originals over at The Internet Archive.
But the problem is not with available disk space (can always have some ever-so-slight compression to get it to just below the threshold), but enough space in RAM for the resident loader portion.
And as for Koronis Rift itself, it wants an IRQ loader, as you have pointed out here already. =) |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
It doesn't need to remain resident after the main program loads.
and it doesn't need to hook kernal.
load stub. stub loads transwarp. transwarp loads logo intro, plays it, then relocates to somewhere it can fit while loading the now compressed main program. decompress, clobbering transwarp.
the vorpal loader is the same. it's not used after main program loads. it hides between $400 and $800, and loads both the intro and the main program from there. the main program clobbers it after. so transwarp can hide there too while loading the main program.
the irq loader that's used after reads standard sectors, so as long as they are in the right place, we are good.
ballblazer is a much easier one, vorpal in high memory there.
ahh the main program loads clear up to $d370. Which is why they used the $400 variant of vorpal but i think it still might be possible to relocate some of the code to the area between $400 and $800 to pull it off?
okay, figured it out. main program CAN be compressed enough to get it to load below transwarp. AR 5 was able to get it down to 193 blocks*254 bytes per block+2048, which loads up to C77E, which won;t clobber transwarp until after it's started, which is fine, cuz it's no longer needed.
Want the 193 block exe that starts the main game? |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Sounds like back to booter business, so you'd just find a crack with IRQ-loading of game data intact and just save the main program in Transwarp format. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
main program in transwarp. intro ALSO in transwarp. looks like there is enough space on the disk fo rit with some compression. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Disk space doesn't seem to be an issue at all, and the (partial) compression on the main program would be just to make it end below $d000.
Couldn't find a proper crack that has IRQ-loading of game data on plain files via directory (the cracks with IRQ-loading still use the same track/sector-based loader), and the music in those cracks (based on the original NTSC release, not the rather bad European Activision release without IRQ-loading) plays at the wrong speed on PAL.
No plans to make a proper crack myself. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
The problem is there's no room for a file based irq loader.
Other crackers better than me have tried.
if you do irq load, you have to use the T/S loader.
Hence me wanting to make the NTSC release work.
plan is as follows.
1) rip lucsfilm logo from PAL Ballblazer, RLE pack and link it with the following basic program, and call it "KORONIS RIFT"
0 SYS 16384:POKE 53265,11:REM BLANK SCREEN
1 POKE54272,0:POKE54273,0:POKE54279,0:POKE54280,0:POKE54286,0:POKE4287,0:REM SILENCE SID
2 LOAD"COPYRIGHT 1985",8
2) save off and pack main program to get it under $d000, and name it "COPYRIGHT 1985"
3) transwarp encode them.
4) save main on tracks 4,7-16. save packed logo on 19-21, stuff transwarp in free sectors elsewhere.
done. I have the two programs ready. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting Zaphodif you do irq load, you have to use the T/S loader. Why? Should be perfectly possible to replace that loader with a modern IRQ loader that loads plain files via directory - and even can decompress on the fly and come out both faster and smaller in the end. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
AS i said, very little room. the main program takes over damn near every bit of memory after running, with barely enough room for it's own T/S irq loader. it's only from $800 to $d370 or so BEFORE it's started.
Any installed IRQ loader will have to fit in the space the old one was in, because there isn't much more.
honestly would probably be easier to NTSC fix the PAL version to correct the music speed, and drop an IRQ loader into it, because it already uses files, but that's above my pay grade. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting ZaphodAny installed IRQ loader will have to fit in the space the old one was in, because there isn't much more. Why, of course. =)
A quick peek in the code revealed that enough space should be available when replacing the IRQ loader/saver (which seems to be a variant of the G.I. Joe loader, and is somewhat scattered in memory).
I'd go for the original NTSC release, as the non-IRQ loading European release might have too many changes that may prohibit re-adding IRQ-loading. (Haven't compared the two versions thoroughly, but some parts of the code seem to have been moved around, i.e., the game might have been rebuilt from a modified source for the latter release). |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
shows what i get for believing Remember about anything. :) They took one look at it and gave up. "there's no room for a better loader."
After all they were wrong about their claimed bugs in Chiller. I have collected both impossible crosses. only one of them even required using another one as a platform to reach.
I really don't know how to do much more than crack and train somewhat.
i suppose you have a memory map and suggestion on which loader to drop in that has a saver too? as far as I know joe loader only loads, though it would probably work. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Another IRQ loader would mainly be "better" in that it would load and save plain files via directory, which is preferable for a proper crack. Slightly increased speed would be a plus (the files to load aren't very big, though).
Currently, i don't know of any combined IRQ loader+saver that would be small enough after install, then load and save files via directory (should be perfectly possible to make one, though).
(What does the game need a saver for, actually? Savegames or hi-scores?)
For your purposes, replacing the IRQ loader would/saver probably not be required. Replacing "intro" and main program with somewhat modified Transwarp-encoded versions would suffice, as you have suggested. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
It's for saving progress. Score is whatever you've dismantled, plus 100k for destroying the level 20 base. End trip dismantles everything for a cash in, but doesn't get you the 100k.
I'd make the release if i had the needed tool. as i said, i have a 27 block lucasfilm logo called KORONIS RIFT that when ran shows the logo, then kernal loads COPYRIGHT 1985, which is the slightly compressed main program at 193 blocks.
If I had a better tool, I can probably make a logo intro that decompresses faster. this one takes between 1 and 2 seconds.
I'm also not entirely sure the save is an IRQ saver. which may help some. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
No worries, the encoder will be released soon. I just want to add a few more games with separate intro and main files.
I'd try not to pack the files at all, as decrunching quickly eats up the fastload speed gain. Only pack to get files to below $d000, and then only partial packing (just an upper part of the program), with an RLE packer (G-Packer V1.0 came in handy).
IRQ saver or not doesn't make much of a difference. Fitting the drive-side saving part alone is most of the problem. =) |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
okay i'm trying g-packer, and dunno what i'm doign wrong. i have main, which is 206 blocks before the packing (going under i/o). it is properly saved off, because i can monitor load it, put in original disk, g 8f0 and it works.
true drive is on, traps are off, and g-packer hangs up no it, and is unable to save a thing. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11089 |
Are you using it in PAL mode? |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
no because the game i'm tryin to pack is NTSC.
sigh. that would explain it i guess.
how DO i tell g-packer to only pack enough to get it under 201 blocks? |
| |
Comos
Registered: May 2004
Posts: 71 |
Quoting Zaphod
how DO i tell g-packer to only pack enough to get it under 201 blocks?
You can't.If you want to squeeze it further, then pack it 1st with some char packer/linker like Lightimizer/Visiomizer or try to use TLR's CLD from cbmtools. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting ComosQuoting Zaphodhow DO i tell g-packer to only pack enough to get it under 201 blocks? You can't.If you want to squeeze it further, then pack it 1st with some char packer/linker like Lightimizer/Visiomizer or try to use TLR's CLD from cbmtools. Re-read the question. You're answering another one that wasn't asked. =)
But anyways, partial packing is manual work. This involves splitting up the unpacked file to a portion that remains unpacked, and another portion at the end of the file to pack.
The packed file will be composed of something like $0801:[BASIC line + depack code][unpacked portion][packed portion]:$d000, where the unpacked portion will remain untouched during decompression, and the packed portion will expand to beyond $d000.
That the unpacked file starts at $08f0 helps a lot here, as the depack code will fit trivially before it.
Bonus if you find a large empty block somewhere below $d000 (bitmap buffer or similar, which will we initialised later anyways). Then you can nicely pack data at $d000+ and put the compressed chunk to that empty block. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
well lemme try to think of example games for that list.
1) rescue on fractalus v2 ntsc. should be a relatively straightforward vorpal to transwarp conversion, and it only uses one directory sector to store high scores (18:2). v1 irq loads instead. actually a major relink of the full remember release of this game would be pretty nice. have it move everything but the main and the trainer screen to the second stage load would work well.
2) staff of karnath https://www.youtube.com/watch?v=5ssSZuNxeXg it would be improved by the blanking the screen for the load. :) It counts because it pauses to show a screen and play music.
3) Scarabaeus. https://www.youtube.com/watch?v=VBIzd2UMTgg has an actual intro that play once, then non irq loads the main game. would be a good example.
4) many later activision games would be a good bet for this. they all show the activision animated logo sequence. Master of the Lamps, Web Dimension, etc. all the later single loads do this. but it's kinda cheating, because the intro isn't game specific. so maybe do one to prove it possible?
5) booga-boo. This has a notable intro before loading the main game. the lack of protection on the existing disk release should make this particularly easy. |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
sigh, got the encoder, but seems it will not write transwarp files to anything but consecutive empty tracks.
without the ability to write anywhere else, there's not enough space free on the disk to write both files.
I am, i kid you not, ONE lousy block too big after g-packer. |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1370 |
Should have a version of TinyCrunch in a few days that packs just enough to get below 201 blocks, leaving as much data in-place as possible. Already works, just need to do a bit more testing and clean up the code a little. |
| |
Krill
Registered: Apr 2002
Posts: 2821 |
Quoting ChristopherJamShould have a version of TinyCrunch in a few days that packs just enough to get below 201 blocks, leaving as much data in-place as possible. Already works, just need to do a bit more testing and clean up the code a little. Would be cool to define an arbitrary limit, to get smaller programs to just enough Transwarp tracks. =) |
| |
Zaphod
Registered: Jun 2012
Posts: 63 |
Quote: Should have a version of TinyCrunch in a few days that packs just enough to get below 201 blocks, leaving as much data in-place as possible. Already works, just need to do a bit more testing and clean up the code a little.
any chance of a version for us poor windows users who don't have python? :) |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1370 |
Zaphod - you could just.. install Python?
There's an installer off the "Downloads" tab at https://www.python.org/ |
| |
map
Registered: Feb 2002
Posts: 27 |
Quoting Zaphodany chance of a version for us poor windows users who don't have python? :)
There are tools like PyInstaller to create .exe existing:
https://www.blog.pythonlibrary.org/2021/05/27/pyinstaller-how-t..
Worked fine for me to create tinycrunch.exe. |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1370 |
Ooh, feel like adding that to the downloads on the csdb entry? My windows-fu is minimal at best. |
| |
map
Registered: Feb 2002
Posts: 27 |
Quoting ChristopherJamOoh, feel like adding that to the downloads on the csdb entry? My windows-fu is minimal at best.
done |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1370 |
Thank you! |
