| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Blog: All Border Double DYPP
So.. a few people had asked me about how I created certain effects.. I love blogging about code (I do this for my game-dev work, for example, and have done on-and-off for 10 years)... so...
https://c64demo.com/?p=19
The website needs a LOT of work, I know .. but for now it gives me a sandbed to upload these texts to.
Please let me know if you spot any mistakes, if you think I've overlooked explaining anything important, if you think I'm stupid and missed some obvious optimisation, etc...
Cheers! |
|
| |
Burglar
Registered: Dec 2004
Posts: 1101 |
thats a very nice blogpost, Raistlin, lots of detail :) |
| |
Quiss
Registered: Nov 2016
Posts: 43 |
That was a good read! I hadn't realized this effect was so memory-constrained.
Your ASM tooling is interesting. (I wonder what a more generic, register-aware version of such a "code interleaver" would look like.)
OT: How do you like Wordpress? I still haven't found a good blogging solution. It seems the more heavy C64 bloggers (Pagetable, LFT) have just rolled their own thing. |
| |
Oswald
Registered: Apr 2002
Posts: 5094 |
interesting I didnt know about this sprite interleave thingie, only "half" sprites, which wastes much more memory. |
| |
Wile Coyote
Account closed
Registered: Mar 2004
Posts: 646 |
Interesting read. Admittedly a large % went over my head.
'This is 624px tall – so, packing 9 lines per 256 byte block, that’s 17,920 bytes of memory.. and then the same again for the 2nd frame of our DYPP, coming to 35,840 bytes. BAD. Basically, we can’t fit it.'
'But herein lies the next trick .. we now look for how many of our 27-byte rows are identical. Our C++ code handles this for us too of course.'
That part made much sense. |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
@Oswald: what is the “half sprite” technique? I never heard of this one.: |
| |
Oswald
Registered: Apr 2002
Posts: 5094 |
Quote: @Oswald: what is the “half sprite” technique? I never heard of this one.:
lets say you have 2 sprites, lets split each in half,
sprite1:
a
b
sprite2:
c
d
A and C representing top part of the sprites, B and C the bottom.
lets make a 3rd sprite shape that looks like:
c
b
this works as an inbetween sprite shape when changing sprite pointers. so I can change to "cb" 3rd sprite shape in the bottom half of "ab" sprite, and anywhere at the top half of "cd" sprite I can change to its final "cd" shape without any visible bugs. Usagi Yojimbo is said to be using this for plexing. |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Ahh, yeah, I guess that’s the same as the 20px interleave method I wrote about for Codebase64 - but needing less tight timing than 20px .. 20px interleave could also be 19px, 18px, whatever … it just comes down to how much overlap you like - the more overlap, the less precise and fast you need to be to update the sprite indices. |
| |
Krill
Registered: Apr 2002
Posts: 2980 |
First known instance to use sprite interleaving is Maniac Mansion, i believe, with a logical sprite height of 20 lines.
Not quite sure, but i think they need to switch sprite pointers in exact raster lines (no double patterns), but do that every 20 (not 21) lines.
The logical sprites can only be positioned on every other line, though. Sprite-Y mod 2 = 0, Badline-Y mod 2 = 1 (or vice versa), neatly avoiding the badlines when switching. |
| |
Oswald
Registered: Apr 2002
Posts: 5094 |
I'm confused about this interleave thing tho. Here is how I think about it: Simply change sprite pointers every 20 lines, and arrange sprite gfx so, that it looks fine on the screen, right ?
whats the difference when we do this every 16 lines ? and what is a logical sprite's position here ?! :D |
| |
Krill
Registered: Apr 2002
Posts: 2980 |
Quoting OswaldSimply change sprite pointers every 20 lines, and arrange sprite gfx so, that it looks fine on the screen, right ? Yes, and make sure that badlines never appear in the lines where you change sprite pointers.
Quoting Oswaldwhats the difference when we do this every 16 lines ? More interrupts with no additional advantage.
Quoting Oswaldand what is a logical sprite's position here ?! :D The raster line in which changed sprite pointers become effective. |
| |
Oswald
Registered: Apr 2002
Posts: 5094 |
thanks ! :-) |
| |
Copyfault
Registered: Dec 2001
Posts: 478 |
Splendid idea to write a whole blog about the routine! Only had a quick glance up to now, but this already made me happy!
Thank you, Raistlin, for sharing all the tricks involved <3 |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Thanks all! I’ll be doing another blog very soon… Very enjoyable to write, actually. |
| |
Youth
Registered: Aug 2003
Posts: 43 |
I love reading stuff like this. Even though it is still too much over my head to tie all this together into an actual working demo effect, it is very inspiring to see these creative solutions to the limitations of the c64. Also cool to see that these top notch coders are not some sort of magicians but theres actually some decent hard thinking and problem solving behind the magic.
Does anyone know more of these sort of blog posts? |
| |
Oswald
Registered: Apr 2002
Posts: 5094 |
Quote: I love reading stuff like this. Even though it is still too much over my head to tie all this together into an actual working demo effect, it is very inspiring to see these creative solutions to the limitations of the c64. Also cool to see that these top notch coders are not some sort of magicians but theres actually some decent hard thinking and problem solving behind the magic.
Does anyone know more of these sort of blog posts?
I recommend articles by bitbreaker from here, but you might find other gems too: https://codebase64.org/doku.php?id=base:demo_programming |
| |
Digger
Registered: Mar 2005
Posts: 437 |
Great blog post Raistlin!
Do I understand it right, for sprite interleave the sprites have to overlap (with 1 pixel line)?
So it's not possible to make it work with 8 sprites in a row? |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
No overlap, no, the sprites are stacked as usual with (at least) 21px spacing.
With no interleave, each sprite of course would be drawing all 21 lines, 0 to 20.
With 20px interleave, you have 1 line (the 20th) where you’re not quite sure whether, after you update the sprite indices, row 0 or row 1 will be being drawn to the screen - so you make sure that line 20 on both rows is identical.
On the next row, at rasterline 40, it’s the 19th line that needs to be identical on rows 1 and 2 - and then the rest of row 2 needs to be slid up such that line 20 on row 2 is the new start point (what would’ve been line 0).
And so on.. each row sliding 1px further.
Sorry, my description sucks… the Codebase64 article includes some diagrams and formula to help with visualising this …. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11386 |
here is something to screw your mind more :) |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Quote: here is something to screw your mind more :)
Hah, nice!
But, yeah, the more that sprite interleave is explained, the more tricky it sounds… the best thing that you can do is just try it out, see for yourself what happens. |
| |
DanPhillips
Registered: Jan 2003
Posts: 39 |
Enigma Force (1986) is the 1st game I saw using the sprite interleaving trick.
Cheers
Dan |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Interesting .. I never played Enigma Force - but remember Shadowfire that came before it. It used sprite interleave?? Just looking at screenshots, i wonted how and why - I will check it out sometime! |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
Hey, thanks for this, it was an interesting read. I've been meaning to write a cycle-interleaving code generator myself, but thus far have just been fitting code fragments in around memory requirements rather than cycle times.
Does your code generator have to deal with any issues around write cycles being absorbed by the first few sprite DMA cycles? |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Yeah, it deals with that - but in a hard-coded way normally. I’ve been slowly adapting the code over time … I’m also not great at C++ despite having a career in it - so my code is very messy… not something that I think I could share - and not worth sharing, to be honest. |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
I did another blog post, by the way.. this time on DXYCP… I don’t want to spam CSDb every time I write a blog - so I’ll post here: https://c64demo.com/?p=156 |
| |
Mixer
Registered: Apr 2008
Posts: 452 |
Great write ups! Looking forward to the next one. |
| |
Burglar
Registered: Dec 2004
Posts: 1101 |
about this "Raistlin Code Generator", I'm assuming this generates unrolled code, not a codegen on c64-side, right?
isn't that expensive in terms of loading/decrunching time? |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Quote: about this "Raistlin Code Generator", I'm assuming this generates unrolled code, not a codegen on c64-side, right?
isn't that expensive in terms of loading/decrunching time?
Yep, it generates full unrolled code. With modern IRQ loaders, like Sparkle and Spindle, it doesn’t noticeably slow things - just load the code in the background of a simpler part if that’s necessary. We do that throughout our demos and that’s part of the art nowadays. |
| |
Krill
Registered: Apr 2002
Posts: 2980 |
Now that explains those three-disk-sided demos. =) |
| |
chatGPZ
Registered: Dec 2001
Posts: 11386 |
Damn i did a lot of "convert unrolled shite to codegens" for previous demos - that was wrong? |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Quote: Now that explains those three-disk-sided demos. =)
Hah, I can’t argue with that. It’s a very different way of making demos to the single-file amazeballs demos, sure.. but hopefully there’s room for both in the scene ;-) |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Quote: Damn i did a lot of "convert unrolled shite to codegens" for previous demos - that was wrong?
Not at all, no. Different style of demo though.
I’m sure my code could be packed down.. but it’s almost never “just” unrolled code… the blit code is variant.
I could spend all my time packing the effects down and get a 3-disk side demo down to 2 sides, possibly, but not definitely… but then you’d probably only see 1 demo from me every 10 years. In fact, you’d probably only see 1 demo full-stop - because packing these demos down isn’t where my interest lies. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11386 |
Quote:the blit code is variant.
sure. it's almost never so bad that a codegen cant be knocked up relatively easy though :) Reminds me i have to try sweet16 for this. |
| |
Krill
Registered: Apr 2002
Posts: 2980 |
Quoting Groepazit's almost never so bad that a codegen cant be knocked up relatively easy though :) Back when coding +H2K i had codegens for all effects.
One effect (the fish-eye lens going across a bitmap) ended up with precompiled unrolled code on the disk, though, and that was because generating the highly-optimised code during run-time took a lot (like 2-3 times) longer than simply loaddecrunching it from disk. :) |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Quote: Quote:the blit code is variant.
sure. it's almost never so bad that a codegen cant be knocked up relatively easy though :) Reminds me i have to try sweet16 for this.
I can PM you some of the generated ASMs if you want to give it a go… more than happy to do that.
I wrote codegens back in the 80s - Delirious 10 would’ve used them for example.. but I was told early on when I returned to the scene that that’s not how most trackmos are done nowadays… and, strange as it felt, I like it this way. Sparkle loads the data so fast, why stress? Having the demo be slick and fluid is more important. Even with codegen, you’re nothing like guaranteed to reduce the #disks needed anyway.. |
| |
Burglar
Registered: Dec 2004
Posts: 1101 |
Quoting RaistlinWe do that throughout our demos and that’s part of the art nowadays. That's very interesting to hear, must feel liberating :)
Quoting RaistlinHaving the demo be slick and fluid is more important. I'd say that's most important :) |
| |
Krill
Registered: Apr 2002
Posts: 2980 |
Quoting RaistlinHaving the demo be slick and fluid is more important. Quoting BurglarI'd say that's most important :) Of course it is, but this isn't mutually exclusive with having run-time code generators. :)
(What does turning the disk a second time do to the flow?) |
| |
Oswald
Registered: Apr 2002
Posts: 5094 |
idk, to me it looks pretty lame that most 'mega'demos today need to turn disk after the intro and 1-2 part. |
| |
Bitbreaker
Registered: Oct 2002
Posts: 508 |
Quote: Yep, it generates full unrolled code. With modern IRQ loaders, like Sparkle and Spindle, it doesn’t noticeably slow things - just load the code in the background of a simpler part if that’s necessary. We do that throughout our demos and that’s part of the art nowadays.
I'm not sure if this is really part of the art, at least it can't be common sense, as i have a different view on this :-D What happens is, that disks then get extended to more disksides or even 40 tracks. Making parts small gives also more freedom in preloading. Also i am no fan of those short offset packers that waste a lot of ratio. In fact loading less and depacking slower does just take the same amount of time, but less diskspace. Not always, but often i do codegenerators when it is about huge structures or unrolled code. But i start with macros and loops first to make the code more readable. Then when you notice that you run out of diskspace and can avoid another diskside, it sure is the first thing i am going to do. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11386 |
Quote:I can PM you some of the generated ASMs if you want to give it a go
doing this for someone elses code is certainly not on my todo list :=) |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
The fact is, and I can tell you this with some certainty, C64 code generators won’t work on 90% of my parts because there are several layers of optimisations that simply couldn’t be done in 6510 in a decent timeframe.
It might sound like an excuse - I guarantee that it’s not. The best that I can imagine could be done is to tokenise some sections - but that’s certainly not going to reduce things enough to save a disk side. And without that, what’s the point? Who cares if a trackmo is 2.5 disk sides instead of 3? |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Quote: Quote:I can PM you some of the generated ASMs if you want to give it a go
doing this for someone elses code is certainly not on my todo list :=)
I simply offered to send you the AsM .. figured that it’s a good way to show you why a 6510 codegen wouldn’t make a huge difference… you wouldn’t be saving much on the disksize of the part. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11386 |
Hence i said "almost". Sure there are cases like this. But very often this is not the case. |
| |
Oswald
Registered: Apr 2002
Posts: 5094 |
Quote: The fact is, and I can tell you this with some certainty, C64 code generators won’t work on 90% of my parts because there are several layers of optimisations that simply couldn’t be done in 6510 in a decent timeframe.
It might sound like an excuse - I guarantee that it’s not. The best that I can imagine could be done is to tokenise some sections - but that’s certainly not going to reduce things enough to save a disk side. And without that, what’s the point? Who cares if a trackmo is 2.5 disk sides instead of 3?
I'm pretty sure interleaving timed raster code and a BOB plotter is quite straightforward, all you need is introduce a variable which keeps track of nr of cycles. then when treshhold is reached pad with nop's and write out the raster code, then goto 10. |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Quote: I'm pretty sure interleaving timed raster code and a BOB plotter is quite straightforward, all you need is introduce a variable which keeps track of nr of cycles. then when treshhold is reached pad with nop's and write out the raster code, then goto 10.
It’s not that simple, no.. there are many optimisations that wouldn’t be easy to implement in a native codegen.. eg.:-
- maintaining whether each dest byte has already been written to (overlaps)
- whether ORA can be used for adjacent chars to save cycles
- whether the raster code is trashing A, X or Y and whether you currently need these for BOB plotting
- my codegen allows, under certain circumstances, for the BOB-plotter to leave a single cycle unused - but this needs to be cleverly wasted in the D016 code
- etc
By all means you could probably code -a- DXYCP or all border double DYPP using native codegen - but I doubt they would be as optimal. Prove me wrong or stop arguing, I say - the challenge is there. |
| |
Burglar
Registered: Dec 2004
Posts: 1101 |
Quoting KrillOf course it is, but this isn't mutually exclusive with having run-time code generators. :) oh definitely, if I look from my own perspective, I write codegens for code, tables and even gfx often, but I only do it when it's straightforward.
I kinda understand Raistlin's point: the options are "spend a lot more time" or "just add another d64 in no time at all". both choices are valid :) |
| |
Bitbreaker
Registered: Oct 2002
Posts: 508 |
There's situations where writing a codegen is a pain in the arse for sure. I for myself just wanted to point out to not generalize things, there is very often a good point to use a codegen. This also allows to be lazy on the ugly parts :-) |
| |
Sparta
Registered: Feb 2017
Posts: 49 |
I really hate to get involved in such a useless debate so I will limit my involvement to this single comment here.
Really, I just wanted to thank you all for appreciating Raistlin's effects (and thus, our demos) and the effort he is putting in his blog. This is clearly shown by the fact that the only argument against them here is using more disk sides than what you'd prefer. Thank you again!
Re: loaders. Bitbreaker, your loader is truly fantastic! As is Krill's loader and Spindle, of course. I could list numerous arguments for using any of the trackmo loaders here, but really, this is personal preference (and somewhat off-topic).
I love you guys all!
Cheers,
Sparta |
| |
Oswald
Registered: Apr 2002
Posts: 5094 |
Quote: It’s not that simple, no.. there are many optimisations that wouldn’t be easy to implement in a native codegen.. eg.:-
- maintaining whether each dest byte has already been written to (overlaps)
- whether ORA can be used for adjacent chars to save cycles
- whether the raster code is trashing A, X or Y and whether you currently need these for BOB plotting
- my codegen allows, under certain circumstances, for the BOB-plotter to leave a single cycle unused - but this needs to be cleverly wasted in the D016 code
- etc
By all means you could probably code -a- DXYCP or all border double DYPP using native codegen - but I doubt they would be as optimal. Prove me wrong or stop arguing, I say - the challenge is there.
since 6510 is turing complete, optimal codegen can be done, thats not a question. anyway at this complexity is rather pointless to make it natively you are right. |
| |
Krill
Registered: Apr 2002
Posts: 2980 |
Quoting Oswaldanyway at this complexity is rather pointless to make it natively you are right. You can still apply a simple filter (separate opcode and operand streams) for better crunchiness of generated code, though. =) |
| |
Perplex
Registered: Feb 2009
Posts: 255 |
Quoting Oswaldsince 6510 is turing complete, optimal codegen can be done, thats not a question.
The memory limitation for keeping track of state when generating optimal code is quite a bit lower when doing it at runtime, though. |
| |
ChristopherJam
Registered: Aug 2004
Posts: 1409 |
Perplex, that's what swap disks are for!
runs |
| |
Danzig
Registered: Jun 2002
Posts: 440 |
Quote: Perplex, that's what swap disks are for!
runs
You could just run the code-generator in the disk drive, best of both worlds ;-) no need to swap floppy disks. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11386 |
Yeah, you have 3 Disks, use them! |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
@Groepaz.. you guys convinced me. As soon as VICE is a 141k install, and uses codegen to create the rest of the emulator, I'll recode my parts on Memento Mori to squeeze it onto 2 disk sides.
141mb is just a little bit too large for a C64 emulator :-p |
| |
chatGPZ
Registered: Dec 2001
Posts: 11386 |
just use Frodo! |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Quote: just use Frodo!
Trololol :-) |
| |
Copyfault
Registered: Dec 2001
Posts: 478 |
Funny to read through this and find yourself pondering wether streaming code-blocks from disc is close to cheating... where there's absolutely no reason to feel so if you ask me. If one finds the challenge attractive to mock up that increadible codegen on C64, it's great. If someone has brilliant ideas to bomb me with effects by the beforementioned code-streaming - fine!
But what do I know... one thing that was not mentioned up to now is that with more disc sides, there usually *are* more turn-disc-parts - and having some of the recent gems in mind, I'd really miss some of them if they hadn't been done!
So, a plus for the disc-sides-to-the-max-approach :) |
| |
Krill
Registered: Apr 2002
Posts: 2980 |
Quoting CopyfaultFunny to read through this and find yourself pondering wether streaming code-blocks from disc is close to cheating... where there's absolutely no reason to feel so if you ask me. Well yes, still you're the first to suggest it could be cheating. =) |
| |
Copyfault
Registered: Dec 2001
Posts: 478 |
Quoting KrillQuoting CopyfaultFunny to read through this and find yourself pondering wether streaming code-blocks from disc is close to cheating... where there's absolutely no reason to feel so if you ask me. Well yes, still you're the first to suggest it could be cheating. =) That's correct - but believe me, I do not at all think it is! Reading the thread just made me feel so... |
| |
Raistlin
Registered: Mar 2007
Posts: 680 |
Sorry… I did another blog. https://c64demo.com/?p=196
@Krill: thanks for your suggestion on splitting the ASM (nuclear compression?). We’re trying it and it -might- work :-) |
| |
Krill
Registered: Apr 2002
Posts: 2980 |
Quoting Raistlin@Krill: thanks for your suggestion on splitting the ASM (nuclear compression?). We’re trying it and it -might- work :-) Worked for me. :) IIRC, saved about 100 bytes in Artefacts even after the static overhead for recombining the two streams after decrunching. So of course, quicker amortisation and more payoff for bigger code than for just a 4K.
There are some rather funky new rules for optimisation, though, like favouring LDA #0 : LDY #0 over LDA #0 : TAY, but then of course this isn't much of an issue for your use-case where the decrunched unrolled code has more important constraints to satisfy. |
