[CSDb] - User Forums - Linking wizardry

Welcome to our latest new user maak ! (Registered 2024-04-18)

You are not logged in - nap

CSDb User Forums

Forums > C64 Coding > Linking wizardry

2014-10-31 14:41

Bitbreaker

Registered: Oct 2002
Posts: 499

Linking wizardry

You wanted to have some linking wizardry, so here's how things were done in Comaland:

The central part:
-----------------

In $demofolder/link there's a Makefile. The example shows the process for doing one side. More imageX.d64 can of course be generated to build more sides, but i shortened things here to give at least a minimal chance to keep the overview. Some additional shellscript with sed wizardry was used to autogenerate a .bat for the windows whiners :-P
Targets like image1.d64 or vice1 can be used to build and fire up only a single side for standalone testing. Still, Alt-W is your best friend ;-)

export PATH := tools:../../link/tools/:$(PATH)
SHELL = /bin/bash
BITNAX = bitnax
D64WRITE = d64write
X64 = x64
MAKE_FLAGS = RELEASE=1

.PHONY: all toolchain vice vice1

all: toolchain image1.d64

toolchain:
        $(MAKE) -C ../../../tool/c6510
        $(MAKE) -C ../../../tool/acme/src
        $(MAKE) -C ../../../tool/dasm
        $(MAKE) -C ../../../tool/bitfire
        $(MAKE) -C ../../../tool/bitnax
        $(MAKE) -C ../../../tool/dreamass

        cp ../../../tool/c6510/c6510 tools/
        cp ../../../tool/acme/src/acme tools/
        cp ../../../tool/dasm/dasm tools/
        cp ../../../tool/dreamass/dreamass tools/
        cp ../../../tool/bitfire/d64write tools/
        cp ../../../tool/bitnax/bitnax tools/

vice: all
        $(X64) -pal -autostart "image1.d64:*" -truedrive -model c64c

vice1: toolchain image1.d64
        $(X64) -pal -autostart "image1.d64:*" -truedrive -model c64c

###############################################################

image1.d64: comaland.prg bootside1.prg tune1.prg rasterrot.prg explode.prg bitmap.prg bitmap_fadeout2.prg shadowscroll.prg shadow_fadeout.prg plotballs_fadein.prg plotballs.prg plotballs_fadeout.prg comalanddef.prg  escos.prg note.prg
        $(D64WRITE) -c $@ --side 1 -a 49 ../bitbreaker/dirart/side1.prg \
                                --boot comaland.prg \
                                -b bootside1.prg \
                                -b rasterrot.prg \
                                -b tune1.prg \
                                -b explode.prg \
                                -b bitmap1.prg \
                                -b bitmap2.prg \
                                -b bitmap3.prg \
                                -b bitmap4.prg \
                                -b bitmap5.prg \
                                -b bitmap_fadeout2.prg \
                                -b comalanddef1.prg \
                                -b comalanddef2.prg \
                                -b plotballs_fadein.prg \
                                -b plotballs.prg \
                                -b plotballs_fadeout.prg \
                                -b shadowscroll1.prg \
                                -b shadowscroll2.prg \
                                -b shadowscroll3.prg \
                                -b shadow_fadeout.prg \
                                -b escos1.prg \
                                -b escos2.prg \
                                -b escos3.prg \
                                -s "erotic note"

##################### SIDE 1 ##################################
note.prg: force_look
        cd ../Axis/note/; $(MAKE) $(MAKE_FLAGS)
        $(BITNAX) --bitfire --sfx 0x4000 -o "erotic note" ../Axis/note/note

comaland.prg: force_look
        cd ../bootloader/stage1/; $(MAKE) $(MAKE_FLAGS) link_exit=0x0100 SIDE=1
        $(BITNAX) --sfx 0x0900 -o $@ ../bootloader/stage1/stage1

bootside1.prg: force_look
        cd ../bootloader/stage2/; $(MAKE) $(MAKE_FLAGS) link_exit=0x2000 SIDE=1
        $(BITNAX) --bitfire -o $@ ../bootloader/stage2/stage2

tune1.prg: ../../Music/dEViLOCk/tune_side10x900.prg
        $(BITNAX) --bitfire -o $@ --load-addr 0xe000 $^

rasterrot.prg: force_look
        cd ../Axis/RasterRotate/; $(MAKE) $(MAKE_FLAGS) link_exit=8192
        $(BITNAX) --bitfire -o $@ ../Axis/RasterRotate/rasterrot

explode.prg: force_look
        cd ../Mirage/FontExplodeMultiplex; $(MAKE) $(MAKE_FLAGS) link_exit=0x300c
        $(BITNAX) --bitfire -o $@ ../Mirage/FontExplodeMultiplex/explode

bitmap.prg: force_look
        cd ../Lavazza/bitmapscroller/; $(MAKE) $(MAKE_FLAGS) link_exit=0x0400
        $(BITNAX) --bitfire -o $(basename $@)1.prg --cut-input-addr 0x3000 0xffff ../Lavazza/bitmapscroller/bitmap
        $(BITNAX) --bitfire -o $(basename $@)2.prg ../Lavazza/bitmapscroller/part2.prg
        $(BITNAX) --bitfire -o $(basename $@)3.prg ../Lavazza/bitmapscroller/part3.prg
        $(BITNAX) --bitfire -o $(basename $@)4.prg ../Lavazza/bitmapscroller/part4.prg
        $(BITNAX) --bitfire -o $(basename $@)5.prg ../Lavazza/bitmapscroller/part5.prg

bitmap_fadeout2.prg: force_look
        cd ../CRT/bitmap_fadeout2/; $(MAKE) $(MAKE_FLAGS) link_exit=0x6400
        $(BITNAX) --bitfire -o $@ ../CRT/bitmap_fadeout2/bitmap_fadeout2

comalanddef.prg: force_look
        cd ../Bob/bob_yazoo/; $(MAKE) $(MAKE_FLAGS) link_exit=0xf000;
        $(BITNAX) --bitfire -o $(basename $@)1.prg --cut-input-addr 0xbff8 0xffff --load-addr 0xe400 ../Bob/bob_yazoo/comalanddef
        $(BITNAX) --bitfire -o $(basename $@)2.prg --cut-input-addr 0x2000 0xbff8 ../Bob/bob_yazoo/comalanddef

plotballs_fadein.prg: force_look
        cd ../bitbreaker/plotball_fadein/; $(MAKE) $(MAKE_FLAGS) link_exit=0x4000;
        $(BITNAX) --bitfire -o $@ --load-addr 0x9b00 ../bitbreaker/plotball_fadein/plotball_fadein

plotballs.prg: force_look
        cd ../bitbreaker/plotballs/; $(MAKE) $(MAKE_FLAGS) link_exit=0x0400;
        $(BITNAX) --bitfire -o $@ ../bitbreaker/plotballs/plotballs

plotballs_fadeout.prg: force_look
        cd ../bitbreaker/plotball_fadeout/; $(MAKE) $(MAKE_FLAGS) link_exit=0x2000;
        $(BITNAX) --bitfire -o $@ ../bitbreaker/plotball_fadeout/plotball_fadeout

shadowscroll.prg: force_look
        cd ../CRT/shadowscroll; $(MAKE) $(MAKE_FLAGS) link_exit=0x0400
        $(BITNAX) --bitfire -o $(basename $@)1.prg --cut-input-addr 0xcff0 0xffff ../CRT/shadowscroll/shadowscroll
        $(BITNAX) --bitfire -o $(basename $@)2.prg --cut-input-addr 0x2000 0xbff8 ../CRT/shadowscroll/shadowscroll
        $(BITNAX) --bitfire -o $(basename $@)3.prg --cut-input-addr 0xbff8 0xcff0 ../CRT/shadowscroll/shadowscroll

shadow_fadeout.prg: force_look
        cd ../CRT/shadow_fadeout/; $(MAKE) $(MAKE_FLAGS) link_exit=0x2000
        $(BITNAX) --bitfire -o $@ ../CRT/shadow_fadeout/shadow_fadeout

escos.prg: force_look
        cd ../Swallow/escos_scroll; $(MAKE) $(MAKE_FLAGS) link_exit=0x0100
        $(BITNAX) --bitfire -o $(basename $@)1.prg --cut-input-addr 0xbff0 0xffff --load-addr 0xe400 ../Swallow/escos_scroll/escos
        $(BITNAX) --bitfire -o $(basename $@)2.prg --cut-input-addr 0x2000 0xbff0 ../Swallow/escos_scroll/escos
        cp ../Swallow/escos_scroll/low.prg $(basename $@)3.prg

###############################################################

clean:
        -rm *.prg *.d64
        -rm ../Axis/note/note
        -rm ../bootloader/stage1/stage1
        -rm ../bootloader/stage2/stage2
        #side1
        -rm ../Axis/RasterRotate/rasterrot
        -rm ../Lavazza/bitmapscroller/bitmap
        -rm ../CRT/bitmap_fadeout2/bitmap_fadeout2
        -rm ../CRT/shadowscroll/shadowscroll
        -rm ../CRT/shadow_fadeout/shadow_fadeout
        -rm ../bitbreaker/plotball_fadein/plotball_fadein
        -rm ../bitbreaker/plotballs/plotballs
        -rm ../bitbreaker/plotball_fadeout/plotball_fadeout
        -rm ../Mirage/FontExplodeMultiplex/explode
        -rm ../Swallow/escos_scroll/escos
        -rm ../Bob/bob_yazoo/comalanddef
        -rm "erotic note"

        #$(MAKE) -C ../../../tool/c6510 clean
        #$(MAKE) -C ../../../tool/acme/src clean
        #$(MAKE) -C ../../../tool/dasm clean
        #$(MAKE) -C ../../../tool/bitfire clean
        #$(MAKE) -C ../../../tool/bitnax clean
        #$(MAKE) -C ../../../tool/dreamass clean

force_look:
        @true

This is the main file to build the demo, there are blocks per side to be built. The part that writes all parts on disk builds up the dependencies to all the other targets.
Each part is build in a saparate subtarget in a separate dir with separate Makefile (tiny and can be copied over and adopted easily) also each part contains the entry point for the next upcoming part (link_exit). The compiled parts are split up and packed with bitnax. The chunks are choosen wisely so that they fit in during loading and not destroy/overwrite any code/data that is still in use. That is the nifty part about linking, stuffing as much as possible data into mem while still showing cool things on the screen. Also parts should split up wisely in size, so that they won't reach or go over the end of mem. If needed, the chunks can also be loaded to different locations by changing their load-address on packing.

The bootloader:
---------------

There's some code fragments for a stage1 and stage2 for the bootloader.

$demofolder/bootloader/stage1/main.asm
- contains code for installing the loader and code for loading stage 2
- contains persistent code that gives exit points for parts to be able to load things without own code (as it might be overwritten)
- contains music call and base irq that can take over the music while no part running. Also it keeps counting up the frame counter to be able to sync things.
- thus each side gets an installer/bootloader for safety reasons for free as well

$demofolder/bootloader/stage2/boot.asm
best placed @ $0100, but set the stackpointer to $ff before doing so.
The side specific stuff is happening here, like loading the music and starting it at the same frame for every try to keep teh demo in sync. Here it is also a good idea to set the music calls to different addresses if the music locations differes from side to side.

Synching:
---------

A sync point needs to be installed everywhere, where loading is not covered by a effect, as it is the only part that is not deterministic (still, longer depack times and such can also shift the sync). Also take care that the sync point waits a reasonable amount of frames longer than loading takes, to respect slower loading floppys. To find out the minimum time, you can freeze the machine after loading with a !byte $02 and read out the frame counter. It is usually a good idea to give about one second extra time.
Also, the demo is not synched to the music, but the music synched to the demo in the final step, or an recurring process. Why not offload some of the pain to the musicians :-P

The Parts:
----------

Each part has a own folder, like:
$demofolder/$codername/$partname

Each of those folders also contains a Makefile like:

ACME = acme
ACMEOPT = -v1 -f cbm

.PHONY: clean

all: bumble

bumble: afli.asm bumble_pic.prg
ifdef RELEASE
        ${ACME} ${ACMEOPT} -DRELEASE=1 -Dlink_exit=$(link_exit) -o $@ $<
else
        ${ACME} ${ACMEOPT} -o $@.prg $<
        ../../../../tool/bitnax/bitnax --sfx 0x9000 -o $@.prg $@.prg
endif

clean:
        -rm bumble bumble.prg

As you see, the target is build in a different way by using the RELEASE define. Also the parts can then make conditional assembly and either take precautions for a release or standalone version.
All this only works if the assembler accepts external defines. So far c6510, acme, dreamass and dasm do, but some only accept decimal values for that, liek dreamass, so take care on the link_exit define

A part loading another might then look like the following:

!ifdef RELEASE {
                ;reset frame counter
                lda #$00
                sta link_frame_count+0
                sta link_frame_count+1

                ;load + depack some part
                jsr link_loadnext_uncr

                ;load some part
                jsr link_loadnext_raw

!ifdef WAIT_FRAME_COUNTER {
                ;sync to frame counter
                +wait_frame_count $0200
} else {
                ;or wait for a trigger from your effect if you are sure it takes longer than loading
trig            lda #$00
                beq trig
}

                ;fadeout the effect
                jsr fadeout

                ;unlink irq to resident irq handler
                +link_player_irq

                ;depack last part loaded and start it
                +link_uncr_jmp link_exit
}

I'd add more details if that codebase64 would just be enabled for editing again :-) This is the suckless approach i guess, but of course not the most comfortable. Now throw in your bloated frameworks :-P

... 57 posts hidden. Click here to view all posts....

2014-11-01 13:50

Axis/Oxyron

Registered: Apr 2007
Posts: 91

Hehe, yes. Never thought about that wrapping problem. I like to do fast paced demos. So 21 Minutes should be enough for everyone! ;o)

2014-11-01 14:43

MagerValp

Registered: Dec 2001
Posts: 1055

Thanks for sharing JackAsser, I have a thread scheduler in my head that I've yet to find any use for, and this is great inspiration.

2014-11-01 14:52

Oswald

Registered: Apr 2002
Posts: 5017

I do that music stuff as gpz, except the music tick. suprised its new stuff to some of you :)

also always keep a music player irq resident "tempirq".

2014-11-01 17:07

Stone

Registered: Oct 2006
Posts: 168

In RGB, the "main" tune plays through all 3 disk sides. This was accomplished by splitting the tune into several parts with a small segment at the end which loops until the next part is ready or the disk is turned. The music player signals when the loop point has been reached and the next tune part can start seamlessly.

Regarding linking: I built my parts with 2 entry points in the same PRG. One cold start (can be run from basic) and a warm start which is called from the demo system. Saves me the trouble of building two versions with different build parameters. Pantaloon doesn't really approve of this though, since it fills up the disk :P

Interesting to read about JackAsser's threading/coroutine system. I'm using coroutines myself (with macros) and I originally got the idea when we were porting the NBA Hangtime arcade game to SNES. There, each object/actor had its own private stack (with local variables). My system (first used in Save Truckstop Alaska/We Are All Connected) is more limited, with just A/X/Y/PC saved. No nested subroutines or stack shenanigans allowed. It uses macros for things like yielding and sleeping and plain rts at the end of a "task".

2014-11-01 21:36

Radiant

Registered: Sep 2004
Posts: 639

Wow, lots of nice reading! I don't have much to add, other than ld65 can be a very powerful tool when linking. In our framework I've added support for splitting up the linker config files, so you can have definitions that are common for all parts of the demo in one or more files, and part specific definitions in others. Most of the work around linking is actually handled by ld65 and Make (and a small, resident "chaining" routine which loads the parts in order).

We also use the linker config for declaring timing symbols (based on a global frame counter). This means you can pretty much move parts around as you want, provided they're well behaved - and provided they don't use the same memory areas...

To solve that in the standard framework setup there are two different memory configs; one using $2000-$dfff and another using $e000-$fff8 for part specific data. The idea is that the smaller parts work as intermediate loader parts while the larger ones are proper effects. The framework starts loading the next part the moment the current part returns from the main thread. Before returning the parts can use all memory between $2000 and $fff8 as they see fit, but when the next part is loading they have to stay away from that memory area.

I've thought about automatically (or at least semi-) splitting up parts in smaller chunks and loading them incrementally, but haven't written any code to do so yet. Also it would be nice to be able to build the demo to run from a specific part from a disk image without running it in standalone PRG mode, if the part for example loads custom data from disk and needs the resident loader to work.

Really like JackAsser's threading approach. *steals* :-)

2014-11-01 23:00

JackAsser

Registered: Jun 2002
Posts: 1987

Quote: Thanks for sharing JackAsser, I have a thread scheduler in my head that I've yet to find any use for, and this is great inspiration.

Any time your re-entrant state machine code in the IRQ becomes too messy you have a use for it in that your code turns from state machine type to linear flowimg code. Adjust the yield-function too your needs, f.e. pausr one frame or even yield to a particular raster line, or to the next music beat etc.

2014-11-02 09:14

doynax
Account closed

Registered: Oct 2004
Posts: 212

How do you guys keep the weird-and-wonderful dependency chains of the build manageable?

Admittedly I've not been working large collaborative demo projects (though I imagine that can only make things worse) but I always seem to end up massaging code and data back and forth in fifteen ways through tools before the build is completed. The result being that I tend spend a significant amount of time tracking down dependency bugs in the build script and forcing full rebuilds at the slightest provocation.

Surely there's got to be a better tool than make out there though, especially the recursive variety. Unfortunately my last attempt to find one mostly turned up slew of tools aimed to ease cross-platform compilation. Personally I eventually ended up kludging together a build engine driven by some Lua scripts but let's just that it's still far from being the ideal tool..

2014-11-02 09:35

Axis/Oxyron

Registered: Apr 2007
Posts: 91

A thing I always thought about, but never implemented yet, is adding a "dev" or "debug" build target to our single parts and full demo build. To check for common sources of error.

In the single parts this target can invoke code before init and after shutdown. To mark the critical zp-adresses and memory areas (music, loader, framework, ...) with a pattern. And check that pattern to not be broken after shutdown. Otherwise an error-message will be shown. Basically like C-compilers do on debug builds with malloc.

Another common problem is, that timing marks that work perfect one day to cover the whole loading time on every device dont fit anymore a few days later. this can be caused by parts growing, or even by other parts earlier on the disk that move the succeeding parts on different tracks and sectors. I can tell you, that phenomenom has bitten us uncountable times, always in the worst moments. And mostly during development (Vice or Ultimate), everything is fine. But then someone does a test on a slow floppy drive and the whole timing is fucked. The "debug" build could check if all timing marks still have enough safety-margin (e.g. 50 frames). And if not, exit the demo with an error-message.

2014-11-02 10:33

Radiant

Registered: Sep 2004
Posts: 639

Quoting doynax

Surely there's got to be a better tool than make out there though, especially the recursive variety. Unfortunately my last attempt to find one mostly turned up slew of tools aimed to ease cross-platform compilation. Personally I eventually ended up kludging together a build engine driven by some Lua scripts but let's just that it's still far from being the ideal tool..

IMO there are currently no better tools out there than Make. Tracking dependencies is a tough problem in general, and like with most tough problems there are no easy solutions. I usually end up hardcoding the dependencies, falling back on full rebuilds occasionally... But that's mostly due to stupid laziness.

2014-11-02 10:59

MagerValp

Registered: Dec 2001
Posts: 1055

Quoting Stein