|
| |
|
Krill's Loader, Repository Version 166 [2018] |
| User rating: | awaiting 8 votes (8 left) |
Credits :
Download :
Look for downloads on external sites:
Pokefinder.org
Production Info
Submitted by Krill on 1 October 2018
Krill's loader
October 2018
Krill@Plush.de
What it is
----------
This is an easy-to-use arbitrarily interruptible flexible general-purpose file-based disk loader system with optional decompression, aiming to provide maximum loading speed while retaining full standard Commodore DOS format compatibility and also having a reasonably small memory footprint.
Being fully standard DOS format compatible, disks and disk images may be written by numerous available third-party copy, transfer, and disk image manipulation programs.
The selection of byte-stream based compression algorithms includes many popular crunchers.
Natively supported drives include all variants of the classic Commodore drive families of 1541, 1570/71, 1581, as well as CMD FD drives, and 1541U. Using the KERNAL fallback option, devices like SD2IEC (slow), IDE64 (fast), and netload are supported non-natively.
Both PAL and NTSC video systems are supported.
The drive's CPU may run custom code provided by the user.
What it isn't
-------------
Simply put, anything that's not a read-only disk driver. Although any of the following features and functionality may be built on top of it, it is not:
- a DOS: it cannot save files, format disks, provide random access below file level, or otherwise organise data on a disk
- a framework of any kind: it's only active when you tell it to, it does not do anything in the background by itself, and the machine is fully yours when the loader is idle
- a memory expansion server: files are not cached in REU, GeoRAM, RAMLink, RAMDrive, or similar devices
- loading in an interrupt handler or another mainline thread: this can easily and flexibly be implemented by the user
Features
--------
- maximum raw loading speed of 7.7 kB/s (~20x) on 1541, typical speed is around 7 kB/s (~18x)
- resident size: $0280 bytes for combined load+depack using Bitnax (fits in $0180..$03ff), $01d4 bytes with tinycrunch (fits in $0200..$03d3), $f6 bytes load-raw only
- IRQ/NMI/DMA/sprites/badlines are allowed without restrictions
- 2bit+ATN protocol (72 cycles per byte), multiple drives on the bus are allowed, but loading only from the primary drive
- supported crunchers: Bitnax, Byteboozer2, Doynax-LZ, Exomizer, Levelcrush, NuCrunch, Pucrunch, Subsizer, tinycrunch
- depacking fully in-place using Bitnax or tinycrunch, no clobbered bytes beyond unpacked data
- loading via directory with 0-terminated filename strings
- loadnext functionality for sequential loading without specifying subsequent filenames
- load address override for raw and crunched files
- PAL/NTSC support, NTSC is optional
- returns status codes for error checking and multiple disk handling
- optional loading and decompression to RAM at $d000..$dfff
- optional memory decompression without loading
- optional loading via shadow directory for unrestricted dir-art
- custom drive code API facilitating co-processing on the drive's CPU
- full on-the-fly block GCR read+decode+checksumming
- native interleave at 100% CPU for the loader is 3 revolutions per track on tracks 18+, 4 on tracks 1-17
- implements Shrydar stepping for minimised seek time to an adjacent track
- free relocation of install routine, resident portion and zeropage variables
Folder structure overview
-------------------------
- loader this
- build output files
- interm intermediate output files
- docs documentation
- include include files
- src source code
- decompress decrunch routines
- drives drive-side code
- hal hardware abstraction layer implementation
- samples example projects
- minexample minimum example
- resources graphics and stuff
- standalone use loader from BASIC prompt
- test test application
- turndisk disk change handling
- tools utilities
- b2 Byteboozer2
- bitnax-07a8c67 Bitnax
- cc1541_source use cc1541 for optimised disk images
- doynamite1.1 Doynax-LZ
- exomizer-3 Exomizer
- nucrunch-1.0.1 NuCrunch
- pucrunch Pucrunch
- subsizer-0.7pre1 Subsizer
- tinycrunch_v1.2 tinycrunch
- wcrush LevelCrush
- shared misc. include files
Building the binaries
---------------------
Prebuilt binaries (install-c64.prg, loader-c64.prg and loadersymbols-c64.inc) can be found in loader/build.
To build them, prerequisites are ca65 (see https://www.cc65.org, http://cc65.github.io/cc65, or https://github.com/cc65/cc65), make, and perl.
In loader or loader/src, run
$ make prg
This will produce (and overwrite) install-c64.prg, loader-c64.prg, and loadersymbols-c64.inc in loader/build.
To specify zeropage, install and resident addresses, run, e.g.,
$ make prg ZP=e8 INSTALL=1000 RESIDENT=ce00
which will produce binaries using zeropage addresses $e8 and up, with the install routine located at $1000 and the resident portion at $ce00.
The two .prg files are regular C-64 program files with 2 bytes load address and can be linked statically using .incbin or similar features of any 6502 assembler/linker.
The symbols file is a text file defining the call addresses and zeropage variables in canonical 6502 assembler syntax and is intended to be included from assembly source code.
Configuration
-------------
The main configuration file is loader/include/config.inc. The individual settings of the options are evaluated at build time and directly control various aspects of the loader's functionality and its memory footprint. This include file can be overridden by setting the EXTCONFIGPATH environment variable to a folder containing a file named loaderconfig.inc.
Note that NTSC support is optional, and disabled in the pre-built binaries. This is because it slightly reduces PAL performance. However, resident portion incarnations with and without NTSC support can be selected at run-time in order to achieve maximum performance for both PAL and NTSC.
Refer to loader/samples for various example projects using different individual external configuration files.
Basic operation
---------------
Before the loader can operate, its drive-side code portion(s) must be installed in the drive(s). This is done by
jsr install; subsequent loader calls will only work on the active drive as denoted by FA = $ae
bcs error
; loader installed successfully
On error, the carry flag is set and the error code is returned in the accu. As the install routine calls various KERNAL routines, the KERNAL ROM must be enabled (e.g., $01 set to $36 or $37).
Note that the KERNAL routines also perform SEI and CLI, so it's advisable to set up any custom interrupt handlers or enable sprites only after installing the loader.
Furthermore, in additon to FA = $ae, some more zeropage variables must have valid values. See the bottom of loader/include/loader.inc for details. This list may not be complete.
The drive-side portion remains resident in the drive. After successful installation, the install routine is not needed any more and may be overwritten. The KERNAL ROM may be disabled and zeropage variables clobbered.
With the installed loader, files can only be loaded from the active drive, which cannot be changed, and all other drives will remain inert:
ldx #<filename
ldy #>filename
jsr loadraw; load without decompression
bcs error; accu contains error code when carry is set after loading
; unpacked file loaded successfully
ldx #<packed
ldy #>packed
jsr loadcompd; load with decompression
bcs error; accu contains error code when carry is set after loading
; packed file loaded successfully
[...]
filename: .petscii "unpacked"
.byte 0
packed: .petscii "packed"
.byte 0
Note that the filename strings are 0-terminated.
While loading using filenames with wildcards ("?" and "*") is not possible, subsequent files following the previously-loaded file can be loaded via a zero-length filename:
ldx #<nextfile
ldy #>nextfile
jsr loadraw; or loadcompd
bcs error
[...]
nextfile: .byte 0
The first file to be loaded by the loader must be loaded via its actual filename. Any subsequent files may then be loaded using the loadnext feature.
It is strongly recommended to always check the carry flag for errors and handle them.
The loader code is completely inert when the loader is idle. Thus, the resident code portion can be buffered away, overwritten, and copied back without any problems. It is also possible to have different incarnations of the resident code portion, such as with and without decompression, or with different decompression routines.
See loader/samples/minexample for a minimal loader usage example.
Refer to loader/include/loader.inc for the definition of the error codes and various ca65-syntax convenience macros illustrating usage of the various loader calls.
Setting the VIC bank
--------------------
The VIC bank may be set by writing to $dd00 at any time after installing the loader, including setting it from an interrupt handler while loading. However, the upper 6 bits must always be set to 0. This means that no masking of the value read from $dd00 must be performed, and setting the VIC bank is as simple as
lda #vicbank; 4 legal values: $00, $01, $02 or $03
sta $dd00
The 4 values are not inverted, i.e., $03 is the lowmost VIC bank at $0000..$3fff, as usual.
Refer to loader/include/loader.inc for a ca65-syntax convenience macro to set the VIC bank.
Zeropage usage
--------------
The loader's zeropage variables may be overwritten while it's idle. While loading, the addresses from loader_zp_first up to and including loader_zp_last as denoted in loadersymbols-c64.inc must not be written to.
Handling disk changes
---------------------
Waiting for a specific disk to be inserted is done by trying to load a file that only exists on that disk in a loop until the file (or its first byte) is loaded successfully.
Thus, it is sufficient to perform
retry:
ldx #<filename
ldy #>filename
jsr loadraw; or loadcompd
bcs retry; branch on error
; file loaded successfully
The detection whether the file exists on the currently inserted disk can be sped up by having the file be merely one block big, or by resetting and then polling its first byte in memory from an interrupt handler and checking it against the expected value (e.g., the unique disk side number) as stored in the file.
Refer to loader/samples/turndisk for an example.
Compressing files
-----------------
Generally, files need to be compressed (and decompressed) in forward (memory address ascending) direction, and the compressed file's load address should be set so that the end of the compressed data aligns with the end of its uncompressed counterpart, such that in-place decompression can be performed.
Refer to the Makefile in loader/ for the "tools" target to build all the crunchers:
$ make tools
Also refer to loader/samples/test/Makefile for the corresponding command line arguments for each of the supported crunchers.
Synchronisation
---------------
For linearly loading and executing productions like demos, the general practice is to tie events such as loading files (and subsequently execute them) to specific music frame counts.
A music frame usually coincides with a video frame, as the player is called once a frame, such that a frame count may be updated with calling the player.
As loading generally does not take a well-determined period of time to finish (there is quite some deviation between different disks, same-model drives, and also loads from the same disk using the same drive), it is a good practice to sync to a minimum value of the frame count rather than an exact one:
wait:
lda framecountlo
cmp nextloadlo
lda framecounthi
sbc nextloadhi
bcc wait; branch if framecount < nextload
; load next file
This way, given enough headroom between loads, after a delayed loading-finished event, the next file would be loaded immediately, with the chain of events eventually catching up with the desired synchronisation.
Dir-art
-------
Changing the amount of file blocks in the directory is strongly discouraged, as this would disturb speculative loading and either slow down loading or cause data to be loaded beyond the file's dedicated memory areas.
However, there are a few ways to have an artful directory without interfering with the loading scheme:
Program files are recognised by the loader regardless of their apparent type, so anything else than PRG is allowed, including deleted PRG files.
For non-files, set their starting track to 0, such that they would not needlessly occupy space in the drive-side directory buffer, and also for the loadnext feature to work correctly.
In order to only have only the first few characters of filenames be significant for identification, set the FILENAME_MAXLENGTH option in the configuration file to the amount of desired chars, e.g. 1, such that chars in the directory-listed filenames beyond the first are ignored, and loading is effectively by names like "1*", "A*", etc.
For full freedom with the directory, including manipulating the amount of file blocks listed, set DIRTRACK in the configuration file to anything else than 18, such that a shadow directory is used on that track by the loader. This shadow directory would list the files normally, but be hidden and separate from the normal directory listing, which can then be manipulated freely.
Consider using Krill's fork of cc1541 (originally conceived by JackAsser) in loader/tools/cc1541_source for creating disk images with a shadow directory.
Optimising for speed
--------------------
To get closer to the maximum speed, a few rules can be followed. These include but are not limited to:
The fewer and the bigger files are required to load a set of data, the faster the data is loaded. Thus, it can be beneficial to pad gaps between loaded data, or to re-arrange data, and merge files, minimizing the number of files loaded.
Generally, it's advisable to store sequentially-loaded files on ascending tracks rather than alternatingly around the directory track 18.
The loader's native interleave is 4 revolutions for tracks 1-17, and 3 revolutions for tracks 18+. Thus, the highest throughput is obtained on tracks 18-24, then 25-30, then 31+, and finally 1-17.
The loader's speculative loading detects interleaves. However, block allocation is expected to follow the same rules as saving with the drive's ROM OS.
This means that when adding the interleave on the current sector number in order to obtain the next one, and there is a wrap (next sector >= number of sectors on this track), subtract number of sectors, then, if new sector is not 0, subtract 1:
sector += interleave;
if (sector >= number_of_sectors[track]) {
sector -= number_of_sectors[track];
if (sector > 0) {
--sector;
}
}
When operating with the loader's native interleave, however, violating above rule might make it load faster. This would simply be
sector = (sector + interleave) % number_of_sectors[track];
While loading to the RAM under the I/O registers at $d000-$dfff is optionally possible, it may slow down loading and increases the size of the resident loader portion. Avoiding this option and working around loading to $d000-$dfff by copying after loading or decompressing to that range only after loading may speed up general loader operation.
Consider using Krill's fork of cc1541 (originally conceived by JackAsser) in loader/tools/cc1541_source for creating disk images with optimised layouts.
Loading in the background
-------------------------
The loader normally loads in the mainline thread. If it shall load within a periodically-executed interrupt handler, i.e., in another mainline thread, the context must be switched accordingly. That is, stack pointer, registers and processor port must be buffered when switching from the current thread, and the context of the thread being switch to must be restored. This can be implemented on top of the loader and is orthogonal to its functionality.
Saving files
------------
Currently, no plug-in for the custom drive-code API implementing save functionality exists.
However, it is possible to uninstall the loader, then save using KERNAL calls, then re-install the loader. Saving using KERNAL calls requires valid KERNAL variables for serial bus operation in zeropage and lowmem. Re-installing the loader has the same requirements as initially installing it.
Limitations
-----------
There must always be at least 2 file entries in the directory.
The final directory block must begin with 00 ff. Some custom disk image creation tools produce an erroneous final directory block, such that its first bytes are 00 00. This is an invalid block length, and the loader rejects blocks with invalid track/sector links or block sizes. In the case of an invalid directory block, the loader keeps retrying to read it, effectively locking up.
As mentioned in the dir-art section, the number of file blocks listed in the directory must not be changed, unless having a shadow directory (with the actual number of file blocks).
If the loader is given extremely little raster time for loading, or even starved for multiple video frames, it's adviseable to enable the DISABLE_WATCHDOG option. This will remove watchdog code from the drive-side code portion, which is otherwise monitoring protocol breaches and would reset the drive on unexpected errors, such as the computer crashing or resetting during loading or cartridge freeze. The rationale is that the drive will become accessible normally without the need to power-cycle or manually reset it.
If not using Bitnax or tinycrunch, compressed files will usually go a few bytes beyond their corresponding uncompressed data. This means that some bytes directly after the uncompressed files' range in memory are overwritten. This can be worked around by not using those areas while loading.
However, when crunched files are larger than their uncompressed counterparts, data before the uncompressed memory region may be overwritten. To work around this, add some padding at the end of the uncompressed files.
License
-------
TL;DR: Please give credit, thanks.
When using this software or a derivative thereof in own works, please give appropriate credit, e.g., "Loader by Krill". This does not need to be on prominent display within the actual production itself, but at least in an accompanying note, readme, info file, or a similar means of attribution.
Copyright 2018 Gunnar Ruthenberg
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice in an appropriate form, e.g., "Loader by Krill" displayed anywhere in the actual production itself and/or accompanying documentation.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Third-party software (particularly in loader/tools and loader/samples/resources) and its derivatives (particularly in loader/src/decompress) are not affected and retain their original licenses.
Thanks
------
For extensive beta-testing, i thank, in alphabetical order:
Adder, Algorithm, ChristopherJam, Jojeli, Nomistake, Paralax, Perplex.
Special thanks to ChristopherJam aka Shrydar, again, for some valuable ideas and input, the concentric circles bitmap, and also for having one of the best test drives (i.e., pickiest wobbliest crapdrives) in the known universe. |
|
|
|
 | Search CSDb |
|
| Navigate | |
|
| Detailed Info | |
|
| Fun Stuff | |
· Goofs (2)
· Hidden Parts
· Trivia
|
|
| Forum | |
|
| Support CSDb | |
|
 |  |
|
