[CSDb] - User Forums - Dealing with (and distributing) large files

You are not logged in - nap

CSDb User Forums

Forums > C64 Coding > Dealing with (and distributing) large files

2014-10-01 10:53

AmiDog

Registered: Mar 2003
Posts: 97

Dealing with (and distributing) large files

Are there any good solutions for dealing with large files, i.e. files of more than 1MB?

The least annoying I could come up with in an emulated environment was to put everything in a large (16MB) REU-file, and DMA the data. I know the 1541U can emulate a large REU, so that might be a solution.

There is the IDE64, but there doesn't seem to be any command line tools to create and manage drive images, making it somewhat annoying to use when crossdeveloping.

Then we have multiple network adapters, which I assume might be a solution, assuming there actually are some ready to use, fast enough, loaders available somewhere?

And last but not least, spreading the data over multiple floppies might work, but would be very annoying unless one has access to a FD-2000/4000 and can use disks of a decent size. I assume it would be very slow though.

Any thoughts?

... 10 posts hidden. Click here to view all posts....

2014-10-03 10:16

chatGPZ

Registered: Dec 2001
Posts: 11114

too hard to run it in the emu? =P

2014-10-03 11:23

Oswald

Registered: Apr 2002
Posts: 5017

indeed. I could find the scpu vice hack, and add the reu file, but drive 8 emulation is disabled, and I find no way to start io.prg

edit: okay, so the code is compiled from a doom code compiled for another cpu, doesnt sound too promising regarding speed. :)

2014-10-03 14:58

chatGPZ

Registered: Dec 2001
Posts: 11114

what about trying the command line given in the readme? ^_^

2014-10-03 15:56

Oswald

Registered: Apr 2002
Posts: 5017

ah okay, didnt notice it, guess it has linux linebreaks and its all in one long line.

still no luck tho. command line attaches the reu image, but io.prg seems not to be loaded. autostart option seems have no effect, list doesnt shows anything, run is not typed out upon start.

2014-10-03 16:32

chatGPZ

Registered: Dec 2001
Posts: 11114

works just fine here... are you using a recent snapshot?

2014-10-03 16:54

Oswald

Registered: Apr 2002
Posts: 5017

okay, found the latest version, runs fine :D

boy this is awesome no matter how slow and unplayable it is. never thought I will play DOOM on a c64!! (in emu:)

I wonder how fast it could be if directly compiled for scpu, and optimized.

wish there was some dithering it could help visibility a lot.

2014-10-03 17:03

Count Zero

Registered: Jan 2003
Posts: 1821

So, thats a nice use case! :)

xscpu64 -config nonexistfiletoresetconf +cart -reu -reusize 16384 -reuimage doom.reu io.prg

Nice one AmiDog - boo, Oswald! :)

Such a distribution is perfectly fine with me - maybe Oswald & friends need help files pointing out details.

Just release with some fun pron WADs people watching nufli vids will rush to play SCPU doom clones :)

But what you are actually asking for is an xscpu64 option which pre-loads the up to 16MB RAM of the REU as well, to speed up the whole process, no?
Soci surely can help out here, I think.

For real hardware some generic loader thing indeed is not too trivial. Bitbanging the serial bus is no fun, network adapters not well spread and I wouldn't bet dimes on 1541U working well with SCPU on real HW either :(

2014-10-04 09:15

Axis/Oxyron

Registered: Apr 2007
Posts: 91

This doom port is surely ugly, but really impressive. Never thought this game could run on that platform. Also speed is not 'that' bad, compared to AGA Amiga ports that are not so much faster.

If this is just the compiled and ported Linux Doom. I would suggest to do the port based on the work of BadMooD, a very impressive Doom port for Atari Falcon.

http://devilsdoorbell.com

The guy behind it did a lot of optimizations on things like AI and collision detection, which took more than 50% of the CPU time.

2014-10-10 09:00

AmiDog

Registered: Mar 2003
Posts: 97

Quoting Oswald

edit: okay, so the code is compiled from a doom code compiled for another cpu, doesnt sound too promising regarding speed. :)

I agree, but it probably isn't as bad as you might think, at least not for 32bit applications. The MIPS is a really simple 32bit RISC CPU. There are no CPU flags to keep track of, it has 32 general purpose registers, and few and simple instructions.

Sure, it is a load/store architecture, which means any memory accesses first goes to an emulated register in the direct page before any processing can be done. On the other hand, a good 65816 compiler would have to cache data in the direct page to achieve good performance. As an example, a 32bit add:

clc
lda <low>
adc <low>
sta <low>
lda <high>
adc <high>
sta <high>

When all addresses are in the direct page, it takes (2+6*4) 26 cycles to complete. If all addresses are in the "far" segment and require long addressing, each memory access take 6 instead of 4 cycles. However, that is only true if the address is in fast SRAM. If it's in DRAM (i.e. SuperRAM) the situation is a lot worse.

Since code and data are in different sections, they will be located far apart. A read from a different column results in a 1 cycle additional delay, and since a column is only 4 bytes, that is always going to be true. If the read is from a different row, which is also very likely since a row is 2-8KB and code and data for a large project will be far from each other, you instead get a 3.5 cycle penalty.

But that's only half of the story, after the memory has been read, the CPU needs to fetch the next opcode, which will be far away from the data, causing another penalty. So all in all, that 6 cycle load will actually take 6 + 3.5 + 3.5 = 13 cycles. Writes are somewhat faster. There is no column penalty and the row penalty is 3.0 cycles. All in all, that 32bit add does take (2+13*4+12.5*2) 79 cycles if the operands are in SuperRAM.

Just image what would happen if your compiler isn't clever enough to cache global data in the direct page before a loop, causing the same value to be read over and over again, taking (2*13) 26 cycles instead of (2*4) 8 cycles each iteration...

2014-11-03 15:14

AmiDog

Registered: Mar 2003
Posts: 97

Quoting Axis/Oxyron

This doom port is surely ugly

And now it can be made to look even worse :-) New release uploaded which fixes low detail mode and map rendering. There's also a 16 color FLI mode now, which does look a little less bad but slows things down even more...

edit: The FLI mode probably doesn't work on the real hardware due to timing differences between VICE and real hardware (bug #543). Also, I haven't bothered to fix the leftmost three columns ("Fli-bug" area) since SuperCPU-style fullscreen FLI doesn't work in VICE currently (bug #545).

Previous - 1 | 2 - Next

Refresh

Subscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search for in
All times are CET.

Search CSDb

Advanced

Users Online

bugjam
Guests online: 140

Top Demos

1 Next Level  (9.8)
2 Mojo  (9.7)
3 Coma Light 13  (9.7)
4 Edge of Disgrace  (9.6)
5 Comaland 100%  (9.6)
6 No Bounds  (9.6)
7 Uncensored  (9.6)
8 Wonderland XIV  (9.6)
9 Bromance  (9.6)
10 Memento Mori  (9.6)

Top onefile Demos

1 It's More Fun to Com..  (9.7)
2 Party Elk 2  (9.7)
3 Cubic Dream  (9.6)
4 Copper Booze  (9.5)
5 TRSAC, Gabber & Pebe..  (9.5)
6 Rainbow Connection  (9.5)
7 Onscreen 5k  (9.5)
8 Wafer Demo  (9.5)
9 Dawnfall V1.1  (9.5)
10 Quadrants  (9.5)

Top Groups

1 Oxyron  (9.3)
2 Nostalgia  (9.3)
3 Booze Design  (9.3)
4 Censor Design  (9.3)
5 Crest  (9.3)

Top Original Suppliers

1 Black Beard  (9.5)
2 Derbyshire Ram  (9.5)
3 hedning  (9.2)
4 Baracuda  (9.1)
5 Irata  (8.5)

Page generated in: 0.052 sec.