Log inRegister an accountBrowse CSDbHelp & documentationFacts & StatisticsThe forumsAvailable RSS-feeds on CSDbSupport CSDb Commodore 64 Scene Database
You are not logged in - nap
CSDb User Forums


Forums > C64 Coding > Dealing with (and distributing) large files
2014-10-01 10:53
AmiDog

Registered: Mar 2003
Posts: 97
Dealing with (and distributing) large files

Are there any good solutions for dealing with large files, i.e. files of more than 1MB?

The least annoying I could come up with in an emulated environment was to put everything in a large (16MB) REU-file, and DMA the data. I know the 1541U can emulate a large REU, so that might be a solution.

There is the IDE64, but there doesn't seem to be any command line tools to create and manage drive images, making it somewhat annoying to use when crossdeveloping.

Then we have multiple network adapters, which I assume might be a solution, assuming there actually are some ready to use, fast enough, loaders available somewhere?

And last but not least, spreading the data over multiple floppies might work, but would be very annoying unless one has access to a FD-2000/4000 and can use disks of a decent size. I assume it would be very slow though.

Any thoughts?
2014-10-01 11:06
chatGPZ

Registered: Dec 2001
Posts: 11386
considering that most people would probably not download these files using their c64.... why not just put them into a .zip and let the user decide where/how to put it on the c64?
2014-10-01 12:02
AmiDog

Registered: Mar 2003
Posts: 97
Still, the code to access the large file(s) would have to be written. Some devices may have direct support for seeking in files, some (most?) do not.

So there are really two problems:

1. Getting the files to some kind of media or storage the C64 can access one way or another.
2. Accessing the files, which may or may not be split into smaller chunks in order to fit on some kind of floppy-disks.
2014-10-01 12:05
chatGPZ

Registered: Dec 2001
Posts: 11386
uhm. of course. what are you trying to achieve anyway, and who is your target audience? a generic solution would have to support most (preferably all) of what you listed basically :)
2014-10-01 19:10
soci

Registered: Sep 2003
Posts: 480
Quoting AmiDog
There is the IDE64, but there doesn't seem to be any command line tools to create and manage drive images, making it somewhat annoying to use when crossdeveloping.

I tend to disagree. There's fusecfs which lets you mount CFS disk images on Linux (and BSDs). The same image can be used in VICE at the same time if you're careful not to write to it. It's also handy for transferring gigabytes of stuff, especially since that a nice quadratic performance bug is gone ;)
Quoting AmiDog
Then we have multiple network adapters, which I assume might be a solution, assuming there actually are some ready to use, fast enough, loaders available somewhere?

There's PCLink on IDEDOS. I use it whenever I have real hardware with me. Works over various physical connections, but I use USB/Ethernet most of the time.

I don't even copy stuff over, as it's completely transparent for file operations. Even seeking works the same way as with a filesystem on CF card (using 0.91 beta). Did I copied all those MP3s to test the MP3 player's tag parsing to a CF card or into VICE? Of course not.

I hacked VICE to emulate USB PCLink for IDE64, but I was lazy to convert it to use sockets instead of pair of pipes, therefore it's not available. Last time I've checked the CS8900 emulation was not accurate enough for Ethernet PCLink without workarounds which degraded performance on real hardware. Yes, I know I'll get a comment soon that maybe I should fix it sometime ;)
2014-10-01 19:47
Count Zero

Registered: Jan 2003
Posts: 1932
What groepaz said - there may also be the question about the data being static or in some way externally modified. If it is static and a game or so you might want to consider a more or less static solution such as an easyflash as a target.

Whenever it comes to speed on an emulated or original system hardly anything can beat a REU anyhow and "more than 1MB" is also a border you may want to tweak on depending on the targetted hardware. So IMHO such decisions depend on more fine grained parameters here :)
2014-10-02 06:49
AmiDog

Registered: Mar 2003
Posts: 97
Quoting soci
I tend to disagree. There's fusecfs which lets you mount CFS disk images on Linux (and BSDs). The same image can be used in VICE at the same time if you're careful not to write to it. It's also handy for transferring gigabytes of stuff, especially since that a nice quadratic performance bug is gone ;)

Yes, for Linux that's a solution. However, I do most of my developing using Windows and Cygwin (which I've put on a USB stick so I can use more or less any PC I have access to without the need to install anything).

I've considered trying to hack that FUSE driver into a command line tool, but not had the time to look into it yet.
2014-10-02 07:01
AmiDog

Registered: Mar 2003
Posts: 97
Quoting Groepaz
uhm. of course. what are you trying to achieve anyway, and who is your target audience? a generic solution would have to support most (preferably all) of what you listed basically :)

Well, while this isn't meant to be taken seriously, it made it quite obvious that some kind of solution is needed to deal with large files. And while the REU hack does work in this case, atleast in en emulated environment, it's limited to 16MB which may be too little...
2014-10-02 08:27
chatGPZ

Registered: Dec 2001
Posts: 11386
i'd just use SD-card then
2014-10-02 17:22
soci

Registered: Sep 2003
Posts: 480
Quoting AmiDog
I've considered trying to hack that FUSE driver into a command line tool, but not had the time to look into it yet.

Sounds interesting. Could be useful for other FUSE filesystems as well.

What about mkisofs or genisoimage? This is part of cygwin.

You can make an iso image with "mkisofs -J directory >file.iso". Feed this to VICE as one of the IDE64 disk images, and it'll emulate a DVD drive. IDEDOS can use it with long filenames, proper directory structure and seekable files.
2014-10-03 07:53
Oswald

Registered: Apr 2002
Posts: 5094
I'd love to see a vid of that scpu-doom, no matter how slow :)insta nerdgasm
 
... 10 posts hidden. Click here to view all posts....
 
Previous - 1 | 2 - Next
RefreshSubscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search   for   in  
All times are CET.
Search CSDb
Advanced
Users Online
Brittle/Dentifrice^(?)
Guests online: 77
Top Demos
1 Next Level  (9.7)
2 13:37  (9.7)
3 Mojo  (9.7)
4 Coma Light 13  (9.6)
5 Edge of Disgrace  (9.6)
6 What Is The Matrix 2  (9.6)
7 The Demo Coder  (9.6)
8 Uncensored  (9.6)
9 Comaland 100%  (9.6)
10 Wonderland XIV  (9.6)
Top onefile Demos
1 No Listen  (9.6)
2 Layers  (9.6)
3 Cubic Dream  (9.6)
4 Party Elk 2  (9.6)
5 Copper Booze  (9.6)
6 Dawnfall V1.1  (9.5)
7 Rainbow Connection  (9.5)
8 Onscreen 5k  (9.5)
9 Morph  (9.5)
10 Libertongo  (9.5)
Top Groups
1 Performers  (9.3)
2 Booze Design  (9.3)
3 Oxyron  (9.3)
4 Triad  (9.3)
5 Censor Design  (9.3)
Top NTSC-Fixers
1 Pudwerx  (10)
2 Booze  (9.7)
3 Stormbringer  (9.7)
4 Fungus  (9.6)
5 Grim Reaper  (9.3)

Home - Disclaimer
Copyright © No Name 2001-2024
Page generated in: 0.108 sec.