| |
AmiDog
Registered: Mar 2003
Posts: 97 |
Dealing with (and distributing) large files
Are there any good solutions for dealing with large files, i.e. files of more than 1MB?
The least annoying I could come up with in an emulated environment was to put everything in a large (16MB) REU-file, and DMA the data. I know the 1541U can emulate a large REU, so that might be a solution.
There is the IDE64, but there doesn't seem to be any command line tools to create and manage drive images, making it somewhat annoying to use when crossdeveloping.
Then we have multiple network adapters, which I assume might be a solution, assuming there actually are some ready to use, fast enough, loaders available somewhere?
And last but not least, spreading the data over multiple floppies might work, but would be very annoying unless one has access to a FD-2000/4000 and can use disks of a decent size. I assume it would be very slow though.
Any thoughts? |
|
| |
chatGPZ
Registered: Dec 2001
Posts: 11135 |
considering that most people would probably not download these files using their c64.... why not just put them into a .zip and let the user decide where/how to put it on the c64? |
| |
AmiDog
Registered: Mar 2003
Posts: 97 |
Still, the code to access the large file(s) would have to be written. Some devices may have direct support for seeking in files, some (most?) do not.
So there are really two problems:
1. Getting the files to some kind of media or storage the C64 can access one way or another.
2. Accessing the files, which may or may not be split into smaller chunks in order to fit on some kind of floppy-disks. |
| |
chatGPZ
Registered: Dec 2001
Posts: 11135 |
uhm. of course. what are you trying to achieve anyway, and who is your target audience? a generic solution would have to support most (preferably all) of what you listed basically :) |
| |
soci
Registered: Sep 2003
Posts: 474 |
Quoting AmiDogThere is the IDE64, but there doesn't seem to be any command line tools to create and manage drive images, making it somewhat annoying to use when crossdeveloping.
I tend to disagree. There's fusecfs which lets you mount CFS disk images on Linux (and BSDs). The same image can be used in VICE at the same time if you're careful not to write to it. It's also handy for transferring gigabytes of stuff, especially since that a nice quadratic performance bug is gone ;)
Quoting AmiDogThen we have multiple network adapters, which I assume might be a solution, assuming there actually are some ready to use, fast enough, loaders available somewhere?
There's PCLink on IDEDOS. I use it whenever I have real hardware with me. Works over various physical connections, but I use USB/Ethernet most of the time.
I don't even copy stuff over, as it's completely transparent for file operations. Even seeking works the same way as with a filesystem on CF card (using 0.91 beta). Did I copied all those MP3s to test the MP3 player's tag parsing to a CF card or into VICE? Of course not.
I hacked VICE to emulate USB PCLink for IDE64, but I was lazy to convert it to use sockets instead of pair of pipes, therefore it's not available. Last time I've checked the CS8900 emulation was not accurate enough for Ethernet PCLink without workarounds which degraded performance on real hardware. Yes, I know I'll get a comment soon that maybe I should fix it sometime ;) |
| |
Count Zero
Registered: Jan 2003
Posts: 1825 |
What groepaz said - there may also be the question about the data being static or in some way externally modified. If it is static and a game or so you might want to consider a more or less static solution such as an easyflash as a target.
Whenever it comes to speed on an emulated or original system hardly anything can beat a REU anyhow and "more than 1MB" is also a border you may want to tweak on depending on the targetted hardware. So IMHO such decisions depend on more fine grained parameters here :) |
| |
AmiDog
Registered: Mar 2003
Posts: 97 |
Quoting sociI tend to disagree. There's fusecfs which lets you mount CFS disk images on Linux (and BSDs). The same image can be used in VICE at the same time if you're careful not to write to it. It's also handy for transferring gigabytes of stuff, especially since that a nice quadratic performance bug is gone ;)
Yes, for Linux that's a solution. However, I do most of my developing using Windows and Cygwin (which I've put on a USB stick so I can use more or less any PC I have access to without the need to install anything).
I've considered trying to hack that FUSE driver into a command line tool, but not had the time to look into it yet. |
| |
AmiDog
Registered: Mar 2003
Posts: 97 |
Quoting Groepazuhm. of course. what are you trying to achieve anyway, and who is your target audience? a generic solution would have to support most (preferably all) of what you listed basically :)
Well, while this isn't meant to be taken seriously, it made it quite obvious that some kind of solution is needed to deal with large files. And while the REU hack does work in this case, atleast in en emulated environment, it's limited to 16MB which may be too little... |
| |
chatGPZ
Registered: Dec 2001
Posts: 11135 |
i'd just use SD-card then |
| |
soci
Registered: Sep 2003
Posts: 474 |
Quoting AmiDogI've considered trying to hack that FUSE driver into a command line tool, but not had the time to look into it yet.
Sounds interesting. Could be useful for other FUSE filesystems as well.
What about mkisofs or genisoimage? This is part of cygwin.
You can make an iso image with "mkisofs -J directory >file.iso". Feed this to VICE as one of the IDE64 disk images, and it'll emulate a DVD drive. IDEDOS can use it with long filenames, proper directory structure and seekable files. |
| |
Oswald
Registered: Apr 2002
Posts: 5023 |
I'd love to see a vid of that scpu-doom, no matter how slow :)insta nerdgasm |
... 10 posts hidden. Click here to view all posts....
|
| Previous - 1 | 2 - Next |
