[CSDb] - User Forums - Error correction when backing up old disks.

You are not logged in - nap

CSDb User Forums

Forums > C64 Coding > Error correction when backing up old disks.

2008-09-01 08:30

Perff
Administrator

Posts: 1680

Error correction when backing up old disks.

<Post edited by Perff on 1/9-2008 11:21>

I guess its rather common today, that one get a read-error when making a backup (d64 or similar) of ones old disks.
I use my own homemade tranfer software which have served me for many years, but it dosn't try to error correct in any way, except reading bad sectors many times, and I plan to improve that somehow.

Before spending several hours thinking up stuff others might aldready have thought off before, I now ask you all for ideas. :)

2008-09-01 09:17

jailbird

Registered: Dec 2001
Posts: 1578

For some reason when I first looked on the topic's title, I read Erection when backing up old disks. But the worst part is, by knowing I'm browsing CSDb it seemed to me absolutely plausible.

2008-09-01 10:38

chatGPZ

Registered: Dec 2001
Posts: 11386

lol :o)

perff: error *correction* is absolutely impossible with 1541 discs, mostly due to the very weak checksums. all software i know does multiple retries on reading, and thats pretty much all you can do :)

2008-09-01 11:11

Perff
Administrator

Posts: 1680

Ok then. I guess I'll just have to write what my thoughts have been so far. Then I'll see if I'll be laughed all the way out of here, or if I have something to work on after all. :)

The basic idea is to read the raw GCR-data many times, and on corrupted sectors you often get different results.
Then I plan to somehow analyze the bit streams I get, and see what I can figure out.

For instance I think the first 10 bits when converted from GCR to regular bytes, result in the number 7. (why actually??)
Also the last 20 bits are afaik normally "01010010100101001010" resulting in two 0's. This could help you align the the bit stream in case there was a missing or extra bit somewhere. (If this is in fact the case???)

Also by simply comparing the different bitstrams you get, you might figure out where in the bitstream the problem is. The strams might be the same up til a certain point, where a few differences occur, or simply a shift, so the streams mis-align after this point.

Finally you also know that only 16 of the 32 GCR-combinations are valid, also helping you to identify where the problem might be.

When you have identified where in the bit stream the problem is, my thoughts was, that you might be able to do something about it? I havn't worked any further on this, but at least you got the super efficient xor-checksum of the 256 bytes to go on!! ;-P

I know there are lot of situations where there is nothing to be done, but I guess there still are many disks which are just a little bit corrupted, so it would be possible to recover it!?!

2008-09-01 11:27

Oswald

Registered: Apr 2002
Posts: 5094

how about creating different d64s of the possible "solutions" and then trying them all out :) this could be automated with extra efforts (is there cpu jam after certain amount of cycles..)

2008-09-01 11:46

chatGPZ

Registered: Dec 2001
Posts: 11386

Quote:

For instance I think the first 10 bits when converted from GCR to regular bytes, result in the number 7. (why actually??)
Also the last 20 bits are afaik normally "01010010100101001010" resulting in two 0's. This could help you align the the bit stream in case there was a missing or extra bit somewhere. (If this is in fact the case???)

don't remember exactly, but it sounds right... probably sector header or sync mark, and then the gap at the end. and yes ofcourse you can (well you HAVE TO) use this to align the data properly :)

Quote:

Finally you also know that only 16 of the 32 GCR-combinations are valid, also helping you to identify where the problem might be.

uhm well, yes. reading raw gcr until all gcr data you get is actually valid is a good idea - and warpcopy does it already, just like starcommander and opencbm :) only the original drive routines return bullshit for illegal gcr =D

Quote:

The basic idea is to read the raw GCR-data many times, and on corrupted sectors you often get different results.
Then I plan to somehow analyze the bit streams I get, and see what I can figure out.

...
Quote:

When you have identified where in the bit stream the problem is, my thoughts was, that you might be able to do something about it? I havn't worked any further on this, but at least you got the super efficient xor-checksum of the 256 bytes to go on!! ;-P

I know there are lot of situations where there is nothing to be done, but I guess there still are many disks which are just a little bit corrupted, so it would be possible to recover it!?!

the basic problem is, there is no way to figure out where in the bitstream the problem is (except for determining illegal gcr data), and there is no way to check wether the decoded data is correct (that xor checksum is totally useless)

Quote:

how about creating different d64s of the possible "solutions" and then trying them all out :) this could be automated with extra efforts (is there cpu jam after certain amount of cycles..)

that sounds like something rambones has done already 20 years ago :o)

2008-09-01 12:20

Perff
Administrator

Posts: 1680

Quote:

the basic problem is, there is no way to figure out where in the bitstream the problem is (except for determining illegal gcr data), and there is no way to check wether the decoded data is correct (that xor checksum is totally useless)

As already stated I havn't though much about this yet, but I have made one simple test.
I have a disk with a persistant read-error on it, and by reading the GCR-bit stream many times, I get two different streams, where the only difference is this:
.....10011..
.....1011...
all the .'s are where the streams are identical, so around here is where the problem is. Note that the last stream is missing a bit.
Because I know the files on the disk, I know what the bitstream should be:
.....10101....

My thoughs was, that even if I didn't know what the bitstream should be, I could figure it out, by trying different permutations of the surrounding bits, which are legal GCR-code, and validate the result up against the useless XOR-checksum. As long as there is only one error, and you only permutate a few bits, I think the checksum is actually usefull!
Perhaps there might even be certain known patterns, which could be implemented into the recovery algorithm? Like the one above?

Again, all errors are probably not as simple as this, but I don't think this is the only disk with such an error, so..
Am I just being overoptimistic, or??

2008-09-01 12:58

chatGPZ

Registered: Dec 2001
Posts: 11386

its a bit optimistic indeed. however, there infact IS a pattern, but it's kinda hard to determine it on gcr level (you can do it on raw level, but thats not possible with a 1541).

what happens when disk age is ... due to physical effects, flux transitions "wander" to their nearest neighbour. so a "long" pulse which is followed by a "short" pulse" becomes a little bit shorter over time, and the neighboured "short" pulse becomes a little bit longer. given enough time (and bad luck =P) that can result in a "long" pulse beeing detected as a "middle" pulse... ie 1000 (long) followed by 10 (short) becomes 100 (mid) followed by 100 (mid). or depending on the actual timing (and bad luck =P) it could become any other combination of mid/short long/mid etc.

correcting this type of error on gcr level is uhm... probably not a good idea =P it works okish on raw level though.

Refresh

Subscribe to this thread:

You need to be logged in to post in the forum.

Search the forum:
Search for in
All times are CET.

Search CSDb

Advanced

Users Online

algorithm
JEZ
rambo/Therapy/ Resou..
kbs/Pht/Lxt
Impetigo/Crescent
Krill/Plush
𝘁𝗡𝗚/FairLight
Guests online: 257

Top Demos

1 Next Level  (9.7)
2 13:37  (9.7)
3 Mojo  (9.7)
4 Coma Light 13  (9.6)
5 Edge of Disgrace  (9.6)
6 What Is The Matrix 2  (9.6)
7 The Demo Coder  (9.6)
8 Uncensored  (9.6)
9 Comaland 100%  (9.6)
10 Wonderland XIV  (9.6)

Top onefile Demos

1 No Listen  (9.6)
2 Layers  (9.6)
3 Cubic Dream  (9.6)
4 Party Elk 2  (9.6)
5 Copper Booze  (9.6)
6 Dawnfall V1.1  (9.5)
7 Rainbow Connection  (9.5)
8 Onscreen 5k  (9.5)
9 Morph  (9.5)
10 Libertongo  (9.5)

Top Groups

1 Performers  (9.3)
2 Booze Design  (9.3)
3 Oxyron  (9.3)
4 Censor Design  (9.3)
5 Triad  (9.3)

Top Original Suppliers

1 Derbyshire Ram  (9.7)
2 Fungus  (9.3)
3 Black Beard  (9.2)
4 Baracuda  (9.2)
5 hedning  (9.1)

Page generated in: 0.05 sec.