CDROM, F2 Reed-Solomon P Q codes in CIRC encoder

Question

I research info about CDROM principies. In standart http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-130.pdf On page 35(45 in pdf) i see CIRC encoder. And his have Q code, and P codes, who calculated by reed-solomon algoritm. I try to confirm this and doit some sample audio tracks(audio track not used scramber as data track) one fill with 0x01 pattern, and one with 0xA5(CIRC interlive byte in packet not bits, and i see Q and P in F3 frame). After i reed this sector from CD(directly from Laser out) with logic analyzer, and decrypt by script. I have this data for track with pattern 0x01

S1 01 01 01 01 01 01 01 01 01 01 01 01 e5 6e 4e c5 01 01 01 01 01 01 01 01 01 01 01 01 ff ff ff ff

S2 01 01 01 01 01 01 01 01 01 01 01 01 e5 6e 4e c5 01 01 01 01 01 01 01 01 01 01 01 01 ff ff ff ff

First byte is Subcode sumbol in this sample SYNC_1 and SYNC_2

For track with pattern 0xA5

S1 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 6b bc a5 72 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 ff ff ff ff

S1 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 6b bc a5 72 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 ff ff ff ff

If see on CIRC all right 12-15 bytes its inverted Q parity and 28-32 P parity(First byte its sybcode his add at F3).

But i cant find algoritm for calculation this bytes, my maths skills very bad. I try calculator from cdrecord, his doit another codes, try some another Reed-Solomon emplemetation but i cant get identical code from this sample. Where i can get worked implementation of this code.

Audio track not used Q, P in sector, audio not have sectors. Audio write by 24 bytes(6 sample) its F1 frames, and directly go to F2 frame, F2 is CIRC interleaver and his use Q and P. But not same as Q and P for data sector on F1 level. Q 4 byte for 24 bytes frame F1 and P is 4 bytes for F1 interleaved frame plus Q. — Vulcanator, Jan 17 '19 at 07:45
I found a link to [edc_ecc.c](https://github.com/Distrotech/cdrtools/tree/master/libedc), which should be helpful. I'll take a look at the code later and see if there's anything I can explain. — rcgldr, Jan 17 '19 at 10:10
In edc_ecc have function encode_L1_Q and encode_L1_P firs doit from 24 bytes 28 byte by add 4 byte of RS code(and add his in between 12-13 originals byte), second doit from 28 bytes 32 byte. Its as CIRC encoder on first and second stage. But value that his calculate not indetical that i read from real cd. I forund this document https://www.ece.uvic.ca/~agullive/reedsolomon405-511-2016.pdf and on page 23 description abot algoritm calculation, but i not see reasisation anythere. — Vulcanator, Jan 18 '19 at 09:13
I deleted prior comments since they were about L2. The L1 encode functions implement multiply by alog[( log(data) + log(RS term) ) % 255]. The log of the RS terms are stored in AQ and AP tables, but it appears AQ table is missing entries for AQ[0], AQ[1], AQ[2]. I'll try to figure out what the actual P and Q generator polynomials are later on. There is a tutorial on RS ECC in this [NASA pdf file](https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19900019023_1990019023.pdf) . — rcgldr, Jan 18 '19 at 10:20

rcgldr · Accepted Answer · 2021-06-22T18:03:48.143

I'm putting what I find in this answer as I make progress, based on the edc_ecc.c web github files. RSL12 is GF(2^8), polynomial x^8 + x^4 + x^3 + x^2 + 1 => hex 11d. All non-zero numbers in the field can be considered as powers of hex 02.

The P generator polynomial in hex is:

(x+01)(x+02)(x+04)(x+08) = 01 x^4 + 0f x^3 + 36 x^2 + 78 x + 40

If you look at the 4 entries for AP[...][31], you see the values 75, 249, 78, 6, these are the decimal logs of hex 0f, 36, 78, 40. Note that AP should be AP[4][28] (not [4][32]), the fix is shown below.

Based on your (now deleted) comment, I "uninverted" Q in the examples you gave in the original question, and using my own RS demo program to calculate the P parities, I now get 00 00 00 00:

01 01 01 01 01 01 01 01 01 01 01 01 1a 91 b1 3a 01 01 01 01 01 01 01 01 01 01 01 01 00 00 00 00
a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 94 43 5a 8d a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 00 00 00 00

The Q generator polynomial is the same as the P generator polynomial. It's used for RS(28,24), but the parity bytes are in the middle, so conventional encoding needs to be modified as explained below.

AQ[][] is wrong, using AQ[3][], to get Q[3], I get 69 instead of 3a:

01 01 01 01 01 01 01 01 01 01 01 01 -- -- -- 69 01 01 01 01 01 01 01 01 01 01 01 01

In addition, AQ[0] only has 21 bytes defined, AQ[1] only has 22 bytes defined, AQ[2] only has 23 bytes defined, AQ[3] has 24 bytes defined, but they are apparently wrong.

There is a workaround, use 4 erasure decoding with locations 12 through 15 marked as erasures (the xx xx xx xx) to do the Q encoding:

01 01 01 01 01 01 01 01 01 01 01 01 xx xx xx xx 01 01 01 01 01 01 01 01 01 01 01 01
a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 xx xx xx xx a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5

After the 4 erasure correction, the Q parity bytes are encoded:

01 01 01 01 01 01 01 01 01 01 01 01 1a 91 b1 3a 01 01 01 01 01 01 01 01 01 01 01 01
a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 94 43 5a 8d a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5 a5

Using the 4 erasure decoding method, I generated a fixed AQ[][]:

static const unsigned char AQ[4][24] =
  {{58,152,173,95,88,43,134,205,143,131,163,75,249,66,151,116,125,184,110,16,58,62,137,113},
   {30,214,148,138,112,154,157,96,49,198,189,249,69,47,147,235,156,47,209,183,138,232,205,120},
   {162,244,13,171,213,236,71,177,253,162,59,78,243,180,186,34,78,136,130,85,108,115,178,246},
   {158,179,101,94,49,140,211,149,137,169,81,6,72,157,122,131,190,116,22,64,68,143,119,22}};

However if you plan on writing a decoder (that fixes erasures and/or errors), then you could use the same method I did of using a 4 erasure decode instead of doing a encode. If I recall correctly, this is how some early DAT (digital audio tape) drives implemented this, as they also had their parity bytes in the middle of data.

AP should be AP[4][28]. P is RS(32,28), 28 bytes of data used to generate 4 bytes of parities. The first 4 values from each row of AP[...][32] should be removed, so it becomes AP[4][28], and encode_L1_P() should be encoding 28 bytes of data (a one line fix as noted below).

static const unsigned char AP[4][28] =
  {{249,142,180,197,5,155,153,132,143,244,101,76,102,155,203,104,58,152,173,95,88,43,134,205,143,131,163,75},
   {205,252,218,199,202,41,136,106,119,238,193,103,123,242,83,178,30,214,148,138,112,154,157,96,49,198,189,249},
   {67,11,131,40,7,41,80,147,151,17,245,253,208,66,228,116,162,244,13,171,213,236,71,177,253,162,59,78},
   {148,186,203,11,161,159,138,149,250,107,82,108,161,209,110,64,158,179,101,94,49,140,211,149,137,169,81,6}};

encode_L1_P() needs one line fixed:

static int
encode_L1_P(inout)
    unsigned char inout[L1_RAW + L1_Q + L1_P];
{
    unsigned char *P;
    int i;

    P = inout + L1_RAW + L1_Q;

    memset(P, 0, L1_P);
    for (i = 0; i < L1_RAW + L1_Q; i++) {   /* fix (remove + L1_P) */
        unsigned char data;

        data = inout[i];
        if (data != 0) {
            unsigned char base = rs_l12_log[data];

            P[0] ^= rs_l12_alog[(base+AP[0][i]) % (unsigned)((1 << RS_L12_BITS)-1)];
            P[1] ^= rs_l12_alog[(base+AP[1][i]) % (unsigned)((1 << RS_L12_BITS)-1)];
            P[2] ^= rs_l12_alog[(base+AP[2][i]) % (unsigned)((1 << RS_L12_BITS)-1)];
            P[3] ^= rs_l12_alog[(base+AP[3][i]) % (unsigned)((1 << RS_L12_BITS)-1)];
        }
    }
    return (0);
}

In response to comments. To generate AQ, I used erasure correction to generate 24 sets of parities:

01 00 00 00 00 00 00 00 00 00 00 00 69 60 bf b7 00 00 00 00 00 00 00 00 00 00 00 00
00 01 00 00 00 00 00 00 00 00 00 00 49 f9 fa 4b 00 00 00 00 00 00 00 00 00 00 00 00
...
00 00 00 00 00 00 00 00 00 00 00 00 9e a7 ab 93 00 00 00 00 00 00 00 00 00 00 01 00
00 00 00 00 00 00 00 00 00 00 00 00 1f 3b cf ea 00 00 00 00 00 00 00 00 00 00 00 01

AQ[][ 0] = hex log2{69 60 bf b7} = decimal {058 030 162 158}
AQ[][ 1] = hex log2{49 f9 fa 4b} = decimal {152 214 244 179}
...
AQ[][22] = hex log2{9e a7 ab 93} = decimal {137 205 178 119}
AQ[][23] = hex log2{1f 3b cf ea} = decimal {113 120 246 022}

An alternative method is to use a full length 255 byte codeword all zeroes, except one byte set to 01 at the appropriate position modulo 255, to generate the parities in the standard way, parities = msg * x^4 % generator. This generates parities 16 bytes beyond their target position, so the offset needs to be adjusted by 255-16 = +239 modulo 255 to produce the target parities:

255_byte_message_offset = 28_byte_message_offset + 239 % 255:

msg[239] = 01 => parities = 69 60 bf b7
msg[240] = 01 => parities = 49 f9 fa 4b
...
msg[010] = 01 => parities = 9e a7 ab 93
msg[011] = 01 => parities = 1f 3b cf ea

Thanks working fine! I could read a some additonal pattern and all workink fine. May be maybe someone will need secor pattern without subcode byte: "00 00 00 00 00 00 00 00 00 00 00 00 61 58 54 6c 00 00 00 00 00 00 00 00 00 00 01 00 ff ff ff ff" "00 00 00 00 00 00 00 00 00 00 00 00 fb 59 83 70 00 00 00 00 00 00 00 00 00 00 ef be ff ff ff ff" "ab ad 00 00 00 00 00 00 00 00 00 00 37 e1 4a 9e 00 00 00 00 00 00 00 00 00 00 be ba ff ff ff ff" Very big thanks for help! — Vulcanator, Jan 18 '19 at 20:02
@Vulcanator - I updated my answer. AP should be AP[4][28], not AP[4][32] (the first 4 values on each line are not used). encode_L1_P needs one line fixed. I deleted my prior comments, since they are now included in my updated answer. You may want to delete you prior comments as well. I also assume you will need to "uninvert" Q before calculating P, then invert both of them after if that is how cd-roms work. — rcgldr, Jan 22 '19 at 20:57
Super. Work fine. Big thanks. In the near future I plan to write an article on the work of the СD I can mention you in the article as a person who helped, and if so, how best to do it? — Vulcanator, Jan 22 '19 at 21:10
@Vulcanator - just a line or a footnote about helping would be fine. You might want to include the guy that created edc_ecc.c (in spite of the corrections that were needed) . Have you checked to see if the data track stuff (L2 ...) is working? — rcgldr, Jan 22 '19 at 21:14
No L2 not tested, but L2 is not hardware part(L1 is hardware and may be because not been tested, you cant read/write L1 data from CDROM), and his use in cdrecord to create sector data for write(if his not working fine cdrecord cant be write good cd). L2 implementation in all CD Writer software, 100% in cdclone and nero in DLL. I think his fine. — Vulcanator, Jan 22 '19 at 21:33
Hi, I am the author of edc_ecc.c. Thanks for checking and fixing my code! It was untested before, because I had no real data to test it with. — hexcoder, Feb 05 '19 at 23:36
@hexcoder its may be intersing for you https://pastebin.com/QM3iNzbb its f3 frames read directly form cd. First byte is subchenal data, 32 is data bytes in f3 frame. Its 5 sectors. And first two frame of 98 is sync bytes for subchanesl. — Vulcanator, Feb 09 '19 at 22:10
@Vulcanator: Thanks very much for the test data! I will use it to test my revised encoder table generator (to be released soon in schilytools/libedc https://sourceforge.net/projects/schilytools/files/?source=directory). — hexcoder, Feb 11 '19 at 09:31
@Vulcanator: the first 5 f3-frames (from 16th and 18th of January) are each from a often repeated data pattern, right? So the delay sections 1 and 2 (C.3 and C.5 in ECMA pdf page 43) have no influence. On the other hand the f3 frames from 10th of february (pastebin.com) need to be processed with delay lines, I assume. From what CD did you get them? Thanks! — hexcoder, Feb 11 '19 at 21:52
@hexcoder - are the delay lines used for the interleave used for by the ECC for the AP[4][32] parities? — rcgldr, Feb 11 '19 at 23:07
@rcgldr no for AP usng only inversion. For AQ using interleave. — Vulcanator, Feb 12 '19 at 15:09
@rcgldr I'm actually confused on how you moved from the erasure correction for Q to generating the AQ array. I understand the concept of using erasure decoding to generate the actual in-middle parity bytes for a given input, but I don't see the link to the division done using AQ. Would you mind detailing this part a bit? — Nicolas Noble, Jun 22 '21 at 05:59
@rcgldr I see, thanks. I fail to see the math behind this however. I'm used to Reed Solomon with the division of the message by the generator, and using the reminder of the division as the parity bytes. Is there a different method I'm not aware of? I'm just curious about placing all my ducks in a row about understanding Reed Solomon in general. — Nicolas Noble, Jun 22 '21 at 13:32
@NicolasNoble - as mentioned in my answer, if an RS decoder handles erasures, then there is no need for a proper encoder, as encoding can be done by setting the parity bytes to zero, marking them as erasures, then correcting the erasures to generate the parities. This is how some DAT tape drives implemented encoding (no actual encode, just using erasure correction, to reduce gate count in hardware). — rcgldr, Jun 22 '21 at 18:19
@rcgldr I see, thanks. I now know what I need to research. My erasure correction is a lot more complex than what you've shown here, as it's tied within the error detection and correction. Do you have any literature about how to do erasure correction only? — Nicolas Noble, Jun 22 '21 at 20:19
@rcgldr I think it just clicked for me: these AP and AQ tables are basically encoding the message's parity bytes with basically a monomial of coefficient 1 for each possible position. This gives you a matrix you can use to encode any further message as they are used as multiplicative identity. I think I see how this encoding works. Thanks for the follow ups :) — Nicolas Noble, Jun 22 '21 at 21:22
@NicolasNoble - I deleted some prior comments. Link to example RS(2^8) interactive demo program [eccdemo8](https://github.com/jeffareid/eccdemo8), which is an erasure + error correcting code, that I used with just erasures to generate AQ. Erasure only code is typically block oriented, used to correct erased blocks of data instead of bytes, and ends up a bit more complicated, involving several steps to generate a correcting matrix used (matrix multiply) with non-erased blocks to regenerate erased blocks. — rcgldr, Jun 23 '21 at 00:50

CDROM, F2 Reed-Solomon P Q codes in CIRC encoder

1 Answers1