Should I keep these line breaks in Base64 encoding for JSON?

Question

I found this function which encodes a stream as a Base64 string. I'm using this string inside JSON. The problem is that the output of this function has line breaks, which is not acceptable in JSON without escaping it. How do I get around this?

const
  Base64Codes:array[0..63] of char=
    'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/';

function Base64Encode(AStream: TStream): string;
const
  dSize=57*100;//must be multiple of 3
var
  d:array[0..dSize-1] of byte;
  i,l:integer;
begin
  Result:='';
  l:=dSize;
  while l=dSize do
   begin
    l:=AStream.Read(d[0],dSize);
    i:=0;
    while i<l do
     begin
      if i+1=l then
        Result:=Result+
          Base64Codes[  d[i  ] shr  2]+
          Base64Codes[((d[i  ] and $3) shl 4)]+
          '=='
      else if i+2=l then
        Result:=Result+
          Base64Codes[  d[i  ] shr  2]+
          Base64Codes[((d[i  ] and $3) shl 4) or (d[i+1] shr 4)]+
          Base64Codes[((d[i+1] and $F) shl 2)]+
          '='
      else
        Result:=Result+
          Base64Codes[  d[i  ] shr  2]+
          Base64Codes[((d[i  ] and $3) shl 4) or (d[i+1] shr 4)]+
          Base64Codes[((d[i+1] and $F) shl 2) or (d[i+2] shr 6)]+
          Base64Codes[  d[i+2] and $3F];
      inc(i,3);
      if ((i mod 57)=0) then Result:=Result+#13#10;
     end;
   end;
end;

Of course all line breaks need to be escaped for JSON, but the question is what to do with these line breaks... Should I escape them and keep them in the encoded string, or should I discard them? I'm not sure if it's a relevant part of Base64, or if this particular piece of code is putting line breaks just to make it easier to read.

Looks like you found the function here: http://stackoverflow.com/questions/14942269/how-to-encode-base64-in-delphi-6/14942464#14942464 — Stijn Sanders, Jul 03 '13 at 15:52
There's a deeper problem. Perhaps it's that your JSON serializer doesn't escape line breaks? Or perhaps line breaks get encoded correctly when serializing, but deserializing doesn't work? Or maybe whoever receives the string doesn't recognize line breaks in Base46-encoded strings? — Rob Kennedy, Jul 03 '13 at 15:58
Wouldn't it be simpler to use on of the base64 implementations that ship with Delphi? — David Heffernan, Jul 03 '13 at 16:02
@David the `EncdDecd` unit was introduced in Delphi 6 I believe - I try to always make things support as many versions as possible. — Jerry Dodge, Jul 03 '13 at 16:04
@Rob I understand there could be different ways to approach this but since I'm still fairly new to JSON and Base64 I'm hoping someone with experience can tell me the recommended approach. For example, can a single line in JSON support binary Base64 data of a large file? — Jerry Dodge, Jul 03 '13 at 16:09
@JerryDodge Your code runs on Delphi 5 through to XE4? Really? You've tested it on all those versions. — David Heffernan, Jul 03 '13 at 16:12
@David no I haven't, but if I give some code to someone else using an older version, it should at least not depend on things from later versions. That's just how I work. — Jerry Dodge, Jul 03 '13 at 16:14
The recommended approach depends on what your requirements are. Right now, the only requirement is the one implied by your claim that a string with line breaks in it is "not acceptable in JSON." I wonder how you arrived at that conclusion. Although it's true that a quoted value in JSON may not have line breaks, it's false that JSON cannot represent string values with line breaks in it. Your JSON serialization library should account for that automatically by escaping line-break characters. — Rob Kennedy, Jul 03 '13 at 16:25
I updated my question to be a little more clear now that I know what the problem was — Jerry Dodge, Jul 04 '13 at 02:51
You haven't really made the question more clear; you've merely added another question, which is a question I already told you is one that cannot be answered without knowing your requirements. What does the recipient of this data expect to see with regard to line breaks? — Rob Kennedy, Jul 04 '13 at 04:56
I agree entirely with Rob. The fact that you have accepted Stijn's answer suggests that you are hiding from the real problem and pushing it out of site. Any real JSON serialization code will escape correctly. Which means that you aren't using real JSON serialization code, or are using it incorrectly. The fact that you want to roll your own base64 makes me think that you have perhaps written your own JSON serialization. — David Heffernan, Jul 04 '13 at 10:05
If Stijn's answer is indeed the answer to your question, then the question should be "how do I suppress emitting of line breaks in this code?" The question you asked appears to be somewhat different. If you do want to get to the bottom of this, you'll need to explain how you are turning base64 strings into JSON. Can you do that? One final comment, the code you are using is really dire. Every single character that is emitted invokes a heap reallocation. Still, you can at least rest easy that the code will run on Delphi 5. — David Heffernan, Jul 04 '13 at 10:07

score 6 · Accepted Answer · answered Jul 03 '13 at 15:48

6

Remove the line with

if ((i mod 57)=0) then Result:=Result+#13#10;

answered Jul 03 '13 at 15:48

Stijn Sanders

35,982
11
45
67

Should I keep these line breaks in Base64 encoding for JSON?

1 Answers1