Swift and golang encoding base64 gives different results

Question

I'm encoding an image with swift using

let base64String = data!.base64EncodedStringWithOptions(NSDataBase64EncodingOptions(rawValue: 0))

the first 222 character of encoded string is /9j/4AAQSkZJRgABAQAASABIAAD/4QBYRXhpZgAATU0AKgAAAAgAAgESAAMAAAABAAEAAIdpAAQAAAABAAAAJgAAAAAAA6ABAAMAAAABAAEAAKACAAQAAAABAAABLKADAAQAAAABAAABLAAAAAD/7QA4UGhvdG9zaG9wIDMuMAA4QklNBAQAAAAAAAA4QklNBCUAAAAAABDUHYzZjwCyBOmACZjs+EJ...

I encoded the same image with golang using

sEnc := base64.StdEncoding.EncodeToString(imageData)

the first 224 character of encoded string is /9j/2wCEAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDIBCQkJDAsMGA0NGDIhHCEyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMv/AABEIASwBLAMBIgACEQEDEQH/xAGiAAABBQEBAQEB...

When I decode the image in swift using

let decodedData = NSData(base64EncodedString: base64String, options: NSDataBase64DecodingOptions(rawValue: 0))
var decodedimage = UIImage(data: decodedData!)

the image looks exactly the same.

When I decode swift generated base64 string in golang with

dec, err := base64.StdEncoding.DecodeString(imageData)

I'm getting error while decoding base64 string: illegal base64 data at input byte 220 which is the + character. But when I decode the golang generated base64 the output correct.

Why are the encoded strings different? How can I fix it?

EDIT:

Swift last 10 bytes eqvr h/9k=
Golang last 10 bytes cvdR//2Q==

EDIT2:

In Golang I'm using ioutil for reading file

b, err := ioutil.ReadFile(fileName)

EDIT3:

Added results for a different test case https://www.dropbox.com/sh/zd1wj7ppjedrx6a/AAAoX15iztTimtav6U_8IF4Ma?dl=0

These two files appear to be _completely_ different. Are you _sure_ you are doing the same encoding to both? — Tim Biegeleisen, Jul 13 '15 at 03:35
@TimBiegeleisen I'm doing the encoding with the codes I provided. The decode is also problematic since they can't decode each others encoded strings. https://golang.org/src/encoding/base64/base64.go?s=1903:1947#L61 — Thellimist, Jul 13 '15 at 03:44
Space in the swift output looks suspicious: `...ACZjs EJ...`. base64 encoded string shouldn't contain spaces. Have a look at http://stackoverflow.com/questions/24769074/base64encodedstringwithoptions-in-swift-fails-with-compile-error — kostya, Jul 13 '15 at 06:02
@kostya I tried the things in that link. but there are still spaces. If options made `NSDataBase64EncodingOptions.Encoding64CharacterLineLength` even `\n` character appears. still it crashes at byte 220 which is the space character — Thellimist, Jul 13 '15 at 07:48
@Thellimist, there are no spaces in the file that you uploaded to dropbox. Why '+' was replaced with space in this string? Have you called URLDecode or similar on it? Also the files that you uploaded to dropbox are valid base64 files but they were created from different files. Please describe the exact steps including all details how you get the error otherwise it is impossible to answer what is wrong. — kostya, Jul 13 '15 at 14:24
@kostya you were right. When I send the data to go server the `+` are converted to spaces. I added `imageData = strings.Replace(imageData, " ", "+", -1)` to go server and the bug got fixed. I'm using https://github.com/nghialv/Net/ for network. Couldn't find anything that could cause this. Also I'm using https://github.com/gin-gonic/gin for routing. Couldn't find anything that can cause this also. — Thellimist, Jul 13 '15 at 15:40
@kostya And still there is the problem where base64 are different but the images are same. I am positive where original images are the same but the encodings are different. Only scenario I can think of is `UIImageJPEGRepresentation(unwrappedImage, 1.0)` function changes the images. — Thellimist, Jul 13 '15 at 15:40
I am not familiar with iOS API but the source files are certainly different. http://stackoverflow.com/questions/17005456/calling-imagewithdatauiimagejpegrepresentation-multiple-times-only-compresses suggests that the function does not return the original image. — kostya, Jul 13 '15 at 15:45
@kostya Yes, I checked. The function does change the image even though the original image is jpeg. — Thellimist, Jul 13 '15 at 15:48
I have the same problem! I use Golang to base64 encode an API key for use in basic auth but the encoding is wrong. I can see the difference too if iI encode it with postman or other tools. — Cyberlurk, Aug 25 '15 at 08:46
@Zahrec the problem was uiimagerepresentation changes the image. The encoding is correct — Thellimist, Aug 25 '15 at 08:57

score 4 · Answer 1 · answered Jul 13 '15 at 03:37

With your Swift-encoded string, this is clearly a JPEG generated in Photoshop (JFIF is the JPEG transport signature):

$ echo '/9j/4AAQSkZJRgABAQAASABIAAD/4QBYRXhpZgAATU0AKgAAAAgAAgESAAMAAAABAAEAAIdpAAQAAAABAAAAJgAAAAAAA6ABAAMAAAABAAEAAKACAAQAAAABAAABLKADAAQAAAABAAABLAAAAAD/7QA4UGhvdG9zaG9wIDMuMAA4QklNBAQAAAAAAAA4QklNBCUAAAAAABDUHYzZjwCyBOmACZjs' | base64 -D | hexdump -C
00000000  ff d8 ff e0 00 10 4a 46  49 46 00 01 01 00 00 48  |......JFIF.....H|
00000010  00 48 00 00 ff e1 00 58  45 78 69 66 00 00 4d 4d  |.H.....XExif..MM|
00000020  00 2a 00 00 00 08 00 02  01 12 00 03 00 00 00 01  |.*..............|
00000030  00 01 00 00 87 69 00 04  00 00 00 01 00 00 00 26  |.....i.........&|
00000040  00 00 00 00 00 03 a0 01  00 03 00 00 00 01 00 01  |................|
00000050  00 00 a0 02 00 04 00 00  00 01 00 00 01 2c a0 03  |.............,..|
00000060  00 04 00 00 00 01 00 00  01 2c 00 00 00 00 ff ed  |.........,......|
00000070  00 38 50 68 6f 74 6f 73  68 6f 70 20 33 2e 30 00  |.8Photoshop 3.0.|
00000080  38 42 49 4d 04 04 00 00  00 00 00 00 38 42 49 4d  |8BIM........8BIM|
00000090  04 25 00 00 00 00 00 10  d4 1d 8c d9 8f 00 b2 04  |.%..............|
000000a0  e9 80 09 98 ec                                    |.....|
000000a5

With your Golang-generated string, nothing of the sort:

$ echo '/9j/2wCEAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDIBCQkJDAsMGA0NGDIhHCEyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMv/AABEIASwBLAMBIgACEQEDEQH/xAGiAAABBQEBAQEB' | base64 -D | hexdump -C
00000000  ff d8 ff db 00 84 00 08  06 06 07 06 05 08 07 07  |................|
00000010  07 09 09 08 0a 0c 14 0d  0c 0b 0b 0c 19 12 13 0f  |................|
00000020  14 1d 1a 1f 1e 1d 1a 1c  1c 20 24 2e 27 20 22 2c  |......... $.' ",|
00000030  23 1c 1c 28 37 29 2c 30  31 34 34 34 1f 27 39 3d  |#..(7),01444.'9=|
00000040  38 32 3c 2e 33 34 32 01  09 09 09 0c 0b 0c 18 0d  |82<.342.........|
00000050  0d 18 32 21 1c 21 32 32  32 32 32 32 32 32 32 32  |..2!.!2222222222|
00000060  32 32 32 32 32 32 32 32  32 32 32 32 32 32 32 32  |2222222222222222|
*
00000080  32 32 32 32 32 32 32 32  ff c0 00 11 08 01 2c 01  |22222222......,.|
00000090  2c 03 01 22 00 02 11 01  03 11 01 ff c4 01 a2 00  |,.."............|
000000a0  00 01 05 01 01 01 01 01                           |........|
000000a8

You need to post your full Go code. It looks like that's the code that's at fault. That run of 0x32 is very suspicious.

I tried your style for another image which is JPG. http://codepad.org/8N4l63JF . Since they don't encode the same way I can't decode it either. — Thellimist, Jul 13 '15 at 04:10

Swift and golang encoding base64 gives different results

1 Answers1