Decoding adjustmentData in .AAE files of pictures taken by the iPhone

Question

I understant .AAE files are supposed to describe the changes that were applied to a certain picture. However I see that most times (apparently not always) there is such a file even when they seem to carry the same data (which seems to indicate that no change was applied to the picture.

Does anyone know how to decode this adjustmentData value?

Example photo 1:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>adjustmentBaseVersion</key>
    <integer>0</integer>
    <key>adjustmentData</key>
    <data>
    bZBNT8JAEIb/y5wrKaBg9mbwIBdJNNGD8TC003bNfjS7U5CQ/nenW0o4eNt95+t9nzNY
    YiyREdQZLEam8EK6bhjUfb5cZBftU5fcgFosVo8Z+KDJMbL2DtSqz6DywSJ/UIhJmmdw
    GN9bV/lh8b7Tpnzt7J4CKJivnx9yyADb9joD8o9FQxbf6KBHLc+gNcjDdmnQu3eQW1j+
    dJGtGIigvs7/3CaHe0MlKA4dyVZi1q6Og48THtNaSVBrh2YTfDv1/e6SmOrRYhACFZoo
    leMYfuRxmtqWa0EROeBAi9yTqw2BupNp7Nhfh5sLzRFd4V0acTxRzm/FC+YhuOZieEli
    XUpYXenELjmeiG9vK4W3MyFqaNY2Xgz03/0f
    </data>
    <key>adjustmentEditorBundleID</key>
    <string>com.apple.camera</string>
    <key>adjustmentFormatIdentifier</key>
    <string>com.apple.photo</string>
    <key>adjustmentFormatVersion</key>
    <string>1.6</string>
    <key>adjustmentTimestamp</key>
    <date>2020-03-21T17:37:31Z</date>
</dict>
</plist>

Example photo2:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>adjustmentBaseVersion</key>
    <integer>0</integer>
    <key>adjustmentData</key>
    <data>
    bZBNT8JAEIb/y5wrKaBg9mbwIBdJNNGD8TC003bNfjS7U5CQ/nenW0o4eNt95+t9nzNY
    YiyREdQZLEam8EK6bhjUfb5cZBftU5fcgFosVo8Z+KDJMbL2DtSqz6DywSJ/UIhJmmdw
    GN9bV/lh8b7Tpnzt7J4CKJivnx9yyADb9joD8o9FQxbf6KBHLc+gNcjDdmnQu3eQW1j+
    dJGtGIigvs7/3CaHe0MlKA4dyVZi1q6Og48THtNaSVBrh2YTfDv1/e6SmOrRYhACFZoo
    leMYfuRxmtqWa0EROeBAi9yTqw2BupNp7Nhfh5sLzRFd4V0acTxRzm/FC+YhuOZieEli
    XUpYXenELjmeiG9vK4W3MyFqaNY2Xgz03/0f
    </data>
    <key>adjustmentEditorBundleID</key>
    <string>com.apple.camera</string>
    <key>adjustmentFormatIdentifier</key>
    <string>com.apple.photo</string>
    <key>adjustmentFormatVersion</key>
    <string>1.6</string>
    <key>adjustmentTimestamp</key>
    <date>2020-03-25T05:45:21Z</date>
</dict>
</plist>

I looked into: https://www.apple.com/DTDs/PropertyList-1.0.dtd and it says this: "<!ELEMENT data (#PCDATA)> " however when I decode the data field using BASE64 it still shows gibberish... — pjp, Apr 04 '20 at 04:27
I did the same thing and the result was a binary plist file, which `plutil -convert xml1` converted to readable XML. Check again? — duozmo, Jan 27 '21 at 13:07

score 2 · Answer 1 · answered May 30 '21 at 00:10

Unfortunately this is not a full answer yet. It might help some, but there's something more that I don't understand.

The most useful answer I found came from http://rae.tnir.org/archives/2016/02/data-inside-apples-aae-files. He takes the contents of the data attribute and runs it through a base64 decoder. This then gives him a binary plist, which he converts to intelligible XML related to slow motion settings on the photo.

Let's do it in more detail.

Start with his adjustmentData info:

    <key>adjustmentData</key>
    <data>
    YnBsaXN0MDDRAQJac2xvd01vdGlvbtIDBAUXV3JlZ2lvbnNUcmF0ZaEG0QcIWXRpbWVS
    YW5nZdIJCgsUVXN0YXJ0WGR1cmF0aW9u1AwNDg8QERITVWZsYWdzVXZhbHVlWXRpbWVz
    Y2FsZVVlcG9jaBABEQEMEQJYEADUDA0ODxUWEhMQAxEC4SI+AAAACAsWGyMoKi03PEJL
    VFpganBydXh6g4WIAAAAAAAAAQEAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAI0=
    </data>

We're going to base64 decode the data value. Whitespace and line breaks don't seem to matter. Use https://www.opinionatedgeek.com/codecs/base64decoder because we're going to want to save the result as it itself is binary and converters that just show text results won't be able to render it appropriately. Copy the data field (YnBs.....AAI0=), decode it, and download the resulting file. Note that it starts with bplist, indicating it's a binary plist for use in the Apple ecosystem.

Now we need to decode the binary plist. He doesn't explain how to do it, but other sites claim you can just open it up in TextMate and it knows how to display it. Indeed. The other Apple suggestion in a comment above is to run plutil -convert xml1 TheDownloadedFile, and it modifies the file in-place from the binary version to an XML version. (These do seem to work on the Mac I borrowed to test it.) I'm on Windows though, and neither suggestion helps. I was able to find a Notepad++ bplist plugin for Notepad++ however. The end result for me as displayed in Notepad++ is

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>slowMotion</key>
    <dict>
        <key>regions</key>
        <array>
            <dict>
                <key>timeRange</key>
                <dict>
                    <key>start</key>
                    <dict>
                        <key>flags</key>
                        <integer>1</integer>
                        <key>value</key>
                        <integer>268</integer>
                        <key>timescale</key>
                        <integer>600</integer>
                        <key>epoch</key>
                        <integer>0</integer>
                    </dict>
                    <key>duration</key>
                    <dict>
                        <key>flags</key>
                        <integer>3</integer>
                        <key>value</key>
                        <integer>737</integer>
                        <key>timescale</key>
                        <integer>600</integer>
                        <key>epoch</key>
                        <integer>0</integer>
                    </dict>
                </dict>
            </dict>
        </array>
        <key>rate</key>
        <real>0.125000</real>
    </dict>
</dict>
</plist>

See also decoding data from plist file. The writeup linked in a comment there was very helpful as well. His plist is not an AAE image, but the same process works. I would have suspected it would for any plist.

Except yours. And mine. I don't know why.

The adjustmentData data value in this original question isn't decoding to something starting with bplist. The same decoder (https://www.opinionatedgeek.com/codecs/base64decoder) is decoding it to:

mO@@†˜+) 箠I4у൓0´v̾4»þ§J8xڽ誽ŸXb,‘ҙ,F¦Bºӽ¾d쓗܀Z,V籲ϠPˆšpޛWΐ¦(˜Ÿrƀܶ:ࠨG-Ϡ5ɃviлwXþ‘ˆ¾ÿ&‡C%(Ȗb֮ŽҚIPk‡|;˜Тš•~㱚ږkA9@‹“
“헟‡ˑ]ۚq<QͯË䡸塸Ib]JX]ꂮ9žoo+…3!jhԶ^ໟ

I had no better success with any other decoder, or messing with whitespace. And I'm having the same issue with the AAE that led me into this mess myself. I note that both you and me have data with frequent / characters and that the two success stories I have linked do not. They "feel" like different encodings to me, but I am not an encoding expert.

I'm also a bit suspicious of the adjustmentFormatVersion in the original AAE. The working example was way back on 1.1. Ours are 1.4 and 1.6. I wonder if the contents used to be a binary plist but no longer are.

score 2 · Answer 2 · answered Dec 08 '22 at 21:57

The adjustmentData is JSON string which is zlib-compressed then base64 encoded. I didn't figure this out myself, ojchase pointed to the base64 encoding, then I found the rest in https://github.com/neilpa/photohack/issues/4#issuecomment-780872343. https://github.com/RhetTbull/osxphotos/issues/384 tells that in some cases the format is different but currently I don't have any examples to examine.

I used python to decode adjustmentData because I found that it has some relevant libraries:

a standard library, plistlib: I tried to use it but adjustmentData was not seem to be a plist binary at any point of decoding/decompression
https://github.com/avibrazil/NSKeyedUnArchiver: I didn't try it out

Code is tested with Python 3.9.12 on Windows 10. Tested with AAE file from iPhone SE (gen 2), iOS version 16.1.1, but the script contains adjustmentData from pjp to answer the question. Both of the example AAE files contain the same adjustmentData.

import base64
import zlib
import json

# adjustmentData merged into one continuous string without whitespaces, stored as bytestring
adjustmentData = b"bZBNT8JAEIb/y5wrKaBg9mbwIBdJNNGD8TC003bNfjS7U5CQ/nenW0o4eNt95+t9nzNYYiyREdQZLEam8EK6bhjUfb5cZBftU5fcgFosVo8Z+KDJMbL2DtSqz6DywSJ/UIhJmmdwGN9bV/lh8b7Tpnzt7J4CKJivnx9yyADb9joD8o9FQxbf6KBHLc+gNcjDdmnQu3eQW1j+dJGtGIigvs7/3CaHe0MlKA4dyVZi1q6Og48THtNaSVBrh2YTfDv1/e6SmOrRYhACFZooleMYfuRxmtqWa0EROeBAi9yTqw2BupNp7Nhfh5sLzRFd4V0acTxRzm/FC+YhuOZieEliXUpYXenELjmeiG9vK4W3MyFqaNY2Xgz03/0f"

# Decode base64 encoded bytes. This will result some not-readable binary data
adjustmentData_decoded = base64.b64decode(adjustmentData)

# Decompress
adjustmentData_decompressed = zlib.decompress(adjustmentData_decoded, -zlib.MAX_WBITS)
adjustmentData_json = json.loads(adjustmentData_decompressed)
    
print(json.dumps(adjustmentData_json, indent=2))

The output is:

{
  "metadata": {
    "masterHeight": 4032,
    "masterWidth": 2268,
    "orientation": 6
  },
  "formatVersion": 1,
  "versionInfo": {
    "buildNumber": "17D50",
    "appVersion": "",
    "schemaRevision": 0,
    "platform": "iOS"
  },
  "adjustments": [
    {
      "formatVersion": 1,
      "enabled": true,
      "settings": {
        "yaw": 0,
        "originalCrop": true,
        "xOrigin": 0,
        "smart": false,
        "width": 4032,
        "yOrigin": 378,
        "straightenAngle": 0,
        "auto": false,
        "height": 2268,
        "constraintHeight": 0,
        "constraintWidth": 0,
        "pitch": 0
      },
      "identifier": "Crop",
      "formatIdentifier": "com.apple.photo"
    }
  ]
}

Additional experiences:

If we don't edit photos we can still have AAE files. As I experience this happens if we use 16:9 aspect ratio. Then iPhone stores the original 4:3 photo and applies the aspect ratio via the AAE file. This makes possible to modify the ratio and orientation of the photo in the editor later.
If I connect the iPhone via USB to my PC then I browse the photos via Windows File Explorer, for each photo I see:
- two almost identical AAE files are stored. The only difference is that one contains adjustmentRenderTypes xml PropertyList key
- two HEIC files: IMG_<number>.HEIC and IMG_E<number>.HEIC. The first is the original 4:3 image, the second seems to have the AAE edits applied.

Screenshot of File Explorer

score 1 · Answer 3 · answered Oct 27 '22 at 19:31

adjustmentData is likely the application-specific sidecar data mentioned in Apple's PhotoKit documentation:

This object provides an application-defined “recipe” you can use to reconstruct the edit.

When an asset is edited, Photos stores a PHAdjustmentData object that is provided by the app (or by the extension) that edited the asset. This object provides whatever information is necessary to reconstruct the edited asset using the previous version of the asset’s content. When a user attempts to edit an asset with your extension, Photos calls this block to learn whether your extension can handle the asset’s past adjustments.

Since this is application-specific data, and is now highly-encoded (no longer just a binary plist), I'd guess it's some kind of archived NSData object (sample custom implementation). However, without knowing what the bytes map to (what the Photos.app encoders do, any enums required), we're probably unable to further decode these. (I have AAE files with version 1.8 on my system, for reference.)

As of iOS 8 (2014), these were probably easily-decodable. If we ever do find or reverse engineer some reference code (from Photos.app or otherwise), it should be easy to encode the info into a .swift shell script.

Decoding adjustmentData in .AAE files of pictures taken by the iPhone

3 Answers3