How to specify hyperparameter configuration to enable HPO in Amazon Personalize?

Question

Do we need to specify the hyperparameters both in algorithmHyperParameters and algorithmHyperParameterRanges? If yes, then should we specify a single value (string as per documentation) in algorithmHyperParameters, but a range (integer in case of integer-valued hyperparameter) in algorithmHyperParameterRanges?

For example: Similar-Items recipe has an integer-valued hyperparameter item_id_hidden_dimension. If I use the following solution_config, where item_id_hidden_dimension is specified only in algorithmHyperParameterRanges and not in algorithmHyperParameters, I get the error:

An error occurred (InvalidInputException) when calling the CreateSolution operation: Provide a hyperparameter that is used in the algorithm: arn:aws:personalize:::algorithm/aws-similar-items

"solution_config": {
                        "algorithmHyperParameters": {},
                        "hpoConfig": {
                          "algorithmHyperParameterRanges": {
                              "integerHyperParameterRanges": [
                                  {
                                      "name": "item_id_hidden_dimension",
                                      "minValue": 30,
                                      "maxValue": 200
                                  }
                              ],
                            "categoricalHyperParameterRanges": [],
                            "continuousHyperParameterRanges": []
                          },
        "hpoResourceConfig": {
                              "maxNumberOfTrainingJobs": "4",
                              "maxParallelTrainingJobs": "2"
                          }
                        }
    }

But if I use the following solution_config, where item_id_hidden_dimension is specified both in algorithmHyperParameterRanges and in algorithmHyperParameters, I still get the same error:

An error occurred (InvalidInputException) when calling the CreateSolution operation: Provide a hyperparameter that is used in the algorithm: arn:aws:personalize:::algorithm/aws-similar-items

"solution_config": {
                        "algorithmHyperParameters": {
                            "item_id_hidden_dimension": "100"
                        },
                        "hpoConfig": {
                          "algorithmHyperParameterRanges": {
                              "integerHyperParameterRanges": [
                                  {
                                      "name": "item_id_hidden_dimension",
                                      "minValue": 30,
                                      "maxValue": 200
                                  }
                              ],
                            "categoricalHyperParameterRanges": [],
                            "continuousHyperParameterRanges": []
                          },
                            "hpoResourceConfig": {
                              "maxNumberOfTrainingJobs": "4",
                              "maxParallelTrainingJobs": "2"
                          }
                        }
                    }

James J · Accepted Answer · 2022-08-18T16:46:44.607

This is caused by an error in the documentation. The hyperparameter names should be item_id_hidden_dim and item_metadata_hidden_dim (note they are dim and not dimension as the documentation states).

This can be confirmed by calling the DescribeRecipe API to get the algorithmArn for the Similar-Items recipe and then calling the DescribeAlgorithm API to get details on the algorithm.

import boto3
import json

personalize = boto3.client('personalize')

response = personalize.describe_recipe(recipeArn = 'arn:aws:personalize:::recipe/aws-similar-items')
print(json.dumps(response['recipe'], indent=2, default=str))

{
  "name": "aws-similar-items",
  "recipeArn": "arn:aws:personalize:::recipe/aws-similar-items",
  "algorithmArn": "arn:aws:personalize:::algorithm/aws-similar-items",
  "featureTransformationArn": "arn:aws:personalize:::feature-transformation/similar-items",
  "status": "ACTIVE",
  "description": "Predicts items similar to a given item based on co-occurrence of items in the user-item interactions dataset and item metadata in the item dataset.",
  "creationDateTime": "2019-06-10 00:00:00+00:00",
  "recipeType": "RELATED_ITEMS",
  "lastUpdatedDateTime": "2022-08-17 00:25:42.935000+00:00"
}

algo_arn = response['recipe']['algorithmArn']
response = personalize.describe_algorithm(algorithmArn = algo_arn)
print(json.dumps(response['algorithm'], indent=2, default=str))

{
  "name": "aws-similar-items",
  "algorithmArn": "arn:aws:personalize:::algorithm/aws-similar-items",
  "algorithmImage": {
    "name": "Item Similarity"
  },
  "defaultHyperParameters": {
    "item_id_hidden_dim": "100",
    "item_metadata_hidden_dim": "100"
  },
  "defaultHyperParameterRanges": {
    "integerHyperParameterRanges": [
      {
        "name": "item_id_hidden_dim",
        "minValue": 30,
        "maxValue": 200,
        "isTunable": true
      },
      {
        "name": "item_metadata_hidden_dim",
        "minValue": 30,
        "maxValue": 200,
        "isTunable": true
      }
    ],
    "continuousHyperParameterRanges": [],
    "categoricalHyperParameterRanges": []
  },
  "defaultResourceConfig": {
    "maxNumberOfTrainingJobs": "20",
    "maxParallelTrainingJobs": "5"
  },
  "trainingInputMode": "File",
  "creationDateTime": "2019-06-10 00:00:00+00:00",
  "lastUpdatedDateTime": "2022-08-17 00:24:41.307000+00:00"
}

Note the hyperparameter names in the last response above.

We will get this error fixed in the documentation ASAP.

How to specify hyperparameter configuration to enable HPO in Amazon Personalize?

1 Answers1