0

I'm trying to use AWS Personalize. After creating dataset and batch inference, I am updating the user-item-interactions with personalize.putEvents (using Javascript SDK, docs)

Snippet:

const awsOpts = { apiVersion, accessKeyId, secretAccessKey, region }
const pEvents = new AWS.PersonalizeEvents(awsOpts)
// ...
const params = {
trackingId, userId, sessionId,
eventList: [{ 
    eventId: (+sentAt) + "",
    sentAt, 
    eventType,
    properties: { itemId }
}]
}
pEvents.putEvents(params, (err, data) => err ? reject(err) : resolve(data))

The events seem to be registered. No errors. After that when I create another batch-inference, I would expect that the new user-items would not appear in the recommendations anymore. But the recommendations in the next batch-inference are unchanged. Am I doing something wrong or am I misunderstanding the putEvents-API-call?

Schema for reference:

{
 "type": "record",
 "name": "Interactions",
 "namespace": "com.amazonaws.personalize.schema",
 "fields": [
    {
        "name": "USER_ID",
        "type": "string"
    },
    {
        "name": "ITEM_ID",
        "type": "string"
    },
    {
        "name": "EVENT_TYPE",
        "type": "string"
    },
    {
        "name": "TIMESTAMP",
        "type": "long"
    }
 ],
 "version": "1.0"
}

One thing seems a bit strange: Cloud watch reports that the lambda was executed twice despite no errors nor timeout exceeded (timeout is set to 10s, and the lambda takes less than 2s). Also Retry attempts is set to 2.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
tokosh
  • 1,772
  • 3
  • 20
  • 37
  • 1
    Isn't it expected behaviour? I would assume, that recommendations are going to change after next retraining with creating a new solution version. It's very similar case to this one: https://stackoverflow.com/questions/57189795/getting-recommendations-for-new-users-with-aws-personalize – PatrykMilewski Jun 08 '20 at 08:46
  • Oh, man. This is how it works, then. Thanks for the comment. It's greatly appreciated. Not dealing with this anymore, but might check it out again. – tokosh Jun 08 '20 at 14:51
  • 1
    I'm sorry but I was wrong, PutEvents API call does change the recommendations. I just tested it and my recommendations were slightly different. I didn't had to retrain the solution. The recommendations changes were instant. – PatrykMilewski Jul 02 '20 at 09:52

3 Answers3

1

User-item recommendations, at least from the runtime API should change without retraining after some events (that is what it is for and that is how we use it), though perhaps (it is something to check) you need to use the runtime API to see the new recommendations.

(edited to clarify that that the API is called the "runtime" API - thanks to PatrykMilewski for seeking the clarification - and that I am by no means certain that the particular API used is the important thing - I do know that when using the runtime API, the events do have an effect though).

D.J.Duff
  • 1,565
  • 9
  • 14
  • Does AWS mention anywhere that there is realtime and non-realtime API? I can't find anything about it. In my opinion there is simply GetRecommendations API call and that's it. Can you please share any sources confirming that: "User-item recommendations, at least from the real-time should change without retraining after some events"? Thanks! – PatrykMilewski Jun 16 '20 at 15:24
  • Dear PatrykMilewski, it's called the "Amazon Personalize Runtime" API ( https://docs.aws.amazon.com/personalize/latest/dg/API_Operations_Amazon_Personalize_Runtime.html ) and only has GetRecommendations and GetPersonalizedRanking calls. The questioner is using batch inference from the "Amazon Personalize" API which may or may not make a difference but I sought to hilight it. – D.J.Duff Jun 18 '20 at 09:01
1

@D.J.Duff (sadly I can't comment)

Are you sure the events added using PutEvent API are considered without retraining ? I have been through the AWS Personalize Doc looking for exactly that and it looked to me that you need to retrain to get those events included and for the runtime api to be able to consider them. Could you point to me where you saw that they are considered without having to retrain ? Thanks

Marc
  • 64
  • 4
  • PatrykMilewski pointed this out in the comments. Not seldom I just fail to read/see the important piece in the doc. – tokosh Jun 18 '20 at 16:04
1

I have tried out the very same usecase with personalize using boto3 sdk. Yes!! Put events can be used to update user-item-interactions. No need to retrain the model after put events. Looks like aws personalize solutions are made compatible to handle the data updated with put events. When I run the next batch inference, after updating interactions with put events API, I could see the change in the recommendations. I verified this with personalized ranking recipe. I could see the recent interacted items getting re-ranked according users recent interactions.

user1506729
  • 11
  • 6
  • Interesting. It did not work for me. Might need to check again... – tokosh Jun 21 '20 at 16:14
  • 1
    Yes, I can confirm that. I was not using Batch recommendations, but normal real-time ones, but calling PutEvents API changes the recommendations and you don't have to retrain the whole solution. – PatrykMilewski Jul 02 '20 at 09:49
  • In my case, using in Java, seems like PutEvents is not failed since there's no exceptions. but how can I confirm the event is done successfully? I reloaded interaction file in S3, but it's not updated. – zeide Sep 08 '20 at 05:33
  • 1
    Just for interested folks coming here to understand, model has to be retrained to include new users/items, but can update recommendations for existing ones already part of the previous solution version. New users will see popular items until they have enough interactions. Details [here](https://docs.aws.amazon.com/personalize/latest/dg/recording-events.html) – Santhosh Oct 23 '20 at 18:46