2

Am working on a project of consumer behaviour analysis on websites and predict the malicious activity of users in real-time. Click data is being collected for each click made by users.

Am using multiple AWS services like kinesis stream, Lambda and sagemaker. I have created an autoencoder model and deployed it as sagemaker endpoint which will be invoked using lambda when it receives new click data from the website through Kinesis stream.

Since sagemaker endpoint contains the only model but click data which lambda function receives is raw data with URLs, texts and date. How can I pass raw data into required preprocessing steps and send processed data to sagemaker endpoint in the required format?

Example of raw data:-

{'URL':'www.amazon.com.au/ref=nav_logo', 'Text':'Home', 'Information':'Computers'}

Darshan M
  • 23
  • 4

1 Answers1

1

You can use Sagemaker inference Pipeline. You need to create preprocessing script comprising of your preprocessing steps and create a Pipeline including Preprocess and model. Deploy pipeline to an endpoint for real time inference.

Reference: https://aws.amazon.com/blogs/machine-learning/preprocess-input-data-before-making-predictions-using-amazon-sagemaker-inference-pipelines-and-scikit-learn/

https://github.com/aws/amazon-sagemaker-examples/blob/master/sagemaker-python-sdk/scikit_learn_inference_pipeline/Inference%20Pipeline%20with%20Scikit-learn%20and%20Linear%20Learner.ipynb

KPH
  • 64
  • 8