How to trigger a specific ipynb file from aws sagemaker notebook instance

Question

I am trying to trigger a sagemaker notebook instance thru lambda function once file is arrived in S3. I had created a sagemaker notebook instance inside i have a folder and multiple ipynb files. I need to run specific ipynb file.

NotebookInstanceName = 'dev-process' NotebookPath= '/home/ec2-user/SageMaker/data-pipeline/entire_process.ipynb'

Also I am trying to trigger first the notebook instance if it is in inactive or failed state.

Answer 1

Why do you want to trigger an notebook (.ipynb) file in a notebook instance? If you want to trigger notebooks you can look at using SageMaker Notebook Jobs.

You could also look at converting your notebook into a Training/Processing Script that can run on a SageMaker Training / Processing Job.

Answer 2

you can utilize sagemaker lifecycle configuration and specify path of your ipynb as below:

#!/bin/bash
set -x

apt update -y

sudo -u ec2-user -i <<'EOF'
# PARAMETERS
ENVIRONMENT=python3
NOTEBOOK_FILE=/home/ec2-user/SageMaker/yash-folder/Example_Notebook.ipynb
echo "Activating conda env"
source /home/ec2-user/anaconda3/bin/activate "$ENVIRONMENT"
echo "Starting notebook"
jupyter nbconvert "$NOTEBOOK_FILE" --ExecutePreprocessor.kernel_name=python --execute --to notebook
echo "Dectivating conda env"
source /home/ec2-user/anaconda3/bin/deactivate

and your lambda function will be able to trigger your sagemaker notebook instance using below code:

import boto3
import logging
def lambda_handler(event, context):
    client = boto3.client('sagemaker')
    client.start_notebook_instance(NotebookInstanceName='yashaswi-instance')
    return 0

you can follow this post for your reference https://medium.com/@yashaswi.kakumanu/struggling-to-schedule-sagemaker-notebooks-ebd7441cc80f