1

based on the aws documentation/example provided here , https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-pipelines/tabular/abalone_build_train_deploy/sagemaker-pipelines-preprocess-train-evaluate-batch-transform.html#Define-a-Transform-Step-to-Perform-Batch-Transformation, a model is created and a batch transform inference can be run on the trained model. it works for this example but if we need a custom inference script, How do we include a custom inference script in the model or model package before we run the batch transformation ?

from sagemaker.transformer import Transformer
from sagemaker.inputs import TransformInput
from sagemaker.workflow.steps import TransformStep

transformer = Transformer(
    model_name=step_create_model.properties.ModelName,
    instance_type="ml.m5.xlarge",
    instance_count=1,
    output_path=f"s3://{default_bucket}/AbaloneTransform",
)

step_transform = TransformStep(
    name="AbaloneTransform", transformer=transformer, inputs=TransformInput(data=batch_data)
)
arve
  • 569
  • 2
  • 10
  • 27

1 Answers1

2

You need a "model repacking step".

From the Amazon SageMaker Workflows FAQ:

Model repacking happens when the pipeline needs to include a custom script in the compressed model file (model.tar.gz) to be uploaded to Amazon S3 and used to deploy a model to a SageMaker endpoint. When SageMaker pipeline trains a model and registers it to the model registry, it introduces a repack step if the trained model output from the training job needs to include a custom inference script. The repack step uncompresses the model, adds a new script, and recompresses the model. Running the pipeline adds the repack step as a training job.

Basically, you can redefine a sagemaker Model by calling the training output as model_data and pass the inference script as entry_point.

Then sequentially, after training the model, redefine the model by changing the entry_point and on the latter you can use the transformer.

This is an example flow taken from a tested code:

my_model = Model(
    image_uri=your_img_uri,
    model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts,
    role=role,
    entry_point='inference_script.py',
    name="your_inference_step_name"
)

step_create_model = ModelStep(
    name="YourInfName",
    step_args=my_model.create(instance_type="ml.m5.xlarge")
)

transformer = Transformer(
    model_name=step_create_model.properties.ModelName,
    instance_count=your_instance_count,
    instance_type=your_instance_type,
    output_path=your_path
)

Of course, instead of the generic Model, you can directly use the one that best suits your requirements (e.g. PyTorchModel etc...).

Giuseppe La Gualano
  • 1,491
  • 1
  • 4
  • 24
  • thanks. i did something similar, i think the only thing i'm missing is, this line => entry_point='inference_script.py'. just including this line will package the model with my inference_scripty.py? i assume this needs to be in the certain folder, or is there a way i can provide path to this file? – arve Dec 31 '22 at 22:08
  • 1
    When you want to train the model, you clearly need to specify the training script. If this does not include inference methods within it (generically this does not happen on fully custom algorithms), then you need to repack the model by specifying the model to be loaded (model_data) previously trained and an inference script in the exact same way you specify the training script in the previous step. So you can specify source_dir and/or code_location for example. – Giuseppe La Gualano Jan 01 '23 at 01:34