0

In code that is running inside a processing step I would like to find details about pipeline execution that the step is part of.

From the step (processing step) I have access to:

  1. Job name
  2. Job ARN

How to find pipeline execution (or it's ARN) based on that?

I explored following things:

One can using boto3 SageMaker client:

  1. List all pipelines
  2. For each pipeline list all executions
  3. For each execution list all steps
  4. Filter the steps to find the execution However, I think there should be a better way?

I tried to explore Lineage API, but it seems that you can't use job ARN as a starting ARN.

Marcin Mycek
  • 1
  • Heya Marcin, welcome to Stackoverflow, did you check this out? https://stackoverflow.com/questions/72721543/how-to-get-sagemaker-pipeline-step-tuning-job-step-information-by-python – Mrk Fldig Feb 02 '23 at 16:10
  • Let me know if that answers it as we should close this as a duplicate :-) – Mrk Fldig Feb 02 '23 at 16:11

0 Answers0