Questions tagged [amazon-sagemaker]

Amazon SageMaker is a fully-managed AWS service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale.

Amazon SageMaker is Amazon's official system for developing machine learning systems in the cloud. It is based on Jupyter notebooks and has the ability to talk with other AWS services in order to explore data, automate processes and conduct various kinds of analysis.

It provides ready-to-use frameworks and algorithms to deal with different use cases and gives the ability to develop fully custom models.

One of the most important capabilities of SageMaker is the ability to deploy models in fully scalable and secure environments.

The only prerequisites for using Amazon SageMaker are an AWS account and an IAM (Identity and Access Management) admin user.


Tagging Recommendation:

Use the tag for all SageMaker-related questions. If it is a question about must also be redirected to this tag as a subset.

For all other SageMaker sub-services, if there is not yet a dedicated tag, it is good to redirect them to SageMaker only if they are closely related.


FAQ:

There are some recurring and important questions that one is faced with when starting to use SageMaker that should not be duplicated:


References


Free SageMaker Programming Books


SageMaker Online Courses


SageMaker Video Tutorials


Official Logo

SageMaker logo

2832 questions
0
votes
0 answers

AWS Sage Maker doesn't show Job Definitions

Out of nowhere, Sage Maker stopped showing Notebook Job Definitions. Access is correct, I cleaned the browser, and there are jobs running from those job definitions, but Sage Maker is not showing the list - what I can do?
0
votes
0 answers

Sagemaker Spark Session - Import aws athena table

I was hoping to import an AWS Athena Database table within a spark session. I have previously setup Notebook instances and used the pyathena library to connect to the athena table and then run Pandas dataframes. However I would like to use Pyspark…
0
votes
1 answer

Update all Sagemaker Jupyter Notebooks with latest git repository changes

I have multiple Sagemaker notebooks created for a training class. I need to make a change in the notebooks before the new class starts. So far, the only way I see to do this is to update the repository with changes and then go into each notebook…
Zak Keirn
  • 807
  • 13
  • 22
0
votes
0 answers

Finetune Llama 2-7B using PDF document in Sagemaker

Can someone give me ideas on how to fine-tune the Llama 2-7B model in Sagemaker using multiple PDF documents, please? For now, I used pypdf and extracted the text from PDF but I don't know how to proceed after this. Even in the AWS documentation,…
0
votes
0 answers

Error: Descriptors cannot not be created directly

Suddently i got this error , even i have downgrade protobuf==3.19.6 still i got this below error TypeError: Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated…
0
votes
0 answers

what's the best way to evaluate Helsinki model with the Huggingface Trainer

I am trying to finetune Helsinki model with the Huggingface trainer based on the documentation found under link 1 and link 2 for a translation task from German to French. I am using cross validation for performance check. I am finetuning the model…
0
votes
0 answers

Converting Python notebook base64 image rendering to JavaScript?

I have the below Python code which renders an image from a SageMaker API endpoint in a Jupyter Notebook. This works fine. I am converting this to a web app, and I'm struggling to decode the image correctly in javascript. My web app gets the image…
dtbaker
  • 4,679
  • 6
  • 28
  • 30
0
votes
1 answer

AWS Lambda & Papermill: Unable to import module 'lambda_function': No module named 'rpds.rpds'

I'm trying to import papermill inside an AWS python lambda function. To to this, I first ran pip3 install papermill -t Desktop/python locally, zipped the python file and uploaded this as a lambda layer. I am now getting this error: { …
0
votes
1 answer

SageMaker complains that /opt/ml/model does not appear to have a file named config.json

I am using a huggingface model alongside a custom pipeline to deploy my model onto SageMaker, my model.tar.gz structure looks like below: ├── added_tokens.json ├── code │ ├── inference.py │ ├── pipeline.py │ └── requirements.txt ├──…
Baiqing
  • 1,223
  • 2
  • 9
  • 21
0
votes
1 answer

Sagemaker AWS llama2 endpoint inference

I am calling the inference endpoint of jumpstart-llama2-foundational-model on AWS sagemaker but it gives me the error below: Error raised by inference endpoint: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received…
ddwivedy
  • 23
  • 1
  • 4
0
votes
1 answer

Sagemaker endpoint Invocation fails from external application

I followed the steps below: Deployed a AWS LLAMA2 foundational model using sagemaker. Added Sagemaker full access policy to my IAM user Ran aws configure for the user above from my local terminal Tried to invoke the sagemaker endpoint from my…
ddwivedy
  • 23
  • 1
  • 4
0
votes
1 answer

How do fix GPT2 Tokenizer error in Langchain map_reduce (LLama2)?

I'm using AWS Sagemaker Jumpstart model for Llama2 13b: meta-textgeneration-llama-2-13b-f On running a Langchain summarize chain with chain_type="map_reduce" I get the below error. I do not have access to https://huggingface.co from my environment.…
apprunner2186
  • 217
  • 1
  • 6
0
votes
0 answers

Sagemaker does not recognize Torch

I'm moving my code from Google Colab to AWS Sagemaker. I do import torch just after %pip install torch and yet I get ModuleNotFoundError: No module named 'torch'! I can remember I had used torch on Sagemaker before and I did not get such error! P.S.…
soroush
  • 45
  • 8
0
votes
1 answer

Moving file from s3 to Sagemaker Instance

So the problem is the following: I have several .zip files of relatively big size (20-30GB) that I have uploaded to s3 Bucket. Since there is quite complex procedure for unzipping them (because in my case the standard way with using Lambda will not…
Keithx
  • 2,994
  • 15
  • 42
  • 71
0
votes
2 answers

Cannot find private key.pem after using openssl

[problem] I generated a private key with the following cmd openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.pem –nocrypt pkcs8: Use -help for summary. Generating RSA private key, 2048 bit long modulus (2…
Mapotofu
  • 268
  • 2
  • 4
  • 15