2

I'm having issues getting boto3 installed on EMR. Here is the bootstrap script I'm currently using:

#!/bin/bash
# Install Python 3 kernel
sudo yum install python3
sudo yum install python3-pip
sudo pip3 install -U boto3

I checked the logs and it says that boto3 was successfully installed, but when I try to import it in a Jupyter notebook I get ModuleNotFoundError: No module named 'boto3'

Josh Jacobson
  • 91
  • 1
  • 4
  • I think bootstrap script is a cleaner approach for installing 3rd party dependencies. The issue you're facing is most likely due to jupyter using a different kernel. See if this helps: https://stackoverflow.com/a/57408712/4245859 – Bitswazsky May 25 '20 at 16:14

1 Answers1

4

Figured it out. Launch a notebook with the pyspark kernel and run this:

sc.install_pypi_package(“boto3”)
Josh Jacobson
  • 91
  • 1
  • 4