PyCharm intellisense for boto3

Question

having problems seeing full intellisense (code completion) options in PyCharm. working with python 3.4 on Windows. the suggests are partially working:

import boto3
s = boto3.Session()  (boto3. will bring up list of methods/params of object boto3)
ec2 = s.resource('ec2') (resource is a suggested method!)
ec2. <<<< this brings up nothing. For some reason PyCharm cant detect that ec2 object would have

while I can work off documentation alone, intellisense is just such a nice feature to have!

ive had similar problems getting it to complete lxml syntax but I thought that was because I had to install lxml directly as a binary (too many hoops to jump through on windows to install it via pip)

Anyone else encounter similar problems?

While we are here, I see a lot of different libraries around using awscli with python: boto, boto3, troposphere etc. What are some advantages of using one over the other. Amazon states that boto3 is the prefered method over boto but for my usage of starting/stopping ec2 instances could be easily done with older boto.

I am using 4.5.2. Wouldn't think it would matter that much but I'll give it a try. — Alex C, Jul 22 '15 at 07:02

score 30 · Answer 1 · edited Jun 15 '19 at 20:19

30

I was frustrated with the same issue. So I decided to parse boto3 documentation and generate wrapper classes from the documentation. Here is the link to project https://github.com/gehadshaat/pyboto3

To install it

pip install pyboto3

To use it

import boto3
s3 = boto3.client('s3')
""" :type : pyboto3.s3 """
# s3. -> will give you autocomplete for s3 methods in pycharm

Make sure that you first:

Install pyboto3 -> pip install pyboto3 | pip3.x install pyboto3
Check your interpreter settings and verify that you see pyboto3 on the list
Do a File -> Invalidate Caches/Restart

After Pycharm restarts you should see intellisense working in your favor and all of the available methods for the service (in the case above s3) you are trying to use available to you!

edited Jun 15 '19 at 20:19

lowlow20

3
3

answered Jul 29 '16 at 02:19

gehad

1,205
3
12
17

1

Thanks for that project, just pip installed it right now and it works like a charm – Patrick Laxton Feb 02 '17 at 07:01
What about for the s3.Bucket? """ :type : pyboto3.s3.Bucket """ doesn't seem to work – Peter Kahn Apr 27 '17 at 21:22
1

@PeterKahn resources are not supported. only client for now. – gehad Apr 28 '17 at 23:30
Thanks. Something is better than nothing. – Peter Kahn Apr 28 '17 at 23:52
1

does not work at my end, using pycharm community 2018.3 – Jeff Nov 30 '18 at 00:29
Looks like it just works for some items. I can get it working for S3 2018.3 but not for instance sagemaker. – Jack Feb 04 '19 at 10:54
pyboto3 supports Python 2.7 only. – avalanchy Mar 19 '21 at 06:30

score 12 · Accepted Answer · answered Jul 28 '15 at 16:22

12

This is happening because all of the methods on the boto3 clients and resource objects are generated at runtime based on a JSON file that describes what operations the service supports. Pycharm would have to have specific knowledge about this process in order to auto complete method names.

For your second question, boto3 is the official AWS SDK for Python. One of the main advantages of boto3 is that because of this JSON model driven process that describes the AWS APIs, most new service features only require a simple model update. This means API updates happen in a quick, consistent, and reliable manner.

But if you're using boto in existing code and it's working for you, feel free to keep using it. You can always install boto3 along side boto if you need to pull in new functionality.

answered Jul 28 '15 at 16:22

jamesls

5,428
1
28
17

6

Thanks for the response! So what is the correct way of developing with boto3? relying on documentation and coding from that? Seems a bit silly to be working with such large chunks of functionality that cannot be validated by any kind of python validator. Another problem I see here is if an API for a Amazon service changes this could potentially break boto3 scripts. I doubt this would happen, but its the downside of not having a standalone library. Is there a way to "precompile" these models? – Alex C Jul 29 '15 at 18:17
1

I would love to know this answer as well. – Pretzel Nov 06 '15 at 15:50
1

Seems to me if an AWS API changes, that's a problem for any SDK, not just Boto3's, and it's not resolved by having the underlying code available for static analysis. Indeed, Boto3 seems much better suited to deal with API changes because it is generated from the specs. If you ask me, the correct way of developing against boto3 is, unsurprisingly, with automated testing -- not necessarily unit testing, but integration testing. Specifically, check out the ubiquitous dry run params that AWS exposes. – jwilner Feb 14 '16 at 20:43
1

Is there a way to precompile these models? (I never considered dot-completion any form of test, so, to me, testing is tangential to the this question) – nik.shornikov Apr 16 '18 at 13:38
2

its a pain in the ass really, but we have to live with it, we cant intellisense boto3 like visual studio does to awssdk. how i wish my boss wanted an aspnet app, makes life easier – Jeff Nov 30 '18 at 00:20

score 6 · Answer 3 · answered Nov 14 '18 at 06:47

The room's getting a little crowded here, but I also have created a boto3 typing solution (GitHub link), boto3_type_annotations. I took the pyboto3 approach and parsed the docstrings of service objects and then programmatically wrote class definitions for them and annotated arguments and return types with the typing module. Unlike pyboto3 I created everything including service resources, paginators, and waiters. There's also an option where I left the docstrings in, so PyCharm's quick documentation will work. But fair warning, that package is really big.

# Without docs
pip install boto3_type_annotations

# Or with docs

pip install boto3_type_annotations_with_docs

score 1 · Answer 4 · answered May 25 '18 at 00:38

Probably not an official method, but I did find something that works. In PyCharm, open python console (tools/python console). The console will also have variable list on the right side. If you initialize a resource object on the console, it will have its sub objects listed in variable object tree. Some limited intellisense as well.

The way I started doing it, is writing code right into the interpreter using variable watch window as a cheat sheet. Once code is written, I copy/paste it into the actual script file. Clunky...

score 1 · Answer 5 · answered Sep 09 '18 at 22:06

I love boto3, but I was also frustrated that every time I want to make a simple ad-hoc request I have to open boto3 documentation. So I wrote autoboto:

https://pypi.org/project/autoboto/

It doesn't just auto-complete. It also returns dataclasses which means that you don't have to look up the names of attributes of the returned objects. PyCharm will tell you what is available.

At the moment, it's also probably very slow because of all the generic serialisation and deserialisation.

score 1 · Answer 6 · answered Oct 31 '19 at 21:25

1

While autocomplete solutions for boto3 are being discussed I'm surprised that nobody mentioned botostubs yet. Works on any IDE and is automatically kept up-to-date.

answered Oct 31 '19 at 21:25

szmktk

103
8

score 1 · Answer 7 · answered Dec 19 '21 at 20:24

1

This works for me. If you are using python3

python3 -m pip install boto3-stubs

python3 -m pip install 'boto3-stubs[essential]'

answered Dec 19 '21 at 20:24

Pujan

3,154
3
38
52

score 0 · Answer 8 · answered Jun 15 '19 at 15:14

Make sure that you:

Install pyboto3 -> pip install pyboto3 | pip3.x install pyboto3
Check your interpreter settings and verify that you see pyboto3 on the list
Do a File -> Invalidate Caches/Restart

After Pycharm restarts you should see intellisense working in your favor and all of the available methods for the service you are trying to use available to you!

score 0 · Answer 9 · answered Jan 29 '21 at 03:18

0

boto3-stubs

Type annotations for boto3 1.16.62 compatible with VSCode, PyCharm, Emacs, Sublime Text, mypy, pyright and other tools.

answered Jan 29 '21 at 03:18

Siva

543
4
6

score 0 · Answer 10 · answered Mar 24 '23 at 14:40

UPDATES 2023-03-24

aws-samples just release an open source boto_session_manager, it has a thin integration layer with boto_stubs, and it has explicit type hint out of the box. With this, it can pop up auto complete and argument hint. Also, you can one click to jump to the detailed document.

PyCharm intellisense for boto3

10 Answers10

Linked