4

I use python3 -m pytest to run my tests. This was working when python3 was pointing to python3.6. I installed python3.7, made python3.7 as my default python3 with these commands:

sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.7 2
sudo update-alternatives --config python3

and installed all the required packages again for python3.7. But now I cannot run my tests anymore. Because it says: /usr/bin/python3: No module named pytest. I'm not using virtual env and the problem is not about a specific module, because it cannot find other modules either. It should be related to python paths. I don't know what to change after switching between python3 versions so it can easily find the required modules in /home/ubuntu/.local/lib/python3.x/site-packages/.

I switched back to python3.6 and getting ModuleNotFoundError: No module named 'jsonpath_ng' now, which was working before switching to python3.7. This is the output of python3 -m site:

sys.path = [
    '/home/ubuntu',
    '/usr/lib/python36.zip',
    '/usr/lib/python3.6',
    '/usr/lib/python3.6/lib-dynload',
    '/home/ubuntu/.local/lib/python3.6/site-packages',
    '/usr/local/lib/python3.6/dist-packages',
    '/usr/lib/python3/dist-packages',
]
USER_BASE: '/home/ubuntu/.local' (exists)
USER_SITE: '/home/ubuntu/.local/lib/python3.6/site-packages' (exists)
ENABLE_USER_SITE: True
Fatemeh Hosseini
  • 71
  • 1
  • 6
  • 1
    You're likely not using the correct PIP with the correct Python interpreter. Try running `python3 -m pip install pytest --user` (can omit the `--user` if installing system-wide). The system's PIP may still be using the wrong Python interpreter, and this may fix your issue. – Alex Huszagh Feb 28 '20 at 21:28
  • 1
    I ran that command and it says: `Defaulting to user installation because normal site-packages is not writeable Requirement already satisfied: pytest in /home/ubuntu/.local/lib/python3.7/site-packages (5.3.5)...` – Fatemeh Hosseini Feb 28 '20 at 21:44
  • Very intriguing. Can you try entering the interpreter and running `import pytest` and also running `import sys; sys.executable`? – Alex Huszagh Feb 28 '20 at 21:53
  • I entered python3 interpreter, ran `import pytest` and didn't show any error, then ran `import sys; sys.executable` and it shows `/usr/bin/python3` – Fatemeh Hosseini Feb 28 '20 at 22:01
  • I'm honestly very confused, because it seems like everything is installed properly. Can you try `file /usr/bin/python3` from the shell? For example, on my end it gives me `/usr/bin/python3: symbolic link to python3.7`. This would confirm it's using the right interpreter. – Alex Huszagh Feb 28 '20 at 22:16
  • I wish I could provide more help, sorry. – Alex Huszagh Feb 28 '20 at 22:16
  • Actually python3 cannot find other modules either: `/usr/bin/python3: No module named pipenv` – Fatemeh Hosseini Feb 28 '20 at 22:17
  • @AlexanderHuszagh it returns `/usr/bin/python3: symbolic link to /etc/alternatives/python3` – Fatemeh Hosseini Feb 28 '20 at 22:18
  • You may want to try asking around community for your distro, as this is not a generic python (or programming) question. You may stumble upon people who use the same distro as you do and even similar setup regarding its configuration, but elsewhere there might be higher concentration of people with specific knowledge and experience. – Ondrej K. Feb 29 '20 at 00:15
  • _But now I cannot run my tests anymore._ - what does it mean? What is the exact error message? Please update the question. – hoefling Feb 29 '20 at 14:06
  • Does this answer your question? [ModuleNotFoundError: No module named 'pytest'](https://stackoverflow.com/questions/55652866/modulenotfounderror-no-module-named-pytest) – Adam Strauss Mar 02 '20 at 05:18
  • @AdamStrauss No, I'm not using virtual env and the problem is not about a specific module. It is about python paths that it cannot find the installed modules in the right site-packages directory. I tried to switch back to python3.6 but it's not working anymore because it cannot find the installed modules in /home/ubuntu/.local/lib/python3.6/site-packages. Although everything I need is installed and I can find them in those paths. But I don't know what has changed that python3 cannot find the right path for the modules. – Fatemeh Hosseini Mar 02 '20 at 05:47
  • What happens if you ran just pytest ? – Kris Mar 02 '20 at 06:02
  • Can you run `python3 -m site` and add the output to the question? – hoefling Mar 02 '20 at 13:19

1 Answers1

2

Thank you to @hoefling, I found the problem by running python3 -m site. I was running tests with my jenkins user and not the root, while I was installing everything for the root user. So USER_SITE didn't actually exist for jenkins user. I installed everything again for the jenkins user and now it can find the modules when I use python3 -m.

Fatemeh Hosseini
  • 71
  • 1
  • 6
  • Glad I could help, although you did find the issue yourself in the end, so not much to thank me for :-) – hoefling Mar 02 '20 at 17:07
  • Can you please explain how do you figure out the jenkin user vs root user. i am having same problem, could not figure out the soluthon. thanks. – nur Jan 07 '21 at 06:41