0

I am creating a vscode extension where I need machine learning tasks to be performed. I have python files that have code that is required in vscode extension. I don't want things to be done using request-response on any python server. What I want is to perform the ML tasks on device (integrated with the vsix).

We have child-process available in js to run basic python file using spawn. It is running fine on both, extension host window and external vscode editor after packaging, with the python code that has basic imports like import sys. But if I try to import some other libraries like numpy, pygments, it is working only on extension host environment, not on other vs window after packaging it. How can I run the typical python code with the vsix?

Below are both the codes that is working fine and not working at all-

TS file (MLOps.ts)-

import { ChildProcessWithoutNullStreams, spawn } from "child_process";
import { join } from "path";
import * as vscode from 'vscode'

export async function pythonOps(): Promise<string> {
    var result = "testt"
    var promise = await new Promise((resolve, reject) => {
        var p = __dirname.split('\\')
        p.pop()
        var path = p.join('\\')
        var pyPath = join(path, 'src', 'py_operations.py')
        var result = "blank result"

        var arg1 = "arg one"
        var arg2 = "arg two"
        var py_process = spawn('python', [pyPath, arg1, arg2])
        py_process.stdout.on('data', (data: any) => {
            vscode.window.showInformationMessage(data.toString())
            result = data.toString()
        })
    })
}

Working Python code (py_operations.py)- This code is working on both, on extension host window and after packaging the extension and installing the vsix on other system.

import sys
print("Some text with: ",sys.argv[0], sys.argv[1], sys.argv[2])
sys.stdout.flush()

Not working Python code- This code is working only on extension host window, and not working after packaging this and isntalling on other system.

import sys
from pygments.lexers.javascript import TypeScriptLexer

lexer = TypeScriptLexer()
src = "alert('text here')"
lexer_tokens = lexer.get_tokens(src)
l = []
for t in lexer_tokens:
    l.append(t[1])

print("list: ",l)
sys.stdout.flush()

How can I run the second python code with packaged vsix?

Tanmay Bairagi
  • 564
  • 5
  • 25
  • Did you find an answer to this? I will be tackling it shortly. I can imagine just copying the dependency source files directly into the extension source files and bundling it all together. It is also possible to use PYTHON_PATH to dynamically point at Python code for it to be importable. It is also possible to import directly from a zip file at times so perhaps the dependency could be in the bundle as a zip file. Not sure. – MichaelJones May 06 '22 at 12:03
  • 1
    you can install the dependencies as per your requirement. You can create a shall script and run that from the extension. This script creates a virtual environment, activate that environment, and then install all the requirements into that using `pip install -r requirements.txt`. After the installation completes, you can run any python file in this environment. You no need to copy all the dependencies into the extension as it will make the plugin heavy to deploy. – Tanmay Bairagi May 13 '22 at 14:39
  • 1
    I cannot help with the import part of this question, but I deliver python scripts with my extension, and have found that a 'nice' way to get the path to the py files is by using the extension context like this: `const pathToTestInterface = context.asAbsolutePath("./python/vTestInterface.py");` Might be nicer than the __dirname stuff you are using – John Paliotta Jan 06 '23 at 16:07

1 Answers1

0

I am not sure if this is the recommended way ( since there is none ) but a very considerate option is to pack a whole installation of Python in your .vsix file.

It is not really that heavy, for example, a barebones installation of Python 3.11.4 packaged with your source code will result in .vsix file of less than 60 MB size. Of course you may also install any python-package you need along with it and that will change the final size but given that most python packages are rather small this is not a limiting factor. Keep in mind that if the size becomes important you may manually remove unnecessary files from both the main installation and the python-packages. Different systems may require different python installations so you might find useful the marketplace functionality of providing multiple .vsix files for different operating systems.

How to ?

Well the most "entry" way (but maybe not the best) is the following :

  1. Use the pip freeze > old_py_packages.txt command to save the titles & versions of all your current python-packages in your machine.

  2. Uninstall all of them so you are left with a clean install of Python. Use the command pip uninstall -r old_py_packages.txt. If you want you may remove from said .txt the packages your extension will be needing to avoid the next step. Here I will recommend that you don't just uninstall the packages as I said but directly uninstall Python from your system and re-install it cleanly. This will save some time since using the pip uninstall command will prompt you for each package and also sometimes it bugs your install (I cannot back it up but I have seen it happen).

  3. Install all the necessary python-packages for your extension through pip install for each dependency.

  4. Find the path where Python is installed (in my Machine it is C:\Users\USERNAME\AppData\Local\Programs\Python\Python311) and copy everything from there in a folder (e.g. local_install/) inside your extension development directory.

  5. Use pip install -r old_py_packages.txt to recover all previously unistalled python packages in your system.

  6. Change your extension.js spawning code to

var py_process=spawn(__dirname.replace(/\\/g,'/')+'/local_install/python.exe',[pyPath, arg1, arg2])

       So your extension uses your packaged Python installation instead of the user's system's one.

  1. Use the vsce package command while in your extension development directory to package everything into a .vsix file.

You will get a .vsix file with a local python installation and all the python packages you need without messing with the end user's installation.

There is also the option of developing your extension in python entirely through this wrapper but it is not maintained.

Ioannis Tsampras
  • 11
  • 4