Spyder: How to edit a python script locally and execute it on a remote kernel?

Question

i am using Spyder 2.3.1 under Windows 7 and have a running iPython 2.3 Kernel on a Rasperry Pi RASPBIAN Linux OS.

I can connect to an external kernel, using a .json file and this tutorial: Remote ipython console

But what now? If I "run" a script (F5), then the kernel tries to exectue the script like:

%run "C:\test.py"

ERROR: File u'C:\\test.py' not found.

This comes back with an error, ofc, because the script lays on my machine under c: and not on the remote machine/raspberry pi. How to I tell Spyder to somehow copy first the script to the remote machine and execute it there?

If I check the "this is a remote kernel" checkbox, I cannot connect to the existing kernel anymore. What does that box mean? Will it copy the script via SSH to the remote machine before execution? If I enter the SSH login information, I get an "It seems the kernel died unexpectedly" error.

Answer 1

The tutorial that you mention is a little bit our of date as Spyder now has the ability to connect to remote kernels. The "This is a remote kernel" checkbox, when checked, enables the portion of the dialog where you can enter your ssh connection credentials. (You should need this unless you have manually opened the required ssh tunnels to forward the process ports of your remote kernel... )

Besides, the ipython connection info (the json file) must correspond to the remote kernel, running on your raspberry pi.

Finally, there is no means at this time to copy the script lying on your local pc when you hit run. The preferred method would actually be the reverse: mount your raspberry pi's filesystem using a tool like sshfs and edit them in place. The plan is to implement an sftp client in Spyder so that it will not be required and you will be able to explore the remote filesystem from Spyder's file explorer.

To summarize:

1) assuming that you are logged in your raspberry pi, launch a local IPython kernel with ipython kernel. It should give you the name of your json file to use, which you should copy to your local pc.

2) in spyder on your local pc, connect to a remote kernel with that json file and your ssh credentials

I know that it is cumbersome, but it is a first step..

Answer 2

After a search in the site-packages\spyderlib directory for the keyword %run, I found the method(in site-packages\spyderlib\plugins\ipythonconsole.py) which constructs the %run command:

    def run_script_in_current_client(self, filename, wdir, args, debug):
    """Run script in current client, if any"""
    norm = lambda text: remove_backslashes(to_text_string(text))
    client = self.get_current_client()
    if client is not None:
        # Internal kernels, use runfile
        if client.kernel_widget_id is not None:
            line = "%s('%s'" % ('debugfile' if debug else 'runfile',
                                norm(filename))
            if args:
                line += ", args='%s'" % norm(args)
            if wdir:
                line += ", wdir='%s'" % norm(wdir)
            line += ")"
        else: # External kernels, use %run
            line = "%run "
            if debug:
                line += "-d "
            line += "\"%s\"" % to_text_string(filename)
            if args:
                line += " %s" % norm(args)
        self.execute_python_code(line)
        self.visibility_changed(True)
        self.raise_()
    else:
        #XXX: not sure it can really happen
        QMessageBox.warning(self, _('Warning'),
            _("No IPython console is currently available to run <b>%s</b>."
              "<br><br>Please open a new one and try again."
              ) % osp.basename(filename), QMessageBox.Ok)

I added the following code to convert paths after else: # External kernels, use %run

            # ----added to remap local dir to remote dir-------
            localpath = "Z:\wk"
            remotepath = "/mnt/sdb1/wk"
            if localpath in filename:
                # convert path to linux path
                filename = filename.replace(localpath, remotepath)
                filename = filename.replace("\\", "/")
            # ----- END mod

now it runs the file on the remote machine when I hit F5. I am on Spyder 2.3.9 with samba share mapped to z: drive.

Answer 3

Another option is to use Spyder cells to send the whole contents of your file to the IPython console. I think this is easier than mounting your remote filesystem with Samba or sshfs (in case that's not possible or hard to do).

Cells are defined by adding lines of the form # %% to your file. For example, let's say your file is:

# -*- coding: utf-8 -*-

def f(x):
    print(x + x)

f(5)

Then you can just add a cell at the bottom like this

# -*- coding: utf-8 -*-

def f(x):
    print(x + x)

f(5)

# %%

and by pressing Ctrl + Enter above the cell line, the full contents of your file will be sent to the console and evaluated at once.

Answer 4

Just thought I'd make my first post to update Roy Cai's answer for Spyder 4 in case anyone is looking for this. Roy's answer worked flawlessly for me. Spyder 4 has moved the relevant code from where it was when he wrote the answer. The method is now in \Lib\site-packages\spyder\plugins\ipythonconsole and the python file is plugin.py.

Everything otherwise works the same as it used to - The place to insert the modified code is the same, and the same update fixes it.

(incidentally - yay for the ability to save login info for logging into remote kernels in Spyder 4!)