1

We are going to start doing a reasonable amount of development on some customization of CKAN via plugins.

Are there any CKAN developer recommendations, hints, tips and/or tricks that people use when developing plugins for CKAN?

Things that we are specifically looking for help on are the following:

Recommendations on how to do development in an IDE (like PyCharm) and move/deploy our code to a remote Vagrant VM (where we will have development instance of ckan running).

How does debugging work in a remote Vagrant VM instance with WSGI? Does anyone do remote debugging with an IDE (like PyCharm) and WSGI?

Then in general how do most CKAN developers do their programming? Do they use Vi? Do they do development in ckan on their local machine? Do they use remote VMs (like with Vagrant/VirtualBox)?

We have tried looking for how anyone does CKAN plugin development and we have not been able to find any useful information at this time and any friendly recommendations would be greatly appreciated.

hooknc
  • 4,854
  • 5
  • 31
  • 60

3 Answers3

2

I develop locally "inside" a complete development install of CKAN. For deploying to the remote production server (in a VM, but I think that is not relevant) this server has a bare git repo for CKAN proper and for each extension. When I push to any of those repos a post-receive hook is triggered that checks out HEAD to /usr/lib/ckan/default/src/ and restarts apache2.

That works quite nicely to deploy fast, keep the code in sync, and rollback fast ;).

This setup is complemented by little scripts for occasionally syncing the development installation with the production install (DB, FileStore, SOLR indexing).

Personally I'm using Emacs right now which I find easier than vim for searching and navigating the code, but everyone has their taste.

hvwaldow
  • 1,296
  • 1
  • 11
  • 13
1

I think the most common way in the ckan community is to use a simple text editor, such as vim, to edit code. Personally I'm using Sublime right now which I find easier than vim for searching and navigating the code, but everyone has their taste.

The Syntastic vim plugin is popular amongst ckan devs for ensuring the code is PEP8 compliant and reflowing text to be 79 characters wide etc. But most editors have this sort of thing in plugins.

It's really helpful to get familiar with debugging with The Python Debugger (pdb) - this appears to be the most popular option for ckan. It's quick to learn and always works. I tried a couple of python IDE debuggers many years ago and found them unreliable, but maybe this is sorted now.

Turn on debug mode in your ckan ini file to deal with web exceptions, but occasionally I turn it off when calling ckan on the command-line e.g. debugging API calls.

To run ckan, either a linux machine or a vagrant/Virtual Box linux vm.

hooknc
  • 4,854
  • 5
  • 31
  • 60
D Read
  • 3,175
  • 1
  • 15
  • 25
0

After a couple years of trying different things, we finally got PyCharm to work with our ckan/Vagrant VM environment.

We tried numerous times trying to get a local instance of PyCharm to talk to our guest Vagrant VM, but that seemed quite fragile and confusing for the developers.

Then someone on our team said something to the effect of "Why don't we just install PyCharm on the Vagrant VM and then use x11 forwarding to see the user interface?"

Seemed like a good idea to try and it has been working great for us. We get all the benefits of an IDE and debugging to boot. Yay!

Here is how we did it. You will have to understand your particular environment and change what you see below to work for what you're doing. You will also need to take care of your own licensing of PyCharm!

VagrantFile

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure(2) do |config|

  config.vm.box = "centos/7"

  # apache port
  config.vm.network "forwarded_port", guest: 5000, host: 5000, auto_correct: false
  # database port
  config.vm.network "forwarded_port", guest: 5432, host: 65432, auto_correct: false
  # solr port
  config.vm.network "forwarded_port", guest: 8080, host: 8080, auto_correct: false

  # For PyCharm
  config.ssh.forward_agent = true
  config.ssh.forward_x11 = true

  # Install PyCharm IDE via shell script
  config.vm.provision "shell", path: "install_pycharm.sh"

end

install_pycharm.sh

#!/bin/sh -e
set -x

cd /tmp
wget https://download.jetbrains.com/python/pycharm-professional-2017.3.1.tar.gz

tar -xvzf pycharm-professional-2017.3.1.tar.gz -C /tmp/

cp -r /tmp/pycharm-2017.3.1 /opt/pycharm

ln -s /opt/pycharm/bin/pycharm.sh /usr/local/bin/pycharm
ln -s /opt/pycharm/bin/inspect.sh /usr/local/bin/inspect

yum -y groupinstall "X Window System"

# Copy .Xauthority file from vagrant to ckan home
cp /vagrant/.Xauthority /usr/lib/ckan
chown ckan:ckan /usr/lib/ckan/.Xauthority

echo "==========================================="
echo "PyCharm will be installed in your Vagrant instance, but...."
echo "now you must run some manual steps to get PyCharm working:"
echo ""
echo " ssh into vagrant"
echo " This ssh will create your /home/vagrant/.Xauthority file, which you need to see the PyCharm GUI."
echo " Then you will need to copy this .Xauthority file to the ckan user to run PyCharm as ckan."
echo ""
echo " sudo cp /home/vagrant/.Xauthority /usr/lib/ckan"
echo " sudo chown ckan:ckan /usr/lib/ckan/.Xauthority"
echo " Set the DISPLAY variable for ckan user as it is set for vagrant when you do echo $DISPLAY: "
echo " export DISPLAY=localhost:11.0 "
echo " To start PyCharm run"
echo " pycharm "
hooknc
  • 4,854
  • 5
  • 31
  • 60