How to experiment source code with `pdb`, `inspect`, `pprint`?

Question

Problem

1. I want to understand source code of Kur (deep learning library)
2. I have no proper training in programming, I prefer to learn by experimenting without prior theoretical knowledge
3. I want an easy tool to help me dig into the detail workings of source code
4. debug tools are like pdb library seem to be a good choice to try
5. But what is the easiest way to get started in experimenting source with pdb?
6. I just want to write one line of code to dig into details, rather than writing a few lines as demonstrated in many examples when you google pdb

In other words, which function of pdb should I use? and How to use it effectively for experimenting source?

Toy Example

1. I want to explore the inner workings of kur dump mnist.yml
2. To make it simple, I want to just explore not beyond __main__.py and kurfile.py.
3. To be more specific, I want to explore dump() and parse_kurfile() in __main__.py, and Kurfile.__init__() in kurfile.py

4. Their relationship is as follows:

   console: kur dump mnist.yml -->
   python: __main__.py : main() --> dump() --> parse_kurfile() -->
   python: kurfile.py : Kurfile class --> __init__() ...
   python: ... the rest is not to be explored

5. Which function of pdb should I use to explore the execution flow from dump() to parse_kurfile() and to Kurfile.__init__() and back to dump() again?

---

Update

How to effectively explore Jupyter notebook using pdb?

• pdb inside Jupyter can't even remember console history, not good

Answer 1

One possible solution

1. use pdb.set_trace only
2. set_trace is trace the details on the level of the current code block, it will not go deeper to the next inner function.

3. for example, when I put a single pdb.set_trace inside dump(), pdb will not help me trace into the function of parse_kurfile(), but stay on the current dump() block:

   def dump(args): """ Dumps the Kurfile to stdout as a JSON blob. """ pdb.set_trace() ### parse kurfile.yml into parts to be used in python code spec = parse_kurfile(args.kurfile, args.engine)

4. If I want to go deeper into parse_kurfile in __main__.py and Kurfile.__init__ in kurfile.py, then I just need to put one pdb.set_trace in each of the two functions, like below:

---

Update

From my experience so far, there are two libraries inspect and pprint go well with pdb library.

Inside library inspect, I use the following functions the most:

1. inspect.getdoc: to see the doc of the function
2. inspect.getmodule: to find out where this function or object come from
3. inspect.getfullargspec: to find out all the inputs the func takes
4. inpsect.getsourceliens: to get the source code of the function

with these functions above, when I want to checkout other functions, I don't have to go to find the source code in editor, I can see them right where I am in the pdb.

From library pprint, you can guess, I use pprint.pprint to print out the source code, the doc in a more readable format right inside pdb.

---

More Update

A working station to explore and experiment source:

1. using atom to split window and see different source files at the same time;
2. use iterm2 to split window and use ipython to execute python or bash code
3. organise them in the following way:

---

More update

During exploring, I want to have all the attributes and methods of a module or class ready at hand.

To achieve it, I can use inspect.getmembers(module or class name) and use iterm2 split window to view it:

---

Update: How to change color of iterm2 for the eyes?

Go to iterm2 preferences, color, change to Tango Dark, to gray the foreground color to make white texts look soft

Change Kur logger color setting to:

## logcolor.py
# Color codes for each log-level.
COLORS = {
    'DEBUG': BLUE,
    'INFO': MAGENTA,
    'WARNING': RED,
    'ERROR': RED,
    'CRITICAL': GREEN
}

Answer 2

How to effectively use pdb in Jupyter notebook?

One way to avoid the drawbacks of pdb in Jupyter:

• download the notebook into py file
• insert codes like import pdb and pdb.set_trace() into the python code
• in console, run python your.py
• Now you can explore this py file as you do in above answer