PrevUpHomeNext

How do I debug my Python extensions?

Greg Burley gives the following answer for Unix GCC users:

Once you have created a boost python extension for your c++ library or class, you may need to debug the code. Afterall this is one of the reasons for wrapping the library in python. An expected side-effect or benefit of using BPL is that debugging should be isolated to the c++ library that is under test, given that python code is minimal and boost::python either works or it doesn't. (ie. While errors can occur when the wrapping method is invalid, most errors are caught by the compiler ;-).

The basic steps required to initiate a gdb session to debug a c++ library via python are shown here. Note, however that you should start the gdb session in the directory that contains your BPL my_ext.so module.

(gdb) target exec python
(gdb) run
>>> from my_ext import *
>>> [C-c]
(gdb) break MyClass::MyBuggyFunction
(gdb) cont
>>> pyobj = MyClass()
>>> pyobj.MyBuggyFunction()
Breakpoint 1, MyClass::MyBuggyFunction ...
Current language:  auto; currently c++
(gdb) do debugging stuff

Greg's approach works even better using Emacs' "gdb" command, since it will show you each line of source as you step through it.

On Windows, my favorite debugging solution is the debugger that comes with Microsoft Visual C++ 7. This debugger seems to work with code generated by all versions of Microsoft and Metrowerks toolsets; it's rock solid and "just works" without requiring any special tricks from the user.

Raoul Gough has provided the following for gdb on Windows:

gdb support for Windows DLLs has improved lately, so it is now possible to debug Python extensions using a few tricks. Firstly, you will need an up-to-date gdb with support for minimal symbol extraction from a DLL. Any gdb from version 6 onwards, or Cygwin gdb-20030214-1 and onwards should do. A suitable release will have a section in the gdb.info file under Configuration - Native - Cygwin Native - Non-debug DLL symbols. Refer to that info section for more details of the procedures outlined here.

Secondly, it seems necessary to set a breakpoint in the Python interpreter, rather than using ^C to break execution. A good place to set this breakpoint is PyOS_Readline, which will stop execution immediately before reading each interactive Python command. You have to let Python start once under the debugger, so that it loads its own DLL, before you can set the breakpoint:

$ gdb python
GNU gdb 2003-09-02-cvs (cygwin-special)
[...]

(gdb) run
Starting program: /cygdrive/c/Python22/python.exe
Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ^Z


Program exited normally.
(gdb) break *&PyOS_Readline
Breakpoint 1 at 0x1e04eff0
(gdb) run
Starting program: /cygdrive/c/Python22/python.exe
Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

Breakpoint 1, 0x1e04eff0 in python22!PyOS_Readline ()
   from /cygdrive/c/WINNT/system32/python22.dll
(gdb) cont
Continuing.
>>> from my_ext import *

Breakpoint 1, 0x1e04eff0 in python22!PyOS_Readline ()
   from /cygdrive/c/WINNT/system32/python22.dll
(gdb) # my_ext now loaded (with any debugging symbols it contains)

Debugging extensions through Boost.Build

If you are launching your extension module tests with Boost.Build using the boost-python-runtest rule, you can ask it to launch your debugger for you by adding "--debugger=debugger" to your bjam command-line:

bjam -sTOOLS=vc7.1 "--debugger=devenv /debugexe" test
bjam -sTOOLS=gcc -sPYTHON_LAUNCH=gdb test

It can also be extremely useful to add the -d+2 option when you run your test, because Boost.Build will then show you the exact commands it uses to invoke it. This will invariably involve setting up PYTHONPATH and other important environment variables such as LD_LIBRARY_PATH which may be needed by your debugger in order to get things to work right.


PrevUpHomeNext