The idea is to use Python's decorator to inject some scaffoldings to the code that's to be tested and debugged, and use Python's ability to dynamically reload source file to fix issues and resume execution. Here is the decorator:
def untrusted(func):
def wrapper_func(*args, **kwds):
try:
return func(*args, **kwds)
except Exception, e:
print str(e) or e.__class__
import pdb
pdb.set_trace()
return wrapper_func
It just calls the function that's being decorated and go into debugging mode if an exception occurs. Here is a function that has a typo, and imagine it's going to be called in the middle of an hour long batch job. @untrusted
def print_date(d):
print d.strftim("%Y-%m-%d")
When the function is called, an exception is raised and caught by the decorator and on the command line it looks like:
'datetime.datetime' object has no attribute 'strftim' --Return-- > /Users/jiayao/examples/jiayao/debug.py(9)wrapper_func()->None -> pdb.set_trace() (Pdb)
Now I can fix the typo in the source code. Then:
(Pdb) import exampleThen the program can be resumed as if no error has ever happened. "import example" loads the source code dynamically so I can execute the correct implementation of "print_date" with the same arguments as the original invocation. Note "import example" will work only once, calling import on the same module more than once has no effect, so to load the source again you need to call "reload(example)" the next time.(Pdb) example.print_date(*args) 2009-06-16
Another example, this time the error is using a module but forgot to import it first:
@untrusted
def match(pattern, text):
return re.search(pattern, text)
match("^abc", "abcde")
Entering debug mode:
global name 're' is not defined --Return-- > /Users/jiayao/examples/jiayao/debug.py(9)wrapper_func()->None -> pdb.set_trace() (Pdb)
Simple calling import re and invoke the function again will not work because the function maintains it's own copy of globals including the imports. "import re" here will not change the small world encapsulated in the function object. So we have to inject the import into it:
(Pdb) import re (Pdb) func.func_globals['re'] = re (Pdb) func(*args) <_sre.SRE_Match object at 0x24e4f0>
One last example, this is dealing with objects and it's a bit more complicated:
class A(object):
def __init__(self, text=None):
self.text = text
@untrusted
def search(self, keyword):
return keyword in self.text
a = A()
a.search("abc")
>python example.py
argument of type 'NoneType' is not iterable
--Return--
> /Users/jiayao/examples/jiayao/debug.py(9)wrapper_func()->None
-> pdb.set_trace()
And we fix "search" function in the source code:
@untrusted
def search(self, keyword):
if self.text:
return keyword in self.text
else:
return False
And we try out the fix:
(Pdb) import example (Pdb) example.A.search(*args) *** TypeError: unbound method wrapper_func() must be called with A instance as first argument (got A instance instead)
This is a strange error at first glance. But the "A" is not the same as the other "A":
(Pdb) example.A < class 'example.A'> (Pdb) type(args[0]) < class '__main__.A'>
So we can not call example.A on an __main__.A object. We can work around this:
(Pdb) new_a = example.A() (Pdb) new_a.__dict__ = args[0].__dict__ (Pdb) example.A.search(new_a, *args[1:]) False
Hope this is useful for some fellow programmers out there. Use the time you saved wisely! :)
