Profilers measure the performance of a whole program to identify where most of the time is spent.
But once youâve found a target function, re-profiling the whole program to see if your changes helped can be slow and cumbersome.
The profiler introduces overhead to execution and you have to pick out the stats for the one function you care about from the report.
I have often gone through this loop while optimizing client or open source projects, such as when I optimized Djangoâs system checks framework (previous post).
The pain here inspired me to create tprof, a targeting profiler for Python 3.12+ that only measures the time spent in specified target functions.
Use it to measure your program before and after an optimization to see if it made any difference, with a quick report on the command line.
For example, say youâve realized that creating pathlib.Path objects is the bottleneck for your code.
You could run tprof like so:
Benchmark with comparison mode
Sometimes when optimizing code, you want to compare several functions, such as âbeforeâ and âafterâ versions of a function youâre optimizing.
tprof supports this with its comparison mode, which adds a âdeltaâ column to the report showing how much faster or slower each function is compared to a baseline.
For example, given this code:
def before():
total = 0
for i in range(100_000):
total += i
return total
def after():
return sum(range(100_000))
for _ in range(100):
before()
after()
âŠyou can run tprof like this to compare the two functions:
$ tprof -x -t before -t after -m example
đŻ tprof results:
function calls total mean ± Ï min ⊠max delta
example:before() 100 227ms 2ms ± 34Όs 2ms ⊠2ms -
example:after() 100 86ms 856Όs ± 15Όs 835Όs ⊠910Όs -62.27%
The output shows that after() is about 60% faster than before(), in this case.
Python API
tprof also provides a Python API via a context manager / decorator, tprof().
Use it to profile functions within a specific block of code.
For example, to recreate the previous benchmarking example within a self-contained Python file:
from tprof import tprof
def before():
total = 0
for i in range(100_000):
total += i
return total
def after():
return sum(range(100_000))
with tprof(before, after, compare=True):
for _ in range(100):
before()
after()
âŠwhich produces output like:
$ python example.py
đŻ tprof results:
function calls total mean ± Ï min ⊠max delta
__main__:before() 100 227ms 2ms ± 83Όs 2ms ⊠3ms -
__main__:after() 100 85ms 853Όs ± 22Όs 835Όs ⊠1ms -62.35%
How it works
tprof uses Pythonâs sys.monitoring, a new API introduced in Python 3.12 for triggering events when functions or lines of code execute.
sys.monitoring allows tprof to register callbacks for only specific target functions, meaning it adds no overhead to the rest of the program.
Timing is done in C to further reduce overhead.
Thanks to Mark Shannon for contributing sys.monitoring to CPython!
This is the second time Iâve used itâthe first time was for tracking down an unexpected mutation (see previous post).
Fin
If tprof sounds useful to you, please give it a try and let me know what you think!
Install tprof from PyPI with your favourite package manager.
May you hit your Q1 targets,
âAdam
Log in
