njs blog

Beautiful tracebacks in Trio v0.7.0

Trio is a new async concurrency library for Python that's obsessed with correctness and usability.

On the correctness side, one of Trio's unique features is that it never discards exceptions: if you don't catch an exception, then eventually it will propagate out the top of your program and print a traceback to help you debug, just like in regular Python. Errors should never pass silently!

But... in Trio v0.6.0 and earlier, these tracebacks also contained a lot of clutter showing how the exception moved through Trio's internal plumbing, which made it difficult to see the parts that were relevant to your code. It's a small thing, but when you're debugging some nasty concurrency bug, it can make a big difference to have exactly the information you need, clearly laid out, without distractions.

And thanks to some hard work by John Belmonte, the just-released Trio v0.7.0 gives you exactly that: clean tracebacks, focused on your code, without the clutter. See below for some before/after comparisons.

Before Trio, I never really thought about where tracebacks came from, and I certainly never changed how I wrote code because I wanted it to produce a different traceback. Making useful tracebacks is the interpreter's job, right? In the process, we had to study how the interpreter manages tracebacks, how they interact with context managers, how to introspect stack usage in third-party libraries, and other arcane details ... but the results are totally worth it.

To me, this is what makes Trio so fun to work on: our goal is to make Python concurrency an order of magnitude friendlier and more accessible than it's ever been before, and that means we're constantly exploring new design spaces, discovering new things, and figuring out new ways to push the limits of the language.

If that sounds like fun to you too, then we're always looking for contributors. And don't worry, you don't need to be an expert on tracebacks or concurrency – the great thing about inventing something new is that we get to figure it out together!

Or, just scroll down to check out our new tracebacks. They're so pretty! 🤩

Simple example

Here's the simplest possible crashing Trio program:

import trio

async def main():
    raise RuntimeError("whoops")

trio.run(main)

With previous Trio versions, this code gave us a traceback like:

Traceback (most recent call last):
  File "error-example.py", line 6, in <module>
    trio.run(main)
  File ".../site-packages/trio/_core/_run.py", line 1277, in run
    return result.unwrap()
  File ".../site-packages/outcome/_sync.py", line 107, in unwrap
    raise self.error
  File ".../site-packages/trio/_core/_run.py", line 1387, in run_impl
    msg = task.context.run(task.coro.send, next_send)
  File ".../site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File ".../site-packages/trio/_core/_run.py", line 970, in init
    self.entry_queue.spawn()
  File ".../site-packages/async_generator/_util.py", line 42, in __aexit__
    await self._agen.asend(None)
  File ".../site-packages/async_generator/_impl.py", line 366, in step
    return await ANextIter(self._it, start_fn, *args)
  File ".../site-packages/async_generator/_impl.py", line 202, in send
    return self._invoke(self._it.send, value)
  File ".../site-packages/async_generator/_impl.py", line 209, in _invoke
    result = fn(*args)
  File ".../site-packages/trio/_core/_run.py", line 317, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File ".../site-packages/trio/_core/_run.py", line 202, in open_cancel_scope
    yield scope
  File ".../site-packages/trio/_core/_multierror.py", line 144, in __exit__
    raise filtered_exc
  File ".../site-packages/trio/_core/_run.py", line 202, in open_cancel_scope
    yield scope
  File ".../site-packages/trio/_core/_run.py", line 317, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File ".../site-packages/trio/_core/_run.py", line 428, in _nested_child_finished
    raise MultiError(self._pending_excs)
  File ".../site-packages/trio/_core/_run.py", line 1387, in run_impl
    msg = task.context.run(task.coro.send, next_send)
  File ".../site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "error-example.py", line 4, in main
    raise RuntimeError("whoops")
RuntimeError: whoops

It's accurate, and I guess it shows off how hard Trio is working on your behalf, but that's about all I can say for it – all the stuff our users care about is drowned in the noise.

But thanks to John's fixes, Trio v0.7.0 instead prints:

Traceback (most recent call last):
  File "error-example.py", line 6, in <module>
    trio.run(main)
  File ".../site-packages/trio/trio/_core/_run.py", line 1328, in run
    raise runner.main_task_outcome.error
  File "error-example.py", line 4, in main
    raise RuntimeError("whoops")
RuntimeError: whoops

Three frames, straight to the point. We've removed almost all of Trio's internals from the traceback. And, for the one line that we can't remove (due to Python interpreter limitations), we've rewritten it so you can get a rough idea of what it's doing even when it's presented out of context like this. (run re-raises the main task's error.)

A more complex example

Here's a program that starts two concurrent tasks, which both raise exceptions simultaneously. (If you're wondering what this "nursery" thing is, see this earlier post.)

import trio

async def crasher1():
    raise KeyError

async def crasher2():
    raise ValueError

async def main():
    async with trio.open_nursery() as nursery:
        nursery.start_soon(crasher1)
        nursery.start_soon(crasher2)

trio.run(main)

Hope your scroll wheel is ready, because here's what old versions of Trio printed for this:

Traceback (most recent call last):
  File "error-example.py", line 14, in <module>
    trio.run(main)
  File ".../site-packages/trio/_core/_run.py", line 1277, in run
    return result.unwrap()
  File ".../site-packages/outcome/_sync.py", line 107, in unwrap
    raise self.error
  File ".../site-packages/trio/_core/_run.py", line 1387, in run_impl
    msg = task.context.run(task.coro.send, next_send)
  File ".../site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File ".../site-packages/trio/_core/_run.py", line 970, in init
    self.entry_queue.spawn()
  File ".../site-packages/async_generator/_util.py", line 42, in __aexit__
    await self._agen.asend(None)
  File ".../site-packages/async_generator/_impl.py", line 366, in step
    return await ANextIter(self._it, start_fn, *args)
  File ".../site-packages/async_generator/_impl.py", line 202, in send
    return self._invoke(self._it.send, value)
  File ".../site-packages/async_generator/_impl.py", line 209, in _invoke
    result = fn(*args)
  File ".../site-packages/trio/_core/_run.py", line 317, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File ".../site-packages/trio/_core/_run.py", line 202, in open_cancel_scope
    yield scope
  File ".../site-packages/trio/_core/_multierror.py", line 144, in __exit__
    raise filtered_exc
  File ".../site-packages/trio/_core/_run.py", line 1387, in run_impl
    msg = task.context.run(task.coro.send, next_send)
  File ".../site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "error-example.py", line 12, in main
    nursery.start_soon(crasher2)
  File ".../site-packages/async_generator/_util.py", line 42, in __aexit__
    await self._agen.asend(None)
  File ".../site-packages/async_generator/_impl.py", line 366, in step
    return await ANextIter(self._it, start_fn, *args)
  File ".../site-packages/async_generator/_impl.py", line 202, in send
    return self._invoke(self._it.send, value)
  File ".../site-packages/async_generator/_impl.py", line 209, in _invoke
    result = fn(*args)
  File ".../site-packages/trio/_core/_run.py", line 317, in open_nursery
    await nursery._nested_child_finished(nested_child_exc)
  File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
    self.gen.throw(type, value, traceback)
  File ".../site-packages/trio/_core/_run.py", line 202, in open_cancel_scope
    yield scope
  File ".../site-packages/trio/_core/_multierror.py", line 144, in __exit__
    raise filtered_exc
trio.MultiError: KeyError(), ValueError()

Details of embedded exception 1:

  Traceback (most recent call last):
    File ".../site-packages/trio/_core/_run.py", line 202, in open_cancel_scope
      yield scope
    File ".../site-packages/trio/_core/_run.py", line 317, in open_nursery
      await nursery._nested_child_finished(nested_child_exc)
    File ".../site-packages/trio/_core/_run.py", line 428, in _nested_child_finished
      raise MultiError(self._pending_excs)
    File ".../site-packages/trio/_core/_run.py", line 1387, in run_impl
      msg = task.context.run(task.coro.send, next_send)
    File ".../site-packages/contextvars/__init__.py", line 38, in run
      return callable(*args, **kwargs)
    File "error-example.py", line 4, in crasher1
      raise KeyError
  KeyError

Details of embedded exception 2:

  Traceback (most recent call last):
    File ".../site-packages/trio/_core/_run.py", line 202, in open_cancel_scope
      yield scope
    File ".../site-packages/trio/_core/_run.py", line 317, in open_nursery
      await nursery._nested_child_finished(nested_child_exc)
    File ".../site-packages/trio/_core/_run.py", line 428, in _nested_child_finished
      raise MultiError(self._pending_excs)
    File ".../site-packages/trio/_core/_run.py", line 1387, in run_impl
      msg = task.context.run(task.coro.send, next_send)
    File ".../site-packages/contextvars/__init__.py", line 38, in run
      return callable(*args, **kwargs)
    File "error-example.py", line 7, in crasher2
      raise ValueError
  ValueError

Accurate, but unreadable. But now, after rewriting substantial portions of Trio's core task management code, we get:

Traceback (most recent call last):
  File "error-example.py", line 14, in <module>
    trio.run(main)
  File ".../site-packages/trio/trio/_core/_run.py", line 1328, in run
    raise runner.main_task_outcome.error
  File "error-example.py", line 12, in main
    nursery.start_soon(crasher2)
  File ".../site-packages/trio/trio/_core/_run.py", line 395, in __aexit__
    raise combined_error_from_nursery
trio.MultiError: KeyError(), ValueError()

Details of embedded exception 1:

  Traceback (most recent call last):
    File "error-example.py", line 4, in crasher1
      raise KeyError
  KeyError

Details of embedded exception 2:

  Traceback (most recent call last):
    File "error-example.py", line 7, in crasher2
      raise ValueError
  ValueError

Reading from the bottom up, the two exceptions each started in their respective tasks, then met and got bundled together into a MultiError, which propagated into the main task's nursery block, and then eventually up out of the call to trio.run.

Now when things go wrong, Trio shows you what you need to reconstruct what happened, and nothing else.

Comments

You can discuss this post on the Trio forum.

Next: Why I'm not collaborating with Kenneth Reitz
Previous: The unreasonable effectiveness of investment in open-source infrastructure