V8 Zero-cost async stack traces

TL;DR This document describes a new mechanism to provide (limited) async stack traces without adding any additional (runtime) overhead to the V8 engine, which only works as long as the user sticks to async functions (i.e. it won’t help with Promise only code).

Project considered finished as of Nov ’18. Shipping enabled by default is blocking on the –harmony-await-optimization flag, but on track for the V8 7.3 release.

Motivation

When programming with async functions, which have recently been added to JavaScript with the ES2017 revision, developers are currently facing the problem that the (non-standard) Error.stack property in V8 only provides a truncated stack trace up to the most recent await. For example consider this snippet:

async function foo(x) {
  await bar(x);
}

async function bar(x) {
  await x;
  throw new Error("Let's have a look...");
}

foo(1).catch(e => console.log(e.stack));

When you execute this in V8 – i.e. in Chrome or Node.js – you’ll see something like this on the console:

Error: Let's have a look...
    at bar (<anonymous>:7:9)

So foo is completely missing from the stack trace, and only bar is shown. This makes sense from the JSVMs perspective, since the function bar suspended execution previously on the await x and was later resumed from the microtask queue (i.e. basically from the event loop), which means that there’s no other function on the stack underneath it now.

Now V8 has been offering support for so-called async stack traces via the Chrome DevTools inspector, which fills the gap here (during development):

You can see the async stack trace showing foo properly on the right (in the “Call Stack” section). This feature also works with code that uses Promises directly or Web APIs like setTimeout() not just for async functions. And it works in Node.js when the inspector is attached. There are two problems with this feature though:

1. It comes with non-trivial overhead.
2. It requires the inspector to run.

Both are show-stoppers here, since async stack traces are an important production feature (companies like Netflix completely ban async/await until the stack trace issue is resolved). Without async stack traces in production it’s really hard to make sense of errors that were logged during execution. While it would be possible to also enrich Error.stack based on the inspector feature, it’d come with significant overhead still, aka it’s not going to be for free (ever).

Solution

The solution outlined here utilizes the fact that the promise chains contain sufficient information to continue execution. For await the suspend and resume points coincide and so we not only know where we would continue, but by coincidence we also know where we came from. So we can utilize the information found in the promise chains, specifically the await continuations, to reconstruct a stack trace that contains all the async functions.

Prerequisites

This however will only work as long as the promise chain has sufficient information at the point where we construct the stack trace (i.e. at the point where the error is thrown). In the case where one promise is resolved with another promise (i.e. returning a promise from a fulfill handler), it doesn’t leave any immediate breadcrumbs in the promise chain, but only after the next tick (when the PromiseResolveThenableJob ran). The way that ES2017 specifies await is problematic here for that exact reason, as await x essentially turns into something like this:

const .promise = @createPromise();
@resolvePromise(.promise, x);
const .throwaway = @createPromise();
@performPromiseThen(.promise,
  res => @resume(.generator_object, res),
  err => @throw(.generator_object, err),
  .throwaway);
@yield(.generator_object, .outer_promise);

Specifically await x always creates a wrapper .promise and resolves that with x. That means that if x is already a promise, there’ll be no direct connection between the x and the .generator_object for the async function, until after the next tick when the PromiseResolveThenableJob chains x and .promise, which in turn holds the handler to resume the .generator_object. So this approach would not work at zero cost, we’d either need to register additional links between the  x and .promise, or harvest the microtask queue(s) to check if there’s some PromiseResolveThenableJob waiting to run.

Fortunately there’s a related recent ECMAScript specification change (tc39/ecma262#1250) by mslekova@, which changes await to operate like this instead:

const .promise = @promiseResolve(x);
const .throwaway = @createPromise();
@performPromiseThen(.promise,
  res => @resume(.generator_object, res),
  err => @throw(.generator_object, err),
  .throwaway);
@yield(.generator_object, .outer_promise);

This change is implemented in V8 behind the –harmony-await-optimization flag. It changes the way await x works by not wrapping x into another promise if x is already a (native) promise. So now the resume handlers for the .generator_object are registered on x directly and we can find them in the promise chains immediately.

Overview

So assuming that await is changed as mentioned above, we are now able to walk up the promise chains and extract relevant information about await sites. Looking at the example mentioned earlier

async function foo(x) {
  await bar(x);
}

async function bar(x) {
  await x;
  throw new Error("Let's have a look...");
}

foo(1).catch(e => console.log(e.stack));

at runtime at the point where the Error is thrown, the VM state looks like this:

On the Call stack we see that we are currently executing bar, which was called from the Await Fulfilled Function. This happened as the result of executing the PromiseReactionJob for the wrapper promise for x (it got wrapped since we passed 1).

So what we can do here is to build the regular stack trace, which will just include bar here, and then, if async stack traces are enabled, peak into the “current microtask” (introduced in this CL), and if that’s a reaction job for await, find the corresponding generator object for the async function. This generator object holds a reference to the promise that corresponds to the async function (as of this CL).

Alternative solution

We also considered an alternative solution where the stack frame for every async function would provide a way to get to its outer promise (via a special .promise variable in the scope). Looking at the relevant data structures in the example above, after the await x in function bar, we find the following:

Each async function has a so-called outer promise which is the JSPromise object that is eventually fulfilled with the value produced by the async function. This special promise is allocated to an internal variable .promise (desugared in Parser and BytecodeGenerator). So when we find that an async function frame is the bottom-most accessible frame (i.e. underneath it there’s only the internal frames for the microtask handling), we can dig into this .promise (which is a pending promise since the async function didn’t finish yet and so it cannot be fulfilled or rejected already) to find its reactions.

In case of async/await only, we should find a PromiseReaction there, whose fulfill_handler is a special closure (see Await Fulfilled Functions in the specification), and whose reject_handler is another special closure (see Await Rejected Functions). These  closures point to a shared AwaitContext (introduced with this change), whose extension slot holds the JSGeneratorObject of the async function that’s currently awaiting on this JSPromise (function foo in this case). So we already discovered that bar was asynchronously called from foo.

Now we can continue with the same logic by looking at the register file of the suspended JSGeneratorObject, which again has a special .promise slot. This way we can build up the async stack frames up to the outermost async function by just utilizing the information that is already present in the system (i.e. without adding any new tracking or shadow stacks).

Arbitrary Promise chains

Due to the zero-cost aspect, we can only support a very limited set of async stack frames here. Specifically the async stack frames shown will always correspond to things that V8 manages itself (i.e. an await in an async function, or maybe a suspended execution inside of a known builtin like Promise.all or Promise.race). For arbitrary promise chains we can still follow the links in the promise chains (as long as there’s only a single link and as long as the developer sticks to using native promises); so something like

async function foo(x) {
  await bar(x).then(y => y);
}

async function bar(x) {
  await x;
  throw new Error("Let's have a look...");
}

foo(1).catch(e => console.log(e.stack));

is still going to work in that it will include both foo and bar in Error.stack. In this case the internals of the VM look like this at the point where the Error is thrown.

We can essentially skip over arbitrary promises in the promise chain (linked via the promise_or_capability field of the PromiseReactions), but we only include await sites in the stack trace, since we don’t know how this other promises ended up the promise chains (it could be because of a direct call to Promise.prototype.then() like in this example, but it could also originate from some other Promise code).

But this is none the less limited to async functions. For example changing foo to an equivalent implementation using only promises is not going to work.

function foo(x) {
  return bar(x).then(y => undefined);
}

Here you won’t see foo in the stack trace anymore.

Await Optimization

As mentioned before, the mechanism described earlier depends on a ECMAScript specification change (tc39/ecma262#1250, explainer) by Maya Lekova, which is flagged by the –harmony-await-optimization in V8 currently (intent to ship). Without this change, there’ll be no reactions on the .promise at the point when the stack trace is constructed, as await instead creates a temporary promises and resolves that with the .promise.

How to deal with multiple reactions?

So what happens when we find a JSPromise that has multiple reactions on it? For example when the same JSPromise was passed to multiple awaits? In that case, there’s no single chain of events and so it’s not really possible to build a meaningful (async) stack trace since we don’t know which branch of the tree corresponds to the actual chain of calls; hence we give up when we find a JSPromise that has no or more than one reaction on it.

Implementation Details

This section discusses some of the details for the implementation that need to be sorted.

How to get to .promise?

There are multiple ways how the stack trace construction machinery could get to the .promise of an async function. The obvious solution would be to parse the function again and determine the stack slot index from the DeclarationScope::promise_var(). This would be pretty solid and easy to implement, but comes at a high cost. Another way would be to extend the ScopeInfo with the index of the .promise variable (in case of async functions); but that comes with an additional memory cost, plus the ScopeInfo is already pretty complex.

Toon Verwaest suggested to just forcibly allocate the .promise variable to stack slot 0 – in case of async functions – during scope resolution. This way the stack trace construction logic just needs to reach out to the first stack slot (either from the stack frame, potentially going through the Deoptimizer for optimized code, or from the register file of a suspended generator). For the sake of simplicity we’ll go with this approach for now.

Async Generators

For async generators there’s no such thing as the .promise that we have for async functions. Instead there’s a new promise for every async request. So for async generators we need to get to the .generator variable instead – a JSAsyncGeneratorObject – and load the promise for the current async request.

So for async generators we could forcibly allocate the .generator variable to the stack slot 0.

Error.stack

The Error.stack property is a non-standard extension that has been available in V8 (and other engines) for a long time. A lot of software depends on this extension (especially in the Node.js world). Initially we plan to just enrich Error.stack with async stack traces when enabled. We will introduce a mechanism to mark the async part of Error.stack by prepending async to the function name (this is orthogonal to the actual implementation of this feature). An async stack trace for the example above will not look like this:

Error: Let’s have a look…

    at bar (<anonymous>)
    at async foo (<anonymous>)

Related to this: There’s a tc39/proposal-error-stacks proposal, which is currently at stage 1 of the process, to standardize Error.prototype.stack.

Error.prepareStackTrace support

In order to support the non-standard Error.prepareStackTrace() API, we expose an additional CallSite.prototype.isAsync() function that tells for any given call site whether it was an asynchronously appended frame.

Promise.all / Promise.race

We might also want to support special builtins here, which have special promise chains that allow us to show corresponding frames in the async call stack. The builtin that comes to mind here is Promise.all. There’s also Promise.race, but that doesn’t leave breadcrumbs in the promise chain, so it’s not going to be detectable at zero cost.

Shipping

We intend to ship this behind a new flag –async-stack-traces, which will imply the –harmony-await-optimization flag for reasons described above.

Results

Initial prototype shows the new flag in action, which goes up to the outermost async function on the (async) call stack.

Resources

• Faster async functions and promises (Maya Lekova & Benedikt Meurer)
• holding on to your performance promises (Maya Lekova & Benedikt Meurer)
• Asynchronous Stack Traces in Node.js (Rod Vagg)

Short link: bit.ly/v8-zero-cost-async-stack-traces
Bug: v8:7522, nodejs/node#11865