Repository navigation
Performance impact of async_hooks #181
Description
Activity
There's been a related discussion in nodejs/promises#31 earlier this year, which yielded a couple of interesting improvements and insights.
Pinging @nodejs/performance @nodejs/async_hooks
Reacted by Benjamin GruenbaumI'd like to use this forum to come to an agreement on what the concrete use cases are (at least estimate that), which aspects of the performance matter the most (i.e. promises created inside of C++ vs. in JS land), and what are useful benchmarks to drive the performance work in a meaningful way.
Particularly about async_hooks and promises or promises in general?
For async_hooks and promises - I think a server-application benchmark could be good here - with contexts through async functions through several layers of calls - I'd then listen to
promiseResolveand write interesting things about the app while sending it "user data" and measure how long it takes. "interesting things" can be things like how many times the database is hit, request latency for different request stages etc.For promises in general there are at least 2-3 good ideas in that thread that can get native promises finally faster than userland alternatives like bluebird. Namely skilling the microtask queue when possible and inlining calls - not to mention async iterators that aren't fast yet but could revolutionize programming in Node.js if they were.
Reacted by Benedikt MeurerI'd like to use this forum to come to an agreement on what the concrete use cases are (at least estimate that), which aspects of the performance matter the most (i.e. promises created inside of C++ vs. in JS land), and what are useful benchmarks to drive the performance work in a meaningful way.
Particularly about async_hooks and promises or promises in general?
Improving promise performance in general is also a good idea, but this particular issue is primarily about the impact of
async_hooks, i.e. about the performance cliff that you fall off when you enableasync_hooks.As for promises in general and in particular
async/awaitand async iterators, we will need good benchmarks as well to drive meaningful performance work. I've started to collect some ideas for promise performance improvements, but I don't feel like it's a good approach to measure our success in terms of the bluebird benchmarks.I think @mcollina had some comments about that before. Can we derive a simple representative performance test from fastify to get things going?
Specifically, regarding
PromiseHooksand performance I would like to see:kDestroyis added. I think in many cases it is straightforward to know when apromisecan no longer be refered too. Thus we could emit thedestroyhook much sooner, in other cases it would be fine to wait for garbage collection, but if V8 could instrument that for us directly it would be great.I've gotten many real-life complains from both APM providers and other
async_hooksusers that they experience a "memory leak". The reality is, that there is no memory leak but that it unintuitive that thedestroyevent is not emitted as soon as possible but rather at garbage collection.I'm guessing this would improve performance a lot, as we would no longer have to wrap the promise object itself to get notified about garbage collection. Essentially we could just create an unreferenced basic object on
kInitand set then just set theasyncIdandtriggerAsyncId(two doubles) on the internal fields.
Extra feature requests that are not performance related.
-
Know the
resolvevalue inkResolve. [WIP] feat(async): fix #740, use async hooks to wrap nodejs async api angular/zone.js#795 currently needs this. -
Don't break
executionAsyncId()andtriggerAsyncId()whenasync_hooksis not enabled. This might be very theoretical. It would involve enablingPromiseHooksunconditionally. If we also hadkDestroywe would only need to set two doubles on the internal fields.
/cc @addaleax
Reacted by Anna Henningsen, Benedikt Meurer, Benjamin Gruenbaum, Matteo Collina, Sathya Gunasekaran and Kaid Wong-
@AndreasMadsen Thanks for the feedback, that's very interesting. These seem to be more on the feature request side.
I think in many cases it is straightforward to know when a promise can no longer be refered too.
Do you have a suggestion/intuition here how to do that?
I'm guessing this would improve performance a lot, as we would no longer have to wrap the promise object itself to get notified about garbage collection. Essentially we could just create an unreferenced basic object on kInit and set then just set the asyncId and triggerAsyncId (two doubles) on the internal fields.
So you're saying that by changing the API and adding machinery to implement
kDestroyon the V8 side we should be able to improve performance significantly?Do you have a suggestion/intuition here how to do that?
Often promises are only "referenced" once, either by
.then()chaining or they are returned and used inawait. At least for the user, it is quite obvious that the promise can't be referenced after that.function wait(ret, ms) { // hint: the promise is directly returned return new Promise(function (resolve, reject) { setTimeout(() => resolve(ret), ms); }); } function main() { // hint: the promise is directly used in `await` without being assigned to a variable await wait(1, 10) // after this line the user expects the `destroy` hook to emit await wait(2, 10) // same for this promise await wait(3, 10) // same for this promise }
So you're saying that by changing the API and adding machinery to implement
kDestroyon the V8 side we should be able to improve performance significantly?Yes, that would be my guess. Wrapping and unwrapping the
promiseobject is definitely the most expensive part of ourPromiseHooksintegration. Not having to do that, should improve performance.Reacted by Benjamin Gruenbaumcc @jaro-sevcik
Often promises are only "referenced" once, either by .then() chaining or they are returned and used in await. At least for the user, it is quite obvious that the promise can't be referenced after that.
If V8 could detect that with escape analysis and optimize the return value of that function to an entirely different implementation that would be awesome. Fast libraries all hack this by having the single listener case optimized - but an engine that can prove it and avoid allocating extra stuff would be awesome. In terms of promise hooks I suspect it would also simplify things a lot for the common case.
On the benchmarks side, we can also consider Hapi v17, which is completely async-await based.
The problem with real-world code is that it typically involves a database, and that makes it very hard to measure small improvements. IMHO we should measure:
- receive a request, performs a query in the DB, returns value as JSON
- receive a request, performs a query in the DB, performs another query in the DB, returns a JSON
- receive a request, performs a query in the DB, send an HTTP request somewhere (maybe using fetch), returns a JSON
I fear this would have to be a synthetic application.
Reacted by Benedikt MeurerI fear this would have to be a synthetic application.
This is what Doxbee did (the bluebird benchmark suite) which you're familiar with - we can fork and extend it.
@benjamingr I think we are all more concern on overall impact of async await rather than a comparative measurement between implementations. Plus, I think we should focus on overall performance including a web framework, a database and HTTP requests rather than just the promise/async await layer.
Reacted by Kaushik VaranasiReacted by Kaushik Varanasi26 remaining items
Accidentally commented on the TSC thread instead of this one. So for future reference, I also did a test run of simple
hapiandkoaservers (using @mcollina'sautocannontester), again with and withoutasync_hooksenabled to get more real-worldish numbers (the Promise benchmarks arguably really stress promises pretty heavily). The results were pretty interesting (with latest Node 9.4.0):
The
koatest is super flaky, so the performance difference could also be noise, but forhapi, which makes heavy use ofasync/await, there's pretty consistent 30% performance drop with just an emptyinithook. See bmeurer/async-hooks-performance-impact for the benchmarks and additional information.And to provide even more data for the discussion: The performance drop also increases with the number of hooks being used (maybe not unsurprising). For example for the
Promisebenchmarks, we see additional performance regressions:
Very interesting data @bmeurer, thanks for the insights on this subject. We definitely hit the
with async_hooks (all)scenario since the current domains implementation on top ofasync_hooksregisters all handlers. That's more in line with what we've seen in production.async/await though will always use native promises.
Does this hold true even even when
await Promise.all([p1, p2])whereconst Promise = require('bluebird')?Does this hold true even even when
await Promise.all([p1, p2])whereconst Promise = require('bluebird')?In that case you'll probably use bluebird promises for the
Promise.alland native promises for theawait, which I guess is not what you want. The ES2017 spec forcesawaitandasync functionto always use native promises (which I consider a good thing FWIW). I think @gsathya brought this up earlier.Reacted by Yahor SiarheyenkaLet me know whether it is not relevant and I should delete my post, but I experienced quite a bad increase in CPU usage when I activated
async_hooks(Node 8 and 11) - see https://theholyjava.wordpress.com/2018/11/01/beware-the-performance-cost-of-async_hooks-node-8/ for details.I would recommend to check out the latest Node 8, Node 10 and Node 11, as the runtimes that are used in the blog post are quite old, or not supported anymore (Node 9). There should have been some improvement in the area.
Overall
async_hooksare very costly, especially with promises.- Thanks. I did try the latest Node 11 (the last graph in the blog post) with no difference.…On Thu, Nov 1, 2018, 11:10 AM Matteo Collina ***@***.***> wrote: I would recommend to check out the latest Node 8, Node 10 and Node 11, as the runtimes that are used in the blog post are quite old, or not supported anymore (Node 9). There should have been some improvement in the area. Overall async_hooks are *very* costly, especially with promises. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#181 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAmJPg_nTlHJe-Qr1xFoG-rEVv8Fq2rGks5uqsilgaJpZM4REc5B> .
Tried it on Node 12.2 on Windows 10 U. Still costly.
regular Bluebird-doxbee: 126 ms. init Bluebird-doxbee: 551 ms. full Bluebird-doxbee: 707 ms. regular Bluebird-parallel: 170 ms. init Bluebird-parallel: 1048 ms. full Bluebird-parallel: 1316 ms. regular Wikipedia: 334 ms. init Wikipedia: 2129 ms. full Wikipedia: 2621 ms. regular hapiserver: 5477.2 reqs. init hapiserver: 3266.6 reqs. full hapiserver: 2727.3 reqs. regular koaserver: 8231.8 reqs. init koaserver: 7309.6 reqs. full koaserver: 6771.1 reqs.I would recommend to check out the latest Node 8, Node 10 and Node 11, as the runtimes that are used in the blog post are quite old, or not supported anymore (Node 9). There should have been some improvement in the area.
Overall
async_hooksare very costly, especially with promises.@mcollina
Can you shortly explain the reasons for the substantial performance impact?Is it possible to reduce performance impact of async_hooks in the future (in future Node.js versions)? Or the performance will be always so bad because of the nature of async_hooks and how they work?
@alekbarszczewski it is mostly blocked on people doing the product work and suggesting concrete improvements.
If anyone can provide pointers of where to start to try and fix this, and/or a tl;dr of why it's so slow, I can try to look into it.
@rochdev my suggestion is to come to the next https://github.com/nodejs/diagnostics meeting. We are looking for a Champion to help with moving async hooks forward.
As per the calendar the next meeting is scheduled for Oct 9 (next wed) and an issue will be opened in the repo next Monday with the meeting links etc.
Reacted by Benjamin GruenbaumI definitely don't expect that it will be an easy task, but it's definitely an issue that is severely impacting APM vendors. I don't know if I'll be able to put a lot of time into it, at least short term, but I'd like to familiarize myself with the issue and how
async_hookscurrently works in general.I'll try to join the next Diagnostics meeting. Thanks for the tip!
Reacted by Benjamin GruenbaumI won't be able to make it to this week WG meeting. I'll make sure I stay available for the next one.
I've already started a related discussions on Twitter earlier here, but Twitter doesn't scale for this kind of discussion, so I thought I should move it here, so it's easier to follow the discussion. For background: @jasnell approached me at NodeConfEU in November 2017, asking for help on the performance of
async_hookson the V8 side, especially when it comes to the Promise lifecycle hooks. And I've talked briefly with @thlorenz about it during the conference (although I have to admit that I was using the wrong terminology and thus kinda ruined the conversation, sorry). And we had chatted briefly about this as well with @trevnorris during one of the CTC-V8 calls.Since Promise based APIs are likely becoming a (big) thing for Node 10 and at the same time there are estimates (i.e. by @ofrobots) that by the end of next year every Node server in enterprise is going to run with
async_hooks, we should start a discussion about this early one to stay ahead of the problem before it becomes a real problem for our users.In the last year @gsathya and @caitp already spent a lot of effort optimizing promises in V8 in general, so we're in a pretty good position wrt. baseline performance!
I'd like to use this forum to come to an agreement on what the concrete use cases are (at least estimate that), which aspects of the performance matter the most (i.e. promises created inside of C++ vs. in JS land), and what are useful benchmarks to drive the performance work in a meaningful way. I think we need both a set of micro-benchmarks to drive certain, in addition to real-world code that makes heavy use of promises, i.e. koa.js or fastify servers maybe? I feel that these benchmarks are most important, otherwise we might easily sink a lot of effort into work that doesn't help real-world use cases in the end. That's why I opened the issue on the benchmarking WG.
Comments and contributions are very welcome!