Repository navigation
Worker threads: ability for tooling to pass data to workers and inject NODE_OPTIONS #30992
Description
Is your feature request related to a problem? Please describe.
nyc 15 is moving away from spawn-wrap, will use process-on-spawn and node-preload in it's place. These two modules ensure nyc gets properly loaded into new processes, but they do not help with worker threads (note spawn-wrap also did nothing to help with worker threads). I'd like to add worker thread support to nyc.
Describe the solution you'd like
nyc needs to pass data to worker threads without requiring modifications to arguments of new Worker(...) run by the code being tested. An idea I have:
process.on('worker', options => {
options.platformData('nyc', nycConfigClonableObject);
options.nodeOptions.push('--require', require.resolve('./lib/wrap-thread.js'));
});The above code assumes that new Worker() internally calculates options.nodeOptions based on the env.NODE_OPTIONS passed to it (see also #29117). This would allow nyc to inject preload modules if necessary without altering the process.env of the new thread.
Describe alternatives you've considered
Setting platform data during process startup could work for some use cases:
const threads = require('worker_threads');
threads.platformData('nyc', dataNeededToInitializeNYC);
threads.platformNodeOptions.push('--require', require.resolve('./wrap-thread.js')));This would work for nyc but maybe another tool would need to calculate the object needed for platformData at the time of new Worker(). This is why my proposal is for 'worker spawn' hooks to ensure this is useful for more than just nyc. I'm also unsure how platformNodeOptions would be incorporated into the env.NODE_OPTIONS given to new Worker().
I considered patching the worker_threads core module. worker.SHARE_ENV is very problematic for this approach as it eliminates the possibility of using environmental variables to get nyc options into the worker threads. We would likely need to patch new Worker() to wrap worker data so that options.workerData = {userData: options.workerData, nycData}, then require('worker_threads').workerData would be patched to return the userData subproperty and we'd create a require('worker_threads').nycData export. This seems like it would be very messy, especially if any other tooling had a need to patch workerData in a similar way. I'm really not interested in this approach even if it could be forced to work.
Activity
bmeck commentedon Jan 6, 2020
It seems like the event listener approach would at least allow some censoring/protecting of options? You could use Object.freeze etc on the options?
coreyfarrell commentedon Jan 7, 2020
@bmeck I prefer the event listener approach since it allows calculation of data and NODE_OPTIONS to happen when the thread is being created instead of needing to know those values way ahead of time, I'm not sure it helps give any special protection to the options that are specified. One drawback of using the events API is that
process.removeAllListeners('worker')could remove my hook. An alternative to the event listener approach would be an API similar toasync_hookswhere the worker hook would only be removable by the code which added it.I'm not sure
optionsis actually the best name for the argument either, I'm very open to suggestions for how to best structure the API. Maybecontextwould be a better name since I was really thinking this would be a class instance with methods to manage data and node options.Given the PR to run loader hooks in a worker thread I have to ask how you see this interacting with loader hooks? I assume loader hooks thread will always start before any
--require's are preloaded and those preloads would not be run inside the loader worker thread? Sorry if this is answered in the PR I really haven't had a chance to dig into the code yet.bmeck commentedon Jan 7, 2020
That is my presumption yes.
worker: add setEnvironmentData/getEnvironmentData
worker: add setEnvironmentData/getEnvironmentData
worker: add setEnvironmentData/getEnvironmentData
github-actions commentedon Mar 4, 2022
There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.
For more information on how the project manages feature requests, please consult the feature request management document.
github-actions commentedon Apr 4, 2022
There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.
For more information on how the project manages feature requests, please consult the feature request management document.