pax_global_header 0000666 0000000 0000000 00000000064 15221147774 0014524 g ustar 00root root 0000000 0000000 52 comment=601124cf86602c1ac88c273a8f99e9c40bea0f16
josdejong-workerpool-601124c/ 0000775 0000000 0000000 00000000000 15221147774 0016151 5 ustar 00root root 0000000 0000000 josdejong-workerpool-601124c/.browserslistrc 0000664 0000000 0000000 00000000006 15221147774 0021235 0 ustar 00root root 0000000 0000000 IE 10
josdejong-workerpool-601124c/.github/ 0000775 0000000 0000000 00000000000 15221147774 0017511 5 ustar 00root root 0000000 0000000 josdejong-workerpool-601124c/.github/workflows/ 0000775 0000000 0000000 00000000000 15221147774 0021546 5 ustar 00root root 0000000 0000000 josdejong-workerpool-601124c/.github/workflows/ci.yml 0000664 0000000 0000000 00000000617 15221147774 0022670 0 ustar 00root root 0000000 0000000 name: Node.js CI
on: [push, pull_request]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [22.x, 24.x, 26.x]
steps:
- uses: actions/checkout@v7
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v6
with:
node-version: ${{ matrix.node-version }}
- run: npm ci
- run: npm test
josdejong-workerpool-601124c/.gitignore 0000664 0000000 0000000 00000001072 15221147774 0020141 0 ustar 00root root 0000000 0000000 /types
# Logs
logs
*.log
# Runtime data
pids
*.pid
*.seed
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
dist
# Dependency directory
# Deployed apps should consider commenting this line out:
# see https://npmjs.org/doc/faq.html#Should-I-check-my-node_modules-folder-into-git
node_modules
# IDE settings
.idea
josdejong-workerpool-601124c/.mocharc.js 0000664 0000000 0000000 00000000140 15221147774 0020174 0 ustar 00root root 0000000 0000000 'use strict';
module.exports = {
timeout: 10000,
'forbid-only': Boolean(process.env.CI)
};
josdejong-workerpool-601124c/.npmignore 0000664 0000000 0000000 00000000757 15221147774 0020161 0 ustar 00root root 0000000 0000000 # Logs
logs
*.log
# Runtime data
pids
*.pid
*.seed
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
# Dependency directory
# Deployed apps should consider commenting this line out:
# see https://npmjs.org/doc/faq.html#Should-I-check-my-node_modules-folder-into-git
node_modules
# IDE settings
.idea
# other
.npmignore
.travis.yml
josdejong-workerpool-601124c/HISTORY.md 0000664 0000000 0000000 00000031530 15221147774 0017636 0 ustar 00root root 0000000 0000000 # workerpool history
https://github.com/josdejong/workerpool
## 2026-07-01, version 10.0.3
- Fix: #548 treat worker as busy while cleaning up canceled task (#549).
Thanks @joshkel.
## 2026-04-16, version 10.0.2
- Fix: #542 clean up listeners when abort triggers. Thanks @joshLong145.
## 2025-11-19, version 10.0.1
- Fix: #523 make `WorkerHandler` resilient against errors without `message`
property.
## 2025-10-21, version 10.0.0
BREAKING CHANGES:
- Feat: implement `queueStrategy` option (FIFO / LIFO / custom) (#518, #517).
This is a new feature without changes in the public API, but the internal
property `pool.tasks` is changed into `pool.taskQueue`. So, changes are
required if you rely on this internal property. Thanks @amaneru55
- Feat: introduce a new `TerminateError` class to differentiate between a
crash and a regular error (see #519, #520). The `TerminateError` class
extends from the class `Error` so no breaking changes in that regard.
But some of the error messages are changed so if you have code that depends
on the exact error messages you may need to change your code accordingly.
Thanks @Julusian.
## 2025-09-10, version 9.3.4
- Fix: #516 improve error handling for errors containing nested classes by
using their `.toJSON` method when available.
## 2025-06-27, version 9.3.3
- Fix: #479 terminate worker even if abortListener resolved (#507).
Thanks @joshLong145.
## 2025-05-29, version 9.3.2
- Fix: #505 npm package missing `/src/generated/` files.
## 2025-05-29, version 9.3.1
- Fix: #505 npm package missing `/src/generated/` files.
## 2025-05-28, version 9.3.0
- Feat: support for events and std streams to parents from an
abort handler (#478). Thanks @joshLong145.
- feat: improve compatibility with standard Promise type (#504).
Thanks @joshkel.
## 2024-10-11, version 9.2.0
- Feat: implement support for abort handlers in the workers (#448).
Thanks @joshLong145.
- Feat: implement `Promise.finally()` on the custom promise class (#388).
Thanks @joshLong145 and @wmertens.
- Fix #460: `vite` example not working (#461). Thanks @datashaman.
## 2024-06-28, version 9.1.3
- Fix transferring from the worker (#455). Thanks @LeonidPolukhin.
## 2024-06-04, version 9.1.2
- Fix: #446 ensure the library can be used with older browsers.
## 2024-04-06, version 9.1.1
- Fix: events not emitted when using (#439). Thanks @t0oF-azpn.
## 2024-01-18, version 9.1.0
- Support capturing the stdout/stderr streams from the workers (#425, #423).
Thanks @cpendery.
## 2023-12-22, version 9.0.4
- Improve the logic to detect the Node.js platform (#421). Thanks @tamuratak.
## 2023-12-20, version 9.0.3
- Improve types: unwrap Promise types (#419). Thanks @joshkel.
## 2023-12-19, version 9.0.2
- Export the `Proxy` type (see #417). Thanks @tamuratak.
- Use workerpool's `Promise` type definition (see #417). Thanks @tamuratak.
## 2023-12-18, version 9.0.1
- Export the types `Pool`, `WorkerPoolOptions`, and `WorkerRegisterOptions`,
see #416. Thanks @joshkel.
## 2023-12-18, version 9.0.0
BREAKING CHANGE: the library now comes with TypeScript type definitions
included. There may be minor differences between the new, embedded type
definitions and the implementation from `@types/workerpool` that was needed
before. For example, `type WorkerPool` is called `type Pool` now.
- Generate TypeScript types from JSDoc comments. Thanks @tamuratak.
## 2023-10-25, version 8.0.0
BREAKING CHANGE: the library now throws an Error when passing unknown or
inherited properties for `workerOpts`, `workerThreadOpts` and `forkOpts`.
- Fix: throw an error in case of unknown properties or inherited properties
in `workerOpts`, `workerThreadOpts` and `forkOpts` to protect against
security issues related to prototype pollution
(see ea5368c5e53d97120b952ffad151a318c1ff073c).
- Fix: #56 document the return type of all functions.
- Docs: added a full usage example with Vite and Webpack5 (#408),
thanks @KonghaYao.
## 2023-10-25, version 7.0.0
BREAKING CHANGE: The setup to bundle `workerpool` has been replaced. This should
be a drop-in replacement, but it may have impact depending on your setup.
- Switched build setup from Webpack to Rollup, see #403. Thanks @KonghaYao.
## 2023-10-11, version 6.5.1
- Fix: `workerThreadOpts` not working when `workerType: auto`, see #357.
## 2023-09-13, version 6.5.0
- Implement support for passing options to web workers constructors (#400,
#322). Thanks @DonatJR.
## 2023-08-21, version 6.4.2
- Fix: a bug in the timeout of termination (#395, #387). Thanks @Michsior14.
## 2023-08-17, version 6.4.1
- Fix: worker termination before it's ready (#394, #387). Thanks @Michsior14.
## 2023-02-24, version 6.4.0
- Support transferable objects (#3, #374). Thanks @Michsior14.
- Implement a new callback `onTerminate` at the worker side, which can be used
to clean up resources, and an option `workerTerminateTimeout` which forcefully
terminates a worker if it doesn't finish in time (#353, #377).
Thanks @Michsior14.
- Pass `workerThreadOpts` to the `onTerminateWorker` callback (#376).
Thanks @Michsior14.
## 2022-11-07, version 6.3.1
- Fix #318: debug ports not being released when terminating a pool.
## 2022-10-24, version 6.3.0
- Implement option `workerThreadOpts` to pass options to a worker of type
`thread`, a `worker_thread` (#357, fixes #356). Thanks @galElmalah.
## 2022-04-11, version 6.2.1
- Fix #343: `.terminate()` sometimes throwing an exception.
## 2022-01-15, version 6.2.0
- Implement callbacks `onCreateWorker` and `onTerminateWorker`. Thanks @forty.
- Fix #326: robustness fix when terminating a workerpool.
## 2021-06-17, version 6.1.5
- Fix v6.1.4 not being marked as latest anymore on npm due to bug fix
release v2.3.4.
## 2021-04-05, version 6.1.4
- Fix terminating a pool throwing an error when used in the browser.
Regression introduced in `v6.1.3`.
## 2021-04-01, version 6.1.3
- Fix #147: disregard messages from terminated workers.
Thanks @hhprogram and @Madgvox.
## 2021-03-09, version 6.1.2
- Fix #253, add `./src` again in the published npm package, reverting the change
in `v6.1.1` (see also #243).
## 2021-03-08, version 6.1.1
- Remove redundant `./src` folder from the published npm package, see #243.
Thanks @Nytelife26.
## 2021-01-31, version 6.1.0
- Implemented support for sending events from the worker to the main thread,
see #51, #227. Thanks @Akryum.
- Fix an issue in Node.js nightly, see #230. Thanks @aduh95.
- Fix #232 `workerpool` not working on IE 10.
## 2021-01-16, version 6.0.4
- Make evaluation of offloaded functions a bit more secure by using
`new Function` instead of `eval`. Thanks @tjenkinson.
## 2020-10-28, version 6.0.3
- Fixes and more robustness in terminating workers. Thanks @boneskull.
## 2020-10-03, version 6.0.2
- Fix #32, #175: the promise returned by `Pool.terminate()` now waits until
subprocesses are dead before resolving. Thanks @boneskull.
## 2020-09-23, version 6.0.1
- Removed examples from the npm package. Thanks @madbence.
## 2020-05-13, version 6.0.0
BREAKING CHANGE: the library entry points are changed and new source maps are
added. This may have impact on your project depending on your setup.
- Created separate library entry points in package.json for node.js and browser.
Thanks @boneskull.
- Generated source maps for both minified and non-minified bundles.
- Removed deprecation warnings for `options.nodeWorker` (renamed to
`options.workerType`) and `pool.clear()` (renamed to `pool.terminate()`).
## 2019-12-31, version 5.0.4
- Fixed #121: `isMainThread` not working when using `worker_threads`.
- Drop official support for node.js 8 (end of life).
## 2019-12-23, version 5.0.3
- Fixed library not working in the browser. See #106.
## 2019-11-06, version 5.0.2
- Fixed environment detection in browser. See #106. Thanks @acgrid.
## 2019-10-13, version 5.0.1
- Fixed #96: WorkerPool not cancelling any pending tasks on termination.
## 2019-08-25, version 5.0.0
- Deprecated option `nodeWorker` and created a new, more extensive option
`workerType` giving full control over the selected type of worker.
Added new option `'web'` to enforce use a Web Worker. See #85, #74.
- In a node.js environment, the default `workerType` is changed from
`'process'` to `'thread'`. See #85, #50.
- Improved detection of environment (`browser` or `node`), fixing wrong
detection in a Jest test environment. See #85.
## 2019-08-21, version 4.0.0
- Pass argument `--max-old-space-size` to child processes. Thanks @patte.
- Removed redundant dependencies, upgraded all devDependencies.
- Fixed Webpack issues of missing modules `child_process` and `worker_threads`.
See #43.
- Bundled library changed due to the upgrade to Webpack 4. This could possibly
lead to breaking changes.
- Implemented new option `maxQueueSize`. Thanks @colomboe.
- Fixed exiting workers when the parent process is killed. Thanks @RogerKang.
- Fixed #81: Option `minWorkers: 'max'` not using the configured `maxWorkers`.
- Fixed not passing `nodeWorker` to workers initialized when creating a pool.
Thanks @spacelan.
- Internal restructure of the code: moved from `lib` to `src`.
## 2019-03-12, version 3.1.2
- Improved error message when a node.js worker unexpectedly exits (see #58).
Thanks @stefanpenner.
- Allocate debug ports safely, this fixes an issue cause workers to exit
unexpectedly if more then one worker pool is active, and the process is
started with a debugger (`node debug` or `node --inspect`).
Thanks @stefanpenner.
## 2019-02-25, version 3.1.1
- Fix option `nodeWorker: 'auto'` not using worker threads when available.
Thanks @stefanpenner.
## 2019-02-17, version 3.1.0
- Implemented support for using `worker_threads` in Node.js, via the new option
`nodeWorker: 'thread'`. Thanks @stefanpenner.
## 2018-12-11, version 3.0.0
- Enable usage in ES6 Webpack projects.
- Dropped support for AMD module system.
## 2021-06-17, version 2.3.4
- Backport fix for Node.js 16, see #309. Thanks @mansona.
## 2018-09-12, version 2.3.3
- Fixed space in license field in `package.json`. Thanks @sagotsky.
## 2018-09-08, version 2.3.2
- Add licence field to `package.json`. Thanks @greyd.
## 2018-07-24, version 2.3.1
- Fixed bug where tasks that are cancelled in a Pool's queue
causes following tasks to not run. Thanks @greemo.
## 2017-09-30, version 2.3.0
- New method `Pool.terminate(force, timeout)` which will replace
`Pool.clear(force)`. Thanks @jimsugg.
- Fixed issue with never terminating zombie child processes.
Thanks @jimsugg.
## 2017-08-20, version 2.2.4
- Fixed a debug issue: look for `--inspect` within argument strings,
instead of exact match. Thanks @jimsugg.
## 2017-08-19, version 2.2.3
- Updated all examples to neatly include `.catch(...)` callbacks.
## 2017-07-08, version 2.2.2
- Fixed #25: timer of a timeout starting when the task is created
instead of when the task is started. Thanks @eclipsesk for input.
## 2017-05-07, version 2.2.1
- Fixed #2 and #19: support for debugging child processes. Thanks @tptee.
## 2016-11-26, version 2.2.0
- Implemented #18: method `pool.stats()`.
## 2016-10-11, version 2.1.0
- Implemented support for registering the workers methods asynchronously.
This enables asynchronous initialization of workers, for example when
using AMD modules. Thanks @natlibfi-arlehiko.
- Implemented environment variables `platform`, `isMainThread`, and `cpus`.
Thanks @natlibfi-arlehiko.
- Implemented option `minWorkers`. Thanks @sergei202.
## 2016-09-18, version 2.0.0
- Replaced conversion of Error-objecting using serializerr to custom
implementation to prevent issues with serializing/deserializing functions.
This conversion implementation loses the prototype object which means that
e.g. 'TypeError' will become just 'Error' in the main code. See #8.
Thanks @natlibfi-arlehiko.
## 2016-09-12, version 1.3.1
- Fix for a bug in PhantomJS (see #7). Thanks @natlibfi-arlehiko.
## 2016-08-21, version 1.3.0
- Determine `maxWorkers` as the number of CPU's minus one in browsers too. See #6.
## 2016-06-25, version 1.2.1
- Fixed #5 error when loading via AMD or bundling using Webpack.
## 2016-05-22, version 1.2.0
- Implemented serializing errors with stacktrace. Thanks @mujx.
## 2016-01-25, version 1.1.0
- Added an error message when wrongly calling `pool.proxy`.
- Fixed function `worker.pool` not accepting both a script and options. See #1.
Thanks @freund17.
## 2014-05-29, version 1.0.0
- Merged function `Pool.run` into `Pool.exec`, simplifying the API.
## 2014-05-14, version 0.2.0
- Implemented support for cancelling running tasks.
- Implemented support for cancelling running tasks after a timeout.
## 2014-05-07, version 0.1.0
- Implemented support for both node.js and the browser.
- Implemented offloading functions.
- Implemented worker proxy.
- Added docs and examples.
## 2014-05-02, version 0.0.1
- Module name registered at npm.
josdejong-workerpool-601124c/LICENSE 0000664 0000000 0000000 00000026112 15221147774 0017160 0 ustar 00root root 0000000 0000000 Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright (C) 2014-2026 Jos de Jong wjosdejong@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
josdejong-workerpool-601124c/README.md 0000664 0000000 0000000 00000054402 15221147774 0017435 0 ustar 00root root 0000000 0000000 # workerpool
[](https://www.npmjs.com/package/workerpool)
[](https://npm-compare.com/workerpool/#timeRange=FIVE_YEARS)
[](https://github.com/josdejong/jsoneditor/pulse)
[](https://github.com/josdejong/workerpool/blob/master/LICENSE)
[](https://app.fossa.io/projects/git%2Bgithub.com%2Fjosdejong%2Fworkerpool?ref=badge_shield)
**workerpool** offers an easy way to create a pool of workers for both dynamically offloading computations as well as managing a pool of dedicated workers. **workerpool** basically implements a [thread pool pattern](http://en.wikipedia.org/wiki/Thread_pool_pattern). There is a pool of workers to execute tasks. New tasks are put in a queue. A worker executes one task at a time, and once finished, picks a new task from the queue. Workers can be accessed via a natural, promise based proxy, as if they are available straight in the main application.
**workerpool** runs on Node.js and in the browser.
## Features
- Easy to use
- Runs in the browser and on node.js
- Dynamically offload functions to a worker
- Access workers via a proxy
- Cancel running tasks
- Set a timeout on tasks
- Handles crashed workers
- Small: 9 kB minified and gzipped
- Supports transferable objects (only for web workers and worker_threads)
## Why
JavaScript is based upon a single event loop which handles one event at a time. Jeremy Epstein [explains this clearly](http://greenash.net.au/thoughts/2012/11/nodejs-itself-is-blocking-only-its-io-is-non-blocking/):
> In Node.js everything runs in parallel, except your code.
> What this means is that all I/O code that you write in Node.js is non-blocking,
> while (conversely) all non-I/O code that you write in Node.js is blocking.
This means that CPU heavy tasks will block other tasks from being executed. In case of a browser environment, the browser will not react to user events like a mouse click while executing a CPU intensive task (the browser "hangs"). In case of a node.js server, the server will not respond to any new request while executing a single, heavy request.
For front-end processes, this is not a desired situation.
Therefore, CPU intensive tasks should be offloaded from the main event loop onto dedicated _workers_. In a browser environment, [Web Workers](http://www.html5rocks.com/en/tutorials/workers/basics/) can be used. In node.js, [child processes](https://nodejs.org/api/child_process.html) and [worker_threads](https://nodejs.org/api/worker_threads.html) are available. An application should be split in separate, decoupled parts, which can run independent of each other in a parallelized way. Effectively, this results in an architecture which achieves concurrency by means of isolated processes and message passing.
## Install
Install via npm:
npm install workerpool
## Load
To load workerpool in a node.js application (both main application as well as workers):
```js
const workerpool = require('workerpool');
```
To load workerpool in the browser:
```html
```
To load workerpool in a web worker in the browser:
```js
importScripts('workerpool.js');
```
Setting up the workerpool with React or webpack5 requires additional configuration steps, as outlined in the [webpack5 section](examples%2Fwebpack5%2FREADME.md).
## Use
### Offload functions dynamically
In the following example there is a function `add`, which is offloaded dynamically to a worker to be executed for a given set of arguments.
**myApp.js**
```js
const workerpool = require('workerpool');
const pool = workerpool.pool();
function add(a, b) {
return a + b;
}
pool
.exec(add, [3, 4])
.then(function (result) {
console.log('result', result); // outputs 7
})
.catch(function (err) {
console.error(err);
})
.then(function () {
pool.terminate(); // terminate all workers when done
});
```
Note that both function and arguments must be static and stringifiable, as they need to be sent to the worker in a serialized form. In case of large functions or function arguments, the overhead of sending the data to the worker can be significant.
### Dedicated workers
A dedicated worker can be created in a separate script, and then used via a worker pool.
**myWorker.js**
```js
const workerpool = require('workerpool');
// a deliberately inefficient implementation of the fibonacci sequence
function fibonacci(n) {
if (n < 2) return n;
return fibonacci(n - 2) + fibonacci(n - 1);
}
// create a worker and register public functions
workerpool.worker({
fibonacci: fibonacci,
});
```
This worker can be used by a worker pool:
**myApp.js**
```js
const workerpool = require('workerpool');
// create a worker pool using an external worker script
const pool = workerpool.pool(__dirname + '/myWorker.js');
// run registered functions on the worker via exec
pool
.exec('fibonacci', [10])
.then(function (result) {
console.log('Result: ' + result); // outputs 55
})
.catch(function (err) {
console.error(err);
})
.then(function () {
pool.terminate(); // terminate all workers when done
});
// or run registered functions on the worker via a proxy:
pool
.proxy()
.then(function (worker) {
return worker.fibonacci(10);
})
.then(function (result) {
console.log('Result: ' + result); // outputs 55
})
.catch(function (err) {
console.error(err);
})
.then(function () {
pool.terminate(); // terminate all workers when done
});
```
Worker can also initialize asynchronously:
**myAsyncWorker.js**
```js
define(['workerpool/dist/workerpool'], function (workerpool) {
// a deliberately inefficient implementation of the fibonacci sequence
function fibonacci(n) {
if (n < 2) return n;
return fibonacci(n - 2) + fibonacci(n - 1);
}
// create a worker and register public functions
workerpool.worker({
fibonacci: fibonacci,
});
});
```
## Examples
Examples are available in the examples directory:
[https://github.com/josdejong/workerpool/tree/master/examples](https://github.com/josdejong/workerpool/tree/master/examples)
## API
The API of workerpool consists of two parts: a function `workerpool.pool` to create a worker pool, and a function `workerpool.worker` to create a worker.
### pool
A workerpool can be created using the function `workerpool.pool`:
`workerpool.pool([script: string] [, options: Object]) : Pool`
When a `script` argument is provided, the provided script will be started as a dedicated worker. When no `script` argument is provided, a default worker is started which can be used to offload functions dynamically via `Pool.exec`. Note that on node.js, `script` must be an absolute file path like `__dirname + '/myWorker.js'`. In a browser environment, `script` can also be a data URL like `'data:application/javascript;base64,...'`. This allows embedding the bundled code of a worker in your main application. See `examples/embeddedWorker` for a demo.
The following options are available:
- `minWorkers: number | 'max'`. The minimum number of workers that must be initialized and kept available. Setting this to `'max'` will create `maxWorkers` default workers (see below).
- `maxWorkers: number`. The default number of maxWorkers is the number of CPU's minus one. When the number of CPU's could not be determined (for example in older browsers), `maxWorkers` is set to 3.
- `maxQueueSize: number`. The maximum number of tasks allowed to be queued. Can be used to prevent running out of memory. If the maximum is exceeded, adding a new task will throw an error. The default value is `Infinity`.
- `workerType: 'auto' | 'web' | 'process' | 'thread'`.
- In case of `'auto'` (default), workerpool will automatically pick a suitable type of worker: when in a browser environment, `'web'` will be used. When in a node.js environment, `worker_threads` will be used if available (Node.js >= 11.7.0), else `child_process` will be used.
- In case of `'web'`, a Web Worker will be used. Only available in a browser environment.
- In case of `'process'`, `child_process` will be used. Only available in a node.js environment.
- In case of `'thread'`, `worker_threads` will be used. If `worker_threads` are not available, an error is thrown. Only available in a node.js environment.
- `workerTerminateTimeout: number`. The timeout in milliseconds to wait for a worker to cleanup it's resources on termination before stopping it forcefully. Default value is `1000`.
- `abortListenerTimeout: number`. The timeout in milliseconds to wait for abort listener's before stopping it forcefully, triggering cleanup. Default value is `1000`.
- `forkArgs: String[]`. For `process` worker type. An array passed as `args` to [child_process.fork](https://nodejs.org/api/child_process.html#child_processforkmodulepath-args-options)
- `forkOpts: Object`. For `process` worker type. An object passed as `options` to [child_process.fork](https://nodejs.org/api/child_process.html#child_processforkmodulepath-args-options). See nodejs documentation for available options.
- `workerOpts: Object`. For `web` worker type. An object passed to the [constructor of the web worker](https://html.spec.whatwg.org/multipage/workers.html#dom-worker). See [WorkerOptions specification](https://html.spec.whatwg.org/multipage/workers.html#workeroptions) for available options.
- `workerThreadOpts: Object`. For `worker` worker type. An object passed to [worker_threads.options](https://nodejs.org/api/worker_threads.html#new-workerfilename-options). See nodejs documentation for available options.
- `onCreateWorker: Function`. A callback that is called whenever a worker is being created. It can be used to allocate resources for each worker for example. The callback is passed as argument an object with the following properties:
- `forkArgs: String[]`: the `forkArgs` option of this pool
- `forkOpts: Object`: the `forkOpts` option of this pool
- `workerOpts: Object`: the `workerOpts` option of this pool
- `script: string`: the `script` option of this pool
Optionally, this callback can return an object containing one or more of the above properties. The provided properties will be used to override the Pool properties for the worker being created.
- `onTerminateWorker: Function`. A callback that is called whenever a worker is being terminated. It can be used to release resources that might have been allocated for this specific worker. The callback is passed as argument an object as described for `onCreateWorker`, with each property sets with the value for the worker being terminated.
- `emitStdStreams: boolean`. For `process` or `thread` worker type. If `true`, the worker will emit `stdout` and `stderr` events instead of passing it through to the parent streams. Default value is `false`.
> Important note on `'workerType'`: when sending and receiving primitive data types (plain JSON) from and to a worker, the different worker types (`'web'`, `'process'`, `'thread'`) can be used interchangeably. However, when using more advanced data types like buffers, the API and returned results can vary. In these cases, it is best not to use the `'auto'` setting but have a fixed `'workerType'` and good unit testing in place.
A worker pool contains the following functions:
- `Pool.exec(method: Function | string, params: Array | null [, options: Object]) : Promise`
Execute a function on a worker with given arguments.
- When `method` is a string, a method with this name must exist at the worker and must be registered to make it accessible via the pool. The function will be executed on the worker with given parameters.
- When `method` is a function, the provided function `fn` will be stringified, send to the worker, and executed there with the provided parameters. The provided function must be static, it must not depend on variables in a surrounding scope.
- The following options are available:
- `on: (payload: any) => void`. An event listener, to handle events sent by the worker for this execution. See [Events](#events) for more details.
- `transfer: Object[]`. A list of transferable objects to send to the worker. Not supported by `process` worker type. See [example](./examples/transferableObjects.js) for usage.
- `Pool.proxy() : Promise