Member-only story
My experiences with concurrency while writing an NPM package
How to Use Every Core on your Machine using NodeJS
Each job takes seconds to complete which is expensive in the long run. Now it takes less than a minute for 3000 jobs! This is the final result.
Background
You have probably used other languages that have developer-friendly ways to multitask complex jobs. Unfortunately, doing this in JavaScript has always been complicated.
For the longest time, JavaScript and NodeJS were limited by the event loop. Code executes asynchronously, but not in true parallel fashion. However, that changed with the release of worker threads in NodeJS.
After discovering this concept, I immediately want to test its full capability. Unfortunately, the existing libraries are overly complex and/or lack true parallel capabilities.
Goal
I want a package that is perfect for small projects. Something that provides a job queue without relying on databases or the filesystem while proving obvious performance benefits.
Problem
Many packages are half-baked implementation of concurrency. For example, some packages have code that look like this.
