NodeJS runtime environment: Libuv Library (Event loop, Thread pool)
Node.js is an open-source, cross-platform JavaScript runtime environment that enables developers to build server-side applications. It runs on the V8 engine and allows for the execution of JavaScript code beyond a web browser.
Node.js offers a platform for executing JavaScript outside of a web browser’s context.
JavaScript was initially designed for browser use and was limited to executing solely within a browser context. Node.js, however, extends its capabilities by allowing developers to run JavaScript outside of a browser environment. The Node.js runtime environment provides all the necessary components to execute a JavaScript program beyond the confines of a browser.
The architecture of Node.js consists of three levels. The top level consists of the Node.js API written in JavaScript, including core packages, streams/buffers, and more. The bottom level includes the Chrome V8 engine, Libuv library, DNS, zlib, and other components. The HTTP module, written in JavaScript, is located at the top level, while the http_parser, written in C, is located at the bottom level. Communication between these components is facilitated by Node.js bindings.
JavaScript is referred to as a synchronous, blocking, single-threaded language because code execution in JavaScript occurs in a sequential, blocking manner. This means that JavaScript will execute one line of code at a time and will wait for the completion of a task before moving on to the next.
Furthermore, JavaScript is single-threaded, meaning it can only process one task at a time. As a result, if a long-running task is executed, the JavaScript runtime environment will block, or freeze, and be unable to process any other requests until the task is complete. This can sometimes lead to performance issues, especially when dealing with complex or time-consuming operations.
· Synchronous execution in JavaScript means that code is executed in a sequential, top-to-bottom fashion, with only one line being executed at a time.
· Due to its synchronous nature, JavaScript is blocking, meaning that it will only execute one task at a time and will wait for its completion before moving on to the next task. If two functions are present, and the first one requires a significant amount of processing time, JavaScript will wait for its completion before executing the second function. This sequential execution of code can result in longer processing times for certain operations, but ensures that code is executed in the order it appears in the script.
· JavaScript is single-threaded, meaning it has only one main thread for executing any code. A thread, in programming, is a process that is used to run a task, and in JavaScript, it has only one main thread which can only perform one task at a time. This single-threaded nature can sometimes result in performance limitations, especially when executing complex or time-consuming operations.
But how we are able to handle multiple requests by just a single thread? Despite being single-threaded, Node.js can handle multiple requests with its asynchronous behavior. This is made possible through the use of the Libuv library, which helps manage asynchronous processing in the Node.js runtime environment. The Libuv library enables Node.js to efficiently handle multiple requests, even with its single-threaded architecture, by allowing tasks to be executed in a non-blocking fashion, without having to wait for the completion of one task before starting another.
What is Libuv?
The Libuv library is a key component in the Node.js runtime environment, responsible for handling asynchronous, non-blocking operations. It’s written in the C language and is open-source and cross-platform. The library abstracts away the complexity of interacting with the operating system by leveraging two crucial features: the thread pool and the event loop. Although these are two of the most important features, there is much more to the Libuv library.
Event Loop
Node.js runtime environment features the V8 engine for executing JavaScript code. The code execution follows a Last-In-First-Out (LIFO) implementation where functions are pushed onto a call stack and returned functions are popped off. Asynchronous code is offloaded to Libuv, which runs the task using the native asynchronous mechanism of the operating system or its thread pool, to ensure the main thread isn’t blocked.
To illustrate this, consider an example of synchronous code execution. It starts with the global execution context (GEC). First, the function of the first console.log statement (console.log(“START”)) is added to the call stack. The statement is logged, and then the function is removed from the stack.
The same process is repeated for the second console.log statement (console.log(“TEXT”)), followed by the third (console.log(“END”)), after which the GEC is removed from the call stack.
Now let’s take a look at asynchronous code execution. In asynchronous code execution, we have three console.log statements. In the second statement, it is inside a callback function that is passed to fs.readFile(). The first statement is the console.log(“START”) function which is pushed into the call stack, and “START” is logged in the console. Then, fs.readFile() is pushed into the call stack. As the operation is asynchronous, its callback function is passed to Libuv which starts to read the file contents on a separate thread. Meanwhile, fs.readFile() is removed from the call stack, and the next console.log(“END”) statement is pushed in, logged in the console, and then removed from the stack.
When the file reading task assigned to the thread pool by Libuv is finished, the corresponding callback function containing the second log statement (console.log(“Read File Content”)) will be added to the call stack. The log message “Read File Content” will be displayed in the console, and the console.log statement will be removed from the call stack. After the callback function finishes executing, it too will be removed from the call stack, and since there’s no more code to run, the Global Execution Context (GEC) will also be popped off.
The Event Loop in Node.js is a design pattern that coordinates the execution of synchronous code in the runtime environment. It acts as an orchestrator, ensuring that the code runs in the appropriate order and without any blocks or delays.
The Event Loop allows Node.js to perform non-blocking I/O operations — despite the fact that JavaScript is single-threaded — by offloading operations to the system kernel whenever possible.
The Event Loop in Node.js coordinates the execution of synchronous code by utilizing a design pattern. The Event Loop has six phases, each with a specific queue: timers, pending callbacks, idle/prepare, poll, check, close callbacks, and a microtask queue (which includes two priority queues: nextTickQueue and Other microtask queue).
· The first phase, timers, holds callbacks from setTimeout() and setInterval(). For Libuv both of them are the same thing, just for setInterval there is a parameter repeat.
· The second phase (queue) is called pending callbacks. This is a queue that gets I/O (Input/Output) operations, but not all of them, the network operations, and error callbacks.
· Third one is called Idle, prepare. We don’t have access to this phase, but the Event Loop can itself gets there, for example before reading a file the Event loop can prepare some data and gets to this phase.
· The fourth phase, poll, holds I/O operations such as file reading.
· In the check phase gets only setImmidiate. This means whenever we call setImmidiate the callback gets to the phase (queue) check. This function is specific to NodeJS and it’s not something that you would come across when you write JavaScript for the browser.
· Close callbacks queue. This contains callbacks associated with the close event for an async task. For example, socket on close and etc.
Finally, there is the microtask queue. The microtask queue has two priority queues, the nextTickQueue and Other microtask queue. Callbacks are added to the nextTickQueue through process.nextTick which is specific to NodeJS, while promises are added to the other microtask queue. And this is very important, “if the Event Loop is not in one of the 6 phases (queues), it goes to implement priority queues (nextTickQueue, Other microtask queue)”. At first, it implements nextTickQueue callbacks then Other Microtask Queue.
The six phases (or queues) in the Event Loop are all part of Libuv, while the two microtask queues are not part of Libuv.
The Event Loop is alive as long as the NodeJS application is up and running. The Event Loop doesn’t spin endlessly, it has a condition (loop alive), if it's false means there are no more callbacks and the Event Loop ends.
Callback functions are executed only when the call stack is empty. The normal flow of execution will not be interrupted to run a callback function.
Let’s conclude the priority order of the queues:
· All user-written synchronous JavaScript code takes priority over asynchronous code that the runtime would like to execute. Only after the call stack is empty, the event loop comes into the picture.
Any callbacks in the microtask queues are executed. First, tasks in the nexTick queue and only then tasks in the promise (Other Microtask Queue) are executed. Callbacks in the microtask queues, if present, are executed.
· All callbacks within the timer phase (queue) are executed.
· Callbacks within network I/O operations are executed.
· All callbacks associated with Idle, prepare queue are executed. (Only the Event Loop can itself gets there).
· Callbacks within the I/O queue are executed. (Example: whenever we start to read files).
· All callbacks in the check queue (phase) are executed. (setImmidiate)
· Callbacks in the close queue are executed.
If the Event Loop is not in one of the 6 phases (queues), it goes to implement priority queues (nextTickQueue, Other microtask queue).
Thread Pool
JavaScript is single-threaded, but in NodeJS, the Libuv open-source library provides a way to implement time-consuming or CPU-intensive operations by not blocking the main thread. The main thread offloads these operations to the Libuv. Libuv has a pool of threads that can use to run some time-consuming tasks. When tasks are done, the associated callback function can be run.
“Libuv thread pool as the name indicates is literally a pool of threads, that NodeJS uses to offload time-consuming tasks and ensures the main thread is not blocked for a long time.”
Every method in NodeJS that has a “sync” suffix always runs on the main thread.
A few async methods (fs.readFile and crypto.pbkdf2) run on a separate thread in the Libuv thread pool. They do run synchronously in their own thread, but as far as the main thread is concerned, it appears as if the method is running asynchronously. The thread pool default size is 4, but it can be changed at startup time by setting process.env.UV_THREADPOOL_SIZE environment variable to ant value (the absolute maximum is 128). By increasing the thread pool size, we are able to shorten the total time to run multiple calls of an asynchronous method.
If the machine running the NodeJS application has 8 cores, then having 1 time-consuming operation (such as fs.readFile or crypto.pbkdf2) will consume 1 thread and 1 core. However, if there are 8 time-consuming operations, they will consume all 8 threads and cores. The execution time of these operations will be nearly the same as for 1 operation, as the UV_THREADPOOL_SIZE is set to 8. Hence, increasing the thread pool size can improve performance, but it is limited by the number of available CPU cores.
Level Up Coding
Thanks for being a part of our community! Before you go:
- 👏 Clap for the story and follow the author 👉
- 📰 View more content in the Level Up Coding publication
- 🔔 Follow us: Twitter | LinkedIn | Newsletter
🚀👉 Join the Level Up talent collective and find an amazing job
