GrabDuck

Node.js Best Practice Exception Handling

:

Following is a summarization and curation from many different sources on this topic including code example and quotes from selected blog posts. The complete list of best practices can be found here



Number1: Use promises for async error handling

TL;DR: Handling async errors in callback style is probably the fastest way to hell (a.k.a the pyramid of doom). The best gift you can give to your code is using instead a reputable promise library which provides much compact and familiar code syntax like try-catch

Otherwise: Node.JS callback style, function(err, response), is a promising way to un-maintainable code due to the mix of error handling with casual code, excessive nesting and awkward coding patterns

Code example - good

doWork()
.then(doWork)
.then(doError)
.then(doWork)
.catch(errorHandler)
.then(verify);

code example anti pattern – callback style error handling

getData(someParameter, function(err, result){
    if(err != null)
      //do something like calling the given callback function and pass the error
    getMoreData(a, function(err, result){
          if(err != null)
            //do something like calling the given callback function and pass the error
        getMoreData(b, function(c){ 
                getMoreData(d, function(e){ 
                    ...
                });
            });
        });
    });
});

Blog quote: "We have a problem with promises" (From the blog pouchdb, ranked 11 for the keywords "Node Promises")

"…And in fact, callbacks do something even more sinister: they deprive us of the stack, which is something we usually take for granted in programming languages. Writing code without a stack is a lot like driving a car without a brake pedal: you don’t realize how badly you need it, until you reach for it and it’s not there. The whole point of promises is to give us back the language fundamentals we lost when we went async: return, throw, and the stack. But you have to know how to use promises correctly in order to take advantage of them."


Number2: Use only the built-in Error object

TL;DR: It pretty common to see code that throws errors as string or as a custom type – this complicates the error handling logic and the interoperability between modules. Whether you reject a promise, throw exception or emit error – using Node.JS built-in Error object increases uniformity and prevents loss of error information

Otherwise: When executing some module, being uncertain which type of errors come in return – makes it much harder to reason about the coming exception and handle it. Even worth, using custom types to describe errors might lead to loss of critical error information like the stack trace!

Code example - doing it right

    //throwing an Error from typical function, whether sync or async
 if(!productToAdd)
 throw new Error("How can I add new product when no value provided?");

//'throwing' an Error from EventEmitter
const myEmitter = new MyEmitter();
myEmitter.emit('error', new Error('whoops!'));

//'throwing' an Error from a Promise
 return new promise(function (resolve, reject) {
 DAL.getProduct(productToAdd.id).then((existingProduct) =>{
 if(existingProduct != null)
 return reject(new Error("Why fooling us and trying to add an existing product?"));

code example anti pattern

//throwing a String lacks any stack trace information and other important properties
if(!productToAdd)
    throw ("How can I add new product when no value provided?");

Blog quote: "A string is not an error" (From the blog devthought, ranked 6 for the keywords “Node.JS error object”)

"…passing a string instead of an error results in reduced interoperability between modules. It breaks contracts with APIs that might be performing instanceof Error checks, or that want to know more about the error. Error objects, as we’ll see, have very interesting properties in modern JavaScript engines besides holding the message passed to the constructor.."


Number3: Distinguish operational vs programmer errors

TL;DR: Operations errors (e.g. API received an invalid input) refer to known cases where the error impact is fully understood and can be handled thoughtfully. On the other hand, programmer error (e.g. trying to read undefined variable) refers to unknown code failures that dictate to gracefully restart the application

Otherwise: You may always restart the application when an error appear, but why letting ~5000 online users down because of a minor and predicted error (operational error)? the opposite is also not ideal – keeping the application up when unknown issue (programmer error) occurred might lead unpredicted behavior. Differentiating the two allows acting tactfully and applying a balanced approach based on the given context

Code example - doing it right

    //throwing an Error from typical function, whether sync or async
 if(!productToAdd)
 throw new Error("How can I add new product when no value provided?");

//'throwing' an Error from EventEmitter
const myEmitter = new MyEmitter();
myEmitter.emit('error', new Error('whoops!'));

//'throwing' an Error from a Promise
 return new promise(function (resolve, reject) {
 DAL.getProduct(productToAdd.id).then((existingProduct) =>{
 if(existingProduct != null)
 return reject(new Error("Why fooling us and trying to add an existing product?"));

code example - marking an error as operational (trusted)

//marking an error object as operational 
var myError = new Error("How can I add new product when no value provided?");
myError.isOperational = true;

//or if you're using some centralized error factory (see other examples at the bullet "Use only the built-in Error object")
function appError(commonType, description, isOperational) {
    Error.call(this);
    Error.captureStackTrace(this);
    this.commonType = commonType;
    this.description = description;
    this.isOperational = isOperational;
};

throw new appError(errorManagement.commonErrors.InvalidInput, "Describe here what happened", true);

//error handling code within middleware
process.on('uncaughtException', function(error) {
    if(!error.isOperational)
        process.exit(1);
});

Blog Quote: "Otherwise you risk the state" (From the blog debugable, ranked 3 for the keywords "Node.JS uncaught exception")

"…By the very nature of how throw works in JavaScript, there is almost never any way to safely “pick up where you left off”, without leaking references, or creating some other sort of undefined brittle state. The safest way to respond to a thrown error is to shut down the process. Of course, in a normal web server, you might have many connections open, and it is not reasonable to abruptly shut those down because an error was triggered by someone else. The better approach is to send an error response to the request that triggered the error, while letting the others finish in their normal time, and stop listening for new requests in that worker"


Number4: Handle errors centrally, through but not within middleware

TL;DR: Error handling logic such as mail to admin and logging should be encapsulated in a dedicated and centralized object that all end-points (e.g. Express middleware, cron jobs, unit-testing) call when an error comes in.

Otherwise: Not handling errors within a single place will lead to code duplication and probably to errors that are handled improperly

Code example - a typical error flow

//DAL layer, we don't handle errors here
DB.addDocument(newCustomer, (error, result) => {
    if (error)
        throw new Error("Great error explanation comes here", other useful parameters)
});

//API route code, we catch both sync and async errors and forward to the middleware
try {
    customerService.addNew(req.body).then(function (result) {
        res.status(200).json(result);
    }).catch((error) => {
        next(error)
    });
}
catch (error) {
    next(error);
}

//Error handling middleware, we delegate the handling to the centrzlied error handler
app.use(function (err, req, res, next) {
    errorHandler.handleError(err).then((isOperationalError) => {
        if (!isOperationalError)
            next(err);
    });
});

Blog quote: "Sometimes lower levels can’t do anything useful except propagate the error to their caller" (From the blog Joyent, ranked 1 for the keywords “Node.JS error handling”)

"…You may end up handling the same error at several levels of the stack. This happens when lower levels can’t do anything useful except propagate the error to their caller, which propagates the error to its caller, and so on. Often, only the top-level caller knows what the appropriate response is, whether that’s to retry the operation, report an error to the user, or something else. But that doesn’t mean you should try to report all errors to a single top-level callback, because that callback itself can’t know in what context the error occurred"


Number5: Document API errors using Swagger

TL;DR: Let your API callers know which errors might come in return so they can handle these thoughtfully without crashing. This is usually done with REST API documentation frameworks like Swagger

Otherwise: An API client might decide to crash and restart only because he received back an error he couldn’t understand. Note: the caller of your API might be you (very typical in a microservices environment)

Blog quote: "You have to tell your callers what errors can happen" (From the blog Joyent, ranked 1 for the keywords “Node.JS logging”)

…We’ve talked about how to handle errors, but when you’re writing a new function, how do you deliver errors to the code that called your function? …If you don’t know what errors can happen or don’t know what they mean, then your program cannot be correct except by accident. So if you’re writing a new function, you have to tell your callers what errors can happen and what they mea


Number6: Shut the process gracefully when a stranger comes to town

TL;DR: When an unknown error occurs (a developer error, see best practice number #3)- there is uncertainty about the application healthiness. A common practice suggests restarting the process carefully using a ‘restarter’ tool like Forever and PM2

Otherwise: When an unfamiliar exception is caught, some object might be in a faulty state (e.g an event emitter which is used globally and not firing events anymore due to some internal failure) and all future requests might fail or behave crazily

Code example - deciding whether to crash

//deciding whether to crash when an uncaught exception arrives
//Assuming developers mark known operational errors with error.isOperational=true, read best practice #3
process.on('uncaughtException', function(error) {
 errorManagement.handler.handleError(error);
 if(!errorManagement.handler.isTrustedError(error))
 process.exit(1)
});


//centralized error handler encapsulates error-handling related logic 
function errorHandler(){
 this.handleError = function (error) {
 return logger.logError(err).then(sendMailToAdminIfCritical).then(saveInOpsQueueIfCritical).then(determineIfOperationalError);
 }

 this.isTrustedError = function(error)
 {
 return error.isOperational;
 }

Blog quote: "There are three schools of thoughts on error handling" (From the blog jsrecipes)

…There are primarily three schools of thoughts on error handling: 1. Let the application crash and restart it. 2. Handle all possible errors and never crash. 3. Balanced approach between the two


Number7: Use a mature logger to increase errors visibility

TL;DR: A set of mature logging tools like Winston, Bunyan or Log4J, will speed-up error discovery and understanding. So forget about console.log.

Otherwise: Skimming through console.logs or manually through messy text file without querying tools or a decent log viewer might keep you busy at work until late

Code example - Winston logger in action

//your centralized logger object
var logger = new winston.Logger({
 level: 'info',
 transports: [
 new (winston.transports.Console)(),
 new (winston.transports.File)({ filename: 'somefile.log' })
 ]
 });

//custom code somewhere using the logger
logger.log('info', 'Test Log Message with some parameter %s', 'some parameter', { anything: 'This is metadata' });

Blog quote: "Lets identify a few requirements (for a logger):" (From the blog strongblog)

…Lets identify a few requirements (for a logger): 1. Time stamp each log line. This one is pretty self explanatory – you should be able to tell when each log entry occured. 2. Logging format should be easily digestible by humans as well as machines. 3. Allows for multiple configurable destination streams. For example, you might be writing trace logs to one file but when an error is encountered, write to the same file, then into error file and send an email at the same time…


Number8: Discover errors and downtime using APM products

TL;DR: Monitoring and performance products (a.k.a APM) proactively gauge your codebase or API so they can auto-magically highlight errors, crashes and slow parts that you were missing

Otherwise: You might spend great effort on measuring API performance and downtimes, probably you’ll never be aware which are your slowest code parts under real world scenario and how these affects the UX

Blog quote: "APM products segments" (From the blog Yoni Goldberg)

"…APM products constitutes 3 major segments:1. Website or API monitoring – external services that constantly monitor uptime and performance via HTTP requests. Can be setup in few minutes. Following are few selected contenders: Pingdom, Uptime Robot, and New Relic 2. Code instrumentation – products family which require to embed an agent within the application to benefit feature slow code detection, exceptions statistics, performance monitoring and many more. Following are few selected contenders: New Relic, App Dynamics 3. Operational intelligence dashboard – these line of products are focused on facilitating the ops team with metrics and curated content that helps to easily stay on top of application performance. This is usually involves aggregating multiple sources of information (application logs, DB logs, servers log, etc) and upfront dashboard design work. Following are few selected contenders: Datadog, Splunk"


The above is a shortened version - see here more best practices and examples