node.js File System appendFile methods

As part of Lab 1 for OSD600, today we will differentiate the various forms of appendFile in the node.js fs library.

The three options that may be used for this type of task are:

  • fs.appendFile(path, data[, options], callback)
  • fs.appendFileSync(path, data[, options])
  • fsPromises.appendFile(path, data[, options])

All of these methods are exist to add text to a text file, which is why they have a path parameter (leading to the file to be appended to) and a data parameter (a string that will be written to the end of the file).

Why are there three of them? Let’s start with their behaviour before discussing why we might need several options.

 

fs.appendFileSync(path, data[, options])

This method goes straight to work, taking the string in data appending it to the file, as expected. However the program must wait for it to be completed before continuing. If the disk is busy, then execution will stall completely until the information is written. Other tasks may be waiting unnecessarily, or the user may think the program has crashed. Then why does this method exist? Let’s revisit it after talking about the alternatives

 

fs.appendFile(path, data[, options], callback)

This method has an additional parameter: a callback function. It is an anonymous function defined as an argument to fs.appendFile. Instead of waiting on the method call to complete, the program will continue on to the next line. The call happens in the background, and calls the callback function after completion. But this makes means that the programmer is responsible for providing that callback function.

 

fsPromises.appendFile(path, data[, options])

This method also allows for asynchronous execution of code, but accomplishes using a Promise. Promises mean that the we can write a chain of callbacks cleanly, instead of nesting them and getting into callback hell. Telling the program to “do A, then when that finished, do B, and then do C, but if an error occurs do D” (all asynchronously) simply looks something like this:

fsPromises.appendFile("myfile.txt", "name: John Smith")

.then(function() { showSuccessDialog();})

.then( function() {  clearInputField(); })

.catch( function (err) { console.log(err); });

They also can return values and throw errors, like synchronous functions.

 

Knowing this, let’s consider when some of these methods are useful (or not useful).

What happens if we are calling appendFile many times in a row on the same file? If it is asynchronous, they may all be trying to execute at the same time. However, scheduling is often taken care of by other forces, and it is possible that they happen in the wrong order, or that one blocks access to the file for the others. In such a case, the non-promise asynchronous method is not desirable. This would be fixed by using appendFileSync.

This would solve either problem, in this case.  If the data being appended is extremely large, or (more likely) you need to append it thousands or millions of times in succession, it may cause the program to hang, since each call must complete before the program goes on to the next line. If the program hangs even for a second or two, it may negatively impact the user experience.

Now consider this example:

for (int i=0; i < novels.length(); i++) {
    fs.writeFile(novels[i] + ".txt", novels[i] + "\n\n");
    appendCopyRightInfo(novels[i] + ".txt", novels[i]);
    fs.appendFileSync(novels[i] + ".txt", novel_bodies[i]);
}

Let’s assume that appendCopyRightInfo and upload work like magic. If we are looping over thousands of novels in our array, then we are spending an unnecessary amount of time on th last step. Assuming nothing after this requires our novel body to be finished appending, we can replace the last call with the asynchronous version fs.appendFile

 

In this version, however, the result is immediately used (twice):

for (int i=0; i < novels.length(); i++) {
    String filename = novels[i] + ".txt"
    fs.writeFile(filename , novels[i] + "\n\n");
    appendCopyRightInfo(filename , novels[i]);
    fs.appendFileSync(filename , novel_bodies[i]);
    upload("myhost", filename );
    fs.unlinkSync(filename);
}

 

instead we could use Promises:

for (int i=0; i < novels.length(); i++) {
    String filename = novels[i] + ".txt"
    fs.writeFile(filename , novels[i] + "\n\n");
    appendCopyRightInfo(filename , novels[i]);
    fsPromises.appendFile(filename , novel_bodies[i]);
    .then(upload("myhost", filename ))
    .then(fsPromises.unlink(filename))
    .catch(function(err) { console.log(err); });
}

 

Of course, things can get a lot more complicated, but having the different options means that we can analyze the problem and take the best path we can think of.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s