Files

The fs module defines a large API for working with files and directories. Most functions in this API are asynchronous callback-based, such as fs.readFile(). Typically, there are also synchronous blocking variants, such as fs.readFileSync(). In addition, Promised-based asynchronous variants are also available from fs/promises module, such as fs.promises.readFile().

Most function from the fs module take a string to specify the path to the file, such as fs.truncate(), but there are other functions that expect a file descriptor to specify a file, such as fs.ftruncate().

Moreover, there are functions in the fs module that start with "l" that operate on symbolic links themselves instead of following them in the filesystem.

File descriptors

A file descriptor is a non-negative integer that uniquely identifies an open file within a process. It is used by the operating system to keep track of all open files and other input/output resources like pipes, sockets, etc.

When a file is opened using fs.open(), a file descriptor is returned. This file descriptor can then be used for low-level file operations like reading and writing bits within a file. A file descriptor remains valid until the file is closed using fs.close(). Processes are allowed to have only a limited number of files open at one time, so it is important to call fs.close() on the file descriptor when you are done with it.

import fs from "fs";

fs.open("data/example.txt", "r", (err, fd) => {
  if (err) throw err;
  console.log(`File descriptor: ${fd}`);
  fs.close(fd, (err) => {
    if (err) throw err;
  });
});

File path

Files are most often specified by path, which is the name of the file itself, plus the hierarchy of directories to file location. If path is absolute, the hierarchy of directories start at the filesystem root. Otherwise, the path is relative to some other path, usually the current working directory.

import os from "os";
import path from "path";

// Important paths
console.log(os.homedir());
console.log(path.sep);

// Path parsing
let p = "src/pkg/test.js";
path.basename(p); // => "test.js"
path.extname(p); // => ".js"
path.dirname(p); // => "src/pkg"
path.basename(path.dirname(p)); // => "pkg"
path.dirname(path.dirname(p)); // => "src"

// Clean paths
path.normalize("a/b/c/../d/"); // => "a\b\d\"
path.normalize("a/./b/"); // => "a\b\"
path.normalize("./a//b//"); // => "a\b\"
path.posix.normalize("./a//b//"); // => "a/b/"

// Combine path segments
path.join("src", "pkg", "script.js"); // => "src\pkg\script.js"
path.posix.join("src", "pkg", "script.js"); // => "src/pkg/script.js"

// Resolve absolute path
console.log(path.resolve());
console.log(path.resolve("script.js"));
console.log(path.resolve("/tmp", "script.js"));
console.log(path.resolve("/a", "/b", "script.js"));
console.log(path.posix.resolve("/a", "/b", "script.js"));

The current working directory is the directory from which the Node process was launched. It may be the same or different to the directory where the script is located depending on whether you run the script from a module or directly from the command line. For example, suppose we have the following script in a module:

// pkg/printDir.js
import { fileURLToPath } from "url";
import { dirname } from "path";

export default function printDir() {
  const __filename = fileURLToPath(import.meta.url);
  const __dirname = dirname(__filename);

  console.log("pkg filename path:", __filename);
  console.log("pkg dirname path:", __dirname);
  console.log("pkg CWD:", process.cwd());
}

It will print a different output for the __filename and __dirname variables, but the process.cwd() will produce the same result.

// cwd.js
import { fileURLToPath } from "url";
import { dirname } from "path";
import printDir from "./pkg/printDir.js";

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

console.log("filename path:", __filename);
console.log("dirname path:", __dirname);
console.log("CWD:", process.cwd());
console.log("=========================");

printDir();

Reading files

You can read the file content all at once, via a stream, or with low-level API. By default, you get the file as a Buffer object, but if you specify encoding, you'll get a decoded string instead.

If the file is small, you can read all of its contents in memory. The following is an example of an asynchronous callback-based approach to read a file:

import fs from "fs";

const filename = "data/example.json";

fs.readFile(filename, { encoding: "utf-8" }, (err, data) => {
  if (err) {
    console.error(err);
    return;
  }

  try {
    let json = JSON.parse(data);
    console.log(json.name);
  } catch (err) {
    console.error(err);
  }
});

In certain scenarios, synchronous blocking code is preferred to read file contents (for example, reading a configuration file when you are starting up a server).

const data = fs.readFileSync(filename, "utf-8");
const json = JSON.parse(data);
console.log(json.version);

The asynchronous Promise-based API to open a file is more concise.

fs.promises
  .readFile(filename, "utf-8")
  .then((data) => console.log(JSON.parse(data).type));

Alternatively, you can use an async/await function with Promise-based API:

async function processFile(filename, encoding = "utf-8") {
  let data = await fs.promises.readFile(filename, encoding);
  let json = JSON.parse(data);
  let description = json.description;
  console.log(description);
}
processFile(filename);

If you can process the contents of a file sequentially and do not need to have entire content of the file in memory, read the file by using a stream.

function printFile(filename, encoding = "utf-8") {
  fs.createReadStream(filename, encoding).pipe(process.stdout);
}

printFile("data/example.txt");

If you need low-level control when reading file contents, you can open a file descriptor and then use fs.read(), fs.readSync(), or fs.promises.read() functions.

fs.open("data/example.json", (err, fd) => {
  if (err) {
    console.error(err.message);
    return;
  }

  const options = {
    buffer: Buffer.alloc(20), // Buffer that data will be written to
    offset: 0, // Position in buffer to write data to
    length: 20, // Number of bytes to read
    position: null, // Where to begin reading from in the file
  };

  try {
    fs.read(fd, options, (err, bytesRead, buffer) => {
      if (err) {
        console.error(err.message);
        return;
      }
      console.log("Bytes read:", bytesRead);
      console.log("Contents:", buffer.toString("utf-8"));
    });
  } catch (err) {
    console.error("Error:", err.message);
  } finally {
    fs.close(fd);
  }
});

In order to read multiple chunks of data with an asynchronous callback-based approach, you need to call nested fs.read() multiple times. Conversely, in a synchronous blocking code you call fs.readSync() sequentially:

function readData(filename) {
  let fd = fs.openSync(filename);

  try {
    let buffer = Buffer.alloc(12);
    fs.readSync(fd, buffer, 0, 5, 0);
    fs.readSync(fd, buffer, 5, 7, 5);

    console.log(buffer.toString());
  } catch (err) {
    console.error("Error:", err.message);
  } finally {
    fs.closeSync(fd);
  }
}
readData("data/example.txt");

Likewise, with async/await style of Promise-based API, you can read chunks of data sequentially, improving code readability:

async function readDataAsync(filename) {
  let fileHandle;

  try {
    fileHandle = await fs.promises.open(filename, "r");

    const buffer = Buffer.alloc(12);
    await fileHandle.read(buffer, 0, 5, 0);
    await fileHandle.read(buffer, 5, 7, 5);

    console.log(buffer.toString());
  } catch (err) {
    console.error("Error:", err.message);
  } finally {
    if (fileHandle) {
      await fileHandle.close();
    }
  }
}
readDataAsync("data/example.txt");

The callback-based version APIs are preferable over the use of the promise APIs when maximal performance (both in terms of execution time and memory allocation) is required.

Writing files

Writing files is similar to reading files. You create a new file by writing to a file that does not exist.

If you want to write entire content of the file, use fs.writeFile(), fs.writeFileSync(), or fs.promises.wirteFile() methods. To append to existing file, use fs.appendFile(), fs.appendFileSync(), or fs.promises.appendFile().

If you want to write to a file in chunks, use fs.createWriteStream() or filehandle.createWriteStream() to write the data to a file sequentially using steams API.

Finally, for a fine-grained control over position within a file at which each chunk is written, open the file with fs.open(), fs.openSync(), or fs.promises.open() and use the resulting descriptor with fs.write(), fs.writeSync(), or filehandle.write() functions. If you have an array of Buffer objects, you can write them with fs.writev(), fs.wirtevSync(), or filehandle.writev() functions.

When you want to open file descriptor of a file for the purpose of modifying the file, you should specify the flag option as a second argument to fs.open(), fs.openSync(), and fs.promises.open() functions:

1. 'a': Open file for appending. The file is created if it does not exist.

2. 'ax': Like 'a' but fails if the path exists.

3. 'a+': Open file for reading and appending. The file is created if it does not exist.

4. 'ax+': Like 'a+' but fails if the path exists.

5. 'as': Open file for appending in synchronous mode. The file is created if it does not exist.

6. 'as+': Open file for reading and appending in synchronous mode. The file is created if it does not exist.

7. 'r': Open file for reading. An exception occurs if the file does not exist.

8. 'rs': Open file for reading in synchronous mode. An exception occurs if the file does not exist.

9. 'r+': Open file for reading and writing. An exception occurs if the file does not exist.

10. 'rs+': Open file for reading and writing in synchronous mode. Instructs the operating system to bypass the local file system cache.

11. 'w': Open file for writing. The file is created (if it does not exist) or truncated (if it exists).

12. 'wx': Like 'w' but fails if the path exists.

13. 'w+': Open file for reading and writing. The file is created (if it does not exist) or truncated (if it exists).

14. 'wx+': Like 'w+' but fails if the path exists.

Note that fs.write() and fs.writeSync() write data to a memory buffer, and the operating system handles the actual writing to the disk at a later time. In cases of unexpected power loss or system crash, the data may be lost.

Flushing data to the storage device ensures that the data written to a file is physically stored on the storage medium, such as a hard drive or SSD, rather than just being held in a temporary memory buffer.

fs.fsync() and fs.fsyncSync() (and also the flush option in fs.writeFile(), fs.appendFile(), fs.createWriteStream()) force the operating system to flush the buffer and write the data to the physical storage, guaranteeing that the data is stored on the disk, which is crucial for critical applications where data loss cannot be tolerated.

However, flushing data to the disk is a relatively expensive operation in terms of I/O performance. Operating systems use various caching mechanisms to improve performance by batching multiple write operations together, leading to more efficient disk usage and better overall performance.

File operations

To copy a file, use fs.copyFile(), fs.copyFileSync(), and fs.promises.copyFile() functions.

To truncate a file, use fs.truncate(), fs.truncateSync(), or fs.promises.truncate() functions.

To rename or move a file, use fs.rename(), fs.renameSync(), or fs.promises.rename() functions.

To delete a file, use fs.unlink(), fs.unlinkSync(), and fs.promises.unlink() functions.

File metadata

To obtain metadate for a file or directory, use fs.stat(), fs.statSync(), and fs.promises.stat() functions.

To change file or directory permissions, use fs.chmod(), , fs.chmodSync(), and fs.promises.chmod() functions. The mode argument used in fs.chmod(), , fs.chmodSync(), and fs.promises.chmod() functions can be constructed from a sequence of three octal digits (e.g. 0o765). The left-most digit (7 in the example), specifies the permissions for the file owner. The middle digit (6 in the example), specifies permissions for the group. The right-most digit (5 in the example), specifies the permissions for others.

To set the owner and group IDs for a file or directory, use fs.chown(), fs.chownSync(), and fs.promises.chown() functions.

To set the access and modification time for a file or directory, use fs.utimes(), fs.utimesSync(), and fs.promises.utimes() functions.

Working with directories

To create a new directory, use fs.mkdir(), fs.mkdirSync(), and fs.promises.mkdir() functions.

To create a directory with random name appended to it, use fs.mkdtemp() and its variants.

To read the contents of an entire directory at once, use fs.readdir() and its variants.

To read the contents of a directory in streaming mode, use fs.opendir() and its variants.