Manipulating files and directories with fs-extra in Node.js

Introduction

When working with Node.js, file system operations are a common requirement. While Node.js provides a built-in fs module for file system operations, it can sometimes be limited or require additional boilerplate code for common tasks. This is where fs-extra comes in.

fs-extra is a drop-in replacement for the native fs module that adds file system methods that aren’t included in the native fs module and adds promise support to the existing fs methods. It provides a more convenient and powerful API for file system operations.

In this article, we’ll explore how to use fs-extra to manipulate files and directories efficiently in Node.js applications.

What is fs-extra?

fs-extra is a third-party npm package that extends the functionality of the built-in Node.js fs module. It includes all the methods from the native fs module, so it can serve as a drop-in replacement, but it also adds many useful methods that make file system operations easier.

Key features of fs-extra include:

• All methods from the native fs module
• Additional convenience methods like copy, move, remove, emptyDir, ensureFile, ensureDir, and more
• Promise support for all methods (in addition to callbacks)
• Recursive directory operations
• Graceful error handling
• Cross-platform compatibility

Installation

To get started with fs-extra, you need to install it in your Node.js project. You can install it using npm or yarn:

# Using npm
npm install --save fs-extra

# Using yarn
yarn add fs-extra

Basic Usage

Once installed, you can import fs-extra in your JavaScript files:

const fs = require('fs-extra');

// Or using ES6 import syntax
import fs from 'fs-extra';

Since fs-extra is a drop-in replacement for the native fs module, you can use it exactly like you would use the native module, but with additional methods available.

Common File Operations

Let’s explore some of the most useful operations you can perform with fs-extra.

Copying Files and Directories

One of the most useful features of fs-extra is the ability to copy files and directories recursively with a single method call.

Copying a Single File

const fs = require('fs-extra');

// Using callbacks
fs.copy('/path/to/source/file.txt', '/path/to/destination/file.txt', (err) => {
  if (err) return console.error(err);
  console.log('File was copied successfully!');
});

// Using promises
fs.copy('/path/to/source/file.txt', '/path/to/destination/file.txt')
  .then(() => console.log('File was copied successfully!'))
  .catch(err => console.error(err));

// Using async/await
async function copyFile() {
  try {
    await fs.copy('/path/to/source/file.txt', '/path/to/destination/file.txt');
    console.log('File was copied successfully!');
  } catch (err) {
    console.error(err);
  }
}

Copying Directories Recursively

// Copy entire directory structure
fs.copy('/path/to/source/directory', '/path/to/destination/directory', (err) => {
  if (err) return console.error(err);
  console.log('Directory was copied successfully!');
});

// With options
fs.copy('/path/to/source', '/path/to/dest', {
  overwrite: true,
  errorOnExist: false
}, (err) => {
  if (err) return console.error(err);
  console.log('Copy completed!');
});

Moving Files and Directories

The move method allows you to move files or directories from one location to another.

// Move a file
fs.move('/path/to/source/file.txt', '/path/to/destination/file.txt', (err) => {
  if (err) return console.error(err);
  console.log('File was moved successfully!');
});

// Move with overwrite option
fs.move('/path/to/source', '/path/to/dest', { overwrite: true }, (err) => {
  if (err) return console.error(err);
  console.log('Move completed!');
});

// Using async/await
async function moveFile() {
  try {
    await fs.move('/path/to/source', '/path/to/dest');
    console.log('Move successful!');
  } catch (err) {
    console.error(err);
  }
}

Removing Files and Directories

The remove method can delete files and directories (including non-empty directories).

// Remove a file
fs.remove('/path/to/file.txt', (err) => {
  if (err) return console.error(err);
  console.log('File was removed!');
});

// Remove a directory and all its contents
fs.remove('/path/to/directory', (err) => {
  if (err) return console.error(err);
  console.log('Directory was removed!');
});

// Using promises
fs.remove('/path/to/unwanted')
  .then(() => console.log('Removed successfully!'))
  .catch(err => console.error(err));

Ensuring Files and Directories Exist

fs-extra provides methods to ensure that files and directories exist, creating them if they don’t.

Ensure Directory Exists

// Ensure directory exists (creates it if it doesn't)
fs.ensureDir('/path/to/directory', (err) => {
  if (err) return console.error(err);
  console.log('Directory exists!');
});

// Using async/await
async function createDirectory() {
  try {
    await fs.ensureDir('/path/to/directory');
    console.log('Directory ensured!');
  } catch (err) {
    console.error(err);
  }
}

Ensure File Exists

// Ensure file exists (creates it if it doesn't)
fs.ensureFile('/path/to/file.txt', (err) => {
  if (err) return console.error(err);
  console.log('File exists!');
});

Empty Directory

You can empty a directory (remove all its contents) while keeping the directory itself:

// Empty a directory
fs.emptyDir('/path/to/directory', (err) => {
  if (err) return console.error(err);
  console.log('Directory emptied!');
});

// Using promises
fs.emptyDir('/path/to/directory')
  .then(() => console.log('Directory emptied!'))
  .catch(err => console.error(err));

Reading and Writing Files

While these methods are available in the native fs module, fs-extra adds promise support:

Reading Files

// Read file with callbacks
fs.readFile('/path/to/file.txt', 'utf8', (err, data) => {
  if (err) return console.error(err);
  console.log(data);
});

// Read file with promises
fs.readFile('/path/to/file.txt', 'utf8')
  .then(data => console.log(data))
  .catch(err => console.error(err));

// Using async/await
async function readFile() {
  try {
    const data = await fs.readFile('/path/to/file.txt', 'utf8');
    console.log(data);
  } catch (err) {
    console.error(err);
  }
}

Writing Files

// Write file
const content = 'Hello, World!';

fs.writeFile('/path/to/file.txt', content, (err) => {
  if (err) return console.error(err);
  console.log('File written successfully!');
});

// Using promises
fs.writeFile('/path/to/file.txt', content)
  .then(() => console.log('File written!'))
  .catch(err => console.error(err));

Output File

The outputFile method is similar to writeFile, but it creates the parent directory if it doesn’t exist:

// Write file and create parent directories if needed
fs.outputFile('/path/to/nested/directory/file.txt', 'Hello!', (err) => {
  if (err) return console.error(err);
  console.log('File written with directories created!');
});

JSON Operations

fs-extra provides convenient methods for reading and writing JSON files:

Reading JSON

// Read JSON file
fs.readJson('/path/to/file.json', (err, obj) => {
  if (err) return console.error(err);
  console.log(obj);
});

// With async/await
async function readJsonFile() {
  try {
    const obj = await fs.readJson('/path/to/file.json');
    console.log(obj);
  } catch (err) {
    console.error(err);
  }
}

Writing JSON

const obj = {
  name: 'fs-extra',
  type: 'module'
};

// Write JSON file
fs.writeJson('/path/to/file.json', obj, (err) => {
  if (err) return console.error(err);
  console.log('JSON written!');
});

// With formatting options
fs.writeJson('/path/to/file.json', obj, { spaces: 2 }, (err) => {
  if (err) return console.error(err);
  console.log('JSON written with formatting!');
});

Path Exists

Check if a path exists:

// Check if path exists
fs.pathExists('/path/to/check', (err, exists) => {
  if (err) return console.error(err);
  console.log(exists); // true or false
});

// Using promises
fs.pathExists('/path/to/check')
  .then(exists => console.log(exists))
  .catch(err => console.error(err));

Advanced Usage Examples

Copying with Filters

You can filter which files to copy using a filter function:

const filterFunc = (src, dest) => {
  // Return true to copy the item
  return !src.includes('node_modules');
};

fs.copy('/src', '/dest', { filter: filterFunc }, (err) => {
  if (err) return console.error(err);
  console.log('Filtered copy complete!');
});

Working with Symbolic Links

// Create symbolic link
fs.ensureSymlink('/path/to/source', '/path/to/symlink', (err) => {
  if (err) return console.error(err);
  console.log('Symlink created!');
});

// Read symbolic link
fs.readlink('/path/to/symlink', (err, linkString) => {
  if (err) return console.error(err);
  console.log(linkString); // prints the path the symlink points to
});

Error Handling Best Practices

When working with file systems, proper error handling is crucial:

async function safeFileOperation() {
  try {
    // Ensure the directory exists
    await fs.ensureDir('/path/to/directory');
    
    // Write file
    await fs.writeFile('/path/to/directory/file.txt', 'Content');
    
    // Read it back
    const content = await fs.readFile('/path/to/directory/file.txt', 'utf8');
    console.log('File content:', content);
    
  } catch (err) {
    if (err.code === 'ENOENT') {
      console.error('File or directory not found');
    } else if (err.code === 'EACCES') {
      console.error('Permission denied');
    } else {
      console.error('An error occurred:', err.message);
    }
  }
}

Performance Considerations

When working with large files or many files, consider these tips:

1. Use streams for large files: For very large files, use streaming methods to avoid loading entire files into memory.

2. Batch operations: When performing many operations, batch them together rather than doing them one by one.

3. Use async methods: Always prefer async methods over sync methods to avoid blocking the event loop.

4. Handle errors gracefully: Always handle errors to prevent your application from crashing.

Common Use Cases

1. Build Scripts

async function buildProject() {
  // Clean build directory
  await fs.emptyDir('./dist');
  
  // Copy source files
  await fs.copy('./src', './dist/src');
  
  // Copy assets
  await fs.copy('./assets', './dist/assets');
  
  console.log('Build complete!');
}

2. Configuration Management

async function loadConfig() {
  const configPath = './config.json';
  
  // Ensure config exists with defaults
  const exists = await fs.pathExists(configPath);
  
  if (!exists) {
    const defaultConfig = {
      port: 3000,
      host: 'localhost'
    };
    await fs.writeJson(configPath, defaultConfig, { spaces: 2 });
  }
  
  // Load config
  return await fs.readJson(configPath);
}

3. Log File Management

async function rotateLogFile() {
  const logFile = './app.log';
  const archiveFile = `./logs/app-${Date.now()}.log`;
  
  // Ensure archive directory exists
  await fs.ensureDir('./logs');
  
  // Move current log to archive
  if (await fs.pathExists(logFile)) {
    await fs.move(logFile, archiveFile);
  }
  
  // Create new empty log file
  await fs.ensureFile(logFile);
}

Conclusion

fs-extra is an invaluable tool for Node.js developers who need to work with the file system. It simplifies many common operations, adds useful functionality not found in the native fs module, and provides a consistent promise-based API.

By using fs-extra, you can:

• Write cleaner, more maintainable code
• Avoid common pitfalls with file system operations
• Handle complex file operations with simple method calls
• Take advantage of both callback and promise-based APIs

Whether you’re building command-line tools, web applications, or automation scripts, fs-extra can help you work with files and directories more efficiently. The examples in this article should give you a solid foundation for using fs-extra in your own projects.