Programmatic text comparison

Group Multiple Text Files by Exact UTF-8 Contents

Compare several UTF-8 files by grouping identical decoded text, preserving whitespace with strict decoding and bounded inputs rather than a similarity score.

By Uddhav Gopani, FounderPublished 27 September 2026Updated 29 September 2026

Group Multiple Text Files by Exact UTF-8 Contents

To compare multiple text files for exact equality, read each file under one encoding rule and group files whose decoded text matches. This identifies duplicate contents and distinct versions without pretending that every difference is a meaningful edit. The Node.js example below reports groups and leaves the input files unchanged.

Choose grouping or change review

Grouping answers which files are exactly the same under the stated rule. A diff answers where two versions differ. A three-way merge answers how two edited versions relate to a shared base. These are different operations, so choose the one your task requires before interpreting the output.

For three files, if A and B contain alpha while C contains beta, the useful grouping is [A, B] and [C]. A single similarity percentage for the whole set would hide which files match. After grouping, compare representative files from different groups if you need to inspect the changes.

Save this as group-text.mjs

import { open } from "node:fs/promises";

async function readUtf8(path) {
  const handle = await open(path, "r");
  try {
    if (!(await handle.stat()).isFile()) throw new Error("Use regular files");
    const chunks = [];
    let size = 0;
    for await (const chunk of handle.createReadStream({ autoClose: false })) {
      size += chunk.length;
      if (size > 4 * 1024 * 1024) throw new Error("File exceeds 4 MiB");
      chunks.push(chunk);
    }
    const bytes = Buffer.concat(chunks);
    const text = new TextDecoder("utf-8", {
      fatal: true, ignoreBOM: true
    }).decode(bytes);
    return { text, size };
  } finally {
    await handle.close();
  }
}

try {
  const paths = process.argv.slice(2);
  if (paths.length < 2 || paths.length > 100)
    throw new Error("Supply 2 to 100 file paths");
  const groups = new Map();
  let totalBytes = 0;
  for (const path of paths) {
    const value = await readUtf8(path);
    totalBytes += value.size;
    if (totalBytes > 16 * 1024 * 1024)
      throw new Error("Combined input exceeds 16 MiB");
    if (!groups.has(value.text)) groups.set(value.text, []);
    groups.get(value.text).push(path);
  }
  console.log(JSON.stringify({
    comparison: "exact decoded UTF-8 text",
    allEqual: groups.size === 1,
    groups: [...groups.values()]
  }, null, 2));
  process.exitCode = groups.size === 1 ? 0 : 1;
} catch (error) {
  console.error(error.message);
  process.exitCode = 2;
}

Run node group-text.mjs a.txt b.txt c.txt, quoting any path that contains spaces. The program accepts two to one hundred path arguments. Its JSON output lists groups in the order their contents first appear, and each group retains the supplied file names.

Understand the exact comparison rule

The reader requires valid UTF-8 and preserves a leading byte-order mark. It also preserves line endings, spaces, case and Unicode representation. Two files that look identical in an editor can therefore belong to different groups if one uses CRLF or a different accent representation.

This is useful when exact text matters. If your workflow deliberately ignores a difference, introduce that normalization as an explicit step and report it. Do not trim or lowercase every file by default, especially when file contents include case-sensitive identifiers or meaningful indentation.

Check the grouping on controlled files

Start with a.txt and b.txt containing the same short text, then c.txt containing a different word. The first two should share a group, and the third should be separate. Next, make all three contents equal and confirm that allEqual becomes true.

Two empty files also share a group. An empty file and a file containing a newline do not. These cases test the boundaries of the rule without relying on a large real document whose differences are difficult to inspect manually.

Keep the bounded-memory contract visible

Each file may contain at most four mebibytes, and the combined accepted byte total is limited to sixteen mebibytes. The program stores decoded text for grouping, so it is intended for modest local batches rather than very large archives.

A missing file, directory, invalid UTF-8 sequence or size-limit failure produces an error and exit code two. The program prints no partial group report in that case. Exit code zero means all supplied texts match; one means the completed comparison found more than one group.

Treat names as labels, not content

Two differently named files can have equal contents, and two files with similar names can be entirely different. The map groups by decoded text, so no filename or extension is used as evidence of equality. Supplying the same path twice will list it twice; deduplicate the argument list first if that is not your intention.

Compare stable saved copies. The script does not freeze a directory while another application edits files. If the versions are changing during the run, repeat the comparison on a consistent snapshot before relying on the groups.

Inspect differences between groups afterward

Choose a representative file from each group and use a text comparison view to review changes. Keep the group report with the selected filenames so that the review remains connected to the actual versions.

Node’s file-system documentation describes the reading APIs used here. Grouping exact text does not establish semantic equivalence, detect plagiarism or merge edits. It provides a clear starting point for those more specific review tasks.