Skip to content

Corrupted file read/write #50717

@ronag

Description

@ronag

We noticed some corrupted files on our system and basically it seems that file read/write does not seem to increment the file handle offset. Hence writes will always write to offset 0 and reads will always read from offset 0.

From preliminary testing the issue seems to have been introduced in v20.3. I suspect #48078 and libuv/libuv#3952.

Host kernel: 5.15.104-0-lts
Docker Daemon: 20.10.9
Docker Image: node:21.1.0-bullseye and 20.9.0-bullseye

The issue only seems to occur when writing/reading to a specific SMB mount.

volumes:
  backup:
    driver: local
    driver_opts:
      type: cifs
      o: 'username=secret,password=secret'
      device: '//somednsname/nxt_backup'
import fs from 'fs'
import assert from 'assert'
const srcPath = '/backup/backup/0/0/0/0006d3befc1c41.mp4/0-TPtirLqtxVa6LP'
const dstPath = srcPath + '.tmp'
let position = 0
const src = await fs.promises.open(srcPath, 'r')
const dst = await fs.promises.open(dstPath, 'wx+')
try {
  while (true) {
    const { bytesRead, buffer } = await src.read() // { position })
    console.log('chunk', bytesRead, position)
    if (bytesRead === 0) {
      break
    }
    const chunk = buffer.subarray(0, bytesRead)
    await dst.write(chunk) // { position })
    position += bytesRead
  }
  await dst.sync()
} finally {
  await dst.close()
  await src.close()
}
const stat = fs.statSync(dstPath)
console.log('copied', dstPath, stat.size, position)
assert(position === stat.size, `${dstPath}: ${position} !== ${stat.size}`)

If we instead pass an explicit position to write/read then everything works as expected.

This also affects, fs.createWriteStream and fs.createReadStream.

Metadata

Metadata

Assignees

No one assigned

    Labels

    fsIssues and PRs related to the fs subsystem / file system.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions