Skip to content

RangeError: Invalid string length with large files. #35973

@denniscm

Description

@denniscm
  • Version: 14.15.0
  • Platform: Windows 10 64-bit
  • Subsystem:

What steps will reproduce the bug?

When i'm trying to read a large Json File (700MB - 26.640.222 lines) using ReadStream i'm getting the error: "RangeError: Invalid string length". Will post the code i'm using to read the file:

const { createReadStream } = require('fs');

const input = createReadStream(`${__dirname}/input/test.json`, { encoding: 'utf8' })
        .on('error', (error) => {
            console.log('Error to read file, check if the filename is correct!');
            console.log(error);
        });

    let buildedChunkData = '';

    input.on('data', data => {
        buildedChunkData += data;
    }).on('end', () => {
            console.log(JSON.parse(buildedChunkData.toString()));
    });

How often does it reproduce? Is there a required condition?

Only when the file to be read is very large.

What is the expected behavior?

Not get the error when reading a large file.

What do you see instead?

This is the error that shows to me:

buildedChunkData += data;
                            ^
RangeError: Invalid string length
    at ReadStream.<anonymous> (C:\Gitlab\Outros\JsonToCsv\event_vehicle_All_editor.js:104:29)
    at ReadStream.emit (events.js:315:20)
    at addChunk (_stream_readable.js:309:12)
    at readableAddChunk (_stream_readable.js:280:11)
    at ReadStream.Readable.push (_stream_readable.js:223:10)
    at internal/fs/streams.js:226:14
    at FSReqCallback.wrapper [as oncomplete] (fs.js:539:5)

Additional information

The weird thing is, i can read this large file with Node 12.19.0 and Node 13.X versions without problems, only in this new versions 14.X that i'm getting this issue. I'm doing something wrong or there is some workaround this? Because it works nice with almost all the files, only in large files that i'm having this issue.

Thank you.

Metadata

Metadata

Assignees

No one assigned

    Labels

    v8 engineIssues and PRs related to the V8 dependency.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions