require "stream/web" produces warning

[zdm]

Hi,
On node v16 it produces experimental warning.

(node:18876) ExperimentalWarning: stream/web is an experimental featur

Could you please use polyfill until stream/web feature will get stable status?

Also instead of using try/catch for load stream/web you can check process.version and load stream/web if node version is >= 16.5.0.

[regseb]

I have the same warning with Node v16.8.0. The Web Streams API has been added in Node v16.5.0 but is experimental. I think you should use web-streams-polyfill until the API is stable (streams.cjs).

(node:23167) ExperimentalWarning: stream/web is an experimental feature. This feature could change at any time
    at emitExperimentalWarning (node:internal/util:227:11)
    at node:stream/web:7:1
    at NativeModule.compileForInternalLoader (node:internal/bootstrap/loaders:312:7)
    at NativeModule.compileForPublicLoader (node:internal/bootstrap/loaders:252:10)
    at loadNativeModule (node:internal/modules/cjs/helpers:41:9)
    at Function.Module._load (node:internal/modules/cjs/loader:804:15)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:94:18)
    at Object.<anonymous> (/home/regseb/testcase/node_modules/fetch-blob/streams.cjs:7:31)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)

[zdm]

stream/web api will be experimental for a year or more, so we are doomed to see this warning everywhere for a long time. ;-(

[jimmywarting]

I want to use the stream version developers will probably use cuz they are not interchangeable.
you can't pipe data from web-streams-polyfill to node's whatwg:stream

Also instead of using try/catch for load stream/web you can check process.version and load stream/web if node version is >= 16.5.0.

I also use this pkg on other non-nodejs places like Deno - Deno don't have stuff like blob's backed up by the filesystem. so i would prefer not to use version detection.

Deno for instances already have a global stream api... but i doubt this will be long lived, both Deno and Node's blob currently support async blobs and they have the potential to read data from the disc in a async manner, but non have a way of actually getting a blob from the disc yet.

[zdm]

Could you, please, move stream/web initialization code to the Blob.prototype.stream function?
Like this:

var initialized;

try {
    const { Blob } = require( "buffer" );

    if ( Blob && !Blob.prototype.stream ) {
        Blob.prototype.stream = function name ( params ) {
            if ( !initialized && !globalThis.ReadableStream ) {
                initialized = true;
                try {
                    Object.assign( globalThis, require( "stream/web" ) );
                }
                catch ( error ) {

                    // TODO: Remove when only supporting node >= 16.5.0
                    Object.assign( globalThis, require( "web-streams-polyfill/dist/ponyfill.es2018.js" ) );
                }
            }

            let position = 0;
            const blob = this;

            return new ReadableStream( {
                "type": "bytes",
                async pull ( ctrl ) {
                    const chunk = blob.slice( position, Math.min( blob.size, position + POOL_SIZE ) );
                    const buffer = await chunk.arrayBuffer();
                    position += buffer.byteLength;
                    ctrl.enqueue( new Uint8Array( buffer ) );

                    if ( position === blob.size ) {
                        ctrl.close();
                    }
                },
            } );
        };
    }
}
catch ( error ) {}

[jimmywarting]

It have been on my mind. we could do something like that.
But we should probably expose a function that can run the initialized part

the stream are needed here also:

fetch-blob/index.js

Lines 154 to 164 in 0e07457

	stream() {
	const it = toIterator(this.#parts, true);
	return new ReadableStream({
	type: 'bytes',
	async pull(ctrl) {
	const chunk = await it.next();
	chunk.done ? ctrl.close() : ctrl.enqueue(chunk.value);
	}
	})
	}

[zdm]

Yes, or export the function and call it in the stream() method.
Let's do this, because this warning is really annoying. ;-)

[jimmywarting]

(It is also possible to run NodeJS with the flag --no-warning)

This project is still quite Browser & Deno friendly and dose not really depend on any built in NodeJS stuff and I would also like to keep it that way.
Deno still lacks ways of creating Blob/Files backed up by the FileSystem. So i'm having thoughts of using this in Deno for another project i have.

So perhaps stream.cjs should expose a global function instead: globalThis._loadStreams() (b/c .cjs can't use esm export)
I have been thinking of using conditional import with top level await, but that requires node v14.14...

[zdm]

Disable warnings is a bad practice, warnings are useful, but not in our case.
Are you really using deno? I am unable to imagine, where it can be used and what for. Only as hobby. ;-)

[jimmywarting]

Are you really using deno? I am unable to imagine, where it can be used and what for. Only as hobby. ;-)

Yea, I use Deno for some own hoppy stuff. not so much for work.
tried convincing them to use Deno, but that would be a major rework. If i start with a new project/startup then i would try to use Deno instead if possible.

[zdm]

I think the correct way is to export from streams.cjs function like this:

/* c8 ignore start */
// 64 KiB (same size chrome slice theirs blob into Uint8array's)
const POOL_SIZE = 65536;

var ReadableStream;

module.exports = function getReadableStream () {
    if ( !ReadableStream ) {
        try {
            ReadableStream = require( "stream/web" ).ReadableStream;
        }
        catch ( error ) {

            // TODO: Remove when only supporting node >= 16.5.0
            ReadableStream = require( "web-streams-polyfill/dist/ponyfill.es2018.js" ).ReadableStream;
        }

        try {
            const { Blob } = require( "buffer" );

            if ( Blob && !Blob.prototype.stream ) {
                Blob.prototype.stream = function name ( params ) {
                    let position = 0;
                    const blob = this;

                    return new ReadableStream( {
                        "type": "bytes",
                        async pull ( ctrl ) {
                            const chunk = blob.slice( position, Math.min( blob.size, position + POOL_SIZE ) );
                            const buffer = await chunk.arrayBuffer();
                            position += buffer.byteLength;
                            ctrl.enqueue( new Uint8Array( buffer ) );

                            if ( position === blob.size ) {
                                ctrl.close();
                            }
                        },
                    } );
                };
            }
        }
        catch ( error ) {}
    }

    return ReadableStream;
};

And then use it where you need ReadableStream.
And you will not pollute globalThis.

[zdm]

What do you think?
If you want I can send pull req.

[zdm]

Hey, man, are you alive? Sorry for disturbing you, but let's close this issue and forget about it.

[jimmywarting]

Sry. I have been busy on other projects

Uhm, okey. Why?