Devise an approach for dealing with pure ESM libraries

[ghost]

Create an empty rescript nextjs project, add bindings for unist-util-visit or mdast-util-to-string, and make use of the binding. You will encounter the following error: "Must use import to load ES module".

mdast-util-to-string links to this very useful article explaining ways to make use of ESM only libraries: https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c.

This issue is complete once we understand what the root cause of this problem is and what workarounds are available, if any. We have no current plans to make use of unist-util-vist. We do use mdast-util-to-string, but we can use an older version which doesn't require ESM for now. We may use other similar utility libraries from the unist-util-* set of libraries. It would be helpful to be prepared to use those libraries when the need arises.

---

I encountered a blocking issue where the es6 and commonjs module systems were clashing. It was unclear what was forcing commonjs. I wasn't able to understand the root cause. It might take some time to understand it.

I implemented a workaround where I implemented my own functionality instead of using the library that was having difficulties. It's working so far, but may encounter issues when trying to use other libraries that are necessary.

Originally posted by @kanishka-work in #296 (comment)

[ghost]

After further research, this looks like the same problem as I am encountering when trying to use an ESM library called mdast-util-to-string, while upgrading dependencies in #225.

[ghost]

This approach from (https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c) may work:

If the package is used in an async context, you could use await import(…) from CommonJS instead of require(…).

[ghost]

The PR #358 demonstrates one way to use a ESM only library. Either do one of the following next:

• Clean up the implementation in Upgrade mdast-util-to-string #358
• Come up with a better way to use an ESM only library

[ghost]

Since this approach will be in the codebase for a while, let's document why the following warning occurs:

bsb: [1/6] src/Unified.ast

  Warning number 103
  /.../Unified.res:2:42-47
  
  1 │ %%raw(`
  2 │ const MdastUtilToStringInternal = (await import('mdast-util-to-string')
      ).toString
  3 │ `)
  4 │ 
  
  FFI warning: Unexpected token `import`, expected the token `)`

[ghost]

This issue is generally on hold as we wait for the dust to settle between nextjs, ReScript, and esm.

[ghost]

let's document why the following warning occurs:

This was done in #412.

[ghost]

After #445, there are very few dependencies remaining. This issue might not arise for a while, so I will close this for now.

[ghost]

"Next.js live" represents the upcoming effort to support esm fully within Next.js: https://nextjs.org/blog/next-11#nextjs-live-preview-release.

[ghost]

I suspect the error I was getting when trying to convert _app.js into rescript was an esm vs commonjs issue.