How can I compile MDX2 to static HTML with components

Question

I'm trying to write an MDX2-to-HTML compiler. I've got most of it working (that is, I can compile a simple .mdx file to HTML), but without importable components.

I have read through the entire MDX2 website, but it's not quite clicking for me. I have a bunch of successful experiments that do various parts, but I haven't quite seen the whole yet.

The Objective

I want to compile a set of MDX2 strings (one main fragment and a set of components) into a static HTML component-- preferrably without accessing the disk.

(This is not an absolute requirement, but I want to run it in an AWS lambda, and I won't have access to all the files at deployment time. I could use the /tmp dir of the lambda, but would rather not for various reasons.)

What I've Tried

Here's what I have so far. This works; it compiles MDX2 to HTML without touching the disk (aside from loading the entrypoint in this example, but it could just as easily be a string from elsewhere). But it doesn't let me do things like import MDX components, and it feels really dirty with the string manipulation I'm doing.

First, my .mdx file:

//example.mdx

# Title

export const Thing = ({foo}) => <>{foo}</>

# Hello, <Thing foo="bar"/>!

This is the content.

Then I have a .js file:

// example.js
import fs from 'node:fs/promises'
import { compile } from '@mdx-js/mdx'
import { renderToString } from 'react-dom/server';
import {Fragment as _Fragment, jsx as _jsx, jsxs as _jsxs} from "react/jsx-runtime";

/**
 * Using a vm to compile the mdx.  Why?  It seems like a way I can add
 * components accessible to the entry markdown file.  Is there a better way?
 *
 * This is the only way I could figure out how to _use_ the compiled module
 * without touching the filesystem.  Is a vfile a better choice?  I haven't
 * used them yet.
 */
import {NodeVM} from 'vm2';

/**
 * Load and compile the ./example.mdx file.  For reasons I don't quite
 * understand, I have to remove any import statements and the default
 * export.  If I don't do this, it throws an error (this could be a VM
 * issue and a reason not to use it).
 */
let compiled = String(await compile(await fs.readFile('example.mdx')))
  .replace(/import .*;/, '')
  .replace(/export default MDXContent;/, '')
;

/**
 * Here, grab the named export from the module.
 */
let namedExport = compiled.match(/export const (\w+)/);
if (namedExport) {
  namedExport = namedExport[1];
}

// Get the compiled version and remove the `export const`
compiled = compiled.replace(/export const (\w+)/, "$1");

/**
 * The compild version of this module needs to export MDXContent and the
 * named export (if there is one). We do that here.
 */
compiled += `
module.exports = {
  MDXContent,
  ${namedExport || "_void: ''"},
}
`;

/**
 * Set up our NodeVM.
 */
const vm = new NodeVM({
  console: 'inherit',
  sandbox: {
    // React resources needed by the module.
    _Fragment,
    _jsx,
    _jsxs,
  },
  require: {
    external : true,
    builtin  : ['fs','path'],
    root     : './',
    external : true,
    mock: {},
  },
});

// Now we've got a component.
let component = vm.run(compiled);

/**
 * For reasons I don't understand, when I render this I get empty HTML
 * comments. This removes them.
 */
console.log(
  renderToString(component.MDXContent())
  .replace(/<!-- -->/igsm,'')
);

And here's my package.json:

{
  "name": "mdx",
  "version": "1.0.0",
  "description": "",
  "type": "module",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@mdx-js/loader": "^2.3.0",
    "@mdx-js/mdx": "^2.3.0",
    "@mdx-js/node-loader": "^2.3.0",
    "@mdx-js/react": "^2.3.0",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-jsx": "^1.0.0",
    "vm2": "^3.9.17"
  }
}

This works-- but feels way too hacky. Since we're using unified under the hood, surely I can just render to static HTML? From what I understand, MDX2 (via @mdx-js/mdx) basically does something like:

mdx → hast → esast

to get from MDX2 to a compiled component. I understand that the esast step is to resolve imports and such. I feel like I should be able to go from there back to hast and just render the fragment, but I don't quite get it.

Next steps ...?

I would love to take a set of MDX2 (and maybe MD) strings and compile everything into a fragment without accessing the disk. I saw mdx-bundler (linked from here in the mdx docs), but I couldn't get it to work. It looks promising, though.

Here's an example of what I would like to get working:

ComponentA

Component A says: {props.data}

ComponentB

<ul>{props.items.map(item => (<li>{item}</li>))}</ul>

entrypoint markdown

# Entrypoint

<ComponentA data="foo"/>
<ComponentB items={1,2,3}/>

Output:

<h1>Entrypoint</h1>

<p>Component A says: foo</p>
<ul><li>1</li><li>2</li><li>3</li></ul>

With some javascript that looks something like:

...
const ComponentA = 'Component A says: {foo}';
const ComponentB = componentBFromSomewhere;
const entrypoint = entrypointFromSomewhere;

function mdxToHtml (e, opt) {
  // ... magic!
}

console.log(mdxToHtml(entrypoint, {components: {ComponentA, ComponentB}}));

Or something like that. Help?

How can I compile MDX2 to static HTML with components

The Objective

What I've Tried

Next steps ...?

0 Answers0