HTTP-Modular: My Node.js Library for Converting Server-Side Functions Into ES Modules

Unleashing the power of ES modules for server-side logic

Photo by Christopher Gower on Unsplash

Recently, I was inspired and developed a remarkable “programming magic.” I encapsulated this idea into a Node.js library called “http-modular.”

The core concept was to transform server-side HTTP interfaces into JavaScript code that adheres to the ESM (ECMAScript Modules) standard. These modules can be effortlessly imported into browsers using the import statement, enabling the invocation of their functions to yield the desired outcomes.

// the server-side logic
...
async function save(data) {
  ......
  return await db.save(data);
}
app.all('/action', modular({save, list, delete}, config.koa));

// the invocation in browser
import {save, list, delete} from 'https://<server.url>:<port>/action';
...
const result = await save(data); // done!
...

Some developers may have a rough understanding that this is similar to a traditional Remote Procedure Call (RPC) but with a notable difference. Because modern browsers natively support ESM imports, there is no need to write bridging code on the frontend, significantly reducing frontend development effort.

To illustrate the concept, let’s explore a project:

Using http-modular to Implement Server-Side KV Storage

Imagine crafting an application on platforms like “CodeSandbox” with a requirement to store data in a database. In the past, this endeavor would entail the following steps:

1. Developing and debugging the storage interface within the “AirCode” environment.
2. Wrapping the storage API on the frontend to communicate with the server-side interface using tools like fetch or axios.

Now, with http-modular, we only need to perform Step 1. Step 2 is no longer necessary. Here's how you can achieve this:

First, create a project in AirCode and add a cloud function called index.mjs.

• If you don’t know how to create an app in AirCode, please follow the document guide.

In index.mjs, add the following code:

const table = aircode.db.table('storage');

async function setItem(key, value) {
  return await table.where({key}).set({value}).upsert(true).save();
}
async function getItem(key) {
  const res = await table.where({key}).findOne();
  return res?.value;
}
async function removeItem(key) {
  return await table.where({key}).delete();
}
async function clear() {
  return await table.drop();
}

With this code, we’ve implemented basic KV storage using the built-in AirCode database.

Next, install the http-modular dependency and export the cloud function module as follows:

export default modular({
  setItem,
  getItem,
  removeItem,
  clear,
}, config.aircode);

Finally, deploy this cloud function using the “Deploy” button. This gives you an endpoint like https://443nrzuwut.us.aircode.run/storage. In your “CodeSandbox” project, you can directly import and use the functions from this endpoint without any additional frontend wrapping:

import * as storage from "<https://443nrzuwut.us.aircode.run/storage>";

function log(msg) {
  const output = document.getElementById("output");
  output.insertAdjacentHTML("beforeEnd", `<div>${msg}</div>`);
}
async function main() {
  await storage.setItem("url", "<https://sandbox.io>");
  log(await storage.getItem("url"));
  await storage.removeItem("url");
  log((await storage.getItem("url")) || "null");
}
main();

By following this approach, you’ve created a convenient way to only develop the server-side code, eliminating the need for frontend API wrapping.

Implementing Authorization

Some may wonder: while your approach is indeed convenient, what if I need to incorporate more intricate functionalities such as authentication logic? How would that be achieved? It’s simpler than you might think; it only requires a touch of functional programming prowess. Let’s modify the existing code to accommodate this:

import aircode from 'aircode';
import {config, context, modular} from 'http-modular';
import {sha1 as hash} from 'crypto-hash';

const auth = context(async (ctx) => {
  const origin = ctx.headers.origin;
  const referer = ctx.headers.referer;
  const xBucketId = ctx.headers['x-bucket-id'];
  if(!/\\.csb\\.app$/.test(origin)) {
    throw new Error(JSON.stringify({error: {reason: 'illegal access'}}));
  }
  
  if(!xBucketId && !referer) {
    throw new Error(JSON.stringify({error: {reason: 'The projectId is missing. You need to add<meta name="referrer" content="no referrer when downtrade"/>to the HTML. For Safari browser, please select the "Unblock Cross Site Tracking Option".'}}));
  }
  const bucket = await hash(xBucketId || referer);
  return aircode.db.table(`storage-${bucket}`)
});
const setItem = context(auth, async (table, key, value) => {
  return await table.where({key}).set({value}).upsert(true).save();
});
const getItem = context(auth, async (table, key) => {
  const res = await table.where({key}).findOne();
  return res?.value;
});
const removeItem = context(auth, async (table, key) => {
  return await table.where({key}).delete();
});
const clear = context(auth, async (table) => {
  return await table.drop();
});
export default modular({
  setItem,
  getItem,
  removeItem,
  clear,
}, config.aircode);

In the provided code, we introduce an auth function responsible for authorization verification. It serves two primary purposes: ensuring that the origin of the call is from https://*.csb.app (the browser sandbox of CodeSandbox) and generating a unique hash based on the referer. This guarantees a distinct storage space for each "CodeSandbox" project, preventing data mixing.

The auth function only returns the data table if the authorization requirements are met. To facilitate this process, we import the context function from http-modular. This higher-order function takes a function as its first argument, which handles the request context and passes the processing result as the first parameter to subsequent cloud functions. Other parameters returned by the client are also forwarded to the cloud functions as subsequent parameters.

In utilizing the context method, it's possible to omit the context-handling function. By providing only a cloud function as an argument, the context will be passed as the first parameter to the cloud function, as demonstrated by the following example:

const echo = context(ctx => ctx.request.body);

With this measure in place, secure data access within the “CodeSandbox” project is assured.

How http-modular Works

Understanding the mechanics behind http-modular is refreshingly simple: it packages HTTP interfaces into JavaScript code that adheres to the ECMAScript Modules (ESM) standard. To illustrate this, let’s delve into the content of an AirCode cloud function after a direct browser request: https://443nrzuwut.us.aircode.run/storage. What emerges is a block of JavaScript code:

function makeRpc(url, func) {
  return async(...args) => {
    const ret = await fetch(url, {
      method: 'POST',
      body: JSON.stringify({func, args}),
      headers: {
        'content-type': 'application/json'
      }
    });
    const type = ret.headers.get('content-type');
    if(type && type.startsWith('application/json')) {
      return await ret.json();
    } else if(type && type.startsWith('text/')) {
      return await ret.text();
    }
    return await ret.arrayBuffer();
  }
}

export const setItem = makeRpc('https://443nrzuwut.us.aircode.run/storage', 'setItem');
export const getItem = makeRpc('https://443nrzuwut.us.aircode.run/storage', 'getItem');
export const removeItem = makeRpc('https://443nrzuwut.us.aircode.run/storage', 'removeItem');
export const clear = makeRpc('https://443nrzuwut.us.aircode.run/storage', 'clear');

The purpose of this code is lucid: it encapsulates diverse methods within Remote Procedure Call (RPC) invocations.

On the server side, the core functions of http-modular are equally streamlined, occupying roughly 60 lines of code:

const sourcePrefix = `
function makeRpc(url, func) {
  return async(...args) => {
    const ret = await fetch(url, {
      method: 'POST',
      body: JSON.stringify({func, args}),
      headers: {
        'content-type': 'application/json'
      }
    });
    const type = ret.headers.get('content-type');
    if(type && type.startsWith('application/json')) {
      return await ret.json();
    } else if(type && type.startsWith('text/')) {
      return await ret.text();
    }
    return await ret.arrayBuffer();
  }
}
`;

function buildModule(rpcs, url) {
  let source = [sourcePrefix];
  for(const key of Object.keys(rpcs)) {
    source.push(`export const ${key} = makeRpc('${url}', '${key}');`);
  }
  return source.join('\\n');
}
const _ctx = Symbol('ctx');
export function context(checker, func) {
  if(!func) {
    func = checker;
    checker = ctx => ctx;
  }
  const ret = async (context, ...rest) => {
    const ctx = await checker(context);
    return await func(ctx, ...rest);
  };
  ret[_ctx] = true;
  return ret;
}
export function modular(rpcs, {getParams, getUrl, getContext, setContentType, setBody}) {
  return async function (...rest) {
    const ctx = getContext(...rest);
    const method = ctx.request?.method || ctx.req?.method;
    if(method === 'GET') {
      setContentType(...rest);
      return setBody(buildModule(rpcs, getUrl(...rest)), ...rest);
    } else {
      const {func, args = []} = await getParams(...rest);
      const f = rpcs[func];
      if(f?.[_ctx]) {
        return setBody(await f(ctx, ...args), ...rest);
      }
      return setBody(await f(...args), ...rest);
    }
  };
}
export default modular;

To elaborate, when an HTTP request’s method is “GET,” the server generates JavaScript code and transmits it to the client. For other HTTP request methods, the appropriate function is executed based on the func and args parameters.

http-modular effortlessly integrates with leading Node.js frameworks and platforms. Its default configuration seamlessly supports a range of frameworks and cloud function platforms, including Express, Koa, Fastify, Nitro, Vercel, and AirCode.

For detailed instructions tailored to each framework or platform, consult the documentation provided within the GitHub repository.

The server-side code example is also included in the GitHub repository. Enthusiasts are encouraged to explore this approach, and you can easily deploy the code to your personal AirCode account by following the “one-click deployment” option outlined in the project’s README.

Despite its simplicity, this paradigm shift holds an intriguing blend of fascination and practicality. I encourage you to try it and experience its potential firsthand.

Thanks for reading.