How to Write a Bundler
Exploring the inner workings
Presented by Paul Sweeney

Paul Sweeney

  • Previously, Senior Engineer @ Ericsson
  • Software Engineer @ Accenture The Dock
  • All opinions are my own

What's this talk about?

  • Bundlers combine and transform files to improve performance
  • The Big 3: Webpack, Rollup, Parcel
  • For me, not a fan of Webpack config
  • Parcel's zero-config is unrealistic
  • Rollup only does production builds - no Hot Module Replacement, slow
  • I decided to write a Rollup dev bundler
  • Here's the basics of a bundler

Anatomy of a Bundle

Dev
  • Concatenation
  • Function Scopes
  • Embedded Source Maps
Prod
  • Tree-Shaking
  • Scope Hoisting
  • No source maps

We'll bundle ES Modules for dev bundles

The Plan

Load
Transform
Generate
Output

Dev Bundle

  • Quick to generate
  • Build modules independently
  • HMR library compatible
  • Eval for multiple source maps

Mapping ESM

  • ESM is statically analyzable
  • Convert to our internal loader
  • Default property for default exports
  • Aligns with ESM dynamic import

Input

Output

The Tools

Abstract Syntax Tree

  • Converts source text to a tree
  • Nodes have types with child nodes
  • Start and end positions in the source
  • Traverse AST to convert source

Acorn Parser

Magic String

  • Can manipulate AST or the string
  • Regex too unreliable
  • Magic String tracks changes
  • Indexes refer to original string
  • Provides source map

Implementation

Loading Files

  • Start with entry file
  • Read in file text
  • Transform code
  • Create ID for module
  • Recursively parse dependencies

Transforming

  • Find ImportDeclaration
  • Convert to loader
  • Track the dependency
  • Convert specifiers
  • Overwrite string
  • Return deps and code.
  • Replace placeholders

Generate Bundle

  • Generate string
  • Append require loader
  • Require entry file
  • Stringify all modules
  • Escape quotes/linebreaks
https://codesandbox.io/s/how-to-write-a-bundler-jokfw

Other Issues

Hot Module
Replacement

  • Modules use function scopes
  • Cache modules in memory
  • Use file watcher
  • Reparse module, send via socket
  • Eval module and invalidate
  • Find accept callback in parents

ESM Bindings

  • Need for circular dependencies
  • Update export value dynamically
  • Only imports do this, nothing else
  • Bundler checks code for changes
  • Webpack breaks names

Source

Webpack Translation

CommonJS

  • CJS is not statically analyzable
  • Our bundler requires ESM
  • Several ways to write
  • Ambiguity with default exports
  • Check common patterns
  • Libraries should publish as ESM

NodeJS Conditionals

  • process.env not in browsers
  • Check for and replace NodeJS APIs
  • Statically check conditional requires
  • Typically has to be if statement
  • Cannot use variables, no execution
  • No ESM equivalent for sync import
  • Libraries should avoid this pattern

Commonly used by Facebook

About that bundler I wrote...

Nollup

github.com/PepsRyuu/nollup

npm install nollup

  • Rollup compatible - reuse config
  • Hot Module Replacement
  • Quick dev bundles and rebuilds
  • Dev middleware & CLI

Conclusion

  • Understanding tools helps solve problems
  • Extend the demo, experiment
  • Not as complex as you think
  • Don't limit yourself, challenge yourself
  • Have fun!

Thanks!

github.com/PepsRyuu

medium.com/@PepsRyuu

twitter.com/PepsRyuu

Sponsors