Categories
Programming

Replacing webpack with vite

In this post I share my experiences with trying to replace webpack with vite for local development of our medium-sized web app. This got longer than I anticipated. See conclusion for the gist of it!

I’ve recently introduced Vite at my company. So far we’ve been using webpack both for production and development. The problem being of course that bundling of our medium-sized TypeScript app (~3000 modules) takes quite some time: In watch mode changes of TSX or LESS files take at least 4 seconds to compile; and that’s on the fastest M1 Pro machines we have. On older devices, and especially also on our non-Mac laptops running Linux, it can take much longer than that. And the initial bundling can easily take up to a minute.

All of these delays can be quite frustrating during development. Also the webpack configuration itself is quite complex. So when we kept hearing more about how satisfied other developers were with vite I took some time to test it properly.

The goal was to use vite only for development, though. Both because I wanted to keep the scope small, and because our production webpack configuration is a bit more complex, especially with regard to chunking. And I wasn’t sure how hard it would be to replicate that using vite, or specifically rollup which is used under the hood for production builds in vite.

Getting Started with Vite

I used Vite’s own Getting Started docs to start out. Since I wanted to integrate it into our own workflow I didn’t use any of the scaffolding or templates. While our app uses a “traditional” Python backend rather than a Node backend and SSR, I started out with a static index.html as Vite seems to expect by default. Getting this to run was relatively easy and I had the first version running in 1 – 2 hours with only minor issues.

Minor Issues

__filename constants

The first problem was that in our code we had been relying on __filename constants that were previously injected by weback. Vite doesn’t define these constants so I had to find a workaround. At first I tried to use import.meta.url instead. That worked for vite, however as I wanted to keep webpack for building production bundles it failed there, because it wasn’t treating our files as ESM modules and therefore import.meta.url wasn’t defined.

For the least effort solution seemed to be writing a vite plugin that injected the constants the same way webpack does. So I had a look at Vite’s Plugin API. The documentation was detailed and clear enough so that implementing that was quite quick and easy using the “transform” hook to prepend the constants to each file:

function injectFilename() {
    return {
        name: 'inject-filename',

        transform(src, id) {
            if (/.*\.tsx?$/.test(id)) {
                // Vite replaces "__filename" in vite.config.js with the path to vite.config.js using a simple regex.
                // This would also match this code here, so the concatenation circumvents that.
                const constantName = '__' + 'filename';
                return `const ${constantName} = "${id}";\n${src}`;
            }
        },
    };
}

Vite caused a bit of confusion initially because within vite.config.js the string “__filename” is simply replaced with the path to the vite config itself, apparently using a simple regex so that this was also replaced in the string that I returned from the transform function. Hence the little concatenation hack above.

But in the end I liked how simple it was to write this plugin to solve the __filename constant problem.

Resolving of LESS Imports

The other problem I had was that none of the LESS CSS imports within LESS files themselves were resolving. While all of our TypeScript modules are importing their styles explicitly, e.g. import 'MyComponent.less' within the LESS files imports were declared implicitly without file extension, e.g. @import 'core'.

Webpack was handling this automatically. But Vite seems to expect explicit file extensions by design. I was considering writing another plugin that uses the “resolve” hook to fix the issue, but in the end I found it easier to simply add the explicit “.less” extension to all imports.

After that all that was missing was the definition of some aliases via the resolve.alias configuration option.

Having Gotten Started

Finally, the first working vite configuration looked something like the following. Not exactly “zero config” like Parcel, but still better than webpack, I suppose.

import { defineConfig, Plugin } from 'vite';

// solely required for passing the current filename to our Logger instances (new Logger(__filename)).
function injectFilename(): Plugin {
    return {
        name: 'inject-filename',

        transform(src, id) {
            if (/.*\.tsx?$/.test(id)) {
                // Vite replaces "__filename" in vite.config.js with the path to vite.config.js using a simple regex.
                // This would also match this code here, so the concatenation circumvents that.
                const constantName = '__' + 'filename';
                return `const ${constantName} = "${id}";\n${src}`;
            }
        },
    };
}

export default defineConfig({
    root: './frontend',
    plugins: [injectFilename()],
    resolve: {
        alias: {
            'core.less': 'frontend/css/app/core.less',
            macros: 'frontend/css/app/macros',
        },
    },
    server: {
        watch: {
            ignored: [
                /\/\.(.+?)\/?/, // ignore all dot files, e.g. .cache, .mypy_cache
            ],
        },
    },
});

Despite of the minor issues that was quite a nice start and I had a bit of a “wow” moment when I saw how quickly the changes were compiled and applied to the running app with HMR working right out of the box. Basically changes were now reflected instantaneously without reloads and without having to wait 5 or so seconds. What a game changer!

This looked already very promising so I kept going.

Backend Integration

Since our app uses what Vite calls a “traditional” backend (based on Python), keeping the static index.html wasn’t an option. Luckily the Vite docs address this issue directly in their Backend Integration section.

The integration ended up being also very simple to do. Instead of relying on vite injecting the necessary code into index.html, I adapted our Jinja2 template as described in Vite’s documentation:

<script type="module">
  import RefreshRuntime from '{{ Config.vite_server_url }}/@react-refresh'
  RefreshRuntime.injectIntoGlobalHook(window)
  window.$RefreshReg$ = () => {}
  window.$RefreshSig$ = () => (type) => type
  window.__vite_plugin_react_preamble_installed__ = true
</script>

<script type="module" src="{{ Config.vite_server_url}}/app/main.tsx"></script>

The first script is copied from the docs almost verbatim and is required for the Hot-Module-Replacement to work. The second script simply includes our main entry point. Since all of our components also import their own styles this also takes care of importing the necessary LESS styles at runtime.

And that was essentially already what was necessary to integrate it into our backend. However, there was still the issue of using vite also in our so called “prerender”, i.e. the “static”, server-generated pages that don’t run our app’s JavaScript code.

Prerender Styles

The “prerender” of our app is the HTML generated by our Python backend which may or may not include our frontend app script. Most pages include the JS entry point, but some don’t. The prerender is a simplified version of our app that already contains some basic elements like the page header and footer. In order for that to look good while the JavaScript files (which import the LESS styles) are still loading, I needed to include the styles already in the server-generated HTML.

Before vite, webpack was taking care of bundling the styles into one big CSS file and that was statically included in our backend template as you’d normally do:

<link rel='stylesheet' href='dist/app.css'>

Now with vite I couldn’t simply do that since during development vite doesn’t generate a single bundle – that’s the whole point. Since I also could not rely on the app’s regular component files to import the styles I had to add explicit imports of all the LESS files into the generated HTML. Well, that is one way to do it. I chose a different approach though.

To “statically” include all the styles for the prerender I created another vite plugin which defines a virtual module called “/virtual/prerenderStyles.ts” which includes all LESS files. The plugin uses glob to list all “.less” files and creates import statements for each of them.

function prerenderStyles(): Plugin {
    const virtualStyleModule = '/virtual/prerenderStyles.ts';

    return {
        name: 'prerender-styles',
        resolveId(id) {
            if (id === virtualStyleModule) {
                return virtualStyleModule;
            }
        },
        load(id) {
            if (id === virtualStyleModule) {
                const files = glob.sync('frontend/**/*.less');
                const imports = files.map((file) => `import '/${file.replace('frontend/', '')}';`).join('\n');
                return imports;
            }
        },
    };
}

This way I can include a single TypeScript file in the prerender’s HTML that takes care of including all the stylesheets:

<script type="module" src="{{ Config.vite_server_url }}/virtual/prerenderStyles.ts"></script>

So this took care of including the stylesheets independent of the actual app’s JavaScript code. Luckily this also works with HMR so in contrast to before with webpack changing LESS styles also is reflected immediately in our prerender.

Linux Problems

At this point vite was working pretty well for me on my MacBook M1 Pro. However, the Linux users in our company still had some massive problems with module requests timing out, being stuck “pending”, and even entire tabs crashing.

It seems like we’re not the only ones with this problem, though it’s not mentioned anywhere in Vite’s documentation. This is especially problematic if your project has large numbers of modules. Maybe it relates to resource limit configuration on Linux systems. By default the number of simultaneously opened files and number of inotify file watchers seems to be limited.

This configuration can be changed. Linux users can check the limits of open files using ulimit -n and the file watcher limits using sysctl fs.inotify. If they are lower than the number of modules in your project you should increase them. See my comment on the GitHub issue for some more details. How to increase these limits may depend on the Linux distribution. Simply using ulimit -n 99999 doesn’t seem to help. Instead some config files need to be adjusted and the system might even have to be rebooted.

However, this didn’t actually solve all of the issues for all of our Chrome users on Linux. Some reported that after adjusting the resource configs that it started to work in Chrome. However, for some the page loading seems to take very long while the browser devtools are open. So, regrettably, for those developers vite is currently not really viable.

Long Initial Loading Times

When Hot Module Replacement works it’s really nice and fast. However, in order to get there the page needs to be loaded at least once, and occasionally a whole page refresh is necessary. And here we’ve having some problems with vite. On my fast M1 MacBook Pro pages load initially within a few seconds, for other devs the first page load after starting vite takes 30, 60, or more seconds. While navigating through the app some initial loads of pages also take several seconds. Vite lazily processing and the browser then loading hundreds or thousands of modules simply seems to take a lot of time. Though this may also be related to resource limits specifically on some Linux systems.

Unfortunately this is rather frustrating, and several of our developers, especially those using Linux on hardware that’s 2 or 3 years old, switched back to webpack for this reason alone. They prefer waiting once for webpack to bundle everything and then having very fast / no loading times when navigating through the app. For some the initial loading takes so long that they prefer webpack even if it means foregoing HMR and waiting 5 or more seconds for webpack to recompile and reload after each change.

On my MacBook the initial loading times were bearable but I agree that it’s not ideal. I hope that the Persistent Cache suggestion on GitHub, or something else that alleviates these issues will be implemented at some point.

In the meantime as one way to improve the situation I created yet another vite plugin that “warms up” the build cache by simply loading the relevant pages of our app immediately after vite has started in order to trigger building all the modules once. This way the first “real” page load by the developer becomes much faster (if they wait for the warmup to finish). The implementation uses a headless browser via puppeteer and I’m not going to post the code here as it’s rather specific to our app and backend.

Without warmup for me the first page load takes around 8 seconds while vite processes the 1500 requests. After warmup the “first” page load only takes 1.3 seconds.

Another way to improve the situation would be to optimize chunking / lazy loading of our app so that only the necessary modules are loaded immediately. The first page definitely doesn’t need all of those 1500 modules. However, this wasn’t an issue with webpack before, and the production bundles are actually reasonably sized. We already do have several different bundles for different entry points to avoid loading all of our around 3000 modules on every page. Ideally vite would handle this situation better by having a persistent cache or something else that makes working with large numbers of modules more performant.

Conclusion

Overall I’m very satisfied with vite. Even though it took a few iterations and custom plugins to overcome some of the problems, the end result for me personally is much better than what I had with webpack before. Starting vite and “warm up” of the cache take 15 to 30 seconds for me – less than webpack’s initial bundling. Full page reloads / hard refreshing take 1 or 2 seconds for me. But most of the time HMR just works so that I can see my changes immediately, where before with webpack I had to always wait 5 seconds or so and reload the page. Therefore I will happily stick with vite now.

Unfortunately not all developers in my company share the same experience, though. This is especially true for developers who don’t have the latest hardware like my MacBook M1 Pro, for example 2 or 3 year old Lenovo laptops running Linux. Due to the long initial loading times, timeouts (maybe due to resource limits), or Chrome flat out freezing, they have to keep using webpack even if it means forgoing HMR and having to wait several seconds after each change.

For our production bundles we will continue to use webpack for now mostly because I didn’t have the time yet to properly evaluate if vite / rollup is sufficient for our use case. If it can achieve similar results (especially regarding chunking) with a simpler configuration, that would be a good reason to completely replace webpack with vite also in production. But while in development Linux/Chrome issues persist we’re forced to keep webpack around anyway.

Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Inline Feedbacks
View all comments