[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Page MenuHomePhabricator

Remove build step from CodeMirror
Closed, ResolvedPublic

Description

A build step was recently added to CodeMirror in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CodeMirror/+/961145.

This is similar to what is done for Extension:Popups, MobileFrontend, among other extensions.

In these extensions, the build step has proved to be a significant hindrance to developers:

  • Ensuring the build step is run before each commit
  • Making simple rebases impossible without checking out the code and re-building
    • Makes almost every patch conflict with every other patch after merging
    • Makes backport deployments more complex
  • Security concerns with compiling code directly from npm

With RL support for ES6 and packageFiles, these repos are looking to remove their build steps: T315929: Reconsider build step in Popups and MobileFrontend

Due to packaging constraints with CodeMirror 6, we now use Webpack to bundle the files,

CodeMirror provides a CJS distribution which should be compatible with ResourceLoader packageFiles: https://github.com/codemirror/basic-setup

If this isn't feasible, the build step should be limited to as few files as possible, i.e. just the CodeMirror library, so that only library updates result in having the run the rebuild scripts.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
MusikAnimal removed a project: Community-Tech.

Indeed, sadly the CJS distros were only discovered some months ago… I created this mess and will clean it up eventually!

We don't use the basic-setup, so the work here would involve migrating the code to use foreign resources for each package instead of NPM. Then the changes to the classes should mostly be just syntactical -- import( 'foo' ) becomes require( 'foo' ) and so forth.

It looks like the distributed CJS files have i.e. require('@codemirror/state') in them, which of course is not understood by ResourceLoader. I would just use the basic-setup package instead, but it too makes the same require statements (it assumes you're using a build step). It's looking like Foreign Resources isn't going to work for this library, unless I register a RL module for each package individually using the same name as the NPM package.

EDIT: See https://discuss.codemirror.net/t/codemirror-6-distribution-package/6705 . An "all-in-one" distro package is not officially offered.

Plan B is to build a small wrapper around the CodeMirror library, so you only need to run the build step for library updates. We basically have that now with the ext.CodeMirror.v6.lib RL module, which is built using Rollup. Beyond that, everything can be managed by RL and instead of i.e. import { EditorState } from '@codemirror/state', we'd use const lib = require( 'ext.CodeMirror.v6.lib' ); and then do lib.EditorState or perhaps even object unpacking (with polyfill if necessary). That would work, but I'm going to sorely miss the benefits of using ESM! Basically everything mentioned in T281781 and T279108. In that sense it feels like I'm taking a step backwards in frontend technologies, but I'm certainly in favor of not having to run a build step every time I want to change the code.

Making simple rebases impossible without checking out the code and re-building

  • Makes almost every patch conflict with every other patch after merging
  • Makes backport deployments more complex

I should mention CodeMirror's build step isn't this intrusive. Looking at https://gerrit.wikimedia.org/r/q/project:mediawiki/extensions/CodeMirror, every patch I see that has a merge conflict is an actual merge conflict in the code and not artifacts of the compiled assets. I think this was a result of moving to Rollup and bundling all vendor code into one file (r1010952).

I think ESM does have an advantage over CJS that it supports better tree-shaking? This could be important for CM6 in order to reduce the bundle size.

I think ESM does have an advantage over CJS that it supports better tree-shaking? This could be important for CM6 in order to reduce the bundle size.

I think that would normally be true, but in our case we're forcing all vendor code (in node_modules/) to be in one output file. Moving to CJS won't change that. As for the other files, we had Rollup configured to bundle them in the same way they would if they were packageFiles in extension.json (the entrypoint file + whatever it requires minus vendor code).

I'm actually making decent progress on this already. The code is the identical to before except for imports and exports at the top/bottom of each file, so the migration isn't that difficult. The big drawback of course is IDEs won't understand what ext.CodeMirror.v6.lib contains, so you won't get autocompletion, static analysis and such anymore :(

It's looking like Foreign Resources isn't going to work for this library …

Plan B is to build a small wrapper around the CodeMirror library, so you only need to run the build step for library updates. We basically have that now with the ext.CodeMirror.v6.lib RL module, which is built using Rollup.

I don't know why it didn't occur to me, but we can do the hybrid. Use Foreign Resources to track the packages and then bundle it with Rollup so it works with ResourceLoader. Boom :)
To note, this would mean six Foreign Resources, one for each package. I'm assuming the scant clutter at Special:Version won't be a problem.

Never mind. I failed to notice the dependency tree. There is no way around it, I'm afraid. CodeMirror 6 requires the use of NPM.

Change #1048371 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] [WIP] Isolate build step to CM6 library, restructure to work with RL

https://gerrit.wikimedia.org/r/1048371

Change #1048371 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] Isolate build step to CM6 library and restructure files to work with RL

https://gerrit.wikimedia.org/r/1048371