How to trick a VDOM
Our story begins with a very particular client that’s technically savvy and picky about the implementation. The request is to make a View Transition API for apps written in Elm.
The challenge
The main request from the client is that this specific piece of software should require no setup in terms of state and updates. The client desires magic. Auto-magic even.
Assuming you know what MVC is, it’s a similar idea in the SPA world of frontend, except it’s called model, view and update. The model holds the entire app state. The view renders html based on the current model. Update is a function that takes 2 arguments: the current model and a message of what happened, and returns a new model.
Model -> View -> (user interaction) -> Msg -> Update -> Model -> View -> and so on…
Choice A: The obvious one
With the way it works in mind, to implement this in Elm “user space”, the user would have to “wire in”:
- our model into their model (no local state allowed)
- our update into their update (no side effects allowed)
It’s the only way it would work, and it would break the request of it all working auto-magically.
On the other hand, doing it in Elm is the only solution where all of the html is managed by Elm’s VDOM. Meaning this is the only option if we want to support “rewinding” view transitions mid-way (like the user pressing back mid-transition).
This was brought up as a major concern and a good reason why the small extra work to set up this piece of software was worth the benefit. But the client pushed back and magic was a hard requirement. So onwards we went to hackier waters.
Choice B: The remaining option
After establishing that we can’t do it in Elm, we’re left with only the option of doing it in Typescript/Javascript. This means that we will have to go around Elm and its VDOM and essentially hide our HTML elements from Elm.
Implementation details
Well, now we know what we’re doing, but we have yet to decide how exactly we’re going to do it.
First thing’s first: in order to provide our magic behaviour, we have to place our implementation in a Web Component that the Elm side will just render.
export const defineViewTransitions = () => { customElements.define("view-transitions", ViewTransitions);};
class ViewTransitions extends HTMLElement { connectedCallback() { // called when our element is added to the DOM } disconnectedCallback() { // called when our element is removed from the DOM }}Now, let’s quickly consider our feature. We want something that drives View Transitions. The software’s job is essentially just to manage and flag the state of HTML elements that are considered containers of a view that needs to transition.
So, identify the right views and know when a view is entering and when it is exiting so we can flag them for targeting in CSS. The identification part can either be explicit from the Elm side, or we can implicitly assume only direct children qualify.
Seems simple so far, but we’re working outside of Elm. So when do these states changes happen? Well, entering happens when Elm adds our view to the DOM, and exiting happens when it removes it. Uh oh.
Looks like we have to detect when Elm adds our view to the DOM, and we have to prevent removal if we want our exit transition to be visible. We have some options here.
The low dependency solution
The most recommended approach to making your own little island (that a VDOM should stay away from) is to just have a static element, like a div, that never changes on the VDOM side (in our case, the Elm side). In this way the VDOM representation of our element is always the same, it doesn’t check what’s inside and it leaves us to do what we want in there. Our only stipulation is that we shouldn’t mutate the DOM tree outside our little island.
So how about a little island element for exiting views and a MutationObserver to track added and removed elements? Both added and removed elements are flagged with a data attribute for their state. However, the removed elements are added to our island, keeping them still on the page while they play their exiting animation, and removed when the transition has ended.
const VIEW_STATE_ATTR = "data-view-state" as const;
class ViewTransitions extends HTMLElement { #enteringContainer: HTMLElement | null = null; #exitingContainer: HTMLElement | null = null; #observer: MutationObserver | null = null;
connectedCallback() { const entering = this.querySelector("entering-views"); const exiting = this.querySelector("exiting-views");
if (!entering || !exiting) return;
this.#enteringContainer = entering; this.#exitingContainer = exiting;
const observer = new MutationObserver((mutations) => { for (const mutation of mutations) { for (const node of mutation.addedNodes) { if (node instanceof Element) this.#onEntering(node); } for (const node of mutation.removedNodes) { if (node instanceof Element) this.#onExiting(node); } } });
observer.observe(entering, { childList: true }); this.#observer = observer; }
disconnectedCallback() { this.#observer?.disconnect(); this.#observer = null; this.#enteringContainer = null; this.#exitingContainer = null; }
#onEntering(el: Element) { el.setAttribute(VIEW_STATE_ATTR, "entering"); }
#onExiting(el: Element) { el.setAttribute(VIEW_STATE_ATTR, "exiting"); if (this.#exitingContainer) { this.#exitingContainer.appendChild(el); } }}This simple version assumes our containers are stable (won’t be replaced by the VDOM), a non-demo version could add some validation mechanism to replace the references and observe the new container. We’re also making a big assumption that children are available in connectedCallback, which in our case is true, but makes this specific VDOM implementation a dependency. Again, on a non-demo version, moving the logic to a method that we can call on a microtask or, failing that, even add to a MutationObserver (that runs only once) on the web component itself would make it safe.
Seems pretty clean. We don’t need to know how Elm adds and removes elements, and we just mutate our island’s DOM with the elements that Elm has already discarded. The only caveat is that our container for exiting elements must share a stacking context with the entering/entered elements of the page. Otherwise it starts looking wonky.
I would say that something like this would be my preferred version, if I had to do it outside of Elm. But the story doesn’t end here.
Hack city (High dependency solution)
The technically savvy client already had some experience with tricking Elm with web components. All within the same element, no islands. And it was indeed for the purpose of view transitions. But it was limited to only 1 entering element and 1 exiting element. The new requirement was to have many, or as many as possible.
It was DOM method overrides.
const VIEW_STATE_ATTR = "data-view-state" as const;
class HackTransitions extends HTMLElement {11 collapsed lines
#onEntering(el: Element) { el.setAttribute(VIEW_STATE_ATTR, "entering"); }
#onExiting(el: Element) { el.setAttribute(VIEW_STATE_ATTR, "exiting"); if (this.#exitingContainer) { this.#exitingContainer.appendChild(el); } }
#handleEnteringNode(node) { switch (node.nodeType) { case Node.ELEMENT_NODE: this.#onEntering(node as Element); break; case Node.DOCUMENT_FRAGMENT_NODE: node.childNodes.forEach((childNode) => { if (childNode.nodeType === Node.ELEMENT_NODE) this.#onEntering(childNode as Element); }); break; default: break; } }
appendChild<T extends Node>(node: T): T { this.#handleEnteringNode(node); return super.appendChild(node); }
removeChild<T extends Node>(node: T): T { switch (node.nodeType) { case Node.ELEMENT_NODE: // call #onExiting and prevent removal for element // #onExiting now has to handle future removal of element this.#onExiting(node as Element); return node; default: return super.removeChild(node); } }
insertBefore<T extends Node>(node: T, child: Node | null): T { this.#handleEnteringNode(node); return super.insertBefore(node, child); }}If I remember correctly, the original implementation that worked with 2 elements simply made sure to put the appended child before the one that was supposed to be removed, but now remains in the DOM. After that simply pray that the VDOM doesn’t look further than the 1 element it thinks exists alone.
Let’s think on our dependencies now. Looks like now we also depend on the VDOM using those specific methods to do its work. Also, currently, our VDOM will certainly be out of sync with the DOM. We have no island to hide the elements away from Elm.
The way forward
The client raised concerns that the low dependency solution causes more work for the browser, which is true. Moving an element to another parent in the MutationObserver causes a second render cycle from the browser after it already finished one. So it does make sense to highly depend on the way VDOM handles the DOM and look for changes that way.
Honestly I’d still go with the 2 containers, I’d just make sure all changes are done in the same event loop task and I’d give the containers a contain CSS property to further optimise the browser rendering process.
Specifying the contain property in CSS can tell the browser in what way to treat the container as isolated. So maybe we can win some performance in layout calculations and the paint stage of rendering. Not 100% sure how big of a difference it would make on wrapper elements so close to the base of the DOM tree.
Oh, and using will-change in CSS, we can probably get a small performance win in the compositing stage. Though to be fair, all of these can be used regardless of implementation choice.
But at this point, the client was very curious to know how far we could take his approach. Flat DOM structure was now a request, if possible. What’s another hack amongst colleagues?
The real trick
At this point, we’re already highly dependant on the VDOM implementation, and we’re overriding DOM methods with wild abandon. So let’s add one more.
class HackTransitions extends HTMLElement { get childNodes(): NodeListOf<ChildNode> { return this.querySelectorAll( `:scope > :not([${VIEW_STATE_ATTR}="exiting"])`, ); } // ...}Woah. What’s going on here? Well… when Elm’s VDOM goes to read childNodes during its render it simply won’t see our exiting views.
Using :scope references the web component itself, which is our sole container. Then after that we use :not() to negate a selector for our exiting state on our attribute. So the query returns all direct children that are not exiting. The big caveat here is that, while we technically kept the types the same by returning a NodeListOf<ChildNode>, we are now returning a static object, not a live object.
Let’s take a look at the VDOM code:
function _VirtualDom_diff(x, y){ var patches = []; _VirtualDom_diffHelp(x, y, patches, 0); return patches;}At first glance it seems like diffing mutates patches and returns them.
If we take a look at this behemoth of a file, we will indeed notice that the VDOM first makes patches and then applies the patches at the end:
function _VirtualDom_applyPatch(domNode, patch){ switch (patch.$) {25 collapsed lines
case __3_REDRAW: return _VirtualDom_applyPatchRedraw(domNode, patch.__data, patch.__eventNode);
case __3_FACTS: _VirtualDom_applyFacts(domNode, patch.__eventNode, patch.__data); return domNode;
case __3_TEXT: domNode.replaceData(0, domNode.length, patch.__data); return domNode;
case __3_THUNK: return _VirtualDom_applyPatchesHelp(domNode, patch.__data);
case __3_TAGGER: if (domNode.elm_event_node_ref) { domNode.elm_event_node_ref.__tagger = patch.__data; } else { domNode.elm_event_node_ref = { __tagger: patch.__data, __parent: patch.__eventNode }; } return domNode;
case __3_REMOVE_LAST: var data = patch.__data; for (var i = 0; i < data.__diff; i++) { domNode.removeChild(domNode.childNodes[data.__length]); } return domNode;
case __3_APPEND: var data = patch.__data; var kids = data.__kids; var i = data.__length; var theEnd = domNode.childNodes[i]; for (; i < kids.length; i++) { domNode.insertBefore(_VirtualDom_render(kids[i], patch.__eventNode), theEnd); } return domNode;
case __3_REMOVE: var data = patch.__data; if (!data) { domNode.parentNode.removeChild(domNode); return domNode; } var entry = data.__entry; if (typeof entry.__index !== 'undefined') { domNode.parentNode.removeChild(domNode); } entry.__data = _VirtualDom_applyPatchesHelp(domNode, data.__patches); return domNode;9 collapsed lines
case __3_REORDER: return _VirtualDom_applyPatchReorder(domNode, patch);
case __3_CUSTOM: return patch.__data(domNode);
default: __Debug_crash(10); // 'Ran into an unknown patch!' }}If you’re wondering where appendChild is called, you can find it in the _VirtualDom_render function.
Thankfully the VDOM (at the time of writing) just iterates and maps into an array of patches, it does not rely on it being a live object.
If you’d like to see it in action, here’s a little demo.
First conclusion
Elm successfully tricked. Client is overjoyed with the hacks. And while I do enjoy the intricacies behind this all, I was left wondering:
Couldn’t we have just written this in Elm but higher up, like wrapping the Program arguments, and avoided the user wiring issue entirely?
Second thoughts
The cool part about writing articles about problems you’ve worked on before is that they tend to pull you back into the problem. And hopefully you’ll always be a better engineer later than when you’ve worked on the problem.
While solving this problem in Elm would have probably worked, and would have been able to reuse exiting views, you’d have to consume more memory in order to hold the state for those views. Either holding entire duplicates of app state per route changed, or ever-growing diffs that need to be applied to the current app state.
Both of those sound pretty terrible to me. And in a sense, completely unnecessary.
When you think about it:
Transitions are about snapshots of the view/render, not about state.
Also, the idea of reusing created views is a bit shaky, since you can’t guarantee that the state actually matches the reused view.
Bonus experimental solution
All our previous solutions heavily rely on Html.Keyed.node for our transitionable views. This is in order to guarantee that Elm will replace our entire exiting view with a new entering view, instead of just precisely editing our exiting view parts to turn it into the entering view. This guarantee makes the views easy to work with.
All our previous work has been done with the idea of making sure rendering the previous view isn’t too expensive. And while making sure we’re not causing two full browser rendering cycles is valid, we just took it for granted that Elm will have to always tear down the entire DOM tree and create a whole new one from scratch.
But this doesn’t have to be the case, does it?
What about not using keyed nodes and taking these steps:
- catching the intent of the app to change routes
- making a copy/clone of our transitionable view
- inserting it to an “island” container for exiting views
- and just letting Elm edit parts of the original view in the most efficient way
This would probably be the most performant solution. But there’s a good chance there would be some inconsistencies in the look of the exiting view compared to what it looked like before it started exiting.
If you want to see the idea in action, here’s another demo.
Maybe it can be polished to do good work, but I leave that to the reader to explore.
Closing words
I hope you enjoyed this journey as much as I did. For me, it’s a real testament to the idea that there is no end to ideas for software problems, as long as the constraints allow it.
At the end of this, I can understand why the browser standard decided to standardise an API for this directly in the browser. I’m very excited to see it reach baseline widely available.