Native Web Apps: React, JavaScript and WebAssembly to rewrite native apps
Florian Rival PARISS: Okay. How was everyone's lunch?
[ Applause ] Good. I had a really successful lunch. I finally
beat all the kids to the desserts. I didn't get any on Monday. So, feeling good. Can we
give a round of applause for our next speaker speaking on native web apps, Florian Rival.
[ Applause ] FLORIAN: Hi, everyone. My name is Florian,
I'm working as a software engineer at Google. And today I want to talk about something that
I really like. It's applications. And when I say applications, it's any kind of application,
but in particular the ones that allow people to create things or to do things that they
were not able to do before. So there is something that I really like. It's video games. That's
why I created GDevelop. It's an open source and cross platform game engine that anyone
can use to create any kind of games. Why anyone? It's a visual editor for games where you can
create. And there are some kinds of visual programming. I will show an example. And anyone
can use it, even those who don't know thousand create games. And then you can get an HTML
game that you can put on the web. So, here is what the software used to look like. I
say used to because it's a software that was created, in C + +, it's a desktop C + + application.
You can see on the right, there's a list of objects in the game. And in the intermediary
of the game. Is, here it's a platform. And what we want to do as an example is to have
when the player is stepping button, we want the button on the platform on the right to
go up in the air. I will show how to do it using visual programming. And so, here we
go. I go to the events tab and I'm adding a new event. An event is a condition, an action,
kind of like an if then. I'm adding a new condition to check if the player is collision
with the tree. And if that's the case, I can add two actions. The first action would be
to change the animation of the button so it's going to the state where it's pressed. It's
animation number one. And I'm adding another action, which is an action to move the platform
up in the air. So, I'm adding to the platform a false on the Y axis. And now, if I press
play, the game is compiled to demonstrate and running in the browser. And we can verify
that the button is working. It's kind of simple for people that don't know how to program
and it's only based on logic. Syntax. So, a few games have been done with it. So, a
few were working quite well. And the game with the cat, but at some point I had some
issues. And the editor was getting quite old. I had lots of cross platform issues. In particular
on macOS and Linux. It wasn't working that well. And iterating on the software of the
quite slow, C++. And it was limited to old UI components, and the UX would need some
enhancements. And the entry barrier for new contributors was quite high. C++, again. So,
it was time for me to react. And I was wondering, would web technologies help? I have been doing
a lot of React before at work. And I was like, okay. That's perfect for making complex UI.
I was iterating. Can we have hundreds of elements, dynamic panels, 2D and 3D visualizations,
and all of the things that make a real application, you know? Can we make an ambitious application
using React? And the same goes for JavaScript. It's perfect for most applications. Or is
it really? Can we reuse an existing native codebase? Should we? Or if we have advanced
situations or computations, what do we do? Or if we need consistent performance? Still,
let's look at the quick post of GDevelop. There is the core of the software, composed
of the C++ classes and defining what a game is, what is an object, and how an event is
composed of conditions and actions. And then all the tools that are transpiring to the
events to JavaScript that you get a running game. And on the top of this there was an
interface in C++ using WX widgets. And there was the interaction with the device. So, my
first idea is, okay, let's replace the interface with React. Because I have an idea that it
could be better. And in a fortified system, I could write adapters for Node.js or even
in the browser simulator file system. We could run this on the JS or a browser or on electrons
so that we have a browser in the JS and you can package the application. And for the central
part, we could use WebAssembly to expose the code to the rest of the JavaScript application.
So, a few notes about WebAssembly. It's a kind of a language that is running in your
browsers. And it's a bit like assembly, meaning that it's not one that you write to, but what
you're compiling to. And at the end, it's simply like a language. Browsers are able
to generate really fast execution of any program that is retaining WebAssembly. And it is running
on actually most recent browsers including on iOS and Android. So, how do we write WebAssembly?
Well, there was the first solution was to use Emscripten. Which is what I did. It's
a project with a C++ WebAssembly compiler. It's a C++, running through a compiler, and
translating it to WebAssembly. And there are solutions for any language you can compile
to WebAssembly. Or even things like AssemblyScript which allows you to create WebAssembly using
a syntax that is really that looks like TypeScript. So, it's really interesting. In
my case, I have been using Emscripten. And I will show you the way it works. So, using
Emscripten, you can download it and sort it. It's pretty easy. And then getting EMCC, the
C++ to WebAssembly compiler. And you can run it on a C or C++ file. And in my case, as
I have a large codebase, I can replace the build to chain that was using the compiler,
and instead of at the end having a binary, I have a WebAssembly. And so, we have the
WebAssembly, which is the C++ codebase compiled to WebAssembly. And we have to expose the
existing classes in C++ so that we could view them in JavaScript. So, let's take an example.
For example, we have a class that is an objects container. So, it's think of it as the list
of objects that you've seen. So, we could have objects and get some objects inside.
And then we can have a layout. It's a bit like a level, again. Which is iterating from
an objects container so it can have a list of objects. And there are additional, name,
to change the name. So, also, we have this. We ought to write bindings. Bindings in my
case I have been using WebIDL. There are a few ways to write it. But in my case, use WebIDL,
we use something that is the interface of your C++ class. You have to write it for all
the classes in your codebase. And this will generate new code that will be doing the bridge
between JavaScript and the WebAssembly module. So, the way it works once you have compiles
the WebAssembly module plus the bindings is that you have a module that you can instant
ship into the JavaScript engine. So, on the browser or in the JS. And when it's an instant
change, here it's called GD. You have this module and you can start creating new classes,
new instances of your classes just like usual JavaScript. And you can call them and it's
working more or less automatically out of the box because the primitive types of your
functions are automatically converted. So, depends on the language that you're using.
C++ or another language. But in my case, C++ and JavaScript are translated to C++. Same
for numbers and for strings. It's also more or less automatic. So it's quite convenient.
If you are passing objects to the parameters, then they are converted to a pointer or a
reference in C++. And you can debug using the input and output stream that's converted
to console.log in JavaScript. With this, we have something that is running in the browser.
Here is the Chrome debugger. And you can see on the top, they also output it's outputted
by the C++ codebase. It's working. And then I can create a new object. So, yeah, it's
working. And it's quite nice to have, you know, with V C++ codebase you have been writing
since a few years. There are a few things to know. First is the memory management requires
care. Because when you're creating a new object, it's quite interesting because it's not garbage
collected. We mean that. If I'm looking at what this layout object is, it's a pointer
to some place in the WebAssembly memory. So, you can think of the WebAssembly memory as
a huge array that is containing bytes. When you're creating a new object, it's already
creating some bytes in this memory. Meaning that if you just drop the reference to the
object in the JavaScript world, the WebAssembly object will still be living in memory. So,
you have to explicitly call GD, to destroy, to remove this content from memory. So, in
the case of using React or any component based framework, if you're creating a framework
when the component is mounted, you have to remember to delete it or you can use an effect
hook if you're using React. So, know that the output files are quite large. For GDevelop,
it's at least 3 megabytes when compared to the WebAssembly module. And honestly, I don't
really care because I'm making a kind of desktop application, a rich application. So, knowing
that I will package one thing and make it a few hundred megabytes. 300 megabytes, more
or less. And something else to know, having a complete test set is really, really useful
because without the kind of heroes that you will be getting when something is going wrong
in the WebAssembly module. And for example, you have the wrong type of parameter. Or if
you forgot parameter, then instead of getting a number of boolean or a pointer in the C++
world, we get undefined which is translated, I guess, zeros. Everything will be broken.
And same thing if you're using a deleted object. Trying to iterate on something that was removed
from memory, and it will crush, basic lip. This being said, we have something that is
working in the browser. So, my next change was, oh, I can create an interface on top
of it is that as good or even better as the one that I use to do in the native world using
C++ before. So, let's see how to do it. Again in my case I was using React. And my change
was to make all of these examples. For example, context menus, trace objects, lots of forms
and buttons and properties. All of this in in the browser. So, we go through a few buttons.
An example that I have been using in the hope that it will the first advice I have is
to find a component library, material UI, React Toolbox or Blueprint. These are examples
of React's component libraries. And the things I was looking at was the extensive list of
high quality components because I don't want to spend time redoing the basic widgets. I
want to build an application, not to make a design system. And good theming support
and accessibility. And some good documentation. Because the world interface will be standing
on this component library. It better be good. I went with Material UI. It was the library
with the most extensive support. And it was allowing me to design quickly. And once I
had it, the first thing I had to get was to display a large list of objects. When they
say large list, in the case of drag and drop, a large list of objects that are living in
the game and can be a few hundred objects. So, this originally here was to use categorization.
So, for example, using react virtualized and react sortable hoc, and you can create a list.
That means that instead of having in the DOM in the browser to have a button for 300 DOM
elements for every object, then you will only have the object that are on the screen in
the DOM. So, 10 or maybe 20 objects. The way it works is that instead of having a list
of objects that is just mapping over an array, you're converting your list to be using the
list component that is referred to by the list and using this property. And you can
say this is the function that you will call to identify every line that is to display
on the screen. And then using React sortable hoc, you can add on top of the list. And you
are using sortable elements and you get a list that is virtualized and you can reorder
and drag and drop. That was the first way to get things that not but working well
even with large lists and big games. And the next thing is that I wanted to have patterns.
And it was as on the screen. As I'm doing a game, people want to customize their workspace.
And so, for this I have been using a tiling window library which is called a react mosaic.
There are a few different libraries like this. But this is the one I've used and it's I
mean, it's working out of the box. So, check it out. Then I've been trying to remake the
events. It's more or less large trees. A large trees arrays with events. An event is composed
of conditions and actions. But it can also have sub events like blocks of code in a traditional
programming language. So, here, again, the solution was to use a virtualization. Previously
I have been using React sortable tree which is a library to make a tree of nodes which
you can reorder. And the interesting thing about this library is it's using React virtualized
under the hood. So, quickly, every node that you see on the screen here is actually a line
in the list. And the lines that are in between the node is just a fake scaffolding that is
making like that is making the tree look like a tree and another a list. So, it's a
customized version of what I have been using. And then at some point I wanted to remake
it was the first thing that I did. And at this moment, I wanted to display the screen
so it can be composed of individual tiles and objects to be displayed. And at this moment
you can forget the DOM and go to be using WebGL. Not using WebGL directly, but a library
called Pixi.JS. Which is the 2D library that I have been using for the game engine. I can
as well use it for the game. And it's super-fast and easy to use and based more or less on
the Flash API. And they also have options like Three.js or Phaser if you want something
that looks more like a game engine. Sometimes you can still rely on the DOM, for example.
Say you have an editor that is displaying an object, here a spaceship, and you want
to know what is the box for the spaceship? What I have been doing is using SVG elements
to display this protocol. And using things like onPointerMove in the DOM, you can drag
and drop. So, you get something a pretty good editor that is very easy to architecture
and to act on because it's all done using React. And it's enough for what you want to
do. Sometimes things are not fast enough. So, the first performance that you can have
is that rendering the picture is too slow. At this point, you can profile the rendering.
Here in the case of React, you can use the React profiler. And in this case, I had an
interesting problem. When I was opening or closing a tab, I had to do the whole thing
was taking like 1 second to be done. And when using the profiler, you can see that every
it's displaying every component that is re rendered when you're doing interactions. And
on the graph, the code, is re rendered when you're doing interactions. And you can see
from right to left I had a whole set of components that was deploying the project manager. The
project manager, it's a bit like a file explorer. And this thing was being re rendered. It was
switching from one user when it was written. So, this was to add some updates or organization
so that it's not re rendered every time you're doing an interaction. So, basically avoid
re rendering if you don't need to. Something to note is that make sure that you measure
performance in production. Because in the case of React, the development build is way
slower than the production build. And something that's I got an issue where I was drag and
dropping things and it was taking like 1 second again to update. But it was a few minutes.
So, I built it in production. And you can have an issue with the calls to WebAssembly.
If you have a lot of calls being made, you're paying the calls to go through the bridge
between JavaScript and WebAssembly every time you are calling a function. This is a case,
I think, Firefox, at least has been optimizing this use case. But if, for example, you have
a list of lots of objects and every time you're mapping on the list, you're calling a function,
at the end, it can add up. It won't be noticeable in most situations. But something to remember
that sometimes it might be useful to store in the JavaScript, to cache in the JavaScript
world the results of some computation if you want to avoid paying the cost to go through
the bridge to do WebAssembly. Sometimes you're reaching a point where your codebase is growing
a lot. So, a few tips on this. In my case it was because I did the project, I got more
and more contributors. The first thing I will tell you is to stick to your components. So,
the idea is that if you have a button that is showing to open the software, you might
think that's a let's change the design of the button so it's smaller on screen. You
can add a font size that is different. What I would recommend is to make a help button.
Basically to create components for as much things as possible. In other words, make a
design system and try to stick to it. Because once you've been doing this thing, it will
be much faster to build new screens and consistent screens. Something I don't know is I'm not
doing any unit tests. Sorry. Actually, I do, but only there is a bit of logic in my components.
And I think it's the case in most it's really where what you are doing that you need tests
for react components. Instead, I'm creating visual stories. So, using Storybook or React
Styleguidist, you can create stories that are displaying your components outside of
your application. I found this button to be faster and way more powerful to first develop
components outside of the application. So, much faster because rather than having to
load the application and then to navigate to a screen to see that your changes are right
or not, you just have to run Storybook and go to the proper story. It's much faster for
development and also for testing. It's not automatic. But in the future, I'm changing
or updating the UI. Or I'm making changes on the component, I can quickly open Storybook
and see my component in a different state and make sure that everything is correct.
It's in particular useful for cases like, for example, to have the handling, the component
showing something that you have never seen when you're developing because everything
is going all right. But at least using Storybook, you can make sure that your components all
have a neutral state that is working. The next advice would be to use types. Flow or
TypeScript. I have been using flow because I was more used to it. And the useful things
with types that they will tell you when you forgot to pass or remove a product or refactoring
something. And help auto completion, avoid silly mistakes and help document objects.
But not useful when you write code, but when you're back on a module that you or someone
has wrote a few months ago. In other words, types are really shining when you or someone
else is coming back and refactoring some parts of the application. And after a few months,
you are a stranger to your own codebase. So, this being done, we now have the core of the
software and an interface that is being made on top of it. And let's see how to package
this. So, we have a few packaging options. We could make a PWA because WebAssembly is
running in the browser. So, it's not an issue. I have been using Electron to have access
to the design system. And also people like to download the application, to own it, in
a way. And having access to the system, you can avoid making something that is only working
on online. And you can use the software offline. And in some cases you can even embed the JavaScript
application inside the native application using Chromium embedded framework. If you
have an application, and you have the manpower to do it. And my advice would be to build
over the native API as much as you can. So, for example, I did two versions of the context
menu. One that is using on the left the map UI context menu, running in a browser. And
on the right, you can seeing the menu using APIs. So, you have the context menu. And I
won't go through the code. But instead of renders some material, I can use the UI to
create this context. And this is helpful for people not helpful, but that makes the application
look like a native application. It does not look web ish. And, yeah. We can know now what's
the result of all of this? Let's take a look. So, this is a new application. And the demo
I'm doing here is the same as before. I will be creating a button object with one when
it's pressed and one animation when it's not pressed. And I will be adding it to the screen
after the button. And doing the same thing as before with the event. When the player
is stepping on the button that the platform is there. So, that's the same more or less
interface as before. But all using React WebAssembly and JavaScript. And here I'm choosing the
player and the button. And there is a condition between these two objects. Then I'm adding
two actions. The first one is to add a false so that the platform is going up in the air.
And then another direction so that the button is going to the state where it's pressed.
So, changing the animation. That's how you do it. And when that will be pressing play,
same as before, the WebAssembly module is going to JavaScript by putting files in and
you get a working game in the browser. Yeah. That's working. In my definition, as a developer,
it's working. But what do our users say? I will be honest. The first feedbacks were really
bad because not because it wasn't the application or built using React or it was using web technologies,
it was just because the new version was lacking a lot of features from the previous one. Even
if I did reuse the existing codebase, I had to remake the whole interface around it. So,
of course I had started a few things and adding new things. So, it was lacking features. But
after a few year D a few years. After one year, okay, there is a real difference. And
it's easier to use and actually I can be more productive using the new version rather than
the old one. And it works. It works so well it's better than the native application. Why
better? I can do ultrafast iteration on it, I can test super easily. I have a near perfect
cross platform and I have a faster startup time. It's faster. Autoupdates, more contributors.
We can try it online and we can even run it on phones. I mean, running it on a phone.
It's not made for a phone so you cannot really use it properly I mean, you can use it properly.
But it will need some enhancement to do it. But my point is that it's working almost everywhere.
So, this was to show that actually we can do things using React, WebAssembly, JavaScript
and all of this new frontend framework can do things that are not only as good as what
we used to do in the native world, but we can be more ambitious. Thank you.
[ Applause ]