You may have opinion at JSConf Europe last year.
And this morning, there was a great introduction to workers.
It is exciting times again for animations in serverside JavaScript run times.
So, here I am to talk about our recent effort to bring a new set of APIs into Node Core.
This may be an opportunity for us to really think about the JavaScript API service in
the ecosystem and create a more universal developer experience across the different
platforms.
So, let's finish this that is right, I'm Joyee, I live in Hangzhou.
I worked on Gaia, and I'm a member of the Node steering committee.
On Twitter or GitHub, Joyee Cheung handle.
Some of you may be aware of this effort of bringing more web APIs into Node Core.
But maybe some folks are just curious because they hear things that are interesting.
So, in case I am using some lingoes that only a very few people understood.
So, what does it mean when we talk about bringing web APIs into Node Core? of well, we are basically
talking about adding browser APIs to the Node.js runtime as buildings.
So, you can run them on the server side without using npm install.
So, in this talk I will try to be objective and summarize how far we have come in this
journey, where we are, and where we are going.
On a teeny tiny side note, Node.js is silent.
So, like many others, I will try to use the correct pronunciation in this talk.
Here is a mental picture of how JavaScript in the wild came to be.
In the beginning before Node came along there were two kinds of APIs that came with the
runtime for JavaScript developers.
There were some JavaScript builtins like date, regular submissions, errors that were part
of the language.
It was created by system 9 and implemented by engines.
There was a host API.
And at that time hosts were basically just browsers.
These APIs were implemented by the browsers and are more or less run by organizations
like W3C.
And then Node was created a started to take off on the server side.
Initially it included some of the web APIs that were already present in the browsers.
Or like at least some APIs that look like what you'll find in browsers.
But Node also introduced several different APIs for doing certain things on the server
side like instead of even target in the browsers.
Because the semantics of the equivalent APIs on the browser side may not make sense for
servers.
Like, for example, there isn't one DOM tree in this execution context for events to bubble
through on the server side.
So, it does not really make sense to implement part of the event target specification in
Node.
So, as time goes by, browsers have developed and synthesized more APIs in the browser to
empower web developers.
While Node also developed several equivalent APIs on the server side with a didn't design.
Because of the popularity of the Node.js runtime, these home-grown APIs gradually became the
defacto factors for the server side run times.
But obviously no one likes to remember two sets of APIs for doing basically the same
thing.
So, over the past few years people have been sending feature requests to Node to bring
the two platforms closer together.
For now, it usually means making the runtime more compatible with browsers.
So, Node did evaluate and implement some of the requests and it is looking into more APIs
from the web.
So, we currently have different types of web APIs implemented in Node Core.
So, some of them are more localize the with the browsers.
And, for instance, we have methods in the browsers, and this is the living standard
of the timers in the HTML specification.
In Node we also have a set of timer APIs, that's similar, but the implementation does
not strictly follow what the browsers do.
For example, it returns an object while browsers usually return a number instead.
The other type of web APIs we have in Node Core are the ones implemented with an existing
specification in mind and behave mostly the same as the APIs that you will find in browsers.
For instance, the new URL implementation in Node Core was developed specifically according
to the URL standard.
This means we did look at the spec test when we implemented them.
And we also have tests.
So, how do we know how close our implementations are to the ones in the browser?
We run a subset of the web platform tests which are tests which are on browsers and
other implementations.
So, to see the current status of the performance in Node, you can look at the status files
under tests, the status folder of the Node project.
There are a few pretty selfexplanatory JSON files documenting the web standards implemented
in Node.
So, this one is not technically a web API addition, but I wanted to mention that there
is now another type of web standards implemented in the core.
We have collaborated with the what is the HTML specification and the first implementation
of JSON modules has landed in the master branch as part of our experimental ECMAScript module.
So, Node implemented first.
So, here we have the web APIs implemented in Node Core that can be alternatives to certain
home-grown Node build teams.
Legacy methods under the URL and the string built in module can be replaced by the new
URL and the URL search params classes.
And searching and coding and decoding methods of buffer and string decoder can be replaced
by text decoder and text decoder as well.
And there is the performance API which, for example, can be used to replace certain timing
methods in the process object.
In addition, we also have some APIs that are not exactly replacements for existing Node
APIs.
But are similar enough in many use cases that it is reasonable to consider them as alternatives.
Be sure to read the documentation if you want to rewrite your code with these APIs.
For example, cue microtask can be used to cue a microtask which will be run asynchronously
which will be used to replace the process if you do not have strict requirements about
the timing you want your task to be run.
There is also web worker which spawns threads in Node.
And it may be used to replace child processes if you are only looking for a way to upload
and you don't necessarily need processes.
So, as of Node 12, we have several web API implementations that are now stable and can
be used in production.
Those are listed on the left here.
And these are available as globals.
So, you don't have to require a builtin to get hold of them.
These are also covered by the web platform task in Node Core.
There may be some additional extensions in these APIs.
There's some minor unspecified behavior differences.
But at least those are fairly limited, and we are aware of them.
We also have a few APIs that are still experimental which means there may still be breaking changes
in the future.
These are currently placed under builtin modules and not on the global object yet.
The current experimental web APIs that we have are workers and performance time being
API.
But they also differ significantly from what you would get from the browsers.
And we do not run web platform tests for them yet.
So, watch out.
So, there is also an implementation of WebAssembly, the JavaScript API of WebAssembly that we
get for free from V8.
But the web API of it has not been implemented yet.
Other than existing APIs, there are a bunch of others that are hunting the issue tracker.
Oops.
There are still under active discussion.
They're web strings which are the foundation of web specifications.
But then we already have Node strings in Core which are also the foundation of many existing
Node Core APIs.
In case you didn't know, there are many different types of strings in Node.
There's string number one.
Which has several issues.
So, they now introduce string number two.
But then they also had some issues.
And then actually introduce string number three which still had issues.
So, there's now a new implementation of strings called Bob which is under development for
some time.
And will hopefully solve all our problems.
Anyway, this is not a string talk.
You can read the documentation if you want to learn more about them.
So, with all the strings, you can imagine how complicated it would be to bring yet another
string into Node Core.
And there is also Fetch.
Which is probably the most requested web API in Node Core.
We just had a session about it this week at the Open JS Coverage Summit in Berlin and
there is now a new work in progress pull request to bring it fresh into Node Core.
Yay!
[ Applause ] So, we have to talk about the history of these
web APIs in Node and what may be coming next.
So, why exactly are we doing this?
One obvious run is with a common API surface, there is less cognitive burden for developers.
We could their documentation, tutorials, tooling.
Instead of developing and maintaining a separate set of educational resources.
This is especially important for just beginners.
At the moment they have to choose between Web APIs and Node APIs when they are just
getting started.
With a common API service, beginners can be less distracted learning about basics.
There are still differences between the two platforms.
But it will be less intimidating when they already learned a bit more about these APIs.
So, another reason for web APIs into Node core, we have more containers compared to
more npm modules.
This is open source and it's natural that contributors come and go.
In Node, even when the existing maintainers of a specific builtin module shop less often,
we have an open governance and an effective nomination and onboarding process to bring
new contributors into the team.
Compared to regular npm modules, this kind of maintenance story fits better with the
Web APIs that are designed as builtins for the host environment.
So, if you have been paying some attention to this topic, you may be aware that it takes
an extraordinary amount of time for this API additions to be accepted into Node Core.
Here I will lay out some of the reasons why there are several requests that keep showing
up in the issue checker but have never really gone anywhere.
So, everything I'm going to talk about later are in the context of the consensusseeking
model of Node Core.
So, Node Core is operated under the consensus from over 100 core collaborators.
These are contributors who have write access to the repository.
For every technical decision, any one of these 100 more than 100 Node Core collaborators
creates an objection.
If consensus cannot be reached within the collaborators, it may come down to a vote
among the members of the tech committee who are a subset of the collaborators.
But we usually try to avoid voting.
We also take community feedback into account even if it's not from someone who has committed
into Node before.
So, here are the common arguments against adding Web APIs into Node Core.
There is still, to some extent, a small core philosophy within Node Core.
It's about providing only the basics functionalities in Core and empower users to implement user
modules instead of building our own opinionated APIs that may become a compatibility or maintenance
burden.
This philosophy has been broken several times in the past.
But at least as far as Web API goes, we are still mostly just trying to expose the existing
functionalities through a different API services.
This is not exactly an idea that's welcomed by everybody either.
Especially when the web API may also lack features that do not make sense for browsers.
Oh, yeah.
And like sometimes they may be necessary for servers.
So, one alternative to adding these APIs in Node is to release them as official modules.
Theoretically for modules that are maintained under the Node organization, it will be possible
to have a maintenance story similar to the one that Node query has.
But some may also argue that it is easier to optimize if it's done in Core because it
can use certain internal APIs.
Or sometimes it may just not be technically possible to implement something without access
to internals.
A part of the philosophical concerns, apart from those, there are concerns about the design
of the web APIs.
They're more than just a bunch of interfaces.
Behind the Design of these APIs, there is just a very different context.
For instance, the browser has a very different security model.
When you fetch an API endpoint, for example, at a script and you want to send credentials
like cookies along with the request.
Fetch as implemented according to the specifications should follow the crossorigin resource protocol
and check the access control allowed header in the HTTP response before invoking that
with the data.
So, that scripts cannot untrusted scripts cannot steal your cookies when the server
is not aware of them.
Somewhere in Node, there isn't really a concept of origins.
At least for now.
So, these are loaded from your local file system and are just trusted by default within
current security model of Node.
If you perform the request using the existing HTTP request method in Node, this security
policy would have to be implemented by the users.
Well, if they do want them.
It is not impossible to implement something like this in Node.
But this may just be confusing for most users because then we'll have two conflicting security
models in Node.
So, when we look at the Fetch specification, the interface itself is just the tip of the
iceberg.
There are many implications under the surface of the API like course origin, consent and
security policy, caching interop with service workers and like potential management.
So, some of this may make sense for now.
Some of them don't.
If we only implement part of the API that we think makes sense for Node, we may confuse
our users more because this will bring another kind of platform compatibility headache to
everyone.
There is also another type of concern.
The ecosystem has come to depend on a lot of existing infrastructure in Node.
And this may differ significantly from their web equivalence.
For example, we have different interfaces to do stringing and to emit events.
As usual, there are even more differences in the underlying design of these infrastructures.
When implementing web APIs in Node, we also need to decide whether we want to introduce
the web infrastructure into Node Core or base the higher-level APIs on the existing Node
infrastructure or just use some instruction layer instead.
We will also need to figure out the interop between these abstractions for other existing
modules.
And this work just takes a lot of time.
So, there are a lot of open questions to answer.
A lot of decisions to be made.
And this, you know, just takes time in the current this is the model in Node Core.
But how do new Web API actually ended up being added to Node Core these days?
It usually starts with a feature request opened in the Node.js/node repository.
To actually make progress on the request, someone, or some group of people, need to
step up and start a prototype.
They do not need to be Node collaborators.
It could be anyone who are willing to invest their time in the work.
Typically, they would create a fork.
Either as a personal fork or as a fork under the Node organization.
And they will hack together an initial invitation and then send a pull request back to the main
repository against the master branch.
At these stages there may be objections coming from collaborators or the community.
Sometimes the proposal just gets stalled and closed.
For example, this is the current status of the feature request for web crypto.
Sometimes, but rarely, there are no objections.
Or these objections get resolved either through discussion or through voting.
Either way, someone must be interested enough in this feature to get it through the consensusseeking
model.
Then the initial implementation may get merged into the master branch.
So, once the feature is merged in master, we'll start iterating on this.
Fixing bugs, optimizing.
At a certain point, depending how visible it is, we may start with the subset of web
platform tests with it and collaborate with the web platform tests upstring as well as
the authors to improve this spec and the test suite.
It is also typical to eventually expose these interfaces to the global object.
But this is depending on various complexities.
At this phase, the feature is still in experimental status.
Depending on the release schedule, this feature may be released to users while it is experimental.
And it may get updated in the release branch with patches back ported and the master branch.
So, this is the current status of web workers and the performance timing API.
Eventually this feature would be, or it is supposed to be, moved out of experiment and
becomes a stable feature.
This is the current status of the URL and encoding implementation.
So, here's a quick summary.
Node query APIs have diverged from the web APIs because they were designed for very different
use cases.
But more and more web APIs have now been added into Node Core.
We have work would through the existing web APIs in Node Core and their status.
Then we looked into the challenges and the workflow of bringing for APIs into Node Core
in the future.
So, finally, we are starting an open standards initiative in Node to collaborate more with
senders and other implementations.
As we said earlier, there are many questions to answer and it takes a lot of energy to
find the answers.
If you are interested, please get involved.
Thank you.
[ Applause ]