The importance of Scala.js

Scala.js has the potential to change the way we compute.

Published on Page is printer-friendly.

The Scala.js compiler 1, made by Sébastien Doeraene 2, turns Scala 34 code into JavaScript, enabling it to run on any web browser and anywhere that Node.js is installed.

Programmers often ask me were Scala.js stands in the bigger picture, when it should be used and if it even makes sense when there is TypeScript; the behemoth who has already taken over the likes of PHP 5 with nearly 20 million weekly downloads 6. TypeScript is proof that strong-typing features of languages like Scala are in high demand, but also a testament to the importance of accessibility and wide distribution - things that Microsoft does particularly well, especially in the recent years, even convincing developers to repatriate back to Windows 7.

Why would a frontend developer take on Scala.js when they probably already have 5 browser tabs open for HTML5 APIs, JavaScript APIs, WebPack documentation, TypeScript guides and the application itself they are developing on? Only to add an extra few tabs: for SBT, Scala and Scala.js. At this point, it becomes a hard sell, but there are very strong cases where Scala.js is in fact a wiser choice. The market of single-page applications is highly competitive with very cool technologies like Svelte 8 and Reason 9 that already compile to JavaScript and offer many benefits of Scala 10, however we will explore different directions where Scala.js is spectacular, including non-frontend uses.

Scala.js enables distribution of applications through npm

> npm install -g scalajs-cli-demo
> scalajs-cli-demo
Hello world from Scala.js!
----
To demonstrate usage of Node.js api, here's some of your environment:
ALLUSERSPROFILE = C:\ProgramData
...

This works Windows, macOS and Linux. You, as the user, did not need to download & install Java just to run my app. As the developer, I did not need to figure out how to package my app with Java and then upload binaries for various OSes. This simply means access to a much wider audience, because the tool is now accessible natively through a widely-deployed ecosystem. I have to say, to do this and get it on npm 11 was very straightforward. The sources are available on GitHub 12.

Scala.js benefits fully from the npm ecosystem

The key ingredient of Scala's success is its native Java interop: hundreds of thousands of Java libraries to do every single task you may need, all at your disposal. Every new improvement to GraalVM is a free improvement for Scala. Because Scala.js targets JavaScript, every improvement to JavaScript (including npm/Node.js) is yet another improvement to Scala.js and Scala.

My excuse for trying Scala on Node.js was simple: I wanted to synchronise some logs over an HTTP EventSource stream, and at the time in 2017, I could not find a single bug-free EventSource library on the JVM. On the other hand, npm had one just sitting there 13, battle-tested across many browsers, and it was a completely painless experience to use this library. All it then took was to publish my app to npm, install it on my target machine and just run my Scala. It uses the full suite of usual Scala things: unit tests in ScalaTest, Optional types and case classes; available on GitHub 14.

Want to publish a trendy desktop app like VS Code or Slack? Electron 15 is at your disposal with its Electron Packager 16 that gives you native installers for all the major platforms. Want to make video games that run both on the browser and the desktop? The Indigo engine 17, built with Scala.js and Electron, enables you to do exactly that, but with all the huge benefits of pure-functional programming, tried & tested tooling, state machines 18 and zero-cost backend-frontend integrations, as well as TypeScript definition support 19; leveraging Scala and TypeScript to the maximum. Or maybe you just want to control Chrome with Puppeteer 20. Or use the Express framework, or run on IoT and serverless platforms.

Scala computations can run in the frontend and the backend

This use case is close to my heart, as I had worked on multiple trading platforms; you must do many computations and validations, to make sure the user gets the correct prices quotes and invalid states are impossible to represent, especially where currencies, exchange rates, and different parameters for financial instruments are concerned. Because of user experience (as you would rather tell the user immediately that something is invalid), you end up reimplementing that logic on the frontend, and in many projects it is done in a different language. Inconsistency ensues, primarily for the reason of duplicated code running out of sync.

Diagram showing the Scala web stack, in particular an arrow demonstrating the link between frontend and backend using Scala itself.
The Scala web stack is a single codebase that has some shared code that is JS/JVM independent ("Pure Scala"). Typically such a project is built with Scala's build tool SBT. Such a project eliminates any possibility of mismatches between the frontend and the backend because the definitions are shared. Example projects are available 22.

However, one trading platform I worked on did have Scala.js, and it was an absolute pleasure to have the same validation code run on the backend and the frontend at the same time. The user gets instant feedback about what is possible, and when they press 'Buy', it always works. No more errors like 'Cannot read property 'y' of undefined' and scrambling to fix production bugs, that I have faced myself even with TypeScript. To do that, the app does not even need to be fully in Scala.js, although it is definitely advantageous.

Computations don't just mean simple sums and state processing; it means orchestrating your application declaratively across the frontend and backend, fusing the two into one cohesive and coherent application through macros, type-level programming 39 and other aspects that are mostly in niche programming languages.

Impact; killer apps

A diagram with arrows pointing from TypeScript/Reason/Svelte to JavaScript
Type-safety 29 and JavaScript are in full hype mode at the moment with compilers like Svelte 24 attaining 42k GitHub stars 23 — don't tell me people don't care about type-safe and correct apps!

Apache Spark was the killer app for Scala, and lead to the conquest of the Big Data niche 21. Scala.js is giving us an opportunity to create another killer app for Scala; given the JavaScript hype and type-safety hype29 24, Scala.js has a window of opportunity to serve an important need of many frontend programmers. The main thing standing in its way, in my opinion, is the dependency on the JVM. Let's take some inspiration from Anders Hejlsberg 25 and Rich Harris 26 and make Scala a native player in this JavaScript ecosystem - their compilers run on JavaScript after all. Once the JVM is no longer a dependency, we are immediately free to run Scala directly in the browser and aim towards that truly mass developer adoption. But let's go further, as there is an entirely different class of mass adoption we could be looking at.

The movement towards end-to-end-encryption

More recently, end-to-end encryption has become a huge deal to users 34. The further we go, the more users demand their data to be processed on their devices, and stored remotely only for synchronisation purposes, rather than relying on remote servers. With the example of Notion, client-side full-text search should be entirely possible to do on the client-side, but this requires a congruent framework that can run on the backend and frontend which is very achievable in the Scala arena.

Full-stack development includes end users

I was highly influenced by Bret Victor's "Learnable programming" 31 32. In part due to this, for years, I wanted to enable my users to compute data directly, without having them (or me) set up a development environment, and download gigabytes of raw data. This would enable them to figure out new exciting views on the data, and I could easily productionise them into scalable pipelines and deliver to users in record time. This desire lead me to explore solutions like Nashorn 30 that let you run JavaScript on the server-side, and Scala via JSR-223, with plenty of issues 33. But the time was not yet ripe.

Diagram of Excel linking with VBA and C#
A part of the typical enterprise stack in any finance company. Usually the end users will develop spreadsheets with computations, then for the side-effects (external interactions), their more technical colleagues or developers will write glue code in Visual Basic for Applications. And finally, were sophistication and full on integrations/reliability are needed, C# will come into the picture.

When you hear 'full stack', you hear 'backend', 'frontend', but nothing about the end user. In fact, the end user is the consumer of the the world's most popular programming language, which happens to also be pure-functional: Excel 35. Used in millions of production systems but almost never on the backend; its main difficulty is that it is not particularly scalable - hence why most serious production systems involving Excel have a Visual Basic layer, combined with another C# layer, which then speaks to external systems. Perhaps this is why very recently, Microsoft released Power Fx 36, possibly one of the most important creations for end-user computing following Microsoft Excel. Power Fx is a 'a general-purpose, strong-typed, declarative, and functional programming language.', targeting 'No-Code, Low-Code and Pro-Code' users 37. Simon Peyton Jones, the co-developer of GHC, the Haskell compiler, is working on Lambdas for Excel, at Microsoft, right now.

Unifying the language stack enables future projects where end users develop spreadsheets, the developers integrate them into production systems as simple functions - because in the end, that is what many of our systems do boil down to. Developers can enhance them with property tests, performance tests, and various other possibilities achievable through static analysis. In theory this is possible already but the mismatch between how Excel does things and how any other language does things is currently too great.

Computations run on the client-side, or on the server-side, wherever it is appropriate; Spark and Big Data is at the fingertips of every user out there. In fact, these ideas were already explored in Scala but were very ahead of their time because Scala.js was very young back then. 38

Full-stack Scala

A diagram of what Scala in the full stack could look like: a backend talking to a frontend which is talking to a spreadsheet.
Scala in the full stack could look like this: computations could be shared all the way from the backend to the end user, implemented by the end user; a REPL available to the end user to do live querying and experimentation for the end user; REPL also available to the frontend and the backend developers. Scala could be presented in the web browser with a simple <script type="scala"> tag, no build tool needed (just 1 import).

The future is in eliminating barriers between end-user computing, the frontend and the backend. Scala is currently at an advantageous position because it is scalable. A scalable language, Scala, should be accessible to end users, not just developers. Given the incredibly wide distribution of JavaScript, if Scala can compile & run entirely on the web browser, end users will be able to write Scala formulas, it could provide the world with its first and final full-stack programming language. The language is there 39, we only need the tools in place.

References

  1. Scala.js

    — Build robust frontend web applications in Scala with Scala.js
  2. Sébastien Doeraene's personal Website

  3. The Scala Programming Language

  4. What is "Scala"? (Scala Algorithms)

    — A condensed jargon-free explanation of what Scala is in the bigger picture of things

  5. Stack Overflow Developer Survey 2020

    — Nearly 65,000 took this comprehensive, annual survey of people who code. Demographics. Most loved, dreaded and wanted technologies. Salary and careers.

  6. typescript

    — TypeScript is a language for application scale JavaScript development
  7. Why I ditched macOS, Linux, and chose Windows for development work — The CRUD blog

  8. Svelte • Cybernetically enhanced web apps

  9. Reason

    — Reason lets you write simple, fast and quality type safe code while leveraging both the JavaScript & OCaml ecosystems.
  10. From first principles: Why I bet on Scala.js

  11. scalajs-cli-demo

    — POC to see if we can publish ScalaJS apps on npmjs.

  12. ScalaWilliam/scalajs-cli-demo

    — Scala.js in your NPM as a CLI app. Easy to publish! - ScalaWilliam/scalajs-cli-demo

  13. EventSource/eventsource

    — EventSource client for Node.js and Browser (polyfill) - EventSource/eventsource

  14. ActionFPS/actionfps-clone-logs

    — Clone live ActionFPS logs into a file. Contribute to ActionFPS/actionfps-clone-logs development by creating an account on GitHub.

  15. What Is Electron and Why Is It So Polarizing?

    — As application frameworks go, nothing is quite as loved or dreaded like Electron. Why is that? Find out more in this article.

  16. electron/electron-packager

    — Customize and package your Electron app with OS-specific bundles (.app, .exe, etc.) via JS or CLI - electron/electron-packager

  17. Indigo · A game engine for programmers.

    — A game engine for programmers.

  18. State machine (Scala language concept)

    — A state machine is the use of `sealed trait` to represent all the possible states (and transitions) of a 'machine' in a hierarchical form.

  19. The Javascript ecosystem for Scala.js! · ScalablyTyped

    — Typescript is now used to describe almost all useful Javascript libraries.

  20. Tweet by Jos Dirksen

    Finished #puppeteer and #scalajs based image scraper and downloader. Got images I wanted for desktop. Slightly overengineered but fun to do!

  21. We Won! How Scala Conquered the Big Data World

    — In 2012, Dean Wampler claimed that Hadoop, and Big Data in general, needed Functional Programming and an alternative to Java. Today he says that Scala has su...     ()
  22. ChristopherDavenport/http4s-scalajsexample — Contribute to ChristopherDavenport/http4s-scalajsexample development by creating an account on GitHub.

  23. sveltejs/svelte

    — Cybernetically enhanced web apps. Contribute to sveltejs/svelte development by creating an account on GitHub.

  24. Svelte, why so much hype ?

    — Since the release of version 3 in April 2019, Svelte has been in the news all the time. But why? What...
  25. Anders Hejlsberg: Introducing TypeScript — TypeScript is a programming language that makes it easier to write cross-platform, application scale, JavaScript that runs in any browser or in any host. Microsoft Technical Fellow Anders Hejlsberg e
  26. Rich Harris: Futuristic Web Development — Edit from the future: check out our followup podcast interview on SvelteKit https://share.transistor.fm/s/859bbd95---This was recorded for Svelte Summit 2020... ()
  27. The Ur Programming Language Family
  28. Adam Chlipala
  29. TypeScript, Seeing Past the Hype - Matthew Gerstman at ReactiveConf 2019 — Matthew Gerstman, senior software engineer at Dropbox, recently presented at ReactiveConf 2019 in Prague the lessons learnt by Dropbox after they migrated to TypeScript four years ago. Gerstman described the good, the bad and the ugly parts of TypeScript, and how TypeScript helped Dropbox to tackle programming at scale.

  30. Scala Nashorn Interaction

    — Scala Nashorn Interaction example with Single Abstract Method and JavaBeans.
  31. Learnable Programming
  32. Bret Victor The Future of Programming — "The most dangerous thought you can have as a creative person is to think you know what you're doing."Presented at Dropbox's DBX conference on July 9, 2013.A... ()

  33. Scala ScriptEngine doesn't seem to be working as before in 2.12.1/2.12.0 via SBT.

    — Scala ScriptEngine doesn't seem to be working as before in 2.12.1/2.12.0 via SBT. - sample.bash

  34. r/Notion - Comment by u/SamLovesNotion on ”🔒 End to End Encryption is MUST!”

    — 289 votes and 87 comments so far on Reddit

  35. Excel Meets Lambda - Prof. Simon Peyton Jones & Prof. Andy Gordon - The Archimedeans × UCCPS

    — Prof. Simon Peyton Jones is a computer scientist who specialises in functional programming languages. He is most known for being one of the designers of Hask...     ()

  36. Microsoft Power Fx overview - Power Platform

    — Overview of the Power Fx language

  37. What is Microsoft Power Fx?

    — Microsoft Power Fx is the low code language for expressing logic across the Microsoft Power Platform. It is the same language that is at the heart of Microsoft Power Apps canvas apps today and is inspired by Microsoft Excel. It enables the full spectrum of development from no code to pro code with no cliffs in between, enabling diverse teams to work together and saving everyone time and money.
  38. Spores: Distributable Functions in Scala — Spores are "small units of possibly mobile functional behavior". They're a closure-like abstraction meant for use in distributed or concurrent environments. ()
  39. Tuples bring generic programming to Scala 3

Subscribe to my newsletter or feed

Comments

For comments, drop me a note on hello@scalawilliam.com.