Let's Encrypt http4s!

The Goal

To run a Scala-http4s HTTP server with Let's Encrypt's certificates on a Linux server.

The Motivation

This tutorial is developed as part of the Let's Encrypt Scala project. My motivation to write this article is to show http4s in the real world, and also announce the new Let's Encrypt Scala project. Your motivation to read this article could be to deploy production apps yourself without having to deal with nginx; or you could just be evaluating whether http4s is suitable to what your company is doing.

If you ever tried to use a Let's Encrypt certificate on the JVM, you will probably have faced a whole world of pain of manual steps (thank god they exist!) in importing keys into Java KeyStores, and then hooking those KeyStores into your SSLContext. Here we show how it can be done more simply.

This tutorial is composed of three key parts: prerequisites, developing the app, and deploying it.

Prerequisites (DevOps & UNIX/Linux required!)

Before we even get to Scala, this will be the most time-demanding part, especially if you have not done DevOps before.

As a gentle note of wisdom: I self-manage only a handful of production apps, that are not of critical importance, and for who have paying subscribers like Scala Algorithms, I simply use Heroku because it comes with the full suite of things for running apps in production: monitoring, log management, certificate management, failover, deployments, and so forth. All of this is very complicated to do yourself. We will not go into how to do this on k8s because production k8s should only be attempted with expert guidance.

Now that we're past this reminder, let's get into the key things that we need to prepare before we can even code up a single line of code, and that is to prepare the deployment environment, details of which are out of scope for this article; but I will include a few resources to get you started if you are new to DevOps. You will also need your own custom domain. If you are on Windows, you will need to set up the Windows Subsystem for Linux in order to run Linux commands like ssh.

  1. Have a Linux server - the simplest and most popular to use are DigitalOcean and Vultr (consider referral links too: DigitalOcean; Vultr). I use Vultr when I need high CPU performance, and DigitalOcean when I want to create one-click servers with pre-loaded apps.
  2. Point a custom domain name to your Linux server (Vultr DNS / DigitalOcean DNS).
  3. Ensure you have a non-root user and you can access it with: DigitalOcean instructions.
  4. Set up certbot on your Linux server
  5. Ensure that Port 8443 is accessible on your server. Some hosts like Vultr block most ports by default. Read: Vultr Firewall.
  6. On your server, set up Java with SDKMan so that our web app can run on the server
  7. On your Shell environment, set up SBT, so that we can prepare our app bundle.

For convenience, you can copy-paste commands from this webpage to your text editor (for security), and then your Shell. But you can also type them out.

For another convenience, if you update the following fields, the snippets will update automatically:

A sanity to check all the pieces are in place:

If any of these commands produce an error, try to resolve it.

# Ensure 'i-am' can see the cert directory (but not certs)
ssh root@scala-geek.info setfacl -m u:i-am:X /etc/letsencrypt/{archive,live}

# Ensure 'i-am' can read the certificates / private keys
ssh root@scala-geek.info setfacl -R -m u:i-am:rX /etc/letsencrypt/{archive,live}/scala-geek.info

# verify the certificate is accessible to the user
ssh i-am@scala-geek.info ls /etc/letsencrypt/live/scala-geek.info/cert.pem
# check Java is available to geek
ssh i-am@scala-geek.info java -version
# Check we have SBT working
sbt -version

Develop the app

Do the development in a directory called letsencrypt-scala. Alternatively, use the Git repository, from which the code is synchronised to this tutorial:

git clone git@github.com:ScalaWilliam/letsencrypt-scala.git

If you use the Git repository, skip over to Deploy & run the app.

All the explanations are in the code comments; do have a read.

Create the Scala build definition file build.sbt:

organization := "com.scalawilliam.rad4s"

scalaVersion := "2.13.6"

// Allow to package the whole app and its dependencies to a ZIP file
enablePlugins(JavaServerAppPackaging)

version := "0.1"
val Http4sVersion = "0.22.0"

// To be able to use the letsencrypt-scala library
resolvers += Resolver.sonatypeRepo("snapshots")

libraryDependencies ++= Seq(
  // Always have this
  "org.http4s" %% "http4s-dsl"  % Http4sVersion,
  "org.http4s" %% "http4s-core" % Http4sVersion,
  // Render HTML with scalatags
  "org.http4s" %% "http4s-scalatags" % Http4sVersion,
  "org.http4s" %% "http4s-server"    % Http4sVersion,
  // http4s has various back-end implementations, even Servlets!
  "org.http4s"       %% "http4s-blaze-server" % Http4sVersion,
  "com.scalawilliam" %% "letsencrypt-ce2"     % "0.0.5-SNAPSHOT"
)

// To enable local development without a certificate
// use the '~ reStart' command in SBT to try this out
reStartArgs += "--insecure"

Global / onChangedBuildSource := ReloadOnSourceChanges

Create the plug-in definition file project/plugins.sbt:

addSbtPlugin("io.spray"         % "sbt-revolver"        % "0.9.1")
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.8.1")

Create the main app: src/main/scala/LetsEncryptHttp4sSampleApp.scala

package com.scalawilliam

import cats.effect.{Blocker, ExitCode, IO, IOApp, Resource}
import cats.implicits.toSemigroupKOps
import com.scalawilliam.letsencrypt.LetsEncryptScala
import org.http4s.HttpRoutes
import org.http4s.blaze.server.BlazeServerBuilder
import org.http4s.implicits.http4sKleisliResponseSyntaxOptionT
import org.http4s.server.staticcontent.resourceServiceBuilder
import org.http4s.server.{Server, defaults}

import java.net.InetAddress
import scala.concurrent.ExecutionContext

object LetsEncryptHttp4sSampleApp extends IOApp {

  /**
    * The pure HTTP parts.
    * This code you can re-use easily in a Jetty or Tomcat servlet
    * or any other server supported by http4s. It's really amazing!
    */
  private def httpRoutes: HttpRoutes[IO] = {
    import org.http4s.dsl.io._
    HttpRoutes.of[IO] {

      /** Capture all the requests */
      case req @ GET -> _ =>
        import org.http4s.scalatags._
        import scalatags.Text.all._

        /** We use scalatags - a cross-platform (Scala-JVM & Scala.js) solution to render HTML without the noise */
        Ok(
          html(
            head(link(rel := "stylesheet", href := "/style.css")),
            body(
              h1("Hello, ",
                 img(src := "/Scala-full-color.svg", alt := "Scala logo"),
                 "!"),
              p(s"Here are the details from your request:"),
              code(req.toString),
              p("Other information:"),
              table(
                tbody(
                  List[(String, Any)](
                    "Remote"          -> req.remote,
                    "Remote user"     -> req.remoteUser,
                    "From"            -> req.from,
                    "Server"          -> req.server,
                    "Server"          -> req.serverAddr,
                    "Server"          -> req.serverPort,
                    "Is secure?"      -> req.isSecure,
                    "Server software" -> req.serverSoftware,
                    "Attributes"      -> req.attributes.toString,
                  ).map {
                    case (k, v) => tr(th(k), td(v.toString))
                  }
                )
              )
            )
          ))
    }
  }

  /** Server assets; in particular the stylesheet for nice visuals */
  private def assetRoutes =
    resourceServiceBuilder[IO](
      "/web-assets",
      Blocker.liftExecutionContext(ExecutionContext.global)
    ).toRoutes

  private def baseServer =
    BlazeServerBuilder
    /** In Scala, execution context allows parallelism, which is important for an HTTP server */
      .apply[IO](ExecutionContext.global)
      .bindHttp(
        /** Get the port and bind hostname from the environment */
        port = sys.env
          .get("HTTP_PORT")
          .flatMap(_.toIntOption)
          .getOrElse(defaults.HttpPort),
        host = sys.env.getOrElse("HTTP_HOST",
                                 InetAddress.getLoopbackAddress.getHostAddress)
      )
      .withHttpApp((assetRoutes <+> httpRoutes).orNotFound)

  private def secureServer: Resource[IO, Server] =
    LetsEncryptScala
      .fromEnvironment[IO]
      .flatMap(_.sslContextResource[IO])
      .flatMap(sslContext => baseServer.withSslContext(sslContext).resource)

  override def run(args: List[String]): IO[ExitCode] = {

    /** To develop locally */
    if (args.contains("--insecure")) baseServer.resource else secureServer
  }.use(_ => IO.never).as(ExitCode.Success)
}

Deploy & run the app

We set up all the environment, developed the Scala. The hard groundwork has been done :-)

Run the following to deploy the app:

(cd letsencrypt-scala && \
sbt 'show dist' && \
scp target/universal/letsencrypt-scala-0.1.zip i-am@scala-geek.info: && \
ssh i-am@scala-geek.info rm -rf /home/i-am/letsencrypt-scala-0.1/ && \
ssh i-am@scala-geek.info unzip letsencrypt-scala-0.1.zip)

This will create a deployable ZIP archive of our app and then upload & extract it.

Run the following to start the app:

ssh i-am@scala-geek.info \
    HTTP_PORT=8443 \
    HTTP_HOST=0.0.0.0 \
    LETSENCRYPT_CERT_DIR=/etc/letsencrypt/live/scala-geek.info \
    /home/i-am/letsencrypt-scala-0.1/bin/letsencrypt-scala

Now you should be able to go to https://scala-geek.info:8443/ and see some request information.

Screenshot showing successful access to the newly created HTTPS website
Here is my result, and it worked first time!

Overview

We developed an http4s app that uses Let's Encrypt certificates, and communicates to the user directly. I highly recommend Heroku for production apps, but if you are experimenting, learning, or want to self-manage for good reasons, then I hope this tutorial really helped you!