Friday, May 21, 2021
Async in Swift
I have glimpsed the future of async and await in Swift and it is gloriously mundane, just as it should be.
Eager as I am to see language-level support for asynchronous programming land in Swift, I was very excited to see the Vapor project launch branches of key components. Naturally, I wanted to try them out.
Although some asynchronous features landed in Swift 5.4, it seems like Swift 5.5, currently in an alpha stage, is likely to be a watershed release when it comes to built-in support for the co-routine style of concurrency enabled by the async and await keywords. Naturally, then, I wanted to test things out in Swift 5.5. In order to do so both in a Linux environment (I’m mostly interested in Swift for server-side programming) and without affecting my day-to-day environment, I decided to take advantage of Microsoft Visual Studio Code’s Dev Containers feature.
Swift 5.5 in Visual Studio Code’s Dev Containers
VS Code’s Dev Containers are a great way to engage in server-side (i.e. Linux-based) development from your normal desktop environment — be that Mac or Windows. But there is some setup involved, especially if you are going to use an alpha-stage version of Swift for which pre-built container images are not yet available. There’s a lot to say about Dev Containers and how they work but I’ll leave most of that to Microsoft’s resources. The TL;DR version is this: if you add a .devcontainer folder to your project with a few key configuration files, you can have VS Code automatically build a complete Linux-based, Docker-based development environment. If you are doing server-side development and need access to databases or other similar resources, you can use Docker Compose to include those dependencies. And if you commit that folder to your source repository, anyone on your team can easily and automatically get the same environment.
Here, then, is my server-side Swift Dev Container configuration:
.devcontainer/devcontainer.json
{
"name": "Swift 5.5 Playground",
"dockerComposeFile": "docker-compose.yml",
"service": "app",
"workspaceFolder": "/workspace",
"settings": {
"terminal.integrated.defaultProfile.linux": "/bin/bash",
"lldb.adapterType": "bundled",
"lldb.executable": "/usr/bin/lldb",
"sde.languageServerMode": "sourcekit-lsp",
"sourcekit-lsp.serverPath": "/usr/bin/sourcekit-lsp"
},
"extensions": [
"vknabel.vscode-swift-development-environment",
"vadimcn.vscode-lldb",
"ms-azuretools.vscode-docker",
"mtxr.sqltools",
"mtxr.sqltools-driver-pg"
],
"forwardPorts": [5432, 8080],
"remoteUser": "vscode"
}
This is, more or less, a standard VS Code configuration file with a few additional keys specific to Dev Containers. You may notice the inclusion of Swift’s language server, SourceKit-LSP; we’ll talk more about that later. Before that, we should take a look at that docker-compose.yml file:
.devcontainer/docker-compose.yml
version: '3'
services:
app:
build:
context: ..
dockerfile: .devcontainer/Dockerfile
volumes:
- ..:/workspace:cached
# Overrides default command so things don't shut down after the process ends.
command: sleep infinity
# Runs app on the same network as the database container, allows "forwardPorts" in devcontainer.json function.
network_mode: service:db
# Uncomment the next line to use a non-root user for all processes.
user: vscode
env_file: .env
db:
image: postgres:13.2
restart: unless-stopped
volumes:
- postgres-data:/var/lib/postgresql/data
environment:
POSTGRES_USER: playground
POSTGRES_DB: playground
POSTGRES_PASSWORD: {make up a password}
volumes:
postgres-data:
The db section ensures that we have a working instance of the estimable open source SQL database, PostgreSQL. (Note also in the devcontainer.json file that Postgres’ port — 5432 — is forwarded so we can access the database outside of the container and even outside of VS Code.) The main action, though, is in the app section; this incorporates the container in which we will do our development. A .env will load any environment variables we need (handy for database connection information), a workspace volume is defined so that our code will live both inside and outside the container, and the Dockerfile that defines the container is called out. Let’s take a look at that next:
.devcontainer/Dockerfile
FROM ubuntu:20.04
RUN export DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true && apt-get -q update && \
apt-get -q install -y \
binutils \
build-essential \
git \
gnupg2 \
libc6-dev \
libcurl4 \
libedit2 \
libgcc-9-dev \
libpython3.8 \
libsqlite3-0 \
libsqlite3-dev \
libstdc++-9-dev \
libxml2 \
libz3-dev \
pkg-config \
tzdata \
zlib1g-dev \
&& rm -r /var/lib/apt/lists/*
# Everything up to here should cache nicely between Swift versions, assuming dev dependencies change little
# gpg --keyid-format LONG -k FAF6989E1BC16FEA
# pub rsa4096/FAF6989E1BC16FEA 2019-11-07 [SC] [expires: 2021-11-06]
# 8A7495662C3CD4AE18D95637FAF6989E1BC16FEA
# uid [ unknown] Swift Automatic Signing Key #3 <swift-infrastructure@swift.org>
ARG SWIFT_SIGNING_KEY=8A7495662C3CD4AE18D95637FAF6989E1BC16FEA
ARG SWIFT_PLATFORM=ubuntu
ARG OS_MAJOR_VER=20
ARG OS_MIN_VER=04
ARG SWIFT_WEBROOT=https://swift.org/builds/swift-5.5-branch
ENV SWIFT_SIGNING_KEY=$SWIFT_SIGNING_KEY \
SWIFT_PLATFORM=$SWIFT_PLATFORM \
OS_MAJOR_VER=$OS_MAJOR_VER \
OS_MIN_VER=$OS_MIN_VER \
OS_VER=$SWIFT_PLATFORM$OS_MAJOR_VER.$OS_MIN_VER \
SWIFT_WEBROOT="$SWIFT_WEBROOT/$SWIFT_PLATFORM$OS_MAJOR_VER$OS_MIN_VER"
RUN echo "${SWIFT_WEBROOT}/latest-build.yml"
RUN set -e; \
# - Grab curl here so we cache better up above
export DEBIAN_FRONTEND=noninteractive \
&& apt-get -q update && apt-get -q install -y curl && rm -rf /var/lib/apt/lists/* \
# - Latest Toolchain info
&& export $(curl -s ${SWIFT_WEBROOT}/latest-build.yml | grep 'download:' | sed 's/:[^:\/\/]/=/g') \
&& export $(curl -s ${SWIFT_WEBROOT}/latest-build.yml | grep 'download_signature:' | sed 's/:[^:\/\/]/=/g') \
&& export DOWNLOAD_DIR=$(echo $download | sed "s/-${OS_VER}.tar.gz//g") \
&& echo $DOWNLOAD_DIR > .swift_tag \
# - Download the GPG keys, Swift toolchain, and toolchain signature, and verify.
&& export GNUPGHOME="$(mktemp -d)" \
&& curl -fsSL ${SWIFT_WEBROOT}/${DOWNLOAD_DIR}/${download} -o latest_toolchain.tar.gz \
${SWIFT_WEBROOT}/${DOWNLOAD_DIR}/${download_signature} -o latest_toolchain.tar.gz.sig \
&& curl -fSsL https://swift.org/keys/all-keys.asc | gpg --import - \
&& gpg --batch --verify latest_toolchain.tar.gz.sig latest_toolchain.tar.gz \
# - Unpack the toolchain, set libs permissions, and clean up.
&& tar -xzf latest_toolchain.tar.gz --directory / --strip-components=1 \
&& chmod -R o+r /usr/lib/swift \
&& rm -rf "$GNUPGHOME" latest_toolchain.tar.gz.sig latest_toolchain.tar.gz \
&& apt-get purge --auto-remove -y curl
# Print Installed Swift Version
RUN swift --version
RUN echo '[ ! -z "$TERM" -a -r /etc/motd ] && cat /etc/motd' \
>> /etc/bash.bashrc; \
echo " ################################################################\n" \
"#\t\t\t\t\t\t\t\t#\n" \
"# Swift Nightly Docker Image\t\t\t\t\t#\n" \
"# Tag: $(cat .swift_tag)\t\t#\n" \
"#\t\t\t\t\t\t\t\t#\n" \
"################################################################\n" > /etc/motd
# [Option] Install zsh
ARG INSTALL_ZSH="false"
# [Option] Upgrade OS packages to their latest versions
ARG UPGRADE_PACKAGES="true"
# Install needed packages and setup non-root user. Use a separate RUN statement to add your own dependencies.
ARG USERNAME=vscode
ARG USER_UID=1000
ARG USER_GID=$USER_UID
COPY .devcontainer/library-scripts/common-debian.sh /tmp/library-scripts/
RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
&& /bin/bash /tmp/library-scripts/common-debian.sh "${INSTALL_ZSH}" "${USERNAME}" "${USER_UID}" "${USER_GID}" "${UPGRADE_PACKAGES}" "true" "true" \
&& apt-get -y install --no-install-recommends lldb python3-minimal libpython3.7 \
&& apt-get autoremove -y && apt-get clean -y && rm -rf /var/lib/apt/lists/* && rm -rf /tmp/library-scripts
# Install SourceKit-LSP
RUN git clone -b "release/5.5" https://github.com/apple/sourcekit-lsp.git \
&& cd sourcekit-lsp \
&& swift build -Xcxx -I/usr/lib/swift -Xcxx -I/usr/lib/swift/Block \
&& cp .build/debug/sourcekit-lsp /usr/bin/
Because the Swift project isn’t yet publishing pre-built Docker containers for Swift 5.5, this Dockerfile is necessarily quite involved. Most of it is taken from Swift’s own Dockerfile for 5.5 with another section taken from Microsoft’s Dev Container Dockerfile for Swift and a final section to compile SouceKit-LSP. Note also the inclusion of library-scripts/common-debian.sh which you can find in Microsoft’s Dev Containers repository and must be included in your .devcontainer folder.
With the dev container definition complete, you can tell VS Code to build and enter your development environment by selecting the command “Remote-Containers: Open Folder in Container”. The first time you run this, it will involve downloading base containers and building your container on top, so it will take a while.
Finally, the Code
With the development environment finally ready, we can turn to actual code. My main goal was to take the regular Vapor template project and tweak it just enough to try out the async and await keywords. I ended up recreating the template manually, but there’s no reason you can’t start with the Vapor toolbox instead. With the toolbox installed, you run the command vapor new {project name}; be sure to answer yes to the question about using Fluent and to select PostgreSQL as the database engine.
To begin, we need to customize the Package.swift file to pull from the relevant async-await branches of the Vapor sources:
Package.swift
// swift-tools-version:5.5
import PackageDescription
let package = Package(
name: "VaporAsyncTry",
dependencies: [
.package(url: "https://github.com/vapor/vapor.git", .branch("async-await")),
.package(url: "https://github.com/vapor/fluent-kit.git", .branch("async-await")),
.package(url: "https://github.com/vapor/fluent.git", .branch("main")),
.package(url: "https://github.com/vapor/fluent-postgres-driver.git", .branch("main"))
],
targets: [
.target(
name: "App",
dependencies: [
.product(name: "FluentKit", package: "fluent-kit"),
.product(name: "Fluent", package: "fluent"),
.product(name: "FluentPostgresDriver", package: "fluent-postgres-driver"),
.product(name: "Vapor", package: "vapor"),
],
swiftSettings: [
.unsafeFlags([
"-Xfrontend", "-enable-experimental-concurrency",
"-Xfrontend", "-disable-availability-checking",
])
]
),
.executableTarget(
name: "Run",
dependencies: [.target(name: "App")]
),
.testTarget(name: "AppTests", dependencies: [
.target(name: "App"),
.product(name: "XCTVapor", package: "vapor"),
])
]
)
Notice also the swiftSettings section with the "-enable-experimental-concurrency" clause. This setting was introduced in Swift 5.4 in order to enable the earliest versions of async await support. I’m hopeful that this setting will not be necessary in the final release of Swift 5.5, but for now it does seem to be needed.
Now, at long last, we can update the example TodoController from Vapor’s default template to take advantage of async and await:
Sources/App/Controllers/TodoController.swift
import Fluent
import Vapor
struct TodoController: RouteCollection {
func boot(routes: RoutesBuilder) throws {
let todos = routes.grouped("todos")
todos.get(use: index)
todos.post(use: create)
todos.group(":todoID") { todo in
todo.put(use: update)
todo.delete(use: delete)
}
}
func index(request: Request) async throws -> [Todo] {
return try await Todo.query(on: request.db).all()
}
func create(request: Request) async throws -> Todo {
let todo = try request.content.decode(Todo.self)
try await todo.save(on: request.db)
return todo
}
func update(request: Request) async throws -> Todo {
guard let todo = try await Todo.find(request.parameters.get("todoID"), on: request.db) else {
throw Abort(.notFound)
}
let updatedTodo = try request.content.decode(Todo.self)
todo.title = updatedTodo.title
try await todo.save(on: request.db)
return todo
}
func delete(request: Request) async throws -> HTTPStatus {
guard let todo = try await Todo.find(request.parameters.get("todoID"), on: request.db) else {
throw Abort(.notFound)
}
try await todo.delete(on: request.db)
return HTTPStatus.ok
}
}
The first thing to notice is that none of the return values are wrapped in EventLoopFuture. Next, observe that there are no nested callbacks, no need for map or flatMap. Function declarations include the async keyword and calls to the database are preceded by await but otherwise this looks just like synchronous code.
One thing I couldn’t get working was Vapor’s unwrap(or: Abort(.notFound)) shortcut for handling database queries that may or may not return a value. Instead, I had to use the somewhat more verbose guard let ... else { throw Abort(.notFound) } construction. Nevertheless, I consider this to be a huge improvement over the nested callbacks previously required.
Regarding sourcekit-lsp
Many — but by no means all — programmers like to have a little boost while programming in the form of intelligent auto-complete, pop-up documentation, etc. Many code editors like VS Code include robust support for these features for many programming languages. Often in the past, each editor had a different API for enabling these features, meaning that a plugin written for TextMate for example wouldn’t work with SublimeText. Microsoft, of all companies, is trying to fix that with their Language Server Protocol, or LSP, specification. LSP allows programming languages to offer universal plugins that work with any text editor supporting LSP.
The Swift programming language has embraced LSP in the form of SourceKit-LSP. In theory, SourceKit-LSP allows editors like VS Code to have all of the same auto-complete features for Swift that Apple’s own Xcode has. In practice, however, SourceKit-LSP has in the past proven to be so slow, buggy, and incomplete as to be nearly useless.
As you can see above, as part of this experiment, I wanted to try out SourceKit-LSP again to see if any progress has been made. I, for one, would very much like to have these features for Swift programming outside of Xcode (Xcode continues to use a different, proprietary engine to drive these features). Alas, I am sorry to say that little progress has been made. SourceKit-LSP remains so slow, buggy, and incomplete as to be nearly useless.