iAchieved.it

Software Development Tips and Tricks

By

Trapping Signals with Swift on Linux

Update: Folks have figured out how to use sigaction which is the preferred way of catching a signal. Jump down to read all about it, both signal and sigaction methods work!

Unix applications frequently have a need to catch, or trap, signals. For example, many daemons will reread their configuration file upon receipt of the SIGHUP signal. A client interface may catch CTRL-C and print Are you sure you want to quit? [yN].

This can be accomplished in Swift on Linux as well by using the signal function from Glibc. The code, which is a play on Adam Sharp’s gist:

Run with swift or compile and execute using swiftc. The application will wait at the select statement. Pressing CTRL-C will generate a SIGINT and the trap routine will execute (and end the application).

To be clear, the application did not exit because the trap closure was executed. The exit occurred because flow of control transferred from the closure to after the select. Think of the line after the select as an implicit exit(0). If you rewrite this as

then the application will not exit after the signal handler executes.

Note: The Signal enumeration is but a small subset of the various signals. Check out man 7 signal for a complete list.

Pretty Printing

There’s no need to add another integer-to-string translation table to get the type of signal received. strsignal is a handy function to get a human-friendly string name for the signal:

Trailing Closures

Trailing closures are one of those syntactical goodies that make Swift a joy to program in. Your first exposure to them is probably implementing a callback function to an HTTP GET request:

The code between the braces is actually the last parameter to the HTTPRequest class initializer. The parameter signature is likely to be (HTTPResponse, NSError) -> Void. Nothing requires you to code this in the form of the trailing closure, but it’s frequently nice to. Just remember that a trailing closure can only be used when the function to be called is the last parameter in the function signature. Those are the rules.

We of course use this feature in our routine:

The trailing closure syntax is used to specify the signal handler which must adhere to the __signalhanlder_t function signature. This signature is simply (Int32) -> Void, which is what our trailing closure is.

What about sigaction?

Truth be told, we ought to be using sigaction rather than signal. The man page says so, and everyone knows better than to argue with a man page.

The behavior of signal() varies across UNIX versions, and has also varied historically across different versions of Linux.   Avoid  its  use:  use sigaction(2) instead.

Unfortunately I haven’t figured out how to use sigaction with Swift on Linux. There are two sigaction definitions in Glibc. One is for the sigaction structure (struct sigaction) and the other is for the sigaction function. Swift can’t seem to distinguish between the two, and only the function is surfaced. Without the structure definition its difficult to call sigaction. Perhaps it is possible to write our own struct sigaction definition and somehow trick Swift into thinking it can be passed into the sigaction function. I haven’t tried, but if someone out there figures it out, leave a comment!

Hold the phone! An update on sigaction from Robert (see comments) allows us to use sigaction! We’ve taken his code (thanks Robert) and updated it a bit:

Try it out!

One Last Example

I mentioned above that the trailing closure didn’t have to be used to provide the signal handler routine. In this code we declare a single function sighandler with a (Int32) -> Void signature and then use it to trap SIGINT, SIGHUP, and SIGUSR1.

Handling Signals

Handling Signals

By

TCP Sockets with Swift on Linux

A long time ago in a galaxy far, far away, software developers wrote client-server applications with TCP/IP sockets. That was before the dark times, before HTTP.

I am of course, joking. HTTP can be leveraged to provide a wide variety of client-server applications and is of course at the base of REST applications. What HTTP brings to the table though is not the plumbing to get packets on the wire, but an agreed upon protocol structure (and to some degree a standard as to what port is being used) for those packets. Action verbs such as GET, POST, PUT, etc. and the HTTP header itself are what makes HTTP ideal for developing client-server applications.

In the end though, at the bottom of the stack, bits and bytes are marshaled through your operating system’s socket interface. The API for interacting with network sockets is quite rich and many a tutorial and books have been written on the topic. IP networking routines in C can be considerably verbose, and were one of the first “real-world” APIs encapsulated in object-oriented routines with C++. That tradition continued with Foundation’s CFStream class and now our Swift swiftysockets API.

Swiftychat

To illustrate how to use TCP/IP network sockets with Swift, we’ll be developing Swiftychat, a basic “chat system” application. It’s quite a naive application, limited in functionality, and would stand no chance in being used in the real world, but, even still, it’s a working example of how to send and receive strings on TCP/IP sockets in Swift.

swiftysockets

Swiftychat will make use of swiftysockets, a Swift Package Manager-ready TCP/IP socket implementation that was originally developed by the Zewo team. Unfortunately due to packaging constraints we have to do a little bit of a dance first to get an underlying C library, Tide, installed on our system. So let’s do that now.

$ git clone https://github.com/iachievedit/Tide
Cloning into 'Tide'...
...
$ cd Tide
$ sudo make install
clang -c Tide/tcp.c Tide/ip.c Tide/utils.c
ar -rcs libtide.a *.o
rm *.o
mkdir -p tide/usr/local/lib
mkdir -p tide/usr/local/include/tide
cp Tide/tcp.h Tide/ip.h Tide/utils.h Tide/tide_swift.h tide/usr/local/include/tide
# copy .a
cp libtide.a tide/usr/local/lib/
mkdir -p /usr/local
cp -r tide/usr/local/* /usr/local/

At some point we believe the Swift Package Manager will be able to compile C libraries that can be linked against in the rest of your package build. Until then, this is the best we can do.

Once Tide is installed we can leverage swiftysockets in our Swiftychat apps.

Start Coding!

Our main.swift file is as simple as it gets. Create a ChatterServer and start it.

main.swift

Of course, a brief main.swift can only mean one thing. Invasion. Oh wait, I’m done with the Star Wars references.

A brief main.swift means our implementation is tucked away in the ChatterServer class, which looks like this:

ChatterServer.swift:

Breaking down our server we have:

1. Initialization

We make use of the init? initializer to signal that nil is a possible return value since both the IP and TCPServerSocket classes (from swiftysockets) can throw an error. IP encapsulates our IP address and port information nicely and we provide an instance of it to the TCPServerSocket initializer. If init succeeds we now have TCP socket on the given port ready to accept incoming connections.

2. The Main Loop

Name the function startListening, start, main, we don’t care. It is the main event loop that accepts new client connections (server!.accept()) and adds them to the list of connected clients. server!.accept() is a blocking function that hangs out and waits for new connections. Pretty standard stuff.

3. Client Management

The rest of the ChatterServer contains all of the “client management” functions. There are a few variables and three routines that manage clients.

Our variables are straightforward:

  • an array of connected clients ([TCPClientSocket])
  • a connection counter that is used to hand out a “client identifier”

The routines are straightforward as well:

  • addClient takes a TCPClientSocket, increments our connection count, and then sets up an NSThread whose sole purpose is “manage” that given client’s connection. As additional connections come in new NSThreads are created for them as well. We’ll talk about the NSThread routine itself in a moment. Once the thread is started addClient will then append the TCPClientSocket onto the end of the array of connected clients.
  • removeClient removes a client from the connected clients list using the filter function to “filter out” the given client. Note here we use the !== identity operator.
  • broadcastMessage is what makes the ChatterServer a chat server. It uses the where keyword to create a filtered array that broadcasts a message created by a client to all of the other connected clients. Again, we use the !== operator.

Recall that a thread is a separate execution path that runs inside the main process. Our server creates a separate thread for each client that is connected. Now, you can argue whether or not that is a good idea, and if we’re designing a server that will eventually handle tens of thousands of clients, I’d argue that it isn’t. For our purposes though we’ll be fine.

Looking once more at our thread:

Our client handling thread also sits in a loop waiting for input through the receiveString method of the TCPClientSocket class. When a string is received the server logs it to the console and then broadcasts a response. If the try results in an error (a disconnect) the server removes the client.

Putting it All Together

Our goal is to, as much as possible, use the Swift Package Manager for building our applications. For an introduction to swiftpm check out our tutorial.

Add the following to Package.swift:

and in a directory named Sources add your main.swift and ChatterServer.swift code.

Running swift build should download and build our two dependencies (Tide and swiftysockets) and compile our application code. If all goes well you’ll have a binary named chatterserver in a directory named .build/debug/.

Testing it Out

Our next tutorial will be writing up a nifty little chat client, but for now we can test our server with the nc (netcat) command. Start your server and then in another terminal window type nc localhost 5555. You should see in your server’s window Client 1 connected. If you hit CTRL-C in your netcat “client” window the server will print a disconnect message along with the reason (like, Connection reset by peer).

For the real test we’ll start up the server and three connected clients.

Chatville

Chatville

In the left-hand terminal our chat server is running. On the right we have 3 clients, each started by typing the command nc localhost 5555. As each client connects the server prints out a connection message.

Recall that our broadcastMessage function excludes the originator of the message from the broadcast. This prevents a client from receiving his own message back (take out the where clause and you will see what we mean).

What’s Next

Using nc as our client is a bit boring. We can’t set a nickname, there’s no “structure” to our messages, no timestamps, etc. In the above example someone receiving a message has no idea who was writing! swiftysockets already has a TCPClientSocket class, why not create a more robust chat client?

Getting the Code

Of course we’ve put together the code for our little chat server here on Github. It also contains a chatterclient project which at the moment is not implemented. If you start with the download, you can type make in the top-level directory and it will build both the client and server. Remember: You must have installed libtide.a and its associated headers before using swiftysockets!

By

Command Line Utilities in Swift

This is another post in a series designed to explore the range of applications that Swift can be applied to on Linux server systems.

Our application will build upon a previous example that uses popen combined with the wget command to call a natural language translation service and translate strings from one language to another like Google Translate. Rather than having a one-shot command line that translates a single string, we’re going to create an interactive “shell” that translates each string entered at a prompt. Here’s a screenshot of the application in action:

Translate all the things!

Translate all the things!

The translator prompt indicates what language it is expecting to receive, as well as what language it will attempt to translate to. For example:

  • en->es Translate from English to Spanish
  • es->it Translate from Spanish to Italian
  • it->ru Translate from Italian to Russian

The program defaults to en->es and provides the commands to and from to change the languages. For example, typing to es will change the language to translate to to Spanish. Typing quit will exit the application.

If the string entered is not a command it is taken verbatim and passed to the translation web service. The result of the translation is then echoed back.

What to Note

If you’re coming from a systems or devops programming background and have never been exposed to Swift before, here are the things you should be taking a look at in the code. I think you will find Swift offers a lot for both types of programmers and will become a welcome addition to the Linux programming toolbox.

  • let variableName = value for assigning constants
  • tuples
  • strings in switch-case statements
  • switch-case statements must be exhaustive
  • computed properties
  • import Glibc which provides access to standard C functions
  • the guard statement
  • Foundation classes such as NSThread and NSNotificationCenter
  • posting notifications to trigger code execution in a different object or thread

Application Design

Our translator is broken into a main routine, two classes, and a globals.swift file. If you are going to follow along you should use the Swift Package Manager and lay out your files in a directory like this:

translator/Sources/main.swift
          /Sources/CommandInterpreter.swift
          /Sources/...
          /Package.swift

The main.swift file is a Swift application’s entry point and the only file that should have executable statements in it (assigning a variable or declaring a class is not an “executable statement” in this context).

main.swift:

As it stands our application has no arguments to process off of the command line, so it:

1. Creates an instance of both the CommandInterpreter and Translator class

2. Adds an observer for a notification named InputNotification (we use a constant INPUT_NOTIFICATION defined in globals.swift)

3. Provides a block of code to execute when the notification is received

4. Starts the interpreter

5. Calls select which blocks the main thread while other threads in the application continue to run

CommandInterpreter

The class CommandInterpreter is responsible for reading characters from standard input, parsing the input string to determine what action should be taken, and then taking that action. If you’re new to Swift we’ve commented some of the basic constructs of the language.

CommandInterpreter.swift:

The CommandInterpreter logic is straightforward. When the start function is called an NSThread is created with a block of code that calls fgetc collecting characters on stdin. When the newline character is encountered (the user hits RETURN) the input is parsed and structured into a Command. This in turn is handed over to the doCommand function to be processed.

Our doCommand function is a simple switch-case statement. If a .Quit command is encountered the application simply calls exit(0). The .SetFrom and .SetTo commands are self-explanatory as well. When it comes to the .Translate command, here is where the Foundation notification system is used. doCommand itself doesn’t translate anything but instead posts an application-wide notification named InputNotification. Anyone listening for this notification (our main thread!) will have its associated callback invoked:

As I mentioned in this post, there is a SILgen crash working with NSNotification userInfo dictionaries, so we are cheating with a global variable named translationCommand. In our block we:

1. Decrease our verbosity a little bit by assigning tc to the contents of translationCommand

2. Call the translate function of our Translator object, passing in the required arguments

3. Provide a block to run with the translation is complete, which

4. Uses the nice guard statement of Swift to bail quickly if there was an error

5. Print our translation!

Translator

The Translator was originally introduced in this post, and we have reused it here.

Translator.swift:

Putting it All Together

To pull everything together we have two more files to create: globals.swift and Package.swift:

globals.swift:

Package.swift:

If everything is in its proper place, a swift build should give you a functioning translator application.

swift build
Cloning https://github.com/iachievedit/CJSONC
Using version 1.0.0 of package CJSONC
Cloning https://github.com/PureSwift/CcURL
Using version 1.0.0 of package CcURL
Compiling Swift Module 'translator' (4 sources)
Linking Executable:  .build/debug/translator

If you want to start off with the code already there, grab it from Github and check out the cmdline_translator directory.

Things to Try!

There are a lot of improvements that can be made to the application. Here’s a list of things you might like to try:

  • add command line arguments to set the default from and to language
  • add command line arguments to run in a non-interactive mode
  • add a swap command which swaps the from and to language
  • add a help command
  • collapse the from and to commands into line, like from en to es
  • fix the bug where typing from or to with no arguments results in a crash
  • require the commands to be proceeded with a character like \ (otherwise you can’t translate “quit”)
  • add localizedDescription strings to our Translator errors, or
  • implement throws on the Translator for when errors occur

Closing Thoughts

I am unabashedly a Swift enthusiast and believe it has a great opportunity to stand along side Perl, Python, and Ruby for “devops tasks” as well as compete with C, C++, and Java for “systems programming.” At the moment it is true that for other than the most trivial single file scripts, you must compile your Swift code into a binary. I am hopeful that situation will change and I can move on from languages that start new blocks with whitespace. Folks are already talking about it on the Swift evolution mailing list in this thread.

By

Foundation on Linux

It is an exciting time to be a Swift enthusiast. The language is now open source and has been ported to Linux. And if that wasn’t enough, folks at Apple and elsewhere across the globe are also working to port over Foundation and Grand Central Dispatch.

If you’re coming to Swift from a mostly Linux background, you might not be aware of either Foundation or GCD, or why they are important. Simply put, Foundation is a
collection of utility classes designed to provide implementations for frequently used routines and tasks.

We’re not going to look at all of the functionality Foundation provides but will instead look at two classes: NSThread and NSNotification. Moreover, the idea behind this post is less about an exhaustive review of the functionality provided by the classes and more about beginning to get our new Swift on Linux users familiarity with Foundation.

NSThread

NSThread is Foundation’s portable thread class. Built upon POSIX threads, NSThread provides you with the ability to create a thread, start it, check its status, etc.

An NSThread is easy to create, and notice that it takes a block as its init argument.

As it stands you can compile the above code, but it won’t do anything because we didn’t start() our thread. Add thread.start() at the end and run it. Here’s what we get:

swift threadexample.swift
Entering thread
10...

That won’t do! Our application exits before the thread is complete. One way to solve this is to raise a “done” flag when the thread completes. We then go to sleep on the main process until the flag is set.

Note that the sleep(1) in our main thread is simply yielding up the CPU so we aren’t creating a busy-wait, but it does have the annoying side effect of not ending execution until after the sleep returns.

Even better is removing the done flag and putting a “main event loop” in, like this:

In this case our thread is now responsible for ending the application. Our select statement is a fake event loop; it never returns but the application continues to run other threads driven by incoming events.

NSNotification

Anyone that has worked with Cocoa and OS X or iOS development has worked with NSNotifications and NSNotificationCenter. In many ways a NSNotificationCenter is like a dbus for your application: objects may post notifications (that is, events), and objects that are interested can subscribe to receive them.

Let’s look at an example; but first, a brief disclaimer. NSNotifications that are posted can be created with a userInfo dictionary that contains additional information about the event. Unfortunately there is a bug in the Swift SILgen on Linux that is preventing the proper casts to extract the information, so we’re going to employ a global variable as a workaround. Normally we wouldn’t do that.

Here’s our code:

It’s pretty simple.

1. Our nc constant is the default NSNotificationCenter for the application. You don’t have to create the default NSNotificationCenter.

2. readThread is an NSThread instance that never exits but sits and reads stdin. Whenever the newline character is encountered it saves the “buffered” characters (stored in input) to availableData and then posts an INPUT_NOTIFICATION via the default NSNotificationCenter.

3. Before starting our readThread we added an observer for the INPUT_NOTIFICATION. The addObserverForName function takes a block which executes when the notification is broadcasted.

4. We start our thread with the NSThread start method.

5. Again, we employ a blocking select to ensure the Linux process doesn’t end (which would unceremoniously kill our thread).

NSNotification on OS X

As a reference of how we should be using userInfo with NSNotification, here is some Swift code for OS X that eventually we’d like to see working on Linux:

Closing Comments

Swift on Linux is very new; as of this writing, less than a month old. Yet with each passing day additional functionality is being added to the Foundation classes, folks are working to port Swift to ARM platforms such as Raspberry Pi and BeagleBone, and the language is evolving towards Swift 3. Our personal belief is that Swift has a great shot at being competitive with NodeJS, Python, Ruby, and Java and C++ for application development on Linux platforms. Hopefully this post and those to come will help application developers explore the possibilities Swift brings.

By

Swift for the Raspberry Pi

Editor’s Note: Swift 3.0 for Raspberry Pi 2 and Pi 3 is now available! Head on over to our Swift 3.0 for Raspberry Pi post for instructions on how to download and install it.

The following is similar instructions to that provided for other ARM devices like BeagleBoneBlack and BeagleBoard X15. There are some slight differences to get clang-3.6 properly installed.

If you’ve already installed the swift-2.2 package with the instructions below and just want to upgrade to the latest version, use sudo apt-get update && sudo apt-get upgrade.

Disclaimer! Swift on ARM systems is very much at an alpha stage at the moment! The package below does not contain a working

12/27/15 Update: The REPL is now somewhat functional on BeagleBone and BeagleBoard, but there is a memory corruption error on the Raspberry Pi 2.
1/4/16 Update: Foundation is now working! Try some of the examples from this post.

You can track the progress of the port of Swift to ARM platforms at bugs.swift.org, and follow @iachievedit on Twitter for updates when new packages are released. To help out, create an account at bugs.swift.org and when you find broken stuff, file bugs and tag them with ‘arm’.

Prerequisites

Both the clang-3.6 and libicu-dev packages need to be installed. In addition, you should use update-alternatives to provide /usr/bin links for clang and clang++. To get clang-3.6 on a Raspberry Pi 2 you can use repositories provided by Robert C. Nelson.

Run the following commands to add the rcn-ee repository:

wget http://repos.rcn-ee.com/debian/pool/main/r/rcn-ee-archive-keyring/rcn-ee-archive-keyring_2015.10.22~bpo90+20151022+1_all.deb
sudo dpkg -i rcn-ee-archive-keyring_2015.10.22~bpo90+20151022+1_all.deb
echo "deb [arch=armhf] http://repos.rcn-ee.com/debian/ jessie main" | sudo tee --append /etc/apt/sources.list
sudo apt-get update

Then, install the dependencies libicu-dev and clang-3.6:

sudo apt-get install libicu-dev
sudo apt-get install clang-3.6
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.6 100
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.6 100

Note: You must use update-alternatives to link /usr/bin/clang and clang++.

Installation

At the moment there are only Raspberry Pi 2 packages for Debian Jessie. I have tested the basic installation on a Raspberry Pi 2.

1. Add the repository key

wget -qO- http://dev.iachieved.it/iachievedit.gpg.key | sudo apt-key add -

2. Add the repository to sources.list

echo "deb [arch=armhf] http://iachievedit-repos.s3.amazonaws.com/ jessie main" | sudo tee --append /etc/apt/sources.list

3. Run apt-get update

sudo apt-get update

4. Install swift-2.2!

sudo apt-get install swift-2.2

Trying it Out

Again, a warning! There is no Swift package manager in this build yet and don’t expect to be able to use swift to run it in REPL mode. You can however compile basic applications, even those using Glibc and Foundation.

root@beaglebone:~# more helloWorld.swift 
print("Hello, world!")

for i in 1...10 {
  print(i)
}
root@beaglebone:~# swiftc helloWorld.swift 
root@beaglebone:~# ./helloWorld 
Hello, world!
1
2
3
4
5
6
7
8
9
10

or, try out swiftcat.swift which uses Glibc.

With the latest 0ubuntu11 release, Foundation is supported too!
countdown.swift

By

Building REST APIs with Zewo

2/20/2016 Update: Zewo has undergone a significant refactoring since this tutorial was originally written. The information below is no longer accurate and I will be working to provide a new tutorial as soon as possible.

Zewo

Zewo is new collection of modules designed to build web applications in Swift. Zewo provides:

  • an HTTP server (Epoch)
  • an HTTP router
  • HTTP request and response entity classes
  • an interface to a Mustache template responder (Sideburns)
  • PostgreSQL and MySQL database adapters

and much more. Although relatively new to the scene, Zewo already has enough functionality to build a complete REST-based web service application. In the past I would have first considered Ruby and Sinatra, but now with the arrival of Swift on Linux, Zewo provides another great option.

A Quick API Design

We’re going to build a REST API similar to that of AT&T’s M2X platform. M2X is self-described as providing time-series data storage, device management, message brokering, event triggering, alarming, and data visualization for your industrial Internet of Things (IOT) products and services. That’s a mouthful. Our API will be less ambitious and implement the time-series data storage piece, but that is enough to showcase some of the capabilities of Zewo.

In the IOT world we can think about devices and streams of data that are associated with the device. For example, a device could be a single-board computer like a BeagleBone or Raspberry Pi with several sensors attached to it. The sensor, in turn, would be providing a stream of datapoints, say temperature readings. These concepts can be modeled nicely in a relational database fronted by a REST API. And, if relational databases aren’t your thing, folks are working on NoSQL interfaces such as MongoDB.

Devices, Streams, and Datapoints

Our model will consist of Device, Stream, and Datapoint.

A Device has the following attributes, or properties:

  • name
  • serial
  • location
  • streams

A Stream has the following properties:

  • name
  • datapoints

And finally, a Datapoint (sometimes referred to as a reading) will be:

  • timestamp
  • value

Now, believe me, this is the not the end-all be-all data model for an IoT application, but it provides enough structure to make our REST API interesting.

The operations we will support are:

  • creating a new device
  • posting a value to a stream
  • retrieving the values of a stream

Of course, this is limited functionality, so it is left up to the reader to add things like:

  • updating the location of the device
  • adding a new stream to a device

So, let’s dive in!

First Things First

The example code provided at the end of the tutorial has only been tested on Linux, and everything done here is on an Ubuntu 14.04 system. To get up and running you will need to:

There are detailed instructions on these steps further below.

Once the prerequisites are installed we’re going to use the Swift Package Manager to bootstrap in our Zewo dependencies. In a directory named iotapp create your Package.swift file:

and then run swift build. You will see the Swift Package Manager download and compile all of the required packages:

Cloning Packages/Epoch
Using version 0.1.0 of package Epoch
Cloning Packages/HTTPParser
Using version 0.1.0 of package HTTPParser
Cloning Packages/CHTTPParser
Using version 0.1.0 of package CHTTPParser
Cloning Packages/HTTP
Using version 0.1.2 of package HTTP
Cloning Packages/Core
Using version 0.1.1 of package Core
Cloning Packages/CURIParser
Using version 0.1.0 of package CURIParser
Cloning Packages/Venice
Using version 0.1.0 of package Venice
Cloning Packages/CLibvenice
Using version 0.1.0 of package CLibvenice
Cloning Packages/Middleware
Using version 0.1.0 of package Middleware
Cloning Packages/Router
Using version 0.1.0 of package Router
Cloning Packages/PostgreSQL
Using version 0.1.0 of package PostgreSQL
Cloning Packages/SQL
Using version 0.1.0 of package SQL
Cloning Packages/CLibpq
Using version 0.1.0 of package CLibpq
Compiling Swift Module 'Core' (15 sources)
Linking Library:  .build/debug/Core.a
Compiling Swift Module 'HTTP' (10 sources)
Linking Library:  .build/debug/HTTP.a
Compiling Swift Module 'HTTPParser' (4 sources)
Linking Library:  .build/debug/HTTPParser.a
Compiling Swift Module 'Venice' (21 sources)
Linking Library:  .build/debug/Venice.a
Compiling Swift Module 'Epoch' (9 sources)
Linking Library:  .build/debug/Epoch.a
Compiling Swift Module 'Router' (1 sources)
Linking Library:  .build/debug/Router.a
Compiling Swift Module 'Middleware' (11 sources)
Linking Library:  .build/debug/Middleware.a
Compiling Swift Module 'SQL' (6 sources)
Linking Library:  .build/debug/SQL.a
Compiling Swift Module 'PostgreSQL' (5 sources)
Linking Library:  .build/debug/PostgreSQL.a

Note: If the above bombed out it is likely due to missing dependencies. You don’t need to perform the bootstrap steps just yet, but if you’re chomping at the bit run the following:

sudo apt-get install libpq-dev
echo "deb [trusted=yes] http://apt.zewo.io/deb ./" | sudo tee --append /etc/apt/sources.list
sudo apt-get update
sudo apt-get install uri-parser http-parser libvenice 

Anatomy of a Zewo REST Application

A Zewo REST application looks similar to other HTTP application stacks such as Rails, Sinatra, etc. At the top of the stack is the familiar HTTP Server. Zewo provides us with Epoch, described as a Venice based HTTP server for Swift 2.2 on Linux. A straightforward main.swift initializes our server:

Next down the stack is our routing functionality, which is an instance of Zewo Router. The router is initialized in Router.swift which looks like this:

The >>> admittedly looks like a little black magic. It is an overloaded operator defined in Zewo Middleware. In this example it provides for logging our API actions.

The route.router("/api", APIv1 >>> log) statement will anchor our web service at /api. Let’s look now at APIv1.swift:

APIv1 is where the real action is happening. Our URL anchor point is now extended to /api/v1 and we’ve specified a number of specific routes from there:

  • GET /api/v1/version
  • POST /api/v1/devices
  • GET /api/v1/devices/:serial
  • POST /api/v1/devices/:serial/streams/:name/value
  • GET /api/v1/devices/:serial/streams/:name

Again, while not the richest API, it serves as starting point.

If you’ve ever worked with Sinatra you will recognize the named parameters in the route patterns. These parameters, such as :serial and :name will be provided to us in our controllers, which we’ll see next.

Controllers

Zewo does all the heavy lifting of receiving an HTTP request and then routing the request to the appropriate controller. It is here though where we have to pick up the request and do something with it.

What is a controller again exactly? According to the Rails ActionController documentation a controller “is responsible for making sense of the request and producing the appropriate output”. This is what our DeviceController does for us:

Recall that it was the APIv1 router that calls deviceController.create when a POST /devices is received. The create function takes a Request object as its only argument, and then returns a Response. In other words, the function is making sense of the request and producing the appropriate output. Making sense of the request is broken down into distinct steps:

  • obtaining the JSON body of the request and extracting required fields name and serial
  • obtaining the location field if it is available, otherwise the location defaults to (0.0, 0.0)
  • extracting the required streams field from the JSON
  • inserting a new Device into the database
  • inserting a new Stream for each stream specified

The response to this request can be one of two types:

  • .BadRequest (HTTP Status 400)
  • .OK (HTTP Status 200)

In both cases we return a JSON body with additional information.

An example of a valid JSON request body for POST /devices looks like this:

If the request was successful we expect a response to look like:

The server returned an additional field id in the JSON response. Some APIs are designed such that the client use the id field for future references to the created object. Our API, on the other hand, will use the serial number to identify a given device.

Records

Observe that the controller does not interact directly with the database. That is the job of our record class, which is modeled after the concept of ActiveRecord in Rails. The DeviceRecord class knows about Postgres, what table is used to manage devices (in fact, it creates that table), and how to insert and find devices. In Rails ActiveRecord is smart enough to read the table schema and create methods such as find on the fly. Our DeviceRecord does not go that far; we aren’t building Rails here.

This is the content of our DeviceRecord.swift file:

At a fundamental level there really isn’t a whole lot going on in this file, with of course the exception that we are issuing Postgres SQL commands to the underlying database adapter and reading the result.

Before leaving the DeviceRecord, notice that insert returns Device. This is not a good idea, and we should change the implementation to return Device? in the event there is an issue with the SQL INSERT. The example code in Github will return Device? so we can catch these errors.

Combining all the components together, top-to-bottom, we get a view of what the “stack” looks like:

Basic Web Service Stack

Basic Web Service Stack

A Working Example

Now that we have a basic idea of how Zewo is used to build a REST web service, let’s run a working example.

First, install of the dependencies you’ll need, which includes libpq-dev for the Postgres headers, as well as some additional packages Zewo requires.

sudo apt-get install libpq-dev
echo "deb [trusted=yes] http://apt.zewo.io/deb ./" | sudo tee --append /etc/apt/sources.list
sudo apt-get update
sudo apt-get install uri-parser http-parser libvenice 

Now, get the code from Github and switch to the basic branch.

git clone https://github.com/iachievedit/iotapp
cd iotapp
git checkout basic

Build the application with swift build and run it:

swift build
.build/debug/iotapp

If Postgres isn’t running or cannot otherwise be contacted, you might see this error:

fatal error: 'try!' expression unexpectedly raised an error: PostgreSQL.Connection.Error.ConnectFailed("could not connect to server: Connection refused\n\tIs the server running on host \"localhost\" (::1) and accepting\n\tTCP/IP connections on port 5432?\ncould not connect to server: Connection refused\n\tIs the server running on host \"localhost\" (127.0.0.1) and accepting\n\tTCP/IP connections on port 5432?\n"): file swift/stdlib/public/core/ErrorType.swift, line 53
Illegal instruction (core dumped)

In this case, install Postgres and create your user and database:

$ sudo apt-get install postgresql
$ sudo su - postgres
$ psql template1
psql (9.4.5)
Type "help" for help.

template1=# CREATE USER iotuser WITH PASSWORD 'iotuser';
CREATE ROLE
template1=# CREATE DATABASE iot_staging;
CREATE DATABASE
template1=# GRANT ALL PRIVILEGES ON DATABASE "iot_staging" to iotuser;
GRANT
template1=# \q
$ exit

Once the application is up and running, we can use cURL or Postman to create a new device. Here we’ll use cURL:

$ curl -X POST -H "Content-Type: application/json" -d '{"name":"BeagleBone", "serial":"N9TT-8GXF-B7GF-RXNC", "location":{"latitude":-34.8836, "longitude":-56.1819}, "streams":[{"name":"cpu-occupancy"}, {"name":"free-memory"}]}' 'http://localhost:8080/api/v1/devices'

Our Zewo application processes the request, logging to STDOUT:

12/23/15, 8:11 PM /home/iotapp/iotapp/Sources/DeviceRecord.swift:23 insert(_:serial:location:): INSERT into devices (name,serial,location) VALUES('BeagleBone','N9TT-8GXF-B7GF-RXNC',POINT(-34.8836,-56.1819)) RETURNING id
POST /api/v1/devices HTTP/1.1
Content-Type: application/json
Accept: */*
User-Agent: curl/7.43.0
Host: localhost:8080
Content-Length: 200

{"name":"BeagleBone",
 "serial":"N9TT-8GXF-B7GF-RXNC",
 "location":{
  "latitude":-34.8836,
  "longitude":-56.1819
  },
  "streams":[
      {"name":"cpu-occupancy"},
      {"name":"free-memory"}
  ]
}
-
HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
Content-Length: 128

{"id":"1","name":"BeagleBone","location":{"longitude":-56.1819,"latitude":-34.8836},"streams":[],"serial":"N9TT-8GXF-B7GF-RXNC"}
----------------------------------------

The response to our command is what we expect, a JSON string with our device!

{"id":"1","name":"BeagleBone","location":{"longitude":-56.1819,"latitude":-34.8836},"streams":[],"serial":"N9TT-8GXF-B7GF-RXNC"}

Try issuing the command again with the same serial number. You should see {"message":"error creating device"} thanks to the logic we added to return a .BadRequest if we couldn’t insert the device.

Creating Streams and Posting Datapoints

We now need to add the functionality that creates streams and datapoints. Unlike with Device we do not need a StreamController. Streams are created directly through a StreamRecord instance in the DeviceController.

Our StreamRecord code looks like this:

Note that our streams schema definition contains device_id, which is a reference to the device that “owns” the stream. Again, unlike with Rails, we are not specifying has_a or belongs_to relationships explicitly; we manage this through code.

Posting Datapoints

A stream is a collection of data points, or values. For example, let’s say our BeagleBone device had a temperature sensor attached to it. We would want to POST a temperature reading to the temperature stream, like this:

curl -X POST -H "Content-Type: application/json" -d '{"value":"77"}' 'http://localhost:8080/api/v1/devices/NFTT-8GXF-B7GF-RXNC/streams/temperature/value'

Our API responds with:

{
  "value": "77",
  "inserted_at": "2015-12-23 21:13:42.763442",
  "id": "1"
}

Some time later the temperature increases to 78 degrees, so another value is posted. Later on, it increases to 80 degrees, and so on. Then, we can retrieve our datapoints via a GET request:

curl -X GET -H "Content-Type: application/json" 'http://localhost:8080/api/v1/devices/NFTT-8GXF-B7GF-RXNC/streams/temperature'

The following data is returned:

{
  "datapoints": [
    {
      "value": "77",
      "inserted_at": "2015-12-23 21:13:42.763442",
      "id": "1"
    },
    {
      "value": "78",
      "inserted_at": "2015-12-23 21:15:30.536468",
      "id": "2"
    },
    {
      "value": "80",
      "inserted_at": "2015-12-23 21:15:54.651966",
      "id": "3"
    }
  ]
}

Here is our implementation of DatapointController. Note how the methods DeviceRecord.findBySerial(serial:) and StreamRecord.findByName(name:,deviceId:) are used to associate the datapoint with the correct device and stream.

Our DatapointRecord implementation looks like:

An exercise for the reader: The DatapointRecord.findByStream(streamId:count:) function is called with a hardcoded value of 10. Extend the API to allow for the number of datapoints to be specified either by a JSON structure in the POST body, or through a URI query parameter. Consider also adding inserted_before and inserted_after filters to control which values to return.

Complete Application

The completed IoT application is on Github on the master branch. With it you should be able to create devices (make sure and specify the streams up front or add an implementation of modifying a device to add a new stream!), and post datapoints to the device streams.

There are some additional files and helper routines in the repository:

  • the +JSON files provide toJSON extension functions to our objects
  • Utils.swift adds a logmsg routine

As I mentioned, there are a lot of capabilities that can be added to this API. Streams can be given datatypes and units, operations which find min and max datapoints in a stream could be added, and so on. For inspiration check out the M2X API.

What’s Next?

After spending time with the Zewo framework over the past week I’m convinced it will become one of the go to building blocks for developing web applications in Swift. This article has focused on building the server side of the REST API, and I’m looking forward to realizing the dream of developing in one language when we look at developing a client side set of classes in Swift that can be used to interact with the service. Stay tuned.

By

Debian Packages for Swift on ARM

June 18, 2016 Update! If you’re looking for Swift 3.0 on Raspberry Pi 2 or 3, come see this post.

Disclaimer! Swift on ARM systems is very much at an alpha stage at the moment! The package below does not contain a working

12/27/15 Update: The REPL is now somewhat functional on BeagleBone and BeagleBoard. Occasionally you may see Execution interrupted. Enter code to recover and continue.
1/4/16 Update: Foundation is now working!

You can track the progress of the port of Swift to ARM platforms at bugs.swift.org, and follow @iachievedit on Twitter for updates when new packages are released. To help out, create an account at bugs.swift.org and when you find broken stuff, file bugs and tag them with ‘arm’. This tagging process helps those working on ARM support track all of the open issues in JIRA.

Background

A number of folks have been working hard to get a port of Open Source Swift working on ARM-based platforms such as the RaspberryPi, BeagleBone, etc. A port request, SR-40 was opened in JIRA and from there several forks of the main Swift repository were created to start the work. William Dillon succeeded in getting a basic port to work, which was no small feat. This work has been a collaboration among a number of users on Github including hpux735, tienex, and others.

Prerequisites

The clang-3.6 and libicu-dev packages need to be installed. In addition, you should use update-alternatives to provide /usr/bin links for clang and clang++.

sudo apt-get install libicu-dev
sudo apt-get install clang-3.6
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.6 100
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.6 100

Editor’s Note: Jump down to the bottom if for whatever reason you absolutely cannot find clang-3.6 for your system.

Installation

At the moment there are packages for Ubuntu 14.04 (Trusty) and Debian 8.2 (Jessie). These have been tested on the BeagleBoard X15, BeagleBone Black, and Wandboard.

1. Add the repository key

wget -qO- http://dev.iachieved.it/iachievedit.gpg.key | sudo apt-key add -

2. Add the appropriate repository to sources.list

Ubuntu 14.04 (Trusty)

echo "deb [arch=armhf] http://iachievedit-repos.s3.amazonaws.com/ trusty main" | sudo tee --append /etc/apt/sources.list

Debian 8.2 (Jessie)

echo "deb [arch=armhf] http://iachievedit-repos.s3.amazonaws.com/ jessie main" | sudo tee --append /etc/apt/sources.list

3. Run apt-get update

sudo apt-get update

4. Install swift-2.2!

apt-get install swift-2.2

Trying it Out

Again, a warning! There is no Swift package manager in this build yet. You can however compile basic applications, including those requiring Glibc and Foundation.

root@beaglebone:~# more helloWorld.swift 
print("Hello, world!")

for i in 1...10 {
  print(i)
}
root@beaglebone:~# swiftc helloWorld.swift 
root@beaglebone:~# ./helloWorld 
Hello, world!
1
2
3
4
5
6
7
8
9
10

or, try out swiftcat.swift which uses Glibc.

With the latest 0ubuntu11 release, Foundation is supported too!
countdown.swift

Stay Tuned!

Folks are working hard to bring Swift to our favorite ARM platforms. Updates will be posted to this article as well as to our @iachievedit Twitter account.

For those using Raspbian

Editor’s Note: This has been tested on at least one Raspberry Pi 3. YMMV.

For some reason clang-3.6 isn’t readily available on Raspbian systems. If you find yourself in the situation of having clang-3.7 or later and want to use the swift-2.2 package, you can use these steps. Yes, this is a bit of trickery.

Before starting, go ahead and install clang-3.7 (or whichever clang version you are using), as well as libicu-dev: sudo apt-get install clang-3.7 libicu-dev.

Then,

  1. Add the iAchieved.it LLC repository key (step #1 above)
  2. Add the Jessie repository.
echo "deb [arch=armhf] http://iachievedit-repos.s3.amazonaws.com/ jessie main" | sudo tee --append /etc/apt/sources.list
  1. Download the .deb file with aptitude download swift-2.2. The downloaded file will include something like 1%3a2.2-0ubuntu11_armhf in it.
  2. Perform the following steps (from Serverfault):
$ ar x swift-2.2_1%3a2.2-0ubuntu11_armhf.deb
$ tar xzf control.tar.gz
# Edit control in a text editor and replace clang-3.6 with clang-3.7
$ tar c control | gzip -c > control.tar.gz
$ ar rcs swift-2.2-new.deb debian-binary control.tar.gz data.tar.xz # Note the order is important

Then, install the new package file: sudo dpkg -i swift-2.2-new.deb. Good luck!

By

Ubuntu Packages for Open Source Swift

Swift 2.2
Quick and easy instructions for installing Open Source Swift 2.2 on Ubuntu 14.04 or 15.10 using apt-get.

1. Add the repository key

wget -qO- http://dev.iachieved.it/iachievedit.gpg.key | sudo apt-key add -

2. Add the appropriate repository to sources.list

Ubuntu 14.04

echo "deb http://iachievedit-repos.s3.amazonaws.com/ trusty main" | sudo tee --append /etc/apt/sources.list

Ubuntu 15.10

echo "deb http://iachievedit-repos.s3.amazonaws.com/ wily main" | sudo tee --append /etc/apt/sources.list

3. Run apt-get update

sudo apt-get update

4. Install swift-2.2!

apt-get install swift-2.2

I am currently serving this package off of Amazon S3 so it may take a bit to download. I am looking at whether or not I can afford utilizing CloudFront.

5. Try it out

git clone https://github.com/apple/example-package-dealer
cd example-packager-dealer
swift build
Compiling Swift Module 'FisherYates' (1 sources)
Linking Library:  .build/debug/FisherYates.a
Compiling Swift Module 'PlayingCard' (3 sources)
Linking Library:  .build/debug/PlayingCard.a
Compiling Swift Module 'DeckOfPlayingCards' (1 sources)
Linking Library:  .build/debug/DeckOfPlayingCards.a
Compiling Swift Module 'Dealer' (1 sources)
Linking Executable:  .build/debug/Dealer

Run it!

.build/debug/Dealer

FAQ

Q. Did Apple build these binaries?
A. No, I built them on my personal server using the instructions I posted here.

Q. What git revisions are included in the build?
A. You can use apt-cache show swift-2.2 to see this information. For example:

# apt-cache show swift-2.2
Package: swift-2.2
Version: 1:2.2-0ubuntu4~trusty1
Architecture: amd64
Maintainer: iachievedit (support@iachieved.it)
Depends: clang-3.6, libicu-dev
Homepage: http://dev.iachieved.it/iachievedit/swift
Priority: optional
Section: development
Filename: pool/main/s/swift-2.2/swift-2.2_2.2-0ubuntu4~trusty1_amd64.deb
Size: 60218098
SHA256: 69d5c38c97fd748af7829cb4c00d51539d0e6c21de9740754d2fa0da71c1bb8e
SHA1: 5872fb2a9c06cc4461d4a26eb9c4350e3b89ac1a
MD5sum: 87e12231e38a75e9126855b803daa4fd
Description: Open Source Swift
 This is a packaged version of Open Source Swift 2.2 built from
 the following git revisions of the Apple Github repositories:
       Clang:  f66c5bb67b
        LLVM:  7bae82deaa
       Swift:  23366f2001
  Foundation:  0b6479b664
Description-md5: 1c037ae5d6761bd55f026676773e5e30

The source tree is untouched for each build.

Q. Do you test the binaries before you upload them?
A. The Swift build process tests the resulting binaries, and I then do run some basic tests and compile my own applications, but there is currently no separate exhaustive test suite.

Q. Are you releasing builds on a set schedule?
A. Not really, though I may try to stay in sync with the releases from Apple. The idea was to get something out for folks to experiment with and start coding Swift on Linux.

Q. Where is everything installed?
A. Everything gets put in /usr, just like installing clang, gcc, etc.

Q. How do I decipher the package version number?
A. This was my first take on what I thought should be an appropriate package version number. Breaking it down, 2.2-0ubuntu3~trusty1 should be:

  • 2.2 is the version of Swift that is packaged
  • -0ubuntu3 indicates that this is the 3rd package for Ubuntu, with the 0 indicating that there is no upstream Debian package upon which this package was based
  • ~trusty1 indicates that this package is for Trusty Tahr

Wily packages do not contain anything like ~wiley1 in their version number such that an upgrade from Trusty to Wiley would automatically upgrade your swift-2.2 package correctly.

I think I got that right, but if you feel otherwise please do drop me a line at support@iachieved.it.

How Does This Work?

I used these awesome instructions on how to host a Debian package repository on Amazon S3. I tried to set up a Launchpad PPA, but quite frankly, got tired of trying to wade through all the metadata required to put together a simple package. I’m sure for hosting distribution repositories it’s all required, but for this it felt like overkill. The folks that develop fpm also have some choice things to say about it.

In addition I’ll be submitting the other bits on how to package and upload to Github.

By

Ubuntu 14.04 Package for Open Source Swift 2.2

Ubuntu 15.10 and 14.04 instructions are now available here!

Quick and easy instructions for installing Open Source Swift 2.2 on Ubuntu 14.04 using apt-get.

1. Add the repository key

wget -qO- http://dev.iachieved.it/iachievedit.gpg.key | sudo apt-key add -

2. Add the repository to sources.list

echo "deb http://iachievedit-repos.s3.amazonaws.com/ trusty main" | sudo tee --append /etc/apt/sources.list

3. Run apt-get update

sudo apt-get update

4. Install swift-2.2!

apt-get install swift-2.2

I am currently serving this package off of Amazon S3 so it may take a bit to download. I am looking at whether or not I can afford utilizing CloudFront.

5. Try it out

git clone https://github.com/apple/example-package-dealer
cd example-packager-dealer
swift build
Compiling Swift Module 'FisherYates' (1 sources)
Linking Library:  .build/debug/FisherYates.a
Compiling Swift Module 'PlayingCard' (3 sources)
Linking Library:  .build/debug/PlayingCard.a
Compiling Swift Module 'DeckOfPlayingCards' (1 sources)
Linking Library:  .build/debug/DeckOfPlayingCards.a
Compiling Swift Module 'Dealer' (1 sources)
Linking Executable:  .build/debug/Dealer

Run it!

.build/debug/Dealer

FAQ

Q. Did Apple build these binaries?
A. No, I built them on my personal server using the instructions I posted here.

Q. What git revisions are included in the build?
A. You can use apt-cache show swift-2.2 to see this information. For example:

# apt-cache show swift-2.2
Package: swift-2.2
Version: 1:2.2-0ubuntu4~trusty1
Architecture: amd64
Maintainer: iachievedit (support@iachieved.it)
Depends: clang-3.6, libicu-dev
Homepage: http://dev.iachieved.it/iachievedit/swift
Priority: optional
Section: development
Filename: pool/main/s/swift-2.2/swift-2.2_2.2-0ubuntu4~trusty1_amd64.deb
Size: 60218098
SHA256: 69d5c38c97fd748af7829cb4c00d51539d0e6c21de9740754d2fa0da71c1bb8e
SHA1: 5872fb2a9c06cc4461d4a26eb9c4350e3b89ac1a
MD5sum: 87e12231e38a75e9126855b803daa4fd
Description: Open Source Swift
 This is a packaged version of Open Source Swift 2.2 built from
 the following git revisions of the Apple Github repositories:
       Clang:  f66c5bb67b
        LLVM:  7bae82deaa
       Swift:  23366f2001
  Foundation:  0b6479b664
Description-md5: 1c037ae5d6761bd55f026676773e5e30

The source tree is untouched for each build.

Q. Do you test the binaries before you upload them?
A. The Swift build process tests the resulting binaries, and I then do run some basic tests and compile my own applications, but there is currently no separate exhaustive test suite.

Q. Are you releasing builds on a set schedule?
A. Not really, though I may try to stay in sync with the releases from Apple. The idea was to get something out for folks to experiment with and start coding Swift on Linux.

Q. Where is everything installed?
A. Everything gets put in /usr, just like installing clang, gcc, etc.

Q. How do I decipher the package version number?
A. This was my first take on what I thought should be an appropriate package version number. Breaking it down, 2.2-0ubuntu3~trusty1 should be:

  • 2.2 is the version of Swift that is packaged
  • -0ubuntu3 indicates that this is the 3rd package for Ubuntu, with the 0 indicating that there is no upstream Debian package upon which this package was based
  • ~trusty1 indicates that this package is for Trusty Tahr

I think I got that right, but if you feel otherwise please do drop me a line at support@iachieved.it.

How Does This Work?

I used these awesome instructions on how to host a Debian package repository on Amazon S3. I tried to set up a Launchpad PPA, but quite frankly, got tired of trying to wade through all the metadata required to put together a simple package. I’m sure for hosting distribution repositories it’s all required, but for this it felt like overkill. The folks that develop fpm also have some choice things to say about it.

In addition I’ll be submitting the other bits on how to package and upload to Github.

By

Building Open Source Swift

Swift 3.0 Linux X86

Editor’s note: If you’re looking for a quick way to start working with Swift 3.0 now, try out our apt-get repository for Trusty Tahr, Wily Werewolf, and Xenial Xerus.

It’s been a little over six months since Apple released its Swift language into the open source community. Since then its various repositories have quickly climbed into the trending pages on Github. The projects are moving fast with thousands of commits a week and features being implemented on a daily basis. While one can certainly download the official SNAPSHOT release from the Swift downloads page, to stay on the bleeding edge you’ll want to learn how to compile your own release.

Getting Started

To effectively work with building Swift from the Github repositories you’re going to need a system with a decent amount of CPU and RAM. I’m personally using an 8-core desktop with 8G of RAM. In addition I suggest compiling against an SSD drive with an EXT4 filesystem. While I trust my long term data storage to ZFS with deduplication, I don’t want to compile against it.

Note: We’re using Ubuntu 15.10, and Apple also supports 14.04 and now 16.04. I haven’t even tried building on a pure Debian release or RedHat installation.

First, let’s install some prerequisites:

sudo apt-get install git cmake ninja-build clang uuid-dev libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config

If you are on Ubuntu 14.04 you will also need to upgrade to Clang 3.6 and upgrade CMake to 3.4.3. Let’s start with getting clang-3.6:

sudo apt-get install clang-3.6
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.6 100
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.6 100

Now we need to check out (sorry, clone) all of the repositories needed to “build Swift”. Fortunately the base swift repository contains an update-checkout script that will clone all of the repositories required:

# mkdir apple # You will want to do this where you have at least 4-6G of free space
# cd apple
# git clone https://github.com/apple/swift.git swift
# ./swift/utils/update-checkout --clone

Building

While not quite as bad as building the Linux kernel, compiling Swift does take a while, particularly the first time. On my system it took roughly 40 minutes to compile and package everything. Rebuilding after updating the repository contents took around 7 minutes with no changes to the source, with most of that time spent executing the post-build testcases.

We’re going to use the buildbot_linux preset for the swift/utils/build-script utility. The preset is defined in build-presets.ini. Here you can see that it goes through specific build and install stages:

Here we go! In your ~/apple directory, or whereever you checked out all of the repositories:

./swift/utils/build-script --preset=buildbot_linux_1510 install_destdir=/tmp/install installable_package=/tmp/swift.tar.gz

You’ll notice our exact preset was buildbot_linux_1510, which for the time being is an alias to buildbot_linux. If you are building on an Ubuntu 16.04 system you can use builtbot_linux_1604. If you prefer you can also choose a different install_destdir and installable_package.

The output of the packaging step is a standard /usr/ directory layout with:

/usr/binThe swift compiler, lldb debugger, and various other tools get put here.
/usr/includeFor now this contains only lldb include headers.
/usr/libAll of the libraries, shared and static, that make up the Swift environment are here. For example, libFoundation.so, libswiftCore.so, etc.
/usr/localThe packaging builtbot creates this directory but nothing lies within.
/usr/shareFor now a single man page is included! Go ahead, try it out with man swift.

To test out your new Swift compiler:

cd /tmp/install
./usr/bin/swift --version
Swift version 3.0-dev (LLVM 7bae82deaa, Clang 4b3278bec5, Swift 0cc49187bb)
Target: x86_64-unknown-linux-gnu

The 3.0-dev information is baked into the source code in Version.cpp by gleaning the information from CMakeLists.txt. The hex strings for LLVM, Clang, and Swift are, you guessed it, the git hash that the binary was built from.

Swift man page

Swift man page

Updating and Rebuilding

To keep current with the latest submissions to the repositories you can use the update-checkout helper script:

./swift/utils/update-checkout --all

After updating to rebuild (without cleaning) use the previous command:

./swift/utils/build-script --preset=buildbot_linux_1510 install_destdir=/tmp/install installable_package=/tmp/swift.tar.gz

To run a distclean of sorts just remove the build/buildbot_linux directory that was created in your top-level directory, or you can selectively remove subdirectories within buildbot_linux to retrigger a build of that component. For example, if we want to clean out the artifacts for Foundation and the Swift Package Manager and then rebuild:

rm -rf build/buildbot_linux/foundation-linux-x86_64 build/buildbot_linux/swiftpm-linux-x86_64
./swift/utils/build-script --preset=buildbot_linux_1404 install_destdir=/tmp/install installable_package=/tmp/swift.tar.gz

Using

I am a fan of using the /opt/vendor directory structure for vendor packages. Once the build is complete you can use rsync to get it placed in an appropriate directory.

sudo rsync -a /tmp/install/ /opt/swift/swift-3.0/

Note: If you aren’t familiar with rsync be aware that trailing slashes are important!

Update your PATH to include /opt/swift/swift-3.0/usr/bin and you should be good to go! Try out your own personal Swift build with the Swift Package Manager.

Get the Code

We’ve included some helper scripts on Github.

# git clone https://github.com/iachievedit/package-swift

Start with running get.sh to clone all of the required Github repositories. Then for your first build, run package.sh and go watch an episode Walking Dead or Game of Thrones. When you get back you should have an install directory with your own version of Swift.

To refresh your sources run update.sh and then rebuild with package.sh.