2/20/2016 Update: Zewo has undergone a significant refactoring since this tutorial was originally written. The information below is no longer accurate and I will be working to provide a new tutorial as soon as possible.
Zewo
Zewo is new collection of modules designed to build web applications in Swift. Zewo provides:
- an HTTP server (Epoch)
- an HTTP router
- HTTP request and response entity classes
- an interface to a Mustache template responder (Sideburns)
- PostgreSQL and MySQL database adapters
and much more. Although relatively new to the scene, Zewo already has enough functionality to build a complete REST-based web service application. In the past I would have first considered Ruby and Sinatra, but now with the arrival of Swift on Linux, Zewo provides another great option.
A Quick API Design
We’re going to build a REST API similar to that of AT&T’s M2X platform. M2X is self-described as providing time-series data storage, device management, message brokering, event triggering, alarming, and data visualization for your industrial Internet of Things (IOT) products and services. That’s a mouthful. Our API will be less ambitious and implement the time-series data storage piece, but that is enough to showcase some of the capabilities of Zewo.
In the IOT world we can think about devices and streams of data that are associated with the device. For example, a device could be a single-board computer like a BeagleBone or Raspberry Pi with several sensors attached to it. The sensor, in turn, would be providing a stream of datapoints, say temperature readings. These concepts can be modeled nicely in a relational database fronted by a REST API. And, if relational databases aren’t your thing, folks are working on NoSQL interfaces such as MongoDB.
Devices, Streams, and Datapoints
Our model will consist of Device
, Stream
, and Datapoint
.
A Device
has the following attributes, or properties:
- name
- serial
- location
- streams
A Stream
has the following properties:
- name
- datapoints
And finally, a Datapoint
(sometimes referred to as a reading) will be:
- timestamp
- value
Now, believe me, this is the not the end-all be-all data model for an IoT application, but it provides enough structure to make our REST API interesting.
The operations we will support are:
- creating a new device
- posting a value to a stream
- retrieving the values of a stream
Of course, this is limited functionality, so it is left up to the reader to add things like:
- updating the location of the device
- adding a new stream to a device
So, let’s dive in!
First Things First
The example code provided at the end of the tutorial has only been tested on Linux, and everything done here is on an Ubuntu 14.04 system. To get up and running you will need to:
- install Swift
- install Postgres
- create a Postgres database named
iot_staging
- create a Postgres user named
iotuser
with the passwordiotuser
There are detailed instructions on these steps further below.
Once the prerequisites are installed we’re going to use the Swift Package Manager to bootstrap in our Zewo dependencies. In a directory named iotapp
create your Package.swift
file:
1 2 3 4 5 6 7 8 9 10 |
import PackageDescription let package = Package( name: "iotapp", dependencies:[ .Package(url:"https://github.com/Zewo/Epoch", majorVersion:0, minor:1), .Package(url:"https://github.com/Zewo/Middleware", majorVersion:0, minor:1), .Package(url:"https://github.com/Zewo/PostgreSQL", majorVersion: 0, minor:1), ] ) |
and then run swift build
. You will see the Swift Package Manager download and compile all of the required packages:
Cloning Packages/Epoch Using version 0.1.0 of package Epoch Cloning Packages/HTTPParser Using version 0.1.0 of package HTTPParser Cloning Packages/CHTTPParser Using version 0.1.0 of package CHTTPParser Cloning Packages/HTTP Using version 0.1.2 of package HTTP Cloning Packages/Core Using version 0.1.1 of package Core Cloning Packages/CURIParser Using version 0.1.0 of package CURIParser Cloning Packages/Venice Using version 0.1.0 of package Venice Cloning Packages/CLibvenice Using version 0.1.0 of package CLibvenice Cloning Packages/Middleware Using version 0.1.0 of package Middleware Cloning Packages/Router Using version 0.1.0 of package Router Cloning Packages/PostgreSQL Using version 0.1.0 of package PostgreSQL Cloning Packages/SQL Using version 0.1.0 of package SQL Cloning Packages/CLibpq Using version 0.1.0 of package CLibpq Compiling Swift Module 'Core' (15 sources) Linking Library: .build/debug/Core.a Compiling Swift Module 'HTTP' (10 sources) Linking Library: .build/debug/HTTP.a Compiling Swift Module 'HTTPParser' (4 sources) Linking Library: .build/debug/HTTPParser.a Compiling Swift Module 'Venice' (21 sources) Linking Library: .build/debug/Venice.a Compiling Swift Module 'Epoch' (9 sources) Linking Library: .build/debug/Epoch.a Compiling Swift Module 'Router' (1 sources) Linking Library: .build/debug/Router.a Compiling Swift Module 'Middleware' (11 sources) Linking Library: .build/debug/Middleware.a Compiling Swift Module 'SQL' (6 sources) Linking Library: .build/debug/SQL.a Compiling Swift Module 'PostgreSQL' (5 sources) Linking Library: .build/debug/PostgreSQL.a
Note: If the above bombed out it is likely due to missing dependencies. You don’t need to perform the bootstrap steps just yet, but if you’re chomping at the bit run the following:
sudo apt-get install libpq-dev echo "deb [trusted=yes] http://apt.zewo.io/deb ./" | sudo tee --append /etc/apt/sources.list sudo apt-get update sudo apt-get install uri-parser http-parser libvenice
Anatomy of a Zewo REST Application
A Zewo REST application looks similar to other HTTP application stacks such as Rails, Sinatra, etc. At the top of the stack is the familiar HTTP Server. Zewo provides us with Epoch, described as a Venice based HTTP server for Swift 2.2 on Linux. A straightforward main.swift
initializes our server:
1 2 3 |
import Epoch Server(port:8080, responder:router).start() |
Next down the stack is our routing functionality, which is an instance of Zewo Router. The router
is initialized in Router.swift
which looks like this:
1 2 3 4 5 6 |
import Router import Middleware let router = Router { route in route.router("/api", APIv1 >>> log) } |
The >>>
admittedly looks like a little black magic. It is an overloaded operator defined in Zewo Middleware. In this example it provides for logging our API actions.
The route.router("/api", APIv1 >>> log)
statement will anchor our web service at /api
. Let’s look now at APIv1.swift
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
import HTTP import Router import Middleware let APIv1 = Router("/v1") { route in route.get("/version") { _ in return Response(status: .OK, json:["version":"1.0.0"]) } // Create a new device route.post("/devices", parseJSON >>> deviceController.create) // Get a device route.get("/devices/:serial", deviceController.show) // Post a datapoint to a stream route.post("/devices/:serial/streams/:name/value", parseJSON >>> datapointController.create) // Get values posted to a stream route.get("/devices/:serial/streams/:name", datapointController.index) } |
APIv1
is where the real action is happening. Our URL anchor point is now extended to /api/v1
and we’ve specified a number of specific routes from there:
GET /api/v1/version
POST /api/v1/devices
GET /api/v1/devices/:serial
POST /api/v1/devices/:serial/streams/:name/value
GET /api/v1/devices/:serial/streams/:name
Again, while not the richest API, it serves as starting point.
If you’ve ever worked with Sinatra you will recognize the named parameters in the route patterns. These parameters, such as :serial
and :name
will be provided to us in our controllers, which we’ll see next.
Controllers
Zewo does all the heavy lifting of receiving an HTTP request and then routing the request to the appropriate controller. It is here though where we have to pick up the request and do something with it.
What is a controller again exactly? According to the Rails ActionController
documentation a controller “is responsible for making sense of the request and producing the appropriate output”. This is what our DeviceController
does for us:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
import Core import HTTP import Middleware let deviceController = DeviceController() final class DeviceController { let devices = try! DeviceRecord() let streams = try! StreamRecord() func create(request:Request) -> Response { guard let json = request.JSONBody else { return Response(status:.BadRequest, json:[ "message":"No device definition found" ]) } guard let name = json["name"]?.stringValue else { return Response(status:.BadRequest, json:[ "message":"name property missing" ]) } guard let serial = json["serial"]?.stringValue else { return Response(status:.BadRequest, json:[ "message":"serial property missing" ]) } var streamName:String guard let streams = json["streams"]?.arrayValue else { return Response(status:.BadRequest, json:[ "message":"streams property missing" ]) } var location = ZERO_POINT if let _location = json["location"] { location["latitude"] = _location["latitude"]!.doubleValue! location["longitude"] = _location["longitude"]!.doubleValue! } if let device = devices.insert(name, serial:serial, location:location) { for s in streams { streamName = s["name"]!.stringValue! let stream = self.streams.insert(streamName, deviceId:device.id) device.addStream(stream) } return Response(status:.OK, json:device.toJSON()) } else { return Response(status:.BadRequest, json:[ "message":"error creating device" ]) } } } |
Recall that it was the APIv1
router that calls deviceController.create
when a POST /devices
is received. The create
function takes a Request
object as its only argument, and then returns a Response
. In other words, the function is making sense of the request and producing the appropriate output. Making sense of the request is broken down into distinct steps:
- obtaining the JSON body of the request and extracting required fields
name
andserial
- obtaining the location field if it is available, otherwise the location defaults to (0.0, 0.0)
- extracting the required
streams
field from the JSON - inserting a new
Device
into the database - inserting a new
Stream
for each stream specified
The response to this request can be one of two types:
.BadRequest
(HTTP Status 400).OK
(HTTP Status 200)
In both cases we return a JSON body with additional information.
An example of a valid JSON request body for POST /devices
looks like this:
1 2 3 4 5 6 7 8 |
{"name":"BeagleBone", "serial":"NATT-8GXF-B7GF-RXNC", "location":{ "latitude":-34.8836, "longitude":-56.1819 }, "streams":[] } |
If the request was successful we expect a response to look like:
1 2 3 4 5 6 7 8 9 10 |
{ "serial": "NATT-8GXF-B7GF-RXNC", "streams": [], "name": "BeagleBone", "location": { "longitude":"-56.1819", "latitude":"-34.8836" }, "id": "14" } |
The server returned an additional field id
in the JSON response. Some APIs are designed such that the client use the id
field for future references to the created object. Our API, on the other hand, will use the serial number to identify a given device.
Records
Observe that the controller does not interact directly with the database. That is the job of our record class, which is modeled after the concept of ActiveRecord
in Rails. The DeviceRecord
class knows about Postgres, what table is used to manage devices (in fact, it creates that table), and how to insert and find devices. In Rails ActiveRecord
is smart enough to read the table schema and create methods such as find
on the fly. Our DeviceRecord
does not go that far; we aren’t building Rails here.
This is the content of our DeviceRecord.swift
file:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
import PostgreSQL import CLibpq import Foundation final class DeviceRecord { let tableName = "devices" let db = Connection("postgresql://iotuser:iotuser@localhost/iot_staging") init() throws { try db.open() try db.execute("CREATE TABLE IF NOT EXISTS \(tableName) (id SERIAL PRIMARY KEY, name VARCHAR(256), serial VARCHAR(256) UNIQUE, location POINT)") } deinit { db.close() } func insert(name:String, serial:String, location:Point) -> Device { let stmt = "INSERT into \(tableName) (name,serial,location) VALUES('\(name)','\(serial)',POINT(\(location["latitude"]!),\(location["longitude"]!))) RETURNING id" logmsg(stmt) let result = try! db.execute(stmt) let id = result[0]["id"]!.string! return Device(id:id, name:name, serial:serial, location:location) } func findBySerial(serial:String) -> Device? { let stmt = "SELECT * from devices where serial = '\(serial)'" logmsg(stmt) let result = try! db.execute(stmt) if result.count > 0 { let id = result[0]["id"]!.string! let name = result[0]["name"]!.string! let serial = result[0]["serial"]!.string! let location = result[0]["location"]!.string! let locationAsPoint = pointFromString(location) return Device(id:id, name:name, serial:serial, location:locationAsPoint) } return nil } } |
At a fundamental level there really isn’t a whole lot going on in this file, with of course the exception that we are issuing Postgres SQL commands to the underlying database adapter and reading the result.
Before leaving the DeviceRecord
, notice that insert
returns Device
. This is not a good idea, and we should change the implementation to return Device?
in the event there is an issue with the SQL INSERT
. The example code in Github will return Device?
so we can catch these errors.
Combining all the components together, top-to-bottom, we get a view of what the “stack” looks like:
A Working Example
Now that we have a basic idea of how Zewo is used to build a REST web service, let’s run a working example.
First, install of the dependencies you’ll need, which includes libpq-dev
for the Postgres headers, as well as some additional packages Zewo requires.
sudo apt-get install libpq-dev echo "deb [trusted=yes] http://apt.zewo.io/deb ./" | sudo tee --append /etc/apt/sources.list sudo apt-get update sudo apt-get install uri-parser http-parser libvenice
Now, get the code from Github and switch to the basic
branch.
git clone https://github.com/iachievedit/iotapp cd iotapp git checkout basic
Build the application with swift build
and run it:
swift build .build/debug/iotapp
If Postgres isn’t running or cannot otherwise be contacted, you might see this error:
fatal error: 'try!' expression unexpectedly raised an error: PostgreSQL.Connection.Error.ConnectFailed("could not connect to server: Connection refused\n\tIs the server running on host \"localhost\" (::1) and accepting\n\tTCP/IP connections on port 5432?\ncould not connect to server: Connection refused\n\tIs the server running on host \"localhost\" (127.0.0.1) and accepting\n\tTCP/IP connections on port 5432?\n"): file swift/stdlib/public/core/ErrorType.swift, line 53 Illegal instruction (core dumped)
In this case, install Postgres and create your user and database:
$ sudo apt-get install postgresql $ sudo su - postgres $ psql template1 psql (9.4.5) Type "help" for help. template1=# CREATE USER iotuser WITH PASSWORD 'iotuser'; CREATE ROLE template1=# CREATE DATABASE iot_staging; CREATE DATABASE template1=# GRANT ALL PRIVILEGES ON DATABASE "iot_staging" to iotuser; GRANT template1=# \q $ exit
Once the application is up and running, we can use cURL or Postman to create a new device. Here we’ll use cURL:
$ curl -X POST -H "Content-Type: application/json" -d '{"name":"BeagleBone", "serial":"N9TT-8GXF-B7GF-RXNC", "location":{"latitude":-34.8836, "longitude":-56.1819}, "streams":[{"name":"cpu-occupancy"}, {"name":"free-memory"}]}' 'http://localhost:8080/api/v1/devices'
Our Zewo application processes the request, logging to STDOUT:
12/23/15, 8:11 PM /home/iotapp/iotapp/Sources/DeviceRecord.swift:23 insert(_:serial:location:): INSERT into devices (name,serial,location) VALUES('BeagleBone','N9TT-8GXF-B7GF-RXNC',POINT(-34.8836,-56.1819)) RETURNING id POST /api/v1/devices HTTP/1.1 Content-Type: application/json Accept: */* User-Agent: curl/7.43.0 Host: localhost:8080 Content-Length: 200 {"name":"BeagleBone", "serial":"N9TT-8GXF-B7GF-RXNC", "location":{ "latitude":-34.8836, "longitude":-56.1819 }, "streams":[ {"name":"cpu-occupancy"}, {"name":"free-memory"} ] } - HTTP/1.1 200 OK content-type: application/json; charset=utf-8 Content-Length: 128 {"id":"1","name":"BeagleBone","location":{"longitude":-56.1819,"latitude":-34.8836},"streams":[],"serial":"N9TT-8GXF-B7GF-RXNC"} ----------------------------------------
The response to our command is what we expect, a JSON string with our device!
{"id":"1","name":"BeagleBone","location":{"longitude":-56.1819,"latitude":-34.8836},"streams":[],"serial":"N9TT-8GXF-B7GF-RXNC"}
Try issuing the command again with the same serial number. You should see {"message":"error creating device"}
thanks to the logic we added to return a .BadRequest
if we couldn’t insert the device.
Creating Streams and Posting Datapoints
We now need to add the functionality that creates streams
and datapoints
. Unlike with Device
we do not need a StreamController
. Streams are created directly through a StreamRecord
instance in the DeviceController
.
Our StreamRecord
code looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 |
import PostgreSQL import CLibpq import Foundation final class StreamRecord { let tableName = "streams" let db = Connection("postgresql://iotuser:iotuser@localhost/iot_staging") init() throws { try db.open() try db.execute("CREATE TABLE IF NOT EXISTS \(tableName) (id SERIAL PRIMARY KEY, name VARCHAR(256), device_id INTEGER NOT NULL)") } deinit { db.close() } func insert(name:String, deviceId:String) -> Stream { let stmt = "INSERT into \(tableName) (name, device_id) VALUES('\(name)', '\(deviceId)') RETURNING id" logmsg(stmt) let result = try! db.execute(stmt) let id = result[0]["id"]!.string! return Stream(id:id, name:name) } func findByName(name:String, deviceId:String) -> Stream? { let stmt = "SELECT * from \(tableName) where name = '\(name)' and device_id = '\(deviceId)'" logmsg(stmt) let result = try! db.execute(stmt) if result.count > 0 { return Stream(id:result[0]["id"]!.string!, name:result[0]["name"]!.string!) } else { return nil } } } |
Note that our streams
schema definition contains device_id
, which is a reference to the device that “owns” the stream. Again, unlike with Rails, we are not specifying has_a
or belongs_to
relationships explicitly; we manage this through code.
Posting Datapoints
A stream is a collection of data points, or values. For example, let’s say our BeagleBone device had a temperature sensor attached to it. We would want to POST a temperature reading to the temperature
stream, like this:
curl -X POST -H "Content-Type: application/json" -d '{"value":"77"}' 'http://localhost:8080/api/v1/devices/NFTT-8GXF-B7GF-RXNC/streams/temperature/value'
Our API responds with:
{ "value": "77", "inserted_at": "2015-12-23 21:13:42.763442", "id": "1" }
Some time later the temperature increases to 78 degrees, so another value is posted. Later on, it increases to 80 degrees, and so on. Then, we can retrieve our datapoints via a GET request:
curl -X GET -H "Content-Type: application/json" 'http://localhost:8080/api/v1/devices/NFTT-8GXF-B7GF-RXNC/streams/temperature'
The following data is returned:
{ "datapoints": [ { "value": "77", "inserted_at": "2015-12-23 21:13:42.763442", "id": "1" }, { "value": "78", "inserted_at": "2015-12-23 21:15:30.536468", "id": "2" }, { "value": "80", "inserted_at": "2015-12-23 21:15:54.651966", "id": "3" } ] }
Here is our implementation of DatapointController
. Note how the methods DeviceRecord.findBySerial(serial:)
and StreamRecord.findByName(name:,deviceId:)
are used to associate the datapoint with the correct device and stream.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
import Core import HTTP import Middleware let datapointController = DatapointController() final class DatapointController { let devices = try! DeviceRecord() let streams = try! StreamRecord() let datapoints = try! DatapointRecord() func create(request:Request) -> Response { logmsg("ENTRY") let deviceSerial = request.parameters["serial"]! let streamName = request.parameters["name"]! guard let json = request.JSONBody, value = json["value"]?.stringValue else { return Response(status:.BadRequest) } guard let device = devices.findBySerial(deviceSerial) else { return Response(status:.NotFound) } guard let stream = streams.findByName(streamName, deviceId:device.id) else { return Response(status:.NotFound) } if let datapoint = datapoints.insert(value, streamId:stream.id) { return Response(status:.OK, json:datapoint.toJSON()) } return Response(status:.BadRequest) } func index(request:Request) -> Response { logmsg("ENTRY") let deviceSerial = request.parameters["serial"]! let streamName = request.parameters["name"]! guard let device = devices.findBySerial(deviceSerial) else { return Response(status:.NotFound) } guard let stream = streams.findByName(streamName, deviceId:device.id) else { return Response(status:.NotFound) } let values = datapoints.findByStream(stream.id, count:10) let jsonValues = [ "datapoints":JSON.from(values) ] return Response(status:.OK, json:JSON.from(jsonValues)) } } |
Our DatapointRecord
implementation looks like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
import PostgreSQL import CLibpq import Foundation final class DatapointRecord { let tableName = "datapoints" let db = Connection("postgresql://iotuser:iotuser@localhost/iot_staging") init() throws { try db.open() try db.execute("CREATE TABLE IF NOT EXISTS \(tableName) (id SERIAL PRIMARY KEY, inserted_at TIMESTAMP WITHOUT TIME ZONE DEFAULT (now() AT TIME ZONE 'utc'), value TEXT, stream_id INTEGER NOT NULL)") } deinit { db.close() } func insert(value:String, streamId:String) -> Datapoint? { let stmt = "INSERT into \(tableName) (value, stream_id) VALUES('\(value)', '\(streamId)') RETURNING id, inserted_at" logmsg("insert datapoint: \(stmt)") do { let result = try db.execute(stmt) let id = result[0]["id"]!.string! let inserted_at = result[0]["inserted_at"]!.string! logmsg("datapoint inserted with id \(id)") return Datapoint(id:id, value:value, inserted_at:inserted_at) } catch { return nil } } func findByStream(streamId:String, count:Int) -> [Datapoint] { let stmt = "SELECT * FROM \(tableName) WHERE stream_id = \(streamId) LIMIT \(count)" logmsg(stmt) var datapoints:[Datapoint] = [] do { let results = try db.execute(stmt) for result in results { let id = result["id"]!.string! let value = result["value"]!.string! let inserted_at = result["inserted_at"]!.string! let datapoint = Datapoint(id:id, value:value, inserted_at:inserted_at) datapoints.append(datapoint) } return datapoints } catch { return [] } } } |
An exercise for the reader: The DatapointRecord.findByStream(streamId:count:)
function is called with a hardcoded value of 10. Extend the API to allow for the number of datapoints to be specified either by a JSON structure in the POST body, or through a URI query parameter. Consider also adding inserted_before
and inserted_after
filters to control which values to return.
Complete Application
The completed IoT application is on Github on the master
branch. With it you should be able to create devices (make sure and specify the streams up front or add an implementation of modifying a device to add a new stream!), and post datapoints to the device streams.
There are some additional files and helper routines in the repository:
- the
+JSON
files providetoJSON
extension functions to our objects Utils.swift
adds alogmsg
routine
As I mentioned, there are a lot of capabilities that can be added to this API. Streams can be given datatypes and units, operations which find min
and max
datapoints in a stream could be added, and so on. For inspiration check out the M2X API.
What’s Next?
After spending time with the Zewo framework over the past week I’m convinced it will become one of the go to building blocks for developing web applications in Swift. This article has focused on building the server side of the REST API, and I’m looking forward to realizing the dream of developing in one language when we look at developing a client side set of classes in Swift that can be used to interact with the service. Stay tuned.
Can you please update this blog since Zewo has changed a couple of times since you originally published it?
Nate, thanks for the comment. Yes, Zewo has undergone quite a transformation; I need to find the time to write up a new tutorial. In the meantime I can add a disclaimer that the information is out of date.