iAchieved.it

Software Development Tips and Tricks

By

Swift 3.0 for Ubuntu 16.04 Xenial Xerus

Swift 3.0

In our zeal to get Swift 3.0 on as many Linux systems as possible we went ahead and got a jumpstart on building for Ubuntu 16.04, also known as Xenial Xerus, for X86 systems. Installation is as easy as using our APT repository with apt-get. Binaries are installed into /opt/swift/swift-3.0, so you will need to update your path after installing the 3.0 package. Editor’s note: For the rationale as to why we use /opt/swift rather than /usr/bin/, see this thread on the swift-dev mailing list.

1. Add the repository key

wget -qO- http://dev.iachieved.it/iachievedit.gpg.key | sudo apt-key add -

2. Add the Xenial repository to sources.list

echo "deb http://iachievedit-repos.s3.amazonaws.com/ xenial main" | sudo tee --append /etc/apt/sources.list

3. Run apt-get update

sudo apt-get update

4. Install swift-3.0!

sudo apt-get install swift-3.0

5. Update that PATH!

export PATH=/opt/swift/swift-3.0/usr/bin:$PATH

6. Try it out

git clone https://github.com/apple/example-package-dealer
cd example-packager-dealer
swift build
Compiling Swift Module 'FisherYates' (1 sources)
Linking Library:  .build/debug/FisherYates.a
Compiling Swift Module 'PlayingCard' (3 sources)
Linking Library:  .build/debug/PlayingCard.a
Compiling Swift Module 'DeckOfPlayingCards' (1 sources)
Linking Library:  .build/debug/DeckOfPlayingCards.a
Compiling Swift Module 'Dealer' (1 sources)
Linking Executable:  .build/debug/Dealer

Run it!

.build/debug/Dealer

FAQ

Q. Did Apple build these binaries?
A. No, I built them on my personal server using the instructions I posted here.

Q. What git revisions are included in the build?
A. You can use apt-cache show swift-3.0 to see this information. For example:

# apt-cache show swift-3.0
Package: swift-3.0
Conflicts: swift-2.2
Version: 1:3.0-0ubuntu10+xenial1
Architecture: amd64
Installed-Size: 370463
Maintainer: iachievedit (support@iachieved.it)
Depends: clang (>= 3.6), libicu-dev
Homepage: http://dev.iachieved.it/iachievedit/swift
Priority: optional
Section: development
Filename: pool/main/s/swift-3.0/swift-3.0_3.0-0ubuntu10+xenial1_amd64.deb
Size: 72513864
SHA256: b1bf548f353466ea72696089a8b666956a2603edb467eb0517e858eb1ba86511
SHA1: 5dd02b14d21f2e821040de3bb1052561653fcfcd
MD5sum: f2c3d3b9517a303cc86558b6c560a8d6
Description: Open Source Swift
 This is a packaged version of Open Source Swift 3.0 built from
 the following git revisions of the Apple Github repositories:
       Clang:  460d629e85
        LLVM:  8d0086ac3e
       Swift:  1abe85ab41
  Foundation:  4c15543f82
Description-md5: a6b1dd247c7584b61692a101d9d0e5fa

The source tree is untouched for each build.

Q. Do you test the binaries before you upload them?
A. The Swift build process tests the resulting binaries, and I then do run some basic tests and compile my own applications, but there is currently no separate exhaustive test suite.

Q. Are you releasing builds on a set schedule?
A. Not really, though I may try to stay in sync with the releases from Apple. The idea was to get something out for folks to experiment with and start coding Swift on Linux.

Q. Where is everything installed?
A. Everything gets put in /opt/swift/swift-3.0/usr.

Q. How do I decipher the package version number?
A. This was my first take on what I thought should be an appropriate package version number. Breaking it down, 3.0-0ubuntu10+xenial1 should be:

  • 3.0 is the version of Swift that is packaged
  • -0ubuntu10 indicates that this is the 2nd package for Ubuntu, with the 0 indicating that there is no upstream Debian package upon which this package was based
  • +xenial1 indicates that this package is for Xenial Xerus

I think I got that right, but if you feel otherwise please do drop me a line at support@iachieved.it.

How Does This Work?

I used these awesome instructions on how to host a Debian package repository on Amazon S3. I tried to set up a Launchpad PPA, but quite frankly, got tired of trying to wade through all the metadata required to put together a simple package. I’m sure for hosting distribution repositories it’s all required, but for this it felt like overkill. The folks that develop fpm also have some choice things to say about it.

The packaging scripts we use to build everything and upload to the repository can be found Github. For Swift 3.0 make sure and look at the swift-3.0 branch.

By

Building Swift 3.0 on a Raspberry Pi 3

Swift 3.0 Swift 3.0 Swift 3.0

9/24/2016 Update: See our latest post on building Swift 3.0 on ARMv7 systems such as the Pi 3.

Editor’s Note: This post was updated on 9/4/2016 to reflect the recent updates for Swift 3.0 (in particular it now includes the build of swift-corelibs-libdispatch).

A number of folks have expressed interest in building Swift 3.0 on their own Raspberry Pi 3. These instructions are for those brave souls who want to do so!

To get started you’re going to need:

Note that this can be done; we’ve done it, and there are group of folks dedicated to working with Swift on ARM-based devices. Don’t give up. If you do decide to give up we aren’t ones to judge, just head over and download our prebuilt Swift 3.0 package for the Raspberry Pi 2 and 3.

Install Xenial

We will not be using Raspbian here, but rather Ubuntu Xenial. Obtain Xenial with curl or wget and then write the filesystem out to your microSD card with dd.

Note: You’ve read this warning before I’m sure, but it is critical that you put in the right device name where we reference YOUR_DEVICE below. Failure to do so could destroy your data (they don’t call dd disk destroyer for nothing).

Using Linux

# wget http://www.finnie.org/software/raspberrypi/ubuntu-rpi3/ubuntu-16.04-preinstalled-server-armhf+raspi3.img.xz
# xzcat ubuntu-16.04-preinstalled-server-armhf+raspi3.img.xz | sudo dd of=/dev/YOUR_DEVICE bs=8M

To find your YOUR_DEVICE insert the microSD into your Linux machine and run dmesg|tail to see something like

[1255451.544316] usb-storage 4-2:1.0: USB Mass Storage device detected
[1255451.544466] scsi host25: usb-storage 4-2:1.0
[1255452.545332] scsi 25:0:0:0: Direct-Access     Generic  STORAGE DEVICE   0817 PQ: 0 ANSI: 6
[1255452.545670] sd 25:0:0:0: Attached scsi generic sg8 type 0
[1255452.828405] sd 25:0:0:0: [sdh] 15523840 512-byte logical blocks: (7.94 GB/7.40 GiB)
[1255452.829402] sd 25:0:0:0: [sdh] Write Protect is off
[1255452.829409] sd 25:0:0:0: [sdh] Mode Sense: 23 00 00 00
[1255452.830400] sd 25:0:0:0: [sdh] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[1255452.835967]  sdh: sdh1
[1255452.840401] sd 25:0:0:0: [sdh] Attached SCSI removable disk

In this example your device is /dev/sdh.

Using OS X (macOS)

I personally prefer to use a separate Linux server for these types of operations, but you can use a Mac as well with homebrew:

# brew install wget xz
# wget http://www.finnie.org/software/raspberrypi/ubuntu-rpi3/ubuntu-16.04-preinstalled-server-armhf+raspi3.img.xz
# xzcat ubuntu-16.04-preinstalled-server-armhf+raspi3.img.xz|sudo dd of=/dev/rdisk3 bs=8m

Note when using dd on the Mac its advisable to use /dev/rYOUR_DEVICE. Using /dev/YOUR_DEVICE can be painfully slow, as explained here.

To find YOUR_DEVICE on a Mac, insert the microSD and run diskutil list in a terminal. Look for the disk that that matches your SD card size (if you have other attached drives of the same size look also at its other attributes to ensure you are using the right one).

# diskutil list
...
/dev/disk5 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *16.0 GB    disk5
   1:             Windows_FAT_32 system-boot             134.2 MB   disk5s1
   2:                      Linux                         15.9 GB    disk5s2

If you are nervous about the steps with diskutil and dd, read this walkthrough to gain some confidence. As an aside, if you’re going to make a habit out of working with the Raspberry Pi the wget, xzcat, and dd commands are a critical part of your toolbox. Get comfortable with them.

Update Xenial

Now that you have Xenial written to the microSD card, insert it into the Raspberry Pi 3 and power up! And yes, we’re assuming you have an Ethernet cable attached or you have network connectivity through the miniUSB. You can try to ssh into the Pi with ssh ubuntu@ubuntu.local first, but if that fails use your router LAN page to determine the IP address. The default password is ubuntu.

# ssh ubuntu@192.168.1.115
The authenticity of host '192.168.1.115 (192.168.1.115)' can't be established.
ECDSA key fingerprint is 62:e9:f9:09:d0:30:3c:c9:0e:47:a3:42:f5:2c:e2:ae.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.115' (ECDSA) to the list of known hosts.
ubuntu@192.168.1.115's password:
You are required to change your password immediately (root enforced)
Welcome to Ubuntu 16.04 LTS (GNU/Linux 4.4.0-1009-raspi2 armv7l)
...
WARNING: Your password has expired.
You must change your password now and login again!
Changing password for ubuntu.
(current) UNIX password:

Once you’ve changed your password log back in with the ubuntu user (and your new password) and update the system with sudo apt-get update.

Warning: Previous versions of this post suggested running sudo apt-get upgrade -y as well. See the comments below for the workarounds if you cannot login via Ethernet afterwards.

Install Build Prerequisites

Compiling Swift requires a number of prerequisites. Get them in one fell swoop with:

# sudo apt-get install -y git cmake ninja-build clang uuid-dev libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config autoconf libtool systemtap-sdt-dev libcurl4-openssl-dev

Fortunately we didn’t have to discover the prerequisites iteratively (you know, when you download a repository and try to build it and discover what the prerequisites are, one at a time) because they are outlined in the Open Source Swift README.

I also like to include htop (sudo apt-get install -y htop) so I can see the Pi compile it’s little heart out:

100%

100%

Add Some Swap

While the Pi 3 does come with an impressive 1GB of RAM, that is still not enough to build Swift from beginning to end (it’s the linking steps that usually get us into trouble). Let’s add some swap to double that amount:

# cd /var/cache
# sudo mkdir swap
# cd swap
# sudo fallocate -l 1G 1G.swap
# sudo mkswap 1G.swap
Setting up swapspace version 1, size = 1024 MiB (1073737728 bytes)
no label, UUID=184d002a-2f15-4b23-8360-8e792badc6a2
# sudo chmod 600 1G.swap
# sudo swapon 1G.swap

Here’s an example of what happens when you don’t have swap enabled:

cd /root/workspace/Swift-3.0-Pi3-ARM-Incremental/build/buildbot_linux/swift-linux-armv7/bin && /usr/bin/cmake -E create_symlink swift swiftc && cd /root/workspace/Swift-3.0-Pi3-ARM-Incremental/build/buildbot_linux/swift-linux-armv7/bin && /usr/bin/cmake -E create_symlink swift swift-autolink-extract
clang: error: unable to execute command: Killed
clang: error: linker command failed due to signal (use -v to see invocation)
ninja: build stopped: subcommand failed.
./swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting

Doubling the amount of your Pi’s RAM with swap space that’s on a microSD card is not something you would typically do. Once we do hit the build phases that require us to dip into swap things will slow down, but the alternative is to crash with an out-of-memory error.

Use package-swift

package-swift is our Github repository that has several helper scripts to make life a little easier building Swift.

# git clone https://github.com/iachievedit/package-swift
# cd package-swift

By default package-swift will check out to the swift-3.0 branch. This branch contains important patches needed to properly build Swift for ARM (to see them take a look in patches/armv7, they are organized by the various repos they patch).

The scripts are pretty simple:

  • get.sh – Downloads all of the required repositories for building Swift
  • update.sh – Updates all of your local repositories with the latest on Github.
  • patch.sh – Applies any patches that the Swift on ARM team has put forward; from time to time we have to apply patches to get things working. Once a patch is no longer necessary we remove it, so before performing the actual build you will want to git pull to get the latest patchset
  • package.sh – Runs the Swift build-script with a preset named buildbot_linux_armv7.

I suggest you look at each shell script and walk through the logic before executing. They are straightforward but it’s worth the time understanding what they are doing. Pay particular attention to the get.sh and update.sh scripts in that they detect we are building on a ARMv7 processor and thus need William Dillon’s swift-llvm fork.

Let’s Do This

Remember to make sure you have:

  • installed all of the prerequisite packages
  • enabled 1GB of swap space
  • cloned package-swift from Github

Now, clone all of the required repositories:

# cd package-swift
# ./get.sh

On my SD card this step took about 15 minutes to download and write everything.

Next, apply patches. It is important to do the step to pick up patches that the Swift on ARM team has developed and tested. From time to time these patches change so make sure and pick up the latest.

# ./patch.sh

Now, compile. The Swift on ARM team uses Jenkins to run this in a headless build job, but if you use nohup to ensure your compile continues even if the terminal detaches you should be fine:

# nohup ./package.sh > swiftbuild.log&

To watch the output of the build use tail -F swiftbuild.log.

Now, be prepared to wait! A clean build of Swift on a Raspberry Pi 3 can take upwards of 6 hours. The reward is well worth it, however, and that is a swift.tar.gz bundle that can be installed on either a Raspberry Pi 2 or Pi 3 running Ubuntu Xenial (do not try to install this on a Raspbian machine, you will be sorry). I like to install Swift in /opt/swift/ like so:

# cd /opt
# mkdir -p swift/swift-3.0
# cd swift/swift-3.0
# tar -xzvf /path/to/swift.tar.gz

You can then set your PATH with export PATH=/opt/swift/swift-3.0/usr/bin:$PATH. To properly use swiftc and swift build you will also need to run the following commands:

sudo apt-get install -y libicu-dev
sudo apt-get install -y clang-3.6
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.6 100
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.6 100

Getting Help

There is a high likelihood that your build will fail the first time. In fact, it is possible that builds for ARM devices such as the Raspberry Pi 2 and 3 will be failing for days until someone is available to fix them. If you are having trouble compiling please feel free to send an e-mail to admin@iachieved.it to get invited to the Swift for ARM Slack community.

Acknowledgements

I should probably put up a separate page just for this; a number of folks have been instrumental in getting Swift going on ARM devices. Folks like PJ have blazed the trail building Swift for the Raspberry Pi 1, and no article on Swift on the Pi is complete without mentioning William Dillon’s work.

By

Swift 3.0 on Raspberry Pi 2 and 3

Swift 3.0 Swift 3.0 Swift 3.0

There are a number of people working to bring Swift 3.0 to as many ARM-based systems as possible. This article is specifically about getting Swift 3.0 for your Raspberry Pi 2 or Raspberry Pi 3 that is running Ubuntu 16 (Xenial Xerus). It has not been tested for Raspbian variants (and likely doesn’t work).

A fair warning: support for Swift 3.0 on the Raspberry Pi (and all ARM devices) is still beta. Use it for prototyping and proof-on-concept work rather than building a product with it. Also, if you’re interested in joining the community of folks working on getting Swift 3.0 for ARM devices, come join us on Slack!

Xenial on a Raspberry Pi

You might not have even known Xenial for the Raspberry Pi existed; I didn’t! To grab it head over to the Ubuntu Wiki and get the distribution for your Pi. You will want to use at least an 8G SD card.

Install Swift 3.0

The team working on Swift for ARM is using Jenkins to build the Raspberry Pi binaries natively on a Raspberry Pi 3. As in, this is the build machine!

Swift 3.0 Build Machine

Swift 3.0 Build Machine

For the curious check out the Jenkins build project. Quite frankly, I’m amazed that it only took 6 hours to compile Swift.

Now, on your Raspberry Pi, grab and extract the Swift 3.0 build artifact into a directory and set your PATH with something like:

Note: You aren’t tied to putting Swift 3.0 in $HOME, I typically use /opt/swift/swift-3.0.

Now, let’s give it a spin!

Create a file called helloWorld.swift:

You can use swift helloWorld.swift to execute the file like a script:

# swift helloWorld.swift
Hello, world!

Or, you can compile the file into an executable with swiftc if you have clang installed and configured properly:

# swiftc helloWorld.swift
# ./helloWorld
Hello world!

If swiftc failed with error: link command failed with exit code 127 there’s a good chance you don’t have clang installed and configured properly:

sudo apt-get update
sudo apt-get install -y libicu-dev
sudo apt-get install -y clang-3.6
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.6 100
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.6 100

Let’s look at a few other tidbits:

Importing Glibc works!

swiftcat.swift:

Compile (swiftc swiftcat.swift) and run (swiftcat)!

Bridging to C routines

Linking against compiled object files works!

escapetext.c:

escapetext.h:

escapeswift.swift:

Compile and link everything together:

# clang -c escapetext.c
# swiftc -c escapeswift.swift -import-objc-header escapetext.h
# swiftc escapeswift.o escapetext.o -o escapeswift -lcurl

And run:

# ./escapeswift "foo > bar"
Escaped text:  foo%20%3E%20bar

Swift Package Manager

Unless you enjoy writing makefiles and build scripts (and trust me, some do), the Swift Package Manager is here to help manage software package dependencies. We’ll be writing more about the SwiftPM available in Swift 3.0, but are excited to provide a version that works on armv7 devices. Try this out:

# mkdir finalCountdown && cd finalCountdown
# swift package init --type executable
Creating executable package: finalCountdown
Creating Package.swift
Creating .gitignore
Creating Sources/
Creating Sources/main.swift
Creating Tests/

Replace the contents of Sources/main.swift with

Now, run swift build to build your finalCountdown application:

# swift build
Compile Swift Module 'finalCountdown' (1 sources)
Linking .build/debug/finalCountdown
# .build/debug/finalCountdown
Entering thread
10...9...8...7...6...5...4...3...2...1...
Exiting thread
Done

moreswift

For a random smattering of Swift 3.0 applications that have been run on both x86 and armv7 systems, check out the moreswift swift-3.0 branch.

What Version Is This?

The current builds available are not tracking the Swift 3.0 preview builds. To determine the Git hashes associated with the swift binary, type swift --version:

# swift --version
Swift version 3.0-dev (LLVM eb140647a6, Clang a9f2183da4, Swift bb43874ba1)

You can then, if you so choose go directly to the Swift repository commit with something like https://github.com/apple/swift/tree/bb43874ba1 to see the history beginning with the last commit incorporated into the build.

Acknowledgements

A lot of people have been apart of the effort to bring Swift to Linux ARM devices. This list is just a few of those individuals. Please do visit their blogs; there is a wealth of information and learnings therein.

By

Handling MQTT Subscriptions with Swift on Linux

In our last post we looked at a Swift MQTT client which published information to a HiveMQ broker. Now let’s turn our attention to writing an MQTT client that subscribes to a topic.

Again, we’ll be using MQTT, code that aims to provide a solid MQTT client library implementation in Swift. If you haven’t yet read the tutorial on building an MQTT publisher with it, I suggest you do so before proceeding!

There are two basic components to writing an MQTT client that subscribes to a topic:

  • subscribing to a topic
  • handling a published message from the broker

Ignoring the boilerplate connection logic for a moment, let’s look at subscribing:

There’s nothing earth shattering here: upon receiving a ConnectedNotification a call is made to subscribe to the topic /hostname/cpu/temperature/value (where hostname is obtained from the OS). Now when the broker receives a message published to this topic it will broadcast it to our client. This is the pub-sub pattern in action.

This Transmission is Coming To You

To receive an MQTT message we need to handle the MQTTDelegate method func mqtt(mqtt: MQTT, didReceiveMessage message: MQTTMessage, id: UInt16 ). This can be readily accomplished using an NSNotification with accompanying userInfo:

Unfortunately for us much the userInfo dictionary remains grounded in using NSObjects as keys, so care must taken to cast String as an NSString when assembling.

In the main.swift implementation we listen for a MessageNotification:

For details on the MQTTMessage class, see the source, but we can gather from the above that two properties are of interest here:

  • topic:String – the topic to which the message was delivered
  • string:String? – the message contents, if any, delivered as a String

Get the Code

MQTTSub is an example implementation of an MQTT client that subscribes to our CPU temperature topic. Obtain it and compile with:

# git clone https://github.com/iachievedit/MQTTSub
# cd MQTTSub
# swift build
# .build/debug/MQTTSub

You will, of course, want to be running the publishing client as well:

# git clone https://github.com/iachievedit/PubSysTemp
# cd PubSysTemp
# swift build
# .build/debug/PubSysTemp

If your publishing client is running, you’ll see something like:

2016-06-13 02:55:43 +0000 - INFO    - Connecting to broker
2016-06-13 02:55:43 +0000 - INFO    - MQTT client has connected to broker.hivemq.com:1883
2016-06-13 02:55:43 +0000 - INFO    - Subscribe to topic
2016-06-13 02:55:43 +0000 - INFO    - didConnectAck
2016-06-13 02:55:43 +0000 - INFO    - didSubscribeTopic /darthvader/cpu/temperature/value
2016-06-13 02:55:44 +0000 - INFO    - Received 33.0 for topic /darthvader/cpu/temperature/value

If you want to have fun, try stress testing your system while running the publisher and subscriber. After stress -c 8 was started up:

PubSysTemp

2016-06-13 03:04:55 +0000 - INFO    - Published temperature to 34.0
2016-06-13 03:05:05 +0000 - INFO    - Published temperature to 37.0
2016-06-13 03:05:15 +0000 - INFO    - Published temperature to 42.0
2016-06-13 03:05:25 +0000 - INFO    - Published temperature to 45.0
2016-06-13 03:05:35 +0000 - INFO    - Published temperature to 47.0

MQTTSub

2016-06-13 03:04:55 +0000 - INFO    - Received 34.0 for topic /darthvader/cpu/temperature/value
2016-06-13 03:05:05 +0000 - INFO    - Received 37.0 for topic /darthvader/cpu/temperature/value
2016-06-13 03:05:15 +0000 - INFO    - Received 42.0 for topic /darthvader/cpu/temperature/value
2016-06-13 03:05:25 +0000 - INFO    - Received 45.0 for topic /darthvader/cpu/temperature/value
2016-06-13 03:05:35 +0000 - INFO    - Received 47.0 for topic /darthvader/cpu/temperature/value

What’s Next?

If you peruse the MQTT library you’ll notice it is lacking SSL support. That’s what’s next! In addition, we’ll be looking at writing additional MQTT tutorials that explore constructing topic hierarchies, using MQTT with real-world sensors, home automation devices, and more.

By

MQTT Clients With Swift on Linux

Swift 3.0 Swift 3.0

Over the past few years I’ve made a living managing Internet of Things (IoT) software development projects. In that time I’ve come to learn a number of protocols for communicating sensor and telemetry information back to “the cloud”. Among the most popular in the IoT space is MQTT, a lightweight protocol that allows for publishing messages to topics, as well as the ability to subscribe to topics. This model is frequently refer to as “pub-sub“.

In addition to working with IoT and MQTT, I am passionate about Swift and the inroads it is making into the server space since being open-sourced and ported to Linux. Naturally it made sense to combine the two areas and start working on an MQTT client implementation in Swift! We’ve taken the liberty to port an iOS implementation of an MQTT client over to Swift 3.0 on a Linux platform. In general this is an example that, indeed, Swift will be making inroads into both the server and IoT domains.

A disclaimer before we get started: the code below is based upon Swift 3.0 which is going through developer previews on Linux right now. To get started with Swift 3.0 on your Ubuntu 14.04 or 15.10 system go here. Or, if you have an armv7 device such as BeagleBone Black, try the ARM port of Swift 3.0!

Sample Application

My first idea was to put together a cool BeagleBone MQTT client that was reading one of the ADC inputs and sending it up to the broker. The input was going to be from a Microchip MCP9700 temperature sensor IC. The sensor IC maximum output voltage is 5.5V so I knew a voltage divider would be involved to keep the voltage input to the BeagleBone under 1.8V. I had the voltage divider all sketched out and everything!

Don't Want to Blow Up the BeagleBone!

Don’t Want to Blow Up the BeagleBone!

Unfortunately while testing the sensor IC with a standard match I got the flame a little too close to the IC package and well, Project Icarus came to an end. Our replacement example is a bit less ambitious, but serves the role of learning how to build an MQTT client with Swift. And it doesn’t involve matches.

MQTT

Our application is built around a port of a client MQTT library (yay open source!). We’ve named it simply MQTT and posted it to GitHub. The library is meant to be used in a Swift application that you can create as follows:

Running swift package init --type executable is going to give you a Swift project shell (think npm init) that you can begin customizing for your purposes. We’re going to edit Package.swift and add our MQTT library dependency:

MQTT Client Delegate

The design of the MQTT library is such that you will create a client class that inherits from MQTT and MQTTDelegate. A very basic implementation looks like this:

As you can probably guess the functions above are delegate methods that are called when the underlying MQTT client connects, publishes a message, subscribes to a topic, etc. It is up to you, the client writer, to fill out the implementation to suit your needs. In our case we’re going to implement the mqttDidDisconnect method as follows:

I’ve mentioned in previous posts that I appreciate the flexibility of posting a notification and having it acted upon by whoever was listening for it. DisconnectedNotification will be handled in our main.swift routine.

main.swift

Let’s turn our attention to main.swift which will instantiate an MQTT-based client. At a fundamental level here is all that’s required:

We want our client to be a bit more robust, so we’ll add in the ability to automatically reconnect if our connection is dropped by doing something like this:

Now, one could argue that we don’t want to exit if the connection to the broker fails. What we could do is set a timer and then rebroadcast our DisconnectedNotification. This will be our approach in the working code detailed further below.

We should publish something useful to the broker, so let’s set up an NSTimer to wake up every ten seconds and obtain the CPU temperature and then post that.

It should be noted here that we set the timer up and then fire() it immediately (so we’re not waiting ten seconds for that first post). Also, you will notice that we’re posting to the topic /<i>clientid</i>/cpu/temperature/value. This is one example of an MQTT topic naming convention and meant strictly as such. As you delve further into designing IoT applications it will become apparent that your naming convention is of considerable importance.

Getting the CPU Temperature

I love working with Linux and the wealth of information that is available in the /sys and /proc filesystems. Unfortunately when you’re dealing with hardware you have to frequently tailor your code to the specific hardware it’s running on. For example, on my x86 server the temperature of the CPU can be obtained by reading /sys/class/hwmon/hwmon0/temp1_input. On the BeagleBoard X15 it is at /sys/class/hwmon/hwmon1/temp1_input. That’s aggravating.

We won’t bother with writing portable code for now, but you should be able to take this example and suit it to your own system’s needs:

Putting it All Together

Let’s put everything together to build a working MQTT client that posts our CPU temperature to broker.hivemq.com. As a bonus, we’ll provide a web page that shows the CPU temperature displayed as a gauge.

We have three files that make up our client:

  • Client.swift
  • CPU.swift
  • main.swift

Each of these files should go in the Sources directory. Let’s look at a complete implementation of each:

Client

Client.swift

CPU

CPU.swift

main

Our main.swift appears to be complicated but it’s not really. The key things to recognize is that we wait for notifications and then set timers based upon those notifications to put our client into motion. For example, nothing good will come of trying to publish if we don’t have an MQTT connection established, so we wait for that notification (from our delegate) before attempting. Once a connection is established we set a 10 second timer and then update the temperature if we are still connected.

Note also the use of a heartbeat timer. Runloops will exit if there are no input sources or timers attached to it, so we use a simple repeating timer to keep a heartbeat (and thus, the runloop) going.

Your Package.swift file should look like:

To make life easier you can grab our code from GitHub. Build with swift build and then run it:

# git clone https://github.com/iachievedit/PubSysTemp
# cd PubSysTemp
# swift build
# .build/debug/PubSysTemp

Remember: this code is written in Swift 3.0! For more information see our post on obtaining Swift 3.0 for Linux.

What’s this Broker Thing Again?

Think of an MQTT broker as an NSNotificationCenter. In iOS one typically obtains a reference to the NSNotificationCenter.defaultCenter() and then posts messages to it. Likewise, to receive a message you register to be called when a named notification is posted (topic anyone?).

With MQTT you need to communicate with a broker. If you’re building an IoT gateway you might run your own broker using Mosquitto or HiveMQ. If you’re writing a tutorial on MQTT you might take advantage of a public broker such as test.mosquitto.org or broker.hivemq.com rather than running your own (like we did!).

In our example above we wrote an MQTT client that publishes data. What about subscribing to that data? This is also the purview of MQTT clients. For our example we’ll use a nice temperature gauge widget to power our temperature display.

Compiling Swift Heats Things Up

Compiling Swift Heats Things Up

What’s important to realize here is that the temperature gauge is actually an (JavaScript-based) MQTT client that has subscribed to receive events published to the /darthvader/cpu/temperature/value topic on broker.hivemq.com. (darthvader is our client’s hostname.)

What’s Next

There’s much more to be done with the MQTT library. As of June 11, 2016 it successfully connects and publishes, but subscribing to a topic is another story. That’s our next issue to tackle.

This is just the beginning, however, for Swift on the server. Organizations such as Zewo are working hard to develop libraries that can be used to develop server-side software on Linux with Swift. In fact, our MQTT library uses Zewo’s VeniceX TCP routines for network IO. Time will tell, but I’m convinced Swift has a bright future ahead of it in more than just iOS development.

Postscript: Please don’t comment on my lousy voltage divider. I nearly failed EE classes and do the best I can.

By

Swift 3.0 on a BeagleBone Black

Swift 3.0 Swift 3.0 Swift 3.0

There are a number of people working to bring Swift 3.0 to as many ARM-based systems as possible. I’m personally interested in the BeagleBoard family of devices and try Swift builds on a BeagleBone Black as soon as they are available. If you’re interested in joining the community of folks working on getting Swift 3.0 for ARM devices come join our community.

We’re assuming here you have a BeagleBone Black running Debian Jessie and have at least 1G of free space available. The base .tgz distribution is nearly 200M, and around 600M extracted. If you’ve booted from a microSD card and taking advantage of its capacity we can show you how to get the most out of it.

The team working on Swift for ARM is using Jenkins to build natively on a BeagleBoard X15. You can find the latest build for BeagleBone Black here. Grab and extract the contents into a directory and set your PATH with something like:

Now, let’s give it a spin!

Create a file called helloWorld.swift:

You can use swift helloWorld.swift to execute the file like a script:

# swift helloWorld.swift
Hello, world!

Or, you can compile the file into an executable with swiftc if you have clang installed and configured properly:

# swiftc helloWorld.swift
# ./helloWorld
Hello world!

If swiftc failed with error: link command failed with exit code 127 there’s a good chance you don’t have clang installed and configured properly:

sudo apt-get update
sudo apt-get install libicu-dev
sudo apt-get install clang-3.6
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.6 100
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.6 100

Let’s look at a few other tidbits:

Importing Glibc works!

swiftcat.swift:

Compile (swiftc swiftcat.swift) and run (swiftcat)!

Bridging to C routines

Linking against compiled object files works!

escapetext.c:

escapetext.h:

escapeswift.swift:

Compile and link everything together:

# clang -c escapetext.c
# swiftc -c escapeswift.swift -import-objc-header escapetext.h
# swiftc escapeswift.o escapetext.o -o escapeswift -lcurl

And run:

# ./escapeswift "foo > bar"
Escaped text:  foo%20%3E%20bar

Swift Package Manager

Unless you enjoy writing makefiles and build scripts (and trust me, some do), the Swift Package Manager is here to help manage software package dependencies. We’ll be writing more about the SwiftPM available in Swift 3.0, but are excited to provide a version that works on armv7 devices. Try this out:

# mkdir finalCountdown && cd finalCountdown
# swift package init --type executable
Creating executable package: finalCountdown
Creating Package.swift
Creating .gitignore
Creating Sources/
Creating Sources/main.swift
Creating Tests/

Replace the contents of Sources/main.swift with

Now, run swift build and your finalCountdown application:

# swift build
Compile Swift Module 'finalCountdown' (1 sources)
Linking .build/debug/finalCountdown
# .build/debug/finalCountdown
Entering thread
10...9...8...7...6...5...4...3...2...1...
Exiting thread
Done

moreswift

For a random smattering of Swift 3.0 applications that have been run on both x86 and armv7 systems, check out the moreswift swift-3.0 branch.

By

Swift for ARM Systems

There has been a lot of excitement over using Swift on ARM devices such as the Raspberry Pi and BeagleBone. Earlier this year a number of folks made tremendous progress in getting Swift 2.2 somewhat working on the Raspberry Pi 2.

As the future of Swift looks ever brighter and the language is beginning to attract more fans its clear that a renewed focus on building a functioning Swift 3.0 toolchain for ARMv7 systems would be appreciated by the community.

Now I’ll be clear: the guy to lead the technical effort isn’t me. What I can do though is help collect together all of the notes from those working on this, organize a team for collaboration, and in some small part provide a bit of hardware to help compile and test things.

What’s Being Built?

Swift 3.0 for armv7. Note that this excludes the original Raspberry Pi and any other system using an armv6 processor.

All of the repos currently being used to build are Apple’s, with one exception, and that’s llvm which uses the hpux735 repository arm branch. At the beginning of every build there is an information dump of what repositories make up the build, like this:

Repository,URL,Branch,Revision
swift,https://github.com/apple/swift.git,master,0c011c328c
llvm,https://github.com/hpux735/swift-llvm,arm,473f9137a9
clang,https://github.com/apple/swift-clang.git,stable,1e6cba3ce3
compiler-rt,https://github.com/apple/swift-compiler-rt.git,stable,a897309947
lldb,https://github.com/apple/swift-lldb.git,master,38dbc3b761
cmark,https://github.com/apple/swift-cmark.git,master,5af77f3c1d
llbuild,https://github.com/apple/swift-llbuild.git,master,437ce6ce0b
swiftpm,https://github.com/apple/swift-package-manager.git,master,2d8ded119d
swift-corelibs-xctest,https://github.com/apple/swift-corelibs-xctest.git,master,244507b687
swift-corelibs-foundation,https://github.com/apple/swift-corelibs-foundation.git,master,d2dc9f3cf9
swift-integration-tests,https://github.com/apple/swift-integration-tests.git,master,98f6c6d992
swift-corelibs-libdispatch,https://github.com/apple/swift-corelibs-libdispatch.git,master,65330e06d9

Build Status

I’ve put up a Jenkins CI server at swift-arm.ddns.net. There are two jobs at the moment:

The goal of the incremental builds is to not run a make clean prior to the build, as even with the X15 the compiles take quite some time (around 4 hours).

For now the build will be issued on demand at the direction of folks doing the development.

Issues Being Tackled

Editor’s note: This list is going to be fluid.

Get Involved

If you are interested in getting involved in helping the port of Swift to ARM devices, come join our Slack team.

By

Moving to Swift 3.0

Swift 3.0

Hey! Interested in an apt-get repository for Swift 3.0? We’ve got packages for Xenial, Wily, and Trusty.

Swift 3.0 is introducing a number of changes, the least of which is moving to conformance of the ratified API naming conventions. If you read through the naming conventions you’ll see that the overarching theme is one of streamlining and simplification.

Let’s take a look at a few examples of this streamlining with this 2.2 script swiftcat.swift:

Swift 3.0 changes the (count:repeatedValue) function signature for array initialization to (repeating:count:). So rather than var buf = [CChar](count:BUFSIZE, repeatedValue:CChar(0)) we will use var buf = [CChar](repeating:0, count:BUFSIZE) (note that we also simplified CChar(0) to just 0).

The fromCString String method has been removed in favor of an initializer in the form of init(cString:). As an initializer it also guarantees an object will be returned, so there’s no need for using the !: print (String(cString:buf), terminator:"")

And finally, CommandLine is used instead of Process. Our new Swift 3.0 version of swiftcat.swift:

Let’s take a look at another change in Swift 3.0 (which I appreciate), and that’s the named parameter is now required for the first argument to a function. For example, in Swift 2.2 the function func translate(text:String, from:String, to:String) would be invoked with something like translate("Hello world", from:"en", to:"es"). In Swift 3.0 the text parameter label is required: translate(text:"Hello world", from:"en", to:"es").

A few more examples of changes made:

Regardless of what you think about the wisdom of this String extension, this is functional Swift 2.2 code:

The Swift 3.0 complains that value of type 'Index' (aka 'String.CharacterView.Index') has no member 'advancedBy'. That’s because the function has changed from advancedBy(_:Index) to advanced(by:Index) (and somewhere buried in the API guidelines is a rationale as to the change).

Our next error is init(start:end:) is unavailable: use the '..<' operator. I must admit that the ..< operator looks weird (there, I said it), but okay, we’ll play along and update the code to return self[Range(startIndex..<endIndex)].

In-N-Out (Burger)

The placement of the inout keyword has also changed. In Swift 2.2 our function for emulating certain ncurses calls looked like this:

That resulted in error: 'inout' before a parameter name is not allowed, place it before the parameter type instead, so now we do this:

Great Renaming

As a part of the Great Renaming the NS prefix has been dropped from much of the Foundation classes. NSThread is now Thread, NSDate is now Date. You get the idea.

In addition to renaming classes, there has been a significant number of API changes centered around simplifying things. A great example of this simplification can be found in formatting dates and creating dates from strings. See our post on handling dates for more examples.

Notifications

Using NSNotificationCenter has also changed between Swift 2.x and Swift 3.0. As a part of the Great Renaming the notification center is now simply named NotificationCenter. For a complete writeup on notifications see our Notifications and userinfo with Swift 3.0 post.

But Wait!

Is that all there is to Swift 3.0 you ask? Of course not! The Swift Evolution repository has a complete list of all of the various changes. Have a look!

Get Started With Swift 3.0

There are a number of ways to start playing with Swift 3.0 today. Apple releases snapshot builds every so often on the official Download Swift page. Make sure and grab one of the Trunk Development (master) snapshots. The snapshots are in the form of .tar.gz files that can be extracted on your filesystem.

If you’re running Ubuntu 14.04 or Ubuntu 15.10 and like the convenience of installing software with apt-get, you can also try out our Swift 3.0 Ubuntu Packages.

Finally, get some practice. There’s some code in our moreswift Github repository that is all written in Swift 2.2. As an exercise download this code and try running with a Swift 3.0 compiler. There’s no telling what you’ll run into; if you find something that isn’t addressed here drop us a line so we can update the post.

By

NSNotifications with userInfo in Swift 2.2

Xcode 7.3 Swift 2.2

Looking for help using notifications with Swift 3.0? Head on over here where we go over the latest Swift 3.0 changes.

When I sit down to blog I never know what will be a popular topic and what won’t. One that definitely surprised me was how popular the original NSNotifications with userInfo in Swift would end up being. Much has changed since Swift first hit the scenes, so I thought we’d provide an update to handling NSNotification userInfo and some working code to help out.

The technique for obtaining the default NSNotificationCenter has remained unchanged, and can be done with let nc = NSNotificationCenter.defaultCenter(). What has changed in the latest versions of Swift is how to specify the selector (i.e., the function that should be called when a notification has posted). Rather than using a bare string like "catchNotification", Xcode will instruct you to use the #selector directive, like this:

In this example we’re instructing the notification center to deliver MyNotification notifications to the catchNotification function of the ViewController class.

Post It!

Now, let’s look at posting (sending) a notification:

The userInfo still takes [NSObject : AnyObject]? as an argument, which we provide as a dictionary literal in Swift. Note that the userInfo values don’t need to be homogeneous (that’s where the AnyObject comes in); we are sending along a String and an NSDate.

Handling Notifications

The guard construct did not exist in Swift when I wrote the original code used to pull apart the userInfo data, but it serves as a good method to unwrap and verify that the expected data is in the userInfo.

To verify that the guard works properly switch out the NSDate() in the call to postNotificationName with a String or some other object. You should see No userInfo found in notification printed to the console.

Example Source

You can try out the code above with a simple iOS project. Create a new Single View Application and replace the contents of ViewController.swift with the following:

By

Encrypting and Obscuring API Keys

Keys, or “credentials”, are a fact of life when working with REST APIs. Sign up for Amplitude for mobile analytics and you get a key to identify yourself and the application you’re working with. If you are using Aeris Weather APIs you get both an ID and secret. If you’re going to access these APIs in your iOS application you have to put those keys somewhere. In a previous post we walked through how to store API keys in property list files. In this tutorial we’re going to encrypt those keys and look at ways on how to access them in your app.

Before we get started, however, I want to make a few comments about security. I am not a security expert, and in all likelihood, neither are you. The original title of this article was Securing API Keys, but on reflection I renamed it to Obscuring API Keys. Nothing in this post will make the claim that this method of obscuring your API keys is impossible to break, because the reality is that anyone intent on stealing your API keys is going to do it. This method will only make it a little harder.

Encrypting Your Keys

To add another level of protection to our API keys, we’re going to use Blowfish Cipher Feedback encryption on them. To use this method of encryption you will want to choose:

  • a secret
  • an initialization vector

The secret must be at least 8 characters in length with a maximum of 56 characters. Choose whatever secret you like; a phrase, jumble of characters, it doesn’t matter. The initialization vector should be a random vector of 8 bytes. Once you’ve chosen these two pieces of data, download our Key Encrypter. Enter the secret and the IV, and then the API key you want to encrypt. For example, let’s say the API key we want to encrypt is 6bbb3654679105f33cf8c491ed9b04df and we’ve chosen dontusethissecret as our secret and decafbadbaddecaf. Enter these values and click Encrypt.

API Key Encrypter

API Key Encrypter

The resulting output is a hexadecimal string that can be fed back through a decryption routine, along, of course, with the original secret and initialization vector.

Note: It is usually at this point someone objects and says, “but your secret and IV will be in your code.” That is true, they will. But your secret can be a substring of a text label in your application, your copyright notice, etc. Foolproof? No. Harder to disassemble. Yes.

Decrypting Your Key

The encryption and decryption routines make use of the Blowfish implementation included in mbed TLS. The routines are written in C and we leverage a bridging header to expose these routines in Swift.

Our decryption function in Swift 4.2 looks like this:

The question, now, is how to use the decryption method and where to put the secret and initialization vector in your code. This is up to you (and there are a variety of ways). Just remember that no matter what you do, if someone is determined to figure it out and extract it, they will. This is just another step they have to go through after decrypting and decompiling the app from a jailbroken phone, etc.

To test out decryption of the generated string, download this example iOS app, open in Xcode, and run it. You should see the original API key 6bbb3654679105f33cf8c491ed9b04df.

Getting the Source

The source code for both the Mac encryption application and an example of how to decrypt an API key is available on Bitbucket:

Feel free to change out the encryption routines; Blowfish CFB is just one of many options to choose from.

Musings

To be honest, I posted this with some trepidation. Security is a topic that draws a lot of attention and frequently, well, developers with an attitude. How often have you heard the phrases:

  • Security through obscurity isn’t!
  • Never write your own cryptographic routine!
  • If you don’t know what you’re doing then don’t!
  • You’re just adding another level of indirection!

and so on. Yet at the same time we’re told to be more security conscious and program with it in mind. REST API keys for accessing services are provided to “everyday developers” and most documentation glosses over how to (more) securely embed them in your mobile app.

In the end, the author of DexGuard for Android sums up storing API keys on the mobile client nicely in a Stack Overflow post:

In the end, it’s an economic trade-off that you have to make: how important are the keys, how much time or software can you afford, how sophisticated are the hackers who are interested in the keys, how much time will they want to spend, how much worth is a delay before the keys are hacked, on what scale will any successful hackers distribute the keys, etc. Small pieces of information like keys are more difficult to protect than entire applications. Intrinsically, nothing on the client-side is unbreakable, but you can certainly raise the bar.” – Eric LaFortune

These are good guidelines to apply when determining what level of security you want to apply to your mobile applications. At some point you have to balance the likelihood of your keys being discovered against what you stand to lose if they do, and upon that decide how much time and effort you’re going to invest in obscuring them in your application.