iAchieved.it

Software Development Tips and Tricks

By

Updating From Such a Repository Can’t Be Done Securely

I recently came across the (incredibly frustrating) error message Updating from such a repository can't be done securely while trying to run apt-get update on an Ubuntu 18.04 LTS installation. Everything was working fine on Ubuntu 16.04.5. It turns out that newer version of apt (1.6.3) on Ubuntu 18.04.1 is stricter with regards to signed repositories than Ubuntu 16.04.5 (apt 1.2.27).

Here’s an example of the error while trying to communicate with the Wazuh repository:

Reading package lists... Done
E: Failed to fetch https://packages.wazuh.com/apt/dists/xenial/InRelease  403  Forbidden [IP: 13.35.78.27 443]
E: The repository 'https://packages.wazuh.com/apt xenial InRelease' is no longer signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

After searching around, we found that this issue has already been reported to the Wazuh project, but the solution of adding [trusted=yes] did not work for a repository that had already been added in /etc/apt. After continued searching, the following solution was finally hit upon:

deb [allow-insecure=yes allow-downgrade-to-insecure=yes] https://packages.wazuh.com/apt xenial main

That is, rather than using [trusted=yes] one can use [allow-insecure=yes allow-downgrade-to-insecure=yes]. Running apt-get update afterwards shows that the InRelease section is ignored, and Release is picked up:

Ign:7 https://packages.wazuh.com/apt xenial InRelease
Hit:8 https://packages.wazuh.com/apt xenial Release

Note that this is obviously a temporary solution, and should only be applied to a misbehaving repository! If you’re so inclined, upvote the Wazuh GitHub issue, as a fix at the repository level would be nice.

By

Ubuntu 18.04 on AWS

Ubuntu 18.04 Bionic Beaver was released several months ago now, and is currently (as of this writing) not available as a Quick Start AMI on AWS. But that’s okay, it is easy to create your own AMI based on 18.04. We’ll show you how!

Some assumptions, though. We’re going to assume you know your way around the AWS EC2 console, and have launched an instance or two in your time. If you haven’t, AWS itself has a Getting Started guide just for you.

Starting with 16.04

First, create an Ubuntu Server 16.04 EC2 instance in AWS with ami-0552e3455b9bc8d50, which is found under the Quick Start menu. A t2.micro instance is fine as we’re only going to be using it to build an 18.04 AMI.

Once the instance is available, ssh to it.

Notice that the OS is Ubuntu 16.04.5. We’re now going to upgrade it to 18.04.1 with do-release-upgrade. First, run sudo apt-get update, followed by sudo do-release-upgrade.

The upgrade script will detect that you are connected via an SSH session, and warn that performing an upgrade in such a manner is “risky.” We’ll take the risk and type y at the prompt.

This session appears to be running under ssh. It is not recommended
to perform a upgrade over ssh currently because in case of failure it
is harder to recover.

If you continue, an additional ssh daemon will be started at port
'1022'.
Do you want to continue?

Continue [yN]

You’ll get another warning about firewalls and iptables. Continue here as well!

To continue please press [ENTER]

Terrific, another warning! We’re about to do some seriously downloading, and hopefully it won’t take 6 hours.

You have to download a total of 173 M. This download will take about
21 minutes with a 1Mbit DSL connection and about 6 hours with a 56k
modem.

Fetching and installing the upgrade can take several hours. Once the
download has finished, the process cannot be canceled.

 Continue [yN]  Details [d]

Of course, press y to continue, and confirm that we also want to remove obselete packages.

Remove obsolete packages?


28 packages are going to be removed.

 Continue [yN]  Details [d]

At this point the installation and upgrade of packages should actually begin. There is a good chance that you’ll be interrupted with a couple screens requesting what version of GRUB and ssh configuration files you want to use. I typically keep the currently installed version of a configuration file, as it is likely I’ve made edits (through Ansible of course) to a given file. Rather than do diffs or merges at this point, I’ll wait until the upgrade is complete to review the files.

Once the upgrade is completed you’ll be prompted to reboot.

System upgrade is complete.

Restart required

To finish the upgrade, a restart is required.
If you select 'y' the system will be restarted.

Continue [yN]

After the reboot is completed, login (via ssh) and you should be greeted with

Welcome to Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-1020-aws x86_64)

Terrific! We have a pristine Ubuntu 18.04.1 LTS instance on Linux 4.15. We’re going to use this instance to make a template (AMI) from which to create more.

To start this process, stop the instance in the EC2 console. Once the instance is stopped, right-click on it and under the Image menu, select Create Image.

AWS will pop up a dialog indicating Create Image request received. with a link for viewing the pending image. Click on this link, and at this point you can name the AMI, as well as refer to it by its AMI ID.

Wait until the Status of the AMI is available before continuing!

Creating An 18.04.1 LTS Instance

Go back to the EC2 console and delete (terminate) the t2.micro instance we created, as it is no longer needed. Then, click Launch Instance and select My AMIs. You should see your new Ubuntu 18.04.1 LTS AMI. Select it and configure your instance (type, storage, security groups, etc.) and launch it!

Once your instance is available, ssh to it and see that you’ve just created an Ubuntu 18.04.1 Bionic Beaver server in AWS, and you have an AMI available to build as many as you like!

By

Hardening Your SSH Servers

DevOps ToolChain, WikiPedia, CC BY-SA 4.0

Why Hardening

Hardening, as I define it, is the process of applying best practices to make it harder for others to obtain unauthorized access to a given service or to the data transmitted by the service. In this post we’ll take a look at hardening SSH access to our server, as well as making it more difficult for others to potentially snoop our SSH traffic.

We’ll be using a fresh AWS EC2 instance running Ubuntu 16.04 for our examples. If you’re running a virtual server in Azure, Digital Ocean, or some other hosting provider, you’ll want to check out how the equivalent of AWS security groups are configured. And of course, these techniques can also be applied to non-virtual systems.

AWS Security Groups

The first step in hardening your SSH server is applying a more restrictive security group to your instance. Think of AWS security groups as custom firewalls you can apply to your instance. Even better, these custom firewalls can apply source-based filtering rules that only allow traffic from subnets or hosts you specify.

Subnet-based rules provides for rules like “Only allow SSH traffic from my development team’s network 10.90.4.0/24” If your internal network is segregated and configured such that developers must authenticate to receive a 10.90.4.0/24 IP address, an additional safeguard is added.

A host-based rule will only allow traffic from a given host (strictly speaking, a given IP address; if a host is behind a NAT then any hosts also behind that NAT will be allowed). This is what we’ll use.

private-ssh-sg

In the above example, we’ve created a security group private-ssh-sg and added a single Inbound rule that allows traffic on port 22 from a specific IP address. This will effectively only allow packets whose source IP is specified in that rule to reach port 22 of the instance.

SSH Cipher Strength

Another technique you can use to harden your SSH server is ensuring that the latest strong key exchange protocols, ciphers, and message authentication code (MAC) algorithms are utilized.

We’ve used several references as a guide to hardening SSH, including Mozilla’s OpenSSH Guidelines as well as ssh-audit, a nice tool designed specifically for this task. Using ssh-audit is as easy as

$ git clone https://github.com/arthepsy/ssh-audit
$ cd ssh-audit
$ ./ssh-audit.py your.ip.address.here

sshaudit

Our first pass uncovers a number of issues:

  • use of weak elliptic curves in the key exchange algorithms supported
  • use of weak elliptic curves in the host-key algorithms supported

ssh-audit goes on to recommend the key exchange, host key, and MAC algorithms to remove.

Let’s look at changes we want to make to our /etc/ssh/sshd_config file:

Protocol 2
HostKey /etc/ssh/ssh_host_ed25519_key

KexAlgorithms curve25519-sha256@libssh.org
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
MACs hmac-sha2-512-etm@openssh.com,hmac-sha2-256-etm@openssh.com,umac-128-etm@openssh.com

Restart your SSH daemon with sudo systemctl restart ssh. Note: It’s a good idea to have a second terminal logged in if you bork your SSH configuration and lock yourself out of your instance.

Once we’ve updated our sshd_config configuration, it’s time to run an audit against it.

Shell

Nice! Strong key exchange, encryption, and MAC algorithms all around.

Two-Factor Authentication

There are different interpretations as to what constitutes two-factor or multi-factor authentication. Many believe that two-factor authentication implies an additional authentication code delivered via text message or provided by a key fob. Others may consider the steps taken to obtain access to a given computing resource as a part of the authentication steps (e.g., to obtain access to a given server you must get past the security guard, provide a retinal scan, and so on). In this example, we’re going to use the former interpretation.

Authy

We’ve chosen to use Authy in this example for two-factor authentication using a time-based one time password. To get started, install the Authy application on your phone (iOS or Android) and follow the quick-start prompts.

After you’ve successfully set up the application on your phone, you can download the app to your desktop or add it to Google Chrome.

Getting Your EC2 Instance Ready

To use the authentication code provided by Authy to add an additional authentication step for SSH logins requires installing the libpam-google-authenticator module and configuring both SSH and PAM.

Install the module with sudo apt-get install libpam-google-authenticator.

Now, as a user that needs to use two-factor authentication, run google-authenticator to get set up. The application will generate several prompts, the first of which is Do you want authentication tokens to be time-based to which you’ll answer “yes”

google-authenticator will then generate QR-code that you can scan with the Authy phone application, as well as a secret key that can be used with the phone, desktop, or browser application. When using the desktop application I prefer just copy-paste of the secret key.

There are additional prompts from the application to follow:

Do you want me to update your "/home/ubuntu/.google_authenticator" file (y/n) y

Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) y

By default, tokens are good for 30 seconds and in order to compensate for
possible time-skew between the client and the server, we allow an extra
token before and after the current time. If you experience problems with poor
time synchronization, you can increase the window from its default
size of 1:30min to about 4min. Do you want to do so (y/n) n

If the computer that you are logging into isn't hardened against brute-force
login attempts, you can enable rate-limiting for the authentication module.
By default, this limits attackers to no more than 3 login attempts every 30s.
Do you want to enable rate-limiting (y/n) y

NB: It is a good idea to save your emergency scratch codes in the event you lose access to the devices that are generating your OTPs.

Now that you’ve configured the authenticator, it’s time to update sshd_config to consult the PAM Google Authenticator module when a user attempts to log in.

Open /etc/ssh/sshd_config as root, and set the following:

ChallengeResponseAuthentication yes
PasswordAuthentication no
AuthenticationMethods publickey,keyboard-interactive

Restart ssh with sudo systemctl restart ssh.

Now, in /etc/pam.d/sshd replace the line @include common-auth with auth required pam_google_authenticator.so.

Once the sshd PAM module has been configured in this manner users will be challenged for a two-factor authentication code, so it’s important that every user on the system be configured with the google-authenticator application.

Test your login!

Try logging in via ssh with the user you’ve just configured for two-factor authentication. You should receive a prompt requesting a verification code (after your key is authenticated).

enter_code

Enter the code displayed on your Authy app (note that all of your Authy apps will display the same code for the same application configuration) to login.

code_entered

One Last Thought on Authy

While writing this post I was doing additional research on two-factor authentication implementations; while Authy supports time-based one time passwords, it supports additional methods that require access to their infrastructure. If you don’t care for providing your cellphone number (and a lot of people don’t), try out Authenticator, a Chrome plugin that doesn’t require any account setup.

You’ll notice if you try out different applications that you can use the same secret key and each application will generate the same code at the same time, hence the time-based one time password.

Recap

There is no such thing as perfect security save turning off the computer, disconnecting all of its cables, putting it in a trunk, filling that with cement, and tossing it into the Pacific. Even that might not be perfect. In the absence of perfection we can put as many barriers in place between our server and others that shouldn’t have access to it. In this post we’ve taken a look at several of those barriers:

  • source-based firewall rules that only allow access on port 22 from a specific IP or subnet
  • hardened key exchange algorithms, ciphers, and MACs for SSH
  • two-factor authentication that requires both public key authentication as well as an OTP code

We did not cover additional techniques such as configuring SSH to listen on a different port; I’ve found that despite explaining that this is done primarily to minimize port-scanning chatter (who wants to sift through auth or fail2ban logs with script kiddie traffic) it never fails to incite the crowd of folks who just learned the phrase security through obscurity to gather up their pitchforks.

If you have any additional recommendations regarding SSH security hardening, please leave a comment!

By

MQTT Last Will and Testament

This is another post in a series on writing MQTT clients using Swift on Linux.

In this post we’ll look at the MQTT Last Will and Testament message. Essentially the LWT message is predefined by a client connecting to the broker. In the event the client abnormally disconnects, the broker will then broadcast the LWT to any subscribers of the LWT message topic.

For example, let’s say you are building a chat application where each client connects to a broker and then subscribes to the topic /chat/hottub. Messages then published to /chat/hottub are received by all of the subscribers. A simple but effective way to join a “chat room” (i.e., MQTT topic).

When a client leaves the chat under normal conditions we might expect a message like “Joe has left /chat/hottub.” This would be easy to do; when the user types /exit then publish an appropriate message and then terminate the client. If a client abnormally disconnects (loss of network, client crash, etc.), then what was given to the broker as the Last Will and Testament message is used instead.

Here’s how we set up the Last Will message for our MQTT client:

willMessage is a MQTTWill property of the MQTT class. MQTTWill is constructed with a topic and message. In our example here the topic will be our chat channel /chat/hottub, and the message will be a JSON string that contains our client’s ID and a simple Abnormal Disconnect string.

Get the Code

We’ve been building Swift 3.0 directly from the latest in the Apple repositories and continually upgrade our code. To use these examples you’ll need to install Swift 3.0 from our apt-get repository.

An example of our MQTT HotTub is in GitHub.

# git clone https://github.com/iachievedit/MQTTHotTub
# cd MQTTHotTub
# swift build

To test things out you’ll want to run the MQTTHotTub twice, so open a second terminal.

1__clear_____build_debug_MQTTHotTub__ssh__and_9__clear_____build_debug_MQTTHotTub__ssh_

Run the clients with .build/debug/MQTTHotTub.

Prisencolinensinainciusol

Our MQTTHotTub client simulates a chat of gibberish. Each published message is delivered as JSON:

{"client":"octxktfo", "message":"Gyxswhz nsoxfnj gz."}
{"client":"ajyhyjic", "message":"Cmr w bzwubzv mwfhtklz."}

When a client receives a message that it did not send it gets filtered out:

Remember, if we are listening to messages published to a given topic, and we publish to that topic, the client will also receive that message back as an echo. This is the purpose of filtering with cid != clientId.

You can see each client logging what they are receiving from other clients:

Received "Wlfu zrqyj tady obxnjl lupihobi nph oapplt nyidmja." from octxktfo
Received "Cmr w bzwubzv mwfhtklz." from ajyhyjic

Now, CTRL-C one of the clients, and notice what is received by the remaining client(s):

Received "Abnormal Disconnect" from octxktfo
Abnormal Disconnect

Abnormal Disconnect

This is the MQTT Last Will and Testament message in action; the aborted client had no opportunity to broadcast that it was unavailable so instead the broker sent out the client’s Last Will on the /chat/hottub topic:

{"client":"\(clientId)","message":"Abnormal Disconnect"}

Do I Need to Have a Last Will?

Need is a strong word, and the answer is no, your MQTT client does not need to supply a Last Will message. There are plenty of additional examples that provide guidance on when you might want to use one.

What’s Next?

We’re continuing to work hard on our MQTT implementation for Swift on Linux. With Last Will and Testament now working our attention will turn to implementing secure MQTT connections (MQTT SSL). Stay tuned!

By

Upgrading CMake for a Happier Swift Build

There have been a number of updates to Open Source Swift that take advantage of newer versions of CMake to build swift on Linux. In particular, the default version of cmake that comes with Ubuntu 14.04 (2.8.12.2) is no longer up to the task.

Let’s get our Ubuntu 14.04 environment updated with CMake 3.4.3, the version proposed by the developers that work in this area.

On systems I’m installing software by source, I usually have a build area in /usr/local/src and an area of archives (so I can keep track of the versions I built) in /usr/local/archive:

As root or using sudo:

# cd /usr/local/archive
# wget https://cmake.org/files/v3.4/cmake-3.4.3.tar.gz
# cd ../src/
# tar -xzvf ../archive/cmake-3.4.3.tar.gz

Now, to configure and build:

# cd cmake-3.4.3
# ./configure --prefix=/usr/local
...
CMake has bootstrapped.  Now run make.
# make

Finally, make install will install cmake and its associated configuration to /usr/local.

# make install
# which cmake
# cmake --version
cmake version 3.4.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).

That’s all it takes. Happy building!

By

Handling MQTT Subscriptions with Swift on Linux

In our last post we looked at a Swift MQTT client which published information to a HiveMQ broker. Now let’s turn our attention to writing an MQTT client that subscribes to a topic.

Again, we’ll be using MQTT, code that aims to provide a solid MQTT client library implementation in Swift. If you haven’t yet read the tutorial on building an MQTT publisher with it, I suggest you do so before proceeding!

There are two basic components to writing an MQTT client that subscribes to a topic:

  • subscribing to a topic
  • handling a published message from the broker

Ignoring the boilerplate connection logic for a moment, let’s look at subscribing:

There’s nothing earth shattering here: upon receiving a ConnectedNotification a call is made to subscribe to the topic /hostname/cpu/temperature/value (where hostname is obtained from the OS). Now when the broker receives a message published to this topic it will broadcast it to our client. This is the pub-sub pattern in action.

This Transmission is Coming To You

To receive an MQTT message we need to handle the MQTTDelegate method func mqtt(mqtt: MQTT, didReceiveMessage message: MQTTMessage, id: UInt16 ). This can be readily accomplished using an NSNotification with accompanying userInfo:

Unfortunately for us much the userInfo dictionary remains grounded in using NSObjects as keys, so care must taken to cast String as an NSString when assembling.

In the main.swift implementation we listen for a MessageNotification:

For details on the MQTTMessage class, see the source, but we can gather from the above that two properties are of interest here:

  • topic:String – the topic to which the message was delivered
  • string:String? – the message contents, if any, delivered as a String

Get the Code

MQTTSub is an example implementation of an MQTT client that subscribes to our CPU temperature topic. Obtain it and compile with:

# git clone https://github.com/iachievedit/MQTTSub
# cd MQTTSub
# swift build
# .build/debug/MQTTSub

You will, of course, want to be running the publishing client as well:

# git clone https://github.com/iachievedit/PubSysTemp
# cd PubSysTemp
# swift build
# .build/debug/PubSysTemp

If your publishing client is running, you’ll see something like:

2016-06-13 02:55:43 +0000 - INFO    - Connecting to broker
2016-06-13 02:55:43 +0000 - INFO    - MQTT client has connected to broker.hivemq.com:1883
2016-06-13 02:55:43 +0000 - INFO    - Subscribe to topic
2016-06-13 02:55:43 +0000 - INFO    - didConnectAck
2016-06-13 02:55:43 +0000 - INFO    - didSubscribeTopic /darthvader/cpu/temperature/value
2016-06-13 02:55:44 +0000 - INFO    - Received 33.0 for topic /darthvader/cpu/temperature/value

If you want to have fun, try stress testing your system while running the publisher and subscriber. After stress -c 8 was started up:

PubSysTemp

2016-06-13 03:04:55 +0000 - INFO    - Published temperature to 34.0
2016-06-13 03:05:05 +0000 - INFO    - Published temperature to 37.0
2016-06-13 03:05:15 +0000 - INFO    - Published temperature to 42.0
2016-06-13 03:05:25 +0000 - INFO    - Published temperature to 45.0
2016-06-13 03:05:35 +0000 - INFO    - Published temperature to 47.0

MQTTSub

2016-06-13 03:04:55 +0000 - INFO    - Received 34.0 for topic /darthvader/cpu/temperature/value
2016-06-13 03:05:05 +0000 - INFO    - Received 37.0 for topic /darthvader/cpu/temperature/value
2016-06-13 03:05:15 +0000 - INFO    - Received 42.0 for topic /darthvader/cpu/temperature/value
2016-06-13 03:05:25 +0000 - INFO    - Received 45.0 for topic /darthvader/cpu/temperature/value
2016-06-13 03:05:35 +0000 - INFO    - Received 47.0 for topic /darthvader/cpu/temperature/value

What’s Next?

If you peruse the MQTT library you’ll notice it is lacking SSL support. That’s what’s next! In addition, we’ll be looking at writing additional MQTT tutorials that explore constructing topic hierarchies, using MQTT with real-world sensors, home automation devices, and more.

By

Creating New Packages with SwiftPM

Swift 2.2 ISC

Back in December I opened an enhancement request in the Swift JIRA site, issue SR-353, to add an npm init-style invocation to swift build for creating all of the boilerplate files needed for a complete Swift package. It wasn’t too long after the request was added that Bastian Rinsche and Tobias Landsberg implemented the feature and had a pull request accepted. Bastian has written up a great post on his experience with opensource Swift and contributing to the community with the implementation of swift build --init.

Let’s look at how it works. First, create a new directory with mkdir for your package or application. We’ll use helloworld as an example: mkdir helloworld. Now, cd into your helloworld directory and run swift build --init.

# cd helloworld
# swift build --init
Creating Package.swift
Creating .gitignore
Creating Sources/
Creating Sources/main.swift
Creating Tests/

Let’s take a look at what swift build --init generates:

  • Package.swift – the “manifest” for this package; swift build --init names your package the same as the directory you invoked it in
  • Sources – SwiftPM convention is that source files are included in a directory called Sources
  • main.swift – Swift applications will have one (and only one) main.swift file as the execution entry point
  • Tests – a directory to hold testcases for the application; more on this in a moment
  • .gitignore – a handy Git gitignore starter file that ignores build artifacts such as .build and Packages (things you don’t want to check in to a repository)

In our helloworld example Package.swift looks like this:

Running swift build at this point will create .build/debug/helloworld which when executed you will see Hello, world!.

If you are building a Swift-based library then remove main.swift and replace it with your library code. Without a main.swift SwiftPM will create a static library archive.

Tests

The Tests directory added by swift build --init is a nod to the future implementation of SR-592, which tracks SE-0019, Swift Package Manager support for automated testing.

Getting the Ubuntu Package

To get started with swift build --init you can grab the latest Ubuntu package for Swift 2.2 (as of this writing, version 2.2-0ubuntu15). See this post for instructions. For more details about using SwiftPM in general, see our introduction here. Happy Swifting!

By

Another Look at Swift Logging

Swift 3.0 ISC

Shortly after Apple released Swift a number of folks provided logging libraries, as the venerable Lumberjack didn’t work out of the box. We even provided a little write-up on using swiftlog, a collection of simple logging routines.

Well, we’re back with an update to swiftlog that focuses on providing a quick way to add log messages to Swift applications on Linux systems. The new features include:

  • SwiftPM-ready
  • color-coded log messages using the awesome Rainbow package
  • ability to write log messages to a file

As with all incarnations of swiftlog, we are not looking to develop the Cadillac of logging frameworks. swiftlog is a very simple and no frills package.

Using swiftlog

To use the new swiftlog just add it to your Package.swift dependencies:

You can begin using the logging facility by importing swiftlog and setting the sLogLevel global variable (it defaults to .None which turns logging off).

All logs equal to or higher in severity are printed. Logs are color-coded as well:

Colored Logs!

Colored Logs!

It doesn’t take a rocket surgeon to determine the options available for sLogLevel, but for completeness here they are:

  • .None
  • .Verbose
  • .Info
  • .Warning
  • .Error

Two additional routines are provided: ENTRY_LOG and EXIT_LOG. These “macros” are designed for tracing function entry and exits in your logs:

This results in three additional log messages:

VERBOSE - ENTRY multiply(_:multiplier:)
VERBOSE - EXIT  multiply(_:multiplier:)
VERBOSE - 10 times 10 equals 100

Finally, we can instruct that logs also be written to a file with slogToFile(atPath:).

The append parameter of slogToFile(atPath:) can be set to false to overwrite the previous contents of the file.

The Code

You can grab the code on Github. If you don’t like the colors I chose for the log levels, hack away! Perhaps a new revision will provide for more customization and options; I’ve deliberately chosen thus far to make this package very simple to use and reduced the number of knobs.

There is an example in the logexample directory of the repository. Enter logexample and type swift build to build and run!

By

Raspberry Pi 2 GPIO with SwiftyGPIO

Working with open source developers worldwide is a fun and rewarding experience. As the cost of computing devices and broadband Internet continues to fall and bring new technologies to people around the globe, developers of all stripes from different cultures and backgrounds come together to collaborate and build awesome things.

Since Apple open sourced the Swift programming language late last year enthusiasts have created Ubuntu packages, ported it to ARM devices such as the Raspberry Pi 2, built web development frameworks, and now Umberto Raimondi has released SwiftyGPIO, a Swift library for interacting with GPIO pins on ARM devices such as the Raspberry Pi and BeagleBone Black.

The SwiftyGPIO README covers all the bases explaining how to use the module. As Umberto notes the Swift Package Manager is currently unavailable in the ARM builds (I’ve been working on getting it to compile but something always preempts me) so we turn to downloading the SwiftyGPIO.swift file with wget and using swiftc to compile and link everything together.

Rock Chalk

Last year I was working with Arduino programming with Xcode and wrote up some Arduino code for blinking LEDs. Let’s do the same with a Raspberry Pi 2 and Swift.

If you’re coming into this cold you’re going to need:

We’re going to use GPIO4 and GPIO27 since they are close to each other on the Pi 2 GPIO header.

Here is our main.swift to blink the two lights back and forth:

To compile and run this code, until SwiftPM for ARM is fixed, we do this:

# wget https://raw.githubusercontent.com/uraimo/SwiftyGPIO/master/Sources/SwiftyGPIO.swift
# swiftc main.swift SwiftyGPIO.swift
# ./main

If you wired your LEDs up properly they should be flashing back and forth!

Blinky Blinky

Blinky Blinky

Pick Your Color

I had a Linrose Tricolor LED lying around so I decided to put it to good use with Swift. In this code example we’ve written a command line application that lets you set the color of the LED (or to turn it off). I’ve marked the code with // 1, // 2 to describe each section.

1. SwiftyGPIO provides canned GPIO definitions for popular board types. In our case we’re using a Raspberry Pi 2.

2. This is purely syntactic sugar to describe GPIO states as On or Off. The code would probably look less cluttered if we removed this.

3. LedColor is a structure to “namespace” definitions for Off, Green, Orange, and Red.

4. The tricolor LED has two anode pins; we will attach one pin to GPIO4 and the other to GPIO27. The application always starts by setting the pin direction to .OUT and Off. Again, because of the way we created an enum for the GPIOState we have to use the .rawValue nonsense.

5. setLedColor takes a tuple of (GPIOState,GPIOState) and [GPIO] array to set the pair of GPIO pins to a certain state.

6. Our application takes a single argument so we guard that we have two (one is the application name). Our color is the second.

7. A switch on the color and a call is made to setLedColor with the appropriate color tuple and GPIO set.

Closing Remarks

SwiftyGPIO is a great API to get started with GPIO manipulation on ARM boards with Swift. Each passing day Swift makes inroads into the maker community and stands a great chance to be the language of choice for single board computer development projects.

By

Ncurses with Swift on Linux

Swift 2.2 ISC Linux

Ncurses is a toolkit for developing “GUI-like” application software that runs under a terminal emulator (Wikipedia). If you aren’t familiar with ncurses, there are a number of resources for learning about its history and capabilities. Here are some of my favorites:

This tutorial will walk you through how to create ncurses applications on Linux with Apple’s Swift programming language. It is surprisingly easy with most of the complexity coming from learning ncurses itself.

Using With Swift

To use ncurses in Swift applications you can add a Package dependency in your application’s Package.swift file like so:

See our Swift Package Manager tutorial for more information on how this dependency works; it is a very simple modulemap around the ncurses header and library.

ncurses primitives

Ncurses is a “true” C API. Each API call is a C function with set of arguments (or what looks like a function call; many ncurses routines are in fact, C macros). Internal library structures and state variables keep track of the state of the screen.

This isn’t a tutorial on ncurses itself (see this for one), but we will show how to perform basic routines and point out areas we cannot use in Swift (and how to get around it!)

Let’s look at three functions we call to set up the screen:

initscr(): initscr is typically the first curses routine to call when initializing a program.
noecho(): noecho turns echoing of characters typed off. One might think character echoing would be desirable, but in most cases if we are dealing with receiving input from the terminal we want a fine control over what is displayed and what isn’t.
curs_set(int): curs_set controls the visual representation of the cursor. The integer value can be 0, 1, or 2 for invisible cursor, normal cursor, or prominent cursor. If you are painting an ASCII art masterpiece the last thing you want left on the screen in a chunky cursor.

Now that our screen is initialized we can use the several ncurses primitives to detect the size of the screen and where the cursor is currently located.

If you Google ncurses get size of screen it will invariably lead you to the “function” getmaxyx, which is actually a macro, not a function. That is an issue, because not only is getmaxyx a macro, it is what Swift defines as a complex macro. From the Swift Programming Language reference: Complex macros are macros that do not define constants, including parenthesized, function-like macros. The reference manual continues: complex macros that are in C and Objective-C source files are not made available to your Swift code. (emphasis mine).

So what’s a developer to do if they want to use getmaxyx? Turn to what the definition of the macro is in /usr/include/curses.h which is:

Using this definition we can use getmaxy and getmaxx together:

stdscr is a global ncurses variable of type WINDOW* and is a handle to the main window.

Alternatively we can create an analogous Swift function to the macro like this:

In this case use of the function would appear as:

Another common macro to use with ncurses is getcuryx which gets the current position of the cursor. Again, we either use getcurx and getcury together, or write a getcuryx function:

Now that we know how to determine the dimensions of our terminal screen and where the cursor is currently located, let’s look at how to move the cursor around, and more importantly, write text to the screen.

ncurses coordinates are frequently given in (y,x) format where y is the line number, and x is the column number. The upper-leftmost coordinate is (0,0). On a “standard” terminal screen defaulted to 24 lines and 80 columns, the lower-rightmost coordinate would be (23,79). Let’s look at writing UL (upper left) and LR (lower right) to the screen in their expected positions.

move is used to place the cursor at a given set of coordinates. Writing UL in the upper-left we’ll use move(0, 0) followed by addstr to add a string.

Note: refresh is used to tell ncurses to update what is visible on the screen. If you move the cursor and addstr without a refresh you will not see a visible change. Get used to calling refresh!

To place LR in the lower-right we want to move the cursor to (23,78) and use addstr("LR").

Our entire application would look like this (make sure and add the signal handling routine unless you don’t mind your terminal window getting trashed when exiting the application):

Write an appropriate Package.swift and place the above code in a main.swift and then build it with swift build.

Try this: Add the strings “LL” and “UR” in their appropriate location on the screen. Add the text “CENTER” centered in the window. Scroll down for a hint on the centerText routine.

One Character at a Time

addch is another primitive that provides us with the capability to add a single character to the screen. The function prototype is addch(const chtype ch) where chtype is a typedef in /usr/include/curses.h, typically of an unsigned. With Swift on Linux the function requires a UInt, so to add a character with addch one would use addch(UInt("*")).

In this quick example we use addch to draw a box using * characters around the screen followed by using addstr to add “Hello world!” in the middle. Try resizing the terminal window. select exits and the application ends. We’ll handle resizing further down.

Handling SIGWINCH

Modern terminals can change size. In the days of VT100s they didn’t, at least not dynamically; the original DEC manual indicates the VT100 had 24 lines by 80 characters or you could switch it to 14 lines by 132 characters. Of course today’s Linux terminal sessions usually start off with 24 lines and 80 columns unless you’ve customized it, but once they start up you can resize them to your heart’s content.

We want our ncurses applications to respond to changes in the size of the window. For example, if our window has a border drawn around it with a drawbox routine we will have to redraw that border if the screen size changes.

Your application process is notified of window size changes via the SIGWINCH signal.

SIGWINCH – Window size change. This is generated on some systems (including GNU) when the terminal driver’s record of the number of rows and columns on the screen is changed. The default action is to ignore it. If a program does full-screen display, it should handle SIGWINCH. When the signal arrives, it should fetch the new screen size and reformat its display accordingly. –GNU Documentation

We’re going to use the mechanism defined in our previous post to handle the signal.

When receiving a SIGWINCH it is very important to run endwin(), refresh(), initscr(), and clear() in that order to “flush” the screen and get it ready for redrawing. Once this sequence is executed we then get our new max (y,x) and redraw our box. For good measure Hello world! is centered in the box.

One last thing on handling signals: you will always want to handle SIGINT and ensure that a call to endwin() is made prior to exiting the application. Failing to do so will leave the terminal you return to (i.e., your shell) in a screwed up state. Try it; leave out the SIGINT handler below and hit CTRL-C to end the application.

main.swift for drawing a box that dynamically refreshes after resizing the terminal now looks like this:

Getting Input with getch

Handling input from a terminal that is under ncurses control is a bit of a pain. Once you see what’s going on it’s not too bad but the first time through is annoying. Let’s take a look at a routine that can intercept input characters, display them on the screen, handle backspaces, and then pass back the contents of the input when the user hits return. I’ve annotated the code with // 1, // 2, etc. to highlight what’s going on.

Note: This code is lifted from a class and is not intended to be copy-paste. The full class is given below.

1. Our input string is a buffer. It’s declared as a static class variable and is used in the getInput routine and a separate redrawInput routine which we’ll cover later. delim and backspace are constants for newline and the DEL character, respectively.

2. getInput returns a string. It is a class method due to the way we interact with ncurses itself (in particular, calling back into an object with a C callback causes issues when handling signals).

3. Clear the input buffer.

4. Move the cursor to the inputLine which is defined as “the last line” in the terminal window. curx will be 0 to start off the routine.

5. Swift is “funny” (nay, advanced) with what it defines as a character (a UnicodeScalar). Ncurses is old school; character routines expect 32-bit integers. Because some character handling routines expect Int32, and some expect UInt32, and our String object wants Character objects to be added, we sort of keep multiple representations of a character hanging around. ic is a UInt32 representation, c is a Character representation.

6. Our switch statement handles what to do with a character when the user types it. If it’s a backspace we ensure that we aren’t backing up over the beginning of the line (the guard statement), and if not, move back one column, delete the character that was just typed, refresh the screen, and then update our input String by using input.characters.dropLast() to get rid of the last character typed.

7. When delim is encountered (Enter/Return key hit), the input line on the screen is cleared out and the input buffer is returned.

8. If the character typed is neither a backspace or a newline it is evaluated as to whether it is printable, and if so addch is called to put the character on the screen. Our cursor is advanced (curx += 1), the screen refreshed, and the character c added to our input buffer. Try taking out the check for isprint and then scroll your mouse wheel. Hilarity ensues.

That in a nutshell is our routine to collect characters into a buffer and display on the screen! We will use this in our next example.

Our Translator App Updated

Starting with this post and continuing with Command Line Utilities in Swift we’ve been building upon an example application that uses a public REST API to translate text from one language to another. We’ve updated the application here to utilize an ncurses interface.

Our first update is to the CommandInterpreter.swift class in that we take out all of the IO as it will now be handled by a new singleton-style class CursesInterface.swift. Here is the revamped CommandInterpreter.swift:

If you compare this routine to our previous versions there is no display of the prompt, no collecting characters, and our .Quit command now doesn’t call exit(0) but rather posts a QUIT_NOTIFICATION.

The main.swift file of the translator has also changed. Again, we will mark up the code with // 1, // 2:

1. Like the previous translator applications we still have a command interpreter and a translator.

2. We register an observer for the INPUT_NOTIFICATION NSNotification. When the trailing closure is called upon receipt of the event it will take the translationCommand and feed it to the translator routine which, in turn, will call the translation REST API service. You can see here our use of class methods of CursesInterface to update the status bar that we are translating as well as post the translation text.

3. It is very important to not simply call exit(0) anywhere lest we want a wonky terminal window left in our wake. Rather than exit out immediately upon the user typing /quit a notification is posted which is handled by CursesInterface.end() (a wrapper around the ncurses endwin() followed by exit(0)). To be prudent we should also catch a lot more signals than SIGINT in the CursesInterface class.

4. The workhorse routine. Start the interface and then enter a while true loop which, on each turn:

  • updates the prompt (which is always from->to)
  • displays the status bar
  • collects input
  • parses the input
  • executes the result of the input

Finally, we get to CursesInterface.swift. Here is where input and output to the application is handled. Nothing gets drawn on the screen elsewhere in the application except through here. CursesInterface is a sort of a singleton-style class, except that rather creating a singleton instance we leverage class methods. I explicitly chose this approach to handle catching the SIGWINCH signal and call back into the class methods in the trailing closure. Think of CursesInterface as a namespace for global state variables and routines that act on those variables. A bit like ncurses itself.

Everything in the above code with the exception of attron(), attroff(), and clrtoeol() has already been covered in some fashion. The attr functions allow for the application of attributes to the character cell. In our case we turn on reverse video to provide a “status bar”. The value of A_REVERSE is provided by ncurses by means of macros, so again, we had to reverse-engineer /usr/include/curses.h to determine it. It’s an exercise to the reader to reason out why it is Int32(1 << 18).

Putting together all of our classes we get a nice UI for translating strings!

Ncurses Translator

Ncurses Translator

Don’t worry about typing or pasting all that code in, links to everything are provided below!

Some Restrictions Apply

Unfortunately with Swift on Linux we cannot access C variadic functions, which means we cannot use, directly, many of the ncurses routines. This includes routines such as:

  • printw
  • wprintw
  • mvprintw

As with the macros that aren’t available, one can work around by providing non-variadic counterparts. As an example, mvprintw has the signature int mvprintw(int y, int x, char *fmt, ...); This can be rewritten as:

and used like this: mvprintw(y:inputLine, x:0, str:input)

Since the signature of the function is different from that of the imported C mvprintw there is no conflict and we can use our Swift version easily.

Getting the Code

We’ve covered a lot of ground in this post and there are a lot of snippets of code to reason through, so we’ve posted all of the examples in our moreswift repository on Github.

git clone https://github.com/iachievedit/moreswift

Take a look at these directories:

  • ncurses_basic – sets up a screen and writes UL and LR in the upper-left and lower-right of a 24×80 terminal. Update this app to detect the actual screen size!
  • ncurses_box_simple – draws a box around the screen with Hello world! written to the center. Resizing the terminal ends the app, fix that!
  • ncurses_box – a working example of being able to resize the box and recenter the text
  • ncurses_translator – Our translator application, complete with handling resizing of the screen, except when the screen is resized any translations already provided are lost. Fix that!

Each directory can be built with swift build. Have fun!

Postscript

If you’re enjoying reading these Swift on Linux tutorials, follow us on Twitter at @iachievedit. This blog post was not written on an iPhone, so any misspellings, grammar errors, or other word crimes are unacceptable. If you find one, drop us a line at feedback@iachieved.it.