How to work with a C library in Linux

I’ll talk about how to get started and play with one of the many C libraries out there on a Linux machine.

Let’s say you want to write and run C code on your Linux machine. I’ll talk about how to install it, get GCC to recognize it, and then use a makefile to simplify building. I’m using Ubuntu 14.04, but this should work in any distro.

The library I’ve used here is libnl – a library for userspace code to use generic netlink sockets, a mechanism which is often used to interact with KLM’s (kernel loadable modules).

First, we need the libraries on our machine.

sudo apt-get install libnl-3-dev libnl-genl-3-dev

libnl-3 is the core library for netlink sockets, and libnl-genl-3 is the library for generic netlink sockets, which is a special type of netlink socket.

If you want to install from the source, use the standard configure and make commands, as listed here.

Now that we have the libnl library with the headers with us, let’s start with a simple piece of C code-

/* userspace.c */
#include <stdio.h>
#include <netlink/netlink.h>
static struct nl_sock *sk;

int main() {
printf("Hello, world.\n");
/* Allocate a new socket */
sk = nl_socket_alloc();
/* Destroy the socket */
nl_socket_free(sk);
return 0;
}

This uses the netlink library for the API’s to allocate and destroy the netlink socket.

Build it using gcc.

$ gcc userspace.c -o userspace -llibnl3 -g -Wall -O3
userspace.c:2:29: fatal error: netlink/netlink.h: No such file or directory
 #include <netlink/netlink.h>

This failed because the gcc compiler was unable to find the include file netlink/netlink.h.

We need to add the header file path to the include path for GCC to pick it up.

$ gcc userspace.c -o userspace -I/usr/include/libnl3 -g -Wall -O3

However, just running this will give still give errors.

 /tmp/ccTtP6p3.o: In function `main':
 /home/saksbhat/userspace.c:10: undefined reference to `nl_socket_alloc'
 /home/saksbhat/userspace.c:11: undefined reference to `nl_socket_free'

GCC knows where to get the header files from, where nl_socket_alloc() and nl_socket_free() are defined, but what about the actual libraries? We also need to add the shared object to the linker.

In Ubuntu 14.04, a quick search (sudo find . -name “*genl*”) showed me the location of the libraries. We need to tell GCC to add this directory path using -L, and link the specific object file using -l.

$ gcc userspace.c -o userspace -I/usr/include/libnl3 -L/lib/x86_64-linux-
gnu/ -lnl-3 -g -Wall -O3

Notice that I used the flag -lnl. This is because the prefix lib and the suffix .a (for static) and .so (for shared) is automatically understood and expected by the -l flag.

GCC first compiled the userspace.c source file into userspace.o object file (which is an assembly code translation), and then linked it, by resolving the dependencies on libnl library and generating an executable.

Instead of giving these command line arguments each time, we can use a makefile that does the same thing. Create a file named makefile in the same folder and write this-

P = userspace
OBJECTS =
CFLAGS = -g -Wall -O3 -I/usr/include/libnl3
LDLIBS = -L/lib/x86_64-linux-gnu/ -lnl-3
CC = c99
$(P): $(OBJECTS)

When you do a make, it will run these 6 instructions to generate the executable.

The first 5 lines define variables that the makefile uses to build. Notice that unlike bash scripts, makefile uses $() to dereference variables.

$(P) points to the userspace.c file. $(OBJECTS) talks about object files userspace.c is dependent on, when linking. These object files are linked first, and then userspace.o is processed.

$(CFLAGS) lists the compiler flags. $(LDLIBS) lists the linker arguments. $(CC) represents the C compiler.

The last line is of the rule that invokes it all. target : dependencies. Our target is the executable userspace and, in this case we don’t have any additional object files apart from userspace.o. We don’t need to list that though, “make” understands that by default.

Let’s add a clean procedure to the makefile.

clean:
    rm $(P)

As earlier, the first line is the target, and the dependencies is blank. The second line is the “recipe”, which in this case is removing the executable. Note that there must be a <TAB> before the recipe.

Run make clean to delete the executable. For more details on makefiles, click here.

Advertisements

Bash scripting: From basics to know-it-enough in ten minutes.

This is a pretty random post, but thought I should write a basic bash script tutorial that gets us up and running in no time. The best way to go about this is a series of challenges in increasing difficulty.

1) Write a bash command to store the current directory’s contents in a file.

Hint 1: The command ls lists what’s in the current directory. To store it in a file, we need to execute this command, send the results to the file as input, and then store it.

Hint 2: The backquote ” ` ” operator is used to execute a command (called command substitution). The ” > ”  operator is used as standard input to the next statement.

Answer 1: `echo ls` > ./file1

Answer 2: ls > ./file1

What happens without the backquote? The ls command will not be executed. Instead the word “ls” will directly be stored in the file. Note that file1 is in the current directory, since we used the ” ./ ” operator. Using cat ./file1, you can verify the file contents. We use a single ” > ” here. A double ” >>” operator would append the output to file1 instead.

2) List information of USB devices connected to system.

Hint 1: The command “dmesg” lists all kernel messages, which include statuses about I/O devices, memory, dma etc, since the computer booted up until the present.

Hint 2: The command “grep” searches across the input data to find matches with a certain word, and displays the matching line or phrase.

Answer: dmesg | grep -i “usb”

dmesg’s output is pipelined as input to grep. Grep receives it and searches for the word “usb” without bothering about case, since we’ve used ” -i “.

3) Assign a variable the eth0 IP address of machine using ifconfig.

Hint 1: Use grep and pipelining to extract the IP address. grep flag -o lists only the match instead of the entire line. grep flag -A prints matching line as well as lines following it.

Answer: ipVar=`ifconfig | grep -A 1 ‘eth0’| grep -o ‘inet addr:[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*’ | grep -o ‘[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*’`

ifconfig lists all interfaces with their IP address in the following manner:

eth0      Link encap:Ethernet  HWaddr 00:20:32:4b:48:f4
             inet addr:10.20.1.120  Bcast:10.20.255.255  Mask:255.255.0.0 …

The IP address (inet addr) is in the next line following eth0 interface information. We first match “eth0” with grep, and print next line using grep -A 1. This pipelines to grep filtering out the “inet addr” portion, which is then pipelined again to another grep, which extracts the IP address.

Note that regex has been used for extracting the IP address. [0-9]* means one or more digits. ” \.” refers to the dot seperating the IP address, and an escape character is used to denote that the special meaning of ” . ” in regex is not considered here.

Verify output using echo $ipVar

MQTT: Scaling up the next level using HiveMQ/ CloudMQTT

My previous post talked about setting a basic MQTT client setup, with one node publishing, and one node subscribing. This post will talk about taking the next step – moving on to a bigger and secure enterprise broker to handle communications.

There are several brokers out there that can be configured and set up to run your own MQTT network.

One option is HiveMQ. It’s an enterprise broker that you can run in your own machine. You can download the setup at their website http://www.hivemq.com/. Do check out the documentation, which has installation instructions too. Once you’ve installed it, run it on a machine, and let your clients connect to the machine’s IP address and port 1883 (by default). The code remains exactly the same as before.

Another option is CloudMQTT (http://www.cloudmqtt.com/) This is a hosted service, which means the broker is on the cloud. When you create an account and add an instance, you’ll get your own special broker in the cloud. A sample code is for a client connecting to their broker is here – http://www.cloudmqtt.com/docs-python.html

Both these services are free to use for a few connections, and paid thereafter. I can’t say which is better, but both have their advantages and disadvantages. HiveMQ has been built from the ground up, and has a lot of plug-ins and features that make it easy to monitor, secure and fast. CloudMQTT is on the cloud, and it’s quick to start off since you don’t need to worry about managing your own broker.

Note that both these options let you have your own private broker. Previously, the broker we used for a public broker where anyone could publish and subscribe.

More coming up…

Simple MQTT client in Python

MQTT is a lightweight protocol that’s ideal for the Internet of Things.

MQTT is based on the publish-subscribe model. This is quite different from the typical network protocol models where a connection is established, after which the client requests the server for information directly, and the server sends back information to the client’s IP address directly. In case of MQTT, nodes connect to a broker and subscribe to a topic. Then they wait. If a node publishes to the same topic in the node, all the other nodes who had subscribed to it receive it.

In the diagram below, Node 5 publishes “Hi!” to the topic “hello/world/”, and Node 1 and Node 3 receive the message since they are subscribing to the topic.

MQTT_fig1

There are various public brokers out there which anyone can use for general testing. A complete list can be found here – http://mqtt.org/wiki/doku.php/public_brokers. I’m using the broker- test.mosquitto.org. All my clients will connect to this broker at the address and port provided. Think of it like public bulletin boards, free for anyone to post and read. Later on, you might want your own private broker for your application. I’ll cover how to do that in my next post.

This is a tutorial to get MQTT clients up and running quickly on Python 2.7 (on a Linux machine). I’m using the oddly named Open Source MQTT broker, Mosquitto. It’s now been donated to the Eclipse Paho project. A lot of old code I saw online on some blogs has to be modified slightly to make it work now.

Part 1: Installation
Hopefully, you’ll have pip installed. It’s a great tool to install python modules, and it’ll save you a lot of hassle.
sudo apt-get install python-pip
Next, install the MQTT broker Mosquitto (or Paho now)-
pip install paho-mqtt

Part 2: Subscribe
Here’s a basic python code to subscribe to a topic hosted by a broker on the local machine itself.

# -*- coding: utf-8 -*-
import paho.mqtt.client as mqtt

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, rc):
print(“Connected with result code “+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe(“hello/world”)

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
print “Topic: “, msg.topic+’\nMessage: ‘+str(msg.payload)

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect(“test.mosquitto.org”, 1883, 60)

# Blocking call that processes network traffic, dispatches callbacks and
# handles reconnecting.
# Other loop*() functions are available that give a threaded interface and a
# manual interface.
client.loop_forever()

This has been borrowed from https://pypi.python.org/pypi/paho-mqtt.

Part 3: Publish
Well now there’s a node which has subscribed to the topic /hello/world and is waiting for someone to publish data. This basic script below will do the job.

# -*- coding: utf-8 -*-
import paho.mqtt.client as mqtt

mqttc = mqtt.Client(“python_pub”)
mqttc.connect(“test.mosquitto.org”, 1883)
mqttc.publish(“hello/world”, “Hello, World!”)
mqttc.loop(2) //timeout = 2s

Part 4: Execute
That’s it. You’re done. Run the subscribe node first and then the publish node. You’ll notice the subscribed node receiving any message sent by the publishing node via the broker.

The next post covers how to expand our MQTT architecture by setting up our own broker.

Character Recognition on a touch screen : Part 2 (Algorithm)

There are various existing methods for character recognition. I used a very intuitive algorithm- The chain code algorithm.

Chain Code
This is basically a notation for recording list of edge points along a contour. It specifies contour direction at each edge in the list. Directions are quantized into one of the 8 directions. Starting at 1st edge in the list and going clockwise around the contour, the direction to the next edge is specified using one of the 8 chain codes. The direction is the chain code for the 8-neighbour of the edge.

Chain Code notation

Chain Code notation

Defining each direction

Defining each direction

Chain Code of an example curve

Chain Code of an example curve

Chain Code Histogram
A chain code histogram counts the frequency of occurrence of each of the 8 directions. This means the information of the order of occurrence of the directions is lost. However, instead of storing the entire chain code, this enables storing only 8 data values.
The loss of accuracy is negligible. The storage space gained is immensely useful for embedded systems which have constrained resources.

Comparing the Chain Code
Once the chain code histogram data for the current coordinates are stored, it has to be compared with the pre-stored character’s histogram data. For that, we find the standard deviation.
The standard deviation is defined as –

std dev

std dev 2

Here n=8.
Thus, we find the similarity between the stroke and pre-stored character by seeing the deviation in the count in each of the 8 values (representing the 8 directions). The stroke is compared with every pre-stored character, and the one with lowest standard deviation wins.

Summary of the algorithm

Summary of the algorithm

Samples from MATLAB
Shown below are sample inputs, taken in MATLAB, from the touchscreen. (Each of the characters has been shown inverted, due to some hardware limitations.) It can be seen that the algorithm correctly recognizes the input by comparing input histogram with each character’s histogram.

Input stroke similar to ‘O’ letter. Notice the similarity of histogram between ‘O’ and input stroke.

Input stroke similar to ‘O’ letter. Notice the similarity of histogram between ‘O’ and input stroke.

Input stroke similar to ‘S’ letter. Notice the similarity of histogram between ‘O’ and input stroke.

Input stroke similar to ‘S’ letter. Notice the similarity of histogram between ‘O’ and input stroke.’

Features of algorithm used
The algorithm is scale and position invariant. Thus, it doesn’t matter where the stroke is made on the touchscreen, or how big the stroke is. What matters is the shape of the stroke.
The algorithm is not rotation invariant. Thus, the strokes will be recognized only in a particular orientation.
The algorithm is very efficient in terms of storage and computations performed. Only 8 data values are stored per character. Comparison is done by performing standard deviation calculations, which are very fast.

Character Recognition on a touch screen : Part1 (Hardware)

As part of my final year project, I got down to the basics. Using a 4 wire resistive touch screen, PSoC-1 (which is a microcontroller + some other cool blocks) , and MATLAB, I got the strokes on the touchscreen to be recognizable by MATLAB in a simple, efficient and fast manner.

I had originally planned on doing the entire thing on PSoC-1, which has a pretty watered down MCU core (M8C). However, lack of time got me to develop the algorithm on MATLAB instead.

Okay so here’s what I used-

1) Nintendo DC Touch Screen
2) Nintendo DC Touch Screen 4-wire Connector
3) PSoC-1 EVAL kit
4) USB-UART cable
5) MATLAB 2011b software

The Big Picture

Process Flow Diagram

Process Flow Diagram

Step 3 is the most important part, of course, and there are various ways of implementing it. I used a modified chain code algorithm, which I’ll describe in Part 2 of this series.

How the touch screen works

The x and y coordinates of a touch on a 4-wire touch screen can be read in two steps. First, Y+ is driven high, Y– is driven to ground, and the voltage at X+ is measured. The ratio of this measured voltage to the drive voltage applied is equal to the ratio of the y coordinate to the height of the touch screen. 5V is applied for the high input. The y coordinate can be calculated as shown in the figure below. The x coordinate can be similarly obtained by driving X+ high, driving X– to ground, and measuring the voltage at Y+. The ratio of this measured voltage to the drive voltage applied is equal to the ratio of the x coordinate to the width of the touch screen. This measurement scheme is shown in the figure below.

Measurement scheme for resistive touch screen

Measurement scheme for resistive touch screen

The PSoC-1 bit

PSoC (Programmable System-on-Chip) is a family of integrated circuits made by Cypress Semiconductor. These chips include a CPU and mixed-signal arrays of configurable integrated analog and digital peripherals.
PSoC 1 has been used for this project. It has an M8C core, with various configurable analog and digital blocks. Each digital block is considered an 8-bit resource that designers can configure using pre-built digital functions or user modules.
The software used to program and design applications for PSoC is PSoC Designer.
In this project, the user modules used are –
1)ADCINC (Analog to Digital Converter) –
The ADCINC is one of the ADC user modules offered with PSoC Designer for the PSoC 1 family of parts. It uses an analog switched capacitor block and one digital block. The ADCINC is an averaging ADC that samples the analog input many times (driven by DataClock) to produce one n-bit digital output sample.
2)PGA (Programmable Gain Amplifier)-
The input signal is buffered with the PGA and connected to the ADC. It uses one
3)PWM8 (Pulse Width Modulation 8-bit) –
2 PWM’s have been used to drive 2 LED’s, one for each axis. It has a programmable period and pulse width.
4)LCD
The character LCD is used to display the values of X and Y axis of touchscreen.

Digital blocks used in PSoC-1
Digital blocks used in PSoC-1

The screenshot shows 3 digital blocks being used.
DBB00 – ADCINC
DBB01 – PWM8_1
DCB02 – PWM8_2
An explanation of its usage has been provided in the next section.

Analog blocks used in PSoC-1
Analog blocks used in PSoC-1

Touch Screen Interfacing

Reading the Touch Screen Input
The input from the four-wire touchscreen comes from 4 wires, typically labelled X1, X2, Y1, Y2. They are all connected to Pins 0,1,3 and 7 of Port 0 on PSoC, and are dynamically configured as analog input when reading is required.
PSoC reads these values, and outputs it to LCD configured on Port 2. It also sends output to 2 LED’s, one for each axis, in pins 0 and 1 of Port 1.

Understanding the Code
The code was written in PSoC Designer. Here is an explanation
Firstly, all modules have been initialised. This is necessary before readings are taken from the touchscreen.
An infinite for loop has been used, since values are constantly being read from the touchscreen using the ADC. Each time the loop runs, it alternates between 2 sets of code. One is to read from the X axis, and the other from the Y axis. Since only 1 ADC has been used, its input is constantly being changed using a MUX.

When reading from the X-axis, Pin 0 and Pin 3 (of Port 0) are set in Strong Drive. Pin 1 and Pin 7 are set in High Z Analog Drive Mode. The input is taken from Pin 1.
When reading from the Y axis, Pin 1 and Pin 7 are set in Strong Drive. Pin 0 and Pin 3 are set in High Z Analog Drive Mode. The input is taken from Pin 3.
To constantly change these drive modes, we use the PRTxDMx registers, as explained in AN2074 (see reference 3).
Before further processing, the values for each axis are averaged out to eliminate any error due to noise.
Once the value has been read from the ADC, it is written as the pulse width in the PWM module. Since an 8-bit value is read, the pulse can vary from 0 to 255. The total period is 255. Thus the LED will glow at a maximum, when the ADC value reads 255, and minimum when it reads a 0.
The value is also written to the LCD.

____

This concludes Part 1 of the series. Part 2 will cover the algorithm which was implemented in MATLAB. Input from the touchscreen was decoded by PSoC, and then processed in MATLAB.