How to work with a C library in Linux

I’ll talk about how to get started and play with one of the many C libraries out there on a Linux machine.

Let’s say you want to write and run C code on your Linux machine. I’ll talk about how to install it, get GCC to recognize it, and then use a makefile to simplify building. I’m using Ubuntu 14.04, but this should work in any distro.

The library I’ve used here is libnl – a library for userspace code to use generic netlink sockets, a mechanism which is often used to interact with KLM’s (kernel loadable modules).

First, we need the libraries on our machine.

sudo apt-get install libnl-3-dev libnl-genl-3-dev

libnl-3 is the core library for netlink sockets, and libnl-genl-3 is the library for generic netlink sockets, which is a special type of netlink socket.

If you want to install from the source, use the standard configure and make commands, as listed here.

Now that we have the libnl library with the headers with us, let’s start with a simple piece of C code-

/* userspace.c */
#include <stdio.h>
#include <netlink/netlink.h>
static struct nl_sock *sk;

int main() {
printf("Hello, world.\n");
/* Allocate a new socket */
sk = nl_socket_alloc();
/* Destroy the socket */
nl_socket_free(sk);
return 0;
}

This uses the netlink library for the API’s to allocate and destroy the netlink socket.

Build it using gcc.

$ gcc userspace.c -o userspace -llibnl3 -g -Wall -O3
userspace.c:2:29: fatal error: netlink/netlink.h: No such file or directory
 #include <netlink/netlink.h>

This failed because the gcc compiler was unable to find the include file netlink/netlink.h.

We need to add the header file path to the include path for GCC to pick it up.

$ gcc userspace.c -o userspace -I/usr/include/libnl3 -g -Wall -O3

However, just running this will give still give errors.

 /tmp/ccTtP6p3.o: In function `main':
 /home/saksbhat/userspace.c:10: undefined reference to `nl_socket_alloc'
 /home/saksbhat/userspace.c:11: undefined reference to `nl_socket_free'

GCC knows where to get the header files from, where nl_socket_alloc() and nl_socket_free() are defined, but what about the actual libraries? We also need to add the shared object to the linker.

In Ubuntu 14.04, a quick search (sudo find . -name “*genl*”) showed me the location of the libraries. We need to tell GCC to add this directory path using -L, and link the specific object file using -l.

$ gcc userspace.c -o userspace -I/usr/include/libnl3 -L/lib/x86_64-linux-
gnu/ -lnl-3 -g -Wall -O3

Notice that I used the flag -lnl. This is because the prefix lib and the suffix .a (for static) and .so (for shared) is automatically understood and expected by the -l flag.

GCC first compiled the userspace.c source file into userspace.o object file (which is an assembly code translation), and then linked it, by resolving the dependencies on libnl library and generating an executable.

Instead of giving these command line arguments each time, we can use a makefile that does the same thing. Create a file named makefile in the same folder and write this-

P = userspace
OBJECTS =
CFLAGS = -g -Wall -O3 -I/usr/include/libnl3
LDLIBS = -L/lib/x86_64-linux-gnu/ -lnl-3
CC = c99
$(P): $(OBJECTS)

When you do a make, it will run these 6 instructions to generate the executable.

The first 5 lines define variables that the makefile uses to build. Notice that unlike bash scripts, makefile uses $() to dereference variables.

$(P) points to the userspace.c file. $(OBJECTS) talks about object files userspace.c is dependent on, when linking. These object files are linked first, and then userspace.o is processed.

$(CFLAGS) lists the compiler flags. $(LDLIBS) lists the linker arguments. $(CC) represents the C compiler.

The last line is of the rule that invokes it all. target : dependencies. Our target is the executable userspace and, in this case we don’t have any additional object files apart from userspace.o. We don’t need to list that though, “make” understands that by default.

Let’s add a clean procedure to the makefile.

clean:
    rm $(P)

As earlier, the first line is the target, and the dependencies is blank. The second line is the “recipe”, which in this case is removing the executable. Note that there must be a <TAB> before the recipe.

Run make clean to delete the executable. For more details on makefiles, click here.

Advertisements

Bash scripting: From basics to know-it-enough in ten minutes.

This is a pretty random post, but thought I should write a basic bash script tutorial that gets us up and running in no time. The best way to go about this is a series of challenges in increasing difficulty.

1) Write a bash command to store the current directory’s contents in a file.

Hint 1: The command ls lists what’s in the current directory. To store it in a file, we need to execute this command, send the results to the file as input, and then store it.

Hint 2: The backquote ” ` ” operator is used to execute a command (called command substitution). The ” > ”  operator is used as standard input to the next statement.

Answer 1: `echo ls` > ./file1

Answer 2: ls > ./file1

What happens without the backquote? The ls command will not be executed. Instead the word “ls” will directly be stored in the file. Note that file1 is in the current directory, since we used the ” ./ ” operator. Using cat ./file1, you can verify the file contents. We use a single ” > ” here. A double ” >>” operator would append the output to file1 instead.

2) List information of USB devices connected to system.

Hint 1: The command “dmesg” lists all kernel messages, which include statuses about I/O devices, memory, dma etc, since the computer booted up until the present.

Hint 2: The command “grep” searches across the input data to find matches with a certain word, and displays the matching line or phrase.

Answer: dmesg | grep -i “usb”

dmesg’s output is pipelined as input to grep. Grep receives it and searches for the word “usb” without bothering about case, since we’ve used ” -i “.

3) Assign a variable the eth0 IP address of machine using ifconfig.

Hint 1: Use grep and pipelining to extract the IP address. grep flag -o lists only the match instead of the entire line. grep flag -A prints matching line as well as lines following it.

Answer: ipVar=`ifconfig | grep -A 1 ‘eth0’| grep -o ‘inet addr:[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*’ | grep -o ‘[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*’`

ifconfig lists all interfaces with their IP address in the following manner:

eth0      Link encap:Ethernet  HWaddr 00:20:32:4b:48:f4
             inet addr:10.20.1.120  Bcast:10.20.255.255  Mask:255.255.0.0 …

The IP address (inet addr) is in the next line following eth0 interface information. We first match “eth0” with grep, and print next line using grep -A 1. This pipelines to grep filtering out the “inet addr” portion, which is then pipelined again to another grep, which extracts the IP address.

Note that regex has been used for extracting the IP address. [0-9]* means one or more digits. ” \.” refers to the dot seperating the IP address, and an escape character is used to denote that the special meaning of ” . ” in regex is not considered here.

Verify output using echo $ipVar

MQTT: Scaling up the next level using HiveMQ/ CloudMQTT

My previous post talked about setting a basic MQTT client setup, with one node publishing, and one node subscribing. This post will talk about taking the next step – moving on to a bigger and secure enterprise broker to handle communications.

There are several brokers out there that can be configured and set up to run your own MQTT network.

One option is HiveMQ. It’s an enterprise broker that you can run in your own machine. You can download the setup at their website http://www.hivemq.com/. Do check out the documentation, which has installation instructions too. Once you’ve installed it, run it on a machine, and let your clients connect to the machine’s IP address and port 1883 (by default). The code remains exactly the same as before.

Another option is CloudMQTT (http://www.cloudmqtt.com/) This is a hosted service, which means the broker is on the cloud. When you create an account and add an instance, you’ll get your own special broker in the cloud. A sample code is for a client connecting to their broker is here – http://www.cloudmqtt.com/docs-python.html

Both these services are free to use for a few connections, and paid thereafter. I can’t say which is better, but both have their advantages and disadvantages. HiveMQ has been built from the ground up, and has a lot of plug-ins and features that make it easy to monitor, secure and fast. CloudMQTT is on the cloud, and it’s quick to start off since you don’t need to worry about managing your own broker.

Note that both these options let you have your own private broker. Previously, the broker we used for a public broker where anyone could publish and subscribe.

More coming up…

Simple MQTT client in Python

MQTT is a lightweight protocol that’s ideal for the Internet of Things.

MQTT is based on the publish-subscribe model. This is quite different from the typical network protocol models where a connection is established, after which the client requests the server for information directly, and the server sends back information to the client’s IP address directly. In case of MQTT, nodes connect to a broker and subscribe to a topic. Then they wait. If a node publishes to the same topic in the node, all the other nodes who had subscribed to it receive it.

In the diagram below, Node 5 publishes “Hi!” to the topic “hello/world/”, and Node 1 and Node 3 receive the message since they are subscribing to the topic.

MQTT_fig1

There are various public brokers out there which anyone can use for general testing. A complete list can be found here – http://mqtt.org/wiki/doku.php/public_brokers. I’m using the broker- test.mosquitto.org. All my clients will connect to this broker at the address and port provided. Think of it like public bulletin boards, free for anyone to post and read. Later on, you might want your own private broker for your application. I’ll cover how to do that in my next post.

This is a tutorial to get MQTT clients up and running quickly on Python 2.7 (on a Linux machine). I’m using the oddly named Open Source MQTT broker, Mosquitto. It’s now been donated to the Eclipse Paho project. A lot of old code I saw online on some blogs has to be modified slightly to make it work now.

Part 1: Installation
Hopefully, you’ll have pip installed. It’s a great tool to install python modules, and it’ll save you a lot of hassle.
sudo apt-get install python-pip
Next, install the MQTT broker Mosquitto (or Paho now)-
pip install paho-mqtt

Part 2: Subscribe
Here’s a basic python code to subscribe to a topic hosted by a broker on the local machine itself.

# -*- coding: utf-8 -*-
import paho.mqtt.client as mqtt

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, rc):
print(“Connected with result code “+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe(“hello/world”)

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
print “Topic: “, msg.topic+’\nMessage: ‘+str(msg.payload)

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect(“test.mosquitto.org”, 1883, 60)

# Blocking call that processes network traffic, dispatches callbacks and
# handles reconnecting.
# Other loop*() functions are available that give a threaded interface and a
# manual interface.
client.loop_forever()

This has been borrowed from https://pypi.python.org/pypi/paho-mqtt.

Part 3: Publish
Well now there’s a node which has subscribed to the topic /hello/world and is waiting for someone to publish data. This basic script below will do the job.

# -*- coding: utf-8 -*-
import paho.mqtt.client as mqtt

mqttc = mqtt.Client(“python_pub”)
mqttc.connect(“test.mosquitto.org”, 1883)
mqttc.publish(“hello/world”, “Hello, World!”)
mqttc.loop(2) //timeout = 2s

Part 4: Execute
That’s it. You’re done. Run the subscribe node first and then the publish node. You’ll notice the subscribed node receiving any message sent by the publishing node via the broker.

The next post covers how to expand our MQTT architecture by setting up our own broker.