logo

flickr.com
IBM Industrial Placement

April to September 2006 (6 months)

Industrial Placement

As part of an Imperial MEng Computing degree, you must spend six months in industry over the summer of your third year. I spent my Industrial Placement at IBM hursley, in a small Pervasive Messaging team, under the management of Andy Stanford-Clark. He is well known for his geeky antics in the field, and it was a lot of fun working in his team! Hursley is a nice place, and IBM has a very relaxed atmosphere.

As part of my degree, I had to keep a log book of my time at IBM, and although it isn't appropriate to publish the whole document, I will include my executive summary, giving an overview of what I worked on.

Logbook Executive Summary

Team Overview

I worked for six months within the Pervasive Messaging Technologies team at IBM under the management of Dr. Andy Stanford-Clark. Pervasive Messaging is a small group under the Websphere software division, focusing on messaging over constrained, largely mobile or remote devices and sensors.

The team is responsible for a product called 'Websphere micro broker', which is a light weight message broker, targeted for mediating between devices on the edge of the IBM messaging solution, which is also able to act as a gateway for these devices to communicate with enterprise level message brokers such as MQ. The current version of the micro broker supports connections over a lightweight publish and subscribe protocol called MQ Transport Telemetry (MQTT).

MQTT supports three specifiable per-message Quality of Service levels which allow an application to guarantee certain levels of delivery assurance: 'Best Efforts', 'At least once', and 'Once and Once Only'. This is achieved in the protocol by multiple low-level, transparent exchanges for messages where delivery assurance is required, and using a pluggable persistence interface used to restore state following a software crash, or power-cycle. And with just a two-byte fixed header, this allows clients to achieve reliable messaging end to end without excessive packet or processing costs.

The micro broker itself is written in Java, and is packaged with a Java MQTT client library, allowing user applications to add MQTT functionality to their program quickly and easily. A C client library (IA93) and a second less-generalised, but very-lightweight Java client library (IA92) are also available as support packs, and are free to download from IBM. IA92 is mainly suitability for small java devices such as mobile phones. The primary, protocol-stack-based implementation is favoured on larger machines for its modularity, sharing much of its code base with the micro broker itself.

My Work

I was fortunate enough to get involved with a range of varied projects. One large development project, consisting of a few discrete components consumed most of my six months, but I was able to share development time between other projects. Some of these were Think Friday projects, where-by IBMers are encouraged to work on interesting problems that may be unrelated to their main business area. Other projects sprung from small requirements that arose during my time within the team. Listed below are the main projects that I was involved with:

  • MQTT Buffered Client - Design and Implementation
  • Event Based Policies - Design and prototype
  • Revision of IA92, an active Service Pack
  • Defect fixing for micro broker
  • Participated in the Hursley Technology eXchange (HTX)
  • Collaborated with an IBM community source project - the Universal Inbox
  • Helped to generalise a home automation, MQTT, SMS gateway project

These projects are described in brief detail below. Implementation details and design discussion can be found in the weekly log-book.

MQTT Buffered Client

I was first asked to look at designing a new MQTT client library with greater network isolation for mobile applications. The existing MQTT client libraries have the fundamental MQTT operations Connect, Disconnect, Publish, Subscribe and Unsubscribe. The semantics of these operations require that an application is first connected before it can publish any messages. In addition, any disconnections from the network (which may be frequent in a mobile environment) must be handled by the user application by repeatedly invoking Connect until it succeeds. In the mean time, all attempts to publish messages will fail ? these must be queued by the application and sent when a connection does exist. If the application wishes to provide any delivery assurance for these messages however, they must be persisted manually.

The ?Buffered Client? solution was to solve these problems by buffering messages internally when disconnected, and simplifying application logic by handling re-connections transparently. Importantly, delivery assurance semantics must be maintained. The following features were requested;

  • Memory Based buffering of messages
  • Delivery assurance through persistence interface
  • J2ME (mobile) concrete persistence implementation
  • Inbound message buffering
  • Connection and Transmission Policy

Since the buffered client library would be used primarily on mobile devices, interoperability with Java 2 Micro Edition (J2ME) was important, and as a related task, I was asked to implement an RMS persistence class which could be plugged in as the persistence mechanism for devices that did not support the current file based persistence.

From the first week, I maintained a Component Level Design (CLD) document working through specification, design considerations and architecture of the Buffered Client, working closely with both the teams? chief architect and my mentor in regular meetings in addition to informal discussions.

The main challenge of this project was in maintaining messages delivery assurance semantics whilst providing inbound and outbound buffering with low memory or processing overhead. By the end of this project, I had produced a distributable Buffered Client package that was interoperable with existing MQTT client library implementations. Using just a subset of the full Java API, the library will run on very limited versions of java, such as J2ME.

I integrated a sub-project, ?event based policies? into the buffered client toward the end of my placement. This project is discussed separately, as it forms a general utility package. When integrated with the buffered client, it allows an application to specify exactly and concisely when the buffered client should connect to a broker and send messages. This enables applications to make most efficient use of network channels, minimizing expense, or saving power by connecting infrequently.

Event Based Policy Definition Language and Utility Package

I designed a lightweight, event based policy definition language, contained within a utility package for re-use. There was interest in using such a package in various areas of the micro broker to specify when a client may wish to connect to a broker, but also how a micro broker itself may wish to connect to other brokers. The package was designed to be easy to use, run under constrained devices, and provide easy extensibility with third party libraries, such as WECM, a library used to gather network connectivity status and network type from laptops and mobile devices.

Design of the policy language borrowed heavily from the material I learnt in the Imperial DoC Compilers course. This allowed me to use techniques to simplify the design of the package, and speak in common terms for my documentation.

Defect Fixing

I spent a month or so working very closely with the micro broker team fixing defects. In which time I experienced the most typical routine of a pervasive messaging developer. I learnt to use CMVC during this time for version control, raising defects and assigning work to myself. I spent a lot of time interacting with colleagues to fix problems, and learnt a lot about the code base, and issues involved with industrial development, and managing medium to large sized products.

Hursley Technology Exchange

My interest in hardware and embedded systems led me to attend some meetings for the upcoming Hursley Technology eXchange (HTX), where-by areas of IBM raise awareness of their technologies through presentations and stands, increasing collaboration between departments. I took responsibility for setting up the 'Mote Computing' demonstration and putting together some slides from colleagues. A mote is simply a small, often wireless smart device or sensor. The demo used a wireless dial mote for navigation between slides, a wireless height sensor attached to the ceiling and a set of scales forming a Body Mass Index calculator demonstration, and a dog door mat mote, all communicating via MQTT.

Universal Inbox Community Project

I also got involved in contributing to an IBM community source project, the Universal Inbox (UIB). The UIB enables a user to receive self describing XML messages based on registered interests. The message provides enough information to describe how it should be rendered, and how it can be interacted with. For example, you may subscribe to your front door. The door may tell you if it is locked, and notify you when someone rings your bell. Whilst a visitor is stood at your door, it may also allow you to interact with the door by unlocking it for entry. Having seen the previous generation UIB which featured in a BBC news article, I was keen to support the new pluggable version, and contributed to the design and implementation of the fundamental communications framework. I attended several design meetings, and was asked to commit my code to the repository.

MQTT Portal

I was asked to incorporate an SMS MQTT portal project which a pre-university student was working on, with another students MQTT Zigbee work on a single machine for demonstration. This led to collaborative work on generalising the gateway to accept typed devices, as opposed to the single X10 devices currently supported for home automation. With extensible device input and output, the gateway can support a range of formats including direct MQTT messaging, X10 control through an MQTT gateway, and UIB Message support. An existing SMS input gateway further increases the product audience.

Learning Outcomes

Technical

  • Programming experience. Java, Ant scripting, and Perl programming
  • Though already familiar with Java, my time at IBM strengthened my confidence many fold. I gained particular experience in design for constrained devices. Working at times with a large code base and existing libraries my program intuition increased, and I felt more comfortable working from a shared repository.
  • Eclipse and UML tooling for efficient design and development
  • I found my knowledge of UML invaluable, and was able to apply design patterns I had learnt at university to real problems. My knowledge of Eclipse itself and IBM Rational tooling also improved, enabling me to be more productive, and communicate my ideas more clearly through design documentation.

General

  • My main development was not on a shared code base, but all my project design and refinement was conducted through group discussion and feedback. I also spent some time working firstly on defect fixing as part of a team, and secondly collaborating on the design and implementation of interesting software such as the UIB, and MQTT portal. I learnt a lot of how this collaboration process is undertaken in industry, and how to communicate my ideas clearly.

Shoutback

by Steven Lovegrove Jump to top