Monday, December 31, 2012

Plotting a weight loss curve from CanOfSleep's WWDiary app

I've been using the Android app WWDiary published by as a food / weight diary. And it works! ... for me anyhow. Another few kg to go and I should be at my goal early in 2013. It's always encouraging to see your long term progress plotted visually. Unfortunately this app has no plotting capability so I wrote a short GnuPlot script to do this.
I manually transcribed the values from the weight diary into file 'weight.dat' with a format that looks like this:

20121226 74.0
20121227 74.6
20121228 73.4
20121229 73.1
20121230 74.1
20121231 74.9

And the following GnuPlot script generates the above chart:

set xdata time
set timefmt "%Y%m%d"
set format x "%d/%m"
set title "Weight loss curve 2012"
set xlabel "Time"
set ylabel "Mass (kg)"
set term png size 800,400
set output "weight.png"
plot 'weight.dat' using 1:2 with linespoints


10 June 2014: I've got an updated howto on doing this which includes extracting the data automatically from the wwdiary database file. See

Wednesday, November 14, 2012

Change of blogger template makes a big difference to traffic

Update (27 June 2013): No I was wrong: that increase in traffic is some weird artifact of that template (a template which I quickly grew to hate and reverted back to something simple a few days ago).

I was experimenting with blog templates last week. I changed my template to one of the new Blogger dynamic templates.  It looked better than the last one, but I didn't think it would make any practical difference.

I was surprised to see this week that my traffic numbers have soared over 300% on last week. Initially I thought I got some coverage at a popular blog or website, but there was no specific traffic source or destination page. I also noticed this surge traffic started on the same day I changed the template.

So there you go... one click and an instant 300% traffic increase!

Unfortunately I didn't have any screen shots of the blog as it was before, and I was unable to find the exact same template again, but this approximately what it looked like before (left) and after (right).


Wednesday, October 31, 2012

Measuring the current drain of a USB device

I've got a evaluation module of the Texas Instruments ADS1292R analog front end (AFE) for electrocardiography (ECG) applications which I'm incorporating into a prototype application. The AFE chip itself doesn't consume much current (10mA while measuring), but it's hard to tell what the entire evaluation board uses considering there is a MCU, LEDs, flash memory and a few other devices on board.

I needed to see what kind of current was being drawn by this board as the intention is to run the application on batteries. The board uses a USB port for both power and communications.

One idea I had was to splice a USB cable and route the 5V line though a multimeter in current mode. However digital loads can be very variable, and it would be difficult to gauge the total charge consumed by an ECG measurement cycle from a multimeter.  I really want a chart of current vs time at a high (millisecond) time resolution.

My solution was to cut a USB cable and add a low value current sense resistor in series on the 5V line. I chose the lowest value I had in my parts drawers: 1.8 ohms. The problem with current sense resistors is that there will be a significant voltage drop at high currents. At the normal USB maximum of 500mA that would be a drop of 0.9V on the 1.8 ohm resistor. Given that the board used 3.3V logic and I expected no where near the 500mA draw I figured 1.8 ohms as an acceptable value.

Sense wires were soldered to both sides of the resistor (orange and white) which were then connected to two probes of an oscilloscope. The scope was then set to display the difference between the two channels. The resulting voltage trace when divided by the resistor value is the current being drawn on the USB bus as a function of time.


In retrospect the choice of 1.8 ohms was too low. It turns out this board does not draw much current.  Due to the resolution limits of the scope the difference between the two voltages is pretty much zero. Yes, I could hook up the sense wires to a differential amplifier... was hoping to avoid any extra electronics. However it did confirm that whatever current was being drawn by the evaluation board is on the low side which is all I need to know right now.

To illustrate this cable in action I connected an Android phone. The host side of the 5V bus is channel 1 (yellow), the device side channel 2 (green) and the difference is in pink. The voltage difference must be divided by the value of the sense resistor (1.8 ohms) to arrive at the current. So that's 400mV peak-to-peak (ie 222mA bursts) and a mean difference voltage of 67mV ie 37mA mean current draw.

Obviously hacking a cable like this isn't going to do wonders for its performance at high speeds. But so far I have not noticed any ill effects.

Philips CFL bulb tear down

I experienced a rare CFL bulb failure last night (I've got about 16 bulbs installed about 12 years ago. This has been the third failure to date). For the past few weeks this bulb was taking longer than normal to ramp up to full brightness, so I suspect a tube failure.

I couldn't resist having a quick peek inside the blown bulb. I opened by prying with a large flat head screwdriver:

This is the electronics board:

Nothing exciting: a transformer, 5.6uF 360V capacitor (the large can on top), 2 x ST 13003A NPN power transistors and a hand full of miscellaneous discretes.

And the tube:

Disclaimer: Don't try this if you don't know what you're doing. There is a trace amount of mercury (toxic!) inside the tube and you can injure yourself on the glass if you break it. Dispose correctly according to local regulations afterwards.

Saturday, October 13, 2012

Searching for old Facebook posts

In the past two weeks two people I know have had difficulty trying to find a link or information from an old Facebook post. It seems that Facebook just can't do search.

Here's a work around:

Select "Account Settings" (down arrow at far top-right of screen). At the bottom of the account settings screen there is a "Download a copy of your Facebook data" link. Click on that and follow the procedure. It takes several hours for Facebook to generate an archive of all your data. Once complete you will be notified by email and a link will be provided where you can download a ZIP archive with all your stuff.

Unpack that ZIP and use your browser to navigate to the unpacked directory. Look for file "wall.html" in directory 'html'. That's a HTML page with all your posts.

Now use the browser's in-page search (usually CTRL-F or use the browser menu) to search for what you're looking for. The browser in-page search is a simple substring search... but hey... it's waaay better than what Facebook can manage.

Friday, August 31, 2012

Weekends make me fat!

I've been trying to shift some weight recently and having some success thanks to a great Android app called 'WWDiary' published by CanOfSleep.

Weekends are never a good time for losing weight, but that's just a hunch. It just occurred to me today I may have enough data to actually prove this.

One of the features of this app is a weight diary which I've been using. So I plugged the data into a data file and ran it through GnuPlot.

I then did an auto correlation of this curve:

And sure enough there is a definite peak at 7, 14 and 21 days. "Proof" that weekends do indeed make me fat! [Will revisit this when I have more data!]

Update (28 May 2014): This post was first posted 31 Aug 2012. After almost two more years of data I thought I should update the auto correlation chart:

The peaks at about 7, 14 and 21 days is getting more pronounced.

Tuesday, July 31, 2012

Transistor abuse: using the avalanching effect of a BJT transistor to generate short pulses

I recently came across an EEVBlog video on a little circuit made popular by Jim Williams in Linear Technology Application Note 47.  This is a simple pulse generator using the breakdown avalanching effect of a BJT transistor which can be used to calculate the bandwidth response of an oscilloscope among other things. Rise times in the order of 300ps are easily achieved this way. The idea of making transistors do things they were not really intended to do is intriguing and I couldn't resist giving it a shot  myself.

The device used in the app note is a Motorola 2N2369, which breaks down at about Vce = 90V. I don't have a PSU that reaches 90V, so I just ordered some low cost, low Vceo transistors.

The circuit used is almost identical to that in App Note 47. I used a larger capacitor (39pF vs 2pF in AN47).

The results have so far been quite pleasing. The following are traces from pulses generated from 3 different BJT devices (see captions under images).  Time base is 20ns / division. My setup is not optimum: the circuit above was implemented on a bread board and the pulses were not coupled to the oscilloscope via 50Ω transmission line. Also my scope is rated at 100MHz bandwidth.

DUT is NXP BFG424F. Avalanching observed when Vce is 15.8V.

DUT is NXP BFG425W. Avalanching observed when Vce is 16.4V.

DUT is NXP BFT25A. Avalanching observed when Vce is 11.8V.

Saturday, June 30, 2012

OKI 900 mobile phone limited tear down

I found my old OKI 900 analog mobile phone while looking for a ferrite rod in a junk box at my parents home. This unit was manufactured in 1991. I acquired it second-hand around 1993. It cost me about a months wages back then.

Googling this phone, it seems to have been a very hackable phone back in it's day. It's based on the 8051 MCU core and there are various ROMs available on the network to do all sorts of interesting things.  However, right now I don't have any interest/time in making any mods to this. Indeed I couldn't even be bothered powering it up. So I'll take Dave Jones advice: "don't power it up, take it apart"... but only out of lazyiness :-)

Saturday, May 12, 2012

Lidl 'Silvercrest' radio teardown

Last week Lidl (a German budget supermarket chain) had a small "transistor" radio on special. Like a lot of their low end electronics it was "Silvercrest" brand. For various reasons I had need of one (actually I was planning to canabilise it for its ferrite rod antenna, but found an alternative source in the mean time).

Usually Lidl stuff is fine. Not brands you'd normally recognize, but solid stuff none-the-less. So this was rather disappointing.
On powering up, the LCD was faulty. The top half of the display wasn't working.

So I had a look inside:

Apart from the grubby PCB (normal enough for low end electronics these days) there was a botched bodge capacitor (not connect on the left terminal) and a wire to the ferrite antenna nicked by a screw.

So I exchanged it for another. But thankfully decided to checked it out in the car before leaving. Radio #2 didn't even power up. So I tried #3 which did work fine. Two out of three failure rate seems a bit high... or maybe I was just unlucky.

The IC in the photo is a Sony CXA1691 AM/FM radio chip. Everything else looked like it was lifted right out of the 1970s. But hey, it's just a "transistor" radio...

Thursday, April 12, 2012

The STM32W-RFCKIT as a 802.15.4 network analyzer on Linux

Summary: The STM32W-RFCKIT together with ST Microelectronics application note AN3406 is a low cost 802.15.4 (ZigBee, MiWi etc) network analyzer. Unfortunately the solution is Windows only and closed source. In this article I have reverse engineered the firmware protocol used in this app note so that a similar solution can be implemented on Linux, Mac OSX and other platforms. I have also written a first draft of this software for Linux.
USB dongle
The STM32W-RFCKIT is a low cost evaluation kit for the STM32W108 802.15.4 SoC from ST Microelectronics. The kit retails for about €33 ($40?) and comprises a USB dongle and a battery powered remote control device with buttons.

I was drawn to this kit because I had recently purchased a STM32F4Discovery board and I was impressed at how easy it was get a GCC tool chain up an running on Linux. I'm currently on the look out for a reliable, low cost 802.15.4 network analyzer solution and given its low cost,  the hope of Linux friendliness and the 802.15.4 transceiver I figured I had a good shot with this hardware.

I was pleasantly surprised to find the kit already comes with a free 802.15.4 network analyzer.  One one of the associated application notes, AN3406, is a 802.15.4 packet sniffer for the dongle. This works by loading special firmware onto the dongle. This then communicates with a Windows server program which translates the dongle's captured 802.15.4 packets (encoded in their own proprietary frame format) into PCAP. The Windows server program also invokes Wireshark and feeds the packet data to it.

Block diagram of the ST Microelectronics AN3406 packet sniffer solution (taken from application note AN3406).
Unfortunately this is a Windows only solution. As a Linux user it's always a pain to have to boot a Windows virtual machine (consuming a huge chunk of my available RAM) just to run one tool. So I set about writing a tool to facilitate the same functionality in Linux.

Although the AN3406 software is free, it is not open source and there is no documentation on the dongle to Wireshark server protocol. So I had to reverse engineer the dongle's protocol to get the same functionality with Linux.

I used the free (and excellent!) serial port monitor tool at to capture the packets to/from the device while opening the Wireshark server and starting a packet capture. Fortunately everything looked very straightforward and it wasn't difficult to write a Linux based server.

The packet structure is

All frames are prefixed with two bytes 0x15, 0xFF. This is followed by a one byte frame length field which is the number of bytes in the frame (excluding the 0x15, 0xFF prefix, the checksum and 0x0C terminator). All frames have a packet type or command code, therefore the minimum valid value for the frame length is 2.

This is followed by a one byte packet type or command code. I notice that frames sent from host to dongle have the command code most significant bit (MSB) cleared while those from the dongle to the host have the command MSB set. For example command 0x10 (set channel) is responded by command 0x90.

Depending on the command there may be zero, one or more data bytes.

Next a frame check sum which is calculated by summing of all bytes from the length field to the last data byte in a 8 bit register and then doing a bitwise NOT.  Finally all frames are terminated with byte 0x0C.

The known command codes are:

Command CodeDirectionDescriptionParameters / Data
0x01 Host to dongle No idea. However it's required for packet capture to work None
0x81Dongle to hostResponse to 0x01 command.One byte: 0x00.
0x10Host to dongleSet 802.15.4 channelFollowed by one byte of data which will contain the channel number (11 to 26).
0x90Dongle to host Set channel responseSame as 0x10
0x11Host to dongleStart relaying packets.No parameters
0x91Dongle to hostStart relay response.No parameters
0x12Host to dongleStop relaying packets.No parameters
0x92Dongle to hostStop relay response.No parameters
0xF0Dongle to host802.15.4 packet data802.15.4 packet metadata (timestamp, RSSI) followed by actual packet data. See section 'Packet capture format' for details.

Packet capture format:

802.15.4 packets are returned along with some packet metadata in the data portion of frames with command code 0xF0. After observing a few frames it quickly became apparent that the actual 802.15.4 packet data started about 8 bytes in. Thus the first 7 bytes is some sort of metadata.

I'll refer to these bytes by their index values starting at 0. Byte 5 remained constant throughout the entire capture and happened to be the same value as the 802.15.4 channel number. So I can safely say that's the channel number. Bytes 1 to 4 also grew in value consistent with a time stamp. As for byte 0, I wasn't sure about that. It seemed random consistent with the LSB of a time stamp, but I thought 5 bytes for a time stamp was odd. Also I figured that there must be a link quality indicator (RSSI) in there somewhere also. Byte 6 looked like a good candidate for the RSSI. So how to know? Plotting these values as a function of time can often reveal their meaning:

Byte 0 (blue) seems to be completely random. So my guess is that's the the least significant byte of a 5 byte time stamp clock. Byte 6 (red) on the other hand must be the RSSI. There are several ZigBee devices on the network. Each device will have an RSSI value based on it's distance/location/antenna that won't vary quickly with time. So each of those red traces corresponds to one ZigBee devices.

So it seems that byte 0, byte 1 and the lower nybble of byte 2 form a 20 bit fraction of a second (ie if treated as an integer each unit is 1/(2^20) seconds.  The upper nybble of byte 2, byte 3 and byte 4 form a 20 bit seconds field.

Putting it all together:

To try this, you must first load the sniffer firmware to the dongle. Right now the only way I know to achieve this is using the AN3406 Windows software. I've documented the procedure in this blog post. And it's also covered in AN3406. However I have no doubt that this can also be accomplished with some of the Linux tools used to program the STM32-F4Discovery board. If you know how, please let me know.

I wrote a translator from this firmware protocol to a PCAP format which can be piped into Wireshark. It's released under the BSD licence. Here is my first draft (version 0.1).

To run:

wireshark -k -i <( ./stm32w-wireshark /dev/ttyACM0 12 )

The command line tool takes two arguments: the device (usually /dev/ttyACM0) and the 802.15.4 channel number (11 to 26). Use the -h flag to get information on more options.

Note: there is a problem with launching Wireshark this way. If you close the Wireshark window, the stm32w-wireshark utility does not die automatically and remains running in the background.  Starting it again will mean that two stm32w-wireshark processes will be grabbing data from the dongle causing massive packet corruption. If you see packet corruption, check for running processes (ps -auxw | grep stm32w-wireshark) and kill any lingering processes. If you have suggestions for a better way of handling this please let me know.

What next?

There are a few lose ends I would like to tidy up (and I would appreciate any suggestions):
  • What's the best way of launching this tool together with Wireshark? Closing the Wireshark window should cause the tool to die. Should I be using named pipes?
  • It's a shame that the RSSI value cannot be recorded in the PCAP file (there simply is no field for arbitrary metadata). So if one wanted to use RSSI information together with Wireshark how can this be achieved? 
  • I would like to be able to upload this firmware without having to boot windows. It's the only Windows dependency left and it would nice to remove it. I suspect one of the standard tools used with the STM32 kits will do the job, but I don't have time to figure that out right now. 

Monday, April 2, 2012

STM32W-RFCKIT as a low cost 802.15.4 / ZigBee network analyzer

Only a few years ago 802.15.4 / ZigBee network analyzers were an expensive affair. Now, many of the low cost evaluation kits from manufacturers such as Microchip, Texas Instruments and ST Microelectronics can be configured as network analyzers.

This morning I took delivery of a ST Microelectronics STM32W-RFCKIT (€33 from Mouser). I was pleasantly surprised that it too has a network analyzer in the form of a Wireshark bridge. These days Wireshark is the only game in town for packet analysis, so this was a smart move.

The kit comes in two pieces: a remote control device with 6 buttons and a dongle for the PC.

Unfortunately all this is Windows software. So I powered up a Windows XP virtual machine, and installed the software from the ZIP file SW application to interface Wireshark packet capture tool (AN3406). I also needed to install Wireshark for Windows. I plugged in the dongle and assigned the USB device to my Windows VM (this is the same as physically plugging in the device if it was a real computer). Then the usual windows new hardware detection and driver installation. (I can never quite understand what is going on with Windows when new hardware is plugged in. I did a reboot also, but that was more out of habit... it might not have been  necessary).

Next, start the STM32W108 Wireshark Server. In the Settings menu set the location of the Wireshark application (usually under Program Files/Wireshark) if this has not been already set.

Right out of the box, the dongle does not have the right firmware installed. So select the Tools menu and select "Flasher". Click the 'Flash' button and all going well, the packet sniffer firmware will be transferred onto the dongle.

Now back to the main window of the Wireshark Server application. Select the right serial port. The Play button should now become active. Hit the play button and all going well Wireshark should start sniffing packets from the dongle.

Linux is my OS of choice, so it would be great to have this working without having to boot a Windows VM. Unfortunately it seems that despite the wise choice of using the Wireshark tool and extolling the virtues of open source code in the documentation, they've neglected to include the source code for the firmware and server software. However it shouldn't be too difficult to reverse engineer. If I get a chance over the coming days I'll give it a stab.

A screen grab of  Wireshark sniffing 802.15.4 / ZigBee packets from the dongle supplied with the STM32W-RFCKIT.
Update (3 April 2012): I just exchanged emails with STM tech support to see if there were willing to share the sniffer firmware source code. Unfortunately they are not. So it's either reverse engineer what's already there or write new firmware.

Update (7 April 2012): I did a quick comparison with the STM32W dongle vs my Microchip ZENA (first version). Both devices are on my desk, about 50cm apart. I ran a Wireshark session on both devices at the same time on the same channel. After a few minutes the STM32W device captured 424 packets vs 290 packets from the ZENA.  It seems the STM32W-RFCKIT makes for a better packet sniffer!

Update (9 April 2012): I've made good progress reverse engineering the firmware protocol. I've now got a Linux command line tool (written in C) that will dump packet hex to terminal and allows the 802.15.4. channel to be changed. I hope to release the first version of the Linux Wireshark server in the next few days.

Update (12 April 2012): The first version of the Linux Wireshark server has been released.

Saturday, March 31, 2012

Irish landed estates database 'heatmap'

I am involved with a project to catalog Irish landed estates in conjunction with the Moore Institute at the National University of Ireland, Galway. Right now the database covers the provinces of Muster and Connaught.

The web site includes an interactive map which visitors can use to browse for houses and estates of interest. I thought it might be interesting to make a 'heat map' of the areas which people found the most interesting.

Every time someone zooms into an area on the map, the database is queried for records matching that rectangular area. The web server logs record all these queries (anonymously of course).

The image on the right was generated by summing about half a million of these database queries. I had intended to super-impose the heat map on a Google map, but unfortunately I don't have the time to complete that right now. However one can see the distinct outline of the west and south of Ireland coast line with hot spots around the cities of Galway and Limerick. (Why there is no hot-spot around Cork city? I have no idea!)

Wednesday, March 14, 2012

The weight ratio of dry pasta to cooked pasta

It's not often you ask Google a simple question and you can't get a straight answer. My question was this: "what is the weight ratio of dry pasta to cooked pasta?"

There is plenty of discussion on this matter, with units such as ounces, cups, Fahrenheit, buschels and microfortnights bandied around. But not one answer that had the simple dimensionless ratio I was looking for.

So it's off to the lab (... well kitchen ...) to do an experiment:
  • 100g fusilli (Lid's finest)
  • 500ml water
  • Bring to boil and boil for 10 minutes
  • Strain pasta
  • Weigh pasta
So what is this magic ratio? Well, this is not an exact science. Obviously al dente pasta is going to be lighter than over cooked. And it's probable that the topology of the pasta will have some influence on it's ability to retain water. Pasta topologies with lots of surface area and enclosed spaces (eg pasta shells) are going to have more water clinging to the surface. These are experiments for another day.

Anyhow, as for fusilli, the cooked weight of 100g of dry pasta after 10 minutes boiling was 208g.  So the weight ratio of dry pasta to cooked pasta is

1 :  2.1

Thursday, March 8, 2012

Visualizing diurnal temperature variation

I've been logging temperature from a ZigBee temperature sensor (located in my shed in the back garden) since about November last year. I thought it would be interesting to see the diurnal (daily) temperature variation over time. On the x-axis are days (from about 10 November 2011 to 7 March 2012) and on the y axis is time-of-day, with midnight being at the very top and bottom. Temperatures vary from about -3C (deep blue) to +15C (red). Periods where there is no data are gray. Sorry no legend or axis markings.

The chart was constructed by averaging temperatures into 15 minute bins (with about 5 samples per bin) and plotting each bin as a rectangle in SVG. I then used rsvg to convert to a PNG image.

For comparison here is a chart for the same time period of my home office temperature. The color scale is different, deep blue being 12C and red 22C.

Wednesday, February 29, 2012

Raspberry Pi now available from Farnell and RS Components

The Raspberry Pi Foundation just announced that they are partnering with Farnell and RS Components to manufacture and distribute their much anticipated 'Raspberry Pi' computer.

This is a sensible move. Else the charity would have to built up an Amazon style logistics operation from scratch -- a huge distraction from their mission to provide affordable and hackable computers for teaching, experimentation and kids to play with. I just managed to get my order in to Farnell. It wasn't easy. Their system was creaking under the load! (at 06:00 in the morning!)

A quick summary of what a Raspberry Pi has to offer: it's an electronics board about the size of a credit card (no casing is provided right now). At its heart is an ARM based Broadcom BCM2835 with integrated CPU, GPU and RAM clocked at 700MHz (and a pretty beefy GPU by all accounts!),  256MB RAM, HDMI video output, SD card for storage, 2 x USB for peripherals and external storage, 1 x 100Mbps ethernet port, 3.5mm jack for audio.  And for hardware hackers, a general purpose IO port (with hardware support for UART, SPI, I2C etc).

Cost? Well it's supposed to be about $35. I got mine for €31.87 + VAT (ie about €35) from Farnell. But that includes shipping: so it's a fair price. (BTW: here is a direct link to the Raspberry Pi  Model B at Farnell).

I look forward to my Pi in the coming days!

Update (1 Mar 2012): Just got an email from Farnell. Despite pushing in my order 13 minutes after the announcement I still didn't make the first batch of 10,000 units. ETA for my Pi: end April 2012 ☹

Monday, February 13, 2012

Getting started with the PIC24FJ64GB002 and PIC32MX110F016B

Microchip's new MPLABX development environment (free download) came out of beta a few weeks ago and the PICkit3 programmer/debugger was on special offer. So I figured this was as good a time as any to pick up a PICkit3 and to get a 'hello world' program running on the PIC24FJ64GB002 and PIC32MX110F016B samples I had in my parts drawer. Both are available in 28 pin DIP packages. (Can't beat DIPs and breadboards for quick prototyping!)

Programming the 8 bit PIC range such as 12F and 16F series with the PICkit2 programmer is simple: just connect the five ICSP pins MCLR, Vdd, GND, PGD, PGC from the programmer to the corresponding pins on the chip. Nothing else is needed. The chip can then be probed and programmed. I prefer to use the command line tool.  For example to verify that a chip was present:
pk2cmd -P
and then I'd upload myprogram.hex firmware with something like
pk2cmd -P -Fmyprogram.hex  -M -R
and apply power at 3.3V for testing (internal oscillator must be configured if there is no external crystal):
pk2cmd -P -T -A3.3

The PIC24FJ64GB002 and PIC32MX110F016B chips are not as straight forward. My naive attempt at treating them like the 8 bit PICs didn't work. I noticed searching Microchip's forums that others were have difficulty also. Like most problems relating to programming MCUs the answer is in the datasheet. You either need to read it cover-to-cover or fumble around and experiment. I'm still not sure which approach gets you there faster, but the latter is more fun :-)

There are a few important differences when compared to the older 12F and 16F series. These chips use a dual power supply. The CPU core runs at a lower voltage (usually 2.2V) and the on-chip peripherals run off a higher voltage (usually 3.3V). The core power can be configured to be supplied from an on-chip source or externally. The on-chip supply is selected by grounding the DISVREG pin and connecting Vcap to ground via a 10µF capacitor [Reference PIC24FJ64GB004 family datasheet, section 2 "Guidelines for getting started with 16-bit microcontrollers"].

Another important difference: the chip must be powered up, even for probing and programming. This is not necessary for the 8 bit PICs that I've been using. This one threw me for a bit: MPLABX with PICkit3 kept telling me no target was connected. Finally I twigged that I should have power applied:
right-click on the project and select 'Properties', select PICkit3, select the Power category from the pull down menu, select the desired voltage (usually something close to 3.3V) and check "Power target circuit from PICkit3" checkbox.

So to summarize the setup for both chips (electrically identical, just different names/functions on some pins).

  • MCLR (pin 1) to programmer MCLR
  • 10k resistor (approx) between MCLR and Vdd
  • Vdd (pin 13) to programmer Vdd
  • Vdd (pin 28) to programmer Vdd
  • Vss (pin 27) to programmer Gnd
  • Vss (pin 8) to programmer Gnd
  • PGED (choose from port 1,2 or 3) to programmer PGD
  • PGEC (chose from port 1,2 or 3) to programmer PGC. Must be the same port as that used for PGED.
  • Vcore (pin 20) to 10µF capacitor to Gnd
  • DISVREG (pin 19) to Gnd
  • Decoupling capacitor between pins 28,27
  • Decoupling capacitor between pins 13,8
  • MCLR (pin 1) to programmer MCLR
  • 10k resistor (approx) between MCLR and Vdd
  • Vdd (pin 13) to programmer Vdd
  • AVdd (pin 28) to programmer Vdd
  • AVss (pin 27) to programmer Gnd
  • Vss (pin 8) to programmer Gnd
  • PGED (choose from port 1,2 or 3) to programmer PGD
  • PGEC (chose from port 1,2 or 3) to programmer PGC. Must be same port as that used for PGED
  • Vcap (pin 20) to 10µF capacitor to Gnd
  • Vss (pin 19) to Gnd
  • Decoupling capacitor between pins 28,27
  • Decoupling capacitor between pins 13,8
You should now be able to program the chip with MPLAB and the PICkit3.

Finally the configuration bits must be set just right if you want the PICkit3 in-circuit debugger to work. The syntax used to defining the configuration bits seem seems to vary from compiler to compiler. For the PIC24FJ64GB002  this is what worked for me using the Microchip C30 C compiler version 3.30c:

        & GCP_OFF
        & GWRP_OFF
        & ICS_PGx2  // NB: set to correct channel
        & FWDTEN_OFF
        & WINDIS_OFF
        & FWPSA_PR32
        & WDTPS_PS8192);
        & FNOSC_FRCPLL
        & OSCIOFNC_ON
        & POSCMOD_NONE
        & PLL96MHZ_ON
        & PLLDIV_DIV2
        & FCKSM_CSDCMD
        & IOL1WAY_OFF);
        & SOSCSEL_IO
        & WUTSEL_FST
        & WPDIS_WPDIS
        & WPEND_WPENDMEM);
        & RTCOSC_LPRC
        & DSBOREN_OFF
        & DSWDTEN_OFF);

The configuration bits I found that worked for the PIC32MX110F016B (Microchip C32, v2.02) are:

#pragma config FPLLODIV = DIV_1 
#pragma config FPLLMUL = MUL_20 
#pragma config FPLLIDIV = DIV_2 
#pragma config FWDTEN = OFF 
#pragma config FPBDIV = DIV_1 
#pragma config WDTPS = PS1 
#pragma config FCKSM = CSECME 
#pragma config OSCIOFNC = OFF 
#pragma config POSCMOD = HS 
#pragma config IESO = ON 
#pragma config FSOSCEN = OFF 
#pragma config FNOSC = PRIPLL 
#pragma config CP = OFF
#pragma config BWP = OFF 
#pragma config PWP = OFF 
#pragma config ICESEL = ICS_PGx2 // NB: set to correct channel
#pragma config DEBUG = OFF

Note that both of these chips have several programming ports. Any can be used for programming. However for debugging you must specify which port is being used in the configuration bits.

Saturday, January 28, 2012

Interfacing Sensirion temperature / humidity sensors to the XBee using 'bit-banging' technique

This article documents an technique to interfacing digital sensors such as the Sensirion SHT21 to an XBee Series 2 radio module running ZigBee end device firmware without using any additional hardware components. The same technique should apply to similar digital sensors.

The XBee Series 2 radio modules from Digi are ideal for developing prototype and low production run wireless sensors. The modules are pre-certified in US, Europe and other regions eliminating the requirement to under go RF certification. In addition to a UART port for communications the XBee module also provides  several IO lines which can be configured in digital input, digital output, ADC and PWM modes.

Digi provide several protocol options including DigiMesh (their own proprietary protocol) and ZigBee. For interoperability with other vendors equipment and security I use ZigBee. Each protocol family has several firmware options which can be loaded on the module: usually Coordinator, Router and End Device.  For battery powered applications the End Device firmware must be used to achieve reasonable battery life. In this mode the module spends most of its time asleep using less than 1µA current. Periodically it will wake (briefly consuming 15 - 40mA), query its parent router to see if there are any waiting packets. If there are none it will go back to sleep. In this mode a set of AAA batteries can last a year or more.

Analog sensors can be coupled to XBee IO lines in ADC mode (10 bits resolution ranging from 0 - 1.2V) and read remotely using Digi's remote sampling API. However analog sensors will probably require signal conditioning circuitry (op amps, filters etc) to make the best use of the ADC voltage range. This conditioning circuitry is likely to require individual calibration to achieve good results.
The headache of analog design can be conveniently side-stepped by using a sensor with a digital IO interface. Sensirion produce a range of temperature / humidity sensors with an impressive resolution (14 bit temperature, 12 bit humidity) and accuracy (less than ±0.3°C for some versions). These sensors talk I2C protocol (or a variant of I2C). But there is a catch: the XBee firmware from Digi does not directly support I2C.

A common solution is to use a low cost MCU as a bridge between the sensor and the XBee's UART. The MCU waits for a command from the network (via the XBee's UART), queries the sensor and relays the result back to the UART for transmission on the ZigBee network. By monitoring the XBee's SLEEP pin the MCU can also spend most of its time in a low power sleep, waking only when the XBee wakes.
There is an alternative... a common solution in a situation where there is no direct hardware support for a serial protocol: "bit banging". Each of the XBee's IO lines can be set high, low or in high impedance (input) state by remote control: everything needed to realize the I2C and SPI protocols. But there is a catch: unlike a MCU bit banging its own IO lines which can happen at clock speeds exceeding 100kHz, remotely bit banging XBee IO lines is a very slow process. Fortunately the Sensirion sensor datasheets do not place any lower bound on the protocol clock speed. So a clocking speed of just 1Hz will work just as well as 100kHz... it's just going to take a while to complete a query.
This is what the test setup looks like. A SHT75 sensor is at the bottom of the photo. A XBee board (the Grove XBee Carrier board from SeeedStudio) with DIO1 and DIO2 connected to the SHT75 clock and data pins respectively. The 3.3V power supply from the board powers the sensor.

Implemenation details:

The details of how to communicate with an XBee module are beyond the scope of this article. A good introduction to the topic is the book Building Wireless Sensor Networks from O’Reilly.

It’s important to note that the I2C protocol requires pull-up resistors on the clock and data lines. Fortunately the XBee has configurable 30k internal pull-up resistors for digital inputs which are configured with the ATPR command. They are enabled by default.

A sensor query begins by first issuing a remote ATIR command to the XBee to initiate frequent IO sampling. I found 100ms sampling period gave good results. ATIR must be followed by ATAC (commit) for the change to take effect. The sampling will keep the (normally sleeping) XBee awake. Also each time a sample packet is transmitted the acknowledgement will let the XBee know if there are incoming packets waiting for it. So it helps keep packet latency relatively low. Samples packets comprise the high/low state of any digital input pins and the ADC value of any pins configured as ADC.

Before proceeding to the next step wait until the first sample arrives. This may take several seconds depending on where the XBee is in its sleep cycle. When it starts to transmit samples you know it is awake and will stay awake.

Now issue ATD commands to set the clock and data lines into the necessary state. For example, ATD13 sets DIO1 into input (or high impedance) state, ATD14 sets DIO1 into a output low (0V) and ATD15 sets DIO1 into output high (3.3V). Any ATD command will need to be followed by an ATAC before the change will take effect.

To read a data line during the read phase of an I2C conversation, wait for a IO sample to arrive after the low-to-high transition of the clock.

Finally when the I2C conversation is complete set ATIR=0 to stop automatic sampling, followed by ATAC. On reception the XBee should go back to sleep.

I found that a short delay (about 100ms) between each AT command was required for acceptable results.

This is an oscilloscope trace of a temperature query of a SHT75 sensor. The SHT7x and SHT1x range use a protocol similar to, but not compatible with I2C. Note the time base is 3.7 seconds per division! The clock is in green, the data in yellow. The blue trace is connected to a third pin which was used for debugging (to help separate out parts of the conversation). Here it is set high while writing out the command (0x03, read temperature) .

The SHT21 and SHT25 are more recent temperature / humidity sensors from Sensirion. These sensors use standard I2C protocol. This trace is a I2C read temperature query to a SHT21 sensor:

Some results:

This is a chart of a few hours of data from a SHT21 and SHT75 connected to an XBee using this technique. For comparison a third wireless sensor, a Digi XS-Z16-CB2R sensor is included. All three sensors were enclosed in an insulated polystyrene box to ensure that all sensors were reading the same temperature and humidity. I've also included the supply voltage (a handy feature of the XBee modules).

One slightly disappointing result is that there are quite a few failed queries (compare the number of  CB2R samples in blue to those in red and green). It seems that with my current implementation of this bit banging technique the success rate is about 66%. I believe this can be significantly improved with some changes to the implementation.


Bit banging serial IO protocols such as I2C, SPI with the XBee IO lines under remote control is feasible. A battery powered sensor unit can be constructed with nothing more than a XBee, a digital sensor (and some means to physically link the sensor to the XBee), a battery holder and suitable housing.  No other components are required.

However there are some down sides: it takes a long time to make a measurement (10 - 30 seconds). During this period the XBee is awake (consuming 15 - 30mA). This is not a problem if  AA or AAA cells are used and the measurements are infrequent (eg once per hour). As currently implemented, reliability is far from perfect (2 out of 3 queries succeed) however I believe this can be improved with some tweaks to the implementation. Also over 100 ZigBee packets are required to complete one measurement: this could be a problem on a congested network.


Unfortunately the test setup is too complex to package up a simple self contained ZIP file to implement this technique. However here is the source code of the main Java class file which implements the necessary XBee AT commands.

 * Implement temperature and humidity queries to a SHT71 and SHT75 sensor by bit banging
 * XBee IO lines. 
 * @author Joe Desbonnet,
public class SHT7x {

 public static final int CMD_TEMPERATURE_READ = 0x03;
 public static final int CMD_HUMIDITY_READ = 0x05;
 private XBeeSeries2 xbee;
 private int clockPin;
 private int dataPin;
 // delay=100, sampleRate=250 does not work
 // delay=150, sampleRate=250 does not work (reliably)
 // delay=200, sampleRate=250 works
 // delay=180, sampleRate=50 works
 // delay=180, sampleRate=100 works

 // delay between sending each packet to the NIC for transmission
 private int delay = 180;
 // ms between each sample
 private int sampleRate = 100;

  * @param xbee XBee proxy object
  * @param clockPin XBee pin used to implement SCK (0 = DIO0, 1 = DIO1 etc)
  * @param dataPin XBee pin used to implement SDA (0 = DIO0 .. etc)
 public SHT7x(XBeeSeries2 xbee, int clockPin, int dataPin) {
  this.xbee = xbee;
  this.clockPin = clockPin;
  this.dataPin = dataPin;

  * Implement short delay. Usually used to space ZigBee packets apart.
 private void delay() {
  try {
  } catch (InterruptedException e) {
   // TODO Auto-generated catch block

 private void clockHigh() throws IOException {
  byte[] param = new byte[1];
  param[0] = XBeeSeries2.HIGH;
  xbee.atCommand("D" + clockPin, param);

 private void clockLow() throws IOException {
  byte[] param = new byte[1];
  param[0] = XBeeSeries2.LOW;
  xbee.atCommand("D" + clockPin, param);

 private void dataHigh() throws IOException {
  byte[] param = new byte[1];
  // Data high is achieved by high impedance state (HIGH_Z) ie digital
  // input mode
  param[0] = XBeeSeries2.HIGH_Z;
  xbee.atCommand("D" + dataPin, param);

 private void dataLow() throws IOException {
  byte[] param = new byte[1];
  param[0] = XBeeSeries2.LOW;
  xbee.atCommand("D" + dataPin, param);

  * Reset communications with sensor. Required if the previous query did not complete.
  * @param xbee
 public void resetComms() throws IOException {

  int i;


  // Pulse clock 9+ times while data high
  for (i = 0; i < 10; i++) {

  * Start sequence.
  * @throws IOException
 private void startSequence() throws IOException {
  * A command is 8 bits (MSB first) followed reading an ack bit from the device. 
  * In this implementation I ignore the result of the ack bit (but there
  * still must be a 9th clock pulse). Bits are written by setting the
  * data pin to either 0V (logic 0) or high impedance (logic 1). The data is
  * read by the sensor during a low to high transition of the clock signal.
  * @param command
  * @throws IOException
 private void sendCommand (int command) throws IOException {
  int i;
  boolean lastBit = true;
  boolean currentBit = false;

  // MSB first
  for (i = 0; i < 8; i++) {
   currentBit = ((command & 0x80) != 0);

   // Only change data pin if there is a change. This reduces the number
   // of ZigBee packets transmitted.
   if (currentBit != lastBit) {
    if (currentBit) {
    } else {
    lastBit = currentBit;

   // Pulse clock
   command <<= 1;

  // If data currenly low bring to high_z/input mode
  if (currentBit == false) {

  // I don't bother to read the ACK bit, but the clock
  // must still be pulsed for it. 
  // don't bother sampling data pin -- will assume all ok


  * Reference datasheet §4.3.
  * @return Temperature in °C
  * @throws IOException
 public float readTemperature() throws IOException {
  int v = makeReading(CMD_TEMPERATURE_READ);
  return -39.7f + 0.01f * (float) v;  
  * Reference datasheet §4.1.
  * @return Humidity as RH%
  * @throws IOException
 public float readHumidity() throws IOException {
  int v = makeReading(CMD_HUMIDITY_READ);
  return (float)(-2.0468 + 0.0367*(double)v - 1.5955e-6*(double)v*(double)v);
  * Reset comms, send start sequence, command and read 16 bits of data.
  * @return
  * @throws IOException
 private int makeReading (int what) throws IOException {

  int i;

  // Configure XBee to send frequent IO samples. This has two important functions.
  // First it keeps the XBee end device awake. Also an end device transmitting a 
  // packet (which must go via its parent) has the side effect of polling the 
  // parent for any incoming packets. So frequent transmission also means low
  // latency in receiving packets.
  byte[] params2 = new byte[2];
  params2[0] = (byte) (sampleRate >> 8);
  params2[1] = (byte) (sampleRate & 0xff);
  xbee.atCommand("IR", params2);
  // Now wait for the first sample to arrive before proceeding. At this point we'll
  // know the end device awake.
  long t0 = System.currentTimeMillis();
  while (xbee.getLastIOSampleTime() < t0) {
  // XBee End Device should now be awake and responsive

  // The last query may not have completed leaving the communications in an undefined
  // state. Reset communications to a known state.

  // Start sequence ref §3.2.

  // Send 8 bit command (and read ack bit)

  // Wait for measurement to complete. We could poll the DATA line. It will be pulled
  // low by the sensor when the reading is complete. However the overhead of doing
  // this makes it not worth the effort.
  try {
  } catch (InterruptedException e1) {
   // ignore

  // Read 16 bits

  int v = 0, sample;
  for (i = 0; i < 16; i++) {

   v <<= 1;


   // wait for IO sample
   t0 = System.currentTimeMillis();
   while (xbee.getLastIOSampleTime() < t0) {
    try {
    } catch (InterruptedException e) {
     // ignore
   // sample = xbee.getIOSample();
   sample = xbee.getLastIOSample();
   if ((sample & 0x04) != 0) {
    v |= 1;

   // write ack bit
   if (i == 7 || i == 15) {
    // write ack bit (0)

  // Return end device to normal sleep pattern by disabling sampling (ie by
  // setting sample period to 0).
  params2 = new byte[2];
  params2[0] = 0;
  params2[1] = 0;
  xbee.atCommand("IR", params2);

  return v;