How I Visualize The Sensors

This is basically the part 2 or extension of my earlier write-up, How I Monitor My Home Temperature. If you haven’t read that, or don’t yet have your sensors all put together this article may be somewhat foreign. The article here focuses on getting the output of your sensors to display in a graphical format so you can visualize the data over different time periods. It’s the polish on the project.

Visualization requires some data logging with an Internet of Things (IoT) aggregator or your own xSQL and Server-Side scripting web page such as PHP, ASP or whatever. In my case, it’s PHP and MySQL, a very famous pairing. The idea is to have your sensors polled at regular, frequent intervals and then to get that poll value stored in a database with a timestamp. That’s really the whole thing. From this seemingly simple plan you can see trends, compare time periods or just log the data to see what the seasons are like. I’ll be focusing on the display of the data so if you did read the aformentioned temperature sensor article you’ll know I have a multitude of Maxim (Dallas) DS18B20 digital temperature sensors scattered about my home. This article includes a power meter displaying the near-realtime Kilo-watts being consumed in my home. I’ll link to how I did that in another article.

How you do this depends on your expectations. I have all the expectations in the world but don’t know which way is best for me to go. If you feel the same hopefully this will clear some of the blurriness up.

So Let’s Do This In Reverse

There are several ways to graphically represent your sensor data.  Real-time bar, line and area graphs plus gauges.  These work a bit differently on different platforms, but you should have an idea of the way you want your data to appear.

Google Gauges

These fall under the real-time type of displaying your data.  They are highly customizable and can be tweaked using config variables to control the UI and the appearance of the gauges.  I use these on the real-time page so at a glance I can see all of the temperatures  and any other statistics.  Out of the box they only display static values, so you query a database, get your value back and then insert it into the code for the gauge to display.  That’s great but it makes you refresh the page to update the sensor data.  There are a lot of people that have written instructions for using AJAX for asynchronous graceful refreshing.  For me, none of those worked.  Follow this link to see how I was able to get the gauges to update automatically every 15 seconds with AJAX and JSON using Thingspeak and Xively.  Here is a real example using my data.  Every minute I post the usage for the previous minute, and every 15 seconds the gauge polls for an update.  Using this system the gauge is always within 15 seconds of the most recent data post.  Follow this link to see how this works and for some sample code for Thingspeak, Xively and Open.Sen.Se.

Http iframes are not shown in https pages in many major browsers. Please read this post for details.

HanSolo Steel Series Canvas

These are pimp. The degree of flexibility is impressive. Nearly every feature is tweakable and can be set to complement your CSS. They read data in a similar way to Google Gauges but the code is a tad trickier. There are at least 30 different designs in the Steel Series portfolio, and each has a unique styling. I was able to incorporate these in place of every Google Gauge I was running and was able to create a few dedicated LCD-type gauges to accomplish things Google could not. For example, my power monitoring page has traditional gauges like the one to the right for kW in near-real-time, kW displayed as a running tally for the current hour, the day and current month. In addition to those I added LCD display gauges to forecast the kWh usage for the same periods, now, current hour, current day and month-to-date, plus another calculating the estimated electricity cost for the month. This one takes into account the price per kW-hours charged in each of the four billing rate tiers. It’s not perfect, the billing days in a given period fluctuates, almost monthly, but it’s close.

Http iframes are not shown in https pages in many major browsers. Please read this post for details.

Line Graphs

Line graphs share most of their attributes with bar and area graphs, so to keep it simple I’ll just show line graphs. To the right is a small example of what you can get out of this. When they are larger they can shine. This particular example is from Thingspeak and is embedded with some subtle color changes from the standard website versions, and I’ve rounded the corners 40px with a little CSS. This one will update in real-time like the gauge. Some of them like the ones on Xively allow you to specify a time period and to change it only takes a click. Google charts can give you some amazing visualizations, and some can even be AJAX-enabled for streamlined refreshing. Follow this link for write-ups on how to do graphing on different IoT platforms.

Http iframes are not shown in https pages in many major browsers. Please read this post for details.

OK, Time To Spew Some Opinions

I have tested the bejeezus out of the various IoT platforms.  Most of them do a good job, some do a great job, and one doesn’t do much (for me at least).  It’s worth noting a few things at this point.  Number 1, I am not paid or compensated in any way to endorse one over another.  This is pure trial and error with my testing.  Number 2, I seem to keep beating on KSDuino.  It’s not that it’s a particularly bad service, it just isn’t as polished as the other patforms are.  The guy running that site has done a good job, kudos to him.  One of the best features of it are the UDP updating.  Super fast.  Number 3, what one platform does great another may not, but just the same, the one that does better at a given function may be bested by the same one it beat in other categories.  Confusing, right?  Platform 1 may be best at function 1 but Platform 2 could be better at function 2.  In other words, your needs will dictate which platform performs best for your needs.  For me, that means complicating things a bit by integrating several of them together to make a super, best-of-breed IoT.  I do things the hard way every turn of the way.

Here is a quick summary of features for the primary platforms:

 Thingspeak.comXively.comOpen.Sen.SePHP snd MySQL
CostFreeFreeFreeFree if hosted privately, may cost if hosted
Graphing OptionsGreat. Excellent embedding options. Gracefully resize with simple parameters. Full auto refresh enabled.Great but the nice views are locked to the website. Embedding options are complicated and look far less appealing. Based on Rickshaw. Embeddable graphs do not auto-refresh.Great but the embedding options for some reason done scale well. If you can incorporate the generated sized they look nice. Full auto refresh enabled.The sky is the limit. rGraph and Google charts & gauges provide a nice set of simple frameworks. Fully embeddable and scalable. Auto refresh for some is simple, others just don't seem possible.
Updating OptionsHTTP GET was the easiest. API well documented. Arduino sketches are provided to assist with the coding. Updating from Arduino takes between 3 and 6 seconds to complete. Offers a way to buly update by CSV file.API well documented but still more complicated code base but was the most streamlined of the mix reviewed. Updaring from Arduino takes 1.4 seconds nearly everytime. Can be tricky to get the client to play nicely with other calls to EtherntClient().Gave up on this one. It was unreliable and caused several update intervals to not report. Hard to say what happened.Simple to do with a single PHP script. Do an HTTP GET from withing Arduino with the vars as URL parameters. PHP parses the URL and creates SQL statements to update the tables. Can be as hard as you want it to be.
AccessibilityFeeds can be retrieved in XML, JSON & CSV. API is helpful.Feeds can be retrieved in XML, JSON, CSV and PNG. API is helpful.Not immediately obvious, but the API is available. JSON only.SQL queries, AJAX, JSON, XML, whatever you want.
ReliabilitySuspect. Several time the website will display a "cannot display" warning. This isn't a standard 4XX error, so the web is up but their backend is suffering. Sometimes there are periods where data is not stored. Related?Bulletproof. It just works.Web UI can be sluggish but uptime is excellentYMMV
External IntegrationNoneNoneExtensive. Can import from Xively in realtime. Can repeat received data out to any URL as an HTTP GET or POST with URL vars. Very powerful.How good are your PHP skills?
Impossible TriangleGood speed
Some control
Great speed
Some control
Unparalleled control
Good scope
Unparalled scope AND control, but you have write the code yourself

Impossible Triangle

Impossible Triangle

So How Do I Use These Platforms?

I’m actually using all four of these in conjunction with each other in one manner or another.  There’s a famous triangle called  a “Triple Constraint” or “Iron Triangle” that most casually refer to as the “Impossible Triangle” because of it’s unobtainable trifecta.  In it are three goals which can rarely be accomplished together.  I decided speed was a critical factor in how quickly the data was jettisoned off to the platform.  That left scope and control, and I could only have one.  Xively doesn’t have much of the scope I was after since their graphing is somewhat limited, but they did have good control with the way the data was reliably posted and the service seems to have excellent uptime.  So since Xively has the best combination of constraints (for me) I post updates there.  From there, Open.Sen.Se monitors the Xively feed for new data and immediately fetches and imports it into it’s database.  From there, Open.Sen.Se acts on the new data and a Xively trigger performs an HTTP POST to my PHP server.  I sanitize and run SQL inserts and from there, and afterward, the PHP script does an HTTP POST to Thingspeak and Open.Sen.Se.  All of this happens in about 5 seconds.  It has a lot of moving parts but until I can settle on a service platform this saves me from having to run the updates natively from the Arduino.  Doing that took about 15 seconds to complete, every minute.  I was missing a lot of main loop executions in that time, which is why I use Xively as it is today.  I have a Raspberry Pi that I’ll look into setting up as the central datastore, and from there POST the JSON out to the cloud IoT’s.  I’m fairly confident the RasPi can handle that load.  If not, there’s always the overclock option or a BeagleBone Black.

Here’s my functional data flow:

Leave a Comment

Your email address will not be published. Required fields are marked *