remote data logging with Arduino – Bringing it all together

Remote data logging with Arduino – Bringing it all together…

Now that we have all the pieces, to tie them all togethers we need to configure our Mango M2M server to accept the data from the base unit and do some additional computations on top of those to drive more meaningful information vs just data.

You may want to familiarise yourself with key mango m2m concepts here.

The way i have setup my M2M box is to log raw data point from the arduino, and then to each of those apply the necessary calculations to arrive at the needed information. To do this i have setup two data sources

HTTP receiver

Contains all data points which are updated by the java application thats acts like the bridge between the base arduino and the mango m2m instance. To set this up :-

1. Add a HTTP Receiver Data source

MangoDataSources

Continue reading

Construction and programming of remote Arduino transreceiver for remote data logging and control

I will divide this post  on remote Arduino transreceiver into following sections :-

  1. Construction of Unit
  2. Logging of data from this unit to the Mango M2M server via the base unit.
  3. Controlling remote devices via algorithms which run on the Mango M2M server.

First, construction of the remote Arduino transreceiver Unit

  1. Construction wise it is fairly similar to the earlier one, only difference being the provision to attach various analog and digital sensors
  2. Programming wise, again similar to the earlier on with following key differences :-
  • Additional code to read sensor value
  • Additional code to transmit sensor reading to the base unit.

Remote Arduino with nRF24L01+ transreceiverRemote Mega Arduino with nRF24L01+ transreceiver

Construction and programming of base unit (arduino transreceiver), connecting to the M2M box

Now that your Mango M2M box is setup, the next step is to construct an arduino transreceiver and connect it to this box.

Hardware needed:-

1. Any Arduino or clone

2. nRF24L01+ wireless transceivers

3. USB cable to connect the Arduino to the Mango M2M server

4. All the usual electronic DIY construction -  connecting wire, multipurpose PCBs, cutter, soldering iron etc etc…

Steps involved:-

1. Construct the Arduino transceivers

2. Programme the Arduino transceivers to work as the base unit for our data logging setup

#define DEBUG 1
#include <avr/pgmspace.h>
#include <RF24Network.h>
#include <RF24.h>
#include <SPI.h>
#include "nodeconfig.h"
#include "printf.h"
#include <Wire.h>
#include <Adafruit_BMP085.h>
#include <util.h>
#include <math.h>
#include "nRF24L01.h"
#include "printf.h"
#include <DHT22.h>
#include <DHT11.h>
#include <OneWire.h>
#include "Arduino.h"
#include "pins_arduino.h"
#include <avr/wdt.h>

#define SAMPLES 4096

#define UNIT_GF_SGROOM 0
#define UNIT_1ST_DROOM 1
#define UNIT_1ST_MBROOM 2
#define UNIT_GF_INVERTOR 3
#define UNIT_FF_INVERTOR 4
#define UNIT_GF_REAR 5
#define UNIT_GF_SGROOMCONTROL 6
#define UNIT_ROOF_CONTROL 7
#define UNIT_R1 9

#include "Arduino.h"

#define sensorUnit 7

//int node_nos[10] = { 00, 01, 02, 03, 04, 05, 011, 012, 021, 031 };

//Temperature chip i/o
#define DHT11PIN 7    // what pin we're connected to

// Universal variables ///////////////////////////////////
const float a = 17.271;
const float b = 237.7;

float outsideDryBulbTemperature;
float outsideWetBulbTemperature;
float outsideRH;

unsigned long lastTxTime;
const unsigned long sensorLoopTime = 10000;
uint16_t this_node; // Our node address
String serialInput; //String for Serial Input

// Delay manager to send pings regularly
const unsigned long interval = 3900; // ms
unsigned long last_time_sent;

// Prototypes for functions to send & handle messages
bool send_T(uint16_t to);
bool send_N(uint16_t to);
bool send_D(uint16_t, char, int, double);

void handle_T(RF24NetworkHeader& header);
void handle_D(RF24NetworkHeader& header);
void handle_N(RF24NetworkHeader& header);
void add_node(uint16_t node);

// Array of nodes we are aware of
const short max_active_nodes = 9;
uint16_t active_nodes[max_active_nodes];
short num_active_nodes = 0;
short next_ping_node_index = 0;

//RF24 radio(9,10); //non mega
RF24 radio(46,48); //mega
RF24Network network(radio);

Adafruit_BMP085 bmp;
dht11 DHT11;

#if sensorUnit == UNIT_ROOF_CONTROL
unsigned long last_time_pumpOn = millis();
const long timeBetweenPumpOnAtMin = 1800000; //1 mins - set this at the timeon at min temp
const long minTempPumpOn = 21;
//const long tempDifferential = 45;
long timePumpOnFor = 30000; // set this at the timeon for at min temp
#endif

void setup(void) {

  //  wdt_enable( WDTO_8S); //enable watchdog
  wdt_reset();
  wdt_disable();

  Serial.begin(57600);
  //store current time
  lastTxTime = millis();

  printf_begin();
  printf_P(PSTR("Sensor Mesh Network\n"));

  // Pull node address out of eeprom, Which node are we?
  this_node = nodeconfig_read();

  // Bring up the RF network
  SPI.begin();
  radio.begin();
  network.begin(/*channel*/102, /*node address*/this_node);

  // unit specific setups

  transmit_startup_information(0); //tx startup information to node 0

  delay(10000);
  pumpOnOnStartup(10000);

}

void loop(void) {
  // Pump the network regularly
  bool ok;
  network.update();

  // Is there anything ready for us?
  while (network.available()) {
    // If so, take a look at it
    RF24NetworkHeader header;
    network.peek(header);

    // Dispatch the message to the correct handler.
    switch (header.type) {
    case 'D':
      handle_D(header);
      break;
    case 'N':
      handle_N(header);
      break;
    case 'C':
      handle_C(header);
      break;
    default:
      printf_P(PSTR("*** WARNING *** Unknown message type %c\n\r"), header.type);
      network.read(header, 0, 0);
      break;
    };
  }
  wdt_reset(); //reset watchdog
  // Send a ping to the next node every 'interval' ms
  unsigned long now = millis();
  if (now - last_time_sent >= interval) {
    last_time_sent = now;

    // Who should we send to?
    // By default, send to base
    uint16_t to = 00;

    // Or if we have active nodes,
    if (num_active_nodes) {
      // Send to the next active node
      to = active_nodes[next_ping_node_index++];

      // Have we rolled over?
      if (next_ping_node_index > num_active_nodes) {
        // Next time start at the beginning
        next_ping_node_index = 0;

        // This time, send to node 00.
        to = 00;
      }
    }

    wdt_reset(); //reset watchdog

    // Normal nodes send a 'T' ping
    if (this_node > 00 || to == 00) {
      //send_T(to);
      //Serial.print("in read and tx");
      read_and_transmit_sensor_data(to);

    }
    // Base node sends the current active nodes out
    else {
      //Serial.print("in send N");
      ok = send_N(to);
    }
    // Notify us of the result
    if (ok) {
      //printf_P(PSTR("I: %lu: APP Send ok\n"), millis());
      Serial.println(".");
    } 
    else {
      printf_P(PSTR("\nI: %lu: Send failed\n"), millis());

      // Try sending at a different time next time
      last_time_sent -= 100;
    }
  }

  // Listen for a new node address
  //nodeconfig_listen();
  read_serial();

  wdt_reset(); //reset watchdog

}

3. Connect the base unit to the mango m2m server using this java application.

import java.io.InputStream;
import java.io.OutputStream;
import gnu.io.CommPortIdentifier;
import gnu.io.SerialPort;
import gnu.io.SerialPortEvent;
import gnu.io.SerialPortEventListener;
import java.util.Enumeration;

public class SerialInterface implements SerialPortEventListener {
    SerialPort serialPort;

//     private static final String PORT_NAMES[] = {
//    "COM21" // Windows
//     };

    private static final String PORT_NAMES[] = { "/dev/ttyUSB0" // Windows
    };

    /** Buffered input stream from the port */
    private InputStream input;
    /** The output stream to the port */
    private OutputStream output;
    /** Milliseconds to block while waiting for port open */
    private static final int TIME_OUT = 2000;
    /** Default bits per second for COM port. */
    private static final int DATA_RATE = 57600;

    public void initialize() {

	CommPortIdentifier portId = null;
	Enumeration portEnum = CommPortIdentifier.getPortIdentifiers();

	// iterate through, looking for the port
	while (portEnum.hasMoreElements()) {
	    CommPortIdentifier currPortId = (CommPortIdentifier) portEnum.nextElement();
	    for (String portName : PORT_NAMES) {
		if (currPortId.getName().equals(portName)) {
		    portId = currPortId;
		    break;
		}
	    }
	}

	if (portId == null) {
	    System.out.println("Could not find COM port.");
	    return;
	}

	try {
	    // open serial port, and use class name for the appName.
	    serialPort = (SerialPort) portId.open(this.getClass().getName(), TIME_OUT);

	    // set port parameters
	    serialPort.setSerialPortParams(DATA_RATE, SerialPort.DATABITS_8, SerialPort.STOPBITS_1,
		    SerialPort.PARITY_NONE);

	    // open the streams
	    input = serialPort.getInputStream();
	    output = serialPort.getOutputStream();

	    // add event listeners
	    serialPort.addEventListener(this);
	    serialPort.notifyOnDataAvailable(true);
	}
	catch (Exception e) {
	    System.err.println(e.toString());
	}
    }

    public synchronized void write (String s)
    {

	try {
	    output.write(s.getBytes());  
	    System.out.print("Wrote to Serial port :");
	    System.out.println(s);
	}
	catch (Exception e) {
	    // TODO Auto-generated catch block
	    System.out.print("********** Failed to Serial port : ");
	    System.out.println(s);
	    e.printStackTrace();
	}
    }

    /**
     * This should be called when you stop using the port. This will prevent port locking on platforms like Linux.
     */
    public synchronized void close() {
	if (serialPort != null) {
	    serialPort.removeEventListener();
	    serialPort.close();
	}
    }

    /**
     * Handle an event on the serial port. Read the data and print it.
     */
    public synchronized void serialEvent(SerialPortEvent oEvent) {

	int data;
	byte[] buffer = new byte[1024];

	if (oEvent.getEventType() == SerialPortEvent.DATA_AVAILABLE) {
	    try {

		int len = 0;
		while ((data = input.read()) != '\n') {
		    if (data &gt; 0) buffer[len++] = (byte) data;
		}
		// System.out.print(new String(buffer,0,len));
		// System.out.print(" ");

		// ----------
		// Displayed results are codepage dependent
		// System.out.print(new String(chunk));

		//
		// String Serialinput = new String(chunk);
		// String[] values = Serialinput.split(",");

		String valueStr = new String(buffer, 0, len);

		String[] PayloadType = valueStr.split(":");
		//System.out.print(PayloadType[0]);
		//System.out.print(",");
		//System.out.print(PayloadType[1]);

		if (PayloadType[0].equals("D")) {
		    System.out.print("Sensor Data:");
		    String[] values = PayloadType[1].trim().split("=");
		    System.out.print(values[0].trim());
		    System.out.print(" = ");
		    System.out.println(values[1].trim());

		    if (values.length == 2) {
			// if (len &gt; 2){

			mangoHttpPost.sendValue(values[0], values[1]);

		    }

		}
		else {
		    System.out.print("Infomation - ");
		    System.out.print(valueStr);
		}

		//
	    }
	    catch (Exception e) {
		System.err.println(e.toString());
	    }
	}
	// Ignore all the other eventTypes, but you should consider the other ones.
    }

}

<– Previous           Next–>

Mango M2M setup using Ubuntu server

Mango M2M setup on a Ubuntu server is fairly straightforward, so i won’t go into too much details on the basic steps involved other than direct you to appropriate links :- Mango M2M setup

1. Install Linux (I used the Ubuntu server version) on any desktop or laptop with a working USB connection. We will use USB connection later to connect to the Arduino base transreceiver. Detailed Ubuntu server installation instructions here

2. Install Tomcat and Mango M2M application. Detailed instructions on Mango M2M setup here

3. Install MYSQL on your Linux server. Detailed instructions here

4. Configure Mango M2M to use MYSQL vs the default debian. Detailed instructions here

The last step is especially important since otherwise you are likely to get very poor  performance from your Mango M2M server once the data size grows.

Home remote data logging and intelligent control with Arduinos and Mango M2M

 

Hardware for remote data logging via Arduino and Mango M2M server

To keep a track of various environmental parameters and use them for intelligent home control, sometime back I set off on this journey to setup a remote data logging system using Arduinos and open source M2M software.

First, the hardware used :-

  1. Arduinos – pretty much any model will do. At a minimum you need two, one for the base receiver and the other for the remote data capture / control unit.
  2. Mango M2M. I got a second hand laptop for Rs.3000/-, converted it into a linux server with Ubuntu Server and installed Mango M2M on it.
  3. Sensors – humidity, temperature, pressure, ultrasound etc etc

Continue reading

Humidex Calculator using Arduino

Given the high cost of running air conditioners, was kicking around the idea of being able to automatically determine when is the right time to use a desert cooler (aka evaporative cooler or swamp cooler) vs an air conditioner. Can across some papers on Humidex / Heat Index which could be used to determine the same, therefore decided to implement a Humidex Calculator using Arduino.

First some theory about Humidex :-

The humidex is an index number used to describe how hot the weather feels to the average person, by combining the effect of heat and humidity. The humidex is a unit-less number based on the dew point, but it is equivalent to dry temperature in degrees Celsius. For example, if the temperature is 30 °C (86 °F), and the calculated humidex is 40, then it indicates the humid heat feels approximately like a dry temperature of 40 °C (104 °F).

According to the Meteorological Service of Canada, a humidex of at least 30 causes “some discomfort”, at least 40 causes “great discomfort” and above 45 is “dangerous”.

Continue reading