Contact sales

How to add a long-lived anomaly detection system to an air quality monitor using Particle’s cloud ledger

Mithun Das article author avatarMithun DasAugust 01, 2024
How to add a long-lived anomaly detection system to an air quality monitor using Particle’s cloud ledger

The Problem

To help people stay safe during fire season, I recently built an air quality monitoring system that sends air quality, co2 & dust readings at certain intervals and sends alerts when readings are beyond a defined threshold (you can read about it here). While it’s been doing its job, sending out alerts based on individual readings, we’ve noticed a problem: false alarms. That system notifies you when a reading is unusual, but since it’s currently only using a singular reading to make that determination, it does occasionally make an error. These false alarms not only waste time but can erode trust in the system, potentially rendering it ineffective when it matters most.

The Solution

We’re going to transform our alert system into a smart, long-term anomaly detection platform. By analyzing data over extended periods, we’ll dramatically increase accuracy and reliability. The key to this upgrade? Particle Ledger.

Particle Ledger allows us to store and analyze data in the cloud per-device, per-product, or per owner account, giving our devices a persistent memory. This means our air quality monitor won’t just react to single readings, but will make informed decisions based on historical trends.

Particle support three types of Ledgers:

  • Cloud Ledger- Stores information in the cloud without syncing to the edge.
  • Device to Cloud Ledger – Device storage that syncs automatically to the cloud
  • Cloud to Device Ledger – Set data in the cloud that will sync to devices

In this tutorial, we’ll focus on using Cloud Ledger to enhance our existing system.

We’ll build upon the previous project, Particle Boron publishes raw sensor data to the topic “aqas-env-raw”.

Ready to make your air quality monitor smarter and more dependable? Let’s get started!

We like to trigger alerts when the co2 level is more than 1000 for the last 12 readings, which lasts one hour considering data is sent every 5 minutes from the device to the particle cloud.

We are going to create two ledgers

  • One for threshold configuration
  • One for time series data 

Access the ledger from the left menu.

Click on “Cloud Ledger” and then enter the name of the ledger as “aqas-thresholds” and save. 

After that, navigate to the instances tab and click on “Create new instance”, add below details and save.

Similarly, create another ledger named “aqas-time-series” and create an instance with an empty array.

Next, navigate to the “Logic” function from the left menu and create an event-based logic function.

Copy/paste the code below and deploy the Logic function. Logic is basically a javascript function that is triggered when an event is received. If you are unfamiliar with Particle logic functions, check out my previous article, which goes into more depth. 

 

import Particle from 'particle:core';




const  findPercentile = (arr, pct)=> {

  // Sort the array in ascending order

  const sortedArr = arr.slice().sort((a, b) => a - b);

  

  // Calculate the index for the 90th percentile

  const index = Math.ceil(sortedArr.length * pct* 0.01) - 1;

  

  // Return the value at the calculated index

  return sortedArr[index];

}





export default function process({ functionInfo, trigger, event }) {

  const deviceThresholdsLedger = Particle.ledger("aqas-thresholds");

  const timeSeriesLedger = Particle.ledger("aqas-time-series");

  const data = JSON.parse(event.eventData);

  

  const co2_threshold = deviceThresholdsLedger.get().data.co2_threshold;

  const no_of_readings = deviceThresholdsLedger.get().data.no_of_readings;

  const co2_ts = timeSeriesLedger.get().data.co2||[];

  co2_ts.push(data.eco2);

  

  

  if(co2_ts.length >= no_of_readings){

    //calculate 90th percentile 

    const pctCO2 = findPercentile(co2_ts,90);

    console.log("percentile value ", pctCO2);

    timeSeriesLedger.set({co2: []});

    if( pctCO2 >=co2_threshold ){

      Particle.publish("aqas-alert", {co2: pctCO2}, { productId: event.productId });

    }

  }else{

    timeSeriesLedger.set({co2: co2_ts});

  }




  Particle.publish("aqas-time-series", {co2: data.eco2, aq: data.aq, seq: co2_ts.length,  deviceId: event.deviceId}, { productId: event.productId, deviceId: event.deviceId });

}

 

 

Now, if you navigate to the Events page, you should see events like below.

Pay attention to the “aqas-alert” event which is triggered when the 90th percentile CO2 reading is above the threshold. 

Let’s create some integrations to display CO2 readings on Ubidots dashboard and send notification to a slack channel.

Slack Notification

We are going to use webhook integration for this. I am assuming you have a slack webhook url to use for this integration. 

Fill Out the details as above – note the event name which must match with your logic code. Save and enable the integration. You can now head over to the Event page again and see if the webhook is successful. You should also get the notification on your slack.

Ubidots Dashboard

Next up, send data to Ubitdots and visualize on the dashboard. Please checkout my previous article on detailed instructions on how to create Ubidots dashboard. 

From the integration page, choose Ubidots and configure the integration as below. Again, it’s important to note the event name. 

Save and enable the integration. Within a few minutes, depending on how frequently your device sends data to particle cloud, your should start receiving data on your Ubidots account.

And there you have it! We’ve taken our air quality monitor from a simple alarm to a smart, data-savvy device. By tapping into Particle Ledger, we’ve given it the power to learn from past data and make smarter decisions. No more crying wolf – just reliable alerts when they really matter.

This project is just the tip of the iceberg for what you can do with IoT and cloud integration. Got ideas brewing? Run into a snag? Or just want to show off your cool upgrades? Swing by community.particle.io. It’s packed with folks who live and breathe this stuff and are always up for a good tech chat.

Thanks for sticking with me through this build. Happy hacking!

Comments are not currently available for this post.