{"id":15658,"date":"2021-10-21T12:45:00","date_gmt":"2021-10-21T19:45:00","guid":{"rendered":"https:\/\/devwww.3cloudsolutions.com\/post\/improving-responsiveness-in-predictive-analytics-with-real-time-change-detection-3\/"},"modified":"2024-01-08T10:36:56","modified_gmt":"2024-01-08T18:36:56","slug":"improving-responsiveness-in-predictive-analytics-with-real-time-change-detection","status":"publish","type":"post","link":"https:\/\/3cloudsolutions.com\/resources\/improving-responsiveness-in-predictive-analytics-with-real-time-change-detection\/","title":{"rendered":"Improving Responsiveness in Predictive Analytics with Real-Time Change Detection"},"content":{"rendered":"

With the advent of Internet of Things (IOT) and the proliferation of connected devices, comes the challenge of monitoring parts for maintenance before they break down. A common approach revolves around getting data from connected devices and performing a statistical test to determine the likelihood of the device failing. While this common approach is robust, it typically involves a significant time investment in exploratory data analysis, feature engineering, training, and testing to build a predictive model. It, therefore, often lacks the agility required to keep up with the monitoring demands of increasingly time-sensitive initiatives.<\/span>\u00a0<\/span><\/p>\n

In this context, the question becomes: how can we ensure a similar degree of rigor, but also improve the timeliness and responsiveness of being able to perform predictive maintenance?<\/span>\u00a0<\/span><\/p>\n

<\/p>\n

\"iStock-1022887882\"<\/h3>\n

A Simple Approach with CUSUM<\/span><\/h2>\n

CUSUM (short for <\/span>\u201cCumulative Sum<\/span>\u201d) control charts allow us to plot the cumulative deviations of a process from the average, and monitor the results versus the average. Once the cumulative deviations from the average reach a certain threshold, we can then alert users that there has been a change.<\/span>\u00a0<\/span><\/p>\n

More formally, suppose there is a metric that we are monitoring in a stationary process whose high values will result in a part\/machine breaking down. We will call the average value of this metric <\/span>\"Capture11\"<\/span> (<\/span>mu<\/span><\/i><\/strong>), and for every observation we will only add positive deviation for the mean. To start, the initial cumulative effect will be 0, \"Capture1-1-1\"<\/span>. Let <\/span>T\u00a0<\/span><\/strong>be\u00a0some threshold value that we want to monitor to see if the cumulative effects go over.<\/span>\u00a0<\/span><\/p>\n

For every newly observed value \"Capture2-2-1\"<\/span> of data, we want to update the cumulative effect as shown. To update for the next <\/span>S<\/span><\/strong>,<\/span> we take only the positive change, since we\u2019re measuring positive deviation or 0. If we reach the threshold value of\u00a0<\/span>T<\/span><\/strong>, we\u00a0alert the user that there has been a change in the process.<\/span>
\n<\/span><\/p>\n

\"Capture3\"<\/p>\n

Below is a visual representation of CUSUM in the increasing direction, with a threshold of 4 and 5 (SH<\/sub><\/span><\/span>\u00a0in this case is\u00a0the variable we are referring to as\u00a0T<\/span>).<\/span>\u00a0<\/span><\/p>\n

<\/span><\/h3>\n

Source. <\/i><\/span><\/span><\/p>\n

Azure Stream Analytics & Power BI Streaming Datasets<\/p>\n

\u00a0<\/span>
\n<\/strong><\/span><\/h2>\n

Azure Stream Analytics<\/a><\/span> is a real-time analytics platform <\/span>designed<\/span>\u00a0to analyze and process large volumes of streaming data from multiple sources simultaneously. The image below <\/span>highlights<\/span>\u00a0a variety of streaming architecture patterns possible in Azure Stream Analytics.<\/span><\/span><\/p>\n

<\/span><\/span><\/span><\/p>\n

For this demo, we will be using Azure Stream Analytics to process\u00a0temperature data using CUSUM, and feed the results\u00a0to\u00a0a\u00a0Power BI\u00a0Report. From there,\u00a0we will demonstrate how to configure data alerts to\u00a0notify\u00a0users when\u00a0the\u00a0process changes\u00a0based on\u00a0a threshold value T<\/span>, <\/span>and\u00a0to\u00a0email\u00a0a\u00a0notification\u00a0to them.<\/span><\/p>\n

 <\/p>\n

Setting Up the Azure Environment<\/span><\/h3>\n

Following this\u00a0<\/span>guide<\/a><\/span>, we will set up\u00a0the following resources\u00a0beforehand. It is recommended to create a test resource group\/environment\u00a0before proceeding.<\/span>\u00a0<\/span><\/p>\n