Using Smartphones as time-bound IoT setup, can save huge capital investments. However, sensor data of a $10 phone is not always be as reliable as the one from a $500 phone. Differences arise in quality of data, frequency of data acquisition, consistency of data correctness or incorrectness, and due to calibration issues.
This is a second article from a series which identifies how smartphone sensors can be used reliably to produce scientific data for IoT applications. Previous article can be found here
Smartphones come with their own specific challenges while attempting to produce reliable, consistent and repeatable sensor data. The peculiarities typically arise from the fact that smartphones come in variety of shapes, size, and quality.
Certain statistical methods are known in the data researchers world, which are useful in ‘filling the gaps’ in various parameters of smartphone sensor datasets to make sure that given software can produce meaningful, reproducible data sets across multiple smartphone hardware sensors.
Data Smoothing Basics
Data smoothing is a technique to remove noise from data set, allowing important pattern to stand out. Using this short-term irregularity can be removed in a time-series data to improve the accuracy of forecasts.
Data noise is random error or variance or outliers in measured values. Below are primary factors for noise in data set.
Attempt to obtain the simplest representation of data which describe the underlying process, while eliminating errors is referred as Curve Fitting or Data Smoothing.
Below are few smoothing techniques
Exponential Smoothing is most popular smoothing techniques due to its flexibility, ease in calculation, and good performance. Exponential Smoothing uses a simple average calculation to assign exponentially decreasing weights starting with the most recent observations. New observations are given relatively more weight in the average calculation than older observations.
Let’s try out Data Smoothing using a simple tool available on most of the computers, Microsoft Excel. Following is a step by step guide to experience power of data smoothing using the spreadsheet program.
Step 1: Open the Excel program. Copy and paste the data which we have to use for data smoothing into columns A and B beginning in row 1 of a blank worksheet.
Step 2: Select the Data ribbon menu, then Data Analysis command on the Analysis tab. A popup box will appear. Scroll down and select Exponential Smoothing. Click OK.
Step 3: The Exponential Smoothing wizard will be displayed. In the Input Range field, select the values from Acceleration column of data table
Step 4: In the Damping factor field, you can provide values from 0 to 1. This will be utilized by exponential smoothing algorithm.
Step 5: In the Output Range field, select cell C2 so that the Smoothed values will be posted from that cell
Step 6: Verify you have checked all remaining boxes as per below fig. Your wizard should be identical to the graphic below:
Step 7: Click OK to 'run' the Exponential Smoothing. Verify your output appears as follows
In above image blue data line is before data smoothening with original data and orange data line represent smooth data points after performing Exponential Smoothing.