Data prediction and anomaly detection

Anomaly detection and prediction of the selected parameters in an arbitrary data algorithms realization using R language.

The system developed by our data science experts consists of a number of modules.

The first module allows performing the statistical and visual exploratory data analysis with a computation of main characteristics distribution for separate parameters and dependencies between them.

Having the analysis results, the user can automatically or manually detect the best group of parameters for prediction.

The second module performs the direct data processing, that is based on:

1. Functions for data pre-processing realization, including data normalization and scaling, clusterization/grouping and initial parameters orthogonal transformation;

2. Training methods and anomaly detection system usage realization based on Gaussian mixture models and confidence interval-based filtering, as well as parameters prediction using neural networks.

The system has the following capabilities:

• result visualization,
• transformation models saving,
• clusterization of neural network for the further usage.

Each module has a robust settings system, so that the user with no deep knowledge in machine learning and algorithms can easily use it.

The system of optimum parameters selection form the selected diapason based on cross-validation was realized in a separate module as well.