Data Intelligence

Data Intelligence is a tool that automatically and intelligently analyzes your datasets before you build a model. It inspects the signals you import and reports a set of easy-to-read metrics (KPIs) that tell you whether your data is balanced, which channels/features matter, how channels relate to each other, and—for time-series data—the best window size and sampling frequency to use.

Data Intelligence supports both classification (predict a class) and regression (predict a continuous value) tasks. Anomaly detection can be treated as a binary classification case. This section describes the dataset analysis process, outlines the steps required to refine and recollect data for improved performance, and explains how insights are derived from the analysis results.

Importance of Data Intelligence

Users usually import time series datasets based on their own knowledge. However, due to some limitations, a comprehensive analysis of the data can yield unsatisfactory results. For example, the sampling frequency can be higher than what the application requires. Also, the training data for a classification task can be unevenly distributed across different classes, or a regression target can be skewed toward a narrow range.

To address these challenges, the Data Intelligence tool helps assess the balance of the dataset (or the distribution of the regression target) and identify the significance of each data channel/feature. The tool not only flags unbalanced datasets but also suggests redundant channels/features for omission. Furthermore, for time-series data the tool helps determine the optimal sampling frequency and window size, enabling you to refine the datasets for improved quality and analysis outcomes.

The Three Data Types

All three data types are plain numeric text files (.csv/.txt). The difference lies only in how each row is interpreted and whether the data has a time axis. Your first step is always to tell the tool how your data is laid out by choosing one of the three data types.

Data Type

Row layout

Time axis

KPIs produced

Continuous

rows = time steps, columns = channels. One long, uninterrupted recording; one value per channel per row.

Yes

Balance / Target Distribution, Channel Correlation, Channel Importance, Window Size & Frequency Finder (the engine cuts windows for you).

Segmented

row = window × channels (a multiple of the channel count). Each row is one ready-made sample.

Yes

Balance / Target Distribution, Channel Correlation, Channel Importance, Sampling Frequency Finder (window size is fixed by you).

Tabular

row = sample, columns = independent features. No time axis; row order does not matter.

No

Feature Correlation, Feature Importance only (no Class Balance / Target Distribution, no window/frequency search).

Note: For regression datasets, each row is prepended with one or more continuous target columns (see Number of Targets below). These target columns are stripped from the row before the remaining columns are interpreted according to the layout above.

Data Settings

The first step is to configure the mandatory parameters for the imported data.

Enter the following information:

  • Task Type: Classification or Regression

  • Dataset Type: Continuous, Segmented, or Tabular

  • Number of Channels (Segmented only): How many channels each window contains. Because a segmented row packs several samples together (window × channels), the tool cannot infer the channel count on its own, so you must declare it up front so each row can be reshaped correctly.

  • Number of Targets (regression only): How many continuous targets are prepended to each data row. Must be at least 1.

  • Sampling Frequency Used (time-series only): The sampling frequency of the data

  • Minimum Frequency (time-series only): The lowest settable frequency division coefficient

Note: Sampling Frequency Used and Minimum Frequency drive the sampling-frequency and window-size search, which only applies to Continuous and Segmented data. Tabular data has no time axis, so these two settings are not required for it.

Note: For Continuous and Tabular data, the number of channels/features is no longer entered here. Each row of a Continuous or Tabular file already is one sample, so the tool infers the channel/feature count directly from the column count of the first file you import (see Loading and Analyzing Data below).

For example, the following figure shows the settings panel for a segmented classification dataset, which still requires Number of Channels, next to a continuous and a tabular settings panel, neither of which asks for it.

Segmented Settings Continuous Settings Tabular Settings

Once the number of channels/features is known (declared up front for segmented data, or resolved automatically after the first file import for continuous/tabular data), you can assign individual aliases to each channel/feature, which aids in distinguishing between them and understanding the analysis report. For regression datasets you can likewise assign an alias to each target.

Managing Classes (Classification)

You no longer need to declare the final number of classes up front. The dataset starts with two classes, and you can grow or shrink the set as you import more data:

  • Click the + button under the class tab to create a new class. Each class name must be unique.

  • Click the x button in the top-right corner of a class tab to remove that class along with all of its imported signals. At least one class must remain.

Add and Delete Class

Loading and Analyzing Data

Steps:

  1. Click the + button beside each class and load files. (For regression, all files are loaded into a single group named “Dataset” since regression has no classes.)

  2. Verify that the automatically recognized Delimiter, number of lines, and number of columns are correct.

  3. Ensure that each file conforms to the specified settings:

    • For continuous data: Every row is one sample, so its column count (after removing any target columns) directly becomes the number of channels. The very first file you import defines that count; every subsequent file must match it.

    • For segmented data: The number of (non-target) columns must equal the window size multiplied by the Number of Channels declared in Data Settings.

    • For tabular data: Every row is one sample, so its column count (after removing any target columns) directly becomes the number of features, exactly as for continuous data.

    • For regression data: Each row additionally begins with Number of Targets continuous target columns, which are removed before the checks above.

  4. Optionally modify the alias for each class, channel/feature, or target to facilitate differentiation.

  5. Click the START DATA ANALYSIS button and proceed with data analysis and generate a report.

Segmented Load Signals and Analysis

Analysis Result

Once you initiate the analysis, the system processes the data for a few seconds and then displays the results. Which KPIs appear depends on your data type and task type.

Class Balance

Note: This metric applies to classification datasets.

Segmented Class Balance

A five-star rating indicates a perfectly balanced dataset, which is optimal for classification tasks. A low balance score warns that one class has far more data than another, which can bias the classifier.

Target Distribution

Note: This metric applies to regression datasets in place of Class Balance.

Target Distribution

For each continuous target, the tool reports the count and the min / max / mean / std, so you can spot skewed or narrow-range targets that may bias a regression model.

Channel / Feature Correlation

Segmented Channel Correlation

Note: This feature is applicable only when the number of channels/features is 2 or more.

Description:

  • Displays a matrix that illustrates the correlation values between different channels (or, for tabular data, features)

  • The higher the absolute value, the higher the correlation between the pair

  • Values closer to zero indicate that the channels/features are independent from each other

  • Highly correlated pairs carry redundant information—you often need only one of them. This information can be used to identify and potentially remove redundant channels/features, helping optimize the dataset for the training step

Channel / Feature Importance

Segmented Channel Importance

Note: This feature requires 2 or more channels/features.

Description:

  • Displays the importance scores for each channel/feature in ranking bars

  • Higher scores indicate greater importance; near-zero scores mean the channel/feature adds little and could often be dropped

  • Classification: importance reflects how much each channel helps distinguish the classes

  • Regression: importance reflects how strongly each channel relates to the continuous target(s). In addition to an averaged score, a per-target breakdown is reported so you can see which channels matter for each target

    Channel Importance Per Target

  • Continuous: since continuous data has no fixed window size yet, the tool windows the data at several candidate window sizes, scores channel importance at each one, and averages the results across all of them. This gives a window-size-independent estimate of how useful each channel is before you have picked a final window size in Best Sampling Params

    Continuous Channel Importance

  • Tabular: importance fuses a statistical measure (p-value based) with a model-based measure (Random Forest). Each source is normalized and averaged so neither dominates purely because of its raw scale

    Tabular Feature Importance

Best Sampling Params

Note: This section applies to time-series data (Continuous and Segmented) only. Tabular data has no time axis, so no sampling-frequency or window-size search is performed.

Parameters:

  • Sampling Frequency:

    • Outputs a recommended division frequency of the original sampling frequency

    • This recommendation is designed to help remove potential high-frequency noise components from the data

    • By reducing the frequency in this manner, you can save resources and power while maintaining essential information

  • Window Size:

    • Outputs a recommended window size

    • Remains unchanged (fixed by you) for segmented data

    • Only searched for continuous data

  • Sampling Duration:

    • The sampling time for a single sample

For continuous data, the tool runs a Window Size & Frequency Finder: it tries many combinations of window length and sampling-frequency division, checks how well each combination lets the data be told apart, and recommends the best pair. When several settings work equally well, the tool prefers the shortest, lightest window so the resulting model stays small and fast. The full set of results is shown as a heat map so you can see the trade-offs.

For segmented data, only the Sampling Frequency Finder runs (the window size is already fixed by your file layout).

Segmented Best Sampling Params

Note (regression): For continuous regression data, the recommended window/frequency pair is chosen using a Random Forest regressor scored by R² (coefficient of determination), whereas classification uses a balanced-accuracy score. The workflow and reported outputs are otherwise the same.

Continuous Data Example

Here, we provide an example of a continuous classification dataset. This dataset represents three-axis acceleration measurements at different states of a fan:

  • The dataset consists of three channels (three acceleration axes)

  • Each data file represents a fan state

  • The sampling frequency is 200 Hz

  • Each data file contains 720,001 lines with three channels per line

Since this is continuous data, Number of Channels is not entered in Data Settings; the tool infers the channel count (three) once the first file is imported, from its three columns.

Continuous Settings

Follow the steps above for data settings and load data for smart analysis.

Analysis Output

Class Balance:

Continuous Class Balance

  • Data balanced with five stars

Channel Correlation:

Continuous Channel Correlation

  • The correlation between channels is low, indicating that there are no redundant channels

Channel Importance:

Continuous Channel Importance

  • Channel_3 carries most of the discriminative information across the averaged window sizes, while Channel_1 and Channel_2 contribute less and could be candidates for removal if the dataset needs to be lightened

Best Sampling Params:

Continuous Best Sampling Params

  • In the window size and sampling frequency matrix, the combination of a frequency of fs/1 and a window size of 64 is the most recommended

Applying Recommendations

You can import continuous data into Data Operation and use the Generate Samples operation with the recommended sampling parameters to generate sample datasets for machine learning projects.

Important: If the recommended sampling frequency is not fs/1, the data should be downsampled according to the division frequency and the recommended window size.

For example, if you get fs/2 division frequency after smart analysis, then the dataset is downsampled and saved as shown below:

Window Size Finder

Regression Example

Regression datasets predict one or more continuous values rather than a class. Because regression has no classes, all imported files are loaded into a single group named Dataset, and each data row begins with the continuous target column(s).

Settings:

  • Set Task Type to Regression

  • Set the Number of Targets (at least 1) and, optionally, an alias for each target

  • Choose the dataset type (Continuous, Segmented, or Tabular). Number of Channels is only entered for Segmented data (shown below); for Continuous and Tabular regression data it is likewise inferred from the column count once targets are stripped off

Regression Settings

Analysis Output

  • Target Distribution replaces Class Balance and reports the count and min/max/mean/std of each target

  • Channel / Feature Importance is scored against the continuous target(s), with both an averaged score and a per-target breakdown

  • Channel / Feature Correlation and the sampling-frequency / window-size finders behave the same as for classification (the correlation and frequency analysis are target-independent)

Tabular Data Example

Tabular data has no time axis: each row is one independent sample, and each column is a separate feature (for example temperature, pressure, or age).

Settings:

  • Set Dataset Type to Tabular

  • Number of Channels is not entered: it is inferred from the number of columns in the first file you import, and each column becomes a feature. Once resolved, you can optionally assign a feature alias to each column

  • Sampling Frequency Used and Minimum Frequency are not required

Tabular Settings

Analysis Output

Tabular data has neither Class Balance nor Target Distribution; it reports only these two KPIs:

  • Feature Correlation — a matrix of pairwise correlation between features; highly correlated pairs are redundant

    Tabular Feature Correlation

  • Feature Importance — a fused score combining a statistical measure with a ML model measure, reported per feature

    Tabular Feature Importance

No window-size or sampling-frequency search is performed for tabular data.

Saving the Report

Finally, you can choose the quality and click the Save PDF button to save the analysis report PDF to the local system.

Save Report

Conclusion

The Data Intelligence tool automatically generates reports on the quality of datasets and provides feedback to users without direct engineering support from NXP. It supports both classification and regression tasks across continuous, segmented, and tabular data. The analyzed features include Class Balance / Target Distribution, Channel / Feature Correlation, Channel / Feature Importance, and the Sampling Frequency and Window Size finders. These features empower you to make informed decisions regarding data recollection or reformatting, ensuring that your datasets are in an optimal state for subsequent processing.