BigQuery Enhances Anomaly Detection for Multivariate Time Series Models

August 21, 2024 at 11:25:09 AM

TL;DR BigQuery ML supports anomaly detection with multivariate time series (ARIMA_PLUS_XREG) models for historical and new data. The ML.DETECT_ANOMALIES function uses ARIMA_PLUS and ARIMA_PLUS_XREG for time series data and autoencoder, k-means, or PCA for IID data. The CREATE MODEL statement creates these models, handling data frequency, irregular intervals, missing data, outliers, and seasonal patterns using various techniques.

BigQuery Enhances Anomaly Detection for Multivariate Time Series Models

You can now use BigQuery ML's multivariate time series ARIMA_PLUS_XREG models for anomaly detection. This feature allows for detecting anomalies in both historical and new data with multiple feature columns and is generally available (GA). You can explore this feature through the "Perform anomaly detection with a multivariate time-series forecasting model" tutorial.

ML.DETECT_ANOMALIES Function

The ML.DETECT_ANOMALIES function in BigQuery ML enables anomaly detection for time series data using ARIMA_PLUS and ARIMA_PLUS_XREG models. For independent and identically distributed random variables (IID) data, it uses autoencoder, k-means, or principal component analysis (PCA) models.

CREATE MODEL Statement for ARIMA_PLUS_XREG Models

The CREATE MODEL statement is used to create multivariate time series models in BigQuery. Forecasting is performed when the model is created, and you can retrieve forecasting values and compute prediction intervals using the ML.FORECAST and ML.EXPLAIN_FORECAST functions.

Time Series Modeling Pipeline

The multivariate ARIMA_PLUS_XREG time series model includes linear external regressors. The modeling pipeline for ARIMA_PLUS time series models performs several functions:

bigquery Time series modeling pipeline

  • Infer data frequency
  • Handle irregular time intervals
  • Handle duplicated timestamps by averaging
  • Interpolate missing data using local linear interpolation
  • Detect and clean spike and dip outliers
  • Detect and adjust abrupt step changes
  • Detect and adjust holiday effects
  • Detect multiple seasonal patterns using Seasonal and Trend decomposition using Loess (STL)
  • Extrapolate seasonality using double exponential smoothing (ETS)
  • Detect and model trends using the ARIMA model and auto.ARIMA algorithm for automatic hyperparameter tuning, selecting the best model based on the lowest Akaike information criterion (AIC).

Have more questions on this topic? Ask our AI assistant for in-depth insights.

Read more from sources 👇

The Only Digital Marketing Feed You'll Ever Need.

Stay informed your way. Tailored updates when and how you want them. 100% Free.

10,000+ Users

500+ Sources

1000+ Tools

Or

Related Posts

Google Updates BigQuery with Multi-Time Series Forecasting and Enhanced Security Features

Google Updates BigQuery with Multi-Time Series Forecasting and Enhanced Security Features

Google Cloud
Google Cloud

Official Source

Official Source

Google Cloud is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Google Cloud Code Extension Adds BigQuery Support in VS Code

Google Cloud Code Extension Adds BigQuery Support in VS Code

Google Cloud
Google Cloud

Official Source

Official Source

Google Cloud is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Top-Notch Google Ads Audit Tool

Top-Notch Google Ads Audit Tool

Featured
BigQuery Introduces Commercial Data Sharing via Cloud Marketplace

BigQuery Introduces Commercial Data Sharing via Cloud Marketplace

Google Cloud
Google Cloud

Official Source

Official Source

Google Cloud is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Google Sheets Integrates BigQuery Saved Queries

Google Sheets Integrates BigQuery Saved Queries

Google
Google

Official Source

Official Source

Google is a Official Source. The source has been verified by Swipe Insight team.

Official Source
BigQuery Expands Search Index Capabilities with INT64 and TIMESTAMP Support

BigQuery Expands Search Index Capabilities with INT64 and TIMESTAMP Support

Google Cloud
Google Cloud

Official Source

Official Source

Google Cloud is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Google Ends Free Access to Gemini in BigQuery, Announces Paid Plans

Google Ends Free Access to Gemini in BigQuery, Announces Paid Plans

Google Cloud
Google Cloud

Official Source

Official Source

Google Cloud is a Official Source. The source has been verified by Swipe Insight team.

Official Source
Google Cloud Enhances Looker with Major User Experience Updates

Google Cloud Enhances Looker with Major User Experience Updates

Sean Zinsmeister
Sean Zinsmeister

Official Source

Official Source

Sean Zinsmeister is a Official Source. The source has been verified by Swipe Insight team.

Official Source

Related Tools

Marketing Auditor logo

Marketing Auditor

Verified Tool

Verified Tool

Marketing Auditor is a Verified Tool. Want to get this badge? Contact us.

Verified Tool

Automated audits for Google Ads and Analytics.

Get Featured Here

Showcase your tool in this list.

Contact Us
Databricks logo

Databricks

Generative AI-powered data intelligence platform

Data Engineering
GA4 SQL logo

GA4 SQL

Verified Tool

Verified Tool

GA4 SQL is a Verified Tool. Want to get this badge? Contact us.

Verified Tool

Generate GA4 BigQuery queries easily

Data Analysis
TapClicks logo

TapClicks

Automated marketing solutions powered by your data

Data Engineering
Stitch logo

Stitch

Automated cloud data pipelines, no coding needed

Data Engineering
Akkio logo

Akkio

AI-powered analytics for agencies

Data Analysis
NinjaCat logo

NinjaCat

AI-powered marketing data and analytics platform

Reporting
Funnel logo

Funnel

Aggregate and analyze marketing data seamlessly

Reporting
Fivetran logo

Fivetran

Effortlessly centralize and move data from any source

Data Engineering
Power My Analytics logo

Power My Analytics

Automate and integrate your marketing data

Reporting

Get Featured Here

Showcase your tool in this list.

Contact Us