Google has announced several updates to BigQuery, introducing new capabilities for vector index management and enhanced data loading options.
Vector Index Statistics and Rebuilding
BigQuery now offers the VECTOR_INDEX.STATISTICS function, allowing users to calculate how much an indexed table's data has drifted between when a vector index was created and the present. When table data has changed significantly enough to require maintenance, users can utilize the ALTER VECTOR INDEX REBUILD statement to rebuild the vector index. This feature is currently available in Preview.
Access Transparency Enhancement
Access Transparency now supports BigQuery data preparation in the GA stage, providing users with enhanced visibility into data processing operations.
Enhanced External Table and Data Loading Options
The CREATE EXTERNAL TABLE and LOAD DATA statements have been expanded with new options, currently available in Preview:
null_markers enables users to define the strings that represent NULL values in CSV files, providing greater flexibility in handling missing data.
source_column_match allows users to specify how loaded columns are matched to the schema. Users can now match columns either by position or by name, offering more precise control over data mapping.
Pattern Recognition with MATCH_RECOGNIZE
BigQuery has introduced the MATCH_RECOGNIZE clause for SQL queries, enabling users to filter and aggregate matches across rows in a table. This pattern recognition functionality is currently available in Preview, expanding BigQuery's analytical capabilities for complex data pattern analysis.
