Google has announced a new feature in BigQuery that allows users to create external datasets linked to existing Spanner databases. This feature is currently in preview.
Key Features:
Federated Querying: Users can query Spanner data using GoogleSQL without moving data to BigQuery storage.
Automatic Sync: Tables in the external dataset automatically reflect updates made in the Spanner source.
Read-Only Access: BigQuery users can query but not modify the external data.
Requirements:
- Users need the BigQuery User (
roles/bigquery.user) IAM role or equivalent permissions.
Limitations:
- Federated query limitations apply.
- Unsupported Spanner column types won't be accessible in BigQuery.
- No data or metadata modifications allowed in BigQuery.
- New table creation, views, or materialized views not supported in the external dataset.
- PostgreSQL dialect in Spanner not supported.
- Various BigQuery features like Write/Read API, row/column-level security, and data masking are not available.
Use Cases:
This feature is particularly useful for organizations wanting to analyze transactional Spanner data alongside other BigQuery datasets without data duplication or complex ETL processes.
While powerful, users should be aware of the feature's limitations and ensure it aligns with their specific data analysis needs before implementation.
