BigQuery has announced significant improvements to its workload management system, particularly focusing on the autoscaling feature. These enhancements are now generally available (GA), offering users more efficient and flexible resource allocation.
Key Improvements in Autoscaling
Immediate Scaling: The autoscaler now scales up instantly when needed.
Precise Scaling: Autoscaling is more accurate, better matching workload demands.
Finer Granularity: Scaling now occurs in 50-slot increments, down from 100.
Flexible Capacity Management: Users can purchase capacity commitments, set baseline slots, and define autoscale max slots in 50-slot steps.
Smoother Downscaling: Capacity can be reduced without resetting the one-minute minimum if more than a minute has passed since the last increase.
Autoscaling Reservations
- No pre-purchase of slot commitments required
- Optional slot commitments provide discounted rates for consistent usage
- Reservations scale to multiples of 50 slots
- Scaling based on actual usage, rounded up to nearest 50 slot increment
- Charges apply for scaled slots, not just used slots
Scale-Down Window
- Autoscaled capacity retained for at least 60 seconds
- Capacity peaks reset the 60-second scale-down window
- After 60 seconds, capacity can be reduced without resetting the window
These updates reflect BigQuery's commitment to providing more efficient and cost-effective data processing solutions, allowing users to better manage their workloads and resources.
