AWS Unveils Native Support for Apache Iceberg V3 in Amazon S3 Tables: A New Era for Petabyte-Scale Analytics

SEATTLE — In a major milestone for enterprise data architecture, Amazon Web Services (AWS) has announced full support for all data types and advanced features in the Apache Iceberg V3 specification within Amazon S3 Tables. The rollout, announced by AWS specialist Daniel Abib, equips data engineering and analytics teams with native mechanisms to handle petabyte-scale data lakes with unprecedented efficiency, reduced latency, and minimized storage overhead.

The update enables organizations to seamlessly provision new V3 tables or perform in-place upgrades on existing V2 architecture. By bridging long-standing gaps in how cloud data lakes ingest and process semi-structured, geospatial, and high-precision temporal data, this release marks a fundamental shift in how modern enterprises design cost-effective, high-performance analytics pipelines.

Main Facts: What the Apache Iceberg V3 Integration Delivers

The integration of Apache Iceberg V3 into Amazon S3 Tables introduces a robust suite of native enhancements designed to eliminate legacy workarounds that have plagued data practitioners for years.

At the core of the release is native support for the complete Iceberg V3 specification, which includes:

• Deletion Vectors: Replacing traditional positional delete files with a compact, binary format that dramatically streamlines row-level updates and compliance deletions.
• Row Lineage: Automatically embedding _row_id and _last_updated_sequence_number attributes into records to simplify incremental data tracking.
• Advanced Data Types: Introducing native support for variant (semi-structured data), nanosecond-precision timestamps, geometry, geography, and unknown types.

Crucially, these features integrate directly with Amazon S3 Tables’ purpose-built storage layer, which automates background tasks such as table compaction, maintenance, replication, and Intelligent-Tiering.

Chronology: The Evolution to Iceberg V3 and AWS Integration

To understand the significance of the Amazon S3 Tables V3 rollout, it is helpful to examine the timeline of open-source data formats and cloud data lake maturation.

• The Rise of Open Table Formats: Over the past half-decade, Apache Iceberg evolved from an internal project at Netflix into the de facto open standard for managing large analytics datasets. By separating the table metadata layer from underlying data files stored in open formats like Parquet, Iceberg brought ACID (Atomicity, Consistency, Isolation, Durability) transactions, schema evolution, hidden partitioning, and time-travel queries to cloud object storage.
• Limitations of the V2 Standard: While Apache Iceberg V2 achieved widespread enterprise adoption, organizations scaling into the multi-terabyte and petabyte range frequently encountered operational bottlenecks. Compliance mandates—such as fulfilling "right to be forgotten" requests—forced engines to generate thousands of small positional delete files. Furthermore, handling JSON strings, geospatial coordinates, or high-precision logs required cumbersome string encoding and expensive run-time parsing.
• The Introduction of Iceberg V3: The open-source community developed the V3 specification to directly address these structural inefficiencies, introducing compact deletion vectors, row lineage tracking, and native data types.
• The AWS Implementation: Building on its history of supporting open-source analytics frameworks, AWS has moved rapidly to integrate V3 capabilities natively. Following the launch of Amazon S3 Tables as a dedicated storage tier for Iceberg, AWS has now rolled out complete V3 support across its global infrastructure at no additional charge beyond standard S3 Tables pricing.

Supporting Data: Overcoming Legacy Bottlenecks

Data lakes operating at scale often face a hidden tax in terms of compute cycles, storage costs, and pipeline complexity. The integration of Apache Iceberg V3 into Amazon S3 Tables directly targets these performance drains with measurable architectural improvements.

Eliminating the Positional Delete Tax

In Iceberg V2, executing a compliance delete—such as removing 50,000 user records from a 2-billion-row table—generated numerous individual positional delete files. Every subsequent query had to scan and reconcile these files until a resource-intensive compaction cycle ran, introducing noticeable query latency.

V3 replaces positional delete files with deletion vectors. Under this model, a compliance delete writes a single, compact binary file. Compaction overhead is drastically reduced, and read operations bypass the latency penalties historically associated with frequent row-level mutations.

The Power of the Variant Data Type

Enterprise telemetry, clickstreams, and application logs frequently arrive as semi-structured JSON payloads where individual events feature divergent schemas (e.g., page views containing URLs versus purchase events containing item arrays). Previously, teams stored these payloads as unstructured strings, forcing every downstream query to parse JSON at read time—a notorious CPU bottleneck.

The V3 variant data type stores semi-structured data natively in a columnar format. During ingestion, the query engine shreds variant data into hidden columns and computes statistics. At query time, these statistics enable advanced file pruning, drastically reducing I/O operations compared to legacy string-parsing workflows.

Architectural Example: Clickstream Analytics

To demonstrate the practical application of these features, consider a retail analytics platform tracking user behavior across web and mobile applications. Using Spark via Amazon EMR, data engineers can define a V3 table with a variant payload without enforcing a rigid, predefined schema:

CREATE TABLE my_catalog.namespace.clickstream (
  event_id bigint,
  event_time timestamp,
  user_id string,
  payload variant
)
USING iceberg
TBLPROPERTIES ('format-version' = '3')

Data ingestion becomes frictionless, allowing disparate event shapes to be written simultaneously:

INSERT INTO my_catalog.namespace.clickstream VALUES
  (1, current_timestamp(), 'user-42',
   PARSE_JSON('"action": "purchase", "amount": 99.99, "items": ["laptop_stand"]')),
  (2, current_timestamp(), 'user-17',
   PARSE_JSON('"action": "page_view", "url": "/products/webcam", "duration_ms": 4200'));

When querying the data, analysts can extract fields directly using variant_get without runtime parsing penalties:

SELECT
  event_id,
  user_id,
  variant_get(payload, '$.action', 'string') AS action,
  variant_get(payload, '$.amount', 'double') AS amount
FROM my_catalog.namespace.clickstream
WHERE variant_get(payload, '$.action', 'string') = 'purchase'
  AND variant_get(payload, '$.amount', 'double') > 50.00

Official Responses and Ecosystem Integration

AWS has positioned this release as part of a broader, cohesive strategy to provide native, cross-service interoperability for Apache Iceberg across the entire analytics stack.

According to technical documentation and architectural guidance released by AWS, the integration spans every layer of the modern data architecture:

• Ingestion and Processing: Teams can write and transform data using Amazon EMR Spark.
• Catalog and Governance: AWS Glue 6.0 provides full Apache Iceberg V3 support and integrates with the Iceberg REST Catalog (IRC) API, ensuring seamless multi-engine interoperability.
• Storage and Optimization: Amazon S3 Tables automatically manages compaction, maintenance, and deletion vector cleanup behind the scenes.
• Query and BI: Downstream analytics engines, including Amazon Redshift, can directly query V3 datasets.

Migration Pathways and Upgrades

To ease the transition for existing enterprises, AWS has engineered backward compatibility into the S3 Tables platform. Organizations can upgrade existing V2 tables to V3 atomically in-place without rewriting underlying data files:

ALTER TABLE my_catalog.namespace.existing_table
SET TBLPROPERTIES ('format-version' = '3')

AWS notes that this is an atomic metadata operation. Existing V2 readers will continue to function on upgraded tables until systems are fully prepared to adopt V3-exclusive capabilities. However, administrators should note that the upgrade is a one-way operation—the Apache Iceberg specification does not support downgrading from V3 to V2.

Implications: The Future of Incremental Data Pipelines

The long-term implications of native Iceberg V3 support on Amazon S3 Tables extend far beyond minor performance tweaks. By introducing row lineage (_row_id and _last_updated_sequence_number), AWS has fundamentally transformed how data engineers build incremental Extract, Transform, Load (ETL) pipelines.

Traditionally, capturing changes in a data lake required expensive full-table scans, partition-based watermarking, or complex change data capture (CDC) state tracking. With V3 row lineage, downstream pipelines can execute highly targeted queries:

SELECT *, _row_id, _last_updated_sequence_number
FROM my_catalog.namespace.clickstream
WHERE _last_updated_sequence_number > 42

By checkpointing sequence numbers, downstream jobs process only newly modified rows on each execution run. This eliminates redundant compute cycles, shrinks pipeline execution windows, and significantly lowers cloud infrastructure expenditures for enterprise data teams.

Availability

Support for all Apache Iceberg V3 data types and capabilities within Amazon S3 Tables is available today across all AWS Regions where S3 Tables are supported. The feature carries no additional licensing fee, with standard S3 Tables storage and request pricing applying. Data engineers can begin provisioning table buckets via the Amazon S3 console, AWS CLI, or integrate management using the AWS MCP Server and associated plugins.