feat(inspect): implement SnapshotsTable scanning by WZhuo · Pull Request #801 · apache/iceberg-cpp

WZhuo · 2026-07-02T08:35:38Z

Summary

Implement Scan() for SnapshotsTable, adding the core scanning infrastructure to MetadataTable and materializing snapshot rows into Arrow arrays via ArrowRowBuilder.

Changes

`metadata_table.h` / `.cc`

Add SnapshotSelection struct with snapshot_id, as_of_timestamp, ref_name optional fields for time-travel queries
Add Scan() virtual method returning Result<ArrowArray> with a convenience zero-arg overload
Add supports_time_travel() concrete method driven by kind(), matching Java's TIME_TRAVEL_TABLE_TYPES static set (currently both kSnapshots and kHistory return false)
Default Scan() returns NotSupported

`snapshots_table.h` / `.cc`

Declare and implement SnapshotsTable::Scan() — iterates source_table()->snapshots() and writes 6 columns (committed_at, snapshot_id, parent_id, operation, manifest_list, summary) via ArrowRowBuilder
committed_at converted to microseconds since epoch, matching Java's snap.timestampMillis() \* 1000

Tests

metadata_table_test_base.h — shared MetadataTableTestBase with MockFileIO + MockCatalog fixture, FinishAndImport(), MakeTestSnapshots(), MakeTableWithSnapshots() helpers
metadata_table_test.cc — 2 framework tests (FactoryRejectsNullSourceTable, SupportsTimeTravel)
snapshots_table_test.cc — 11 tests (Construct, SchemaMatchesIcebergSchema, 9 scan data tests matching Java TestDataTaskParser fixture)
history_table_test.cc — 1 schema test

- Add Scan() virtual method and Scan() convenience overload to MetadataTable - Add SnapshotSelection struct for time-travel snapshot resolution - Add supports_time_travel() concrete method driven by kind() - Implement SnapshotsTable::Scan() to materialize snapshot rows via ArrowRowBuilder

Copilot

Pull request overview

Implements initial metadata-table scanning support by adding a Scan() API to MetadataTable (with snapshot-selection parameters for future time-travel) and providing a concrete SnapshotsTable::Scan() implementation that materializes snapshot rows into Arrow arrays. The PR also restructures/extends the metadata-table test suite to validate schemas and snapshot scanning behavior.

Changes:

Added SnapshotSelection and a virtual MetadataTable::Scan() API (with a convenience overload) plus supports_time_travel().
Implemented SnapshotsTable::Scan() to emit snapshot rows (6 columns) via ArrowRowBuilder.
Added/expanded tests and wired new test sources into the metadata-table test target.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/iceberg/inspect/metadata_table.h	Adds `SnapshotSelection`, `Scan()` API, and `supports_time_travel()` declaration/docs.
src/iceberg/inspect/metadata_table.cc	Implements default `Scan()` + `supports_time_travel()` and wires factory for kinds.
src/iceberg/inspect/snapshots_table.h	Declares `SnapshotsTable::Scan()` override.
src/iceberg/inspect/snapshots_table.cc	Implements snapshot scanning into Arrow via `ArrowRowBuilder`.
src/iceberg/test/metadata_table_test.cc	Simplifies base setup and adds `SupportsTimeTravel` test.
src/iceberg/test/metadata_table_test_base.h	New shared fixture/helpers for metadata table tests.
src/iceberg/test/snapshots_table_test.cc	New tests validating snapshots table construction/schema/scan output.
src/iceberg/test/history_table_test.cc	New schema test for history table.
src/iceberg/test/CMakeLists.txt	Adds new test sources to the metadata-table test target.

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/iceberg/inspect/metadata_table.cc:38

supports_time_travel() is currently hard-coded to return false. That matches today’s two Kind values, but it’s easy to forget to update once additional metadata table kinds are added, and it doesn’t reflect the docstring/PR description that this is kind-driven. Consider switching on kind() and making the non-exhaustive case unreachable to keep future additions honest.

bool MetadataTable::supports_time_travel() const noexcept { return false; }

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

+    metadata_ = std::make_shared<TableMetadata>(
+        TableMetadata{.format_version = 2, .schemas = {schema}, .current_schema_id = 1});


+    auto metadata = std::make_shared<TableMetadata>(
        TableMetadata{.format_version = 2, .schemas = {schema}, .current_schema_id = 1});


+TEST_F(SnapshotsTableTest, ScanEmptySnapshotList) {
+  // A table with zero snapshots should return zero rows.
+  ICEBERG_UNWRAP_OR_FAIL(auto empty_table,
+                         MakeTableWithSnapshots({}, /*current_snapshot_id=*/-1));


Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/iceberg/inspect/metadata_table.cc:38

supports_time_travel() is documented (and described in the PR) as being driven by the metadata table kind, but the implementation always returns false regardless of kind. This makes the API misleading and forces future kinds to remember to update callers rather than the method itself.

bool MetadataTable::supports_time_travel() const noexcept { return false; }

+  for (const auto& snapshot : source_table()->snapshots()) {
+    // column 0: committed_at (timestamptz -> int64 micros)


+  static std::shared_ptr<::arrow::RecordBatch> FinishAndImport(ArrowArray&& array,
+                                                               const Schema& schema) {
+    ArrowSchema c_schema{};
+    EXPECT_THAT(ToArrowSchema(schema, &c_schema), IsOk());
+    auto arrow_schema = ::arrow::ImportSchema(&c_schema).ValueOrDie();
+
+    // ImportRecordBatch takes ownership of the array and releases it.
+    return ::arrow::ImportRecordBatch(&array, arrow_schema).ValueOrDie();
+  }


WZhuo marked this pull request as ready for review July 2, 2026 08:46

WZhuo force-pushed the inspect branch 3 times, most recently from 28c4513 to a53f8ce Compare July 6, 2026 03:37

WZhuo changed the title ~~feat(inspect): expand MetadataTable framework for all 16 metadata table types~~ feat(inspect): implement SnapshotsTable scanning Jul 6, 2026

WZhuo force-pushed the inspect branch 3 times, most recently from 9bc23e1 to 04b657f Compare July 7, 2026 02:04

WZhuo force-pushed the inspect branch from 04b657f to efc76b8 Compare July 14, 2026 06:23

WZhuo force-pushed the inspect branch from efc76b8 to 3b11911 Compare July 17, 2026 02:19

manuzhang added this to the 0.4.0 milestone Jul 21, 2026

manuzhang requested a review from Copilot July 21, 2026 03:39

Copilot started reviewing on behalf of manuzhang July 21, 2026 03:42 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Comment thread src/iceberg/inspect/metadata_table.h Outdated

Comment thread src/iceberg/inspect/metadata_table.cc Outdated

Comment thread src/iceberg/test/metadata_table_test_base.h Outdated

Comment thread src/iceberg/inspect/metadata_table.h Outdated

fix(inspect): address metadata table review feedback

6c112cc

Copilot AI review requested due to automatic review settings July 22, 2026 05:32

Copilot started reviewing on behalf of WZhuo July 22, 2026 05:33 View session

Copilot AI reviewed Jul 22, 2026

View reviewed changes

Comment thread src/iceberg/test/snapshots_table_test.cc

Comment thread src/iceberg/test/snapshots_table_test.cc

test(inspect): address snapshots table review feedback

791474e

Copilot AI review requested due to automatic review settings July 23, 2026 01:44

Copilot started reviewing on behalf of WZhuo July 23, 2026 01:45 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread src/iceberg/test/metadata_table_test_base.h Outdated

Potential fix for pull request finding

a5c20c2

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 24, 2026 01:38

Copilot started reviewing on behalf of WZhuo July 24, 2026 01:38 View session

Fix formatting issues in metadata_table_test_base.h

ab62d36

Copilot AI reviewed Jul 24, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 24, 2026 01:43

Copilot started reviewing on behalf of WZhuo July 24, 2026 01:44 View session

Copilot AI reviewed Jul 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(inspect): implement SnapshotsTable scanning#801

feat(inspect): implement SnapshotsTable scanning#801
WZhuo wants to merge 5 commits into
apache:mainfrom
WZhuo:inspect

WZhuo commented Jul 2, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		metadata_ = std::make_shared<TableMetadata>(
		TableMetadata{.format_version = 2, .schemas = {schema}, .current_schema_id = 1});

		auto metadata = std::make_shared<TableMetadata>(
		TableMetadata{.format_version = 2, .schemas = {schema}, .current_schema_id = 1});

		for (const auto& snapshot : source_table()->snapshots()) {
		// column 0: committed_at (timestamptz -> int64 micros)

Uh oh!

Conversation

WZhuo commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

metadata_table.h / .cc

snapshots_table.h / .cc

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

WZhuo commented Jul 2, 2026 •

edited

Loading

`metadata_table.h` / `.cc`

`snapshots_table.h` / `.cc`