Skip to content

Deprecate FromArrowArray in favour of ArrowSession array imports#8927

Draft
robert3005 wants to merge 2 commits into
rk/deprecatefrom
rk/deprecate-arrow-arrays
Draft

Deprecate FromArrowArray in favour of ArrowSession array imports#8927
robert3005 wants to merge 2 commits into
rk/deprecatefrom
rk/deprecate-arrow-arrays

Conversation

@robert3005

@robert3005 robert3005 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

Stacked on #8918. Second step of making ArrowSession the authoritative Arrow ⇄ Vortex conversion API: this PR deprecates the array-import traits (FromArrowArray; IntoArrowArray was already deprecated and has no remaining users) and migrates every usage to session methods.

What changes are included in this PR?

  • FromArrowArray is #[deprecated], and each of its impl bodies is extracted into a named, directly invocable function in vortex_arrow::convert (from_arrow_primitive, from_arrow_decimal*, from_arrow_temporal, from_arrow_bytes, from_arrow_byte_view, from_arrow_boolean, from_arrow_struct, from_arrow_list, from_arrow_list_view, from_arrow_fixed_size_list, from_arrow_null, from_arrow_dictionary, from_arrow_run_array, from_arrow_dyn, from_arrow_batch). The trait impls are thin shims over these, so the trait can eventually be deleted.
  • ArrowSession gains from_arrow_array_nullable(&dyn Array, nullable) — the field-less import. No top-level extension dispatch is possible without a Field, but nested container fields still route through registered import plugins.
  • ArrowImportVTable::from_arrow_array now receives &ArrowSession (mirroring from_arrow_field from Deprecate all non ArrowSession DType imports and exports #8918); the uuid, geo (×8), json, tensor, and parquet-variant plugins convert their storage through the session.
  • The session is threaded as an argument everywhere — no default-session statics: ParquetVariant::from_arrow_variant{,_nullable} take &ArrowSession (from vtables/kernels), native_geometry_scalar_from_wkb takes &ArrowSession (vortex-duckdb passes its crate session), and vx_array_from_arrow takes a const vx_session* and imports through the FFI schema's Field, so Arrow extension types round-trip.
  • All consumers migrated: vortex-datafusion, vortex-python, vortex-ffi, vortex-tui, vortex-bench, compat-gen, vortex-layout, and the [convert] doc snippet in the vortex facade. RecordBatch imports now go through ArrowSession::from_arrow_record_batch, keeping array dtypes consistent with the session-derived schemas from Deprecate all non ArrowSession DType imports and exports #8918.

What APIs are changed? Are there any user-facing changes?

  • FromArrowArray (trait + method) is deprecated; replacements are ArrowSession::{from_arrow_array, from_arrow_array_nullable, from_arrow_record_batch} and the typed vortex_arrow::convert::from_arrow_* functions.
  • ArrowImportVTable::from_arrow_array gains a session: &ArrowSession parameter (breaking for plugin implementors).
  • ParquetVariant::from_arrow_variant and vortex_geo::extension::native_geometry_scalar_from_wkb gain a session parameter; vx_array_from_arrow gains a const vx_session* parameter (C header regenerated).

🤖 Generated with Claude Code

@codspeed-hq

codspeed-hq Bot commented Jul 23, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 1849 untouched benchmarks
⏩ 46 skipped benchmarks1


Comparing rk/deprecate-arrow-arrays (154677c) with rk/deprecate (452e451)2

Open in CodSpeed

Footnotes

  1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on rk/deprecate (8adb394) during the generation of this report, so 0d65210 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

robert3005 and others added 2 commits July 23, 2026 17:04
Follow-up to #8918: `FromArrowArray` is now #[deprecated] and every usage is
replaced with `ArrowSession` methods, making the session the authoritative
Arrow array import path.

- `ArrowSession` gains `from_arrow_array_nullable(&dyn Array, bool)` for
  imports with no Arrow `Field` in hand; nested fields still dispatch
  extension import plugins.
- `ArrowImportVTable::from_arrow_array` now receives the `ArrowSession` so
  plugins convert storage through the session (uuid, geo, json, tensor,
  parquet-variant updated).
- All consumers (datafusion, python, ffi, tui, bench, compat-gen, layout,
  facade doc snippet) migrated to `from_arrow_array` /
  `from_arrow_array_nullable` / `from_arrow_record_batch`.
- `vx_array_from_arrow` now imports through the session using the FFI
  schema's field, so Arrow extension types round-trip.
- `IntoArrowArray` was already deprecated and has no remaining users.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
…ions

- `ParquetVariant::from_arrow_variant{,_nullable}` take an `&ArrowSession`
  threaded from vtables/kernels, so shredded storage children resolve
  through the registered import plugins.
- `native_geometry_scalar_from_wkb` takes an `&ArrowSession`; vortex-duckdb
  passes its crate session.
- `vx_array_from_arrow` takes a `const vx_session*` (header regenerated).
- Extract each `FromArrowArray` impl body into a named, invocable
  `vortex_arrow::convert::from_arrow_*` function (primitive, decimal,
  temporal, bytes, byte-view, boolean, struct, list, list-view,
  fixed-size-list, null, dictionary, run-ends, dyn dispatch, record batch);
  the deprecated trait impls are now thin shims so the trait can eventually
  be deleted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005
robert3005 force-pushed the rk/deprecate-arrow-arrays branch from 2bdf0b7 to 154677c Compare July 23, 2026 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant