Deprecate FromArrowArray in favour of ArrowSession array imports#8927
Draft
robert3005 wants to merge 2 commits into
Draft
Deprecate FromArrowArray in favour of ArrowSession array imports#8927robert3005 wants to merge 2 commits into
robert3005 wants to merge 2 commits into
Conversation
Merging this PR will not alter performance
Comparing Footnotes
|
Follow-up to #8918: `FromArrowArray` is now #[deprecated] and every usage is replaced with `ArrowSession` methods, making the session the authoritative Arrow array import path. - `ArrowSession` gains `from_arrow_array_nullable(&dyn Array, bool)` for imports with no Arrow `Field` in hand; nested fields still dispatch extension import plugins. - `ArrowImportVTable::from_arrow_array` now receives the `ArrowSession` so plugins convert storage through the session (uuid, geo, json, tensor, parquet-variant updated). - All consumers (datafusion, python, ffi, tui, bench, compat-gen, layout, facade doc snippet) migrated to `from_arrow_array` / `from_arrow_array_nullable` / `from_arrow_record_batch`. - `vx_array_from_arrow` now imports through the session using the FFI schema's field, so Arrow extension types round-trip. - `IntoArrowArray` was already deprecated and has no remaining users. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Robert Kruszewski <github@robertk.io>
…ions
- `ParquetVariant::from_arrow_variant{,_nullable}` take an `&ArrowSession`
threaded from vtables/kernels, so shredded storage children resolve
through the registered import plugins.
- `native_geometry_scalar_from_wkb` takes an `&ArrowSession`; vortex-duckdb
passes its crate session.
- `vx_array_from_arrow` takes a `const vx_session*` (header regenerated).
- Extract each `FromArrowArray` impl body into a named, invocable
`vortex_arrow::convert::from_arrow_*` function (primitive, decimal,
temporal, bytes, byte-view, boolean, struct, list, list-view,
fixed-size-list, null, dictionary, run-ends, dyn dispatch, record batch);
the deprecated trait impls are now thin shims so the trait can eventually
be deleted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Robert Kruszewski <github@robertk.io>
robert3005
force-pushed
the
rk/deprecate-arrow-arrays
branch
from
July 23, 2026 16:23
2bdf0b7 to
154677c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Stacked on #8918. Second step of making
ArrowSessionthe authoritative Arrow ⇄ Vortex conversion API: this PR deprecates the array-import traits (FromArrowArray;IntoArrowArraywas already deprecated and has no remaining users) and migrates every usage to session methods.What changes are included in this PR?
FromArrowArrayis#[deprecated], and each of its impl bodies is extracted into a named, directly invocable function invortex_arrow::convert(from_arrow_primitive,from_arrow_decimal*,from_arrow_temporal,from_arrow_bytes,from_arrow_byte_view,from_arrow_boolean,from_arrow_struct,from_arrow_list,from_arrow_list_view,from_arrow_fixed_size_list,from_arrow_null,from_arrow_dictionary,from_arrow_run_array,from_arrow_dyn,from_arrow_batch). The trait impls are thin shims over these, so the trait can eventually be deleted.ArrowSessiongainsfrom_arrow_array_nullable(&dyn Array, nullable)— the field-less import. No top-level extension dispatch is possible without aField, but nested container fields still route through registered import plugins.ArrowImportVTable::from_arrow_arraynow receives&ArrowSession(mirroringfrom_arrow_fieldfrom Deprecate all non ArrowSession DType imports and exports #8918); the uuid, geo (×8), json, tensor, and parquet-variant plugins convert their storage through the session.ParquetVariant::from_arrow_variant{,_nullable}take&ArrowSession(from vtables/kernels),native_geometry_scalar_from_wkbtakes&ArrowSession(vortex-duckdb passes its crate session), andvx_array_from_arrowtakes aconst vx_session*and imports through the FFI schema'sField, so Arrow extension types round-trip.[convert]doc snippet in thevortexfacade. RecordBatch imports now go throughArrowSession::from_arrow_record_batch, keeping array dtypes consistent with the session-derived schemas from Deprecate all non ArrowSession DType imports and exports #8918.What APIs are changed? Are there any user-facing changes?
FromArrowArray(trait + method) is deprecated; replacements areArrowSession::{from_arrow_array, from_arrow_array_nullable, from_arrow_record_batch}and the typedvortex_arrow::convert::from_arrow_*functions.ArrowImportVTable::from_arrow_arraygains asession: &ArrowSessionparameter (breaking for plugin implementors).ParquetVariant::from_arrow_variantandvortex_geo::extension::native_geometry_scalar_from_wkbgain a session parameter;vx_array_from_arrowgains aconst vx_session*parameter (C header regenerated).🤖 Generated with Claude Code