add preprocessing benchmarking#216
Conversation
| "generation_kwargs": [ | ||
| { "n_samples": 100000, "n_features": 5000, "noise": 1.0 }, | ||
| { "n_samples": 1000000, "n_features": 500, "noise": 1.0 }, | ||
| { "n_samples": 10000000, "n_features": 50, "noise": 1.0 } |
There was a problem hiding this comment.
Perhaps it could add cases with sparse data instead of something this large.
There was a problem hiding this comment.
https://github.com/uxlfoundation/scikit-learn-intelex/blob/main/sklearnex/preview/preprocessing/_data.py#L83 unfortunately not something we handle in the current moment (due to IncrementalBasicStatistics not handling sparse data).
There was a problem hiding this comment.
Also the reason for such big data is to match order of magnitude to linear regression sizing https://github.com/IntelPython/scikit-learn_bench/blob/main/configs/regular/linear_model.json#L11
|
@icfaust Were you able to run this in a CI job? |
Saw that CI was popping and held back. Will be testing it once CIs pass to see what comes out. |
2 participants
Description
Following uxlfoundation/scikit-learn-intelex#3020
Done with claude.
Checklist:
Completeness and readability
Testing