-
Notifications
You must be signed in to change notification settings - Fork 52
Add sponsor project: Numba and llvmlite in the browser #362
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| import useDocusaurusContext from '@docusaurus/useDocusaurusContext'; | ||
| import GetAQuotePage from '@site/src/components/fundable/GetAQuotePage'; | ||
|
|
||
| export default function FundablePage() { | ||
| const { siteConfig } = useDocusaurusContext(); | ||
| return ( | ||
| <GetAQuotePage/> | ||
| ); | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| import useDocusaurusContext from '@docusaurus/useDocusaurusContext'; | ||
| import LargeProjectCardPage from '@site/src/components/fundable/LargeProjectCardPage'; | ||
|
|
||
| export default function FundablePage() { | ||
| const { siteConfig } = useDocusaurusContext(); | ||
| return ( | ||
| <LargeProjectCardPage/> | ||
| ); | ||
| } |
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,77 @@ | ||||||
| #### Overview | ||||||
|
|
||||||
| [Numba](https://numba.pydata.org/) is the standard | ||||||
| just-in-time (JIT) compiler for numerical Python, widely used | ||||||
| across the scientific stack to accelerate compute-heavy code. | ||||||
| [Pyodide](https://pyodide.org/) brings CPython to the browser | ||||||
| through WebAssembly and powers JupyterLite, while | ||||||
| [emscripten-forge](https://github.com/emscripten-forge) | ||||||
| provides a conda-based package distribution for the same | ||||||
| target. Today, Numba cannot run in either environment. | ||||||
|
|
||||||
| We propose to make Numba and its JIT backend, | ||||||
| [llvmlite](https://github.com/numba/llvmlite), | ||||||
| browser-compatible using WebAssembly, and thus enabling | ||||||
| Pyodide and emscripten-forge users to JIT-compile numerical | ||||||
| code just like they do on a native CPython interpreter. | ||||||
|
|
||||||
| #### Why Numba in the Browser Matters | ||||||
|
|
||||||
| In a native CPython environment, many paths exist to make | ||||||
| numerical Python fast: Numba's JIT compilation, | ||||||
| multiprocessing, native C extensions, or offloading to a GPU. | ||||||
| In the browser, multithreading/processing and GPU access are | ||||||
| mostly unavailable, and shipping pre-compiled native | ||||||
| extensions for everything is impractical. In addition, the | ||||||
|
Comment on lines
+24
to
+25
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Isn't "shipping pre-compiled native extensions" exactly what we do in emscripten-forge? Why would it be more impractical for the browser than for native code?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The key word is everything here. We can't ship compiled extensions for every loop and small code snippet. That is what is impractical on native and in the browser. Maybe it can be written better.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But this doesn't make sense either. The equation is for same for Wasm than it is for native code. Performance-critical Python scientific code is never written in pure Python. It's already available either in C/Rust/C++, or as Numba/Cython.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 2-5x slower for pure Python code sounds severe, but it's also small compared to the 50-100x faster that you can get with C/C++/Rust vs. pure Python. So you would want native extensions in the same situations regardless of whether the Python interpreter is running in Wasm or not.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess that depends on the situation and absolute numbers as well, 2x can be decisive or not at all. Underlying is the argument that you have more incentives to JIT compile Python in the browser:
And then you have libraries that use Numba anyway for acceleration and that might work out of the box if we had it available. |
||||||
| overhead of the Python interpreter is larger in the browser | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not convinced the overhead of the Python interpreter is larger in the browser, unless you can link to an independent study.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That's generally cited in the pyodide community. The standard numbers are <2.5x native for compiled stuff (based on a 2019 paper), 2-5x for python (cited on pyodide issues in their roadmap and more based on their benchmarks). If you actually look at the pyodide benchmarks you could say 2 -10x though. I'll put a reference in. I don't think they have rigorously isolated the effect of the interpreter.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pyodide is not independent of course in that sense.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, perhaps link to a recent evaluation by PyOdide, then?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On performance numbers: I did a pretty long search and didn't find papers newer than the 2019 one. That paper "Not so Fast" is already an evolution to the older WASM paper which showed close to native performance for a different benchmark set. I recommend reading both. But the lack of newer studies is why new published benchmarks (even just as blog posts) for us, and for specific domains, are so important. The numbers I recently shared internally are all over the place, sometimes WASM is even a bit faster, often a bit (<2x) and sometimes much slower for compiled (10x+). But much is in the 2-5x range as the often cited numbers. My own numbers aren't able to isolate the interpreter overhead. Pyodide has more numbers, but unfortunately they don't run their CI benchmarks regularly anymore it seems. You need to run them locally or find snapshots in issues. And even with those numbers available, to me the python browser overhead is anecdotal because I am unsure those benchmarks isolate the interpreter well (not because they are bad but because I didn't understand myself whether they do. And again, maybe I didn't find the right sources and others have a better understanding). |
||||||
| and many existing libraries use Numba, sometimes as hard | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| requirements. JIT compilation is therefore an interesting | ||||||
| route to improved performance in a fully client-side, | ||||||
| browser-based Python environment. | ||||||
|
|
||||||
| #### Current Progress | ||||||
|
|
||||||
| We have already patched llvmlite and demonstrated that it runs | ||||||
| in the browser including compilation and execution of the WASM | ||||||
| code it generates. A live demo is available at | ||||||
| https://notebook.link/@anutosh491/llvmlite. | ||||||
|
|
||||||
| We also have a working demo of Numba's `@jit` decorator | ||||||
| operating on scalar functions in the browser. However, several | ||||||
| issues have been identified that must be overcome to make | ||||||
| Numba work generally (e.g. with arrays). These likely include | ||||||
| WebAssembly's lack of writable-and-executable memory pages | ||||||
| (which rules out MCJIT), unsupported atomic instructions | ||||||
| without shared memory, variadic C calls that cannot be lowered | ||||||
| to WASM, and linkage incompatibilities in the LLVM WASM PIC | ||||||
| backend. But these seem surmountable based on our experience. | ||||||
|
Comment on lines
+42
to
+47
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would remove the details, some of which are probably wrong.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yea could do. It would be nice to show that we have some idea about the non trivial parts though. |
||||||
|
|
||||||
| #### Why QuantStack | ||||||
|
|
||||||
| QuantStack is uniquely positioned to deliver this work. We | ||||||
| have shipped several JIT and WebAssembly projects in the | ||||||
| scientific Python ecosystem, including | ||||||
| [xeus-cpp](https://github.com/jupyter-xeus/xeus-cpp), a C++ | ||||||
| interpreter running in the browser via Clang and LLVM compiled | ||||||
| to WebAssembly. Our team also includes Numba core developers, | ||||||
| giving us a direct upstream relationship. | ||||||
|
Comment on lines
+56
to
+57
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not even sure I still have commit rights on Numba, and we have nobody else in the Numba team. So this sentence is a lie AFAICT.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, core developer is not correct? Or the upstream relationship?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have not contributed to Numba since 2017 or 2018, I think. I don't have any reliable evaluation of my upstream relationship, but I wouldn't count on it. By default you cannot assume that we are in a privileged position. I do have expertise in Numba, though, and the internals don't seem to have changed much. So you could stress that we have in-house Numba expertise (and such expertise is rather rare).
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok that sounds good to me. We can just write it as such. |
||||||
|
|
||||||
| #### Proposed Work | ||||||
|
|
||||||
| The goal of this project is to: | ||||||
|
|
||||||
| - upstream llvmlite compatibility with browser environments, | ||||||
| based on our existing patches. | ||||||
| - build a proof of concept for general Numba use in the browser to | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would say it's up to us to show a proof of concept before getting some funding. A sponsor will want dependable upstream changes for full-blown Numba functionality.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I thought about this as well. But my understanding is that the proof of concept itself is a significant undertaking, and probably the main challenge right now. Under that hypothesis, funding already for the PoC would be necessary because otherwise we may not be able to stem it ourselves.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure the effort to demonstrate JIT-acceleration of NumPy array code is that significant. @anutosh491 could perhaps give more insight. |
||||||
| - demonstrate, ideally all of Numba, or at least a large part of it, in the browser | ||||||
| - demonstrate the ability to run libraries that depend on it (such as pytensor/pymc) in the browser | ||||||
| - gather and understand the changes required | ||||||
| - upstream the required changes | ||||||
|
|
||||||
| Note that this is a proof of concept effort, we will advance | ||||||
| incrementally, deliver sub-parts, and adjust the plan based | ||||||
| on new learnings. | ||||||
|
|
||||||
| ##### Are you interested in this project? Either entirely or | ||||||
| partially, contact us for more information on how to help us | ||||||
| fund it. | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's remove the MCJIT tidbit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes