Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory allocation decoupling #751

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

andreaTP
Copy link
Collaborator

@andreaTP andreaTP commented Jan 31, 2025

Problem:

Working on using Chicory with real-world Wasm payloads, I noticed a huge pressure on the memory_grow op code.
Basically, some C module, will keep growing the memory in very very small chunks, and, currently, this operation is really expensive as it needs a full copy to a new ByteBuffer.

Proposal:

While it's definitely possible to tweak the WASM module memory allocator and swap in something more optimized, it's much better for us to decouple the guest module consumption.
In this way, we can decide to plug in more advanced and sophisticated heuristics based on the usage.

Implementation:

The implementation puts in places the necessary pieces, the "Default" allocator is not greatly smarter than the previous, but it already improves performance in the use case I was looking at.
It should be a bit better overall, but it definitely has margin for improvement.

Notes:

I provided an implementation to preserve the current behavior when necessary, but this is a behavioral change and we should roll it (if approved) with a minor version bump (i.e. 1.1.X).

@electrum : I'd be extremely interested if you take a little time to try out those changes with Python 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant