8000 WAMR upgrade by csegarragonz · Pull Request #734 · faasm/faasm · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

WAMR upgrade #734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 50 commits into from
Apr 4, 2023
Merged

WAMR upgrade #734

merged 50 commits into from
Apr 4, 2023

Conversation

csegarragonz
Copy link
Collaborator
@csegarragonz csegarragonz commented Mar 28, 2023

In this PR I bump the WAMR reference, further simplify our diff with upstream WAMR, and use the newest public API calls to invoke wasm functions.

Most importantly, as part of the diff simplification, I remove some places in the WAMR codebase where we were hardcoding the memory layout. This was to prevent WAMR from doing some optimisations to the WASM's module memory. Instead of hacking WAMR (harder to maintain), I move to using a workaround in our compilation toolchain. I agree that all Faasm-compiled WASM modules will have this workaround (even if they don't run in WAMR), but the overhead is minimal (just a very small function that isn't even called), and really helps maintaining alignment with upstream WAMR. See faasm/cpp#113

I also make a couple of fixes here and there that make LAMMPS work on WAMR (!).

To-Do before merge:

This PR makes LAMMPS work on WAMR including migration (!).

@csegarragonz csegarragonz self-assigned this Mar 28, 2023
@csegarragonz csegarragonz marked this pull request as ready for review March 30, 2023 15:10
@csegarragonz csegarragonz force-pushed the wamr-upgrade branch 3 times, most recently from d020c48 to a603582 Compare March 30, 2023 16:00

// Set dummy argv to capture return value
std::vector<uint32_t> argv = { 0 };
bool success = wasm_runtime_call_wasm(execEnv, func, 0, argv.data());
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only reason I change this here is because aot_create_exec_env_and_call_function is not even exposed as an API function anymore, so this file would not compile.

@@ -32,7 +28,7 @@ int doRunner(int argc, char* argv[])
usleep(1000 * 500);

for (const auto& m : req->messages()) {
faabric::Message result = sch.getFunctionResult(m.id(), 20000);
faabric::Messa 8000 ge result = sch.getFunctionResult(m.id(), 20000 * 100);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the local pool runner it makes sense to have longer timeouts, as we may want to run some longer running functions for debug purposes.

As a reminder, the local pool runner is a pool_runner that can be invoked from the command line. For example, the following runs a LAMMPS simulation without requiring any changes to faasm (i.e. any new lammps_runner etc):

 WASM_VM=wamr inv run.pool lammps main --cmdline '-in faasm://lammps-data/in.controller.wall'

@@ -11,6 +11,7 @@ set(WAMR_BUILD_SPEC_TEST 0)
add_definitions(-DWAMR_FAASM=1)

# Set AOT mode and JIT for code generation
set(WAMR_BUILD_INTERPRETER 1)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was set by default implicitly through setting WAMR_BUILD_AOT, but I prefer to make it explicit.

uint32_t returnValue = argv[0];

// Check function result
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Luckily, WAMR's error checking has improved since we first wrote this code, so we don't need such cumbersome checks.

// Note, for some reason WAMR sets the return value in the argv array you
// pass it, therefore we should provide a single integer argv even though
// it's not actually used
std::vector<uint32_t> argv = { 0 };

// Invoke the function
bool success = aot_create_exec_env_and_call_function(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function was never part of the public API, and I think its not even available anymore.

We use the public API function wasm_runtime_call_wasm.

{
module->validateNativePointer(requestPtrPtr, sizeof(MPI_Request));
MPI_Request* requestPtr = reinterpret_cast<MPI_Request*>(requestPtrPtr);
faabric::util::unalignedWrite<faabric_request_t*>(
Copy link
Collaborator Author
@csegarragonz csegarragonz Mar 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unalignedWrite was actually smashing other values in the WASM stack, what was making LAMMPS hang mid execution, and what made me have a very entretaining week 🤣

SPDLOG_WARN("MEM - unable to reclaim unmapped memory {} at {}",
pageAligned,
offset);
// TODO - this log statement should be a warning, but for some reason
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, I don't know what's going on with this, but I will look into it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now fixed as per faasm/cpp#113

{
auto moduleInstance = this->underlying().getModuleInstance();
uint32_t wasmOffset =
wasm_runtime_module_malloc(moduleInstance, size, nativePtr);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This essentially calls the malloc symbol in wasi-libc. Hence why we need to export it here.

@csegarragonz csegarragonz requested a review from Shillaker March 31, 2023 10:35
@csegarragonz csegarragonz merged commit 07408c4 into main Apr 4, 2023
@csegarragonz csegarragonz deleted the wamr-upgrade branch April 4, 2023 12:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant
0