Issue to execute function when on k8 #895

krho999 · 2024-11-24T14:03:16Z

Hey,

I have an issue executing anything on the deployed k8 cluster created on 4 VMs with kubeadm.
As a start I have ran the faasmctl deploy.k8 which created the init file and I tested the endpoints via curl to be sure.

I had to add this to the ymls as I am executing stuff on control-plane and it didn`t want to spin without it

 tolerations:
  - key: "node-role.kubernetes.io/control-plane"
    operator: "Exists"
    effect: "NoSchedule"

There was no change in env vars in the ymls files, I am running the wamr worker and upload ymls and i can upload and compile the hello.cpp with the cpp.cli container however after running compile and successful upload with 200, the invoke will get to infinit loop on waiting for response as the worker pod will crash with

[13:41:21] [1] [I] Starting state server
[13:41:21] [1] [I] Starting snapshot server
[13:41:22] [1] [I] Starting point-to-point server
[13:41:22] [1] [I] Starting function call server
[13:41:23] [1] [I] Starting endpoint
[13:41:23] [1] [I] Starting HTTP endpoint on 8080, 4 threads
Caught stack backtrace:
/build/faasm/bin/pool_runner(_ZN7faabric4util11handleCrashEi+0x21)[0x561a3b750e81]
/build/faasm/bin/pool_runner(_Z12crashHandleri+0x9)[0x561a3b750e49]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fc841686520]
/build/faasm/bin/pool_runner(aot_call_function+0x109)[0x561a3997aa39]
/build/faasm/bin/pool_runner(wasm_runtime_call_wasm+0x3c1)[0x561a3994e831]
/build/faasm/bin/pool_runner(_ZN4wasm14WAMRWasmModule21executeCatchExceptionEiPviiRSt6vectorIjSaIjEE+0xbf)[0x561a380e07df]
/build/faasm/bin/pool_runner(_ZN4wasm14WAMRWasmModule19executeWasmFunctionEiRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0x6e)[0x561a380dff6e]
/build/faasm/bin/pool_runner(_ZN4wasm14WAMRWasmModule15executeFunctionERN7faabric7MessageE+0x74)[0x561a380dfa64]
/build/faasm/bin/pool_runner(_ZN4wasm10WasmModule11executeTaskEiiSt10shared_ptrIN7faabric19BatchExecuteRequestEE+0x115)[0x561a3810d825]
/build/faasm/bin/pool_runner(_ZN7faaslet7Faaslet11executeTaskEiiSt10shared_ptrIN7faabric19BatchExecuteRequestEE+0x1e5)[0x561a3809e0d5]
/build/faasm/bin/pool_runner(_ZN7faabric8executor8Executor16threadPoolThreadESt10stop_tokeni+0x2b6)[0x561a3b6ca756]
/build/faasm/bin/pool_runner(_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJSt11_Bind_frontIMN7faabric8executor8ExecutorEFvSt10stop_tokeniEJPS6_EES7_iEEEEE6_M_runEv+0x3b)[0x561a3b6ceb6b]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253)[0x7fc841969253]
/lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7fc8416d8ac3]
/lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x7fc84176a850]
[13:54:07] [79] [E] Destructing scheduler without shutting down first
pure virtual method called
terminate called without an active exception
Caught stack backtrace:
/build/faasm/bin/pool_runner(_ZN7faabric4util11handleCrashEi+0x21)[0x561a3b750e81]
/build/faasm/bin/pool_runner(_Z12crashHandleri+0x9)[0x561a3b750e49]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fc841686520]
/lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7fc8416da9fc]
/lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7fc841686476]
/lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x7fc84166c7f3]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2b9e)[0x7fc84192fb9e]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xae20c)[0x7fc84193b20c]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xae277)[0x7fc84193b277]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xaefa5)[0x7fc84193bfa5]
/build/faasm/bin/pool_runner(+0x786a7bf)[0x561a3b74c7bf]
/lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc253)[0x7fc841969253]
/lib/x86_64-linux-gnu/libc.so.6(+0x94ac3)[0x7fc8416d8ac3]
/lib/x86_64-linux-gnu/libc.so.6(+0x126850)[0x7fc84176a850]

This seems like something either I did wrong when building this VMs and connecting them as cluster or something I did wrong when deploying the k8 via faasmctl. Does anyone have an idea where I could be messing up?
The VMs are ubuntu VMS with:
OS Image: Ubuntu 22.04.5 LTS
Operating System: linux
Architecture: amd64
cpu: 4

Any help is appreciated, the overall goal is trying to run different HPC workloads on this and see how this performs.

Kind regards

The text was updated successfully, but these errors were encountered:

csegarragonz · 2024-11-25T16:02:37Z

Hi @krho999 , many thanks for using Faasm!

How have you deployed Faasm on Kubernetes? Have you used faasmctl?
We deploy Faasm regularly on Azure's Kubernetes Service, also as part of our CI/CD, so I would expect it to work.

Also, if you are getting started with Faasm, I would recommend deploying on a docker compose cluster first. Have you tried setting a developlment cluster?

krho999 · 2024-12-09T20:12:07Z

Hey,

yes I run the compose version and could execute your h 8000 ellomp and hellompi but when running custom script I will get the pool_runner error

2024-12-09 21:08:39 Error: Number of processes must be 2
2024-12-09 21:06:13 Testing crash handler backtrace:
2024-12-09 21:06:13 /build/faasm/bin/pool_runner(_ZN7faabric4util17setUpCrashHandlerEi+0x76)[0x5600ac8e5f86]
2024-12-09 21:06:13 /build/faasm/bin/pool_runner(_ZN7faabric6runner11FaabricMain15startBackgroundEv+0x21)[0x5600ac89d481]
2024-12-09 21:06:13 /build/faasm/bin/pool_runner(main+0xfe)[0x5600a92301ee]
2024-12-09 21:06:13 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f1508229d90]
2024-12-09 21:06:13 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f1508229e40]
2024-12-09 21:06:13 /build/faasm/bin/pool_runner(_start+0x25)[0x5600a9230025]
2024-12-09 21:07:58 Testing crash handler backtrace:
2024-12-09 21:07:58 /build/faasm/bin/pool_runner(_ZN7faabric4util17setUpCrashHandlerEi+0x76)[0x55ad46039f86]
2024-12-09 21:07:58 /build/faasm/bin/pool_runner(_ZN7faabric6runner11FaabricMain15startBackgroundEv+0x21)[0x55ad45ff1481]
2024-12-09 21:07:58 /build/faasm/bin/pool_runner(main+0xfe)[0x55ad429841ee]
2024-12-09 21:07:58 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f4eaea29d90]
2024-12-09 21:07:58 /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f4eaea29e40]
2024-12-09 21:07:58 /build/faasm/bin/pool_runner(_start+0x25)[0x55ad42984025]

this is from the docker on my Mac. The script is MPI Cartesian implementation I can add it here in case you want to have a look. When I use a normal HPW cluster the MPI works with no issue for me so not sure what can be wrong here.

I might rework it from Cartesian to normal Rank MPI com and see if that helps, unfortunately on the kodeamn cluster I have from VMs none of the scripts work so I think that might be a hardware issue

Small update, After refactoring to common world coms it still throws different error regarding gatherRv but the script caalculates.

#include <iostream>
#include <chrono>
#include <cmath>
#include <mpi.h>

#include "a1-helpers.hpp"

using namespace std;

int main(int argc, char **argv)
{
    int max_iterations = 2000;
    double epsilon = 1.0e-3;

    // default values for M rows and N columns
    int N = 4200;
    int M = 4200;

    //process_input(argc, argv, N, M, nx, ny, max_iterations, epsilon);
    
    // Note that you need to initialize MPI first

    int rank, size;

    MPI_Init(&argc, &argv);
    MPI_Status status;
    MPI_Request req;
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    auto time_compute_1 = MPI_Wtime();
    
    int i, j;
    double diffnorm;
    int iteration_count = 0;
    double pDiffnorm;
    
    int local_rows = M / size; 
    int local_cols = N;         

    if (rank == 0) {
        local_rows += M % size;
    } 

    // You can also use Mat data struct (see helpers), in which case you allocate like: Mat U(M, N) 
    // and then use it like a regular 2D array
    Mat pU(local_rows + 2, local_cols + 2);
    Mat W(local_rows + 2, local_cols + 2);
    Mat U(M,N);
    
    for (int j = 0; j < local_rows + 2; j++) {
        pU(j, 1) = 20.0;  
    }

    for (int j = 0; j < local_rows + 2; j++) {
        pU(j, local_cols) = 40.0; 
    }

    if (rank == size - 1) {  
        for (int i = 0; i < local_cols + 2; i++) {
            pU(local_rows, i) = 100.0;  
        }
    }

    iteration_count = 0;
    do
    {
        iteration_count++;
        diffnorm = 0.0;
        pDiffnorm = 0.0;

        if(rank == 0){
            for (i = 2; i < local_rows + 1; ++i) {
                for (j = 2; j < local_cols + 1; ++j) {
                        W(i, j) = (pU(i, j + 1) + pU(i, j - 1) + pU(i + 1, j) + pU(i - 1, j)) * 0.25;
                        pDiffnorm += (W(i, j) - pU(i, j)) * (W(i, j) - pU(i, j));
                    }
            }
            for (i = 2; i < local_rows + 1; ++i)
                for (j = 2; j < local_cols + 1; ++j)
                    pU(i, j) = W(i, j);
            
            // Send the data to the processes
            MPI_Isend(&pU(local_rows, 1), local_cols, MPI_DOUBLE, rank+1, 0, MPI_COMM_WORLD, &req); // Send to the bottom
            MPI_Recv(&pU(local_rows + 1, 1), local_cols, MPI_DOUBLE, rank+1, 0, MPI_COMM_WORLD, &status); // Receive from any source
        }

        if(rank != 0 && rank != size - 1){
            for (i = 2; i < local_rows + 1; ++i) {
                for (j = 1; j < local_cols + 1; ++j) {
                        W(i, j) = (pU(i, j + 1) + pU(i, j - 1) + pU(i + 1, j) + pU(i - 1, j)) * 0.25;
                        pDiffnorm += (W(i, j) - pU(i, j)) * (W(i, j) - pU(i, j));
                    }
            }
            for (i = 2; i < local_rows + 1; ++i)
                for (j = 1; j < local_cols + 1; ++j)
                    pU(i, j) = W(i, j);

            MPI_Isend(&pU(local_rows, 1), local_cols, MPI_DOUBLE, rank+1, 0, MPI_COMM_WORLD, &req); // Send to the bottom
            MPI_Isend(&pU(1, 1), local_cols, MPI_DOUBLE, rank-1, 0, MPI_COMM_WORLD, &req); // Send to the top

            MPI_Recv(&pU(local_rows + 1, 1), local_cols, MPI_DOUBLE, rank+1, 0, MPI_COMM_WORLD, &status); // Receive from the bottom
            MPI_Recv(&pU(0, 1), local_cols, MPI_DOUBLE, rank-1, 0, MPI_COMM_WORLD, &status); // Receive from the top
        }

        if(rank == size - 1){
            for (i = 1; i < local_rows; ++i) {
                for (j = 1; j < local_cols + 1; ++j) {
                        W(i, j) = (pU(i, j + 1) + pU(i, j - 1) + pU(i + 1, j) + pU(i - 1, j)) * 0.25;
                        pDiffnorm += (W(i, j) - pU(i, j)) * (W(i, j) - pU(i, j));
                    }
            }
            for (i = 1; i < local_rows; ++i)
                for (j = 1; j < local_cols + 1; ++j)
                    pU(i, j) = W(i, j);

            MPI_Isend(&pU(1, 1), local_cols, MPI_DOUBLE, rank-1, 0, MPI_COMM_WORLD, &req); // Send to the top
            MPI_Recv(&pU(0, 1), local_cols, MPI_DOUBLE, rank-1, 0, MPI_COMM_WORLD, &status); // Receive from the top
        }

        MPI_Allreduce(&pDiffnorm, &diffnorm, 1, MPI_DOUBLE, MPI_SUM, MPI_COMM_WORLD); // allreduce diffnorm type sum
        diffnorm = sqrt(diffnorm); // all processes need to know when to stop
    } while (epsilon <= diffnorm && iteration_count < max_iterations);

    // // Now gather the data with MPI_Gatherv using MPI_COMM_WORLD
    // int* counts = new int[size];
    // int* displs = new int[size];
    // int local_size = local_rows * local_cols;

    // // Populate counts and displs arrays for gathering the data from all processes
    // for (int i = 0; i < size; ++i) {
    //     counts[i] = local_size; // Each process sends its local size
    //     displs[i] = i * local_size; // Displacements based on rank
    // }

    // // Gather the result into the global U matrix
    // MPI_Gatherv(&pU(0, 0), local_size, MPI_DOUBLE, &U(0, 0), counts, displs, MPI_DOUBLE, 0, MPI_COMM_WORLD);

    auto time_compute_2 = MPI_Wtime();

    if(rank == 0){
        Mat seqU(M,N);
        heat2d_sequential(max_iterations, epsilon, seqU, iteration_count);
        cout << "Computed in " << iteration_count << " iterations and " << chrono::duration<double>(time_compute_2 - time_compute_1).count() << " seconds." << endl;
        cout << "Final check: " << ( verify(seqU, U) ? "OK" : "NOT OK") << endl;
    }

    MPI_Finalize();

    return 0;
}

I have commented out the GatherRv as that has a different error running in the planner saying it could not map the MPI function.

csegarragonz · 2024-12-10T10:34:31Z

Hey @krho999 , very interesting to see that you are using Faasm's MPI support!

We have done a lot of work to support MPI and OpenMP, particularly extending faabric to act as a messaging and state layer. You can find a lot of scripts to execute large-scale MPI/OpenMP jobs here, and the code to cross-compile them here.

Regarding MPI_Gatherv, it is not implemented. Long story short is that to run MPI applications with Faasm we need to implement every MPI symbol, and we have only implemented the ones needed to run our evaluation workloads. See here. You can see the list of MPI functions that are cross-compiled and executed in each CI/CD run here.

Regarding cartessian communicators, they should work. IIRC we only support 2D communicators, but not 3D. In the src/wamr/mpi.cpp file that I linked before, you will see that all functions end up calling an MPI implementation in faabric. This is the relevant file, where you will see the implementation of the cartesian communicator.

If you let me know more about your use case I may be able to help further.

Lastly, is it possible that to deploy Faasm on a self-managed K8s cluster (not on AKS) you need to do a bit more work. But it should be relatively straightforward.

krho999 · 2024-12-11T17:59:46Z

Thanks for all the information.

Regarding the MPI, I have changed the implementation as above and the calculation works now with rank so that is okay, I can rework the gatherv and have that also work with some other MPI called FAASM supports.

For the customer K8, I was able to deploy it but even the hello.cpp in the d 8000 emo folder will fail with the above pool_runner issue which seems like the hardware I am using isn't working correctly after compiling via the cpp.cli. The CPU is older and has SIMD only in limited support so I was only able to run WAMR worker and upload.
To evaluate this assumption after that I have tried only docker-compose on the master node running the composed version I use now locally but the same demo hello.cpp had the same pool_runner error.

Could this be the issue?

While I am working on my project, I also have a question. When invoking how do I pass a parameter to my executable? like the size of a matrix or the number of MPI processes I request? I cannot figure that out

krho999 · 2024-12-11T20:28:57Z

I have an additional question as I am also trying to compile an extended version of the stencil with both OMP and MPI but during compilation, I run into a problem:

1/1] Linking CXX executable mpi_my/a1_MPI_OMP.wasm
wasm-ld: error: unable to find library -lfaasmp
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

I used inv libfaasmp.build to reinstall the function and this CMake to include it based on the omp and mpi CMakes

if (CMAKE_SYSTEM_NAME STREQUAL "WASI")
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} --target=$ENV{FAASM_WASM_TRIPLE} -fopenmp")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} --target=$ENV{FAASM_WASM_TRIPLE} -fopenmp")
    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Xlinker --shared-memory")
    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Xlinker --features=atomics,bulk-memory,simd128,relaxed-simd,mutable-globals,sign-ext")
else ()
    find_package(OpenMP REQUIRED)
    include_directories(${OPENMP_INCLUDE_DIR} ${LLVM_INCLUDE_DIRS}/openmp)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
endif ()

function(mpi_omp_func exec_name dir_path)
    faasm_func(${exec_name} ${dir_path})

    target_link_libraries(${exec_name} faasmp faasmpi mpi)

    if (NOT CMAKE_SYSTEM_NAME STREQUAL "WASI")
        target_link_libraries(${exec_name} pthread)
    endif ()

    # Add to list of all funcs
    set(ALL_MPI_FUNCS ${ALL_MPI_FUNCS} ${exec_name} PARENT_SCOPE)
    set(ALL_OMP_FUNCS ${ALL_OMP_FUNCS} ${exec_name} PARENT_SCOPE)
endfunction(mpi_omp_func)

# Example usage of the combined function
mpi_omp_func(a1_MPI_OMP a1_MPI_OMP.cpp)

which is in custom mpi_omp folder and was added to the higher-level cmake add_subdirectory(mpi_omp)
Is there a particular place you define the paths as I cannot find any solution for this?

csegarragonz · 2024-12-12T09:16:27Z

Regarding your hardware compatibility: that is a very good point. The pool_runner stack trace is not very helpful, as this is what it prints any time it crashes. If you suspect it is a CPU feature issue, you should first run, as you are doing, on docker compose, as it is easy to re-build the binaries. Then try experimenting with different hardware features enabled/disabled. You need to set them in two places: (1) in the compilation toolchain (WASM compiler & linker flags, e.g. search for simd here, and re-build your WASM binary, (2) in the compilation from WASM to AOT machine code here.

If you are running in a test CLI environment, after re-building the codegen binary you need to run inv codegen your func and inv run.pool your func. If you are using a compose cluster, but still invoking functions to the worker, you should re-build the upload binary, inv dev.cc upload_server, from outside the CLI container re-start it faasmctl restart -s upload, and also flush the workers of any cached machine code: faasmctl flush.workers. Then you can re-upload and re-invoke the function.

To pass command line arguments, bot the inv run.pool local execution, as well as faasmctl invoke take a --cmdline "foo bar baz" argument that will pass "foo bar baz" to the binary's argc/argv.

Lastly, a limitation of Faasm is that we cannot run hybrid (MPI + OMP) programs. The main reason why it currently does not work, related to the error you are seeing, is that to use OMP we need to build the WASM toolchain and sysroot with the wasm32-wasi-threads target. This is because we need atomic WASM instructions. The MPI libraries target a different sysroot based on wasm32-wasi. That being said, we should be able to compile the libraries in MPI with the threads target too. No one has done it, though.

krho999 · 2024-12-12T16:23:43Z

Thanks for all the help, I will have a look into the MPI+OMP and try to run MPI on the wasm32-wasi-threads. Would you be able to give me some starting guidelines for rebuilding the MPI lib or would it be as simple as changing a variable? I am basically trying to see how this can be used in a current state and do some experiments on it as a part of my Master thesis and would be keen to try to make this work for my 5-point stencil so I can compare it to some less demanding HPW workload like MapReduce.

csegarragonz · 2024-12-13T09:33:41Z

Compiling libfaasmpi to wasm32-wasi-threads should be very easy. You just need to add a flag in ./tasks/libfaasmpi to take a threads boolean argument, that then you can pass to the generic build_faasm_lib.

You will see that changing the target is only changing the environment variables that we use in our sysroot. After compilation, you can check if the libraries are where you expect them to be by ls-ing: /usr/local/faasm/llvm-sysroot/wasm32-wasi-threads.

I expect more complications will arise at runtime, when you try to run the code.

krho999 · 2024-12-13T10:58:03Z

Aa okay I see, but this is the inv libfaasmpi.build right? That seems to be broken in the cpp.cli throwing

(cpp) root@a5980f405241:/code/cpp# inv  libfaasmpi.build
cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX=/usr/local/faasm/llvm-sysroot/lib/wasm32-wasi -DCMAKE_TOOLCHAIN_FILE=/usr/local/faasm/toolchain/tools/WasiToolchain.cmake /code/cpp/libfaasmpi
-- Faasm building STATIC libraries
-- Faasm building target 
System is unknown to cmake, create:
Platform/WASI to use this system, please post your config file on discourse.cmake.org so it can be added to cmake
Your CMakeCache.txt file was copied to CopyOfCMakeCache.txt. Please post that file on discourse.cmake.org.
CMake Error at CMakeLists.txt:31 (add_subdirectory):
  add_subdirectory given source
  "/code/cpp/libfaasmpi/../third-party/faabric/src/mpi" which is not an
  existing directory.


-- Libfaasmpi WebAssembly build
-- Configuring incomplete, errors occurred!

Which seems to be related to the way how the environment variable copied the fabric into itself, I manually put it into the third-party folder but that gives a different issue which I am trying to figure out

(cpp) root@a5980f405241:/code/cpp# inv  libfaasmpi.build
cmake -GNinja -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF -DCMAKE_INSTALL_PREFIX=/usr/local/faasm/llvm-sysroot/lib/wasm32-wasi -DCMAKE_TOOLCHAIN_FILE=/usr/local/faasm/toolchain/tools/WasiToolchain.cmake /code/cpp/libfaasmpi
-- Faasm building STATIC libraries
-- Faasm building target 
System is unknown to cmake, create:
Platform/WASI to use this system, please post your config file on discourse.cmake.org so it can be added to cmake
Your CMakeCache.txt file was copied to CopyOfCMakeCache.txt. Please post that file on discourse.cmake.org.
-- Faabric MPI native build
CMake Error at /code/cpp/third-party/faabric/src/mpi/CMakeLists.txt:41 (faabric_lib):
  Unknown CMake command "faabric_lib".

since the Cmake text files should be linked correctly but the actual function cannot be retrieved from the main cmake here, am I missing something in the setup why this is happening?

Thank you for all the responses, this thread is beneficial for understanding the FAASM better.

csegarragonz · 2024-12-17T10:28:57Z

Have you cloned faasm with recursive submodules? cpp/third-party/faabric is a submodule of cpp.

krho999 · 2024-12-26T13:37:41Z

Yes as in the read me

krho999 · 2025-01-04T14:56:16Z

Nwm i have fixed that one, one question though as I also wanted to try the faasm python scheduling and followed the quick steps you provided here unfortunately in GCP the kubernetes worker throws

[14:42:50] [69] [E] Error executing _start: Exception: failed to call unlinked import function (env, __faasm_get_py_user)
[14:42:50] [69] [E] Task 649141 threw exception. What: Error executing WASM function with WAMR
[00:25:09:412 - 7DFB7D7FF640]: warning: failed to link import function (env, __faasm_read_state_offset)

with a lot of those warnings. I am not sure if there is a step missing and I am looking in your documentation what could be causing this just had no luck so far. Any Idea why this would happen?

The plan is to try to train tensor CNN with the chain and then merge the models to get the final model out.

csegarragonz · 2025-01-06T18:25:06Z

Unfortunately, Python support is broken at this point :-(

That being said, I would expect a slightly different error than the one you are seeing...
I am guessing you have compiled and uploaded cpython beforehand?

krho999 · 2025-01-08T21:44:21Z

Yes ran the these commands as provided at the documentation page

faasmctl cli.python
inv cpython.func
inv cpython.upload
inv func.uploadpy hello
inv func.invoke python hello

unfortunately the above is what I get, it was just an attempt to see the difference in performance.

As I said, Python support is broken. The reason why I did not recognize the error is because, when it worked, it worked only with the WAVM WASM runtime. Now, the default WASM runtime is WAMR.

Python is not even supported in WAMR (i.e. not even broken), hence the errors you are seeing.

We are interested in resurrecting Python support for Faasm with WAMR, is this something you would be interested in contributing? I could provide some pointers.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue to execute function when on k8 #895

Issue to execute function when on k8 #895

Issue to execute function when on k8 #895

Issue to execute function when on k8 #895

Comments