[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pthreads #3266

Merged
merged 278 commits into from
Jun 3, 2015
Merged

Pthreads #3266

merged 278 commits into from
Jun 3, 2015

Conversation

juj
Copy link
Collaborator
@juj juj commented Mar 17, 2015

This set of commits implements initial pthreads library support for Emscripten code, and allows native code to utilize multithreading with the pthreads API. This backs on experimental research on the JavaScript SharedArrayBuffer and Atomics APIs from https://docs.google.com/document/d/1NDGA_gZJ7M7w1Bh8S0AoDyEqwDdRh4uSoTPSNn77PFk/edit?usp=sharing , and as such is preliminary and unsupported in other browsers except current Firefox Nightly builds. By default pthreads support is disabled, and the feature must be explicitly enabled by specifying the compiler flag -s USE_PTHREADS=1 and the linker flag -lpthread.

This is for experimental use only at the moment, since none of the threading specification has yet been standardized, and it may change a lot for the final proposal, even to the point of making large parts of this pull request obsolete.

This PR contains some unit tests, but more is available in https://github.com/juj/posixtestsuite (especially see emscripten-core/posixtestsuite@ed4bdd6 ), which contains ~300 unit tests for POSIX conformance for pthreads.

There is an accompanying pull request to the LLVM backend at emscripten-core/emscripten-fastcomp#67 .

@@ -703,6 +707,7 @@ try:
pre_js += open(shared.path_from_root('src', 'emrun_prejs.js')).read() + '\n'
post_js += open(shared.path_from_root('src', 'emrun_postjs.js')).read() + '\n'


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

newline here seems unnecessary

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ops. Removed.

@kripken
Copy link
Member
kripken commented Mar 23, 2015

Very impressive work!

Bunch of questions above. I am still getting into the code, the answers will help me.

@gunyarakun
Copy link
Contributor

I'm interested in running threads with pthread API in emscripten.

So I tried to build and run a sample pthread program and run posixtestsuite, but I failed to execute.

Could you point what's wrong I did?

I was using the browser FirefoxNightly 39.0a1 (2015-03-23) and emscripten/fastcomp on pthreads branch.

testing with simple pthread program

I fetched a pthread sample source code from http://en.wikipedia.org/wiki/POSIX_Threads

I built html/js with emcc and run it with emrun.

emcc wikipedia.c -lpthread -o wikipedia.html -s TOTAL_MEMORY=268435456 -s PTHREAD_POOL_SIZE=40 -s USE_PTHREADS=1 --emrun 
emrun --browser firefox_nightly wikipedia.html

The error below shows up continuously.

still waiting on run dependencies:
dependency: pthreads
(end of list)

Here is the source code.

#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <assert.h>

#define NUM_THREADS     5

void *perform_work(void *argument)
{
   int passed_in_value;

   passed_in_value = *((int *) argument);
   printf("Hello World! It's me, thread with argument %d!\n", passed_in_value);

   /* optionally: insert more useful stuff here */

   return NULL;
}

int main(void)
{
   pthread_t threads[NUM_THREADS];
   int thread_args[NUM_THREADS];
   int result_code, index;

   // create all threads one by one
   for (index = 0; index < NUM_THREADS; ++index) {
      thread_args[index] = index;
      printf("In main: creating thread %d\n", index);
      result_code = pthread_create(&threads[index], NULL, perform_work, (void *) &thread_args[index]);
      assert(0 == result_code);
   }

   // wait for each thread to complete
   for (index = 0; index < NUM_THREADS; ++index) {
      // block until thread 'index' completes
      result_code = pthread_join(threads[index], NULL);
      printf("In main: thread %d has completed\n", index);
      assert(0 == result_code);
   }

   printf("In main: All threads completed successfully\n");
   exit(EXIT_SUCCESS);
}

testing with posixtestsuite

I challenged to run tests in posixtestsuite (https://github.com/juj/posixtestsuite)

  • Modify emcc/run_emcc_tests.py emrun option '--browser' to use firefox_nightly
cmd = ['emrun', '--kill_start', '--kill_exit', '--timeout=15', '--silence_timeout=15', '--browser=firefox_nightly', out]
  • Install emcc/run_all_tests.py and emcc/run_emcc_tests.py into a directory in $PATH (/usr/local/bin)
  • Execute tests in conformance directory.
cd conformance/interfaces
run_all_tests.py

The test started well and launched Firefox, but the results are fail like following.

=== FINISHED ===
3 tests failed:
1-1.c, 2-1.c, 3-1.c

@juj
Copy link
Collaborator Author
juj commented Mar 24, 2015

@gunyarakun: There is currently an undocumented issue that you will need to work around: in Firefox there is a limit to how many web workers you can spawn. This is bug https://bugzilla.mozilla.org/show_bug.cgi?id=1052398 . To work around this, navigate to about:config and find the pref dom.workers.maxPerDomain and set it to a big enough value, e.g. 50. The value should be larger than what you pass in to -s PTHREAD_POOL_SIZE=x, or otherwise you will see that the run dependencies will never finish. Setting the value to at least 50 should allow running the open posix test suite as well, since some of those tests use quite large number of simultaneous threads.

@juj
Copy link
Collaborator Author
juj commented Mar 24, 2015

@kripken: Thanks for the reviews, I think I should have addressed all of those in the new batch of commits.

@gunyarakun
Copy link
Contributor

@juj: thank you for your advice. I changed dom.workers.maxPerDomain and toolkit.startup.max_resumed_crashes (for not showing the dialog to be safe mode) and posixtestsuite seemed working well.

But the sample program following doesn't work well. Is my program wrong or is it a known problem?

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <pthread.h>

static void *thread_func(void *vptr_args) {
    for (int i = 0; i < 20; i++) {
        puts("  b");
        sleep(1);
    }

    return NULL;
}

int main(void) {
    pthread_t thread;

    if (pthread_create(&thread, NULL, thread_func, NULL) != 0) {
        return EXIT_FAILURE;
    }

    for (int i = 0; i < 20; i++) {
        puts("a");
        sleep(1);
    }

    if (pthread_join(thread, NULL) != 0) {
        return EXIT_FAILURE;
    }

    return EXIT_SUCCESS;
}

juj added 25 commits June 1, 2015 15:11
… to call to sbrk()), while main thread is also doing memory management, would cause a deadlock where pthread and main thread would wait on each others' mutexes. Add a test.
… calls again and again if the call flow leads to multiple nested operations.
…thread_mutex to pass the test params to compilation.
…und, do it as u32 version with a CAS loop. TODO: revert this commit once https://bugzilla.mozilla.org/show_bug.cgi?id=1141986 lands.
…ently so that JS code that interacts with Emscripten HEAP when it is a SAB can operate normally.
…mber of detected logical cores on the system and emscripten_force_num_logical_cores() which allows overriding the value returned by this function. Add a new command line variable -s PTHREAD_HINT_NUM_CORES=x to allow choosing the default value when navigator.hardwareConcurrency is not supported.
…ser. Add new option -s IN_TEST_HARNESS=1 and have the test harness skip the pthread tests if run on a browser that doesn't support them.
…mall delay to report test being skipped before the test window is closed so that the XHR request can reach the test harness.
@juj
Copy link
Collaborator Author
juj commented Jun 1, 2015

The other pull request should be now in better shape. Rebased this on top of the latest incoming and reran the tests on my Windows box, which look ok.

juj added a commit that referenced this pull request Jun 3, 2015
@juj juj merged commit 6dc579b into emscripten-core:incoming Jun 3, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants