8000 gcc 6 issue · Issue #343 · gunrock/gunrock · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

gcc 6 issue #343

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mahmoodn opened this issue Jun 24, 2018 · 31 comments
Closed

gcc 6 issue #343

mahmoodn opened this issue Jun 24, 2018 · 31 comments

Comments

@mahmoodn
Copy link
mahmoodn commented Jun 24, 2018

Is there any way to workaround gcc-6 without downgrading the whole gcc version? When I run make, I get the following error

mahmood@orca:build$ make
[  1%] Building NVCC (Device) object gunrock/CMakeFiles/gunrock.dir/__/externals/moderngpu/src/gunrock_generated_mgpucontext.cu.o
In file included from /usr/local/cuda/include/host_config.h:50:0,
                 from /usr/local/cuda/include/cuda_runtime.h:78,
                 from <command-line>:0:
/usr/local/cuda/include/crt/host_config.h:121:2: error: #error -- unsupported GNU version! gcc versions later than 6 are not supported!
 #error -- unsupported GNU version! gcc versions later than 6 are not supported!
  ^~~~~
CMake Error at gunrock_generated_mgpucontext.cu.o.cmake:222 (message):
  Error generating
  /home/mahmood/gunrock/build/gunrock/CMakeFiles/gunrock.dir/__/externals/moderngpu/src/./gunrock_generated_mgpucontext.cu.o


gunrock/CMakeFiles/gunrock.dir/build.make:119: recipe for target 'gunrock/CMakeFiles/gunrock.dir/__/externals/moderngpu/src/gunrock_generated_mgpucontext.cu.o' failed
make[2]: *** [gunrock/CMakeFiles/gunrock.dir/__/externals/moderngpu/src/gunrock_generated_mgpucontext.cu.o] Error 1
CMakeFiles/Makefile2:117: recipe for target 'gunrock/CMakeFiles/gunrock.dir/all' failed
make[1]: *** [gunrock/CMakeFiles/gunrock.dir/all] Error 2
Makefile:94: recipe for target 'all' failed
make: *** [all] Error 2
mahmood@orca:build$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/6/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 6.4.0-8ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 6.4.0 20171010 (Ubuntu 6.4.0-8ubuntu1) 

The cuda is 9.1 and ubuntu is 17.10. Any idea about that?

@jowens
Copy link
Contributor
jowens commented Jun 24, 2018

This is a CUDA issue, not a Gunrock issue. CUDA only supports certain combinations of OS+compiler.

@jowens jowens closed this as completed Jun 24, 2018
@mahmoodn
Copy link
Author

But I have installed toolkit 9.1 with that version without any problem! The document is described at medium.
The default gcc in ubuntu 17.10 is 7. However, cuda need gcc-6 and I did installed cuda with gcc-6. I have compiled other programs, e.g. lammps and etc without similar errors.

@jowens
Copy link
Contributor
jowens commented Jun 24, 2018

We have repeatedly seen that issues with building and running Gunrock on compiler/OS combinations not on this list (https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html) are actually issues with the compiler/OS combination. I'm glad other stuff works for you. Gunrock uses a fair amount of C++11 features and CUDA 9 features and perhaps we stress the compiler/OS combination more than other packages.

@neoblizz
Copy link
Member

And to add to that, you don't have to downgrade gcc, just use update-alternatives and add a gcc-6 installation in there along with others. That will make all of your CUDA 9 features work, and you can switch to the newest gcc-7 for all your other needs.

@mahmoodn
Copy link
Author

@neoblizz : Yes I did that.
@jowens : I will ask at the devtalk and come back later.

@mahmoodn
Copy link
Author
mahmoodn commented Jun 25, 2018

OK I ran the following commands to explicitly set CC and CXX to gcc-6 and g++-6, respectively. However, I now get a new error message.

mahmood@orca:build$ export CC=/usr/bin/gcc-6 
mahmood@orca:build$ export CXX=/usr/bin/g++-6 
mahmood@orca:build$ export BOOST_ROOT=/home/mahmood/gunrock/boost_1_64_0
mahmood@orca:build$ cmake ..
-- The C compiler identification is GNU 6.4.0
-- The CXX compiler identification is GNU 6.4.0
-- Check for working C compiler: /usr/bin/gcc-6
-- Check for working C compiler: /usr/bin/gcc-6 -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/g++-6
-- Check for working CXX compiler: /usr/bin/g++-6 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found CUDA: /usr/local/cuda-9.1 (found suitable version "9.1", minimum required is "7.5") 
-- Could NOT find Boost
-- Boost version: 1.64.0
-- Found the following Boost libraries:
--   system
--   filesystem
--   timer
--   chrono
CMake Warning (dev) at /usr/share/cmake-3.9/Modules/FindOpenMP.cmake:200 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "c" will no longer be dereferenced when the policy is
  set to NEW.  Since the policy is not set the OLD behavior will be used.
Call Stack (most recent call first):
  /usr/share/cmake-3.9/Modules/FindOpenMP.cmake:324 (_OPENMP_GET_FLAGS)
  cmake/FindOpenMP.cmake:70 (FIND_PACKAGE)
  CMakeLists.txt:49 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP
CMake Warning at cmake/FindMetis.cmake:47 (MESSAGE):
  Metis was requested but support was not found, run `sudo apt-get install
  metis` or `/dep/install_metis.sh` for installation.
Call Stack (most recent call first):
  CMakeLists.txt:50 (include)


-- Project Added: bc
-- Project Added: bfs
-- Project Added: cc
-- Project Added: pr
-- Project Added: sssp
-- Project Added: hits
-- Project Added: salsa
-- Project Added: wtf
-- Project Added: topk
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mahmood/gunrock/build
mahmood@orca:build$ make
[  1%] Building NVCC (Device) object gunrock/CMakeFiles/gunrock.dir/__/externals/moderngpu/src/gunrock_generated_mgpucontext.cu.o
[  2%] Building NVCC (Device) object gunrock/CMakeFiles/gunrock.dir/app/bfs/gunrock_generated_bfs_app.cu.o
In file included from /usr/local/cuda-9.1/include/common_functions.h:50:0,
                 from /usr/local/cuda-9.1/include/cuda_runtime.h:115,
                 from <command-line>:0:
/usr/local/cuda-9.1/include/crt/common_functions.h:64:24: error: token ""__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."" is not valid in preprocessor expressions
 #define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."
                        ^
/usr/local/cuda-9.1/include/crt/common_functions.h:64:24: note: in definition of macro ‘__CUDACC_VER__’
 #define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/cuda-9.1/include/crt/common_functions.h:64:24: error: token ""__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."" is not valid in preprocessor expressions
 #define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."
                        ^
/usr/local/cuda-9.1/include/crt/common_functions.h:64:24: note: in definition of macro ‘__CUDACC_VER__’
 #define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CMake Error at gunrock_generated_bfs_app.cu.o.cmake:222 (message):
  Error generating
  /home/mahmood/gunrock/build/gunrock/CMakeFiles/gunrock.dir/app/bfs/./gunrock_generated_bfs_app.cu.o


gunrock/CMakeFiles/gunrock.dir/build.make:63: recipe for target 'gunrock/CMakeFiles/gunrock.dir/app/bfs/gunrock_generated_bfs_app.cu.o' failed
make[2]: *** [gunrock/CMakeFiles/gunrock.dir/app/bfs/gunrock_generated_bfs_app.cu.o] Error 1
CMakeFiles/Makefile2:117: recipe for target 'gunrock/CMakeFiles/gunrock.dir/all' failed
make[1]: *** [gunrock/CMakeFiles/gunrock.dir/all] Error 2
Makefile:94: recipe for target 'all' failed
make: *** [all] Error 2
mahmood@orca:build$ 

Does the error mean that gunrock doesn't support cuda 9.1?! That is odd because cmake said found suitable version "9.1", minimum required is "7.5"

@jowens
Copy link
Contributor
jowens commented Jun 25, 2018

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/i 8000 ndex.html says for Ubuntu 17.10, you need gcc 7.2.0.

@neoblizz
Copy link
Member

@mahmoodn the error that you have is still in CUDA because you don't have the right gcc support. You cannot mix and match the gcc and cuda versions as you like, please follow their wiki guide for the appropriate version you need and then use update-alternatives to set the gcc version instead of exporting it. Exporting doesn't always guarantee it (even if it shows up if you run gcc --v). We have no issues running gunrock on CUDA 9.1, with our Ubuntu 16.04 and gcc 5.4.0.

@mahmoodn
Copy link
Author

@jowens: That document is about cuda 9.2. However, as I said before, I have installed cuda 9.1 on Ubuntu 17.10 at the time that 9.2 had not been released. I haven't upgraded cuda yet.

So, at the time that I decided to installed cuda 9.1, it said that gcc7 is unsupported. So, I used gcc 6 and it went smoothly and I am actually able to build some other programs using gcc6, nvcc and other staffs.

If you say that installed cuda is incompatible with gcc, then how it was possible to built the toolkit?! If there is any mismatch between the gcc command and cuda, then all programs must fail. However, I have built some programs with makefiles and nvcc.

@neoblizz: I really don't want to use any version as I like... again let me summarize:
ubuntu 17.10
gcc-7 (default with ubuntu)
gcc-6 installed via apt and configured via update-alternative
cuda 9.1 which is built using gcc-6

Please see below that although I have specified the alternatives and I am able to build some cuda examples, the gunrock fails

mahmood@orca:~$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 10
update-alternatives: using /usr/bin/gcc-7 to provide /usr/bin/gcc (gcc) in auto mode
mahmood@orca:~$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-6 20
update-alternatives: using /usr/bin/gcc-6 to provide /usr/bin/gcc (gcc) in auto mode
mahmood@orca:~$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 10
update-alternatives: using /usr/bin/g++-7 to provide /usr/bin/g++ (g++) in auto mode
mahmood@orca:~$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-6 20
update-alternatives: using /usr/bin/g++-6 to provide /usr/bin/g++ (g++) in auto mode
mahmood@orca:~$ sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30
update-alternatives: warning: forcing reinstallation of alternative /usr/bin/gcc because link group cc is broken
mahmood@orca:~$ sudo update-alternatives --set cc /usr/bin/gcc
mahmood@orca:~$ sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30
update-alternatives: warning: forcing reinstallation of alternative /usr/bin/g++ because link group c++ is broken
mahmood@orca:~$ sudo update-alternatives --set c++ /usr/bin/g++
mahmood@orca:~$ sudo update-alternatives --config gcc
There are 2 choices for the alternative gcc (providing /usr/bin/gcc).

  Selection    Path            Priority   Status
------------------------------------------------------------
* 0            /usr/bin/gcc-6   20        auto mode
  1            /usr/bin/gcc-6   20        manual mode
  2            /usr/bin/gcc-7   10        manual mode

Press <enter> to keep the current choice[*], or type selection number: 0
mahmood@orca:~$ sudo update-alternatives --config g++
There are 2 choices for the alternative g++ (providing /usr/bin/g++).

  Selection    Path            Priority   Status
------------------------------------------------------------
* 0            /usr/bin/g++-6   20        auto mode
  1            /usr/bin/g++-6   20        manual mode
  2            /usr/bin/g++-7   10        manual mode

Press <enter> to keep the current choice[*], or type selection number: 0
mahmood@orca:~$ cd gunrock/build/
mahmood@orca:build$ export BOOST_ROOT=/home/mahmood/gunrock/boost_1_64_0
mahmood@orca:build$ cmake ..
-- The C compiler identification is GNU 6.4.0
-- The CXX compiler identification is GNU 6.4.0
-- Check
8000
 for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE 
-- Found CUDA: /usr/local/cuda-9.1 (found suitable version "9.1", minimum required is "7.5")
-- Could NOT find Boost
-- Boost version: 1.64.0
-- Found the following Boost libraries:
--   system
--   filesystem
--   timer
--   chrono
CMake Warning (dev) at /usr/share/cmake-3.9/Modules/FindOpenMP.cmake:200 (if):
  Policy CMP0054 is not set: Only interpret if() arguments as variables or
  keywords when unquoted.  Run "cmake --help-policy CMP0054" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  Quoted variables like "c" will no longer be dereferenced when the policy is
  set to NEW.  Since the policy is not set the OLD behavior will be used.
Call Stack (most recent call first):
  /usr/share/cmake-3.9/Modules/FindOpenMP.cmake:324 (_OPENMP_GET_FLAGS)
  cmake/FindOpenMP.cmake:70 (FIND_PACKAGE)
  CMakeLists.txt:49 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP
CMake Warning at cmake/FindMetis.cmake:47 (MESSAGE):
  Metis was requested but support was not found, run `sudo apt-get install
  metis` or `/dep/install_metis.sh` for installation.
Call Stack (most recent call first):
  CMakeLists.txt:50 (include)


-- Project Added: bc
-- Project Added: bfs
-- Project Added: cc
-- Project Added: pr
-- Project Added: sssp
-- Project Added: hits
-- Project Added: salsa
-- Project Added: wtf
-- Project Added: topk
-- Configuring done
-- Generating done
-- Build files have been written to: /home/mahmood/gunrock/build
mahmood@orca:build$ make
[  1%] Building NVCC (Device) object gunrock/CMakeFiles/gunrock.dir/__/externals/moderngpu/src/gunrock_generated_mgpucontext.cu.o
[  2%] Building NVCC (Device) object gunrock/CMakeFiles/gunrock.dir/app/bfs/gunrock_generated_bfs_app.cu.o
In file included from /usr/local/cuda-9.1/include/common_functions.h:50:0,
                 from /usr/local/cuda-9.1/include/cuda_runtime.h:115,
                 from <command-line>:0:
/usr/local/cuda-9.1/include/crt/common_functions.h:64:24: error: token ""__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."" is not valid in preprocessor expressions
 #define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."
                        ^
/usr/local/cuda-9.1/include/crt/common_functions.h:64:24: note: in definition of macro ‘__CUDACC_VER__’
 #define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/cuda-9.1/include/crt/common_functions.h:64:24: error: token ""__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."" is not valid in preprocessor expressions
 #define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."
                        ^
/usr/local/cuda-9.1/include/crt/common_functions.h:64:24: note: in definition of macro ‘__CUDACC_VER__’
 #define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."
                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
CMake Error at gunrock_generated_bfs_app.cu.o.cmake:222 (message):
  Error generating
  /home/mahmood/gunrock/build/gunrock/CMakeFiles/gunrock.dir/app/bfs/./gunrock_generated_bfs_app.cu.o


gunrock/CMakeFiles/gunrock.dir/build.make:63: recipe for target 'gunrock/CMakeFiles/gunrock.dir/app/bfs/gunrock_generated_bfs_app.cu.o' failed
make[2]: *** [gunrock/CMakeFiles/gunrock.dir/app/bfs/gunrock_generated_bfs_app.cu.o] Error 1
CMakeFiles/Makefile2:117: recipe for target 'gunrock/CMakeFiles/gunrock.dir/all' failed
make[1]: *** [gunrock/CMakeFiles/gunrock.dir/all] Error 2
Makefile:94: recipe for target 'all' failed
make: *** [all] Error 2
mahmood@orca:build$ nvcc -v
nvcc fatal   : No input files specified; use option --help for more information
mahmood@orca:build$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Nov__3_21:07:56_CDT_2017
Cuda compilation tools, release 9.1, V9.1.85
mahmood@orca:build$ nvcc -ccbin g++
nvcc fatal   : No input files specified; use option --help for more information
mahmood@orca:build$ cd ~/samples/0_Simple/matrixMul
mahmood@orca:matrixMul$ make clean && make
rm -f matrixMul matrixMul.o
rm -rf ../../bin/x86_64/linux/release/matrixMul
"/usr/local/cuda-9.1"/bin/nvcc -ccbin g++ -I../../common/inc  -m64    -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_70,code=compute_70 -o matrixMul.o -c matrixMul.cu
"/usr/local/cuda-9.1"/bin/nvcc -ccbin g++   -m64      -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_37,code=sm_37 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_70,code=compute_70 -o matrixMul matrixMul.o
mkdir -p ../../bin/x86_64/linux/release
cp matrixMul ../../bin/x86_64/linux/release
mahmood@orca:matrixMul$


I think cmake wrongly set or forget somethings about the correct gcc version. Therefore, during the compilation it uses gcc-7. Though, this is a guess.

@neoblizz
Copy link
Member
neoblizz commented Jun 25, 2018

@mahmoodn cmake is using the right gcc-6, if you say it is supported. Please navigate to /usr/local/cuda/include/crt/common_functions.h and comment out line 64, this is still a cuda related issue and has showed up in tensorflow and other libraries as well. Recompile after that.

@neoblizz
Copy link
Member

EDIT: @mahmoodn before that, could you try using boost 1.58.0 instead?

@mahmoodn
Copy link
Author

@neoblizz
Without modifying cuda source, boost 1.58.0 passed that error. Later I got

[  5%] Building NVCC (Device) object gunrock/CMakeFiles/gunrock.dir/app/pr/gunrock_generated_pr_app.cu.o
/home/mahmood/gunrock/boost_1_58_0/boost/smart_ptr/detail/sp_counted_base_gcc_x86.hpp(75): warning: variable "tmp" was set but never used

/home/mahmood/gunrock/boost_1_58_0/boost/mpl/set/aux_/item.hpp(46): warning: "char (&operator||(const boost::mpl::s_item<T, Base> &, boost::mpl::aux::type_wrapper<T> *))[boost::mpl::next<Base::order>::type::value]" declares a non-template function -- add <> to refer to a template instance

/home/mahmood/gunrock/boost_1_58_0/boost/mpl/set/aux_/item.hpp(47): warning: "boost::mpl::aux::no_tag operator%(const boost::mpl::s_item<T, Base> &, boost::mpl::aux::type_wrapper<T> *)" declares a non-template function -- add <> to refer to a template instance

/home/mahmood/gunrock/boost_1_58_0/boost/mpl/set/aux_/item.hpp(61): warning: "boost::mpl::aux::yes_tag operator%(const boost::mpl::s_mask<T, Base> &, boost::mpl::aux::type_wrapper<T> *)" declares a non-template function -- add <> to refer to a template instance

/home/mahmood/gunrock/boost_1_58_0/boost/mpl/set/aux_/item.hpp(75): warning: "boost::mpl::aux::no_tag operator%(const boost::mpl::s_unmask<T, Base> &, boost::mpl::aux::type_wrapper<T> *)" declares a non-template function -- add <> to refer to a template instance

Error: Internal Compiler Error (codegen): "there was an error in verifying the lgenfe output!"

CMake Error at gunrock_generated_pr_app.cu.o.cmake:282 (message):
  Error generating file
  /home/mahmood/gunrock/build/gunrock/CMakeFiles/gunrock.dir/app/pr/./gunrock_generated_pr_app.cu.o


gunrock/CMakeFiles/gunrock.dir/build.make:6661: recipe for target 'gunrock/CMakeFiles/gunrock.dir/app/pr/gunrock_generated_pr_app.cu.o' failed
make[2]: *** [gunrock/CMakeFiles/gunrock.dir/app/pr/gunrock_generated_pr_app.cu.o] Error 1
CMakeFiles/Makefile2:117: recipe for target 'gunrock/CMakeFiles/gunrock.dir/all' failed
make[1]: *** [gunrock/CMakeFiles/gunrock.dir/all] Error 2
Makefile:94: recipe for target 'all' failed
make: *** [all] Error 2

Although internal compiler errors are related to gcc, I would like to test another version of boost, e.g. 1.60.0
!

@neoblizz
Copy link
Member

@mahmoodn more on this: h2oai/h2o4gpu#607

@neoblizz
Copy link
Member

From the looks of it, they have taken care of a lot of issues within 9.1 in CUDA 9.2, and that makes sense because subversions usually means that there were bugs significant enough (like the one you see), to release a new version.

@neoblizz
Copy link
Member

We have also seen this same issue, here: #334

@mahmoodn
Copy link
Author

Yes I removed -std=c++11 option.
Thanks a lot. I can now see the binary files.

@neoblizz
Copy link
Member

@mahmoodn have you tried running ctest on it?

@mahmoodn
Copy link
Author

Yes it was fine. I also went to the dataset folder and made hollywood. I am trying to run one of the datasets. The document for simple example (here), seems to be a dev document. Is that the first step to understand gunrock?

@neoblizz
Copy link
Member

@mahmoodn great, running a dataset is fairly simple. For example: path/to/bin/bfs market /path/to/dataset, it may also be worth watching the presentation we gave it GTC that explains the underlying programming model and the new api under dev-refactor branch: http://on-demand.gputechconf.com/gtc/2018/video/S8586/

@mahmoodn
Copy link
Author
mahmoodn commented Jun 26, 2018

Consider the following run


mahmood@orca:hollywood-2009$ ~/gunrock/build/bin/bfs market hollywood-2009.mtx 
Loading Matrix-market coordinate-formatted graph ...
Reading from hollywood-2009.mtx:
  Parsing MARKET COO format (1139905 nodes, 115031232 directed edges)... Done parsing (17s).
  Converting 1139905 vertices, 115031232 directed edges (unordered tuples) to CSR format...
Done converting (7s).

Degree Histogram (1139905 vertices, 112751422 edges):
    Degree   0: 32662 (2.87%)
    Degree 2^0: 24445 (2.14%)
    Degree 2^1: 50032 (4.39%)
    Degree 2^2: 99398 (8.72%)
    Degree 2^3: 174673 (15.32%)
    Degree 2^4: 231414 (20.30%)
    Degree 2^5: 219393 (19.25%)
    Degree 2^6: 139586 (12.25%)
    Degree 2^7: 78822 (6.91%)
    Degree 2^8: 44379 (3.89%)
    Degree 2^9: 25905 (2.27%)
    Degree 2^10: 13729 (1.20%)
    Degree 2^11: 5233 (0.46%)
    Degree 2^12: 230 (0.02%)
    Degree 2^13: 4 (0.00%)

  Converting 1139905 vertices, 112751422 directed edges (unordered tuples) to CSR format...
Done converting (7s).
Using 1 GPU: [ 0 ].
Using traversal-mode LB
__________________________
0	 4	 0	 queue3 	 oversize :	 7409384 ->	 52557065
0	 5	 0	 queue3 	 oversize :	 52557065 ->	 56516755
--------------------------
iteration 0 elapsed: 130.076170 ms, src = 0, #iteration = 11
Computing reference value ...
CPU BFS finished in 472.594971 msec. cpu_search_depth: 11

Label Validity: 
CORRECT

First 40 labels of the GPU result:
[0:0 1:1 2:1 3:1 4:5 5:5 6:5 7:5 8:6 9:6 10:6 11:6 12:6 13:6 14:6 15:6 16:6 17:6 18:6 19:6 20:6 21:6 22:6 23:6 24:6 25:6 26:5 27:6 28:6 29:6 30:6 31:6 32:6 33:6 34:6 35:6 36:6 37:6 38:6 39:6 ]

	Memory Usage(B)	 #keys0,0	 #keys0,1
GPU_0	 1223950336	 7409384	 56516755
	 queue_sizing =	 6.500001 	 49.580233

 [BFS] finished.
 avg. elapsed: 130.0762 ms
 iterations: 11
 min. elapsed: 130.0762 ms
 max. elapsed: 130.0762 ms
 rate: 865.7489 MiEdges/s
 src: 0
 nodes_visited: 1069126
 edges_visited: 112613306
 nodes queued: 1069126
 edges queued: 112613306
 load time: 48844.0242 ms
 preprocess time: 98.7210 ms
 postprocess time: 46.8349 ms
 total time: 49601.1329 ms

I want to know
1- which time is the measurement of GPU?
2- When I see the procss offloads to the GPU (using nvidia-smi), the GPU load isn't 100%. It remains 0%.
3- How can I control the number of cpu threads when the program is running on GPU?

@neoblizz
Copy link
Member

The average elapsed time, min elapsed and max elapsed are GPU measurements. Preprocessing and post processing and before and after the computation as the name suggest.

Nvidia-smi maybe isn't the best way to check GPU utilization when the process is only 130ms long. Are you running nvidia-smi in a different window? What GPU are you using?

I don't quite understand why you'd need that control, because after the work is set by the CPU, it doesn't do much. We don't have an api allowing such control, and there didn't seem any use case. What are you trying to do there?

@mahmoodn
Copy link
Author

When the task is offloaded to gpu, what does min and max elapsed mean? Is that related to more than one iteration for running the program?

For the threads, I just want to measure some gpu features for research. So, I would like to know what happens when two or eight bfs threads compete on the gpu resource? Just to measure some architectural features. Currently at some initial steps are cores reach 100% then one one bfs threads is seen.

@neoblizz
Copy link
Member

Min and max elapsed is due to the multiple runs, yes, more than one iterations averaged out for the avg elapsed.

I believe you are able to do so, we don't have an api for that as far as I know. @sgpyc might be able to comment on this.

Also worth looking is the dev-refactor branch, which has the refactored code and cleaner api. If you can elaborate further on what your research is looking for. We may be able to suggest some things.

@Michael2397
Copy link

The problem is gcc version.
CUDA 9.1, with our Ubuntu 16.04 and gcc 5.4.0. I am is Ok.

following this
mahmood@orca:$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-7 10
update-alternatives: using /usr/bin/gcc-7 to provide /usr/bin/gcc (gcc) in auto mode
mahmood@orca:
$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-6 20
update-alternatives: using /usr/bin/gcc-6 to provide /usr/bin/gcc (gcc) in auto mode
mahmood@orca:$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-7 10
update-alternatives: using /usr/bin/g++-7 to provide /usr/bin/g++ (g++) in auto mode
mahmood@orca:
$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-6 20
update-alternatives: using /usr/bin/g++-6 to provide /usr/bin/g++ (g++) in auto mode
mahmood@orca:$ sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc 30
update-alternatives: warning: forcing reinstallation of alternative /usr/bin/gcc because link group cc is broken
mahmood@orca:
$ sudo update-alternatives --set cc /usr/bin/gcc
mahmood@orca:$ sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++ 30
update-alternatives: warning: forcing reinstallation of alternative /usr/bin/g++ because link group c++ is broken
mahmood@orca:
$ sudo update-alternatives --set c++ /usr/bin/g++
mahmood@orca:~$ sudo update-alternatives --config gcc
There are 2 choices for the alternative gcc (providing /usr/bin/gcc).

Selection Path Priority Status

  • 0 /usr/bin/gcc-6 20 auto mode
    1 /usr/bin/gcc-6 20 manual mode
    2 /usr/bin/gcc-7 10 manual mode

Press to keep the current choice[*], or type selection number: 0
mahmood@orca:~$ sudo update-alternatives --config g++
There are 2 choices for the alternative g++ (providing /usr/bin/g++).

Selection Path Priority Status

  • 0 /usr/bin/g++-6 20 auto mode
    1 /usr/bin/g++-6 20 manual mode
    2 /usr/bin/g++-7 10 manual mode

@neoblizz
Copy link
Member
neoblizz commented Dec 6, 2018

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#system-requirements

@archenroot
Copy link

@neoblizz - we are ready to go with cuda10 against gunrock?

@neoblizz
Copy link
Member

@archenroot yes, I have tested gunrock on CUDA 10 and latest NVIDIA drivers, work perfectly.

@archenroot
Copy link
archenroot commented Dec 17, 2018

@mahmoodn - just for your info, I am testing Gunrock Java interface (to finally merge it into JavaCPP presets) and as Gentoo user I already migrated to GCC 7. By facing similar issues as you did. I finally found following setup which compiles gunrock. So my env is:

  • dev-util/nvidia-cuda-sdk-9.2.88
  • dev-util/nvidia-cuda-toolkit-9.2.88
  • sys-devel/gcc-7.3.0-r4 (r4 might be internal Gentoo subversion, not sure on this)
  • dev-libs/boost-1.65.1_pre (I had some issues with Gentoo stable 1.63 and with 1.65 as well)

Still after this I had some issues in boost:
more than one operator "==" matches these operands

I found these discussed at Torch repos: torch/cutorch#797

So I did not nice, but dirty change:
sedinplace 's/CUDA_NVCC_FLAGS -std=c++11/CUDA_NVCC_FLAGS -std=c++11 -D__CUDA_NO_HALF_OPERATORS__/g' $(find ./ -iname CMakeLists.txt)

note sedinplace is a bash function defined in JavaCPP presets to help custimizing the code to be wrapped by Java access layer (JNI in this case):

function sedinplace {
    if ! sed --version 2>&1 | grep -i gnu > /dev/null; then
        sed -i '' "$@"
    else
        sed -i "$@"
    fi
}

I just add another custom NVCC_FLAG into existing defintiion: -D__CUDA_NO_HALF_OPERATORS__ which fixed/removed the error reported from boost.

So in this setup (although not supported by Nvidia) works, I am just not sure why I didn't faced your issue with version as soon as I have in my crt/host_config.h still these lines:

#if __GNUC__ > 7

#error -- unsupported GNU version! gcc versions later than 7 are not supported!

#endif /* __GNUC__ > 7 */

Additionally I had to disable VER check in CUDA:

zangetsu@venus: pts/10: 142 files 72Mb -> grep /opt/cuda/include/crt/common_functions.h -e "CUDACC_VER"
*#define __CUDACC_VER__ "__CUDACC_VER__ is no longer supported.  Use __CUDACC_VER_MAJOR__, __CUDACC_VER_MINOR__, and __CUDACC_VER_BUILD__ instead."

I think this latest one is something with Gunrock build system, but devs are stable, so it is again something to do with my setup. Still I made it working. And I am also going to do now the merge request regarding Java API for Gunrock :-)

CC6F

@jowens
Copy link
Contributor
jowens commented Dec 17, 2018

__CUDACC_VER__ appears to be used in one of the submodules (CUB), but not in Gunrock itself.

@archenroot
Copy link
archenroot commented Dec 17, 2018

@jowens - was receiving in my env exactly this:
BVLC/caffe#5994

Not sure what I did, but don't receive it anymore, so don't need to comment out the line in *.h file...not sure what caused this originally (I did yesterday multiple changes to env)...

@neoblizz
Copy link
Member

If the problem persist, you can hack these defines as well, by just defining them yourself. Or passing it during compile -D__CUDACC_VER_=<version>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants
0