Okay, an update on this.
1) This is fixed now, thank you. We just needed to add a custom target, as you
said.
2) This is trickier and, unfortunately, still not working. We are already
adding -fPIC to CMAKE_CXX_FLAGS, should that not be enough? I also tried adding
it to both CMAKE_CXX_FLAGS_DEBUG and CMAKE_CXX_FLAGS_RELEASE, with no effect.
Looking into FindCUDA.CMake at CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS, I find
code very similar to what you are describing. Are you saying that in addition
to what is below, we need to add what you proposed? This is what I see.
--
foreach(config ${CUDA_configuration_types})
string(TOUPPER ${config} config_upper)
# Add config specific flags
foreach(f ${CUDA_NVCC_FLAGS_${config_upper}})
list(APPEND config_specific_flags $<$<CONFIG:${config}>:${f}>)
endforeach()
set(important_host_flags)
_cuda_get_important_host_flags(important_host_flags
${CMAKE_${CUDA_C_OR_CXX}_FLAGS_${config_upper}})
foreach(f ${important_host_flags})
list(APPEND flags $<$<CONFIG:${config}>:-Xcompiler> $<$<CONFIG:${config}>:${f}>)
endforeach()
endforeach()
James Bigler wrote:
2. It looks as though CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS is only
looking at the configuration specific flags. You can add the flag
specifically to all your configs (e.g. CMAKE_CXX_FLAGS_DEBUG) or you
could try adding these lines of code in your FindCUDA.cmake file
somewhere in CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS after set(flags).
It looks as though this was overlooked (sorry).
set(important_host_flags)
_cuda_get_important_host_flags(important_host_flags
${CMAKE_${CUDA_C_OR_CXX}_FLAGS})
foreach(f ${important_host_flags})
list(APPEND flags -Xcompiler ${f})
endforeach()
On Mon, Jan 5, 2015 at 3:43 PM, Irwin Zaid
<irwin.z...@physics.ox.ac.uk <mailto:irwin.z...@physics.ox.ac.uk>> wrote:
Alright, this is a lot of progress!
1) We are using Makefiles. I agree with you about the dependency
graph, so I'll try and sort that out. I'll let you know what the
result is.
2) I just checked and, indeed, the *_intermediate_link.o file is
not being passed -fPIC. Is this our problem? What is the correct fix?
Irwin
James Bigler wrote:
On Mon, Jan 5, 2015 at 1:57 PM, Irwin Zaid
<irwin.z...@physics.ox.ac.uk <mailto:irwin.z...@physics.ox.ac.uk>
<mailto:irwin.z...@physics.ox.__ac.uk
<mailto:irwin.z...@physics.ox.ac.uk>>> wrote:
Hi James,
Thanks for the quick reply! As I mentioned, we've hit two issues.
The first is the project dependencies one, which I'll try and
describe more a bit below. I'm not a CMake expert, so please bear
with me.
The second is what I've put under 2).
The only CMake build dependency changes when doing separable
compilation
are found in CUDA_LINK_SEPARABLE_____COMPILATION_OBJECTS.
Basically what
this does is create a new rule to build an intermediate link
file. For
everything but some versions of MSVC generators it adds a custom
command
to generate this intermediate link file. The other case adds a
custom
command that runs as a PRE_LINK command to generate the object
file (the
reasons for this is a bug in VS custom command dependency
scanning), but
this should happen during link phase and not compile phase.
Nothing in here should change what happens before the target is
built,
though.
1) Okay, I understand that, but I do think we saw a different
behaviour when we switched to separable compilation. Let me
describe
what we are doing.
We generate part of our library from a simple program (call the
simple program 'gen', which generates a source file 'gen.hpp')
that
we want to execute before compiling our library (call our library
'main'). We set this up with the following:
- add_executable(...) is called for 'gen'
- add_custom_command(...) sets up a command that executes 'gen'
- set_source_files_properties(..____.) is called to set
'gen.hpp' as
having the PROPERTY of GENERATED
- add_dependencies(main gen) is called to establish 'main' depends
on 'gen'
So far, this has only failed for CUDA with separable compilation.
(It has worked for all of our other configurations. including CUDA
without separable compilation.)
Have we done something wrong? Is there some additional information
we can look at to figure out what's going on?
What kind of generator are you using (e.g. makefile)?
Here's what I'm thinking might be the problem, though I'm not
sure why
it would have worked without CUDA_SEPARABLE_COMPILATION.
There's a dependency between gen and gen.hpp (encoded in the
call to
add_custom_command(OUTPUTS gen.hpp COMMAND gen)).
There's a dependency between main and gen (can't start
building main
until gen is built).
There's a dependency between gen.hpp and main (gen.hpp is an input
source to main, so it needs to be built as part of main).
What I don't see is a dependency between gen.hpp and all the cuda
sources that might use it as input. So from a dependency graph
standpoint a makefile (if one is being used in this case) is
entirely
free to start compiling the CUDA code once the gen target has been
satisfied.
What you need is another target that builds the gen.hpp file
which can
be forced to run before main starts to build. There are more
than one
way to do this, and I'm not sure what the best option is, but
you might
try this:
add_custom_command(OUTPUTS gen.hpp ....)
add_custom_target(make_gen_hpp DEPENDS gen.hpp)
add_dependency(main make_gen_hpp)
2) A second problem we've run into is an error when trying to
link a
CUDA shared library with separable compilation. This is
specifically
a Linux problem, on Mac it is fine. A static library is also fine,
working for both Linux and Mac.
The particular error is "relocation R_X86_64_32S against
`__nv_module_id' can not be used when making a shared object;
recompile with -fPIC". However, we are already compiling with
-fPIC.
I can confirm that -fPIC appears to be passed to both the host
compiler for non-CUDA source and via -Xcompiler -fPIC for CUDA
source.
This error occurs when trying to link all the different object
files
together of our library.
Do you have any idea of what this could be?
I'm not sure which object file wasn't compiled with -fPIC, but
I would
suspect it might be the intermediate link object file. FindCUDA is
supposed to pass this flag along (see
function(_cuda_get_important___host_flags)), but you might
want to verify
this with a 'make VERBOSE=1' and look for
<target_name>_intermediate___link.o (substitute your target
name in or
leave it out of the search string).
James
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
