This test failed on Windows because clang is called "clang.exe" not "clang". Fixed in r279772.
On Wed, Aug 24, 2016 at 8:21 AM, Samuel Antao via cfe-commits < cfe-commits@lists.llvm.org> wrote: > Author: sfantao > Date: Wed Aug 24 10:21:05 2016 > New Revision: 279632 > > URL: http://llvm.org/viewvc/llvm-project?rev=279632&view=rev > Log: > clang-offload-bundler - offload files bundling/unbundling tool > > Summary: > One of the goals of programming models that support offloading (e.g. > OpenMP) is to enable users to offload with little effort, by annotating the > code with a few pragmas. I'd also like to save users the trouble of > changing their existent applications' build system. So having the compiler > always return a single file instead of one for the host and each target > even if the user is doing separate compilation is desirable. > > This diff proposes a tool named clang-offload-bundler (happy to change the > name if required) that is used to bundle files associated with the same > user source file but different targets, or to unbundle a file into separate > files associated with different targets. > > This tool supports the driver support for OpenMP under review in > http://reviews.llvm.org/D9888. The tool is used there to enable separate > compilation, so that the very first action on input files that are not > source files is a "unbundling action" and the very last non-linking action > is a "bundling action". > > The format of the bundled files is currently very simple: text formats are > concatenated with comments that have a magic string and target identifying > triple in between, and binary formats have a header that contains the > triple and the offset and size of the code for host and each target. > > The goal is to improve this tool in the future to deal with archive files > so that each individual file in the archive is properly dealt with. We see > that archives are very commonly used in current applications to combine > separate compilation results. So I'm convinced users would enjoy this > feature. > > This tool can be used like this: > > `clang-offload-bundler -targets=triple1,triple2 -type=ii > -inputs=a.triple1.ii,a.triple2.ii -outputs=a.ii` > > or > > `clang-offload-bundler -targets=triple1,triple2 -type=ii > -outputs=a.triple1.ii,a.triple2.ii -inputs=a.ii -unbundle` > > I implemented the tool under clang/tools. Please let me know if something > like this should live somewhere else. > > This patch is prerequisite for http://reviews.llvm.org/D9888. > > Reviewers: hfinkel, rsmith, echristo, chandlerc, tra, jlebar, ABataev, > Hahnfeld > > Subscribers: whchung, caomhin, andreybokhanko, arpith-jacob, > carlo.bertolli, mehdi_amini, guansong, Hahnfeld, cfe-commits > > Differential Revision: https://reviews.llvm.org/D13909 > > Added: > cfe/trunk/test/Driver/clang-offload-bundler.c > cfe/trunk/tools/clang-offload-bundler/ > cfe/trunk/tools/clang-offload-bundler/CMakeLists.txt > cfe/trunk/tools/clang-offload-bundler/ClangOffloadBundler.cpp > Modified: > cfe/trunk/test/CMakeLists.txt > cfe/trunk/tools/CMakeLists.txt > > Modified: cfe/trunk/test/CMakeLists.txt > URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/ > CMakeLists.txt?rev=279632&r1=279631&r2=279632&view=diff > ============================================================ > ================== > --- cfe/trunk/test/CMakeLists.txt (original) > +++ cfe/trunk/test/CMakeLists.txt Wed Aug 24 10:21:05 2016 > @@ -29,6 +29,7 @@ list(APPEND CLANG_TEST_DEPS > clang-format > c-index-test diagtool > clang-tblgen > + clang-offload-bundler > ) > > if(CLANG_ENABLE_STATIC_ANALYZER) > > Added: cfe/trunk/test/Driver/clang-offload-bundler.c > URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/ > clang-offload-bundler.c?rev=279632&view=auto > ============================================================ > ================== > --- cfe/trunk/test/Driver/clang-offload-bundler.c (added) > +++ cfe/trunk/test/Driver/clang-offload-bundler.c Wed Aug 24 10:21:05 2016 > @@ -0,0 +1,222 @@ > +// > +// Generate all the types of files we can bundle. > +// > +// RUN: %clang -O0 -target powerpc64le-ibm-linux-gnu %s -E -o %t.i > +// RUN: %clangxx -O0 -target powerpc64le-ibm-linux-gnu -x c++ %s -E -o > %t.ii > +// RUN: %clang -O0 -target powerpc64le-ibm-linux-gnu %s -S -emit-llvm -o > %t.ll > +// RUN: %clang -O0 -target powerpc64le-ibm-linux-gnu %s -c -emit-llvm -o > %t.bc > +// RUN: %clang -O0 -target powerpc64le-ibm-linux-gnu %s -S -o %t.s > +// RUN: %clang -O0 -target powerpc64le-ibm-linux-gnu %s -emit-ast -o > %t.ast > + > +// > +// Generate an empty file to help with the checks of empty files. > +// > +// RUN: touch %t.empty > + > +// > +// Generate a couple of files to bundle with. > +// > +// RUN: echo 'Content of device file 1' > %t.tgt1 > +// RUN: echo 'Content of device file 2' > %t.tgt2 > + > +// > +// Check help message. > +// > +// RUN: clang-offload-bundler --help | FileCheck %s --check-prefix CK-HELP > +// CK-HELP: {{.*}}OVERVIEW: A tool to bundle several input files of the > specified type <type> > +// CK-HELP: {{.*}}referring to the same source file but different targets > into a single > +// CK-HELP: {{.*}}one. The resulting file can also be unbundled into > different files by > +// CK-HELP: {{.*}}this tool if -unbundle is provided. > +// CK-HELP: {{.*}}USAGE: clang-offload-bundler [subcommand] [options] > +// CK-HELP: {{.*}}-inputs=<string> - [<input file>,...] > +// CK-HELP: {{.*}}-outputs=<string> - [<output file>,...] > +// CK-HELP: {{.*}}-targets=<string> - [<offload kind>-<target triple>,...] > +// CK-HELP: {{.*}}-type=<string> - Type of the files to be > bundled/unbundled. > +// CK-HELP: {{.*}}Current supported types are: > +// CK-HELP: {{.*}}i {{.*}}- cpp-output > +// CK-HELP: {{.*}}ii {{.*}}- c++-cpp-output > +// CK-HELP: {{.*}}ll {{.*}}- llvm > +// CK-HELP: {{.*}}bc {{.*}}- llvm-bc > +// CK-HELP: {{.*}}s {{.*}}- assembler > +// CK-HELP: {{.*}}o {{.*}}- object > +// CK-HELP: {{.*}}gch {{.*}}- precompiled-header > +// CK-HELP: {{.*}}ast {{.*}}- clang AST file > +// CK-HELP: {{.*}}-unbundle {{.*}}- Unbundle bundled file into several > output files. > + > +// > +// Check errors. > +// > +// RUN: not clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.i,%t.tgt1,%t.tgt2 -outputs=%t.bundle.i -unbundle 2>&1 | > FileCheck %s --check-prefix CK-ERR1 > +// CK-ERR1: error: only one input file supported in unbundling mode. > +// CK-ERR1: error: number of output files and targets should match in > unbundling mode. > + > +// RUN: not clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu -inputs=%t.i,%t.tgt1,%t.tgt2 > -outputs=%t.bundle.i 2>&1 | FileCheck %s --check-prefix CK-ERR2 > +// RUN: not clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.i,%t.tgt1 -outputs=%t.bundle.i 2>&1 | FileCheck %s > --check-prefix CK-ERR2 > +// CK-ERR2: error: number of input files and targets should match in > bundling mode. > + > +// RUN: not clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.i,%t.tgt1,%t.tgt2 -inputs=%t.bundle.i 2>&1 | FileCheck %s > --check-prefix CK-ERR3 > +// CK-ERR3: error: only one output file supported in bundling mode. > +// CK-ERR3: error: number of input files and targets should match in > bundling mode. > + > +// RUN: not clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu -outputs=%t.i,%t.tgt1,%t.tgt2 > -inputs=%t.bundle.i -unbundle 2>&1 | FileCheck %s --check-prefix CK-ERR4 > +// RUN: not clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.i,%t.tgt1 -inputs=%t.bundle.i -unbundle 2>&1 | FileCheck %s > --check-prefix CK-ERR4 > +// CK-ERR4: error: number of output files and targets should match in > unbundling mode. > + > +// RUN: not clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.i,%t.tgt1,%t.tgt2.notexist -outputs=%t.bundle.i 2>&1 | > FileCheck %s --check-prefix CK-ERR5 > +// RUN: not clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.i,%t.tgt1,%t.tgt2 -inputs=%t.bundle.i.notexist -unbundle 2>&1 | > FileCheck %s --check-prefix CK-ERR5 > +// CK-ERR5: error: Can't open file {{.+}}.notexist: No such file or > directory > + > +// RUN: not clang-offload-bundler -type=invalid > -targets=host-powerpc64le-ibm-linux-gnu,openmp-powerpc64le- > ibm-linux-gnu,openmp-x86_64-pc-linux-gnu -inputs=%t.i,%t.tgt1,%t.tgt2 > -outputs=%t.bundle.i 2>&1 | FileCheck %s --check-prefix CK-ERR6 > +// CK-ERR6: error: invalid file type specified. > + > +// RUN: not clang-offload-bundler 2>&1 | FileCheck %s --check-prefix > CK-ERR7 > +// CK-ERR7-DAG: clang-offload-bundler: for the -type option: must be > specified at least once! > +// CK-ERR7-DAG: clang-offload-bundler: for the -inputs option: must be > specified at least once! > +// CK-ERR7-DAG: clang-offload-bundler: for the -outputs option: must be > specified at least once! > +// CK-ERR7-DAG: clang-offload-bundler: for the -targets option: must be > specified at least once! > + > +// RUN: not clang-offload-bundler -type=i -targets=hxst-powerpcxxle-ibm- > linux-gnu,openxp-pxxerpc64le-ibm-linux-gnu,xpenmp-x86_xx-pc-linux-gnu > -inputs=%t.i,%t.tgt1,%t.tgt2 -outputs=%t.bundle.i 2>&1 | FileCheck %s > --check-prefix CK-ERR8 > +// CK-ERR8: error: invalid target 'hxst-powerpcxxle-ibm-linux-gnu', > unknown offloading kind 'hxst', unknown target triple > 'powerpcxxle-ibm-linux-gnu'. > +// CK-ERR8: error: invalid target 'openxp-pxxerpc64le-ibm-linux-gnu', > unknown offloading kind 'openxp', unknown target triple > 'pxxerpc64le-ibm-linux-gnu'. > +// CK-ERR8: error: invalid target 'xpenmp-x86_xx-pc-linux-gnu', unknown > offloading kind 'xpenmp', unknown target triple 'x86_xx-pc-linux-gnu'. > + > +// RUN: not clang-offload-bundler -type=i -targets=openmp-powerpc64le- > linux,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.i,%t.tgt1,%t.tgt2 -outputs=%t.bundle.i 2>&1 | FileCheck %s > --check-prefix CK-ERR9A > +// RUN: not clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,host-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.i,%t.tgt1,%t.tgt2 -outputs=%t.bundle.i 2>&1 | FileCheck %s > --check-prefix CK-ERR9B > +// CK-ERR9A: error: expecting exactly one host target but got 0. > +// CK-ERR9B: error: expecting exactly one host target but got 2. > + > +// > +// Check text bundle. This is a readable format, so we check for the > format we expect to find. > +// > +// RUN: clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.i,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.i > +// RUN: clang-offload-bundler -type=ii -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.ii,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.ii > +// RUN: clang-offload-bundler -type=ll -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.ll,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.ll > +// RUN: clang-offload-bundler -type=s -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.s,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.s > +// RUN: clang-offload-bundler -type=s -targets=openmp-powerpc64le- > ibm-linux-gnu,host-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.tgt1,%t.s,%t.tgt2 -outputs=%t.bundle3.unordered.s > +// RUN: FileCheck %s --input-file %t.bundle3.i --check-prefix CK-TEXTI > +// RUN: FileCheck %s --input-file %t.bundle3.ii --check-prefix CK-TEXTI > +// RUN: FileCheck %s --input-file %t.bundle3.ll --check-prefix CK-TEXTLL > +// RUN: FileCheck %s --input-file %t.bundle3.s --check-prefix CK-TEXTS > +// RUN: FileCheck %s --input-file %t.bundle3.unordered.s --check-prefix > CK-TEXTS-UNORDERED > + > +// CK-TEXTI: // __CLANG_OFFLOAD_BUNDLE____START__ > host-powerpc64le-ibm-linux-gnu > +// CK-TEXTI: int A = 0; > +// CK-TEXTI: test_func(void) > +// CK-TEXTI: // __CLANG_OFFLOAD_BUNDLE____END__ > host-powerpc64le-ibm-linux-gnu > +// CK-TEXTI: // __CLANG_OFFLOAD_BUNDLE____START__ > openmp-powerpc64le-ibm-linux-gnu > +// CK-TEXTI: Content of device file 1 > +// CK-TEXTI: // __CLANG_OFFLOAD_BUNDLE____END__ > openmp-powerpc64le-ibm-linux-gnu > +// CK-TEXTI: // __CLANG_OFFLOAD_BUNDLE____START__ > openmp-x86_64-pc-linux-gnu > +// CK-TEXTI: Content of device file 2 > +// CK-TEXTI: // __CLANG_OFFLOAD_BUNDLE____END__ > openmp-x86_64-pc-linux-gnu > + > +// CK-TEXTLL: ; __CLANG_OFFLOAD_BUNDLE____START__ > host-powerpc64le-ibm-linux-gnu > +// CK-TEXTLL: @A = global i32 0 > +// CK-TEXTLL: define {{.*}}@test_func() > +// CK-TEXTLL: ; __CLANG_OFFLOAD_BUNDLE____END__ > host-powerpc64le-ibm-linux-gnu > +// CK-TEXTLL: ; __CLANG_OFFLOAD_BUNDLE____START__ > openmp-powerpc64le-ibm-linux-gnu > +// CK-TEXTLL: Content of device file 1 > +// CK-TEXTLL: ; __CLANG_OFFLOAD_BUNDLE____END__ > openmp-powerpc64le-ibm-linux-gnu > +// CK-TEXTLL: ; __CLANG_OFFLOAD_BUNDLE____START__ > openmp-x86_64-pc-linux-gnu > +// CK-TEXTLL: Content of device file 2 > +// CK-TEXTLL: ; __CLANG_OFFLOAD_BUNDLE____END__ > openmp-x86_64-pc-linux-gnu > + > +// CK-TEXTS: # __CLANG_OFFLOAD_BUNDLE____START__ > host-powerpc64le-ibm-linux-gnu > +// CK-TEXTS: .globl {{.*}}test_func > +// CK-TEXTS: .globl {{.*}}A > +// CK-TEXTS: # __CLANG_OFFLOAD_BUNDLE____END__ > host-powerpc64le-ibm-linux-gnu > +// CK-TEXTS: # __CLANG_OFFLOAD_BUNDLE____START__ > openmp-powerpc64le-ibm-linux-gnu > +// CK-TEXTS: Content of device file 1 > +// CK-TEXTS: # __CLANG_OFFLOAD_BUNDLE____END__ > openmp-powerpc64le-ibm-linux-gnu > +// CK-TEXTS: # __CLANG_OFFLOAD_BUNDLE____START__ > openmp-x86_64-pc-linux-gnu > +// CK-TEXTS: Content of device file 2 > +// CK-TEXTS: # __CLANG_OFFLOAD_BUNDLE____END__ openmp-x86_64-pc-linux-gnu > + > +// CK-TEXTS-UNORDERED: # __CLANG_OFFLOAD_BUNDLE____START__ > openmp-powerpc64le-ibm-linux-gnu > +// CK-TEXTS-UNORDERED: Content of device file 1 > +// CK-TEXTS-UNORDERED: # __CLANG_OFFLOAD_BUNDLE____END__ > openmp-powerpc64le-ibm-linux-gnu > +// CK-TEXTS-UNORDERED: # __CLANG_OFFLOAD_BUNDLE____START__ > host-powerpc64le-ibm-linux-gnu > +// CK-TEXTS-UNORDERED: .globl {{.*}}test_func > +// CK-TEXTS-UNORDERED: .globl {{.*}}A > +// CK-TEXTS-UNORDERED: # __CLANG_OFFLOAD_BUNDLE____END__ > host-powerpc64le-ibm-linux-gnu > +// CK-TEXTS-UNORDERED: # __CLANG_OFFLOAD_BUNDLE____START__ > openmp-x86_64-pc-linux-gnu > +// CK-TEXTS-UNORDERED: Content of device file 2 > +// CK-TEXTS-UNORDERED: # __CLANG_OFFLOAD_BUNDLE____END__ > openmp-x86_64-pc-linux-gnu > + > +// > +// Check text unbundle. Check if we get the exact same content that we > bundled before for each file. > +// > +// RUN: clang-offload-bundler -type=i -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.i,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bundle3.i -unbundle > +// RUN: diff %t.i %t.res.i > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > +// RUN: clang-offload-bundler -type=ii -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.ii,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bundle3.ii -unbundle > +// RUN: diff %t.ii %t.res.ii > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > +// RUN: clang-offload-bundler -type=ll -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.ll,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bundle3.ll -unbundle > +// RUN: diff %t.ll %t.res.ll > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > +// RUN: clang-offload-bundler -type=s -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.s,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bundle3.s -unbundle > +// RUN: diff %t.s %t.res.s > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > +// RUN: clang-offload-bundler -type=s -targets=openmp-powerpc64le- > ibm-linux-gnu,host-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.tgt1,%t.res.s,%t.res.tgt2 -inputs=%t.bundle3.s -unbundle > +// RUN: diff %t.s %t.res.s > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > + > +// Check if we can unbundle a file with no magic strings. > +// RUN: clang-offload-bundler -type=s -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.s,%t.res.tgt1,%t.res.tgt2 -inputs=%t.s -unbundle > +// RUN: diff %t.s %t.res.s > +// RUN: diff %t.empty %t.res.tgt1 > +// RUN: diff %t.empty %t.res.tgt2 > +// RUN: clang-offload-bundler -type=s -targets=openmp-powerpc64le- > ibm-linux-gnu,host-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.tgt1,%t.res.s,%t.res.tgt2 -inputs=%t.s -unbundle > +// RUN: diff %t.s %t.res.s > +// RUN: diff %t.empty %t.res.tgt1 > +// RUN: diff %t.empty %t.res.tgt2 > + > +// > +// Check binary bundle/unbundle. The content that we have before bundling > must be the same we have after unbundling. > +// > +// RUN: clang-offload-bundler -type=bc -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.bc,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.bc > +// RUN: clang-offload-bundler -type=gch -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.ast,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.gch > +// RUN: clang-offload-bundler -type=ast -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.ast,%t.tgt1,%t.tgt2 -outputs=%t.bundle3.ast > +// RUN: clang-offload-bundler -type=ast -targets=openmp-powerpc64le- > ibm-linux-gnu,host-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -inputs=%t.tgt1,%t.ast,%t.tgt2 -outputs=%t.bundle3.unordered.ast > +// RUN: clang-offload-bundler -type=bc -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.bc,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bundle3.bc -unbundle > +// RUN: diff %t.bc %t.res.bc > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > +// RUN: clang-offload-bundler -type=gch -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.gch,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bundle3.gch > -unbundle > +// RUN: diff %t.ast %t.res.gch > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > +// RUN: clang-offload-bundler -type=ast -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.ast,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bundle3.ast > -unbundle > +// RUN: diff %t.ast %t.res.ast > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > +// RUN: clang-offload-bundler -type=ast -targets=openmp-powerpc64le- > ibm-linux-gnu,host-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.tgt1,%t.res.ast,%t.res.tgt2 -inputs=%t.bundle3.ast > -unbundle > +// RUN: diff %t.ast %t.res.ast > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > +// RUN: clang-offload-bundler -type=ast -targets=openmp-powerpc64le- > ibm-linux-gnu,host-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.tgt1,%t.res.ast,%t.res.tgt2 -inputs=%t.bundle3.unordered.ast > -unbundle > +// RUN: diff %t.ast %t.res.ast > +// RUN: diff %t.tgt1 %t.res.tgt1 > +// RUN: diff %t.tgt2 %t.res.tgt2 > + > +// Check if we can unbundle a file with no magic strings. > +// RUN: clang-offload-bundler -type=bc -targets=host-powerpc64le-ibm- > linux-gnu,openmp-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.bc,%t.res.tgt1,%t.res.tgt2 -inputs=%t.bc -unbundle > +// RUN: diff %t.bc %t.res.bc > +// RUN: diff %t.empty %t.res.tgt1 > +// RUN: diff %t.empty %t.res.tgt2 > +// RUN: clang-offload-bundler -type=bc -targets=openmp-powerpc64le- > ibm-linux-gnu,host-powerpc64le-ibm-linux-gnu,openmp-x86_64-pc-linux-gnu > -outputs=%t.res.tgt1,%t.res.bc,%t.res.tgt2 -inputs=%t.bc -unbundle > +// RUN: diff %t.bc %t.res.bc > +// RUN: diff %t.empty %t.res.tgt1 > +// RUN: diff %t.empty %t.res.tgt2 > + > +// Some code so that we can create a binary out of this file. > +int A = 0; > +void test_func(void) { > + ++A; > +} > > Modified: cfe/trunk/tools/CMakeLists.txt > URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/ > CMakeLists.txt?rev=279632&r1=279631&r2=279632&view=diff > ============================================================ > ================== > --- cfe/trunk/tools/CMakeLists.txt (original) > +++ cfe/trunk/tools/CMakeLists.txt Wed Aug 24 10:21:05 2016 > @@ -5,6 +5,7 @@ add_clang_subdirectory(driver) > add_clang_subdirectory(clang-format) > add_clang_subdirectory(clang-format-vs) > add_clang_subdirectory(clang-fuzzer) > +add_clang_subdirectory(clang-offload-bundler) > > add_clang_subdirectory(c-index-test) > > > Added: cfe/trunk/tools/clang-offload-bundler/CMakeLists.txt > URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang- > offload-bundler/CMakeLists.txt?rev=279632&view=auto > ============================================================ > ================== > --- cfe/trunk/tools/clang-offload-bundler/CMakeLists.txt (added) > +++ cfe/trunk/tools/clang-offload-bundler/CMakeLists.txt Wed Aug 24 > 10:21:05 2016 > @@ -0,0 +1,19 @@ > +set(LLVM_LINK_COMPONENTS support) > + > +add_clang_executable(clang-offload-bundler > + ClangOffloadBundler.cpp > + ) > + > +set(CLANG_OFFLOAD_BUNDLER_LIB_DEPS > + clangBasic > + LLVMBitWriter > + LLVMObject > + ) > + > +add_dependencies(clang clang-offload-bundler) > + > +target_link_libraries(clang-offload-bundler > + ${CLANG_OFFLOAD_BUNDLER_LIB_DEPS} > + ) > + > +install(TARGETS clang-offload-bundler RUNTIME DESTINATION bin) > > Added: cfe/trunk/tools/clang-offload-bundler/ClangOffloadBundler.cpp > URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang- > offload-bundler/ClangOffloadBundler.cpp?rev=279632&view=auto > ============================================================ > ================== > --- cfe/trunk/tools/clang-offload-bundler/ClangOffloadBundler.cpp (added) > +++ cfe/trunk/tools/clang-offload-bundler/ClangOffloadBundler.cpp Wed Aug > 24 10:21:05 2016 > @@ -0,0 +1,681 @@ > +//===-- clang-offload-bundler/ClangOffloadBundler.cpp - Clang format > tool -===// > +// > +// The LLVM Compiler Infrastructure > +// > +// This file is distributed under the University of Illinois Open Source > +// License. See LICENSE.TXT for details. > +// > +//===------------------------------------------------------ > ----------------===// > +/// > +/// \file > +/// \brief This file implements a clang-offload-bundler that bundles > different > +/// files that relate with the same source code but different targets > into a > +/// single one. Also the implements the opposite functionality, i.e. > unbundle > +/// files previous created by this tool. > +/// > +//===------------------------------------------------------ > ----------------===// > + > +#include "clang/Basic/FileManager.h" > +#include "clang/Basic/Version.h" > +#include "llvm/ADT/StringMap.h" > +#include "llvm/ADT/StringSwitch.h" > +#include "llvm/Bitcode/ReaderWriter.h" > +#include "llvm/IR/Constants.h" > +#include "llvm/IR/LLVMContext.h" > +#include "llvm/IR/Module.h" > +#include "llvm/Object/Binary.h" > +#include "llvm/Object/ELFObjectFile.h" > +#include "llvm/Object/ObjectFile.h" > +#include "llvm/Support/CommandLine.h" > +#include "llvm/Support/FileSystem.h" > +#include "llvm/Support/Path.h" > +#include "llvm/Support/Program.h" > +#include "llvm/Support/Signals.h" > + > +using namespace llvm; > +using namespace llvm::object; > + > +static cl::opt<bool> Help("h", cl::desc("Alias for -help"), cl::Hidden); > + > +// Mark all our options with this category, everything else (except for > -version > +// and -help) will be hidden. > +static cl::OptionCategory > + ClangOffloadBundlerCategory("clang-offload-bundler options"); > + > +static cl::list<std::string> > + InputFileNames("inputs", cl::CommaSeparated, cl::OneOrMore, > + cl::desc("[<input file>,...]"), > + cl::cat(ClangOffloadBundlerCategory)); > +static cl::list<std::string> > + OutputFileNames("outputs", cl::CommaSeparated, cl::OneOrMore, > + cl::desc("[<output file>,...]"), > + cl::cat(ClangOffloadBundlerCategory)); > +static cl::list<std::string> > + TargetNames("targets", cl::CommaSeparated, cl::OneOrMore, > + cl::desc("[<offload kind>-<target triple>,...]"), > + cl::cat(ClangOffloadBundlerCategory)); > +static cl::opt<std::string> > + FilesType("type", cl::Required, > + cl::desc("Type of the files to be bundled/unbundled.\n" > + "Current supported types are:\n" > + " i - cpp-output\n" > + " ii - c++-cpp-output\n" > + " ll - llvm\n" > + " bc - llvm-bc\n" > + " s - assembler\n" > + " o - object\n" > + " gch - precompiled-header\n" > + " ast - clang AST file"), > + cl::cat(ClangOffloadBundlerCategory)); > +static cl::opt<bool> > + Unbundle("unbundle", > + cl::desc("Unbundle bundled file into several output > files.\n"), > + cl::init(false), cl::cat(ClangOffloadBundlerCategory)); > + > +/// Magic string that marks the existence of offloading data. > +#define OFFLOAD_BUNDLER_MAGIC_STR "__CLANG_OFFLOAD_BUNDLE__" > + > +/// The index of the host input in the list of inputs. > +static unsigned HostInputIndex = ~0u; > + > +/// Obtain the offload kind and real machine triple out of the target > +/// information specified by the user. > +static void getOffloadKindAndTriple(StringRef Target, StringRef > &OffloadKind, > + StringRef &Triple) { > + auto KindTriplePair = Target.split('-'); > + OffloadKind = KindTriplePair.first; > + Triple = KindTriplePair.second; > +} > +static bool hasHostKind(StringRef Target) { > + StringRef OffloadKind; > + StringRef Triple; > + getOffloadKindAndTriple(Target, OffloadKind, Triple); > + return OffloadKind == "host"; > +} > + > +/// Generic file handler interface. > +class FileHandler { > +public: > + /// Update the file handler with information from the header of the > bundled > + /// file > + virtual void ReadHeader(MemoryBuffer &Input) = 0; > + /// Read the marker of the next bundled to be read in the file. The > triple of > + /// the target associated with that bundle is returned. An empty string > is > + /// returned if there are no more bundles to be read. > + virtual StringRef ReadBundleStart(MemoryBuffer &Input) = 0; > + /// Read the marker that closes the current bundle. > + virtual void ReadBundleEnd(MemoryBuffer &Input) = 0; > + /// Read the current bundle and write the result into the stream \a OS. > + virtual void ReadBundle(raw_fd_ostream &OS, MemoryBuffer &Input) = 0; > + > + /// Write the header of the bundled file to \a OS based on the > information > + /// gathered from \a Inputs. > + virtual void WriteHeader(raw_fd_ostream &OS, > + ArrayRef<std::unique_ptr<MemoryBuffer>> > Inputs) = 0; > + /// Write the marker that initiates a bundle for the triple \a > TargetTriple to > + /// \a OS. > + virtual void WriteBundleStart(raw_fd_ostream &OS, StringRef > TargetTriple) = 0; > + /// Write the marker that closes a bundle for the triple \a > TargetTriple to \a > + /// OS. > + virtual void WriteBundleEnd(raw_fd_ostream &OS, StringRef TargetTriple) > = 0; > + /// Write the bundle from \a Input into \a OS. > + virtual void WriteBundle(raw_fd_ostream &OS, MemoryBuffer &Input) = 0; > + > + FileHandler() {} > + virtual ~FileHandler() {} > +}; > + > +/// Handler for binary files. The bundled file will have the following > format > +/// (all integers are stored in little-endian format): > +/// > +/// "OFFLOAD_BUNDLER_MAGIC_STR" (ASCII encoding of the string) > +/// > +/// NumberOfOffloadBundles (8-byte integer) > +/// > +/// OffsetOfBundle1 (8-byte integer) > +/// SizeOfBundle1 (8-byte integer) > +/// NumberOfBytesInTripleOfBundle1 (8-byte integer) > +/// TripleOfBundle1 (byte length defined before) > +/// > +/// ... > +/// > +/// OffsetOfBundleN (8-byte integer) > +/// SizeOfBundleN (8-byte integer) > +/// NumberOfBytesInTripleOfBundleN (8-byte integer) > +/// TripleOfBundleN (byte length defined before) > +/// > +/// Bundle1 > +/// ... > +/// BundleN > + > +/// Read 8-byte integers from a buffer in little-endian format. > +static uint64_t Read8byteIntegerFromBuffer(StringRef Buffer, size_t pos) > { > + uint64_t Res = 0; > + const char *Data = Buffer.data(); > + > + for (unsigned i = 0; i < 8; ++i) { > + Res <<= 8; > + uint64_t Char = (uint64_t)Data[pos + 7 - i]; > + Res |= 0xffu & Char; > + } > + return Res; > +} > + > +/// Write 8-byte integers to a buffer in little-endian format. > +static void Write8byteIntegerToBuffer(raw_fd_ostream &OS, uint64_t Val) { > + > + for (unsigned i = 0; i < 8; ++i) { > + char Char = (char)(Val & 0xffu); > + OS.write(&Char, 1); > + Val >>= 8; > + } > +} > + > +class BinaryFileHandler final : public FileHandler { > + /// Information about the bundles extracted from the header. > + struct BundleInfo final { > + /// Size of the bundle. > + uint64_t Size = 0u; > + /// Offset at which the bundle starts in the bundled file. > + uint64_t Offset = 0u; > + BundleInfo() {} > + BundleInfo(uint64_t Size, uint64_t Offset) : Size(Size), > Offset(Offset) {} > + }; > + /// Map between a triple and the corresponding bundle information. > + StringMap<BundleInfo> BundlesInfo; > + > + /// Iterator for the bundle information that is being read. > + StringMap<BundleInfo>::iterator CurBundleInfo; > + > +public: > + void ReadHeader(MemoryBuffer &Input) { > + StringRef FC = Input.getBuffer(); > + > + // Initialize the current bundle with the end of the container. > + CurBundleInfo = BundlesInfo.end(); > + > + // Check if buffer is smaller than magic string. > + size_t ReadChars = sizeof(OFFLOAD_BUNDLER_MAGIC_STR) - 1; > + if (ReadChars > FC.size()) > + return; > + > + // Check if no magic was found. > + StringRef Magic(FC.data(), sizeof(OFFLOAD_BUNDLER_MAGIC_STR) - 1); > + if (!Magic.equals(OFFLOAD_BUNDLER_MAGIC_STR)) > + return; > + > + // Read number of bundles. > + if (ReadChars + 8 > FC.size()) > + return; > + > + uint64_t NumberOfBundles = Read8byteIntegerFromBuffer(FC, ReadChars); > + ReadChars += 8; > + > + // Read bundle offsets, sizes and triples. > + for (uint64_t i = 0; i < NumberOfBundles; ++i) { > + > + // Read offset. > + if (ReadChars + 8 > FC.size()) > + return; > + > + uint64_t Offset = Read8byteIntegerFromBuffer(FC, ReadChars); > + ReadChars += 8; > + > + // Read size. > + if (ReadChars + 8 > FC.size()) > + return; > + > + uint64_t Size = Read8byteIntegerFromBuffer(FC, ReadChars); > + ReadChars += 8; > + > + // Read triple size. > + if (ReadChars + 8 > FC.size()) > + return; > + > + uint64_t TripleSize = Read8byteIntegerFromBuffer(FC, ReadChars); > + ReadChars += 8; > + > + // Read triple. > + if (ReadChars + TripleSize > FC.size()) > + return; > + > + StringRef Triple(&FC.data()[ReadChars], TripleSize); > + ReadChars += TripleSize; > + > + // Check if the offset and size make sense. > + if (!Size || !Offset || Offset + Size > FC.size()) > + return; > + > + assert(BundlesInfo.find(Triple) == BundlesInfo.end() && > + "Triple is duplicated??"); > + BundlesInfo[Triple] = BundleInfo(Size, Offset); > + } > + // Set the iterator to where we will start to read. > + CurBundleInfo = BundlesInfo.begin(); > + } > + StringRef ReadBundleStart(MemoryBuffer &Input) { > + if (CurBundleInfo == BundlesInfo.end()) > + return StringRef(); > + > + return CurBundleInfo->first(); > + } > + void ReadBundleEnd(MemoryBuffer &Input) { > + assert(CurBundleInfo != BundlesInfo.end() && "Invalid reader info!"); > + ++CurBundleInfo; > + } > + void ReadBundle(raw_fd_ostream &OS, MemoryBuffer &Input) { > + assert(CurBundleInfo != BundlesInfo.end() && "Invalid reader info!"); > + StringRef FC = Input.getBuffer(); > + OS.write(FC.data() + CurBundleInfo->second.Offset, > + CurBundleInfo->second.Size); > + } > + > + void WriteHeader(raw_fd_ostream &OS, > + ArrayRef<std::unique_ptr<MemoryBuffer>> Inputs) { > + // Compute size of the header. > + uint64_t HeaderSize = 0; > + > + HeaderSize += sizeof(OFFLOAD_BUNDLER_MAGIC_STR) - 1; > + HeaderSize += 8; // Number of Bundles > + > + for (auto &T : TargetNames) { > + HeaderSize += 3 * 8; // Bundle offset, Size of bundle and size of > triple. > + HeaderSize += T.size(); // The triple. > + } > + > + // Write to the buffer the header. > + OS << OFFLOAD_BUNDLER_MAGIC_STR; > + > + Write8byteIntegerToBuffer(OS, TargetNames.size()); > + > + unsigned Idx = 0; > + for (auto &T : TargetNames) { > + MemoryBuffer &MB = *Inputs[Idx++].get(); > + // Bundle offset. > + Write8byteIntegerToBuffer(OS, HeaderSize); > + // Size of the bundle (adds to the next bundle's offset) > + Write8byteIntegerToBuffer(OS, MB.getBufferSize()); > + HeaderSize += MB.getBufferSize(); > + // Size of the triple > + Write8byteIntegerToBuffer(OS, T.size()); > + // Triple > + OS << T; > + } > + } > + void WriteBundleStart(raw_fd_ostream &OS, StringRef TargetTriple) {} > + void WriteBundleEnd(raw_fd_ostream &OS, StringRef TargetTriple) {} > + void WriteBundle(raw_fd_ostream &OS, MemoryBuffer &Input) { > + OS.write(Input.getBufferStart(), Input.getBufferSize()); > + } > + > + BinaryFileHandler() : FileHandler() {} > + ~BinaryFileHandler() {} > +}; > + > +/// Handler for text files. The bundled file will have the following > format. > +/// > +/// "Comment OFFLOAD_BUNDLER_MAGIC_STR__START__ triple" > +/// Bundle 1 > +/// "Comment OFFLOAD_BUNDLER_MAGIC_STR__END__ triple" > +/// ... > +/// "Comment OFFLOAD_BUNDLER_MAGIC_STR__START__ triple" > +/// Bundle N > +/// "Comment OFFLOAD_BUNDLER_MAGIC_STR__END__ triple" > +class TextFileHandler final : public FileHandler { > + /// String that begins a line comment. > + StringRef Comment; > + > + /// String that initiates a bundle. > + std::string BundleStartString; > + > + /// String that closes a bundle. > + std::string BundleEndString; > + > + /// Number of chars read from input. > + size_t ReadChars = 0u; > + > +protected: > + void ReadHeader(MemoryBuffer &Input) {} > + StringRef ReadBundleStart(MemoryBuffer &Input) { > + StringRef FC = Input.getBuffer(); > + > + // Find start of the bundle. > + ReadChars = FC.find(BundleStartString, ReadChars); > + if (ReadChars == FC.npos) > + return StringRef(); > + > + // Get position of the triple. > + size_t TripleStart = ReadChars = ReadChars + BundleStartString.size(); > + > + // Get position that closes the triple. > + size_t TripleEnd = ReadChars = FC.find("\n", ReadChars); > + if (TripleEnd == FC.npos) > + return StringRef(); > + > + // Next time we read after the new line. > + ++ReadChars; > + > + return StringRef(&FC.data()[TripleStart], TripleEnd - TripleStart); > + } > + void ReadBundleEnd(MemoryBuffer &Input) { > + StringRef FC = Input.getBuffer(); > + > + // Read up to the next new line. > + assert(FC[ReadChars] == '\n' && "The bundle should end with a new > line."); > + > + size_t TripleEnd = ReadChars = FC.find("\n", ReadChars + 1); > + if (TripleEnd == FC.npos) > + return; > + > + // Next time we read after the new line. > + ++ReadChars; > + } > + void ReadBundle(raw_fd_ostream &OS, MemoryBuffer &Input) { > + StringRef FC = Input.getBuffer(); > + size_t BundleStart = ReadChars; > + > + // Find end of the bundle. > + size_t BundleEnd = ReadChars = FC.find(BundleEndString, ReadChars); > + > + StringRef Bundle(&FC.data()[BundleStart], BundleEnd - BundleStart); > + OS << Bundle; > + } > + > + void WriteHeader(raw_fd_ostream &OS, > + ArrayRef<std::unique_ptr<MemoryBuffer>> Inputs) {} > + void WriteBundleStart(raw_fd_ostream &OS, StringRef TargetTriple) { > + OS << BundleStartString << TargetTriple << "\n"; > + } > + void WriteBundleEnd(raw_fd_ostream &OS, StringRef TargetTriple) { > + OS << BundleEndString << TargetTriple << "\n"; > + } > + void WriteBundle(raw_fd_ostream &OS, MemoryBuffer &Input) { > + OS << Input.getBuffer(); > + } > + > +public: > + TextFileHandler(StringRef Comment) > + : FileHandler(), Comment(Comment), ReadChars(0) { > + BundleStartString = > + "\n" + Comment.str() + " " OFFLOAD_BUNDLER_MAGIC_STR "__START__ "; > + BundleEndString = > + "\n" + Comment.str() + " " OFFLOAD_BUNDLER_MAGIC_STR "__END__ "; > + } > +}; > + > +/// Return an appropriate handler given the input files and options. > +static FileHandler *CreateFileHandler(MemoryBuffer &FirstInput) { > + if (FilesType == "i") > + return new TextFileHandler(/*Comment=*/"//"); > + if (FilesType == "ii") > + return new TextFileHandler(/*Comment=*/"//"); > + if (FilesType == "ll") > + return new TextFileHandler(/*Comment=*/";"); > + if (FilesType == "bc") > + return new BinaryFileHandler(); > + if (FilesType == "s") > + return new TextFileHandler(/*Comment=*/"#"); > + if (FilesType == "o") > + return new BinaryFileHandler(); > + if (FilesType == "gch") > + return new BinaryFileHandler(); > + if (FilesType == "ast") > + return new BinaryFileHandler(); > + > + llvm::errs() << "error: invalid file type specified.\n"; > + return nullptr; > +} > + > +/// Bundle the files. Return true if an error was found. > +static bool BundleFiles() { > + std::error_code EC; > + > + // Create output file. > + raw_fd_ostream OutputFile(OutputFileNames.front(), EC, > sys::fs::F_None); > + > + if (EC) { > + llvm::errs() << "error: Can't open file " << OutputFileNames.front() > + << ".\n"; > + return true; > + } > + > + // Open input files. > + std::vector<std::unique_ptr<MemoryBuffer>> InputBuffers( > + InputFileNames.size()); > + > + unsigned Idx = 0; > + for (auto &I : InputFileNames) { > + ErrorOr<std::unique_ptr<MemoryBuffer>> CodeOrErr = > + MemoryBuffer::getFileOrSTDIN(I); > + if (std::error_code EC = CodeOrErr.getError()) { > + llvm::errs() << "error: Can't open file " << I << ": " << > EC.message() > + << "\n"; > + return true; > + } > + InputBuffers[Idx++] = std::move(CodeOrErr.get()); > + } > + > + // Get the file handler. We use the host buffer as reference. > + assert(HostInputIndex != ~0u && "Host input index undefined??"); > + std::unique_ptr<FileHandler> FH; > + FH.reset(CreateFileHandler(*InputBuffers[HostInputIndex].get())); > + > + // Quit if we don't have a handler. > + if (!FH.get()) > + return true; > + > + // Write header. > + FH.get()->WriteHeader(OutputFile, InputBuffers); > + > + // Write all bundles along with the start/end markers. > + auto Input = InputBuffers.begin(); > + for (auto &Triple : TargetNames) { > + FH.get()->WriteBundleStart(OutputFile, Triple); > + FH.get()->WriteBundle(OutputFile, *Input->get()); > + FH.get()->WriteBundleEnd(OutputFile, Triple); > + ++Input; > + } > + return false; > +} > + > +// Unbundle the files. Return true if an error was found. > +static bool UnbundleFiles() { > + // Open Input file. > + ErrorOr<std::unique_ptr<MemoryBuffer>> CodeOrErr = > + MemoryBuffer::getFileOrSTDIN(InputFileNames.front()); > + if (std::error_code EC = CodeOrErr.getError()) { > + llvm::errs() << "error: Can't open file " << InputFileNames.front() > << ": " > + << EC.message() << "\n"; > + return true; > + } > + > + MemoryBuffer &Input = *CodeOrErr.get(); > + > + // Select the right files handler. > + std::unique_ptr<FileHandler> FH; > + FH.reset(CreateFileHandler(Input)); > + > + // Quit if we don't have a handler. > + if (!FH.get()) > + return true; > + > + // Read the header of the bundled file. > + FH.get()->ReadHeader(Input); > + > + // Create a work list that consist of the map triple/output file. > + StringMap<StringRef> Worklist; > + auto Output = OutputFileNames.begin(); > + for (auto &Triple : TargetNames) { > + Worklist[Triple] = *Output; > + ++Output; > + } > + > + // Read all the bundles that are in the work list. If we find no > bundles we > + // assume the file is meant for the host target. > + bool FoundHostBundle = false; > + while (!Worklist.empty()) { > + StringRef CurTriple = FH.get()->ReadBundleStart(Input); > + > + // We don't have more bundles. > + if (CurTriple.empty()) > + break; > + > + auto Output = Worklist.find(CurTriple); > + // The file may have more bundles for other targets, that we don't > care > + // about. Therefore, move on to the next triple > + if (Output == Worklist.end()) { > + continue; > + } > + > + // Check if the output file can be opened and copy the bundle to it. > + std::error_code EC; > + raw_fd_ostream OutputFile(Output->second, EC, sys::fs::F_None); > + if (EC) { > + llvm::errs() << "error: Can't open file " << Output->second << ": " > + << EC.message() << "\n"; > + return true; > + } > + FH.get()->ReadBundle(OutputFile, Input); > + FH.get()->ReadBundleEnd(Input); > + Worklist.remove(&*Output); > + > + // Record if we found the host bundle. > + if (hasHostKind(CurTriple)) > + FoundHostBundle = true; > + } > + > + // If no bundles were found, assume the input file is the host bundle > and > + // create empty files for the remaining targets. > + if (Worklist.size() == TargetNames.size()) { > + for (auto &E : Worklist) { > + std::error_code EC; > + raw_fd_ostream OutputFile(E.second, EC, sys::fs::F_None); > + if (EC) { > + llvm::errs() << "error: Can't open file " << E.second << ": " > + << EC.message() << "\n"; > + return true; > + } > + > + // If this entry has a host kind, copy the input file to the output > file. > + if (hasHostKind(E.first())) > + OutputFile.write(Input.getBufferStart(), Input.getBufferSize()); > + } > + return false; > + } > + > + // If we found elements, we emit an error if none of those were for the > host. > + if (!FoundHostBundle) { > + llvm::errs() << "error: Can't find bundle for the host target\n"; > + return true; > + } > + > + // If we still have any elements in the worklist, create empty files > for them. > + for (auto &E : Worklist) { > + std::error_code EC; > + raw_fd_ostream OutputFile(E.second, EC, sys::fs::F_None); > + if (EC) { > + llvm::errs() << "error: Can't open file " << E.second << ": " > + << EC.message() << "\n"; > + return true; > + } > + } > + > + return false; > +} > + > +static void PrintVersion() { > + raw_ostream &OS = outs(); > + OS << clang::getClangToolFullVersion("clang-offload-bundler") << '\n'; > +} > + > +int main(int argc, const char **argv) { > + llvm::sys::PrintStackTraceOnErrorSignal(argv[0]); > + > + cl::HideUnrelatedOptions(ClangOffloadBundlerCategory); > + cl::SetVersionPrinter(PrintVersion); > + cl::ParseCommandLineOptions( > + argc, argv, > + "A tool to bundle several input files of the specified type <type> > \n" > + "referring to the same source file but different targets into a > single \n" > + "one. The resulting file can also be unbundled into different files > by \n" > + "this tool if -unbundle is provided.\n"); > + > + if (Help) > + cl::PrintHelpMessage(); > + > + bool Error = false; > + if (Unbundle) { > + if (InputFileNames.size() != 1) { > + Error = true; > + llvm::errs() > + << "error: only one input file supported in unbundling mode.\n"; > + } > + if (OutputFileNames.size() != TargetNames.size()) { > + Error = true; > + llvm::errs() << "error: number of output files and targets should > match " > + "in unbundling mode.\n"; > + } > + } else { > + if (OutputFileNames.size() != 1) { > + Error = true; > + llvm::errs() > + << "error: only one output file supported in bundling mode.\n"; > + } > + if (InputFileNames.size() != TargetNames.size()) { > + Error = true; > + llvm::errs() << "error: number of input files and targets should > match " > + "in bundling mode.\n"; > + } > + } > + > + // Verify that the offload kinds and triples are known. We also check > that we > + // have exactly one host target. > + unsigned Index = 0u; > + unsigned HostTargetNum = 0u; > + for (StringRef Target : TargetNames) { > + StringRef Kind; > + StringRef Triple; > + getOffloadKindAndTriple(Target, Kind, Triple); > + > + bool KindIsValid = !Kind.empty(); > + KindIsValid = KindIsValid && > + StringSwitch<bool>(Kind) > + .Case("host", true) > + .Case("openmp", true) > + .Default(false); > + > + bool TripleIsValid = !Triple.empty(); > + llvm::Triple T(Triple); > + TripleIsValid &= T.getArch() != Triple::UnknownArch; > + > + if (!KindIsValid || !TripleIsValid) { > + Error = true; > + llvm::errs() << "error: invalid target '" << Target << "'"; > + > + if (!KindIsValid) > + llvm::errs() << ", unknown offloading kind '" << Kind << "'"; > + if (!TripleIsValid) > + llvm::errs() << ", unknown target triple '" << Triple << "'"; > + llvm::errs() << ".\n"; > + } > + > + if (KindIsValid && Kind == "host") { > + ++HostTargetNum; > + // Save the index of the input that refers to the host. > + HostInputIndex = Index; > + } > + > + ++Index; > + } > + > + if (HostTargetNum != 1) { > + Error = true; > + llvm::errs() << "error: expecting exactly one host target but got " > + << HostTargetNum << ".\n"; > + } > + > + if (Error) > + return 1; > + > + return Unbundle ? UnbundleFiles() : BundleFiles(); > +} > > > _______________________________________________ > cfe-commits mailing list > cfe-commits@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits >
_______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
