MELT plugin: test fopen
Hello, I would like to present you a small plugin, which could be a good exemple of a MELT use case. This plugin allows to monitor that after every call to the fopen function, we have a test on the pointer returned by fopen (monitoring that it is not null). It creates a pass after SSA and works on gimple. It firstly matchs the gimple_call on fopen and save the tree corresponding to the FILE *. Then we check that the next instruction is a gimple_cond between the saved tree and the NULL ptr. When no test are found, the following error is returned: "test_fopen_cfile.c:35:11: warning: fopen not followed by a test on his returned pointer [enabled by default]" I think MELT is particulary adaptated when we have to match tree or gimples like I do here. For the moment I have only used it on small test file. I will try to see what it gives on a more realistic small to medium application. The code can be find on github: https://github.com/Piervit/GMWarn . The idea is to add more plugins. If you have some ideas or remarks, I am interested (however I still have to learn a lot from both GCC and MELT). If you try the code you will need to use the GCC MELT branch (or there is some changes to do in the Makefile to have it with MELT as plugin). Regards!
customizable warnings with a GCC plugin
Hi, My name is Pierre Vittet and my GSOC application as been selected. My project is about writing a plugin allowing GCC users to add some simple warnings, being useful in their particular project. The user should be able to add rules like "when I got a call to a foo function, I would like to be sure that a check is made to monitor that it doesn't return somethings null" or "when I call this foo function, I would like it to be followed by that bar function". I will use MELT in order to realize this plugin. You can read the detail of my application here: http://pvittet.com/GSOC/GSOC.pdf . I have also started a page on the GCC wiki, it is pretty empty for now but is going to grow with the project: http://gcc.gnu.org/wiki/CustomizableWarningPlugin . For now, I am able to write simple plugins, two exemples can be found here (https://github.com/Piervit/GMWarn). However, that is still difficult to test them on real size programs. Mainly because it can be difficult to build a complete program using GCC MELT (the branch). For exemple GCC MELT has mainly been tested on C programs, so if there is a C++ part, it mights fails, I have got also some problems when the configure scripts had to check GCCMELT. There is a fresh release of MELT as a plugin which will simplify things, I am going to report few issues coming from it. I think MELT is improving , and I am confident in being able to compile on real programs (I could already build readline for exemple). So, I will be ready to start real coding of the plugin for the GSOC starting date (Mai 23). For the moment I would like to give the following monitoring opportunity to the user: - Checking that the returned value of a call to foo function is tested to be (not) NULL. - Checking that the returned value of a call to foo function is tested to be (not) negative. - Checking that the returned value of a call to foo function is tested to be (not) zero. - Checking that a call to foo is immediately followed by a call to the bar function. - Checking that a call to foo is followed in the same function by a call to the bar function. My checks are limited to a function because I am going to work at the GIMPLE representation, and pass are run successively for each function. I am not sure that MELT is ready for IPA pass (or that might be experimental work). There is sufficient things to do with GIMPLE pass. Do you see any others interesting cases to check? Another big question: For this type of plugin, what would be the best position to insert my pass. For now, I try to add it just after the SSA pass and as it worked, I keep it at the position. Is it a good choice? I need a pass which is always run (or my plugin will not be inserted) and which has a correct GIMPLE representation. I guess the most difficult point of my project might be to organise how my pass will run the different checks. The user will have the opportunity to run different checks in one compilation. I could choice to make one pass for every checks. This would mean to have a first pass that parse the arguments and set the others pass (Is it possible to create new pass while we are already in one pass ?). The others idea would be to have just one big pass, which is able to run for each GIMPLE statement the different checks in parallel. I am not sure that would be really nice. What is your opinion on this? If you have feedbacks from what is given here, I would enjoy to read them. If you have some documentation which could be related to the pass organisation (I already had a look at the very good GCC tutorial from Albert Cohen (http://www.hipeac.net/node/746)) or to anything looking interesting for my project, it would be great. Thanks for your reading, I hope my English is not so bad. Pierre Vittet
Re: customizable warnings with a GCC plugin
Using MELT is always heavily discussed :). I thinks this project is really an opportunity to see how useful is MELT. Limitation doesn't really come from the syntax (infix or not) but from what is already implemented (cannot make code replacement, not a complete implementation of IPA). However I don't need those functionnalities in my plugin and I will be able to use some nice functionnalities of MELT (pattern matching on trees to just give one). As said, user will not have to know about the Lisp syntax, he will just have to install MELT as a plugin and to add some rules (in the easiest possible syntax). For plugin develloper, it is an opportunity to see how MELT works. About the security issue, I guess that if someone find something very good to detect potential buffer overflow, it does not have to be coded in a plugin but in GCC itself. Moreover, as Basile said, I don't know much about those questions. I don't think it is possible to write a plugin able to help respecting coding rules for every project. I thinks there is always some particular issues comming from the specific project we are working on which make this quite impossible. That why I think, giving to the user an opportunity to easily set his on rules is the best choice. If the user knows that using the unsafe_foo function need a check or another function to be called, this is a way to improve security. Thanks for your mails! On 29/04/2011 16:25, David Brown wrote: On 29/04/2011 13:16, Basile Starynkevitch wrote: On Fri, 29 Apr 2011 12:30:45 +0200 David Brown wrote: There is a lot of interesting and useful work that could be done here. Melt is a nice idea, but the big barrier (for me, anyway) is the language - it's Lisp, which is very different to other languages that I've used. Pierre Vittet knows MELT quite well and has already successfully used it (and he did found some bugs in MELT that I was happy to correct). He is one of the most active posters on the gcc-melt-fre...@googlegroups.com list. We (Pierre& me Basile) met regularily (with Alexandre Lissy). Pierre is perhaps one of the best (and few) MELT users. So MELT is definitely no more an obstacle for him! I wasn't thinking of MELT's language as an obstacle for Pierre - but for us mere mortal C programmers. That's why this is such a good idea for a project if it lets people like me get extra checks without having to learn MELT and Lisp. And regarding the Lispy syntax, I do have a infix syntax in the works. However, contrarily to many others, and conforming to Pierre's experience, the hard part is not learning MELT syntax (which is looking similar to e.g. Elisp or Scheme or Common Lisp code, so learning MELT is fast if you did try Elisp or Scheme before), the hard part is learning GCC internals. Pierre struggled much more understanding where to insert his pass and what precise kind of Gimple he wants to detect than with the MELT syntax. So my feeling is even more than before that MELT lispy syntax is not important: providing an infix syntax to MELT (which in my eyes would be parsed into exactly the same abstract MELT trees as the current lisped syntax) is a significant amount of work, and I am not sure at all it will buy new users to MELT. Hence a question to David Brown: will he use MELT if it had an infix syntax (but all the rest of MELT being the same)? [I am not sure, and this is why making an infix syntax is not my priority] Perhaps I wasn't clear here (or perhaps I misunderstood Pierre's first post). I am not suggesting changing the language for MELT - I assume it uses Lisp because the developers thought that's a good language for such pattern-matching applications. While my Lisp knowledge is very rudimentary, I can easily see it's a better choice than, for example, C - I am a big fan of choosing appropriate languages for the task in hand, rather than trying to do everything with the one language. Of course, I am a bigger fan of people using languages I already know - I am sure Python is the ideal language for MELT :-) As far as I understand Pierre's project, it will let users like me get configurable warnings using a simple syntax on the gcc command line - and that's great. The language used by Pierre is then an implementation detail that doesn't affect me as a user (unless I want to fiddle with the source myself) - it doesn't matter whether I like Lisp or not, because I won't see the Lisp code. As to whether I would be more likely to use MELT if it had another language - probably I would be a little more likely to try it. But as you say, the language is only one part of the issue - understanding how gcc works and how to fit in with it is a far bigger challenge. What would make me much more likely to try it is lots of examples, or library-style code so that I could have something useful (not "hello, world!") to start with, that
[MELT] a successful compilation of GCC using a plugin
Hello,
I would like to announce that I have been able to compile GCC while
checking it with a simple MELT plugin that I have wrote. This plugin is
integrated into GCC by using the MELT plugin. I guess this is the first
successfull use of MELT with a huge real program like GCC.
For those who are not really aware about MELT and how it works:
MELT is available into 2 forms:
-A GCC branch
-A GCC plugin
Here I have use MELT as a GCC plugin.
This GCC plugin loads a melt.so library which has severals modes.
Firstly I have use the translatetomodule mode. This mode allows me to
translate my plugin (which is a file "test_fopen.melt") into several C
files and then into a dynamic test_fopen.so.
The command is the following:
gcc
-fplugin=/usr/lib64/gcc/x86_64-mandriva-linux-gnu/4.6.0/plugin/melt.so \
-fplugin-arg-melt-mode=translatetomodule \
-fplugin-arg-melt-arg=/home/mpierre/programmation/melt/test_fopen/test_fopen.melt
\
-c empty.c 2>fopen_compil_debug.txt
The second step is to compile GCC using CFLAGS which define that we use
the MELT plugin with the mode test_fopen which is defined in my plugin
and precising the location of the test_fopen.so. This allows me to
execute the pass defined in my plugin.
The command is the following:
make CFLAGS="-fplugin=/usr/lib64/gcc/x86_64-mandriva-linux-
gnu/4.6.0/plugin/melt.so -fplugin-arg-melt-mode=test_fopen
-fplugin-arg-melt-init=@@:test_fopen
-fplugin-arg-melt-module-path=$build
-fplugin-arg-melt-source-path=$build -Wno-error"
What does my test_fopen.melt plugin?
It searchs each call to the fopen function and get the lhs (under the
form of a tree).
After the fopen calls, it looks at the next gimple. If this not a
gimple_cond_equal or gimple_cond_notequal between the lhs of the call
and NULL, it returns a warning, explaining that the call was not tested.
This is not perfect for now, it can find some false positifs, in some
special cases, like this one:
s->file = fd < 0 ? F_OPEN(path, fmode) : (FILE*)fdopen(fd, fmode);
if (s->file == NULL) {
...
}
...
It comes from the fact, that at GIMPLE state, the expression is separate
into severals and the test is not made immediatly but after a few gimples.
However, it sends also some pertinents warnings:
-in gcc/gcc.c line 3422, function driver_handle_option:
There is a call to fopen without test, I don't know much about what it
does but can we be sure that the fopen always return somethings not NULL.
note: in reality this is not the fopen function but the fopen_unlocked
function which is called at this step (using a #define). I have adapted
my plugin to search also for fopen_unlocked which call fopen without
testing more the returned value.
-in libcpp/files.c line 1468, function read_name_map:
there is another call to the fopen function.
The full log of my GCC compilation can be found here:
http://pvittet.com/GSOC/log.txt .
The plugin that I have written can be found here:
https://github.com/Piervit/GMWarn/tree/master/test_fopen .
If you have any questions or remarks, I would be glad to read them :)
Re: [MELT] a successful compilation of GCC using a plugin
On 05/05/2011 21:46, Diego Novillo wrote: On Thu, May 5, 2011 at 15:16, Diego Novillo wrote: I've always found MELT interesting as an exercise in impossibility and [ ... ] There was some confusion about my meaning here. It was meant as a compliment. Sorry for the confusion. Diego. Yeah, Basile's work is very impressive. From what I can say (Basile might answer better), the idea is to offer more and more abstraction in order to manipulate as easily as possible the GCC structures. In particular, MELT emphasizes on offering rich pattern matching features (on tree, gimple...). On a (very?) long term view, we could imagine a language centered on pattern matching request, helping the user to extract informations (such has "return every basic block that contains a call to foo()").
basic bloc chaining: using dominance
Hello, I am working on a plugin at the GIMPLE state, I am parsing basic blocks and I need to check that a call to foo() is only present once in a function. Howerver, it can be present several times if it is in different basic blocks and only one is executed at execution time. I think the most convenient way is to use dominance relation between the basic blocks: I can warn in a basic block A calling the foo() function only if there is a block B calling foo and dominating A. In others cases, I cannot be sure that there is several calls to foo(). In the file gcc/dominance.c, there is a dominated_by_p function which allows to test dominance between 2 basic blocks and I would like to use it to solve this problem. I would like to have your opinion, does it looks the google solution to the problem or is there another way to do this? If this is a good solution, I will implement a primitive in MELT allowing to use dominated_by_p function in MELT. Thanks! Pierre Vittet My Google Summer Of Code projet: http://www.google-melange.com/gsoc/project/google/gsoc2011/piervit/15001
Re: basic bloc chaining: using dominance
On 10/05/2011 16:23, Ian Lance Taylor wrote: Pierre Vittet writes: I am working on a plugin at the GIMPLE state, I am parsing basic blocks and I need to check that a call to foo() is only present once in a function. Howerver, it can be present several times if it is in different basic blocks and only one is executed at execution time. I think the most convenient way is to use dominance relation between the basic blocks: I can warn in a basic block A calling the foo() function only if there is a block B calling foo and dominating A. In others cases, I cannot be sure that there is several calls to foo(). In the file gcc/dominance.c, there is a dominated_by_p function which allows to test dominance between 2 basic blocks and I would like to use it to solve this problem. I would like to have your opinion, does it looks the google solution to the problem or is there another way to do this? If this is a good solution, I will implement a primitive in MELT allowing to use dominated_by_p function in MELT. Yes, dominated_by_p and friends are the right way to test for basic block dominance. Note that you have to build the graph first; see uses of calculate_dominance_info. Ian First, thanks for your help. I have looked at several function using calculate_dominance_info(). From what I understand, when you have finish to use it, you have to clear the structure by making a free_dominance_info(). In the function flow_loops_find (file gcc/cfgloop.c), there is a call to calculate_dominance_info() without call to free_dominance_info(). I feel it is a bug, no?
using plugin and lto: problem linking c-pragma symbol
Hello, I try to use the plugin pragma-plugin.c which is given in the testsuite (gcc/testsuite/g++.dg/plugin/pragma_plugin.c), I have compiled it as a shared library. If I try it on a simple c file it works, however, if I use -flto it causes an error: gcc -fplugin=./pragma_plugin.so -flto test.c lto1: error: cannot load plugin ../pragma_plugin.so ../pragma_plugin.so: undefined symbol: c_register_pragma lto-wrapper: /usr/bin/gcc returned 1 exit status /usr/lib/gcc/i486-slackware-linux/4.7.0/../../../../i486-slackware-linux/bin/ld: lto-wrapper failed collect2: error: ld returned 1 exit status I don't know much about LTO but I guess it is important to be able to use langage specific function (such as c_register_pragma) in plugin even when using lto. I have this need in others plugins (I would like to have pragma support in MELT). Maybe a solution could be to define c_register_pragma and others related functions as weak references (however it mights not works for all plateform). What would be the better way to patch this? Pierre Vittet
new wiki page: MELT performance
Hello, I have written a new page on the wiki (http://gcc.gnu.org/wiki/MELTPerformanceTest) about the performance of GCC using a MELT plugin. This can be usefull to see what take too much time in MELT (it looks like we spend a lot of time in the garbage collector, this might be highly improved). This can also help to follow the evolution of MELT in term of performance. I will also make available the plugin that I have used for the test, in order, to make new tests on the same basis. If you have any recommendation on others possibles test or on the way to run and report them, mail me! thanks
Re: GSOC - Student Roundup
Hi, I am Pierre Vittet, one of the GSOC students. I am writing a plugin which is a simple statical analysis tools. The idea is to write some tests (like testing that a function call is tested to return somethings (not) null, or testing that a call to a given function is followed by a call to another function.), that can be parametrized by the user (to be run on the functions he wants to check). I feel it can be a useful tool (however I am ready to fight with false positives). I am using MELT (http://gcc-melt.org/) for this, I know that is quite a hot topic in the community and it is not much considered. MELT aims at abstracting GCC internal with a higher level language. I think that it can be a good way to enter into GCC, because it permits to parse quite quikly the GCC internal, gimple especially. The major feature of MELT is his pattern matching on tree ability. Part of my work is to improve MELT (as I need some features which were not implemented), such as having handling pragma in MELT plugins or gluing some GCC dominance functions (from gcc/dominance.c) However, like the others students which answered to this thread, the main difficulty is understanding and eventually modifying GCC itself. That is quite time consumming, as you can have, even to write a simple patch, to know well several parts of GCC (I am often grepping for a given struct or function). I also thing that each pass could be more detailled. Sometimes, it is hard to find a pass in the code from it's dump file name. I had to write a patch for the trunk, to add a data field to pragma handlers, this was quite a simple patch, however, I had to make it compatible with the fact that one of the function I was modifying was used many times in differents parts of GCC. Currently, I need to understand how works C++ mangling if I want my plugin to work with C++ (I am going to write a new mail about this), as I need to convert the string given by the user into the function(s) that it represents. About the state of my plugin, I have written the core of the module. It can use user's input from different entries (I have already written an entrie using the pragma, and an entrie using direct argument (given when invoking GCC), searchs if it matchs a corresponding test and generate a new GCC Pass from this. This new pass is inserted after ssa pass, as I got a representation which suits to me (I can find the information my tests need). But the choice of inserting here is quite empirical (would have you choice another pass?). The plugin is mainly tested for C code, even if I would like to have C++ working too. I have started implementing tests, for the moment I use a test which checks that there is a test checking for a NULL pointer (if resCall == NULL for exemple) after a given function call, and return a warning if there is no test. I have also started implementing a test checking that a given is immediatly followed by another given function. I would also try to add a test checking that a call to a given function is followed by a call to another function in the same function bodies, I know that I will have to handle the basic block hierarchie, that why I glued dominance functions into MELT. I think, I am in time, however, this is sometimes hard to evaluate: for exemple, I don't know really how much time, I will take to manage C++ function in my plugin. I feel sad, not being able to use IRC from my web access (in my school)... and so only communicating by mail with the GCC community. However, I thanks the community for their answer and support. The point to improve, I guess, is having more people reviewing patchs (even if, I understand that it requires a lot of time and skills). Thanks Pierre Vittet On 05/07/2011 21:08, Daniel Carrera wrote: Hello Philip + Dimitrios Thanks for your posts. I am another GSOC student. I am working on the Fortran front-end of GCC (gfortran). Like most GFortran developers, my background is more in the natural sciences (astrophysics in my case) rather than computer science. My project is to help add coarray support for GFortran. Coarrays are a cool new feature in the Fortran 2008 standard that give Fortran native support for parallel programming (as opposed to using MPI or OpenMP). Scientific simulations (especially in parallel clusters) are the bread and butter of Fortran. Learning how GCC works has been difficult but rewarding. My work is split between the libgfortran library, which is straight forward enough, and the Fortran front-end, which at first looked like gobbledygook. Today I can actually understand most front-end functions when I read them, but it took a long time to get to this point. GCC has a scary learning curve. The libgfortran library is more forgiving for the beginner. We are using MPI as the backend to implement coarrays and I have very much enjoyed learning MPI. So far my contributions have been modest, but now th
C++ mangling, function name to mangled name (or tree)
Hello, I am working on a plugin which permits simple static analysis. I would like my plugin to work with C++ but I have to take in account the C++ mangling. I would like user of the plugin to give in arguments the name of the functions on which he would like a test to be run. That means that I must convert the string containing a function name (like "myclass::init") and get either the mangled name or the tree corresponding to the function. I know that there might be several results (functions with the same name and different arguments), a good policy for me would be to recover every concerned functions (at least for the moment). I guess what I want to do is possible, because there are already some tools doing it (like gdb). Also, when I use fdump-ipa-all, I get something interesting in the .visibility file: int Application::test(int)/1 @0xb6cff6b4 (asm: _ZN11Application4testEi) availability:available analyzed reachable body externally_visible finalized called by: int main()/2 (1.00 per call) (can throw external) calls: int printf(const char*, ...)/4 (1.00 per call) (can throw external) We get both the "real name" and the mangled one (_ZN11Application4testEi). But I don't know where I can find such translating. I have looked at cp/mangle.c, which has some interesting function, for exemple giving the mangle named from the corresponding tree, but in this case, how can I get the tree? If you have any idea, thanks! Cheers, Pierre Vittet
Re: announce: MELT plugin 0.8rc2 for 4.6
I got a bug if we use mawk instead of gawk however it is not the bug of Allan, as it just end the script early (before starting compiling). I have also been able to compile the current branch with a 32 bits system (I can't compile the plugin because my 32 bits system has no gcc 4.6). So it looks to appear only on 32 bits when compiling the plugin. Pierre Vittet
Re: IPA and LTO
Hello, If local_function_and_variable_visibility was not a simple IPA pass it would not have been called once per file but once per function (as it is with GIMPLE pass). I feel this is normal that this pass is run 2 times because it is run before any link operations. However, I don't know exactly how and when ld is called and which passes run after this. Pierre Vittet On 13/07/2011 17:54, AJM-2 wrote: What you say is in line with my understanding, however when I instrument the execute function of ipa-function-and-variable-visibility (local_function_and_variable_visibility()) I note that: gcc -flto a.c b.c causes the pass to be called twice (presumably once per file). If I split the compilation into two stages, then in the link stage gcc -flto a.o b.o the pass is never called. Conversely, the gate of IPA-Points-to does seem to be called three times at link time (presumably once for each file and then once for all together). I cannot discover the cause of the different behaviours here.
Re: C++ mangling, function name to mangled name (or tree)
Hello, sorry to answer that late (I didn't saw your mail in my mailbox + I was preparing me for RMLL/Libre software meeting). Your solution looks to be a nice one, I am goiing to try it and I will post the result of my experiment. I was not aware of that hook. Thanks! Pierre Vittet Hello, Have you considered the reverse way to do that. I mean, why don't you hook on the PLUGIN_PRE_GENERICIZE event to catch all function bodies, and then compare the argument the user gave you to current_function_name() (that will returns you the full protoype of the current function, ie: malloc full name is "void* malloc(size_t)"). Then, you can store the FUNCTION_DECL tree if there's a match and use it for later processing. That's how i proceed for my plugins. Romain Geissler
Re: C++ mangling, function name to mangled name (or tree)
On 14/07/2011 12:42, Romain Geissler wrote: At that time i didn't know you were working on Melt, and for now the few things i know about it is that it's mainly abut hooking the pass manager (or am i wrong ?) So all those useful events like PLUGIN_PRE_GENERICIZE or PLUGIN_FINISH_TYPE don't seems to be catchable through the Melt API (again, i'm maybe wrong, but if not it should be easy to add that feature to Melt). Yeah, first use of MELT is to add new passes, however we have some others hooks, for exemple I already added a hook to use pragma in MELT. I am going to add a hook for PLUGIN_PRE_GENERICIZE in the MELT API. I saw your mail about pushing PLUGIN_FINISH_TYPE hook in the trunk, that will be very usefull for both MELT and plugins in a general sense. I have seen you were correcting things in the MELT build system, that is not easy I think Is it working ? Take me (and Basile) informed. From my system, it looks melt-sources directory is not correctly installed, moreover, I am not sure that meltgc_make_load_melt_module (in melt-runtime.c) search correctly this source (for exemple I can't understand why we try to find them in temporary directory). Basile Starynkevitch does not work this week (normally), however, I am sure, he will help us when he came back. Pierre Vittet
write after approval status
Hello, I might need to commit on the MELT branch in the next weeks, however Basile Starynkevitch which is the maintainer for this branch will not be able to commit my patchs as he will be on holliday. So, I request a Write after approval status, as from what I have understood I need this status even to have the ability to commit in a GCC branch different from the trunk. If it is possible (but I guess it is not) I could simply have a write approval limited to the MELT branch. If I get write after approval status, It will not change my way to send patch to the trunk, as I need to get a positif answer from a reviewer before sending it. From, the MELT branch, Basile agrees to allow me to commit patch, he will also be able to review my change when he comes back and to revert if it is really necessary. This ability to commit on the MELT branch is quite important for me, in order to complete my GSOC project. Here are some links to the patchs I already proposed (and were accepted): [PATCH] c-pragma: adding a data field to pragma_handler: http://gcc.gnu.org/ml/gcc-patches/2011-06/msg00067.html which was finally updated in: http://gcc.gnu.org/ml/gcc-patches/2011-06/msg00854.html [PATCH, MELT] add dominance functions http://gcc.gnu.org/ml/gcc-patches/2011-05/msg01306.html [PATCH, MELT] Add PRE_GENERICIZE callback support in MELT http://gcc.gnu.org/ml/gcc-patches/2011-07/msg01311.html [PATCH, MELT] correct meltgc_read_from_val without location http://gcc.gnu.org/ml/gcc-patches/2011-06/msg01888.html [PATCH, MELT] pragma support in MELT http://gcc.gnu.org/ml/gcc-patches/2011-06/msg01861.html [PATCH, MELT] fix useless forcing of GCC garbage collector http://gcc.gnu.org/ml/gcc-patches/2011-05/msg00648.html [PATCH, MELT] correcting path error in the Makefile.in: http://gcc.gnu.org/ml/gcc-patches/2011-05/msg01232.html Pierre Vittet
[GSOC] Customizable warnings with a GCC plugin
Hello,
As the GSOC ending is approching, I tried on a real project (GNU Grub)
the plugin that I am developping.
This plugin allows the user to add warnings when compiling, depending of
rules that he has previously written. Here is an exemple of possible rules :
(testNull "grub_malloc")
This means that when there is a call to function "grub_malloc", we will
check that it is immediately followed by a condition, testing that what
was returned by grub_malloc was (not) null.
(testNull "grub_malloc")
We also have operator testZero and test testNeg, allowing to work on
functions that return int.
We could also use such rule (event if it is quite unusual):
(testNull "function_that_modify_first_arg" 1)
It means that instead of testing that there is a condition on the result
of the function, we test that there is a test on the first argument of
the function.
Using this feature has more sense with the following type of test:
(testFollowedBy "grub_bufio_open" 0 "grub_bufio_close" 1)
This means that in a function body, if you have a call to the function
"grub_bufio_open", we will check that the returned variable (0) is later
tested to be first argument (1) of a call to grub_bufio_close.
There is also a testNotFollowedBy working the same way which emit a
warning if the first function is followed by the second one.
The last kind of test is "testImmediatlyFollowedBy" testing that a call
to a function is immediatly followed by a call to another one, for
exemple we might want to use :
(testImmediatlyFollowedBy "chroot" 1 "chdir" 1)
The plugin seems to return quite interesting warnings without to much
false positives. False positives comes from several particular
situations, for exemple for testFollowedBy, if you have someting like that:
int myFunction(void)
{
void * myPtr = first_function_to_be_tested();
if(something does not works)
{
return ERROR;
}
second_function(myPtr);
return 0;
}
It will return a warning as you can exit the function without running
second_function. The problem is that it is more a design problem than a
problem from the plugin.
If you want to see the detail of my result when running a few tests on
Grub, you can have a look on the grub mailing list :
https://lists.gnu.org/archive/html/grub-devel/2011-08/msg9.html .
I would be glad to have your opinion on my plugin. Do you thing in can
be really useful in real project ? Would you use it ? Have you idea of
tests that we could run over GCC or another project ? I guess this can
be a pretty good tools, but it needs to have clear coding standard in
order to know what should be tested.
For the moment, the plugin has only be tested on C code, this might
works with only few changes for C++. It needed the following patchs
http://www.mail-archive.com/gcc-patches@gcc.gnu.org/msg13086.html which
had been commited yesterday + some changes in MELT (the plugin is
developped in MELT). Moreover as the plugin uses some newly added
functionalities of MELT, it might be quite hard to use until the release
of MELT 0.9 (which might be ready for the end of August). I have named
the plugin Talpo, it means mole in esperanto, the idea is that we dig
blindly into GCC finding the information we want, the project is
available here : https://gitorious.org/talpo. I will focus on
documentation and help next week.
Thanks for your interest!
Pierre Vittet
Re: [GSOC] Customizable warnings with a GCC plugin
Hello,
thanks for you mail. You are right my plugin returns a 'false' warning
on your case. That should be possible to handle this case but it needs
some search: when parsing func,we should request to look at the Guard
constructor (either by inserting a new pass or maybe by using LTO (but I
should look deeper at this)) to check if there is a test or no.
For the moment my plugin's 'view' is limited to a single function but
that should be something to improve.
Pierre Vittet
On 12/08/2011 18:00, Jonathan Wakely wrote:
> On 12 August 2011 15:54, Pierre Vittet wrote:
>>
>> For the moment, the plugin has only be tested on C code, this might
>> works with only few changes for C++.
>
> Would your example tests for grub warn about the following C++ code?
>
> struct Guard {
> Guard(void* p) : p(p) { if (!p) throw std::bad_alloc(); }
> ~Guard() { grub_free(p); }
> void* p;
> };
>
> void func(grub_size_t n)
> {
> Guard g(grub_malloc(n));
> // do something with g.p
> }
>
> For it to be useful for C++ it would be necessary to not warn about that code.
>
[GSOC] code contribution + documentation
Hello, As GSOC is approching it's end, I would like to get precision of how project's result should be made available. I open a quite general topic however I guess it heavily depends of each project. In particular, if I made some contributions to GCC and MELT, the main part of my project is a plugin. For now it is hosted on a gitorious repository (https://gitorious.org/talpo). As it uses MELT, it will be integrated into MELT branch, as an example tools (I am going to see this with Basile Starynkevitch next week). However I would like to know if it makes sense to have documentation and/or link to Talpo available on the GCC wiki? I guess there should be a link at least in the page referencing existing plugins, I will add it. At the beginning of the GSOC, I created a wiki page about the project as I wanted visibility: http://gcc.gnu.org/wiki/CustomizableWarningPlugin. It is rather outdated now, I should use it as a complete documentation or delete it. What make sense ? I know plugins are often considered external, on the others side this could carries useful information about what a plugin can do, how I used GIMPLE representation... (it might be useful to GCC newcomers and people interested by the plugin). Thanks! Pierre Vittet PS: I you want to know more about my plugin: http://gcc.gnu.org/ml/gcc/2011-08/msg00251.html https://gitorious.org/talpo/talpo/blobs/master/README
Re: role of dump_file notably for/in plugins?
Hello, >From what I understand, the question is not really how do we dump passes but what a pass (from a plugin or not) should dump? So the question is what do you expect from a dump file? Do you expect to have the full internal representation of the code after the end of the pass? For many dump files, we just get this (lower,cfg, ssa...). Or do you expect something more oriented on what the pass did? Like "this pass remove three unreable basicblocks, they were located here, here and here"? Maybe there is no particular rules and it just depends of the pass. However I guess there should some clear rules about this. Dump files can be quite useful to understand the compiler behavior (sometimes it is a huge task to search using the debugger). Thanks. Pierre Vittet
misbehaviour with md5_process_bytes and maybe in optimization
Hello, I recently asked for some help as I got a problem when using md5_process_bytes (in libiberty/md5.c): http://gcc.gnu.org/ml/gcc-help/2011-09/msg00126.html, http://gcc.gnu.org/ml/gcc-help/2011-09/msg00127.html and it appears that there is a bug in md5_process_bytes. The bug can conduct to a miscomputed md5 result. It tooks time to me to make the bug reproducible but I was finally able to do so. The fact is that it only appears in very particular situation. I have written a small gcc plugin, allowing to reproduce it (see attachment). The bad news is that the bug only appears when use libiberty compiled in -g -O0 (it works well with -O2). It is quite sad, because It could means another bug in an optimization function. I have attached a README which detail how to use the plugin and how to explain the bug. I have tried to explain as good as possible (and I apologize for my very bad english). The bug appears when: 1) We use libiberty compiled with -O0 2) We first call md5_process_bytes with a less than 64 bits buffer (we call his size len1). 3) We make a new call of md5_process_bytes with a buffer which has a size len2 such as: len2 > 127 + 65 (so test in line 228 of md5.C will be true) 128 -len1 != Mulint with Mulint % __alignof__ (md5_uint32) != 0 (so condition on line 238 is true) len2 - (128 - len1) = Mul64 and Mul64 such as Mul %64=0 (so the loop of line 239 is broken with len = 64, this leads to the bug as, line 249, (len & ~63) = 64 and we shift the buffer without processing the data). Please, can you reproduce the bug? Is there any useful informations I can add? Must I contact somebody from libiberty (I don't know the status of this library (is this part of gcc or from another project?)). I already sent a patch correcting this issue (it does not correct the fact that we don't get the bug with an optimised libiberty): http://gcc.gnu.org/ml/gcc-patches/2011-09/msg01098.html. It has not been reviewed, could someone reviews this? Thanks! Pierre Vittet md5sum_plugin.tar.gz Description: application/gzip
Re: misbehaviour with md5_process_bytes and maybe in optimization
Thanks for your interest, I just checked revision 179127 of GCC. Last revision is 177700, it has not been change for 6 weeks. My file is the same as this one: http://gcc.gnu.org/viewcvs/trunk/libiberty/md5.c?revision=177700&view=markup in libiberty/md5.c, function md5_process_bytes start line 203. On 23/09/2011 17:13, Ian Lance Taylor wrote: > Pierre Vittet writes: > >> The bug appears when: >> 1) We use libiberty compiled with -O0 >> 2) We first call md5_process_bytes with a less than 64 bits buffer (we >> call his size len1). >> 3) We make a new call of md5_process_bytes with a buffer which has a >> size len2 such as: >> len2 > 127 + 65 (so test in line 228 of md5.C will be true) line 228 is the following:if (len > 64) >> 128 -len1 != Mulint with Mulint % __alignof__ (md5_uint32) != 0 (so >> condition on line 238 is true) line 238 is the following: if (UNALIGNED_P (buffer)) >> len2 - (128 - len1) = Mul64 and Mul64 such as Mul %64=0 (so the loop of >> line 239 is broken with len = 64, this leads to the bug as, line 249, >> (len & ~63) = 64 and we shift the buffer without processing the data). line 239 is the following: while (len > 64) line 249: buffer = (const void *) ((const char *) buffer + (len & ~63)); > > The line numbers you mention do not correspond to any version of > libiberty/md5.c that I can see. Can you list the exact line for each > line number you mention, so that your explanation is easier to follow? > Thanks. I give about the same explanation in the README (which is in the attached archive of my previous mail) but I does not use line number but direct quote of the code. It mights be more easy to try the plugin with gdb but it needs to compile libiberty.a with -O0. > > Ian >
