Hi, @tromey: We have a question for you: the problem is detailed here and our question to you is at the bottom. Thanks!
So the line problem we are having in pph is much bigger then we originally thought. The problem is: As we've had issues with before: the chain of name bindings in any cpp_bindings is inserted at the head, such that when we stream it back in, the last one put on the chain by the parser is read first. This is a problem in the linemap case because when we read the last name, we also read it's input_location (line and column) and try to replay that location by calling linemap_line_start (so far so good....). This issue is that linemap_line_start EXPECTS to be called with lines in increasing orders (but since the bindings are backwards in the chain we actually call it with lines in a decreasing order), the logic in linemap_line_start is such that if the line# is less then the previous line# it was called with, it assumes this must be a new file and creates a new file entry in the line_table by calling linemap_add (which in our case is WRONG!!). >From a comment found in libcpp/line-map.c, linemaps clearly assume this fact: "Since the set (line_table) is built chronologically, the logical lines are monotonic increasing, and so the list is sorted and we can use a binary search." @crowl: Maybe this is the issue with resume_scope?? I'm just thinking that since it seems lookups are fairly tied to the line_table, and the fact that it should be built in a monotonically increasing order. In the past we have solved similar issues (caused by the backwards ordering), by replaying whatever it was in the correct order (before doing anything else), and then simply loading references using the pph_cache when the real ones were needed. BUT we can't do this with source_locations as they are a simple typedef unsigned int, not actual structs which are passed by pointer value... Potential solution? : I thought of a few ways to solve this: 1) Catch the calls to the linemap functions when generating the pph, store them with the pph and replay them first before anything else when reading the pph file. The problem with that is that to when trying to obtain the source location for a given binding streamed in: after calling linemap_line_start when needed, we call linemap_position_for_column which ASSUMES that we are currently on the last line which linemap_line_start was called with (and thus doing the full replay first would not really work here, is there another function we can call that can take a line as an argument as well as a column??) 2) We could potentially output a reference to the source_location and use that to do what we always do using the cache in this sort of situation, however, we would need to figure out exactly which integer we need to output a reference of (since they are passed around by value...). This would be kinda hacky, and would also imply building some hackery to go around the fact that the lto version of the input location doesn't need to use references (i.e. we would need a streamer hook, but two different functions signatures...). I don't think this is a nice way of going about it, but could be an option... @tromey: I hear you are the person in the know when it comes down to linemaps, do you have any hint on how we could rebuild a valid line_table given we are obtaining the bindings from the last one in the file to the first one (that is, using pph (pre-parsed headers) where we are trying to restore a cached version of the parser state for a header)? Thanks, Gabriel
