[lldb-dev] Reporting bugs which only affect (semi-proprietary) downstream consumers.

[Adam HARRIES via lldb-dev]

Hi all,

I've recently taken over maintenance of my company's llvm+lldb branch,
where we have added support for our in-house architecture (in llvm) as well
as support for debugging through both hardware and our simulator. Our llvm
fork is public/open source, however many of our runtime libraries and
drivers (which are linked into lldb, clang, etc, and provide built-ins and
driver support etc) are not.

While attempting to update our branch from llvm-11 to llvm-12 we came
across a commit[1] in lldb which quite reliably causes a deadlock when we
launch a process to debug a core dump. Luckily, said commit simply modifies
some concurrency primitives, and reverting it is sufficient to fix the bug
without any further effects. We are quite confident that the commit is the
issue, as we performed a thorough bisect which maintained "our" code
unchanged throughout.

Unfortunately, however, we are unable to reproduce this bug in any "open"
architectures (such as x86-64, AArch64, etc), so are not entirely sure how
we should go about reporting the bug. Additionally, it makes it difficult
to open a discussion regarding whether the commit is correct (and thus we
may need to modify our additions to lldb to match new implicit behaviour),
as third parties may be unable to reproduce the issue. Finally, as the bug
results in a deadlock (which requires a sigkill to end) we won't (as I
understand it) be able to use a "Reproducer" to demonstrate the bug to
third parties.

Although we are able to "solve" the issue locally (by reverting the
commit), we feel that the better solution would be to feed back our
findings to the community and solve the issue, rather than (privately)
sweeping it under the rug. As components of our compiler are proprietary,
however, this process becomes difficult due to the reasons listed above.

To summarise, there are two main questions that I feel unable to answer:
- Is there an existing process for reporting bugs that only affect third
parties, and which cannot be reproduced in "core" targets.
- To what extend is it possible to discuss (or report) bugs "on faith" - as
in without any concrete evidence that a third party can reproduce.

We are currently looking into opening up our build process so that we are
able to distribute binary libraries to enable third parties to build our
compiler + debugger, but as this is currently a work-in-progress it is
unfortunately not a solution to this issue.

Many thanks in advance for any and all advice.
Yours,

-- 
*Adam Brouwers-Harries*
Compiler Engineer
aharr...@upmem.com

[1] Please note, I have specifically not named this commit as I wish to
better understand the "meta"-bug filing process, and I do not wish to
publicly assign blame for any bugs without understanding how and why I can
do so respectfully and properly.
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Reporting bugs which only affect (semi-proprietary) downstream consumers.

[Raphael “Teemperor” Isemann via lldb-dev]

Hi Adam,

I think the best idea is to comment on the commit on Phabricator ( 
reviews.llvm.org ) as it seems to be a relatively recent change. Otherwise if 
you can somehow provide way to reproduce the deadlock using only code you can 
share + LLVM.org sources then filing a bug would be an option too.

Regarding what information you should provide: Pretty much everything that you 
can share would help. At least the backtrace of all threads in the deadlocked 
state would be good to know. And of course the commit your bisect stopped at if 
it's a bug report. From there people might have an idea how to reproduce the 
issue in a unit test or via the SB API (or what could be going wrong in your 
downstream fork).

And I believe you can't use the reproducer feature here as that requires having 
the respective LLDB binary to replay (which you probably can't share).

- Raphael

> On 22 Jun 2021, at 19:10, Adam HARRIES via lldb-dev  
> wrote:
> 
> Hi all, 
> 
> I've recently taken over maintenance of my company's llvm+lldb branch, where 
> we have added support for our in-house architecture (in llvm) as well as 
> support for debugging through both hardware and our simulator. Our llvm fork 
> is public/open source, however many of our runtime libraries and drivers 
> (which are linked into lldb, clang, etc, and provide built-ins and driver 
> support etc) are not. 
> 
> While attempting to update our branch from llvm-11 to llvm-12 we came across 
> a commit[1] in lldb which quite reliably causes a deadlock when we launch a 
> process to debug a core dump. Luckily, said commit simply modifies some 
> concurrency primitives, and reverting it is sufficient to fix the bug without 
> any further effects. We are quite confident that the commit is the issue, as 
> we performed a thorough bisect which maintained "our" code unchanged 
> throughout.
> 
> Unfortunately, however, we are unable to reproduce this bug in any "open" 
> architectures (such as x86-64, AArch64, etc), so are not entirely sure how we 
> should go about reporting the bug. Additionally, it makes it difficult to 
> open a discussion regarding whether the commit is correct (and thus we may 
> need to modify our additions to lldb to match new implicit behaviour), as 
> third parties may be unable to reproduce the issue. Finally, as the bug 
> results in a deadlock (which requires a sigkill to end) we won't (as I 
> understand it) be able to use a "Reproducer" to demonstrate the bug to third 
> parties. 
> 
> Although we are able to "solve" the issue locally (by reverting the commit), 
> we feel that the better solution would be to feed back our findings to the 
> community and solve the issue, rather than (privately) sweeping it under the 
> rug. As components of our compiler are proprietary, however, this process 
> becomes difficult due to the reasons listed above.
> 
> To summarise, there are two main questions that I feel unable to answer: 
> - Is there an existing process for reporting bugs that only affect third 
> parties, and which cannot be reproduced in "core" targets. 
> - To what extend is it possible to discuss (or report) bugs "on faith" - as 
> in without any concrete evidence that a third party can reproduce.
> 
> We are currently looking into opening up our build process so that we are 
> able to distribute binary libraries to enable third parties to build our 
> compiler + debugger, but as this is currently a work-in-progress it is 
> unfortunately not a solution to this issue. 
> 
> Many thanks in advance for any and all advice.
> Yours,
> 
> -- 
> Adam Brouwers-Harries
> Compiler Engineer
> aharr...@upmem.com 
> 
> [1] Please note, I have specifically not named this commit as I wish to 
> better understand the "meta"-bug filing process, and I do not wish to 
> publicly assign blame for any bugs without understanding how and why I can do 
> so respectfully and properly.
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] Reporting bugs which only affect (semi-proprietary) downstream consumers.

[Greg Clayton via lldb-dev]

> On Jun 22, 2021, at 10:10 AM, Adam HARRIES via lldb-dev 
>  wrote:
> 
> Hi all, 
> 
> I've recently taken over maintenance of my company's llvm+lldb branch, where 
> we have added support for our in-house architecture (in llvm) as well as 
> support for debugging through both hardware and our simulator. Our llvm fork 
> is public/open source, however many of our runtime libraries and drivers 
> (which are linked into lldb, clang, etc, and provide built-ins and driver 
> support etc) are not. 
> 
> While attempting to update our branch from llvm-11 to llvm-12 we came across 
> a commit[1] in lldb which quite reliably causes a deadlock when we launch a 
> process to debug a core dump. Luckily, said commit simply modifies some 
> concurrency primitives, and reverting it is sufficient to fix the bug without 
> any further effects. We are quite confident that the commit is the issue, as 
> we performed a thorough bisect which maintained "our" code unchanged 
> throughout.
> 
> Unfortunately, however, we are unable to reproduce this bug in any "open" 
> architectures (such as x86-64, AArch64, etc), so are not entirely sure how we 
> should go about reporting the bug. Additionally, it makes it difficult to 
> open a discussion regarding whether the commit is correct (and thus we may 
> need to modify our additions to lldb to match new implicit behaviour), as 
> third parties may be unable to reproduce the issue. Finally, as the bug 
> results in a deadlock (which requires a sigkill to end) we won't (as I 
> understand it) be able to use a "Reproducer" to demonstrate the bug to third 
> parties. 
> 
> Although we are able to "solve" the issue locally (by reverting the commit), 
> we feel that the better solution would be to feed back our findings to the 
> community and solve the issue, rather than (privately) sweeping it under the 
> rug. As components of our compiler are proprietary, however, this process 
> becomes difficult due to the reasons listed above.
> 
> To summarise, there are two main questions that I feel unable to answer: 
> - Is there an existing process for reporting bugs that only affect third 
> parties, and which cannot be reproduced in "core" targets. 

I don't believe there is a formal process for this. Though I would suggest just 
submitting a bug and attaching stack traces of your deadlock. Loading a core 
file is very similar across all targets, so I can't imagine this being hard to 
reproduce with another core file? Is there something special about your core 
file or setup? I know that logging used to be able to cause deadlocks due to 
the Module::GetDescription(...) that tried to take the module lock. It no 
longer does this on top of tree. 
> - To what extend is it possible to discuss (or report) bugs "on faith" - as 
> in without any concrete evidence that a third party can reproduce.
> 
> We are currently looking into opening up our build process so that we are 
> able to distribute binary libraries to enable third parties to build our 
> compiler + debugger, but as this is currently a work-in-progress it is 
> unfortunately not a solution to this issue. 
> 
> Many thanks in advance for any and all advice.
> Yours,

I would go ahead and debug the deadlock, attach repro steps for how you are 
loading your core file (exact commands or APIs that are being used) and then 
maybe attach the output "bt all" so we can see all of the threads and see what 
is deadlocking your LLDB.

Greg

> 
> -- 
> Adam Brouwers-Harries
> Compiler Engineer
> aharr...@upmem.com 
> 
> [1] Please note, I have specifically not named this commit as I wish to 
> better understand the "meta"-bug filing process, and I do not wish to 
> publicly assign blame for any bugs without understanding how and why I can do 
> so respectfully and properly.
> ___
> lldb-dev mailing list
> lldb-dev@lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] [cfe-dev] [llvm-dev] Mailing List Status Update

[James Y Knight via lldb-dev]

On Mon, Jun 21, 2021 at 3:53 PM Chris Lattner via cfe-dev <
cfe-...@lists.llvm.org> wrote:

> On Jun 9, 2021, at 10:50 AM, Philip Reames via llvm-dev <
> llvm-...@lists.llvm.org> wrote:
>
> Specific to the dev lists, I'm very hesitant about moving from mailing
> lists to discourse.  Why?
>
> Well, the first and most basic is I'm worried about having core
> infrastructure out of our own control.  For all their problems, mailing
> lists are widely supported, there are many vendors/contractors available.
> For discourse, as far as I can tell, there's one vendor.  It's very much a
> take it or leave it situation.  The ability to preserve discussion archives
> through a transition away from discourse someday concerns me.  I regularly
> and routinely need to dig back through llvm-dev threads which are years
> old.  I've also recently had some severely negative customer experiences
> with other tools (most recently discord), and the thought of having my
> employability and ability to contribute to open source tied to my ability
> to get a response from customer service teams at some third party vendor I
> have no leverage with, bluntly, scares me.
>
> Second, I feel that we've overstated the difficulty of maintaining mailing
> lists.  I have to acknowledge that I have little first hand experience
> administering mailman, so maybe I'm way off here.
>
> Hi Philip,
>
> First, despite the similar names, Discord is very different than
> Discourse.  Here I’m only commenting about Discourse, I have no opinion
> about Discord.
>
>
> In this case, I think we need to highly weight the opinions of the people
> actively mainlining the existing systems.  It has become clear that the
> priority isn’t “control our own lists”, it is “make sure they stay up” and
> “get LLVM people out of maintaining them”.
>
> The ongoing load of maintaining these lists (including moderation) and of
> dealing with the security issues that keep coming up are carried by several
> individuals, not by the entire community.  I’m concerned about those
> individuals, but I’m also more broadly concerned about *any* individuals
> being solely responsible for LLVM infra.  Effectively every case we’ve had
> where an individual has driving LLVM infra turns out to be a problem.  LLVM
> as a project isn’t good at running web scale infra, but we highly depend on
> it.
>

I agree that the maintenance issue is definitely a problem which needs to
be solved. And there is some urgency, given the recent problems which
resulted in a need to manually subscribe people to the lists.

But, the proposal on the table doesn't appear to actually address this
issue, because the maintainers of llvm mailman will still continue to be
responsible for keeping it functioning, for the mailing lists which were
not proposed to be migrated. On the other hand, having osci.io run a
mailman3 service for us does seem to be a way to solve this -- and doesn't
require discarding mailing lists entirely.

It seems clear to me that we should outsource this to a proven vendor.
> Your concerns about discourse seem very similar to the discussion about
> moving to Github (being a single vendor who was once much smaller than
> Microsoft).  I think your concerns are best addressed by having the IWG
> propose an answer to “what is our plan if Discourse-the-company goes
> sideways?"
>
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

[lldb-dev] [Bug 39166] Better error messages when ptrace() fails on linux

[via lldb-dev]

https://bugs.llvm.org/show_bug.cgi?id=39166

Apoorv Sachan  changed:

   What|Removed |Added

   Assignee|lldb-dev@lists.llvm.org |apoorv.99.sac...@gmail.com

-- 
You are receiving this mail because:
You are the assignee for the bug.___
lldb-dev mailing list
lldb-dev@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev