potiuk commented on issue #25088: URL: https://github.com/apache/datafusion/issues/25088#issuecomment-6375013688
> Yes, I see. We are feeling something similar in Arrow and DataFusion. DataFusion only has 1200 contributors, but recently we have seen a deluge of PRs. Each new PR (including AI dumps) consumes a lot of PR time on the Apache shared runners Oh absolutely. This is what I proposed, including the design and 5-stacked PR implementing weeks ago only to get some PMC members respond, "if it's about saving time to others we do not care". And since then I spent a lot of time convincing people and pushing my way through so that it gets accepted - to the point that I **just** sent lazy consensus result about it. You can track the whole discussion there: https://lists.apache.org/thread.html/krl1bzszlfgfz70m494rm8jq1ph6dr01 Unfortunately, during the last few days while I was waiting for the lazy consensus to pass, we were somewhat frequently nudged, added to the mailing list threads, pointed in slakc and some issues were created. That was quite distracting to our work because we constantly had to constantly respond to such nudges, despite my explicit explanation to the people nudging us that we were working on it. This led me to worry that objectors might break the lazy consensus, further delaying things. It's sometimes very unproductive to try to motivate motivated people by nudging them. Each of us has an example of the manager coming every hour and asking "Are we there yet"? That distracts people and de-focuses them as we all know. Bad idea in general, especially if you tell "Working on it". And yet we have not seen any apology for those distractions. Luckily the consensus was not broken and we can proceed now. > left a comment (apache/datafusion#25088) > <https://github.com/apache/datafusion/issues/25088#issuecomment-6374466898> > > On arrow <https://github.com/apache/arrow> we limit non-collaborators to > 3 PR at a time. Will merge apache/arrow#55164 > <https://github.com/apache/arrow/pull/55164> once > apache/infrastructure-asfyaml#135 > <https://github.com/apache/infrastructure-asfyaml/pull/135> is complete. > Oh absolutely again - I have proposed (or rather my Magpie skill that I created for that very purpose) in https://lists.apache.org/thread.html/6bgpcflsqow9b9b3r022yyf135m63352 - that you could replace your custom pull_request_target workfow with .asfyaml feature - that I not only contributed so that others (not only us) could use it but also developed and contributed the whole testing framework to infra that infra or anyone else can use to test .ASF YAML changes: https://github.com/apache/infrastructure-asfyaml/pull/111 and https://github.com/apache/infrastructure-asfyaml/pull/133 - because my PR for that waited for weeks. And yes - we limited PRs to 5 in Airflow. Some of our contributors had 30 PRs, but I convinced others in Airflow and drove the change (including closing 200 PRs that were exceessive and taking the pain of it). If not that pre-emptive move, we would likely have 15x more traffic today. Luckily - we acted on that without any nudging, because we care. And yes it also required getting to consensus: You can see lazy consensus here - and earlier discussions I had to navigate: https://lists.apache.org/thread.html/901dt9qdqjtqrh7ow5cdnd8sp0dl3llp BTW. My (or rather my skill's proposal that I sent yesterday) might help Arrow to use 30% - 40% less of ASF resources - and rather quickly. It also proposes you to get rid of pull_request targets. You can also implement more optimizations by following the infra page I just got merged today when I described all kinds of optimizations with examples (and links to the Magpie SKILL that will help your agent to do those optimizations very quickly) : https://infra.apache.org/github-actions-optimization.html. Some of those optimizations are basic and simple - and the 30% - 40% optimisation is mostly driven by switching push "complete builds" to "scheduled" canary builds we use in Airflow for 2 years now - this is one single optimization that helps busy repos like ours a lot. And yes you **MUST** get rid of all pull_request_target workflow by the end of November as they will just **stop working** ASF infra will **not** enable it for the organisation. It's far too dangerous of a footgun.. You can also refer to another document I wrote - where I explained one of the ways you can replace pull_request_target with much safer workflow: https://infra.apache.org/github-actions-policy.html - see "Safer replacement for pull_request_target" and examples - again, taken from Magpie where I tested and perfected it. I am glad we are all working in the same direction. And I think it would benefit all of us if we share finding easler - and not in the form "You are using too much resources - fix it immediately" repeated 7 times in a day when we are pretty advanced in solving that very problem, documenting it for others to follow - and hack - even developing tools that others can use to make their life easier, BTW. I REALLY heartily invite everyone to use Magpie - the "repo-health" family of skills - which includes "workflow optimisation skill" - does all the analysis and prepares improvement proposals that you can ask the agent to do for you. HEARTILY RECOMMEND. J, — > Reply to this email directly, view it on GitHub > <https://github.com/apache/datafusion/issues/25088?email_source=notifications&email_token=AAERMI7KX53VPBTG6EYKNZD5VPABNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMMZXGQ2DMNRYHE4KM4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-6374466898>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAERMI645RHIZOVJ7QWE5TL5VPABNAVCNFSNUABFKJSXA33TNF2G64TZHMZTKOBZGE3TGMJYHNEXG43VMU5TKMZYHE2DEMBQGY4KC5QC> > . > You are receiving this because you were mentioned. > On Fri, Oct 23, 2026 at 11:01 PM Oleks V ***@***.***> wrote: > *comphead* left a comment (apache/datafusion#25088) > <https://github.com/apache/datafusion/issues/25088#issuecomment-6374643429> > > Thing that concerns me is > https://docs.github.com/en/actions/reference/security/securely-using-pull_request_target#default-policy-for-pull_request_target > > that disables pull_request_target even on public repos, which I believe > also applicable for contributor forks > > — > Reply to this email directly, view it on GitHub > <https://github.com/apache/datafusion/issues/25088?email_source=notifications&email_token=AAERMI4HECYIR7BYVY4MEWD5VPBIZA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMMZXGQ3DIMZUGI42M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#issuecomment-6374643429>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAERMI4FRWQ5YRLSCB5DQXL5VPBIZAVCNFSNUABFKJSXA33TNF2G64TZHMZTKOBZGE3TGMJYHNEXG43VMU5TKMZYHE2DEMBQGY4KC5QC> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
