Hi again, It seems that I've bumped into the first barrier, since in order to run the tests, I'd need permissions to create branches. Do you think that would be possible? Or for that I'd need to be a comitter?
/Mario. ________________________________ From: Mario Salazar de Torres <mario.salazar.de.tor...@est.tech> Sent: Wednesday, October 12, 2022 9:08 PM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: https://concourse.apachegeode-ci.info/ is shutting down Hi, Yes, I was already aware of Apache Jenkins instance. However, GH actions provides more flexibility for the developer. So, considering nobody has any argument against it, I'll start to make some tests today with GH actions. If GHA doesn't work out, we can consider for sure other choices like Jenkins. /Mario ________________________________ From: Dan Smith <dasm...@vmware.com.INVALID> Sent: Saturday, October 8, 2022 1:02 AM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: https://concourse.apachegeode-ci.info/ is shutting down If github actions don't pan out, we previously ran geode CI on the apache jenkins instance. According to the instructions I should be able to grant folks access if anyone wants to try the jenkins option - https://cwiki.apache.org/confluence/display/INFRA/Jenkins -Dan ________________________________ From: Mario Salazar de Torres <mario.salazar.de.tor...@est.tech> Sent: Friday, October 7, 2022 1:57 PM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: https://concourse.apachegeode-ci.info/ is shutting down ⚠ External Email Hi everyone, Following up on this, I looked into the view of the ASF regarding GitHub Actions and as they describe here: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FBUILDS%2FGitHub%2BActions%2Bstatus&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BLJeO6f3%2Fk92Vye%2BYHWRWA2MyCezbRsM6UoEHrojOh8%3D&reserved=0 among lots of places in the bu...@apache.org list, there are mainly 2 issues: * Per-Enterprise limitations. Currently GitHub is giving for free an Enterprise status for the Apache organization. And the thing is GitHub establishes a concurrent job limit per Enterprise, in this case 180 jobs, as can be seen here: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Factions%2Flearn-github-actions%2Fusage-limits-billing-and-administration%23usage-limits&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=glfd3pwp6afDPSbTRVzIuTkWH3MPYRIUsfAUR1SuJBM%3D&reserved=0. Also, you can see there is a disclaimer stating that this limit might be increased on demand, which TBH is something I don't know yet if happened. On the other side, apparently there is a quota for the amount of time executing GitHub Actions per month and organization (ref: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2F6lbqr0f6mqt9s8ggollp5kj2nv7rlo9s&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=wkXtuIAX6Y7fjENG875VnHRfPqzBj2kLFowxK1Jd6Fc%3D&reserved=0) so this is another downside. As an alternative, ASF is/has been in conversations with GitHub so per-repository quotas/tracking is set into place, but it seems this is not happening any time soon, as this will surely help ASF to keep assign credits to each project using GHA. * Security concerns. There are several concerning security concerns to which ASF propose several mitigations, which will require to change the way I wanted to approach the pipelines. Still, this part can be solved, and also, the biggest security concerns are related to self-hosted runners, but that's not going to be the case due to the lack of sponsors and also, btw, ASF is only approving the use of them for an specific set of projects and only under ASF's support and supervision. [https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.githubassets.com%2Fimages%2Fmodules%2Fopen_graph%2Fgithub-logo.png&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ODE79SLkU2tSmqxauJnG3MJ%2BRgx%2B5wueYmD3%2FJKXCVk%3D&reserved=0]<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Factions%2Flearn-github-actions%2Fusage-limits-billing-and-administration%23usage-limits&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=glfd3pwp6afDPSbTRVzIuTkWH3MPYRIUsfAUR1SuJBM%3D&reserved=0> Usage limits, billing, and administration - GitHub Docs<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.github.com%2Fen%2Factions%2Flearn-github-actions%2Fusage-limits-billing-and-administration%23usage-limits&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=glfd3pwp6afDPSbTRVzIuTkWH3MPYRIUsfAUR1SuJBM%3D&reserved=0> Job matrix - A job matrix can generate a maximum of 256 jobs per workflow run. This limit applies to both GitHub-hosted and self-hosted runners. Workflow run queue - No more than 500 workflow runs can be queued in a 10 second interval per repository. If a workflow run reaches this limit, the workflow run is terminated and fails to complete. docs.github.com Worth mentioning that Jarek Potiuk, from the Airflow project, is one of the ASF members which are pushing for GitHub Actions to improve so that ASF projects can use it, so it seems Airflow is the best project for me to take a look at. It's also worth mentioning that Jarek recently published the usage stats for GitHub Actions inside the ASF (https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fv52q29pcr9ky0r3omlt3s1gy66osf9lz&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=0Y7SrQqNCfkbfinoNeFa9DXmk06bGe8QqiiCUb4orLM%3D&reserved=0), and even taking into account its limitations, it seems that there is a growing trend, so I guess as time comes by more and more ASF projects will adopt GA as their CI solution. Still, I think, considering we don't have a CI in place right now, it would be a good idea to run some tests and see how much time it takes for the pipelines to run using ASF GH Enterprise infra. /Mario ________________________________ From: Michael Oleske <mole...@vmware.com.INVALID> Sent: Friday, October 7, 2022 7:56 PM To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconcourse.apachegeode-ci.info%2F&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4mFx0p7luTOec1pkrDKQf9%2BmlkRONCB%2FDBUP%2BeIy8DI%3D&reserved=0 is shutting down Apache Hawq<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhawq%2Factions&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=7G0OuyONexRMtfqbB0THZ84twuZeUsR5bEhVcGF9pos%3D&reserved=0> uses github actions, and I suspect many others use it as well (from a quick search of the Apache github org<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsearch%3Fl%3DYAML%26q%3Dorg%253Aapache%2Bworkflows%26type%3DCode&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4i7dDSJu%2FQapmumzCiYNxchC6jo5DDsAkXLzoWNMF%2BY%3D&reserved=0>). So I suspect you won't have a difficult time with asf infrastructure -michael ________________________________ From: Anthony Baker <bak...@vmware.com.INVALID> Sent: Friday, October 7, 2022 08:53 To: dev@geode.apache.org <dev@geode.apache.org> Subject: Re: https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconcourse.apachegeode-ci.info%2F&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4mFx0p7luTOec1pkrDKQf9%2BmlkRONCB%2FDBUP%2BeIy8DI%3D&reserved=0 is shutting down ⚠ External Email Sounds like good idea to me! You might check with ASF Infra to see their views on GF actions. If you can find another project that has already done this, it’s a lot easier. Anthony > On Oct 7, 2022, at 8:24 AM, Mario Salazar de Torres > <mario.salazar.de.tor...@est.tech> wrote: > > ⚠ External Email > > Hi everyone, > > Regarding this, some time ago Michael Oleske and myself were looking into > implementing GitHub Actions as the de-facto CI solution for Geode Native. Now > Concourse CI is not available anymore, I'd like to propose GH actions as the > alternative, at least for Geode Native. > I have been preparing the pipelines and there are still some details to work > out, but it looks promising, you can check I.E this run: > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNordix%2Fgeode-native%2Factions%2Fruns%2F3176127664&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Wgwp%2Bf8a7wQtzN2SgQn7bmdIWz5eAhZpEWa3Ky%2BUxfk%3D&reserved=0 > > So, I wanted to ask what you think about this proposal. > My idea is if nobody presents any arguments against it by next Wednesday, > 12th of October, I'll start to work on it. > > Also, when you say ASG-hosted CIs, you mean like whole CI solutions right? Do > you happen to know if they provide hardware resources on its own? I mention > it because it would be ideal to connect GH actions to that hardware so > executions would take less time. > > /Mario > ________________________________ > From: Anthony Baker <bak...@vmware.com.INVALID> > Sent: Thursday, September 29, 2022 8:32 PM > To: dev@geode.apache.org <dev@geode.apache.org> > Subject: > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconcourse.apachegeode-ci.info%2F&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4mFx0p7luTOec1pkrDKQf9%2BmlkRONCB%2FDBUP%2BeIy8DI%3D&reserved=0 > is shutting down > > At 3pm PST today the > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconcourse.apachegeode-ci.info%2F&data=05%7C01%7Cdasmith%40vmware.com%7C303729bb7daa4c52ed0408daa8a68bcf%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C638007730578843543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4mFx0p7luTOec1pkrDKQf9%2BmlkRONCB%2FDBUP%2BeIy8DI%3D&reserved=0 > site will be shutting down. Apologies for the short notice, I just received > this information today. > > As discussed on other threads, there are ASF-hosted CI alternatives. I’m > happy to point any volunteers in the right direction. > > > Anthony > > > ________________________________ > > ⚠ External Email: This email originated from outside of the organization. Do > not click links or open attachments unless you recognize the sender.
