Hi Cloudberry Dev Team, We currently use Ubuntu 22.04 (Jammy Jellyfish) as the host operating system for our Cloudberry instances.
We would like to request to add a build step for Jammy, so that we can ensure that it builds successfully on our platform. We can provide the necessary Docker files and testing scripts ourselves. Additionally, we have some free compute resources available in our cloud https://yandex.cloud/en/services/compute , which could be used for your build and testing process. This would be at no cost. We use Terraform for our infrastructure, although there may be some differences between it and AWS. However, I believe that it would not be difficult to migrate from AWS to YC. On Thu, Nov 14, 2024 at 12:09 PM Ed Espino <esp...@apache.org> wrote: > Hi Cloudberry Dev Team, > > I wanted to provide some additional important details about the CI > implementation that I forgot to mention in my previous email. > > The CI build process will also: > > - Create a development RPM package for el9 (Rocky Linux 9) that will be > available for download from the workflow page > - Use this RPM in the testing phase > - Utilize a minimal testing container to validate that: > - The RPM correctly pulls in all required runtime dependencies > - Package installation works as expected > - installcheck can execute successfully in a clean environment > > This minimal testing container approach helps ensure that we're properly > capturing all runtime dependencies and that the package will work correctly > in production environments. > > A significant infrastructure change is that we plan to execute these > workflows on GitHub-hosted runners instead of the previously used > self-hosted AWS VMs. This shift will simplify our infrastructure > requirements and maintenance overhead. > > Regarding test suite execution, there are some important considerations: > > Currently Disabled Tests: > > - cbdb_parallel > - instr_in_shmem_verify > > These test suites are currently commented out as they're experiencing > execution issues in the GitHub-hosted runner environment. We'll need to > review their implementation to determine what modifications are needed to > make them compatible. > > As we move forward with the CI implementation, we anticipate we may > encounter flaky tests. To maintain development velocity, we'll need to > prioritize addressing any test stability issues quickly when they arise. > This will be crucial for maintaining an efficient development workflow for > our community. > > Best regards, > -=e > > On Wed, Nov 13, 2024 at 11:43 PM Ed Espino <esp...@apache.org> wrote: > > > Hi Cloudberry Dev Team, > > > > I'd like to discuss a series of planned Pull Requests aimed at > > implementing Continuous Integration (CI) support for Apache Cloudberry. > The > > changes will be introduced incrementally across multiple repositories to > > establish a robust CI pipeline. > > > > The implementation will come in two main phases: > > > > 1. Docker Container Infrastructure (cloudberry-devops-release) > > - Introducing Docker containers for building and testing Cloudberry > > on Rocky Linux 9 > > - Publishing build artifacts to DockerHub > > - Adding GitHub workflows to automate container building process > > - Repository: https://github.com/apache/cloudberry-devops-release > > 2. CI Pipeline Implementation (cloudberry main repository) > > - Implementing CI workflows for building and testing on Rocky Linux > > 9 > > - Initial focus on unittest-check and installcheck-good (with > > Optimizer disabled) > > - Adding new GitHub workflow configurations > > - Introducing supporting CI scripts from cloudberry-devops-release > > - Repository: https://github.com/apache/cloudberry > > > > As part of this effort, we will also: > > > > - Clean up legacy build scripts to reduce technical debt > > - Evaluate moving the build scripts back to the main cloudberry > > repository > > > > The CI implementation will be actively developed over the next few months > > to support our initial release processes. We'll be using simplified CI > > scripts initially, with plans to expand functionality as we progress. > > > > The current separation between devops-release and main repository allows > > us to: > > > > - Maintain clear separation of concerns > > - Iterate on CI infrastructure independently > > - Reduce complexity in the main repository > > - Enable easier maintenance and updates of CI components > > > > I welcome any feedback or suggestions on this approach before I begin > > submitting the PRs. > > > > Best regards, > > -=e > > -- > > Ed Espino > > Apache Cloudberry (incubating) & MADlib > > > > > -- > Ed Espino > Apache Cloudberry (incubating) & MADlib >
