Re: [D] [Proposal] Upgrade the PostgreSQL version from 14 to 16 [cloudberry]
GitHub user oppenheimer01 added a comment to the discussion: [Proposal] Upgrade the PostgreSQL version from 14 to 16 Greate idea ! I think GitHub Projects is a good tool for PostgreSQL version upgrade work. A new project named 'Merge postgres 16' has been created for traking the upgrade work. https://github.com/orgs/apache/projects/497/views/1 GitHub link: https://github.com/apache/cloudberry/discussions/1095#discussioncomment-13128832 This is an automatically sent email for dev@cloudberry.apache.org. To unsubscribe, please send an email to: dev-unsubscr...@cloudberry.apache.org - To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
Re: Discussion: Change PAX Plugin to Be Disabled by Default?
Thanks for repeated discussions and feedback. More people still prefer not to compile PAX by default, so we do not compile PAX by default. Once again thank you all. Best regards, Max Yang On Sat, May 10, 2025 at 11:15 PM Jianghua Yang wrote: > For our current use case, I’d like to minimize the Pax build to reduce > dependencies and avoid unnecessary components. Specifically, I’ve disabled > the following CMake options: > option(USE_MANIFEST_API "Use manifest API" OFF) > option(USE_PAX_CATALOG "Use manifest API, by pax impl" OFF) > option(BUILD_GTEST "Build with Google Test" OFF) > option(BUILD_GBENCH "Build with Google Benchmark" OFF) > option(BUILD_TOOLS "Build with Pax tools" OFF) > > This configuration allows us to build only the core modules without pulling > in submodules or extra third-party libraries. Please let me know if any > additional flags should be adjusted for this lightweight setup. > > Leonid Borchuk 于2025年5月10日周六 07:25写道： > > > Hi, all > > > > I really like the PostgreSQL approach - configure && make && make > install. > > And usually there are no additional packages or builds required. > Postgresql > > seems to be compiled everywhere - even on coffee machine. It would be > great > > to see the same for cloudberry. Since PAX is quite complex feature it > would > > be better to have a special option --enable-pax. > > > > I believe we will continue to create RPM and DEB packages with all > features > > enabled, so users can take precompiled binaries and start using PAX from > > scratch. > > > > But for those who want to contribute, they can easily compile their first > > binary and then enable features step by step. > > > > On Fri, May 9, 2025 at 11:05 AM Dianjin Wang > > wrote: > > > > > Hi Jiaqi and all, > > > > > > Thanks for your reply and for sharing your insights. I’d like to > > > respond with a few thoughts from a community and release readiness > > > perspective. > > > > > > 1. Lack of transparency in introducing the PAX > > > > > > While I fully respect the engineering efforts behind PAX, we must also > > > reflect on how the feature was introduced. There was no public design > > > proposal or open discussion prior to its contribution. The PR was open > > > for a few weeks, but it was too large to review effectively. The goals > > > behind PAX might be well-understood within certain internal teams, but > > > not yet widely discussed in the Cloudberry community. At that time, we > > > rushed to merge it into the main branch to catch up with the release - > > > I also gave my +1 to the related PRs. > > > > > > This is a valuable lesson for all of us — especially for large, > > > impactful features, we need more transparency and community > > > discussion, just like the ongoing one around perfmon[1]. I'm not meant > > > to blame anyone, it's one part of the story of incubation. > > > > > > The key point is that the default enabling behaviour of PAX is also > > > against the description in the doc[2]. This is why we have this > > > discussion thread. I have created one PR on setting the PAX to be > > > disabled by default[3]. Compared to the thousands of file changes in > > > PAX, mine is too small, however, we have such a deep conversation > > > here, quite interesting and proud. > > > > > > 2. PAX is enabled by default is beyond the users' expectations for now > > > > > > The concern is not whether PAX is valuable — it’s about how users > > > perceive and experience it. > > > > > > Having PAX enabled by default breaks the general expectation from most > > > users familiar with Greenplum or PostgreSQL. We must admit that most > > > of Cloudberry users will be ones who migrate from the open-source > > > Greenplum. This difference can lead to: > > > > > > * Confusing build failures > > > * Extra burden on users' Cloudberry installation > > > * Frustration during first-time installation, which may cause them to > > > give up on Cloudberry entirely > > > > > > Already saw the failure happening from the PPMC members, I bet it will > > > be the same for the general users, we can imagine that. > > > > > > Our goal is to help PAX mature through community feedback. The best > > > way is to make it opt-in — let users choose to enable it, experiment > > > with it, and report issues, rather than forcing it upon them. > > > > > > 3. Let’s give users the power to choose > > > > > > Instead of enabling PAX by default, we can: > > > > > > * Provide clear instructions on how to enable it > > > * Share blog posts and documentation explaining its advantages and > > roadmap > > > * Present it at community events (meetups, talks, etc.) > > > > > > This gives users the opportunity to understand what PAX is, why it > > > matters, and decide whether it’s right for them. `--enable-pax` is > > > meaningfully different from `--disable-pax`; the former has less > > > burden on users' operations. > > > > > > Eventually, if the community widely adopts PAX and achieves consensus, > > > we can consider enab
Re: [D] [Proposal] Upgrade the PostgreSQL version from 14 to 16 [cloudberry]
GitHub user edespino added a comment to the discussion: [Proposal] Upgrade the PostgreSQL version from 14 to 16 **Thanks for initiating this important proposal. I fully support the goal of aligning Apache Cloudberry with upstream PostgreSQL enhancements. That said, there are several foundational areas that need clarification and community discussion to ensure the upgrade is successful, maintainable, and inclusive of all stakeholders.** --- ### 🚩 Release Lifecycle Policy: Unclear The proposal asserts: > "Apache Cloudberry 2.0 will remain supported beyond PostgreSQL 14’s EOL." > "Both 2.0 and 3.0 are planned for long-term support." However, the project has **not yet published or discussed a formal release lifetime policy**. This makes it difficult for users, downstream integrators, and commercial adopters to plan around version stability and upgrade timelines. Key open questions include: * What does “long-term support” mean in practice? * Will Apache Cloudberry versions follow PostgreSQL’s \~5-year support window? * Will maintenance continue for EOL'd upstream kernels via backports or security patches? 📌 **Recommendation:** A **minimal versioning and support policy** should be drafted and published. This can be done by **any contributor, committer, or PPMC member**, and does not need to be authored by the original proposal writer. Even a placeholder policy would help clarify expectations. Here’s a suggested draft format to seed discussion: 📄 Example: Minimal Versioning and Support Policy ```markdown ## Apache Cloudberry Versioning and Support Policy (DRAFT) Apache Cloudberry follows a major.minor.patch format: - Major: Kernel upgrades or breaking changes (e.g., PG14 → PG16) - Minor: Backward-compatible feature additions - Patch: Bug and security fixes | Version | PG Base | Status | Maintenance Target | |-|-|--|--| | 2.0.x | PG14| Active | Through at least 2027¹ | | 3.0.x | PG16| Planned | PG16 EOL + 1 year (tentative)| ¹ PG14 EOL is in 2026; Cloudberry 2.0 will continue receiving critical fixes beyond that date. Upgrades will follow standard PostgreSQL tooling (e.g., `pg_upgrade`) where feasible. Changes to this policy will be proposed on the dev@ mailing list. ``` --- ### 🔄 Upgrade Path for Users: Not Addressed There is no mention of how users are expected to upgrade from Apache Cloudberry 2.0 (PG14) to 3.0 (PG16). Key questions: * Will in-place upgrades using `pg_upgrade` be supported? * Will logical dump/restore be required? * Are any major compatibility caveats expected (e.g., catalog changes, extension rewrites)? 📌 **Recommendation:** Outline the expected upgrade path or link to a follow-up issue/mailing list thread where this will be clarified. --- ### 🌿 Development Workflow and Branching: Needs Clarification At the time of this proposal, Apache Cloudberry 2.0 has not yet been officially released. As such: * The `main` branch is currently tracking work for the 2.0 release. * A `REL_2_STABLE` branch will be created when 2.0 is cut. * The PG16 upgrade work is **expected to follow immediately on `main`**. This raises an important question: > **If PG16 work begins directly on `main`, and this work is long-running, how > will that impact other contributors and feature development?** > Will it effectively "hold `main` hostage" for months, making it difficult to > keep it in a releasable state? 📌 **Recommendation:** * Consider doing PG16 upgrade work in a **dedicated feature branch** (e.g., `pg16-integration`) to keep `main` clean and releasable. * Allow post-2.0 fixes and small enhancements to continue on `main`. * Establish a merge/rebase plan for when PG16 work is production-ready. Keeping `main` stable supports CI, release planning, and encourages ongoing community engagement outside the upgrade track. --- ### 🧪 Test Plan: Needs Clarification While the proposal estimates \~3 months for regression fixes, it doesn’t specify: * Whether parallel testing against PG14 and PG16 will be done * If test framework changes will be required * What success criteria define a “complete” upgrade 📌 **Recommendation:** Expand the proposal to include: * Testing scope across extensions and planner/executor behavior * Integration with CI for continuous validation * Clear thresholds for test suite compliance --- ### 🧩 Extension and Component Coordination: Missing Apache Cloudberry includes many in-core components and bundled extensions (e.g., PXF, gpfdist, PL languages, workload manager). A PostgreSQL upgrade will almost certainly affect: * Internal APIs * Plan node compatibility * System catalog behavior 📌 **Recommendation:** * List all bundled extensions and components expected in 3.x. * Assign or identify maintainers for each. * Track PG16 compatibility status (via GitHub issues or a shared audit doc). --- ### 🌐 Downstream Ecosystem Impact: Private and Commercial Com
Re: [D] [Proposal] Upgrade the PostgreSQL version from 14 to 16 [cloudberry]
GitHub user oppenheimer01 added a comment to the discussion: [Proposal] Upgrade the PostgreSQL version from 14 to 16 Cloudberry 2.0 will maintain support regardless of PostgreSQL 14s end-of-life (EOL). Both versions 2.0 and 3.0 are committed to long-term support cycles. GitHub link: https://github.com/apache/cloudberry/discussions/1095#discussioncomment-13128587 This is an automatically sent email for dev@cloudberry.apache.org. To unsubscribe, please send an email to: dev-unsubscr...@cloudberry.apache.org - To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
Re: [D] [Ideas] Submodule Pinning [cloudberry]
GitHub user tuhaihe added a comment to the discussion: [Ideas] Submodule Pinning I recently did a test following Ed's suggestions from PR #1084 comments. It worked, as I can track versions more effectively on my test branch: https://github.com/tuhaihe/cloudberrydb/commits/submodule-update05/, which also incorporates the commit SHA as a reliable reference. I also took a look at the manual on `.gitmodules`, where I found a helpful example for upgrading a submodule version, similar to Ed's suggestion: https://git-scm.com/docs/gitsubmodules#_workflow_for_a_third_party_library. By using this method, we can avoid updating the `.gitmodules` file manually. Of course, we can specify the branch in the `.gitmodules` file as well like your way! Here needs a tradeoff. Anyway, let's move forward. GitHub link: https://github.com/apache/cloudberry/discussions/1083#discussioncomment-13127819 This is an automatically sent email for dev@cloudberry.apache.org. To unsubscribe, please send an email to: dev-unsubscr...@cloudberry.apache.org - To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
Re: Discussion: Change PAX Plugin to Be Disabled by Default?
Hi, Leonid On 2025/05/10 14:23:29 Leonid Borchuk wrote: > Hi, all > > I really like the PostgreSQL approach - configure && make && make install. Me too. > And usually there are no additional packages or builds required. Postgresql > seems to be compiled everywhere - even on coffee machine. It would be great > to see the same for cloudberry. Since PAX is quite complex feature it would > be better to have a special option --enable-pax. Agree. Unlike AO, PAX, as a contrib module, I think we should avoid adding --enable-pax or --disable-pax flags entirely. Instead, we should treat it as a standard PostgreSQL contrib module. That means: It gets built(compiled and installed) only if users explicitly install it from its directory and follows the usual extension workflow (CREATE EXTENSION) Extensions should follow the rule of Postgres, users who need PAX can enable it the same way they would any other extension. - To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
Re: Discussion: Change PAX Plugin to Be Disabled by Default?
Hi, Leonid On 2025/05/10 14:23:29 Leonid Borchuk wrote: > Hi, all > > I really like the PostgreSQL approach - configure && make && make install. Me too. > And usually there are no additional packages or builds required. Postgresql > seems to be compiled everywhere - even on coffee machine. It would be great > to see the same for cloudberry. Since PAX is quite complex feature it would > be better to have a special option --enable-pax. Agree. Unlike AO, PAX, as a contrib module, I think we should avoid adding --enable-pax or --disable-pax flags entirely. Instead, we should treat it as a standard PostgreSQL contrib module. That means: It gets built(compiled and installed) only if users explicitly install it from its directory and follows the usual extension workflow (CREATE EXTENSION) Extensions should follow the rule of Postgres, users who need PAX can enable it the same way they would any other extension. - To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
Re: Discussion: Change PAX Plugin to Be Disabled by Default?
Hi, Leonid On 2025/05/10 14:23:29 Leonid Borchuk wrote: > Hi, all > > I really like the PostgreSQL approach - configure && make && make install. Me too. > And usually there are no additional packages or builds required. Postgresql > seems to be compiled everywhere - even on coffee machine. It would be great > to see the same for cloudberry. Since PAX is quite complex feature it would > be better to have a special option --enable-pax. Agree. Unlike AO, PAX, as a contrib module, I think we should avoid adding --enable-pax or --disable-pax flags entirely. Instead, we should treat it as a standard PostgreSQL contrib module. That means: It gets built(compiled and installed) only if users explicitly install it from its directory and follows the usual extension workflow (CREATE EXTENSION) Extensions should follow the rule of Postgres, users who need PAX can enable it the same way they would any other extension. - To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
Re: Discussion: Change PAX Plugin to Be Disabled by Default?
Sorry for the duplicate emails—must have been a network glitch. On 2025/05/14 03:34:34 Zhang Mingli wrote: > > Hi, Leonid > > On 2025/05/10 14:23:29 Leonid Borchuk wrote: > > Hi, all > > > > I really like the PostgreSQL approach - configure && make && make install. > > Me too. > > > And usually there are no additional packages or builds required. Postgresql > > seems to be compiled everywhere - even on coffee machine. It would be great > > to see the same for cloudberry. Since PAX is quite complex feature it would > > be better to have a special option --enable-pax. > > Agree. > > Unlike AO, PAX, as a contrib module, I think we should avoid adding > --enable-pax or --disable-pax flags entirely. > > Instead, we should treat it as a standard PostgreSQL contrib module. That > means: > It gets built(compiled and installed) only if users explicitly install it > from its directory and follows the usual extension workflow (CREATE EXTENSION) > > Extensions should follow the rule of Postgres, users who need PAX can enable > it the same way they would any other extension. > > - > To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org > For additional commands, e-mail: dev-h...@cloudberry.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
Re: Discussion: Change PAX Plugin to Be Disabled by Default?
Will it require a major refactoring effort to install the PAX using the EXTENSION method? We are close to the new release; if so, hope we can evolve in the future release. Best, Dianjin Wang On Wed, May 14, 2025 at 11:34 AM Zhang Mingli wrote: > > > Hi, Leonid > > On 2025/05/10 14:23:29 Leonid Borchuk wrote: > > Hi, all > > > > I really like the PostgreSQL approach - configure && make && make install. > > Me too. > > > And usually there are no additional packages or builds required. Postgresql > > seems to be compiled everywhere - even on coffee machine. It would be great > > to see the same for cloudberry. Since PAX is quite complex feature it would > > be better to have a special option --enable-pax. > > Agree. > > Unlike AO, PAX, as a contrib module, I think we should avoid adding > --enable-pax or --disable-pax flags entirely. > > Instead, we should treat it as a standard PostgreSQL contrib module. That > means: > It gets built(compiled and installed) only if users explicitly install it > from its directory and follows the usual extension workflow (CREATE EXTENSION) > > Extensions should follow the rule of Postgres, users who need PAX can enable > it the same way they would any other extension. > > - > To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org > For additional commands, e-mail: dev-h...@cloudberry.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@cloudberry.apache.org For additional commands, e-mail: dev-h...@cloudberry.apache.org
