Re: [D] Reduce test duplication between core and PAX storage? [cloudberry]

2025-07-17 Thread via GitHub
GitHub user avamingli edited a comment on the discussion: Reduce test duplication between core and PAX storage? +1 for this, one pain point is that when we search for a test SQL in the code, it appears multiple times, including instances in both Pax and SingleNode test, most of them are copi

Re: [D] Reduce test duplication between core and PAX storage? [cloudberry]

2025-07-17 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Reduce test duplication between core and PAX storage? +1 for this, one pain point is that when we search for a test statement in the code, it appears multiple times, including instances in both Pax and SingleNode test, most of them are

Re: [D] [Ideas] Remove Bundled Python Packages in Cloudberry Main Repo [cloudberry]

2025-07-16 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: [Ideas] Remove Bundled Python Packages in Cloudberry Main Repo Hi, These files are pointed out by the Incubator PMC members and are not encouraged to be included in our upcoming Cloudberry 2.0 source files. So it's a good time to remove

Re: [D] Support Parallel Processing of Window Functions. [cloudberry]

2025-07-16 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Support Parallel Processing of Window Functions. @leborchuk Thank you so much for sharing your insightful observations and experiences. I will make the necessary adjustments to ensure our tests are robust. A preliminary test indicates

Re: [D] Support Parallel Processing of Window Functions. [cloudberry]

2025-07-16 Thread via GitHub
GitHub user leborchuk added a comment to the discussion: Support Parallel Processing of Window Functions. > During develop, I found that the result of Window Agg without Order By clause > is unstable. > > Referring to the SQL 2011 standard, it states that if ORDER BY is omitted, > the order

Re: [D] Support Parallel Processing of Window Functions. [cloudberry]

2025-07-16 Thread via GitHub
GitHub user jianlirong added a comment to the discussion: Support Parallel Processing of Window Functions. In my personal opinion, we should modify the corresponding SQL statement and add ORDER BY. Although we're discussing window functions here, the issue reflected in this example is essenti

Re: [D] Support Parallel Processing of Window Functions. [cloudberry]

2025-07-16 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Support Parallel Processing of Window Functions. > To pass parallel test cases, we need to modify the SQL in that case. Postgres discussion: https://www.postgresql.org/message-id/flat/fbb5c0d7-4a96-4dd1-9a26-5dfccfac667a%40Spark AFAK,

Re: [D] Support Parallel Processing of Window Functions. [cloudberry]

2025-07-15 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Support Parallel Processing of Window Functions. During develop, I found that the result of Window Agg without Order By clause is unstable. Referring to the SQL 2011 standard, it states that if ORDER BY is omitted, the order of rows in

Re: [D] Add cloudberry-mcp-server for integration with LLM apps [cloudberry]

2025-07-13 Thread via GitHub
GitHub user yjhjstz added a comment to the discussion: Add cloudberry-mcp-server for integration with LLM apps refer to `https://github.com/HenkDz/postgresql-mcp-server`, most mcp written by TypeScript, so to be cloudberry extension, kind of weird. GitHub link: https://github.com/apache/clou

Re: [D] Add cloudberry-mcp-server for integration with LLM apps [cloudberry]

2025-07-11 Thread via GitHub
GitHub user reshke added a comment to the discussion: Add cloudberry-mcp-server for integration with LLM apps I dont see why this cannot be an extension actually GitHub link: https://github.com/apache/cloudberry/discussions/1068#discussioncomment-13734926 This is an automatically sent

Re: [D] Add cloudberry-mcp-server for integration with LLM apps [cloudberry]

2025-07-11 Thread via GitHub
GitHub user yjhjstz added a comment to the discussion: Add cloudberry-mcp-server for integration with LLM apps Maybe a single alone repo is better . GitHub link: https://github.com/apache/cloudberry/discussions/1068#discussioncomment-13732754 This is an automatically sent email for dev@

Re: [D] Add cloudberry-mcp-server for integration with LLM apps [cloudberry]

2025-07-10 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: Add cloudberry-mcp-server for integration with LLM apps Hey, how's the progress on this proposal? Since developing it as an extension with PGRX or C/C++ is somewhat challenging, we can instead build this feature on the FastMCP foundation,

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-09 Thread via GitHub
GitHub user leborchuk added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion +1 for duckDB. But have one question, are there any security reasons we should be aware of for using duckDB data sources? (For examp

Re: [D] Make UNION Parallel [cloudberry]

2025-07-08 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Make UNION Parallel > Specifically, we can implement a Parallel-oblivious Append approach, where > multiple workers operate independently without sharing state, rather than a > Parallel-aware Append that requires coordination among worke

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-08 Thread via GitHub
GitHub user yjhjstz added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion I’d like to propose enhancing CloudBerry by integrating DuckDB, leveraging its [Data Sources](https://duckdb.org/docs/stable/data/data_

Re: [D] Support Parallel Processing of Window Functions. [cloudberry]

2025-07-07 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Support Parallel Processing of Window Functions. Yes, and the focus is more on optimizer changes, specifically leveraging Motion to redistribute data based on the Partition By clauses of window functions. Once the plan is established, e

Re: [D] Add Support for the diskquota extension in Apache Cloudberry [cloudberry]

2025-07-07 Thread via GitHub
GitHub user my-ship-it added a comment to the discussion: Add Support for the diskquota extension in Apache Cloudberry Hi @nicko858, do you still have problems or resolved by yourself? Feel free to share building results if you need any help form the community. GitHub link: https://github.co

Re: [D] [Ideas] Build some like greenplum_fdw for GPDB or ADB-to-ADB connector for Arenadata, for Cloudberry to run queries between one and more CBDB clusters [cloudberry]

2025-07-07 Thread via GitHub
GitHub user my-ship-it added a comment to the discussion: [Ideas] Build some like greenplum_fdw for GPDB or ADB-to-ADB connector for Arenadata, for Cloudberry to run queries between one and more CBDB clusters Thanks for working on it. Could you please share design documents here so that more

Re: [D] Support Parallel Processing of Window Functions. [cloudberry]

2025-07-07 Thread via GitHub
GitHub user my-ship-it added a comment to the discussion: Support Parallel Processing of Window Functions. Leverage parallel approach of MPP to implent executor parallelism seems to be a viable path. In fact, because Cloudberry data has distribution information, we can implement executor par

Re: [D] Support Parallel Processing of Window Functions. [cloudberry]

2025-07-07 Thread via GitHub
GitHub user cnliuchong added a comment to the discussion: Support Parallel Processing of Window Functions. When large amounts of data, the window function is performance low. If parallel execution is possible, the performance will be greatly improved. GitHub link: https://github.com/apache/c

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-04 Thread via GitHub
GitHub user jianlirong added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion It appears that data import and export is indeed a frequently used and frequently discussed functionality in MPP databases. Based o

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-04 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion With `transform`, you could do anything to data before it's inserted into the database. For example, it can work as an independent t

Re: [D] Shall we use Golang to manage the cluster? [cloudberry]

2025-07-03 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Shall we use Golang to manage the cluster? Hi @boogabee Thank you so much for sharing the backstory behind this decision—it truly brings back memories of our time at Pivotal. Those were such formative years, working alongside talented

Re: [D] Shall we use Golang to manage the cluster? [cloudberry]

2025-07-03 Thread via GitHub
GitHub user boogabee added a comment to the discussion: Shall we use Golang to manage the cluster? There was going to be a move to golang for the greenplum utilities, and there had been a significant amount of work put into that effort. The issue in front of us at the time was lots of conflic

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-03 Thread via GitHub
GitHub user ZTE-EBASE added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion Based on the current implementation, the gpfdist end can add format conversion logic to transform columnar storage formats into nati

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-03 Thread via GitHub
GitHub user ZTE-EBASE added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion As mentioned above, the enhanced features focus on addressing the issue of gpfdist being co-located with files and also supporting

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-02 Thread via GitHub
GitHub user gfphoenix78 added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion The CUSTOM format assumes row-based data stream. I'm not sure it's suitable for column-base storage format, needs more spike. Git

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-02 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion Indeed, it's a good perspective. I had forgotten about this approach; it seems promising and definitely worth a try. GitHub link:

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-02 Thread via GitHub
GitHub user gfphoenix78 added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion The discussion seems to support more protocols for external tables, not multiple data sources for a single external table. To be c

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-02 Thread via GitHub
GitHub user ZTE-EBASE added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion Yes, our implementation relies on libssh along with the arrow/parquet libraries. This approach is tailored to specific business requ

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-02 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion Seems need to implement HDFS/SFTP client codes in gpfdist, then why not directly use FDW? GitHub link: https://github.com/apache/

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-01 Thread via GitHub
GitHub user ZTE-EBASE edited a comment on the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion Minimize kernel code changes by reusing the gpfdist protocol. Add an sftp/hdfs protocol marker and use it to call the corresponding

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-01 Thread via GitHub
GitHub user ZTE-EBASE added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion Minimize kernel code changes by reusing the gpfdist protocol. Add an sftp/hdfs protocol marker and use it to call the corresponding

Re: [D] Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion [cloudberry]

2025-07-01 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Extend the gpfdist tool to support SFTP/HDFS protocols for high-performance multi-source data ingestion Hi, interesting! As far as I know, gpfdist is a protocol for external tables, and the gpfdist tool is an implementation of that pro

Re: [D] [Proposal] Introducing PL/Container to Cloudberry – Feedback Requested [cloudberry]

2025-07-01 Thread via GitHub
GitHub user jianlirong added a comment to the discussion: [Proposal] Introducing PL/Container to Cloudberry – Feedback Requested This is a very useful feature and an excellent start. By inheriting from PostgreSQL and Greenplum Database, Cloudberry has had very comprehensive UDF support from d

Re: [D] Make UNION Parallel [cloudberry]

2025-06-30 Thread via GitHub
GitHub user avamingli edited a discussion: Make UNION Parallel ### Description In PostgreSQL, the UNION operator can leverage parallel processing through the Parallel Append node with subnodes. ```sql explain (costs off, verbose) select * from t1 union select * from t2; Q

Re: [D] [Proposal] Introducing PL/Container to Cloudberry – Feedback Requested [cloudberry]

2025-06-30 Thread via GitHub
GitHub user yjhjstz edited a discussion: [Proposal] Introducing PL/Container to Cloudberry – Feedback Requested ### Proposers @yjhjstz @my-ship-it ### Proposal Status Under Discussion ### Abstract Hi everyone, We’re proposing to bring [PL/Container](https://github.com/greenplum-db/plco

Re: [D] Cloudberry Database Roadmap 2024 [cloudberry]

2025-06-25 Thread via GitHub
GitHub user jianlirong added a comment to the discussion: Cloudberry Database Roadmap 2024 What do you mean by "Native"? We are working on a FDW named datalake_fdw, which will support the functionalities of access to Iceberg tables(including read and write). That FDW will be open sourced soon

Re: [D] Cloudberry Database Roadmap 2024 [cloudberry]

2025-06-25 Thread via GitHub
GitHub user yjhjstz added a comment to the discussion: Cloudberry Database Roadmap 2024 We can refer to paper https://dl.acm.org/doi/pdf/10.1145/3722212.3724459 from databricks, make Iceberg as first class table like heap table that called "Native". GitHub link: https://github.com/apache/cl

Re: [D] Cloudberry Database Roadmap 2024 [cloudberry]

2025-06-25 Thread via GitHub
GitHub user yjhjstz added a comment to the discussion: Cloudberry Database Roadmap 2024 I found this https://www.crunchydata.com/blog/crunchy-data-warehouse-postgres-with-iceberg-for-high-performance-analytics to prove my standpoint. GitHub link: https://github.com/apache/cloudberry/discuss

Re: [D] Cloudberry Database Roadmap 2024 [cloudberry]

2025-06-24 Thread via GitHub
GitHub user tuhaihe edited a discussion: Cloudberry Database Roadmap 2024 > [!note] > Hi everyone, since Cloudberry has joined the ASF Incubator, the new roadmap > for Cloudberry is now up for discussion. We welcome your feedback: > https://lists.apache.org/thread/wo1lsly8h7q8c4sxrwpw73ylbnzrc

Re: [D] Upgrade from Cloudberry 1.6.0 to 2.0.0 [cloudberry]

2025-06-19 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: Upgrade from Cloudberry 1.6.0 to 2.0.0 Hi @xanadu-dev, the PPMC member @edespino replied on the Dev mailing list - https://lists.apache.org/thread/1r06ykd8vf0sl2qbxm7rr97lxk448k6p, FYI. Welcome to continue this conversation there. GitHub

Re: [D] Upgrade from Cloudberry 1.6.0 to 2.0.0 [cloudberry]

2025-06-19 Thread via GitHub
GitHub user xanadu-dev added a comment to the discussion: Upgrade from Cloudberry 1.6.0 to 2.0.0 Would this also been needed for further releases or will everything maintain compatibility after apache-cloudberry:2.0.0 ? GitHub link: https://github.com/apache/cloudberry/discussions/1175#discu

Re: [D] Upgrade from Cloudberry 1.6.0 to 2.0.0 [cloudberry]

2025-06-19 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: Upgrade from Cloudberry 1.6.0 to 2.0.0 Hi, for the upgrade, you can try to use cbcopy: https://cloudberry.apache.org/docs/sys-admin/migration-and-upgrade GitHub link: https://github.com/apache/cloudberry/discussions/1175#discussioncommen

Re: [D] Cloudberry Database Roadmap 2024 [cloudberry]

2025-06-19 Thread via GitHub
GitHub user sobolevna added a comment to the discussion: Cloudberry Database Roadmap 2024 Hello! What type of integration with Iceberg do you mean? Native, PXF, some other methods? And are you planning to integrate with Paimon tables? GitHub link: https://github.com/apache/cloudberry/discus

Re: [D] Proposal: Enable Parallel DQA Plans (with streaming hash agg) [cloudberry]

2025-06-19 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Proposal: Enable Parallel DQA Plans (with streaming hash agg) Done in https://github.com/apache/cloudberry/pull/1173 GitHub link: https://github.com/apache/cloudberry/discussions/914#discussioncomment-13518642 This is an automatic

Re: [D] [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? [cloudberry]

2025-06-12 Thread via GitHub
GitHub user avamingli edited a comment on the discussion: [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? In GPDB/CBDB, if a feature affects ORCA, its GUC typically follows the format optimizer_xxx. Else, most GUCs do not guarantee ORCA will use the feature by

Re: [D] [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? [cloudberry]

2025-06-12 Thread via GitHub
GitHub user avamingli added a comment to the discussion: [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? In GPDB/CBDB, if a feature affects ORCA, its GUC typically follows the format optimizer_xxx. Else, most GUC does not guarantee ORCA will use the feature by

Re: [D] [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? [cloudberry]

2025-06-12 Thread via GitHub
GitHub user yjhjstz added a comment to the discussion: [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? Orca planner does not support answer_query_using_materialized_views yet. So change answer_query_using_materialized_views to true, a little bit confused. Git

Re: [D] Add Support for the diskquota extension in Apache Cloudberry [cloudberry]

2025-06-12 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: Add Support for the diskquota extension in Apache Cloudberry Hey, I forked one here: http://github.com/cloudberry-contrib/diskquota. Welcome to help create PR to fix these issues. GitHub link: https://github.com/apache/cloudberry/discuss

Re: [D] [Ideas] Build some like greenplum_fdw for GPDB or ADB-to-ADB connector for Arenadata, for Cloudberry to run queries between one and more CBDB clusters [cloudberry]

2025-06-06 Thread via GitHub
GitHub user roseduan added a comment to the discussion: [Ideas] Build some like greenplum_fdw for GPDB or ADB-to-ADB connector for Arenadata, for Cloudberry to run queries between one and more CBDB clusters I am working on the issue. Firstly, we decide to use parallel retrieve cursor to suppo

Re: [D] [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? [cloudberry]

2025-06-05 Thread via GitHub
GitHub user avamingli added a comment to the discussion: [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? Yes, this is an inevitable trade-off. However, compared to the significant time and resource savings provided by AQUMV—especially when handling large-scale

Re: [D] Proposal: Enable Parallel DQA Plans (with streaming hash agg) [cloudberry]

2025-06-04 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Proposal: Enable Parallel DQA Plans (with streaming hash agg) Hi, we recently encountered the same issue in a customer's environment, so I'd like to revisit this topic. In the customer's setup, there are numerous DISTINCT queries runnin

Re: [D] [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? [cloudberry]

2025-06-04 Thread via GitHub
GitHub user yjhjstz added a comment to the discussion: [Ideas] Shall we enable answer_query_using_materialized_views by default in v2.0? plan time increased ? we can offer some data to discuss. GitHub link: https://github.com/apache/cloudberry/discussions/1142#discussioncomment-13374237 ---

Re: [D] Fast Filtering for Materialized Views when Answering Query. [cloudberry]

2025-06-02 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Fast Filtering for Materialized Views when Answering Query. part2 https://github.com/apache/cloudberry/pull/1138 GitHub link: https://github.com/apache/cloudberry/discussions/1115#discussioncomment-13350818 This is an automaticall

Re: [D] 2.x vs 1.6 and how to get 2.x [cloudberry]

2025-05-28 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: 2.x vs 1.6 and how to get 2.x Hi @dbaid, Thank you for your interest in Cloudberry! Our community is currently working on Release 2.0, and while we are very close to completing it, it will take a bit more time before we can deliver the f

Re: [D] Add Support for the diskquota extension in Apache Cloudberry [cloudberry]

2025-05-28 Thread via GitHub
GitHub user vsbace edited a comment on the discussion: Add Support for the diskquota extension in Apache Cloudberry Well i have a one problem when i was trying to compile DISKQUOTA extension on Cloudberry 1.6.0 and i need help in this situation if it possible: `/opt/diskquota/src/diskquota.c

Re: [D] Add Support for the diskquota extension in Apache Cloudberry [cloudberry]

2025-05-28 Thread via GitHub
GitHub user vsbace added a comment to the discussion: Add Support for the diskquota extension in Apache Cloudberry Hi, i had solved this problem but catched the next problem: `root@cbdb01:/opt/diskquota/build# make [ 12%] Building C object CMakeFiles/diskquota.dir/src/diskquota_utility.c.o /opt

Re: [D] Add Support for the diskquota extension in Apache Cloudberry [cloudberry]

2025-05-27 Thread via GitHub
GitHub user vsbace added a comment to the discussion: Add Support for the diskquota extension in Apache Cloudberry DISKQUOTA source i was taken from this place: https://github.com/arenadata/diskquota GitHub link: https://github.com/apache/cloudberry/discussions/1036#discussioncomment-1328471

Re: [D] Add Support for the diskquota extension in Apache Cloudberry [cloudberry]

2025-05-27 Thread via GitHub
GitHub user vsbace edited a comment on the discussion: Add Support for the diskquota extension in Apache Cloudberry DISKQUOTA source i taken from this place: https://github.com/arenadata/diskquota GitHub link: https://github.com/apache/cloudberry/discussions/1036#discussioncomment-13284719

Re: [D] Add Support for the diskquota extension in Apache Cloudberry [cloudberry]

2025-05-27 Thread via GitHub
GitHub user vsbace added a comment to the discussion: Add Support for the diskquota extension in Apache Cloudberry Well i have a one problem when i was trying to compile DISKQUOTA extension on Cloudberry 1.6.0 and i need help in this situation if it possible: `/opt/diskquota/src/diskquota.c:

Re: [D] Add Support for the diskquota extension in Apache Cloudberry [cloudberry]

2025-05-27 Thread via GitHub
GitHub user vsbace edited a comment on the discussion: Add Support for the diskquota extension in Apache Cloudberry Well i have a one problem when i was trying to compile DISKQUOTA extension on Cloudberry 1.6.0 and i need help in this situation if it possible: `/opt/diskquota/src/diskquota.c

Re: [D] Add Support for the diskquota extension in Apache Cloudberry [cloudberry]

2025-05-27 Thread via GitHub
GitHub user vsbace added a comment to the discussion: Add Support for the diskquota extension in Apache Cloudberry Hello, we are planning to use the DISK QUOTA extension in Cloudberry DB. And I think this extension is mostly used in current Greenplum installations, and it will be popular if it

Re: [D] [Ideas] Submodule Pinning [cloudberry]

2025-05-26 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: [Ideas] Submodule Pinning As we’re approaching the 2.0.0 release, I noticed that there hasn’t been significant progress on PR #1084. If the author and reviewers are unable to reach a consensus in the near term, I’d suggest we consider defe

Re: [D] Mirror Segment Fail Over [cloudberry]

2025-05-25 Thread via GitHub
GitHub user SeihaHoy edited a discussion: Mirror Segment Fail Over Currently I am testing cloudberry on Virtual Machines with a Coordinator Node and 2 Segment nodes with each has 2 primary segments and 2 mirror segments. I applied default mirror configuration to the database. After that I simu

Re: [D] Mirror Segment Fail Over [cloudberry]

2025-05-25 Thread via GitHub
GitHub user SeihaHoy edited a discussion: Mirror Segment Fail Over Currently I am testing cloudberry on Virtual Machines with a Coordinator Node and 2 Segment nodes with each has 2 primary segments and 2 mirror segments. I applied default mirror configuration to the database. After that I simu

Re: [D] Support jsonlog implementation [cloudberry]

2025-05-21 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: Support jsonlog implementation Already replied in #1120, but would like to copy my reply here again for your information: > There is a proposal related to upgrading the PG14 to PG16. Perhaps your work > can be a part of this proposal: >

Re: [D] Fast Filtering for Materialized Views when Answering Query. [cloudberry]

2025-05-21 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Fast Filtering for Materialized Views when Answering Query. > As an initial step in the overall plan, this part could be optimized by > utilizing other system tables, such as gp_matview_aux, to skip unnecessary > scans. https://github.

Re: [D] [Proposal] Upgrade the PostgreSQL version from 14 to 16 [cloudberry]

2025-05-19 Thread via GitHub
GitHub user oppenheimer01 added a comment to the discussion: [Proposal] Upgrade the PostgreSQL version from 14 to 16 Hi Ed, Thank you for helping to sort out the remaining issues. Here are my thoughts on these issues ### Release Lifecycle Policy: Unclear - Thanks for making it clearer. It

Re: [D] Proposal: Binary Data Support for libpq's Extended Protocol with Consumption Tracking [cloudberry]

2025-05-15 Thread via GitHub
GitHub user avamingli closed a discussion: Proposal: Binary Data Support for libpq's Extended Protocol with Consumption Tracking ### Description We( @avamingli and @wangliang03) propose enhancing libpq's Extended Protocol to support binary data transmission while adding message tracking cap

Re: [D] Proposal: Binary Data Support for libpq's Extended Protocol with Consumption Tracking [cloudberry]

2025-05-15 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Proposal: Binary Data Support for libpq's Extended Protocol with Consumption Tracking Merged. GitHub link: https://github.com/apache/cloudberry/discussions/1092#discussioncomment-13165280 This is an automatically sent email for de

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-14 Thread via GitHub
GitHub user leborchuk added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ Hi, @fanfuxiaoran I have a number of ideas about what can be done to improve diagnosis. May I ask you a few quick questions below? You can answer them in detail or

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-14 Thread via GitHub
GitHub user leborchuk added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ We have a similar solution for Open-GPDB: https://github.com/open-gpdb/yagp_hooks_collector/tree/YAGP-0.0.2-WIP. Right now, we have an open-sourced component that i

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-14 Thread via GitHub
GitHub user leborchuk added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ Hi @fanfuxiaoran, very thanks for your work! I have a thought on how to properly review and merge such complex functionality. Since we have agreed not to merge it i

Re: [D] Proposal: Binary Data Support for libpq's Extended Protocol with Consumption Tracking [cloudberry]

2025-05-14 Thread via GitHub
GitHub user avamingli added a comment to the discussion: Proposal: Binary Data Support for libpq's Extended Protocol with Consumption Tracking https://github.com/apache/cloudberry/pull/1098 GitHub link: https://github.com/apache/cloudberry/discussions/1092#discussioncomment-13141396 Thi

Re: [D] [Proposal] Upgrade the PostgreSQL version from 14 to 16 [cloudberry]

2025-05-13 Thread via GitHub
*Recommendation:** * List all bundled extensions and components expected in 3.x. * Assign or identify maintainers for each. * Track PG16 compatibility status (via GitHub issues or a shared audit doc). --- ### 🌐 Downstream Ecosystem Impact: Private and Commercial Components Apache Cloudberr

Re: [D] [Proposal] Upgrade the PostgreSQL version from 14 to 16 [cloudberry]

2025-05-13 Thread via GitHub
GitHub user oppenheimer01 added a comment to the discussion: [Proposal] Upgrade the PostgreSQL version from 14 to 16 Greate idea ! I think GitHub Projects is a good tool for PostgreSQL version upgrade work. A new project named 'Merge postgres 16' has been created for traking the upgrade wo

Re: [D] [Proposal] Upgrade the PostgreSQL version from 14 to 16 [cloudberry]

2025-05-13 Thread via GitHub
GitHub user oppenheimer01 added a comment to the discussion: [Proposal] Upgrade the PostgreSQL version from 14 to 16 Cloudberry 2.0 will maintain support regardless of PostgreSQL 14s end-of-life (EOL). Both versions 2.0 and 3.0 are committed to long-term support cycles. GitHub link: https

Re: [D] [Ideas] Submodule Pinning [cloudberry]

2025-05-13 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: [Ideas] Submodule Pinning I recently did a test following Ed's suggestions from PR #1084 comments. It worked, as I can track versions more effectively on my test branch: https://github.com/tuhaihe/cloudberrydb/commits/submodule-update05/,

Re: [D] [Proposal] Upgrade the PostgreSQL version from 14 to 16 [cloudberry]

2025-05-12 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: [Proposal] Upgrade the PostgreSQL version from 14 to 16 I have two questions from here: - For Cloudberry 2.x, will it also be EOF by 2026, following the PostgreSQL versioning policy[^1]? - Will the Cloudberry 3.x be the PG16 built-in, an

Re: [D] Add cloudberry-mcp-server for integration with LLM apps [cloudberry]

2025-05-12 Thread via GitHub
GitHub user yjhjstz added a comment to the discussion: Add cloudberry-mcp-server for integration with LLM apps we can write extension using pgrx(which Build Postgres Extensions with Rust) or c/c++ . GitHub link: https://github.com/apache/cloudberry/discussions/1068#discussioncomment-13122505

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-11 Thread via GitHub
GitHub user fanfuxiaoran added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ @edespino I forked the libsigar https://github.com/cloudberry-contrib/sigar. GitHub link: https://github.com/apache/cloudberry/discussions/1087#discussioncomm

Re: [D] Add cloudberry-mcp-server for integration with LLM apps [cloudberry]

2025-05-11 Thread via GitHub
GitHub user yangs16 added a comment to the discussion: Add cloudberry-mcp-server for integration with LLM apps Not sure about this. MCP-server is expected to be integrated with LLM clients/agents. If it's implemented as a Cloudberry extension, how will the LLM clients/agents interact with it?

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-08 Thread via GitHub
GitHub user edespino added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ Hi @fanfuxiaoran, Thanks — I think your plan to clone and maintain a Linux-only SIGAR fork outside the perfmon tree makes a lot of sense. It keeps things modular and

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-08 Thread via GitHub
GitHub user my-ship-it added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ > @tuhaihe & @my-ship-it - Given the amount of work needed to get SIGAR into > shape to be used with perfmon, can you share your thoughts on excluding > perfmon fo

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-08 Thread via GitHub
GitHub user tuhaihe added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ Given the significant effort required for the perfmon PR, I believe it's reasonable to exclude it from the upcoming 2.0.0 release. We can include this functionality i

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-08 Thread via GitHub
GitHub user edespino added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ @tuhaihe & @my-ship-it - Given the amount of work needed to get SIGAR into shape to be used with perfmon, can you share your thoughts on excluding perfmon for the up

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-08 Thread via GitHub
GitHub user edespino added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ Given the amount of work involved in preparing the SIGAR fork and ensuring it’s ready for inclusion under ASF policy, I suggest we exclude perfmon from the upcoming

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-08 Thread via GitHub
GitHub user fanfuxiaoran added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ @edespino Thanks for your detailed response and summary. - Sigar is implemented by reading /proc on linux. I think we don't bother to implement another 'sigar

Re: [D] [Ideas] Cleanup of `deploy/*` Directory [cloudberry]

2025-05-08 Thread via GitHub
GitHub user tuhaihe closed a discussion: [Ideas] Cleanup of `deploy/*` Directory ### Description Hi, I would like to propose a cleanup of the `deploy/` directory. Upon reviewing the files in this directory, I noticed that there are some old URLs that may no longer be valid, and we need to en

Re: [D] [Ideas] Improve Version Information for Source Builds [cloudberry]

2025-05-08 Thread via GitHub
GitHub user tuhaihe closed a discussion: [Ideas] Improve Version Information for Source Builds ### Description Currently, when Cloudberry is built from a source zip file (not cloned via git), the version information displayed is not matched the right info, always like this: ``` PostgreSQL 1

Re: [D] [Ideas] Remove Unused Files (concourse/*, hd-ci/*) After Cherry-Pick Completion [cloudberry]

2025-05-08 Thread via GitHub
GitHub user tuhaihe closed a discussion: [Ideas] Remove Unused Files (concourse/*, hd-ci/*) After Cherry-Pick Completion ### Description As part of the ongoing cherry-pick process from the Greenplum archive, we have identified two directories (`concourse/` and `hd-ci/*`) that are no longer in

Re: [D] [Ideas] Cleanup of `deploy/*` Directory [cloudberry]

2025-05-08 Thread via GitHub
GitHub user tuhaihe closed the discussion with a comment: [Ideas] Cleanup of `deploy/*` Directory This has been done in PR #1090 GitHub link: https://github.com/apache/cloudberry/discussions/948#discussioncomment-13072515 This is an automatically sent email for dev@cloudberry.apache.org

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-08 Thread via GitHub
GitHub user edespino added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ ## 📌 Note: Replacing libsigar in gpsmon Hi @fanfuxiaoran, Following up on the effort to replace libsigar within gpsmon: while SIGAR is Apache-licensed and technical

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-08 Thread via GitHub
GitHub user edespino added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ Hi @fanfuxiaoran , Thanks for pointing out libstatgrab as a potential replacement for libsigar. One important licensing detail to be aware of: > While the libr

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-08 Thread via GitHub
GitHub user fanfuxiaoran added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ @edespino For the options to replace SIGAR, I prefer libstatgrab as it is a cross-platform library, but it provides a more limited set of collected parameters c

Re: [D] Add cloudberry-mcp-server for integration with LLM apps [cloudberry]

2025-05-07 Thread via GitHub
GitHub user yjhjstz added a comment to the discussion: Add cloudberry-mcp-server for integration with LLM apps Can we impl mcp-server also be an extension ? not just write by python ? GitHub link: https://github.com/apache/cloudberry/discussions/1068#discussioncomment-13070361 This is a

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-07 Thread via GitHub
GitHub user edespino added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ ## Building `perfmon` with SIGAR on Rocky Linux 9 – Dependency Discovery, Gaps, and Recommendations** While working to build the `perfmon` components (`gpsmon`, `gpm

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-07 Thread via GitHub
GitHub user edespino added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ ### 📚 User Documentation Plans? This is a feature that will directly impact end users — it introduces an extension, SQL tables, background workers, and runtime hooks

Re: [D] Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ [cloudberry]

2025-05-07 Thread via GitHub
GitHub user fanfuxiaoran added a comment to the discussion: Introducing the [perfmon] Extension for Cloudberry Database Monitoring​ > @fanfuxiaoran > > Thanks for confirming that `libsigar` is still in use. However, I’d strongly > encourage us to reconsider that position. Continuing to dep

  1   2   3   >