Re: Re:Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-08 Thread Joice Jacob
Hi Kaka & Doris Community, Thank you for your suggestion. Following your advice, I manually placed the "_orc_acid_version" file into the HDFS directory of Hive. I'm pleased to report that this solution has resolved the issue, and the integration with Doris is now functioning correctly. Your guida

Re: Re:Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-07 Thread Joice Jacob
Yes, I will check. I will update you by EOD today. Thanks for support. Thanks & Regards Joice On Sun, Oct 8, 2023, 7:48 AM kaka chen wrote: > Hi Joice, > > BTW,could you pls try to query it by Doris after putting the > "_orc_acid_version" > file to the hdfs directory of hive manually? Thanks. >

Re: Re:Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-07 Thread kaka chen
Hi Joice, BTW,could you pls try to query it by Doris after putting the "_orc_acid_version" file to the hdfs directory of hive manually? Thanks. Best regards, Kaka kaka chen 于2023年10月8日周日 10:10写道: > Hi Joice, > > Got it, thanks very much. We will setup a cdp env and do some > compatibility work

Re: Re:Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-07 Thread kaka chen
Hi Joice, Got it, thanks very much. We will setup a cdp env and do some compatibility work. Best regards, Kaka Joice Jacob 于2023年10月7日周六 14:58写道: > Hi,Dear Doris Community, > I wanted to provide an update on my recent experience with integrating > Doris with both Apache Hive and CDP Hive, as w

Re: Re:Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-06 Thread Joice Jacob
Hi,Dear Doris Community, I wanted to provide an update on my recent experience with integrating Doris with both Apache Hive and CDP Hive, as well as seek insights into a particular observation. In my integration with Apache Hive, I noticed that transaction tables retain the "_orc_acid_version" fil

Re: Re:Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-06 Thread kaka chen
Hi Joice: Thanks your reporting. It seems the root cause of this issue is missing "_orc_acid_version" file. >From Hive version >= 3.0, delta/base files will always have file '_orc_acid_version' with value >= '2'. Maybe the hive3 of HDP has similar issue? https://issues.apache.org/jira/browse/HIVE-

Re: Re:Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-06 Thread Joice Jacob
Hi, I've checked the Hive data directory, and I couldn't find the "_orc_acid_version" file. I have attached screenshots for your reference. Are there any recommended workarounds or alternative approaches that I can consider to resolve this issue? I'm open to exploring different solutions to ensure

Re:Re:Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-06 Thread Mingyu Chen
Could you list your hive data dir, to see what files exist? like: _orc_acid_version -- Best Regards Mingyu Chen Email: morning...@apache.org At 2023-10-05 22:31:58, "Mingyu Chen" wrote: >Oh, I see, let me check it again. > > > > >-- > >Best Regards >Mingyu Chen > >Email: >morning.

Re:Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-05 Thread Mingyu Chen
Oh, I see, let me check it again. -- Best Regards Mingyu Chen Email: morning...@apache.org At 2023-10-05 21:49:28, "Joice Jacob" wrote: >Hi, >I wanted to share some important information regarding the CDP Hive version >that I am currently using, which is Hive 3.1.3000.7.1.7.0-551. > >In

Re: Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-05 Thread Joice Jacob
Hi, I wanted to share some important information regarding the CDP Hive version that I am currently using, which is Hive 3.1.3000.7.1.7.0-551. In CDP Hive version 3.x, a significant change has been introduced regarding managed tables. By default, managed tables in Hive 3.x are considered transacti

Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-05 Thread Joice Jacob
Hi Guangdong Liu, please find the attachment for DESCRIBE FORMATTED baby output On Thu, Oct 5, 2023 at 6:27 PM Guangdong Liu wrote: > Can you see the detailed information of the table baby? > > `DESCRIBE FORMATTED baby;` > > -- > > Best Regards > > > > Liugddx > liug...@gmail.com >

Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-05 Thread Guangdong Liu
Can you see the detailed information of the table baby? `DESCRIBE FORMATTED baby;` -- Best Regards Liugddx liug...@gmail.com Joice Jacob 于2023年10月5日周四 13:30写道: > Dear Doris Community & Jiafeng.Zhang > , > Thank you for your prompt response and your willingness to assist with t

Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-04 Thread Jiafeng.Zhang
Can you provide your hive catalog creation statement, your fe.log log at that time, and your cdp hive version? This will help us locate the problem, thank you. You can also try specifying your hive version in your catalog statement: "hive.version=2.1.0" Joice Jacob 于2023年10月5日周四 00:45写道: > I am

Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-04 Thread Joice Jacob
I am reaching out once again to seek assistance and share a specific issue I've encountered while integrating Doris with Cloudera-flavored Hive. The error message I'm facing is as follows: detailMessage = get file split failed for table: baby1, err: java.lang.Exception: Hive 2.x versioned full-acid

Re: Issue Integrating Hive on CDP Distribution with Doris 2.0.1.1

2023-10-04 Thread Joice Jacob
Dear Doris Community, I have an update on the issue I previously mentioned regarding the integration of Hive on a CDP distribution with Doris 2.0.1.1. After further investigation, I have identified that the issue is related to Hive managed tables being transactional by default. This appears to be