Re: [VOTE] Release Spark Connectors v1.0.0 for Apache Doris(Incubating)
I tried to build the source with thrift 0.16.0 (which is installed in my box). I got below error, /apache-doris-spark-connector-3.1.2-2.12-1.0.0-incubating-src/target/generated-sources/thrift/org/apache/doris/thrift/TStatus.java:[527,58] no suitable method found for readListBegin(byte) method org.apache.thrift.protocol.TProtocol.readListBegin() is not applicable (actual and formal argument lists differ in length) method org.apache.thrift.protocol.TCompactProtocol.readListBegin() is not applicable (actual and formal argument lists differ in length) Did I miss something? Willem Jiang Twitter: willemjiang Weibo: 姜宁willem On Tue, Mar 1, 2022 at 11:03 AM 陈明雨 wrote: > > Hi All, > > > > > This is a call for vote to release Spark Connectors v1.0.0 for Apache > Doris(Incubating). > And this is the first release of doris-spark-connector, we just move the code > base from [1] to [2]. > > There are 2 outputs for different Spark and Scala version: > > > > > - apache-doris-spark-connector-2.3.4-2.11-1.0.0-incubating > > - apache-doris-spark-connector-3.1.2-2.12-1.0.0-incubating > > > > > The release candidates: > > https://dist.apache.org/repos/dist/dev/incubator/doris/spark-connector/1.0.0/ > > > > > Maven 2 staging repository: > > https://repository.apache.org/content/repositories/orgapachedoris-1006/org/apache/doris/doris-spark-connector/ > > > > > Git tag for the release: > > https://github.com/apache/incubator-doris-spark-connector/tree/2.3.4-2.11-1.0.0 > > https://github.com/apache/incubator-doris-spark-connector/tree/3.1.2-2.12-1.0.0 > > > > > Keys to verify the Release Candidate: > > https://downloads.apache.org/incubator/doris/KEYS > > > > > Look at here for how to verify this release candidate: > > http://doris.incubator.apache.org/community/release-and-verify/release-verify.html > > > > > The vote will be open for at least 72 hours or until necessary number of > votes are reached. > > Please vote accordingly: > > [ ] +1 approve > > [ ] +0 no opinion > > [ ] -1 disapprove with the reason > > > > [1] https://github.com/apache/incubator-doris > [2] https://github.com/apache/incubator-doris-spark-connector > > -- > > 此致！Best Regards > 陈明雨 Mingyu Chen > > Email: > chenmin...@apache.org - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
Re:Re: [VOTE] Release Spark Connectors v1.0.0 for Apache Doris(Incubating)
We are using thrift@0.13 for now. Maybe thrift@0.16 changed its API. -- 此致！Best Regards 陈明雨 Mingyu Chen Email: chenmin...@apache.org At 2022-03-03 17:36:36, "Willem Jiang" wrote: >I tried to build the source with thrift 0.16.0 (which is installed in my box). >I got below error, >/apache-doris-spark-connector-3.1.2-2.12-1.0.0-incubating-src/target/generated-sources/thrift/org/apache/doris/thrift/TStatus.java:[527,58] >no suitable method found for readListBegin(byte) >method org.apache.thrift.protocol.TProtocol.readListBegin() is not >applicable > (actual and formal argument lists differ in length) >method org.apache.thrift.protocol.TCompactProtocol.readListBegin() >is not applicable > (actual and formal argument lists differ in length) > >Did I miss something? > > >Willem Jiang > >Twitter: willemjiang >Weibo: 姜宁willem > >On Tue, Mar 1, 2022 at 11:03 AM 陈明雨 wrote: >> >> Hi All, >> >> >> >> >> This is a call for vote to release Spark Connectors v1.0.0 for Apache >> Doris(Incubating). >> And this is the first release of doris-spark-connector, we just move the >> code base from [1] to [2]. >> >> There are 2 outputs for different Spark and Scala version: >> >> >> >> >> - apache-doris-spark-connector-2.3.4-2.11-1.0.0-incubating >> >> - apache-doris-spark-connector-3.1.2-2.12-1.0.0-incubating >> >> >> >> >> The release candidates: >> >> https://dist.apache.org/repos/dist/dev/incubator/doris/spark-connector/1.0.0/ >> >> >> >> >> Maven 2 staging repository: >> >> https://repository.apache.org/content/repositories/orgapachedoris-1006/org/apache/doris/doris-spark-connector/ >> >> >> >> >> Git tag for the release: >> >> https://github.com/apache/incubator-doris-spark-connector/tree/2.3.4-2.11-1.0.0 >> >> https://github.com/apache/incubator-doris-spark-connector/tree/3.1.2-2.12-1.0.0 >> >> >> >> >> Keys to verify the Release Candidate: >> >> https://downloads.apache.org/incubator/doris/KEYS >> >> >> >> >> Look at here for how to verify this release candidate: >> >> http://doris.incubator.apache.org/community/release-and-verify/release-verify.html >> >> >> >> >> The vote will be open for at least 72 hours or until necessary number of >> votes are reached. >> >> Please vote accordingly: >> >> [ ] +1 approve >> >> [ ] +0 no opinion >> >> [ ] -1 disapprove with the reason >> >> >> >> [1] https://github.com/apache/incubator-doris >> [2] https://github.com/apache/incubator-doris-spark-connector >> >> -- >> >> 此致！Best Regards >> 陈明雨 Mingyu Chen >> >> Email: >> chenmin...@apache.org > >- >To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org >For additional commands, e-mail: dev-h...@doris.apache.org
be????????
??doris beOOM?? ?? 2022-03-04 10:54:43 doris??detailMessage = tablet 45664199 has few replicas: 1, alive backends: [10004] cetnos 8 6416??64G ?? less /var/log/messageOOM?? Mar 4 10:54:38 narwal-doris-be-0004 systemd[1]: Started Process Core Dump (PID 1881819/UID 0).Mar 4 10:54:41 narwal-doris-be-0004 systemd-coredump[1881820]: Core file was truncated to 2147483648 bytes.Mar 4 10:55:00 narwal-doris-be-0004 systemd-coredump[1881820]: Process 2800630 (palo_be) of user 0 dumped core.#012#012Stack trace of thread 1877601:#012#0 0x039480a2 memcpy (/mnt/be/lib/palo_be)#012#012Stack trace of thread 2800630:#012#0 0x7fccb330efc8 n/a (n/a)Mar 4 10:55:01 narwal-doris-be-0004 systemd[1]: systemd-coredump@1-1881819-0.service: Succeeded.Mar 4 10:55:03 narwal-doris-be-0004 systemd[1]: session-12.scope: Succeeded. dump??coredumpctl info 2800630 PID: 2800630 (palo_be) UID: 0 (root) GID: 0 (root) Signal: 11 (SEGV) Timestamp: Fri 2022-03-04 10:54:38 CST (1h 10min ago) Command Line: /mnt/be/lib/palo_be Executable: /mnt/be/lib/palo_be Control Group: / Slice: -.slice Boot ID: ca3ef395d7c547d2aecb1c251097066f Machine ID: 501f93b5c19d4ca38db845c29176e3c5 Hostname: narwal-doris-be-0004 Storage: /var/lib/systemd/coredump/core.palo_be.0.ca3ef395d7c547d2aecb1c251097066f.2800630.164636247800.lz4 (truncated) Message: Process 2800630 (palo_be) of user 0 dumped core. Stack trace of thread 1877601: #0 0x039480a2 memcpy (/mnt/be/lib/palo_be) Stack trace of thread 2800630: #0 0x7fccb330efc8 n/a (n/a) - To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
Re: be异常退出
目前的堆栈看起来不足以支撑做出判断； 如果线上有开core dump的话，可以用gdb palo_be core_dump文件看看堆栈 后者看下be.out是否进程怪盗时的堆栈 刘波 <270309...@qq.com.invalid> 于2022年3月4日周五 12:25写道： > 尊敬的开发者，您好： > 我们的doris > be节点经常会挂掉其中几个，经分析资源情况正常，具体信息可参见附件，日志层面未发现是OOM，未能定位出异常，请求协助，具体信息如下： >*时间： 2022-03-04 10:54:43* >doris报错*：detailMessage = tablet 45664199 has few replicas: 1, > alive backends: [10004]* >环境信息：华为云cetnos 8 64位，16核64G > >系统日志： less /var/log/message（*非OOM*） > > Mar 4 10:54:38 narwal-doris-be-0004 systemd[1]: Started Process Core Dump > (PID 1881819/UID 0). > > Mar 4 10:54:41 narwal-doris-be-0004 systemd-coredump[1881820]: Core file > was truncated to 2147483648 bytes. > > Mar 4 10:55:00 narwal-doris-be-0004 systemd-coredump[1881820]: Process > 2800630 (palo_be) of user 0 dumped core.#012#012Stack trace of thread > 1877601:#012#0 0x039480a2 memcpy > > (/mnt/be/lib/palo_be)#012#012Stack trace of thread 2800630:#012#0 > 0x7fccb330efc8 n/a (n/a) > > Mar 4 10:55:01 narwal-doris-be-0004 systemd[1]: > systemd-coredump@1-1881819-0.service: Succeeded. > > Mar 4 10:55:03 narwal-doris-be-0004 systemd[1]: session-12.scope: > Succeeded. > > > dump信息：*coredumpctl info 2800630* > >PID: 2800630 (palo_be) >UID: 0 (root) >GID: 0 (root) > Signal: 11 (SEGV) > Timestamp: Fri 2022-03-04 10:54:38 CST (1h 10min ago) > Command Line: /mnt/be/lib/palo_be > Executable: /mnt/be/lib/palo_be > Control Group: / > Slice: -.slice >Boot ID: ca3ef395d7c547d2aecb1c251097066f > Machine ID: 501f93b5c19d4ca38db845c29176e3c5 > Hostname: narwal-doris-be-0004 >Storage: > /var/lib/systemd/coredump/core.palo_be.0.ca3ef395d7c547d2aecb1c251097066f.2800630.164636247800.lz4 > (truncated) >Message: Process 2800630 (palo_be) of user 0 dumped core. > > Stack trace of thread 1877601: > #0 0x039480a2 memcpy (/mnt/be/lib/palo_be) > > Stack trace of thread 2800630: > #0 0x7fccb330efc8 n/a (n/a) > > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org > For additional commands, e-mail: dev-h...@doris.apache.org -- 王博 Wang Bo
