toohandsome opened a new issue, #29947: URL: https://github.com/apache/doris/issues/29947
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description 我们目前的日志是通过java组件来把日志发送到redis中,然后日志服务消费redis中的数据到es中.存放在es中进行搜索,每天大概100G, 查询经常超时,希望用 doris 来代替 因为有业务上实时查询执行sql的需求,我大概看了一下官网的文档,目前比较适合我们的导入日志方式应该就是 insert 方式, 但是 文档好像不太建议这么做   是否可以提供关闭事务的方式来支持, 同时是否可以新增数据订阅的功能? 实现新入库数据的通知 我看了B站 Apache Doris [日志场景解决方案](https://www.bilibili.com/video/BV1d14y1r7KR/?vd_source=2428fc03aeed1b6c07d7f526c226b25d),里面并没有具体落地的方案 ---------------------------------------------------------------------------------------- Our current logs are sent to Redis through Java components, and then the log service consumes the data from Redis to ES for searching. It is stored in ES for approximately 100GB per day, and the query often times out. We hope to use Doris instead Because there is a demand for real-time query and execution of SQL in business, I have roughly looked at the documentation on the official website. Currently, the most suitable way for us to import logs is probably the insert method, But the document doesn't seem to suggest doing so much ! [image]（ https://github.com/apache/doris/assets/46422252/1cda6b38-c4cb-42c3-b5a5-8a56c9b5dc9d ） ! [image]（ https://github.com/apache/doris/assets/46422252/cba5f3a2-e1f4-4eda-9d41-319bdf095623 ） Can we provide a way to close transactions to support it, and can we also add data subscription functionality? Implement notification of new incoming data I looked at Apache Doris on Bilibili [Log Scenario Solution]（ https://www.bilibili.com/video/BV1d14y1r7KR/?vd_source=2428fc03aeed1b6c07d7f526c226b25d ）There is no specific implementation plan inside ### Use case 支持客户端控制当前导入是否需要事务 支持数据订阅推送 ----------------------------- Support client control over whether transactions are required for the current import Support data subscription push ### Related issues _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
