orpiske commented on issue #329: URL: https://github.com/apache/camel-kafka-connector-examples/issues/329#issuecomment-1040029641
Hello, > Hello > > I am getting message.max.bytes error when trying to read logs files (pain text) of 300MB. As far as I know, the connector moves the whole file in one record to kafka. I haven't really tried it, but I'd look at using a converter for splitting the file. > Is any option to split the file in mini batches and have an atomic transaction. In case the connector fails in the middle of > the processing, reprocess only from where it left? Not yet. I am implementing that feature in Camel Core as part of CAMEL-15562. It's progressing, and should be in Core in a few versions if everything goes alright. > > Regards -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
