On Tue, 04 Aug 2015 11:49:38 -0700 Rob Miller <[email protected]> wrote:
> Answers in-line below, although I'm not sure you're going to like them... thanks as always. > > I have started work on a heka output and encoder plugin for kairosdb's rest > > endpoint. > Is a custom output really needed? Ideally you'd just use the standard > HttpOutput. i'll try it out. i presume it honors response codes, and when combined with disk buffer, updates the cursor upon 2xx response? > There's currently no support for delaying the RabbitMQ ack until the received > message has been written to disk. All AMQP interactions happen in the > AMQPInput. Disk buffering, if it happens, doesn't occur until between the > message matcher and the filter or output plugin. oh, i seemed to recall acks could be postponed until further processing down the pipe. I thought this came up as a differentiatior compared to logstash in one of the older videos on vimeo. but maybe i'm mistaken. or did this change? > > I couldn't find any config option to the disk buffer that controls how > > often/after how much data sync() is called. > There's no relationship btn filter / output disk buffering and the details of > any particular input plugin. I was just curious how often the disk buffer syncs data to disk or what kind of semantics it uses to determine when data that comes in (the disk buffer code), when it is safely stored on disk, and if there's any knobs to control the behavior like sync_every x ms or sync_after x messages > Just that Heka is *not* claiming to offer full delivery guarantees at this > time. We're trying to be explicit about the risks and trade-offs. Fair enough, no worries. cheers, Dieter _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
