On Friday 29 August 2014 10:33:12 Chris Knipe wrote: > Unfortunately not (we are talking NNTP specifically here if it isn't > obvious yet) - and the data rates on the network in terms of the > stream we're looking at over 1Gbps (generated with the TAKETHIS > commands). The other seemingly obvious problem that I am going to > have by handing the work off to a separate process (or even thread for > that matter) is that I need to inform the remote client on the socket > in real time whether I want / do not want / if an error occurred to > the article in question issued by the command from the client. > > It's times like this that I *really* wish I knew how to code in C - no > offence to anyone, again. Even in .NET doing something like this is > almost trivial by using separate event handlers for receiving data > from the socket - again which is why I firmly believed that something > like this could have been done via POE (yes, this would be handled via > threads - but it's done and managed by the .NET modules, and thus > require -very- little code, or understanding for that matter, from the > programmer's point of view). > > Oh well - it's back to the drawing board for this one then. > > -- > > Regards, > Chris Knipe
Hi Chris, I've not read the whole thread in details but what you need seems fairly simple. I think you're wrong when you said about implementing it within POE or any NET:: module because what you are after is definitely application layer and not module layer. I think what you want has been covered by Carl. Put very abtractly you need * one task (process, thread, whatever) to handle communications This can either be one task per connection which would be easiest to implement using the traditional process of forking on each incoming socket connection described in the first post, or can be one process handling all concurrent connections, possibly using POE. I believe that the notes your read about avoiding having one task reading a socket and another task writing a socket is to do with the actual TCP conversation, and I can see how that could cause issues. By having one dispatcher per connection, you can syncronise the sending / receiving per IP connection. * one task per work unit (each article). Have one process accept the job from the communications task, work on the article asyncronously, and return the status back to the communications task. There are many different ways within unix / Perl for inter-process comms for this part of the job. Depending on the workload, a model similar to Apache httpd where there are maximum X processes running, which I believe periodicallty die and get replaced to prevent them going stale. Gary -- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] http://learn.perl.org/
