Re: [Beowulf] transcode Similar Video Processing on Beowulf?

2014-04-16 Thread Gavin W. Burris
Think of RPC as a POST and a simple queue as a GET. On Wed 04/16/14 01:41PM -0400, Mark Hahn wrote: > >I think this an interesting distinction, between RPC and simple queues. > >RPC relies on a client calling on a known remote server, where simple > >queues allow a horde of workers to pull message

Re: [Beowulf] transcode Similar Video Processing on Beowulf?

2014-04-16 Thread rf
> "Roland" == Mark Hahn writes: >> successfully) it is now ready to securely manage stuff in the >> cloud. Our secure remote execution engine is also based on >> ZMQ/Curve. Roland> interesting, though queues seem like a potentially odd Roland> choice. don't you normally

Re: [Beowulf] transcode Similar Video Processing on Beowulf?

2014-04-16 Thread Mark Hahn
I think this an interesting distinction, between RPC and simple queues. RPC relies on a client calling on a known remote server, where simple queues allow a horde of workers to pull messages off as they are available. I can see a use case for either depending on scale. I'm unforgivably implemen

Re: [Beowulf] transcode Similar Video Processing on Beowulf?

2014-04-16 Thread Gavin W. Burris
I think this an interesting distinction, between RPC and simple queues. RPC relies on a client calling on a known remote server, where simple queues allow a horde of workers to pull messages off as they are available. I can see a use case for either depending on scale. Cheers. On Wed 04/16/14 1

Re: [Beowulf] transcode Similar Video Processing on Beowulf?

2014-04-16 Thread Mark Hahn
successfully) it is now ready to securely manage stuff in the cloud. Our secure remote execution engine is also based on ZMQ/Curve. interesting, though queues seem like a potentially odd choice. don't you normally want RPC-like behavior, rather than just queueing? (I use SSH to provide my secure

Re: [Beowulf] transcode Similar Video Processing on Beowulf?

2014-04-16 Thread Mark Hahn
I'm trying to understand this from a perspective of conventional HPC. cop-out but we're not keen to reinvent the wheel. It provides statekeeping and job queues in one package; replacing it wouldn't be "statekeeping" is just tracking queued/running/done jobs, right? trivial but wouldn't be a

Re: [Beowulf] transcode Similar Video Processing on Beowulf?

2014-04-16 Thread rf
> "James" == James Harrison writes: James> -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 James> On 14/04/2014 14:13, Gavin W. Burris wrote: >> Hi, Lawrence. >> >> Maybe rolling something with a ZeroMQ / 0MQ would be a good idea. >> This could be a very fast way to pas

Re: [Beowulf] transcode Similar Video Processing on Beowulf?

2014-04-16 Thread James Harrison
On 15/04/14 14:02, Gavin W. Burris wrote: >>> > Yes! No doubt! The "simple" queues presuppose a massive > distributed system to take advantage of. Bonus points if that > system can interchangeably be an in-house cluster or major cloud > provider. > > I would be very interested to hear what you