On 03/18/2014 09:58 PM, spykee wrote:
I agree, starting 20K threads/sec does not sound like I good idea. But
since I'm not familiar with these other technologies that you use, I
don't understand why you need to fire a thread per message. Why can't
you have a dedicated sender thread that reads everything in your
queue-table and sends it (through whatever api you have to the MQ
system). You could either have the sender thread check the queue table
periodically, or have it sleep on a monitor which the trigger action
grabs when adding new records to the queue table.
First you answered to this question yourself( previous response). For each
call of the stored procedure a new thread will be created to execute the
stored procedure.
I'm sorry but I don't understand this sentence. Why do you say that a
call to a stored procedure spawns a new thread?. Derby most certainly
does not spawn a new thread when executing a stored procedure. You
could, of course add code to your stored procedure which spawns a new
thread, but why would you? If the SP takes a long time to run you're
only blocking a single connection, not the entire database.
Anyway, looks like you have some ideas. Good luck :)
At this I referred, and I want to avoid creating one
thread per message even if the messages are produced at different times (
e.g. message_1 at t1, and message_2 at t2, t1 < t2).
Second, I read that pooling from a table ( perform a periodically select
c1,c2.... from table_T where.... to see if something changed) is not an
optimal choice, but I like the idea with the monitor ( same thing here, for
each notification from the db " Hey a new row was inserted, wake up and take
the message", I will execute a stored procedure -> one thread per
notification, but since this is just a notifier, less bytes, the network
load is lower, but I will have to make a new call from my monitor thread to
the database.)
Using the monitor approach I will have to make *2 steps:*
- throw a notifier message from the db to the monitor thread ( execute a
stored procedure)
- the monitor thread will then have a logic, and then query the table for
the changes and publish them to the queue
-> this approach will ALLOW me batching the messages ( this should improve
the performance)!!
With the first approach I have to make *1 step:*
- throw a message directly to the queue
-> this approach will NOT allow me batching the messages( this should
decrease the performance)
Why can't everything from the Derby trigger to the message queue be
inside the same jvm?
Everything will run on the same JVM, or different JVM ( this is why I will
use a JMS queue). For the moment there will be same JVM.
I will implement both solutions :
- monitor solution
- my first solution
- compare how many messages are sent per second/minute for each solution,
and use the approach which is the best.
Kind regards,
George
--
View this message in context:
http://apache-database.10148.n7.nabble.com/Re-Derby-replication-system-Need-help-tp138003p138081.html
Sent from the Apache Derby Users mailing list archive at Nabble.com.
--
Regards,
Dyre
