What about the things I spent time suggesting in my last email? Did you try them because I was actually curious if they made any impact.
On Thu, Dec 3, 2009 at 11:29 AM, eaf <[email protected]> wrote: > > I'm sorry if I sounded that way. Did mean to. :) > > Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 > chip > and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm > > Line offset difference is due to some minor logging changes I made to see > who's allocating timers and how often. This way I found MOH streaming and > that RTP still allocates timers even when it's set to none in the profile. > > I feel that this platform turned out to be underpowered for FS because it > cannot meet its scheduling expectations. I guess, some degree of kernel > tweaking or setting priorities will fix that. Meanwhile I just got rid of > the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms > thread in two (one blocked and waiting for new commands in the SQL queue, > the other one checking registrations and gateways with 1sec interval), and > don't know yet what to do about the timer thread. > > Again, I apologize for stupid or accusing questions, I'm just trying to see > how FS can be made friendlier to this board. Or the board be made > friendlier > to FS ;) > > > Anthony Minessale-2 wrote: > > > > If you see that message then your machine/os/combo is having some > problems > > keeping up. > > It's not the timer missing anything its the monotonic clock detecting a 1 > > second or more differential from what its next prediction for the time > > should be. The best way to trigger this would be to suspend FS with > > control-z or attach to it with gdb blocking the entire process, that 1ms > > thread would have to miss 1000 iterations to trigger that warning. > > > > Btw, that error message is at line 471 not 473 so you are using modified > > code. > > > > Its possible your box has a bad monotonic timer, you can set > > > > > > > > under <settings> in switch.conf.xml > > > > We are now starting to guess you are using some small embedded type > > platform > > perhaps? > > I've run FS even on a nokia n810 and never caused that message to fire. > > > > if 1 call can interrupt the cpu enough to cause noticeable issues you > > might > > want to consider running the process at a > > greater priority by using the -hp command line arg or at least nice it > > > > Why don't you tell us the whole story about what OS/platform you are > using > > here rather that form conjectures about what is wrong with our code that > > thousands of people are happy with. > > > > > > > > > > > > > > > > On Thu, Dec 3, 2009 at 8:55 AM, eaf <[email protected]> wrote: > > > >> > >> Btw, I have these popping up in my logs from time to time: > >> > >> 2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314 > >> (sofia/external/[email protected]) Running State Change CS_HANGUP > >> 2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration > >> Detected! Syncing Clock > >> > >> In this case an incoming call rang to both FS and Asterisk, Asterisk > >> picked > >> up, but the surge of activity made FS timer thread miss a beat or two. > >> > >> > >> eaf wrote: > >> > > >> > Oh, it's not just one timer thread... Why, why is sql_thread keeps on > >> > checking for messages every millisecond? Couldn't there be some > >> signalling > >> > implemented that will make the thread suspend on condition variable or > >> a > >> > socket/pipe in between? > >> > > >> > #0 do_sleep (t=1000) at src/switch_time.c:109 > >> > #1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) > >> at > >> > src/switch_core_sqldb.c:783 > >> > > >> > Why does this sofia_profile_worker_thread keeps on looping checking > for > >> > the queue? Have a semaphore! > >> > > >> > #0 do_sleep (t=1000) at src/switch_time.c:109 > >> > #1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30, > >> > obj=0x80f2490) at sofia.c:978 > >> > > >> > Nothing's happening on the box, but there are three threads that > >> pretend > >> > to be actively busy with smth. Others at least sleep for hundreds of > >> > milliseconds, not for one. > >> > > >> > And there is even infrastructure present to do blocking pops: i.e. why > >> > couldn't sqldb thread do queue_pop() instead of queue_trypop() > >> intermixed > >> > with 1ms sleeps? This looping is such a waste... > >> > > >> > > >> > eaf wrote: > >> >> > >> >> As I see it, switch_cond_next() currently is just a do_sleep(1000). > >> Yes, > >> >> it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" > >> >> overrides that. > >> >> > >> >> Yeah, there is a global timestamp... It's easy to workaround that for > >> RTP > >> >> who calls switch_micro_time_now()... But if somebody accesses > >> >> runtime.timestamp directly, it's gonna be tough to grep for that. If > >> only > >> >> this was C++... > >> >> > >> >> I'll play around. Never liked polling too much. Never could've > guessed > >> >> that polling could be so useful for scalability ;) My naive > >> >> implementation would've pulled timestamp via system calls and > would've > >> >> done sleeping by passing exact interval to select() instead of > syncing > >> >> with a pacing thread. Which would be dead-quiet at idle time, but, of > >> >> course, would stop scaling at some point due to excessive number of > >> >> system calls. > >> >> > >> >> Thanks. > >> >> > >> >> > >> >> Michael Jerris wrote: > >> >>> > >> >>> In short. No, you can not for many reasons. The milisecond tic is > >> >>> used throughout the code even when there is not any calls up. You > >> can > >> >>> grep for switch_cond_next if you would like to see where but it is > >> >>> required to keep our global timestamp and for pacing the scheduler > >> >>> among other services that run all the time. > >> >>> > >> >>> Mike > >> >>> > >> >>> On Dec 2, 2009, at 7:31 PM, eaf <[email protected]> wrote: > >> >>> > >> >>>> > >> >>>> Can I reduce resolution of that timer thread 10 times? I mean, I > >> >>>> glanced > >> >>>> through the code, and see that among others (are there others?) RTP > >> >>>> and IVR > >> >>>> set up their timers that are subsequently managed by this thread. > >> >>>> RTP timers > >> >>>> should be eliminated by that setting you've suggested. IVR timers > >> >>>> are set at > >> >>>> 20ms... So, if the thread is set to wake up every 10ms instead of > >> >>>> 1ms it > >> >>>> should be able to wake up those IVR timers just fine. Right? > >> >>>> > >> >>>> That's a cool design to have one dedicated thread that maintains > >> >>>> accurate > >> >>>> timing and then broadcasts via condition variables to hundreds of > >> >>>> other > >> >>>> threads events that they can register for. I'm sure it's one of the > >> >>>> reasons > >> >>>> why FS scales so much better than Asterisk. But for poor low-end > >> >>>> setups that > >> >>>> sit in the closet, eat only 6W of power and hardly ever run more > >> >>>> than two > >> >>>> calls at the same time, can I hack it somehow to be more UNIX- > >> >>>> friendly? I.e. > >> >>>> make it stuck in select() or recv() when there is nothing to do, > >> call > >> >>>> clock_gettime() right from the thread that wants and when it wants > >> >>>> to know > >> >>>> current time? > >> >>>> > >> >>>> Say, what if that thread is made to suspend on a condition variable > >> >>>> in case > >> >>>> if there are no timers registered in TIMER_MATRIX? Then, if some > >> other > >> >>>> thread comes up and adds its timer into the matrix, it could wake > up > >> >>>> the > >> >>>> timer thread and enjoy accurate timing as needed, on demand? And > in- > >> >>>> between > >> >>>> the calls, when there is no RTP or IVR, it will all go silent? I > >> mean, > >> >>>> sitting on a wait queue in the kernel is way better than go back > and > >> >>>> forth > >> >>>> incrementing counters that nobody even needs at the moment? > >> >>>> > >> >>>> > >> >>>> Anthony Minessale-2 wrote: > >> >>>>> > >> >>>>> idle is a 4 letter word to a realtime application. > >> >>>>> > >> >>>>> The core keeps a single high-priority thread to keep 1ms timing > and > >> >>>>> expands > >> >>>>> that broadcasting > >> >>>>> to hundreds or thousand of threads who need accurate timing. > >> >>>>> > >> >>>>> Your choppy audio is caused by linksys lying about the packet len > >> >>>>> that > >> >>>>> it's > >> >>>>> using and we set our timer > >> >>>>> to the wrong speed. > >> >>>>> > >> >>>>> > >> >>>> > >> >>>> -- > >> >>>> View this message in context: > >> >>>> > >> http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html > >> >>>> Sent from the Freeswitch-users mailing list archive at Nabble.com. > >> >>>> > >> >>>> > >> >>>> _______________________________________________ > >> >>>> FreeSWITCH-users mailing list > >> >>>> [email protected] > >> >>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > >> >>>> UNSUBSCRIBE: > http://lists.freeswitch.org/mailman/options/freeswitch- > >> >>>> users > >> >>>> http://www.freeswitch.org > >> >>> > >> >>> _______________________________________________ > >> >>> FreeSWITCH-users mailing list > >> >>> [email protected] > >> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > >> >>> UNSUBSCRIBE: > >> http://lists.freeswitch.org/mailman/options/freeswitch-users > >> >>> http://www.freeswitch.org > >> >>> > >> >>> > >> >> > >> >> > >> > > >> > > >> > >> -- > >> View this message in context: > >> http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html > >> Sent from the Freeswitch-users mailing list archive at Nabble.com. > >> > >> > >> _______________________________________________ > >> FreeSWITCH-users mailing list > >> [email protected] > >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > >> UNSUBSCRIBE: > http://lists.freeswitch.org/mailman/options/freeswitch-users > >> http://www.freeswitch.org > >> > > > > > > > > -- > > Anthony Minessale II > > > > FreeSWITCH http://www.freeswitch.org/ > > ClueCon http://www.cluecon.com/ > > Twitter: http://twitter.com/FreeSWITCH_wire > > > > AIM: anthm > > MSN:[email protected] <msn%[email protected]>< > msn%[email protected]<msn%[email protected]> > > > > GTALK/JABBER/PAYPAL:[email protected]<paypal%[email protected]> > <paypal%[email protected]<paypal%[email protected]> > > > > IRC: irc.freenode.net #freeswitch > > > > FreeSWITCH Developer Conference > > sip:[email protected] <sip%[email protected]>< > sip%[email protected]<sip%[email protected]> > > > > iax:[email protected]/888 > > googletalk:[email protected]<googletalk%3aconf%[email protected]> > <googletalk%3aconf%[email protected]<googletalk%253aconf%[email protected]> > > > > pstn:213-799-1400 > > > > _______________________________________________ > > FreeSWITCH-users mailing list > > [email protected] > > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users > > http://www.freeswitch.org > > > > > > -- > View this message in context: > http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html > Sent from the Freeswitch-users mailing list archive at Nabble.com. > > > _______________________________________________ > FreeSWITCH-users mailing list > [email protected] > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users > http://www.freeswitch.org > -- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:[email protected] <msn%[email protected]> GTALK/JABBER/PAYPAL:[email protected]<paypal%[email protected]> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:[email protected] <sip%[email protected]> iax:[email protected]/888 googletalk:[email protected]<googletalk%3aconf%[email protected]> pstn:213-799-1400
_______________________________________________ FreeSWITCH-users mailing list [email protected] http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
