Re: [opensource-dev] Client Plugin System Design

[Morgaine]

Ricky, I'm not sure even where to begin, because your proposed solution is
simply direct embedding of one or more scripting language systems into the
address space of the viewer.  This ignores every single advantage of
process-separated plugins and replaces them with a catalog of problems that
could fill a book.

Direct embedding is what everyone does when they want a simplistic
sequential scripting facility for configuring or testing a host
application.  We haven't proposed it ourselves not because we've had a brain
lapse and can't see the simplest solution, but because the simplest solution
is a complete disaster when misapplied as a general scripting facility for
user-written extensions.  I regularly embed Lua into host programs your way,
but using that approach here would have so many disadvantages and so few
advantages that I have avoided it intentionally.

I'll try to summarize some key issues for you, although this will leave much
unsaid.  Let's call the conventional approach Direct Language Embedding, or
*DLE* for brevity.


   - Unlike process-based plugins, DLE normally provides no concurrent
   execution.  This means that if you want your plugins to execute concurrently
   you will have to implement either multitasking or threading in the viewer,
   both of which involve shared-state multiprogramming.


   - Shared-state multiprogramming is hard and error-prone because of race
   conditions, non-determinacy, atomicity and locking, transactional integrity,
   as well as deadlock, livelock, and various other highly unfriendly
   critters.  I'm not going to justify this further here as it would take too
   long, but I did my research in this area, and anyone who thinks it's not
   hard needs to study the topic another 10 years.  A reliable solution
   requires operating system quality design, so this job really needs to be
   left to the operating system,


   - Because shared-state multiprogramming is hard, the execution engine for
   client-side scripts would be large, complex, and very hard to get right.  It
   would be literally a multi-year effort to make it work correctly, and it
   would always be hanging by a thread because full-coverage testing is very
   difficult owing to state complexity and timing dependencies.


   - In contrast to DLE, process-based plugins would not require any
   multiprogramming in the viewer at all.  The C++ host side can be completely
   sequential in its processing of messages arriving from plugins, while the
   plugins themselves would run with full concurrency courtesy of the operating
   system.  This makes for simple, reliable, and determinist viewer code.


   - DLE would add the entire code and data space of each embedded language
   runtime into the viewer's address space.  This is bloat with a capital 'B',
   and because virtually every language has a different embedding mechanism
   into the hosting C++ code, there will be very little code reuse.
   Application stability is inversely proportional to application size, so
   bloating the viewer is asking for trouble, not to mention large maintenance
   costs.


   - Because of the preceding issue, in practice DLE will never allow more
   than one or two different language runtimes to be integrated into the
   viewer, and hence a language-agnostic approach to client-side scripting is
   effectively barred.


   - Without language agnosticism, a small proportion of users will be
   favored by the chosen language or runtime while a larger proportion will be
   disadvantaged and their primary skills will not be harnessed.  This is a
   waste of people's skills, time and productivity.


   - Without language agnosticism, you are stuck with the facilities
   provided by the chosen languages or runtime.  If another language has a
   great feature that is highly appropriate for a particular plugin task, too
   bad, you cannot use it.  This is not very empowering, and it's an
   unnecessary restriction.


   - With DLE, the language runtimes would be directly *linked* with the
   viewer code, and hence licensing restrictions would come into play.  This
   would add a whole new rat's nest of problems and effectively make embedding
   of many useful languages impossible, whereas the license barrier provided by
   sockets avoids this issue entirely.


   - Because all plugins run in the address space of the viewer in DLE, the
   opportunity for programs to interfere with each other or with the viewer is
   large.  The viewer's stability drops to that of its least stable plugin.


   - With DLE, security is a nightmare.  Every 3rd party plugin can
   potentially alter something in the viewer unless it runs sandboxed, but if
   it runs sandboxed then it cannot provide much (or any) local interfacing, so
   the power of such plugins is weak.  Fast C/C++ plugins are almost out of the
   question for this reason.


   - Even if low inherent power is accepted so that sandboxing can provide
   security, this sandboxing can

[opensource-dev] SLPlugin lagging my viewer like crazy, maybe it was a bad idea from the start?

[Lance Corrimal]

Hey folks,


I've been noticing the following behaviour (on linux):

after some time logged i on snowglobe (latest svn checkout of 2009/trunk) I 
find that there are several copies of SLPlugin running in the background, and 
each of them loaded every single browser plugin installed on my system.
I doubt that I have come across prims that have flash or pdf files on urls on 
them (yet), since the general population has only just started playing with 
that.

Anyhow, the result of all those (useless?) processes in the background is 
heavy swapping, manifesting itself in "lag like hell" (framerate drops from 
around 25 to 1.5).
A relog fixes it (of course), but only for a short time.

...Add the possibility of all those nasty flash exploits running wild, and the 
whole "Fully interactive html on a prim" idea suddenly has a slightly rotten 
smell for me... which gets seriously stronger when I look at viewer 2.0 with 
its "html on an attachment irregardless of parcel media settings".

Anyways, shouldn't SLPlugin exit when it is done doing what it thought it 
should be doing?


bye,
LC
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Client Plugin System Design

[Rob Nelson]

On Sun, 2010-03-07 at 18:19 -0800, Ricky wrote:
> So far, barring any LL concepts, we have (as far as I know so far!)
> two designs of plugin system:
> 1: Socket-based plugins - as suggested by Morgaine.
> 2: D-Bus or similar existing IPC tool.
> 3: C++ Dynamically Shared Objects - my suggestion.

You forgot interpreted scripts, such as Lua.  Already implemented and
working in the Luna viewer.  A lot easier to develop for, as well, since
users don't need to have an API to use it. All they need to know is how
to make it work and where to dump their files.  

The largest issue is sandboxing and threading, actually.  I still have
to make an override for the io.* functions that will not access
materials outside of lua/data or the plugin's folder, and creating a
working event queue is becoming a major pain in the neck.

Fred Rookstown
Luna Viewer (http://luna-viewer.googlecode.com)

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] SLPlugin lagging my viewer like crazy , maybe it was a bad idea from the start?

[Opensource Obscure]

I got this too, but in my case, "system plugins" 
didn't get loaded.

Quicker workaround for restoring performances:
kill -9 the single processes, without need to
relog. As said - workaround, not solution.

I'm not competent enough to give you a better reply,
but this was already discussed here a few weeks ago,
there are 1 or 2 JIRA entries about it.

I don't see this happening on Viewer 2, BTW, 
so I guess a fix for Snowglobe exists as well.

HTH,
opensource obscure
--
LOL(65,43,21)
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] SLPlugin lagging my viewer like crazy, maybe it was a

[Lance Corrimal]

Am Montag, 8. März 2010 13:15:59 schrieb Opensource Obscure:


sounds like killall -9 SLPlugin

> I'm not competent enough to give you a better reply,
> but this was already discussed here a few weeks ago,
> there are 1 or 2 JIRA entries about it.
> 
> I don't see this happening on Viewer 2, BTW,
> so I guess a fix for Snowglobe exists as well.

found them... SNOW-318 and SNOW-416... neither one has a patch, or a reference 
to how that was fixed in viewer 2.0
Ah well, guess it's time to import my 2.0 svn snapshot into kdevelop and poke 
around.
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Client Plugin System Design

[Tigro Spottystripes]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Would there be an approach that would allow both all the benefits you
mention, plus letting the process based plugins have access to the bare
metal performance? (somthing like a way to have the plugins instruct the
client to add the desired processing routines to it's per frame cycle or
whatever) Perhaps an hybrid approach, where with secure sockets
communication and perhaps user intervention (a dialog to grant or deny
access to an specific plugin) the external plugins could interface with
the client guts like DLE plugins?

Actually, how much is the performance loss if the plugins aren't of the
DLE kind?

On 8/3/2010 05:22, Morgaine wrote:
> Ricky, I'm not sure even where to begin, because your proposed solution
> is simply direct embedding of one or more scripting language systems
> into the address space of the viewer.  This ignores every single
> advantage of process-separated plugins and replaces them with a catalog
> of problems that could fill a book.
> 
> Direct embedding is what everyone does when they want a simplistic
> sequential scripting facility for configuring or testing a host
> application.  We haven't proposed it ourselves not because we've had a
> brain lapse and can't see the simplest solution, but because the
> simplest solution is a complete disaster when misapplied as a general
> scripting facility for user-written extensions.  I regularly embed Lua
> into host programs your way, but using that approach here would have so
> many disadvantages and so few advantages that I have avoided it
> intentionally.
> 
> I'll try to summarize some key issues for you, although this will leave
> much unsaid.  Let's call the conventional approach Direct Language
> Embedding, or *DLE* for brevity.
> 
> * Unlike process-based plugins, DLE normally provides no concurrent
>   execution.  This means that if you want your plugins to execute
>   concurrently you will have to implement either multitasking or
>   threading in the viewer, both of which involve shared-state
>   multiprogramming.
> 
> * Shared-state multiprogramming is hard and error-prone because of
>   race conditions, non-determinacy, atomicity and locking,
>   transactional integrity, as well as deadlock, livelock, and
>   various other highly unfriendly critters.  I'm not going to
>   justify this further here as it would take too long, but I did my
>   research in this area, and anyone who thinks it's not hard needs
>   to study the topic another 10 years.  A reliable solution requires
>   operating system quality design, so this job really needs to be
>   left to the operating system,
> 
> * Because shared-state multiprogramming is hard, the execution
>   engine for client-side scripts would be large, complex, and very
>   hard to get right.  It would be literally a multi-year effort to
>   make it work correctly, and it would always be hanging by a thread
>   because full-coverage testing is very difficult owing to state
>   complexity and timing dependencies.
> 
> * In contrast to DLE, process-based plugins would not require any
>   multiprogramming in the viewer at all.  The C++ host side can be
>   completely sequential in its processing of messages arriving from
>   plugins, while the plugins themselves would run with full
>   concurrency courtesy of the operating system.  This makes for
>   simple, reliable, and determinist viewer code.
> 
> * DLE would add the entire code and data space of each embedded
>   language runtime into the viewer's address space.  This is bloat
>   with a capital 'B', and because virtually every language has a
>   different embedding mechanism into the hosting C++ code, there
>   will be very little code reuse.  Application stability is
>   inversely proportional to application size, so bloating the viewer
>   is asking for trouble, not to mention large maintenance costs.
> 
> * Because of the preceding issue, in practice DLE will never allow
>   more than one or two different language runtimes to be integrated
>   into the viewer, and hence a language-agnostic approach to
>   client-side scripting is effectively barred.
> 
> * Without language agnosticism, a small proportion of users will be
>   favored by the chosen language or runtime while a larger
>   proportion will be disadvantaged and their primary skills will not
>   be harnessed.  This is a waste of people's skills, time and
>   productivity.
> 
> * Without language agnosticism, you are stuck with the facilities
>   provided by the chosen languages or runtime.  If another language
>   has a great feature that is highly appropriate for a particular
>   plugin task, too bad, you cannot use it.  This is not very
>   empowering, and it's an unnecessary restriction.
> 
> * With DLE, the language runtimes would be d

Re: [opensource-dev] SLPlugin lagging my viewer like crazy, maybe it was a

[Dickson, Mike (ISS Software)]

I've seen this behaviour as well in Viewer 2 on a Fedora 12 x64 system.

Mike

-Original Message-
From: opensource-dev-boun...@lists.secondlife.com 
[mailto:opensource-dev-boun...@lists.secondlife.com] On Behalf Of Lance Corrimal
Sent: Monday, March 08, 2010 6:47 AM
To: opensource-dev@lists.secondlife.com
Subject: Re: [opensource-dev] SLPlugin lagging my viewer like crazy, maybe it 
was a

Am Montag, 8. März 2010 13:15:59 schrieb Opensource Obscure:


sounds like killall -9 SLPlugin

> I'm not competent enough to give you a better reply,
> but this was already discussed here a few weeks ago,
> there are 1 or 2 JIRA entries about it.
> 
> I don't see this happening on Viewer 2, BTW,
> so I guess a fix for Snowglobe exists as well.

found them... SNOW-318 and SNOW-416... neither one has a patch, or a reference 
to how that was fixed in viewer 2.0
Ah well, guess it's time to import my 2.0 svn snapshot into kdevelop and poke 
around.
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Client Plugin System Design

[Morgaine]

Lua provides a fine example of Direct Language Embedding.  The language was
designed expressly for being embedded in a host application as well as for
being extended with calls to other languages.  It's no surprise then that it
is very clean and simple to use in this way.  I like Lua a lot.

Unfortunately, as you know from your own work, Rob, there is a big
difference between using an embedded language inline with its host
application's main or single thread of execution, and turning the language
into a concurrent execution environment that can run multiple user scripts
at the same time, which we need here.

As I explained briefly to Ricky, to achieve the latter requires a
multiprogrammed runtime to be created within the viewer, using either one of
the multitasking approaches such as preemptive timeslicing plus a scheduler
or alternatively coroutine-style cooperative yielding, or else using system
threads to run multiple language runtimes concurrently.  Whichever approach
is used, a concurrent runtime would have to be added to the viewer to handle
either the internal multitasking or the interations between system threads,
and on top of that, one would then add an API to the world-related aspects
of the viewer.  It would be a non-trivial job.

In the case of Lua, the clear winner for DLE is to spawn a system thread
running a Lua VM for every Lua user script that needs to run, and let the
system worry about executing them concurrently and scheduling between them.
That's easy, but much hard work remains because simply having N separate Lua
scripts running without interacting with the viewer is of no use at all, so
a communications system needs to be built within the viewer to manage its
interactions with these concurrent threads.  (Letting each Lua script access
any needed viewer code directly and concurrently through a simple C++
binding  is of course not viable --- the viewer is not internally
thread-safe, and locking calls gets us deep into concurrency horrors.)

I like using Lua as it is designed to be used, running inline with an
application's main thread of control (ie. no concurrency).  I use it for
flexible configuration and very easy testing of applications.  Lua is also
quite well suited for implementing large chunks of applications, everything
excluding those functions that need to run at bare-metal speed like
rendering.  For example, it would be perfectly viable for the entire 2D GUI
of the viewer to written as a suite of Lua scripts, each one invoked from
C++ only when the appropriate GUI element requires attention.  However,
these scripts would have to be short and non-blocking --- one would not have
the freedom of writing Lua scripts in the normal way with arbitrary loops
and extended calls to external facilities, and multiple Lua scripts would
not run concurrently.

This is why the process-based approach is being recommended.  It eliminates
the work of implementing a multiprogrammed environment for scripts, letting
the system do all the hard work (*really* hard work, because of
concurrency).  Instead, it requires socket handling, but that is quite short
and simple sequential code.  Depending on how one wishes to serialize
communications, the amount of work here can vary --- calling JSON
encode/decode routines is typically very easy, but handling buffering and
exceptions can introduce some significant work.

That's the in-viewer work overhead of this approach:  socket handling and
serialization.  It's a *lot* less than the work overhead in DLE, and the
flexibility that results from users being able to write plugins in any
language they want and calling any libraries they want and using normal
programming styles is a major win.


Morgaine.




===

On Mon, Mar 8, 2010 at 10:44 AM, Rob Nelson wrote:

> On Sun, 2010-03-07 at 18:19 -0800, Ricky wrote:
> > So far, barring any LL concepts, we have (as far as I know so far!)
> > two designs of plugin system:
> > 1: Socket-based plugins - as suggested by Morgaine.
> > 2: D-Bus or similar existing IPC tool.
> > 3: C++ Dynamically Shared Objects - my suggestion.
>
> You forgot interpreted scripts, such as Lua.  Already implemented and
> working in the Luna viewer.  A lot easier to develop for, as well, since
> users don't need to have an API to use it. All they need to know is how
> to make it work and where to dump their files.
>
> The largest issue is sandboxing and threading, actually.  I still have
> to make an override for the io.* functions that will not access
> materials outside of lua/data or the plugin's folder, and creating a
> working event queue is becoming a major pain in the neck.
>
> Fred Rookstown
> Luna Viewer (http://luna-viewer.googlecode.com)
>
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
_

Re: [opensource-dev] Client Plugin System Design

[Morgaine]

Yes, there is a way, Tigro.

In the Multi-Process
Clientdesign
that we started back when AWG was active, we loosely defined
"Facility Optimization", which would be done mainly through communication *
accelerators*.

The general idea is that if profiling shows that the job being done by a
socket-linked plugin is being held back by socket throughput or latency or
by serialization overheads, then an *accelerator *pathway can be added
between viewer and plugin.  In most cases this would involve setup of a
shared memory segment for exclusive use by the viewer and this one single
plugin.  Every plugin requiring such acceleration would have its own
"private" shared memory segment, shared only with the viewer, not with other
plugins.

Using shared memory means that socket and serialization overheads vanish
completely of course, since data transfer can be done by reference.  As
always though, there is a tradeoff:  the viewer is now involved in two-party
multiprogramming with respect to each accelerated plugin.  As anyone who has
worked in concurrency knows, here be dragons.

Nevertheless, it is perfectly viable for occasional, narrowly-defined high
speed tasks.  This would require special programming within the viewer, both
to manage shared memory segments and also to implement the by-reference
communications system, for instance using mailboxes or atomic variables.
While viable, it would not be easy, and one might wish to question (on a
case-by-case basis) whether dynamically loading a shared native code library
might not be more appropriate.

Still, such accelerators could be used if desired, and if they are
implemented *transparently* so that any plugin could have its socket pathway
accelerated on demand then this would be a major win for all data-intensive
plugins.


Morgaine.







On Mon, Mar 8, 2010 at 1:51 PM, Tigro Spottystripes <
tigrospottystri...@gmail.com> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Would there be an approach that would allow both all the benefits you
> mention, plus letting the process based plugins have access to the bare
> metal performance? (somthing like a way to have the plugins instruct the
> client to add the desired processing routines to it's per frame cycle or
> whatever) Perhaps an hybrid approach, where with secure sockets
> communication and perhaps user intervention (a dialog to grant or deny
> access to an specific plugin) the external plugins could interface with
> the client guts like DLE plugins?
>
> Actually, how much is the performance loss if the plugins aren't of the
> DLE kind?
>
> On 8/3/2010 05:22, Morgaine wrote:
> > Ricky, I'm not sure even where to begin, because your proposed solution
> > is simply direct embedding of one or more scripting language systems
> > into the address space of the viewer.  This ignores every single
> > advantage of process-separated plugins and replaces them with a catalog
> > of problems that could fill a book.
> >
> > Direct embedding is what everyone does when they want a simplistic
> > sequential scripting facility for configuring or testing a host
> > application.  We haven't proposed it ourselves not because we've had a
> > brain lapse and can't see the simplest solution, but because the
> > simplest solution is a complete disaster when misapplied as a general
> > scripting facility for user-written extensions.  I regularly embed Lua
> > into host programs your way, but using that approach here would have so
> > many disadvantages and so few advantages that I have avoided it
> > intentionally.
> >
> > I'll try to summarize some key issues for you, although this will leave
> > much unsaid.  Let's call the conventional approach Direct Language
> > Embedding, or *DLE* for brevity.
> >
> > * Unlike process-based plugins, DLE normally provides no concurrent
> >   execution.  This means that if you want your plugins to execute
> >   concurrently you will have to implement either multitasking or
> >   threading in the viewer, both of which involve shared-state
> >   multiprogramming.
> >
> > * Shared-state multiprogramming is hard and error-prone because of
> >   race conditions, non-determinacy, atomicity and locking,
> >   transactional integrity, as well as deadlock, livelock, and
> >   various other highly unfriendly critters.  I'm not going to
> >   justify this further here as it would take too long, but I did my
> >   research in this area, and anyone who thinks it's not hard needs
> >   to study the topic another 10 years.  A reliable solution requires
> >   operating system quality design, so this job really needs to be
> >   left to the operating system,
> >
> > * Because shared-state multiprogramming is hard, the execution
> >   engine for client-side scripts would be large, complex, and very
> >   hard to get right.  It would be lit

Re: [opensource-dev] Potential inventory problem?

[Nexii Malthus]

Happens just as well on any grid. I'm just as confused as anyone here what
this error message means, but this was brought up before and I vaguely
remember a linden saying that wasn't the source of the lag problems related
to inventory.

Are you trying to track down the texture picker freezing bug?

- Nexii

On Mon, Mar 8, 2010 at 12:45 AM, John Hurliman wrote:

> When logging into OpenSim with Snowglobe I see a lot (dozens) of these
> messages in the log file:
>
> 2010-03-08T00:33:18Z WARNING: LLInventoryPanel::modelChanged:
> d24c5c7b-4d64-45eb-9286-91fb0f307b24 is in model and in view, but STRUCTURE
> flag not set
>
> What does this message mean? Am I sending inventory items in the wrong
> order, or with missing data? I'm trying to track down a client freezing bug
> in Snowglobe and the official viewer that might be related to inventory, and
> found this.
>
> John
>
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Management Algorithm

[Kelly Linden]

We are not out to write a new malloc for mono.  What we have is a system
that throws an exception when the memory used by the script hits a certain
threshold (64k currently).  This exception is caught so we can "crash" the
script.  The future small scripts and big scripts projects will add new
functions to set and get this threshold value, allowing scripters to
effectively control how much memory is reserved for their script.  We will
continue to use mono's default memory management within the reserved memory
thresholds.  It is a much simpler problem to solve.

 - Kelly

On Sun, Mar 7, 2010 at 5:50 AM, Carlo Wood  wrote:

> Lets assume that the *average* script uses
> 8 to 16 kB of real memory. LL's design allocates
> 64 kB regardless, leading to an overhead of
> 400 to 800% (meaning they need to buy 5 to
> 9 times the memory that is theoretically needed).
>
> In that light, I gave it some more thought, and
> think something as complex as my rmalloc's algorithm,
> with it's 19% overhead, isn't needed (note that
> it's both faster than gmalloc as well as three
> times more efficient; so complexity is not always
> a bad thing ;).
>
> Nevertheless, in this case, since the scripts
> use a maximum of 64 kB, you can use the
> following design:
>
> Let each allocated block be a power of 2
> kilobytes, with a maximum of 64 kB (and a
> minimum of 1 KB, or 4 if even the tiniest
> script needs that).
>
> It is easy to see that this would lead
> to an overhead of 25% on average per
> allocated block.
>
> We'd still have to deal with "holes" of a
> full 64 kB where blocks became totally
> unused, but normally scripts in objects are
> started all at once when a sim reboots, and
> only seldom stopped. The scripts that will
> most likely attribute to random starting
> and stopping of scripts will be the scripts
> in attachments. A worst case scenario would
> be one where there are 50 avies in a sim
> (during a meeting), then a new avie enters
> with some scripts which need to be allocated
> at the top of the heap; then the previous
> 50 avies leave. That would create a hole
> in the heap of the size of all the scripts
> of those 50 avies. If script memory would
> be relocatable, then this problem doesn't
> exist of course. I can't simply estimate
> the average overhead caused by this, but
> because the algorithm described here is
> basically the one used by gmalloc (which
> in my test used 62% overhead) I'm pretty
> sure that it will be less than -say- 100%
> overhead; still 4 to 8 times more efficient
> than the current design on the table.
>
> The API for this design would be something
> like the following:
>
> namespace script_memory_management {
>
> void* ll_sbrk(ptrdiff_t);   // Increment the size of the heap
> int   ll_brk(void*);// Set the size of the heap explicitely
>
> void* ll_malloc64(void);// Get a new 64 kB block.
> void  ll_free64(void*); // Free such a block.
>
> void* ll_malloc(size_t s);  // Allocate s bytes of memory for a script.
> void  ll_free(void*);   // Free it again.
>
> ...
>
> Assuming here that scripts cannot deal with
> relocation, otherwise one should also have:
>
> void* ll_realloc(size_t s); // Allocate a different size of memory.
>
>
> ll_malloc then will round the number of requested bytes up
> to the nearest power of 2 (kBs) and retrieve a block from one
> of the free lists (maintained for 32, 16, 8, 4, 2 and 1 kB)
> (note that if scripts only seldom use 1 or 2 kB it might
> be more efficient to use a minimum of 2 or 4 kB instead of 1).
>
> Each 64 kB block would contain either one 64 kB allocation,
> or two 32 kB block allocations, or four 16 kB block allocations,
> etc, but never allocations of mixed sizes, thus making it
> easy to find a free block of such size.
>
> A free list is usually maintained by adding pointers inside
> the (unused) memory blocks, linking them together to a linked
> list of free memory blocks of a given size. That causes allocating
> to be instant, but freeing memory requires to traverse this
> list in order to update it's pointers. With the number of
> scripts that normally run in a sim this will not be a problem.
>
> --
> Carlo Wood 
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Client Plugin System Design

[Lawson English]

Morgaine wrote:
>
>
> PS. With regards to "Networking code in every plugin just to connect 
> to the client", networking is made available by the operating system 
> to every process through system calls or system subroutines, ie. the 
> thinnest interface possible.  There is no bloat or overhead involved.  
> Particular languages sometimes pretty up the system interface a 
> little, but these bindings do not normally introduce any significant 
> overhead. Throughput and latency of socket communications is not a 
> significant issue either --- I've measured them in an environment 
> which emulated this design pattern, and the level of performance might 
> even suit some rendering tasks.
>

When extra speed is needed, the shared memory design of the media plugin 
can be used to augment/replace socket connections.  The only caveat 
there is that each OS has a limit on how many shared buffers can be used 
per process/machine.

On Mac OS X, without some superuser tweaking and maybe a restart, it's 8 
per process and 32 per machine. Other *nix have more I think and I have 
no idea what Windows allows. Regardless, its something that should be 
used sparingly.


Lawson





___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Management Algorithm

[Michael Schlenker]

Am 08.03.2010 um 18:46 schrieb Kelly Linden:

> We are not out to write a new malloc for mono.  What we have is a system that 
> throws an exception when the memory used by the script hits a certain 
> threshold (64k currently).  This exception is caught so we can "crash" the 
> script.  The future small scripts and big scripts projects will add new 
> functions to set and get this threshold value, allowing scripters to 
> effectively control how much memory is reserved for their script.  We will 
> continue to use mono's default memory management within the reserved memory 
> thresholds.  It is a much simpler problem to solve.
> 
While your at it, how about a static analyzer for mono/LSL that determines 
guaranteed lowest memory consumption for a script and sets the threshold there. 

That would have the benefit of easing scripters work by providing useful 
defaults for all the easy cases without them having to do anything at all. 

The scheme should only break down if the mono GC behaves weird. In that case 
scripters have a huge problem anyway as they cannot set any threshold without 
being crashed at random by a lazy GC.

Michael
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Potential inventory problem?

[John Hurliman]

Yes, I'm trying to push out the first release of a set of PHP grid services
for OpenSim and the texture picker freeze bug is a showstopper. It obviously
doesn't happen on SLGrid and it wasn't happening in the past with OpenSim. A
packet trace didn't show anything interesting (doesn't appear to be
triggered by a specific packet at all). My current guess is that the client
is making a bad assumption about information the server sent earlier on. It
might have to do with the library since I am not sending a library skeleton
at the moment.

Any other clues on the freezing issue would be much appreciated. The warning
is more of a curiosity but it could uncover another hidden expectation the
open source server software is not meeting.

John

On Mon, Mar 8, 2010 at 9:21 AM, Nexii Malthus  wrote:

> Happens just as well on any grid. I'm just as confused as anyone here what
> this error message means, but this was brought up before and I vaguely
> remember a linden saying that wasn't the source of the lag problems related
> to inventory.
>
> Are you trying to track down the texture picker freezing bug?
>
> - Nexii
>
> On Mon, Mar 8, 2010 at 12:45 AM, John Hurliman wrote:
>
>> When logging into OpenSim with Snowglobe I see a lot (dozens) of these
>> messages in the log file:
>>
>> 2010-03-08T00:33:18Z WARNING: LLInventoryPanel::modelChanged:
>> d24c5c7b-4d64-45eb-9286-91fb0f307b24 is in model and in view, but STRUCTURE
>> flag not set
>>
>> What does this message mean? Am I sending inventory items in the wrong
>> order, or with missing data? I'm trying to track down a client freezing bug
>> in Snowglobe and the official viewer that might be related to inventory, and
>> found this.
>>
>> John
>>
>> ___
>> Policies and (un)subscribe information available here:
>> http://wiki.secondlife.com/wiki/OpenSource-Dev
>> Please read the policies before posting to keep unmoderated posting
>> privileges
>>
>
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Client Plugin System Design

[Morgaine]

Indeed, Lawson.

You may have seen it, but I provided a little more detail about accelerating
plugin communications using Shared Memory segments in my
answerto
Tigro.

In practice it's not easy to come up with an SM-accelerated path mechanism
that preserves the semantics of communications on the socket path
transparently, so I expect that these two paths would be kept separate.

It wouldn't be too hard to implement an SM-based path into the viewer in
parallel with the socket-based channel.  I expect that most of the design
battles would be about what shared data model to use.  A sequential mailbox
carrying by-reference payloads?  A DOM-like structure?  Locking with strong
conventions?  Lock-free update techniques?  Software transactional memory?
Retro semaphores and condition variables?  Single assignment and no mutable
state?

Unfortunately all of these mechanisms have disadvantages and dangers ---
there is no silver bullet in this area.  The only way to win the SM
multiprogramming game is to not play it, which is where we came in: socket
messaging with no shared state.  It's unfortunate that, occasionally, we
have to break this rule for performance reasons.


Morgaine.




=

On Mon, Mar 8, 2010 at 6:40 PM, Lawson English  wrote:

> Morgaine wrote:
>
>>
>>
>> PS. With regards to "Networking code in every plugin just to connect to
>> the client", networking is made available by the operating system to every
>> process through system calls or system subroutines, ie. the thinnest
>> interface possible.  There is no bloat or overhead involved.  Particular
>> languages sometimes pretty up the system interface a little, but these
>> bindings do not normally introduce any significant overhead. Throughput and
>> latency of socket communications is not a significant issue either --- I've
>> measured them in an environment which emulated this design pattern, and the
>> level of performance might even suit some rendering tasks.
>>
>>
> When extra speed is needed, the shared memory design of the media plugin
> can be used to augment/replace socket connections.  The only caveat there is
> that each OS has a limit on how many shared buffers can be used per
> process/machine.
>
> On Mac OS X, without some superuser tweaking and maybe a restart, it's 8
> per process and 32 per machine. Other *nix have more I think and I have no
> idea what Windows allows. Regardless, its something that should be used
> sparingly.
>
>
> Lawson
>
>
>
>
>
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Client Plugin System Design

[Argent Stonecutter]

On 2010-03-08, at 10:06, Morgaine wrote:
> Unfortunately, as you know from your own work, Rob, there is a big  
> difference between using an embedded language inline with its host  
> application's main or single thread of execution, and turning the  
> language into a concurrent execution environment that can run  
> multiple user scripts at the same time, which we need here.

You could use the old fashioned real-time control system approach,  
make it event-driven (like LSL), and simply say that each script has  
to run to completion each time through the event loop... and outlaw  
long running event handlers.

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Snowglobe as an mixed reality platform

[Philippe (Merov) Bossut]

Hi Morgaine,

I'm surprised and disappointed you're reading so much into this. Personal
(and irrelevant) feelings aside, my answer was motivated strictly by the
following:
- There is an already existing set of code with experimental results and it
looks very nice indeed.
- There's a chance all this good work might get lost forever if it doesn't
find a host.
- According to Tuomas, it's relatively isolated on the fringe of the code
and doesn't need rearchitecture.
- The Snowglobe community is certainly the best qualified to look at this
code and see how we could merge it if at all.

Based on that, the minimum we can do as a community is to show respect for
the work done by Tuomas and his group and welcome them. I see no reason to
turn them down and you haven't provided any BTW.

Note that I'm not offering to merge the project "as is" but lay out some
steps for Tuomas to take so that we can collectively study that code,
namely: log something in JIRA and attach a patch so we can look at the code.

I hope others on the list feel the same and that we'll find some enthusiast
folks to look into Tuomas project and see if and how it can fit into
Snowglobe (hint: I'm unlikely to have the bandwidth to do this). The fact
that it might need preliminary work being done on the client-scripting side
of things is, IMO, a plus (give us something tangible to gauge our
thinking).

Regards,
- Merov
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] SLPlugin lagging my viewer like crazy, maybe it was a

[Tayra Dagostino]

On Mon, 8 Mar 2010 13:46:32 +0100
Lance Corrimal  wrote:

> Am Montag, 8. März 2010 13:15:59 schrieb Opensource Obscure:
> 
> 
> sounds like killall -9 SLPlugin

if u killall all plugins stop to work, keep first child active, it fork
other child sometime for some strange reason one of plugin begin to
waste cpu and memory

i don't understand why if 2 plugin are declared in mimetype defualt
file from 4 to 6 child spawned isn't unusefull? i think is better
one plugin executed per type or in a mixed land wioth a lot of media a
new powerfull pc will be killed anyway
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Potential inventory problem?

[Rob Nelson]

Hahaha, I remember back when Open Grid Services was written in PHP;  I
still have it on my webserver somewhere.  I think I submitted a patch to
you guys several years ago (regarding a SQL injection exploit) where I
accidentally left in a bunch of cursing and racial slurs.

If I contribute again, I think I'll be a little more careful.

Also, I'll take a peek into what's causing that problem;  I'm having to
redo the UI in Lua anyway, so I may as well figure out what is causing
your issues.

Fred Rookstown

On Mon, 2010-03-08 at 11:17 -0800, John Hurliman wrote:
> Yes, I'm trying to push out the first release of a set of PHP grid
> services for OpenSim and the texture picker freeze bug is a
> showstopper. It obviously doesn't happen on SLGrid and it wasn't
> happening in the past with OpenSim. A packet trace didn't show
> anything interesting (doesn't appear to be triggered by a specific
> packet at all). My current guess is that the client is making a bad
> assumption about information the server sent earlier on. It might have
> to do with the library since I am not sending a library skeleton at
> the moment.
> 
> Any other clues on the freezing issue would be much appreciated. The
> warning is more of a curiosity but it could uncover another hidden
> expectation the open source server software is not meeting.
> 
> John
> 
> On Mon, Mar 8, 2010 at 9:21 AM, Nexii Malthus 
> wrote:
> Happens just as well on any grid. I'm just as confused as
> anyone here what this error message means, but this was
> brought up before and I vaguely remember a linden saying that
> wasn't the source of the lag problems related to inventory.
> 
> 
> Are you trying to track down the texture picker freezing bug?
> 
> 
> - Nexii
> 
> 
> On Mon, Mar 8, 2010 at 12:45 AM, John Hurliman
>  wrote:
> 
> 
> When logging into OpenSim with Snowglobe I see a lot
> (dozens) of these messages in the log file:
> 
> 2010-03-08T00:33:18Z WARNING:
> LLInventoryPanel::modelChanged:
> d24c5c7b-4d64-45eb-9286-91fb0f307b24 is in model and
> in view, but STRUCTURE flag not set
> 
> What does this message mean? Am I sending inventory
> items in the wrong order, or with missing data? I'm
> trying to track down a client freezing bug in
> Snowglobe and the official viewer that might be
> related to inventory, and found this.
> 
> John
> 
> 
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep
> unmoderated posting privileges
> 
> 
> 
> 
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting privileges


___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

[opensource-dev] SnowGlobe/viewer 2 streaming media issue with new prim textures

[Lawson English]

https://jira.secondlife.com/browse/SNOW-566

This is a reference to: https://jira.secondlife.com/browse/VWR-17151


There are 14 votes for VWR-17151 in pJIRA, but I can't find a 
reference/link to it in any SnowGlobe jira and the issue shows up in my 
hand-built SnowGlobe 2 viewer, AND to some extent, in the viewer plugin 
test app bundled with SG2.


Lawson

___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Snowglobe as an mixed reality platform

[Nexii Malthus]

Hey,

Been waiting on this a while. Has there been any progress been made Tuomas?
As Merov already mentioned we're willing to give help.

If you need a place to host documentation and/or code;
- You can use the wiki to create a page for documentation over @
https://wiki.secondlife.com/wiki/Open_Source_Portal
- You can also create a
JIRA with a patch submission @ jira.secondlife.com/ which we'll be able to
review and help fix up.

This mailing list as well as with the help of combination of a wiki page we
could get up to speed on a design.

I'm sure there are many eager developers here who'd chime in about getting
this project alive and kicking.

- Nexii

On Mon, Mar 8, 2010 at 9:42 PM, Philippe (Merov) Bossut  wrote:

> Hi Morgaine,
>
> I'm surprised and disappointed you're reading so much into this. Personal
> (and irrelevant) feelings aside, my answer was motivated strictly by the
> following:
> - There is an already existing set of code with experimental results and it
> looks very nice indeed.
> - There's a chance all this good work might get lost forever if it doesn't
> find a host.
> - According to Tuomas, it's relatively isolated on the fringe of the code
> and doesn't need rearchitecture.
> - The Snowglobe community is certainly the best qualified to look at this
> code and see how we could merge it if at all.
>
> Based on that, the minimum we can do as a community is to show respect for
> the work done by Tuomas and his group and welcome them. I see no reason to
> turn them down and you haven't provided any BTW.
>
> Note that I'm not offering to merge the project "as is" but lay out some
> steps for Tuomas to take so that we can collectively study that code,
> namely: log something in JIRA and attach a patch so we can look at the code.
>
> I hope others on the list feel the same and that we'll find some enthusiast
> folks to look into Tuomas project and see if and how it can fit into
> Snowglobe (hint: I'm unlikely to have the bandwidth to do this). The fact
> that it might need preliminary work being done on the client-scripting side
> of things is, IMO, a plus (give us something tangible to gauge our
> thinking).
>
> Regards,
> - Merov
>
>
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Snowglobe as an mixed reality platform

[Morgaine]

I would wish everything to be as honest and open as you've described it,
Merov.

The problem is that the open source / TPV community has been battered around
the head too much recently (by factors outside of your control, I'm sure),
and when the answer to everything seems to be "No" except when it involves a
corporate partner, it doesn't lead to comfortable thoughts.

The situation has been *SO* bad recently that even small hints like your "*it
might need preliminary work being done on the client-scripting side of
things*" are a significant win --- that's how depressed the situation is
currently.  Yet notice that you don't even state that the community will
have any say in it --- your remarks are entirely consistent with Firefly
continuing as a secret internal project and all community requirements being
ignored.  "Must try harder."

Hopefully this will be remedied, and the spirit of cooperation rekindled.
Client-side scripting is a bit of a weather vane for how well relations are
progressing, and it's quite easy to turn thing around.  It just needs some
good will to be shown, and open dialog to return.

There was going to be a new openness after Viewer 2.0 was released, wasn't
there?  We heard that said multiple times around the end of 2009.  It still
remains to appear though.


Morgaine.





=

On Mon, Mar 8, 2010 at 9:42 PM, Philippe (Merov) Bossut  wrote:

> Hi Morgaine,
>
> I'm surprised and disappointed you're reading so much into this. Personal
> (and irrelevant) feelings aside, my answer was motivated strictly by the
> following:
> - There is an already existing set of code with experimental results and it
> looks very nice indeed.
> - There's a chance all this good work might get lost forever if it doesn't
> find a host.
> - According to Tuomas, it's relatively isolated on the fringe of the code
> and doesn't need rearchitecture.
> - The Snowglobe community is certainly the best qualified to look at this
> code and see how we could merge it if at all.
>
> Based on that, the minimum we can do as a community is to show respect for
> the work done by Tuomas and his group and welcome them. I see no reason to
> turn them down and you haven't provided any BTW.
>
> Note that I'm not offering to merge the project "as is" but lay out some
> steps for Tuomas to take so that we can collectively study that code,
> namely: log something in JIRA and attach a patch so we can look at the code.
>
> I hope others on the list feel the same and that we'll find some enthusiast
> folks to look into Tuomas project and see if and how it can fit into
> Snowglobe (hint: I'm unlikely to have the bandwidth to do this). The fact
> that it might need preliminary work being done on the client-scripting side
> of things is, IMO, a plus (give us something tangible to gauge our
> thinking).
>
> Regards,
> - Merov
>
>
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Potential inventory problem?

[Nexii Malthus]

It does happen on the SL grid. At least it did for a long while, I have
admit I use the texture picker not that often these days, but it has burned
me pretty badly.

How bad is the freeze? How long is it?

Edit: Found this JIRA, might be relevant to the problem I think, the freeze
bug used to exist in 1.21 and 1.22 and was fixed with 1.23.
http://jira.secondlife.com/browse/VWR-8818


- Nexii

On Mon, Mar 8, 2010 at 10:33 PM, Rob Nelson wrote:

> Hahaha, I remember back when Open Grid Services was written in PHP;  I
> still have it on my webserver somewhere.  I think I submitted a patch to
> you guys several years ago (regarding a SQL injection exploit) where I
> accidentally left in a bunch of cursing and racial slurs.
>
> If I contribute again, I think I'll be a little more careful.
>
> Also, I'll take a peek into what's causing that problem;  I'm having to
> redo the UI in Lua anyway, so I may as well figure out what is causing
> your issues.
>
> Fred Rookstown
>
> On Mon, 2010-03-08 at 11:17 -0800, John Hurliman wrote:
> > Yes, I'm trying to push out the first release of a set of PHP grid
> > services for OpenSim and the texture picker freeze bug is a
> > showstopper. It obviously doesn't happen on SLGrid and it wasn't
> > happening in the past with OpenSim. A packet trace didn't show
> > anything interesting (doesn't appear to be triggered by a specific
> > packet at all). My current guess is that the client is making a bad
> > assumption about information the server sent earlier on. It might have
> > to do with the library since I am not sending a library skeleton at
> > the moment.
> >
> > Any other clues on the freezing issue would be much appreciated. The
> > warning is more of a curiosity but it could uncover another hidden
> > expectation the open source server software is not meeting.
> >
> > John
> >
> > On Mon, Mar 8, 2010 at 9:21 AM, Nexii Malthus 
> > wrote:
> > Happens just as well on any grid. I'm just as confused as
> > anyone here what this error message means, but this was
> > brought up before and I vaguely remember a linden saying that
> > wasn't the source of the lag problems related to inventory.
> >
> >
> > Are you trying to track down the texture picker freezing bug?
> >
> >
> > - Nexii
> >
> >
> > On Mon, Mar 8, 2010 at 12:45 AM, John Hurliman
> >  wrote:
> >
> >
> > When logging into OpenSim with Snowglobe I see a lot
> > (dozens) of these messages in the log file:
> >
> > 2010-03-08T00:33:18Z WARNING:
> > LLInventoryPanel::modelChanged:
> > d24c5c7b-4d64-45eb-9286-91fb0f307b24 is in model and
> > in view, but STRUCTURE flag not set
> >
> > What does this message mean? Am I sending inventory
> > items in the wrong order, or with missing data? I'm
> > trying to track down a client freezing bug in
> > Snowglobe and the official viewer that might be
> > related to inventory, and found this.
> >
> > John
> >
> >
> > ___
> > Policies and (un)subscribe information available here:
> > http://wiki.secondlife.com/wiki/OpenSource-Dev
> > Please read the policies before posting to keep
> > unmoderated posting privileges
> >
> >
> >
> >
> > ___
> > Policies and (un)subscribe information available here:
> > http://wiki.secondlife.com/wiki/OpenSource-Dev
> > Please read the policies before posting to keep unmoderated posting
> privileges
>
>
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Potential inventory problem?

[Nexii Malthus]

Ah, the comment by Qarl Linden in the JIRA issue nails the issue for that
specific freeze bug:
"i'm not sure how our external svn is configured - but i can explain the
root of the problem. it's with our memory allocator (malloc/new/etc.) we
switched back at 1.21, and the behavior of the inventory system seemed to
trigger some kind of horrible worst-case scenario. with my demo above, i
switched to dmalloc. for the real fix in 1.23, we invoked some windows magic
to improve the allocator performance."

Might that help for this bug?

- Nexii

On Tue, Mar 9, 2010 at 12:19 AM, Nexii Malthus wrote:

> It does happen on the SL grid. At least it did for a long while, I have
> admit I use the texture picker not that often these days, but it has burned
> me pretty badly.
>
> How bad is the freeze? How long is it?
>
> Edit: Found this JIRA, might be relevant to the problem I think, the freeze
> bug used to exist in 1.21 and 1.22 and was fixed with 1.23.
> http://jira.secondlife.com/browse/VWR-8818
>
>
> - Nexii
>
>
> On Mon, Mar 8, 2010 at 10:33 PM, Rob Nelson 
> wrote:
>
>> Hahaha, I remember back when Open Grid Services was written in PHP;  I
>> still have it on my webserver somewhere.  I think I submitted a patch to
>> you guys several years ago (regarding a SQL injection exploit) where I
>> accidentally left in a bunch of cursing and racial slurs.
>>
>> If I contribute again, I think I'll be a little more careful.
>>
>> Also, I'll take a peek into what's causing that problem;  I'm having to
>> redo the UI in Lua anyway, so I may as well figure out what is causing
>> your issues.
>>
>> Fred Rookstown
>>
>> On Mon, 2010-03-08 at 11:17 -0800, John Hurliman wrote:
>> > Yes, I'm trying to push out the first release of a set of PHP grid
>> > services for OpenSim and the texture picker freeze bug is a
>> > showstopper. It obviously doesn't happen on SLGrid and it wasn't
>> > happening in the past with OpenSim. A packet trace didn't show
>> > anything interesting (doesn't appear to be triggered by a specific
>> > packet at all). My current guess is that the client is making a bad
>> > assumption about information the server sent earlier on. It might have
>> > to do with the library since I am not sending a library skeleton at
>> > the moment.
>> >
>> > Any other clues on the freezing issue would be much appreciated. The
>> > warning is more of a curiosity but it could uncover another hidden
>> > expectation the open source server software is not meeting.
>> >
>> > John
>> >
>> > On Mon, Mar 8, 2010 at 9:21 AM, Nexii Malthus 
>> > wrote:
>> > Happens just as well on any grid. I'm just as confused as
>> > anyone here what this error message means, but this was
>> > brought up before and I vaguely remember a linden saying that
>> > wasn't the source of the lag problems related to inventory.
>> >
>> >
>> > Are you trying to track down the texture picker freezing bug?
>> >
>> >
>> > - Nexii
>> >
>> >
>> > On Mon, Mar 8, 2010 at 12:45 AM, John Hurliman
>> >  wrote:
>> >
>> >
>> > When logging into OpenSim with Snowglobe I see a lot
>> > (dozens) of these messages in the log file:
>> >
>> > 2010-03-08T00:33:18Z WARNING:
>> > LLInventoryPanel::modelChanged:
>> > d24c5c7b-4d64-45eb-9286-91fb0f307b24 is in model and
>> > in view, but STRUCTURE flag not set
>> >
>> > What does this message mean? Am I sending inventory
>> > items in the wrong order, or with missing data? I'm
>> > trying to track down a client freezing bug in
>> > Snowglobe and the official viewer that might be
>> > related to inventory, and found this.
>> >
>> > John
>> >
>> >
>> > ___
>> > Policies and (un)subscribe information available here:
>> > http://wiki.secondlife.com/wiki/OpenSource-Dev
>> > Please read the policies before posting to keep
>> > unmoderated posting privileges
>> >
>> >
>> >
>> >
>> > ___
>> > Policies and (un)subscribe information available here:
>> > http://wiki.secondlife.com/wiki/OpenSource-Dev
>> > Please read the policies before posting to keep unmoderated posting
>> privileges
>>
>>
>> ___
>> Policies and (un)subscribe information available here:
>> http://wiki.secondlife.com/wiki/OpenSource-Dev
>> Please read the policies before posting to keep unmoderated posting
>> privileges
>>
>
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Potential inventory problem?

[Nexii Malthus]

Ah, the comment by Qarl Linden in the JIRA issue nails the issue for that
specific freeze bug:
"i'm not sure how our external svn is configured - but i can explain the
root of the problem. it's with our memory allocator (malloc/new/etc.) we
switched back at 1.21, and the behavior of the inventory system seemed to
trigger some kind of horrible worst-case scenario. with my demo above, i
switched to dmalloc. for the real fix in 1.23, we invoked some windows magic
to improve the allocator performance."

Might that help for this bug?

- Nexii

On Tue, Mar 9, 2010 at 12:19 AM, Nexii Malthus wrote:

> It does happen on the SL grid. At least it did for a long while, I have
> admit I use the texture picker not that often these days, but it has burned
> me pretty badly.
>
> How bad is the freeze? How long is it?
>
> Edit: Found this JIRA, might be relevant to the problem I think, the freeze
> bug used to exist in 1.21 and 1.22 and was fixed with 1.23.
> http://jira.secondlife.com/browse/VWR-8818
>
>
> - Nexii
>
>
> On Mon, Mar 8, 2010 at 10:33 PM, Rob Nelson 
> wrote:
>
>> Hahaha, I remember back when Open Grid Services was written in PHP;  I
>> still have it on my webserver somewhere.  I think I submitted a patch to
>> you guys several years ago (regarding a SQL injection exploit) where I
>> accidentally left in a bunch of cursing and racial slurs.
>>
>> If I contribute again, I think I'll be a little more careful.
>>
>> Also, I'll take a peek into what's causing that problem;  I'm having to
>> redo the UI in Lua anyway, so I may as well figure out what is causing
>> your issues.
>>
>> Fred Rookstown
>>
>> On Mon, 2010-03-08 at 11:17 -0800, John Hurliman wrote:
>> > Yes, I'm trying to push out the first release of a set of PHP grid
>> > services for OpenSim and the texture picker freeze bug is a
>> > showstopper. It obviously doesn't happen on SLGrid and it wasn't
>> > happening in the past with OpenSim. A packet trace didn't show
>> > anything interesting (doesn't appear to be triggered by a specific
>> > packet at all). My current guess is that the client is making a bad
>> > assumption about information the server sent earlier on. It might have
>> > to do with the library since I am not sending a library skeleton at
>> > the moment.
>> >
>> > Any other clues on the freezing issue would be much appreciated. The
>> > warning is more of a curiosity but it could uncover another hidden
>> > expectation the open source server software is not meeting.
>> >
>> > John
>> >
>> > On Mon, Mar 8, 2010 at 9:21 AM, Nexii Malthus 
>> > wrote:
>> > Happens just as well on any grid. I'm just as confused as
>> > anyone here what this error message means, but this was
>> > brought up before and I vaguely remember a linden saying that
>> > wasn't the source of the lag problems related to inventory.
>> >
>> >
>> > Are you trying to track down the texture picker freezing bug?
>> >
>> >
>> > - Nexii
>> >
>> >
>> > On Mon, Mar 8, 2010 at 12:45 AM, John Hurliman
>> >  wrote:
>> >
>> >
>> > When logging into OpenSim with Snowglobe I see a lot
>> > (dozens) of these messages in the log file:
>> >
>> > 2010-03-08T00:33:18Z WARNING:
>> > LLInventoryPanel::modelChanged:
>> > d24c5c7b-4d64-45eb-9286-91fb0f307b24 is in model and
>> > in view, but STRUCTURE flag not set
>> >
>> > What does this message mean? Am I sending inventory
>> > items in the wrong order, or with missing data? I'm
>> > trying to track down a client freezing bug in
>> > Snowglobe and the official viewer that might be
>> > related to inventory, and found this.
>> >
>> > John
>> >
>> >
>> > ___
>> > Policies and (un)subscribe information available here:
>> > http://wiki.secondlife.com/wiki/OpenSource-Dev
>> > Please read the policies before posting to keep
>> > unmoderated posting privileges
>> >
>> >
>> >
>> >
>> > ___
>> > Policies and (un)subscribe information available here:
>> > http://wiki.secondlife.com/wiki/OpenSource-Dev
>> > Please read the policies before posting to keep unmoderated posting
>> privileges
>>
>>
>> ___
>> Policies and (un)subscribe information available here:
>> http://wiki.secondlife.com/wiki/OpenSource-Dev
>> Please read the policies before posting to keep unmoderated posting
>> privileges
>>
>
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Limits UI

[Lear Cale]

It would be nice if everything were free, too.

The issue is memory *allocation*.  If a script only uses 16K but is
allocated 64K, that 64K counts against the server's actual memory
allocation limit.

So, cool, wouldn't it be nice to only allocate what is actually
requested?  Well that implies rewriting a lot of code to use memory
differently, and requires frequent reallocation of memory (as the
needs grow or shrink).  There is a real cost to this, in terms of
programming effort and in terms of runtime costs.

Until now, script memory has seemed to be a free lunch.  Well, the
free lunch is over, and we'll have to deal with it.  I won't believe
that it's feasible to simply "report the actual memory used" until
someone who really understands Mono memory allocation and our
scripting language's arrays (the main memory users) says that there
actually is a feasible solution along these lines.

Change causes upset.  This will be an issue.  IMHO, though, failing to
address the problem would be worse.

Regards
Jeff

On Sun, Mar 7, 2010 at 6:20 AM, Marine Kelley  wrote:
> Well we have two mutually exclusive solutions here.
>
> Either Mono scripts are given a hard memory limit that we (the scripters)
> can change within the scripts, with all the overhead work that it implies
> (i.e. modifying hundreds of scripts before issuing an update, and having to
> know upfront how much memory will be taken exactly), which means that in
> regards to the scripts memory usage UI, the script will use exactly as much
> as the limit it has requested, no matter whether it really uses it or not.
> This gives wasted memory and false information.
>
> Or, Mono scripts are given a hard memory limit that we cannot change, and
> they report exactly as many bytes as they use at any time. But we shouldn't
> be able to change the limit ourselves, because it wouldn't make sense to do
> so, it would only be restraining ourselves if we set less than 64k, and
> wasting memory space if we set more than 64k.
>
> In both cases, the question of whether the script crashes when reaching the
> limit or not is not related.
>
> I seriously, and I mean seriously, think that choosing the first option is
> going to hurt the established scripters very badly, and therefore the grid
> as a whole. To me scripts should report exactly as much memory as they use,
> not more, and should not require the scripters to modify them to report
> something that could be computed by the sims more accurately anyway.
>
> Of course it is tempting to tell the scripters "you can now decide how much
> memory to allow, and that way you are certain it will report the amount you
> have set", as much as it is tempting to shift the workload of allocating
> script memory onto the scripters since LL can't seem do it.
>
> Remember, we are now going to have limits on a service that didn't have them
> before. For the same price. All in the sake of stabilizing the grid. Ok for
> me. This will already hurt scripters who will have to adapt bad scripts. But
> now we are told we are going to also adapt good scripts as well ! I repeat,
> this is unacceptable.
>
> Marine
>
>
> On 7 March 2010 03:02, Frans  wrote:
>>
>> As for the dynamic vs fixed memory usage. Of course it would make sense to
>> have dynamic memory usage, but I haven't seen a response yet on how to solve
>> the problem that Kelly described, about scripts suddenly running out of
>> available memory to use, when they fill up lists with info, etc. And break
>> because of it. Or is this considered not to be a big problem?
>
>
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Limits UI

[Lear Cale]

Babbage has already said that LSO code will be "charged" 64K even
though it only uses 16K.  Perhaps he's changed that decision -- is
that the case?

Regards
Jeff

On Sat, Mar 6, 2010 at 9:02 PM, Frans  wrote:
> In response to the OP. I agree the UI will have to present that information
> differently. As it is now people will merely make a decision on memory usage
> and choose LSL scripts, and remove mono scripts. Likely negatively impacting
> their own experience. Scripters will be driven to compile scripts as LSL for
> marketing reasons, to make a product report a lower value
> Preventing LL from achieving it's goal of a full move towards MONO.
>
> As for the dynamic vs fixed memory usage. Of course it would make sense to
> have dynamic memory usage, but I haven't seen a response yet on how to solve
> the problem that Kelly described, about scripts suddenly running out of
> available memory to use, when they fill up lists with info, etc. And break
> because of it. Or is this considered not to be a big problem?
>
> --
> Jeroen Frans
> Virtual World Technology Specialist.
> VesuviusGroup.com
> SL: Frans Charming
>
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Limits UI

[Lear Cale]

It used to be that simple, with original LSO.

With mono, it's more complicated, because two instances of the *exact*
same script can share the code segments.  I wonder whether this
sharing ability will be taken into account.  Ideally yes, because it's
best when the metrics used to measure something actually reflects the
thing you want to measure and optimize.

Regards

On Sun, Mar 7, 2010 at 8:21 AM, Marine Kelley  wrote:
> As far as I understand it a script (the item you manipulate in-world) is a
> bytecode (a compiled version of your code without the comments, the spaces
> and the returns) loaded into the sim, plus its simple, constant-size
> variables (integer, floats), all of which forming the stack. Then you have
> all the varying-size variables, namely strings, keys and lists, which all
> form the heap.
>
> The 64k limit concerns the sum of the bytecode, the stack and the heap. The
> size of the two former is fixed and determined at compile time, while the
> size of the latter is variable.
>
> Someone please correct me if I'm wrong, this is experience talking, I do not
> have access to the server-side code of the script handling.
>
>
> On 7 March 2010 13:11, Tony Dodd  wrote:
>>
>> May I ask a probably dumb question here? It's probably been answered but
>> I can't find the answer now.
>>
>> Presumably a script uses two resources, the byte code in its assembly and
>> a slab of memory allocated for its stack/heap. And am I right that the
>> former can be shared between multiple copies of a script whereas the latter
>> would have to be allocated separately to each script?
>>
>> OK, the question is, does the 64K limit refer to both of these together or
>> just to the stack/heap? If I have a mono script that is quite large but
>> doesn't allocate any lists, will each instance of that that runs be using up
>> 64K of memory?
>>
>> If there's an obvious place I should look I apologise but without knowing
>> this I find it hard to assess how much impact memory limits will have on my
>> own scripts.
>>
>> Maldoror Bowman
>>
>>
>>
>>
>> ___
>> Policies and (un)subscribe information available here:
>> http://wiki.secondlife.com/wiki/OpenSource-Dev
>> Please read the policies before posting to keep unmoderated posting
>> privileges
>
>
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting
> privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Question regarding llSetLinkPimitiveParamsFast() function in 1.38.0

[Lear Cale]

It's generally a bad idea to change behavior dramatically, and much
safer to add a new function.

Dramatic changes in behavior cause new bugs, or cause old but
insignificant bugs to become significant.

New functions don't.



On Sun, Mar 7, 2010 at 9:39 AM, Obsidian Kindragon
 wrote:
> Hi all,
>
> I've a quick question regarding the new llSetLinkPimitiveParamsFast()
> function in 1.38.0. Why did LL opt for a new function instead of just
> removing the delay from the current llSetLinkPrimitiveParams() function?
> I can't conceive any case where removing the delay from the current
> function would break any existing content.
>
> - Obsidian Kindragon (AKA Obsidian Stormwind)
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Management Algorithm

[Lear Cale]

huh?  Can you point to existing technology for this analyzer?  Seems
to me like it would require an oracle.



On Mon, Mar 8, 2010 at 2:03 PM, Michael Schlenker
 wrote:
>
> Am 08.03.2010 um 18:46 schrieb Kelly Linden:
>
>> We are not out to write a new malloc for mono.  What we have is a system 
>> that throws an exception when the memory used by the script hits a certain 
>> threshold (64k currently).  This exception is caught so we can "crash" the 
>> script.  The future small scripts and big scripts projects will add new 
>> functions to set and get this threshold value, allowing scripters to 
>> effectively control how much memory is reserved for their script.  We will 
>> continue to use mono's default memory management within the reserved memory 
>> thresholds.  It is a much simpler problem to solve.
>>
> While your at it, how about a static analyzer for mono/LSL that determines 
> guaranteed lowest memory consumption for a script and sets the threshold 
> there.
>
> That would have the benefit of easing scripters work by providing useful 
> defaults for all the easy cases without them having to do anything at all.
>
> The scheme should only break down if the mono GC behaves weird. In that case 
> scripters have a huge problem anyway as they cannot set any threshold without 
> being crashed at random by a lazy GC.
>
> Michael
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting privileges
>
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Management Algorithm

[Lear Cale]

Please tell me how to register for this list.  I was registered for
sldev, but don't remember how.
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Limits UI

[Tigro Spottystripes]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

What if the memory allocation worked kinda like those bouncing bars on
top of those audiofrquency/spectrum/VUmeter readouts on some stereos and
many computer audio players, where it's fast to go up, but then it falls
slowly back till it hits the current level or is bumped back up bit it
during the slow fall?

On 8/3/2010 22:40, Lear Cale wrote:
> It would be nice if everything were free, too.
> 
> The issue is memory *allocation*.  If a script only uses 16K but is
> allocated 64K, that 64K counts against the server's actual memory
> allocation limit.
> 
> So, cool, wouldn't it be nice to only allocate what is actually
> requested?  Well that implies rewriting a lot of code to use memory
> differently, and requires frequent reallocation of memory (as the
> needs grow or shrink).  There is a real cost to this, in terms of
> programming effort and in terms of runtime costs.
> 
> Until now, script memory has seemed to be a free lunch.  Well, the
> free lunch is over, and we'll have to deal with it.  I won't believe
> that it's feasible to simply "report the actual memory used" until
> someone who really understands Mono memory allocation and our
> scripting language's arrays (the main memory users) says that there
> actually is a feasible solution along these lines.
> 
> Change causes upset.  This will be an issue.  IMHO, though, failing to
> address the problem would be worse.
> 
> Regards
> Jeff
> 
> On Sun, Mar 7, 2010 at 6:20 AM, Marine Kelley  wrote:
>> Well we have two mutually exclusive solutions here.
>>
>> Either Mono scripts are given a hard memory limit that we (the scripters)
>> can change within the scripts, with all the overhead work that it implies
>> (i.e. modifying hundreds of scripts before issuing an update, and having to
>> know upfront how much memory will be taken exactly), which means that in
>> regards to the scripts memory usage UI, the script will use exactly as much
>> as the limit it has requested, no matter whether it really uses it or not.
>> This gives wasted memory and false information.
>>
>> Or, Mono scripts are given a hard memory limit that we cannot change, and
>> they report exactly as many bytes as they use at any time. But we shouldn't
>> be able to change the limit ourselves, because it wouldn't make sense to do
>> so, it would only be restraining ourselves if we set less than 64k, and
>> wasting memory space if we set more than 64k.
>>
>> In both cases, the question of whether the script crashes when reaching the
>> limit or not is not related.
>>
>> I seriously, and I mean seriously, think that choosing the first option is
>> going to hurt the established scripters very badly, and therefore the grid
>> as a whole. To me scripts should report exactly as much memory as they use,
>> not more, and should not require the scripters to modify them to report
>> something that could be computed by the sims more accurately anyway.
>>
>> Of course it is tempting to tell the scripters "you can now decide how much
>> memory to allow, and that way you are certain it will report the amount you
>> have set", as much as it is tempting to shift the workload of allocating
>> script memory onto the scripters since LL can't seem do it.
>>
>> Remember, we are now going to have limits on a service that didn't have them
>> before. For the same price. All in the sake of stabilizing the grid. Ok for
>> me. This will already hurt scripters who will have to adapt bad scripts. But
>> now we are told we are going to also adapt good scripts as well ! I repeat,
>> this is unacceptable.
>>
>> Marine
>>
>>
>> On 7 March 2010 03:02, Frans  wrote:
>>>
>>> As for the dynamic vs fixed memory usage. Of course it would make sense to
>>> have dynamic memory usage, but I haven't seen a response yet on how to solve
>>> the problem that Kelly described, about scripts suddenly running out of
>>> available memory to use, when they fill up lists with info, etc. And break
>>> because of it. Or is this considered not to be a big problem?
>>
>>
>> ___
>> Policies and (un)subscribe information available here:
>> http://wiki.secondlife.com/wiki/OpenSource-Dev
>> Please read the policies before posting to keep unmoderated posting
>> privileges
>>
> ___
> Policies and (un)subscribe information available here:
> http://wiki.secondlife.com/wiki/OpenSource-Dev
> Please read the policies before posting to keep unmoderated posting privileges
> 
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.12 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkuVsUoACgkQ8ZFfSrFHsmUv3gCePSEwRWsVSf9vh1YVijEITi4t
RcQAniPxfg6VROIG+BewpHFxa2IR8G/X
=u9CF
-END PGP SIGNATURE-
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep

Re: [opensource-dev] Potential inventory problem?

[John Hurliman]

That certainly sounds like the symptoms I'm seeing. One of my two CPUs is
pegged, memory allocation for the Snowglobe process is all over the place
(jumping up and down by 10s or 100s of MB at a time), and the hang seems to
be indefinite. I haven't left it for more than three minutes, but by that
time the server has decided you are timed out since the client isn't sending
any network traffic.

We also just confirmed that the current OpenSim master isn't triggering this
bug, so it's a subtle difference between what the stock OpenSim inventory
connector is doing and what mine is doing. It would be really nice to know
what that difference is, but it sounds like Qarl wasn't sure either. I just
found out I'm sending folders with Version = 0 and I'm told that version
should start at 1. I fixed that and it hasn't resolved the issue, but I'll
keep plugging away.

John

On Mon, Mar 8, 2010 at 4:23 PM, Nexii Malthus  wrote:

> Ah, the comment by Qarl Linden in the JIRA issue nails the issue for that
> specific freeze bug:
> "i'm not sure how our external svn is configured - but i can explain the
> root of the problem. it's with our memory allocator (malloc/new/etc.) we
> switched back at 1.21, and the behavior of the inventory system seemed to
> trigger some kind of horrible worst-case scenario. with my demo above, i
> switched to dmalloc. for the real fix in 1.23, we invoked some windows magic
> to improve the allocator performance."
>
> Might that help for this bug?
>
> - Nexii
>
> On Tue, Mar 9, 2010 at 12:19 AM, Nexii Malthus wrote:
>
>> It does happen on the SL grid. At least it did for a long while, I have
>> admit I use the texture picker not that often these days, but it has burned
>> me pretty badly.
>>
>> How bad is the freeze? How long is it?
>>
>> Edit: Found this JIRA, might be relevant to the problem I think, the
>> freeze bug used to exist in 1.21 and 1.22 and was fixed with 1.23.
>> http://jira.secondlife.com/browse/VWR-8818
>>
>>
>> - Nexii
>>
>>
>> On Mon, Mar 8, 2010 at 10:33 PM, Rob Nelson > > wrote:
>>
>>> Hahaha, I remember back when Open Grid Services was written in PHP;  I
>>> still have it on my webserver somewhere.  I think I submitted a patch to
>>> you guys several years ago (regarding a SQL injection exploit) where I
>>> accidentally left in a bunch of cursing and racial slurs.
>>>
>>> If I contribute again, I think I'll be a little more careful.
>>>
>>> Also, I'll take a peek into what's causing that problem;  I'm having to
>>> redo the UI in Lua anyway, so I may as well figure out what is causing
>>> your issues.
>>>
>>> Fred Rookstown
>>>
>>> On Mon, 2010-03-08 at 11:17 -0800, John Hurliman wrote:
>>> > Yes, I'm trying to push out the first release of a set of PHP grid
>>> > services for OpenSim and the texture picker freeze bug is a
>>> > showstopper. It obviously doesn't happen on SLGrid and it wasn't
>>> > happening in the past with OpenSim. A packet trace didn't show
>>> > anything interesting (doesn't appear to be triggered by a specific
>>> > packet at all). My current guess is that the client is making a bad
>>> > assumption about information the server sent earlier on. It might have
>>> > to do with the library since I am not sending a library skeleton at
>>> > the moment.
>>> >
>>> > Any other clues on the freezing issue would be much appreciated. The
>>> > warning is more of a curiosity but it could uncover another hidden
>>> > expectation the open source server software is not meeting.
>>> >
>>> > John
>>> >
>>> > On Mon, Mar 8, 2010 at 9:21 AM, Nexii Malthus 
>>> > wrote:
>>> > Happens just as well on any grid. I'm just as confused as
>>> > anyone here what this error message means, but this was
>>> > brought up before and I vaguely remember a linden saying that
>>> > wasn't the source of the lag problems related to inventory.
>>> >
>>> >
>>> > Are you trying to track down the texture picker freezing bug?
>>> >
>>> >
>>> > - Nexii
>>> >
>>> >
>>> > On Mon, Mar 8, 2010 at 12:45 AM, John Hurliman
>>> >  wrote:
>>> >
>>> >
>>> > When logging into OpenSim with Snowglobe I see a lot
>>> > (dozens) of these messages in the log file:
>>> >
>>> > 2010-03-08T00:33:18Z WARNING:
>>> > LLInventoryPanel::modelChanged:
>>> > d24c5c7b-4d64-45eb-9286-91fb0f307b24 is in model and
>>> > in view, but STRUCTURE flag not set
>>> >
>>> > What does this message mean? Am I sending inventory
>>> > items in the wrong order, or with missing data? I'm
>>> > trying to track down a client freezing bug in
>>> > Snowglobe and the official viewer that might be
>>> > related to inventory, and found this.
>>> >
>>> > John
>>> >
>>> >
>>> > ___
>>> > Policies and (un)subscribe infor

Re: [opensource-dev] Script Memory Limits UI

[Argent Stonecutter]

On 2010-03-08, at 19:41, Lear Cale wrote:
> Babbage has already said that LSO code will be "charged" 64K even
> though it only uses 16K.  Perhaps he's changed that decision -- is
> that the case?

I'd like some clarification there too, because I read that they'd  
backed down on that idea (which is good). If they've changed their  
minds again, I'd like to know.
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Limits UI

[Maggie Leber (sl: Maggie Darwin)]

On Mon, Mar 8, 2010 at 8:40 PM, Lear Cale  wrote:
> It would be nice if everything were free, too...
> Until now, script memory has seemed to be a free lunch.  Well, the
> free lunch is over, and we'll have to deal with it.

I wouldn't mind "dealing with" lunch not being free if a crappy
bloated scripting implementation hadn't eaten it.
___
Policies and (un)subscribe information available here:
http://wiki.secondlife.com/wiki/OpenSource-Dev
Please read the policies before posting to keep unmoderated posting privileges

Re: [opensource-dev] Script Memory Limits UI

[Marine Kelley]

Change requires work. Unnecessary, unwanted, and uncalled for in this  
instance. We have to adapt to handle a part of the task that LL was  
supposed to do themselves. Oh of course this is a hard job, allocating  
memory dynamically in an environment like this. Perhaps it is  
impossible. I have yet to hear a Linden say, in all honesty, "sorry  
guys, we can't do it as initially planned, we have to ask you to  
participate by tailoring your scripts, because we can't do it from our  
side". What I have heard so far is "you will be provided tools to  
adapt to the change that is taking place". The two mean exactly the  
same thing, but a little honesty does not hurt. This additional  
workload was not planned, is a shift of work that we were not supposed  
to take in charge in the first place, with no compensation, so I'd  
have liked a little explanation at least.

I am not against the limits. Of course scripts need limits, and they  
actually always had some, only they were hidden, unmanaged and had to  
be discovered through trial and error. As I said, "I am ok with that".  
What I am not ok with, is the fact that I have to review every single  
script of mine to try to find out how much memory they need in the  
worst case. Finding said worst case is already a challenge in itself.  
I have thousands of scripts in-world, not even counting the ones in my  
products that I will have to issue an update for. Because wait a  
little and you'll see merchants start to advertize how little the  
memory footprint of their scripts is compared to the next competitor,  
regardless of how crappy said scripts are.

Anyway this is a moot point now. As far as I understand it, LL has  
decided that their implementation would require us to adapt our  
scripts ourselves, exactly like I described.



On 9 mars 2010, at 02:40, Lear Cale  wrote:

> It would be nice if everything were free, too.
>
> The issue is memory *allocation*.  If a script only uses 16K but is
> allocated 64K, that 64K counts against the server's actual memory
> allocation limit.
>
> So, cool, wouldn't it be nice to only allocate what is actually
> requested?  Well that implies rewriting a lot of code to use memory
> differently, and requires frequent reallocation of memory (as the
> needs grow or shrink).  There is a real cost to this, in terms of
> programming effort and in terms of runtime costs.
>
> Until now, script memory has seemed to be a free lunch.  Well, the
> free lunch is over, and we'll have to deal with it.  I won't believe
> that it's feasible to simply "report the actual memory used" until
> someone who really understands Mono memory allocation and our
> scripting language's arrays (the main memory users) says that there
> actually is a feasible solution along these lines.
>
> Change causes upset.  This will be an issue.  IMHO, though, failing to
> address the problem would be worse.
>
> Regards
> Jeff
>
> On Sun, Mar 7, 2010 at 6:20 AM, Marine Kelley  
>  wrote:
>> Well we have two mutually exclusive solutions here.
>>
>> Either Mono scripts are given a hard memory limit that we (the  
>> scripters)
>> can change within the scripts, with all the overhead work that it  
>> implies
>> (i.e. modifying hundreds of scripts before issuing an update, and  
>> having to
>> know upfront how much memory will be taken exactly), which means  
>> that in
>> regards to the scripts memory usage UI, the script will use exactly  
>> as much
>> as the limit it has requested, no matter whether it really uses it  
>> or not.
>> This gives wasted memory and false information.
>>
>> Or, Mono scripts are given a hard memory limit that we cannot  
>> change, and
>> they report exactly as many bytes as they use at any time. But we  
>> shouldn't
>> be able to change the limit ourselves, because it wouldn't make  
>> sense to do
>> so, it would only be restraining ourselves if we set less than 64k,  
>> and
>> wasting memory space if we set more than 64k.
>>
>> In both cases, the question of whether the script crashes when  
>> reaching the
>> limit or not is not related.
>>
>> I seriously, and I mean seriously, think that choosing the first  
>> option is
>> going to hurt the established scripters very badly, and therefore  
>> the grid
>> as a whole. To me scripts should report exactly as much memory as  
>> they use,
>> not more, and should not require the scripters to modify them to  
>> report
>> something that could be computed by the sims more accurately anyway.
>>
>> Of course it is tempting to tell the scripters "you can now decide  
>> how much
>> memory to allow, and that way you are certain it will report the  
>> amount you
>> have set", as much as it is tempting to shift the workload of  
>> allocating
>> script memory onto the scripters since LL can't seem do it.
>>
>> Remember, we are now going to have limits on a service that didn't  
>> have them
>> before. For the same price. All in the sake of stabilizing the  
>> grid. Ok for
>> me. This wil