Hi, On Sun, Mar 22, 2009 at 09:37:55PM +0200, Sergiu Ivanov wrote: > <olafbuddenha...@gmx.net> writes: > > On Sun, Feb 22, 2009 at 08:56:50PM +0200, Sergiu Ivanov wrote:
> > I said running multiple translator instances in one process, not > > sharing one translator instance among clients... That's quite a > > different thing :-) > > Hm, that's really a different thing... I understand that this question > will be offtopic, but I'll still ask you: do your words imply that we > will somehow run multiple processes within a single process?.. Well, processes are defined by address spaces... But I believe I know what you mean: running several programs (or instances of a program) as totally independent threads in a single address space, so logically they are almost like multiple processes, although they share the address space, and thus are technically one process. This already came up in some earlier discussion (regarding libchannel I guess, but I don't remember for sure) -- and my opinion is that this is not a good idea. It would trade added complexity and lowered robustness in exchange for only little saving in resource usage: address spaces are not really that expensive. Of course, the cost of the address spaces alone is quite substantial when dealing with really many processes (in the order of many thousands) -- but then, even having a kernel thread per logical process would also be very expensive. Leaving out the address spaces is not a real remedy for this. The only way to be really efficient is such a situation, is having a single thread handle multiple clients -- but in this case, I don't think it's possible to logically seperate the instances, i.e. we end up with a monolithic server... Which, as I explained in the other discussion, is something I'm rather sceptical about. However, I believe that really needing thousands of translator instances *in parallel* is rather an ususual case, which probably will occur only in extreme situations, unless something is poorly designed. A much more realistic situation is only having a few instances at most runnig in parallel at any given moment, while many more instances are created and destroyed in short succession. Here, the major problem is not resource usage of running processes, but rather the cost of process startup. And I believe that this should be the main optimization target. So the idea is to have only the first instance do a full translator startup. It would do as much common initialization as possible; and then some mystical lightweight mechanism would be used for launching the other instances from there -- something that is much cheaper than normal process creation... > > I mean that the filter invokes the special RPC to obtain a version > > of the node without surplus shadows, returning this to the actual > > client. Thus the procedure is not transparent to the filter -- it > > needs to know that nsmux is involved and invoke the special RPC -- > > but it is transparent to the actual client: it gets a "clean" node, > > devoid of undesired shadows, just as if they never existed. > > Aha, I see. It means I've understood this thing correctly. > > Another question now (probably, I'm repeating myself already...): how > problematic is that the filter should know about nsmux? After all, the > filter's main real use case is running in a dynamic translator stack. > I understand that having the filter capable of running as a normal > translator would be a nice option, but I fail to find the absence of > this feature a very bad thing. Actually, this doesn't prevent the filter from running outside nsmux. It only means that the filter must be aware when it is running on nsmux, and handle it specially. That's not really a big deal technically -- but it's not very elegant, as it means things are not as orthogonal as I'd like them to be. nsmux is not fully transparent -- certain kinds of other programs need to handle it specially to work correctly... > > The RPC for getting the underlying node logically would belong to > > the file interface (fs.defs) [...] > > The alternative is creating a new interface just for this special > > call. We wouldn't need to touch existing interfaces; but it would be > > rather unelegant... > > I am somehow more inclined to creating a new special interface for > nsmux... Could please point out the reasons why you consider this > solution rather unelegant? See above: logically, the RPC for obtaining the underlying node belongs with the existing interfaces. The operation is not really filter-specific -- it could be useful in other situations as well. The RPC for getting a non-shadowed version of the node on the other hand is very specific, so having a distinct interface only for this is probably really the best approach. > >> [...] there is no already existing RPC for going one translation > >> layer lower. > > > > My point is that traversing bottom-to-top isn't any more natural, as > > it requires obtaining the untranslated node at the bottom of the > > stack, and there is no existing RPC for that either. > > Hm, I think I cannot understand something properly here: we *do* have > the possibility to get the untranslated node at the bottom of the > stack by opening the node with O_NOTRANS, don't we? No. We can't obtain the untranslated version of a node we have. All we can do is reopen the same *file name* -- which is a totally different thing! The filter does *not* have the file name of the node it filters. The hack with O_NOTRANS on translator startup to obtain the untranslated node, only works with special handling in nsmux. You can't do that when running the filter on a normal file system, implementing only the standard interfaces. It requires a new operation -- even if this new operation is hacked as a special case in an existing RPC. > >> I thought we could merge the functionality in a single node because > >> it seemed to me that another node would mean another context > >> switch... [...] > > (Note that this would actually be a case of translator stacking > > optimization -- i.e. a use case for the "mobility framework" > > Frederik is working on. I'm not quite sure whether it's better to > > create special solutions for various use cases first, and only later > > factor out a generic stacking framework, or only work on such > > optimizations once the generic stacking framework is in place...) > > Hm... I'm trying to follow your discussion with Frederik, but I'm not > sure I can understand how this could be a use case for the ``mobility > framework''. I guess I should go and read the latest mail in you > discussion, which I skipped do to lack of time. What you wanted to do here is avoiding a context switch, by transparently stuffing the functionality of two logically distinct translators in a single process. This is *exactly* what translator stacking is about... -antrik-
