Re: uvm_map improvements

2010-04-01 Thread Bob Beck
> All hail memory management hardware! All hail a quick and honorable death. None of this surviving without honor in our kernel. We want to die quckly.

Re: uvm_map improvements

2010-04-01 Thread Otto Moerbeek
On Thu, Apr 01, 2010 at 09:56:19AM -0600, Bob Beck wrote: > > stupid idea, perhaps, but would it be possible to recycle the idea of > > having some sort of canaries at the end of *each* page, thus disposing > > of the need to have guard pages? Or would that be too costly? > > Oh sure, that'll wor

Re: uvm_map improvements

2010-04-01 Thread Bob Beck
> stupid idea, perhaps, but would it be possible to recycle the idea of > having some sort of canaries at the end of *each* page, thus disposing > of the need to have guard pages? Or would that be too costly? Oh sure, that'll work Ok, I'll just ensure when your file gets written out it has a 64 by

Re: uvm_map improvements

2010-04-01 Thread Toni Mueller
Hi, On Fri, 26.02.2010 at 00:31:35 -0700, Theo de Raadt wrote: > space when it is under high contention. But there is a massively > understated benefit that comes from filling the address space with > unallocated gaps. The gaps, though only on a page boundary, are > finding a lot of bugs. LOT

Re: uvm_map improvements

2010-03-04 Thread Ted Unangst
On Thu, Mar 4, 2010 at 11:39 AM, Anton Maksimenkov wrote: > Since I not found any support of that idea I think it is better to > keep things simple. So I introduce implementation of the "ReScan" > idea. Independent of any technical merits, I think this has more to do with release timing and avail

Re: uvm_map improvements

2010-03-04 Thread Anton Maksimenkov
Since I not found any support of that idea I think it is better to keep things simple. So I introduce implementation of the "ReScan" idea. The diff must be almost the same as in the first message in thread. I added some logic into uvm_map_findspace(). If it can't found reasonable gap using hint it

Re: uvm_map improvements

2010-02-27 Thread Anton Maksimenkov
2010/2/26 Anton Maksimenkov : > The idea might be like this. We don't use uvm_map_hint(). In > uvm_map_findspace() we get entry with biggest space. Then we generate > random from range 0...MIN(space-length, 256M). Then we search first > entry with space >= length+that_random. It surely exists, at l

Re: uvm_map improvements

2010-02-26 Thread Anton Maksimenkov
2010/2/26 Anton Maksimenkov : > *result = entry->start + that_random; ur-r-r-gh *result = entry->end + that_random; -- antonvm

Re: uvm_map improvements

2010-02-26 Thread Anton Maksimenkov
2010/2/26 Anton Maksimenkov : >> + ReScan: It might be better solution. As I said to art@, we might generate random exactly in uvm_map_findspace(). The idea might be like this. We don't use uvm_map_hint(). In uvm_map_findspace() we get entry with biggest space. Then we generate random from range

Re: uvm_map improvements

2010-02-26 Thread Anton Maksimenkov
> + int try_rescan = 0; Oh my mistake, sorry. It was so quick draft... Of course try_rescan must be not 0 before the first scan: try_rescan = 1; > + ReScan: > tmpsearch.space = length; > tmp = RB_NFIND(uvm_tree_space, &map->rb_space, &tmpsearch); > if (tmp == NULL)

Re: uvm_map improvements

2010-02-26 Thread Anton Maksimenkov
2010/2/26 Theo de Raadt : >> But I just want to add POSSIBILITY to change this. Let sysadmin decide >> that in some case it NEED "shrink randomness", doesn't matter which >> bugs _may_ will show themselfs. > No. We choose to not do that Ok, I got this. Let it be. But what about this: 2010/2/26 O

Re: uvm_map improvements

2010-02-26 Thread Otto Moerbeek
On Fri, Feb 26, 2010 at 12:52:56PM +0500, Anton Maksimenkov wrote: > 2010/2/26 Otto Moerbeek : > >> 2. > ... > >> I think that we may introduce some variable - sysctl variable or > >> malloc.conf flag - which will prevent sys_mmap() or uvm_map_hint() > > If fragmentation really is a problem for yo

Re: uvm_map improvements

2010-02-26 Thread Theo de Raadt
> 2010/2/26 Otto Moerbeek : > >> 2. > ... > >> I think that we may introduce some variable - sysctl variable or > >> malloc.conf flag - which will prevent sys_mmap() or uvm_map_hint() > > If fragmentation really is a problem for you, my first reaction is: > > use a machine with a bigger adress spac

Re: uvm_map improvements

2010-02-25 Thread Anton Maksimenkov
2010/2/26 Otto Moerbeek : >> 2. ... >> I think that we may introduce some variable - sysctl variable or >> malloc.conf flag - which will prevent sys_mmap() or uvm_map_hint() > If fragmentation really is a problem for you, my first reaction is: > use a machine with a bigger adress spacce. No deal.

Re: uvm_map improvements

2010-02-25 Thread Theo de Raadt
> There are cases when we need memory rather than randomization. > Dedicated database server is such case. Postgres can give more > performance if it can use as much memory as userspace can allocate. > Many big random holes quickly leads to fails of allocations. So then > we must dramatically decre

Re: uvm_map improvements

2010-02-25 Thread Otto Moerbeek
On Fri, Feb 26, 2010 at 10:58:09AM +0500, Anton Maksimenkov wrote: > 2010/2/20 Anton Maksimenkov : > > it was started discussion about uvm_map.c improvements. I prepared the > > diff and trying to explain it. > > In addition to that diff I want to discuss two problems. > > 1. > AFAIK, the usersp

Re: uvm_map improvements

2010-02-25 Thread Anton Maksimenkov
2010/2/20 Anton Maksimenkov : > it was started discussion about uvm_map.c improvements. I prepared the > diff and trying to explain it. In addition to that diff I want to discuss two problems. 1. AFAIK, the userspace malloc() uses mmap (sys_mmap) to do it's job. As we can see, sys_mmap() uses uvm

uvm_map improvements

2010-02-20 Thread Anton Maksimenkov
Hi. In this thread http://marc.info/?l=openbsd-tech&m=126494185625212&w=2 it was started discussion about uvm_map.c improvements. I prepared the diff and trying to explain it. First, I introduce 2 trees here: - uvm_tree_start, the RB_TREE indexed by entry's "start" - uvm_tree_space, the RB_TREE