By overwhelmingly popular demand, I present you with the libpager rework. I have been pushing data through this code for the last few days and it seems to be quite stable. I will follow with patches for libdiskfs and ext2fs once I have some comments on this work.
Thanks. 2002-04-04 Neal H Walfield <[EMAIL PROTECTED]> * pager.h (struct pager): Bring into scope. (struct pager_ops): New structure. Contains the call backs: read, write, unlock, report_extent, clear_user_data and dropweak. Replaces the linker-time call backs: pager_read_page, pager_write_page, pager_unlock_page, pager_report_extenet, pager_clear_user_data and pager_drop_weak. Functions now also have a multipage interface and take block, not byte, offsets (i.e. off_t's, not vm_address_t's). (pager_cleate): Take a struct pager_ops to support new interface. Take the size of the desired user_page_info structure rather than a user_pager_info structure to correspond with the new semantics. (pager_get_error): Follow new semantics: take the page number rather than a byte offset. (pager_sync_come): Likewise. (pager_flush_come): Likewise. (pager_return_some): Likewise. (pager_memcpy): Likewise. (pager_offset_page): Remove obsolete function. (pager_read_page): Removed in favor of call back in struct pager_ops. (pager_write_page): Likewise. (pager_unlock_page): Likewise. (pager_report_extent): Likewise. (pager_clear_user_data): Likewise. (pager_dropweak): Likewise. (pager_data_supply): New function. (pager_data_unavailable): Likewise. (pager_data_read_error): Likewise. (pager_data_write_error): Likewise. (pager_data_unlock): Likewise. (pager_data_unlock_error): Likewise. * priv.h (struct pager): Change UPI from a struct user_pager_info to a flex array at the end of the structure to match the new pager_create semantics. Add the pager_ops call back structure. (PM_PAGEINWAIT): Macro no longer used. (PM_WRITEWAIT): Adjust value. (PM_INIT): Likewise. (PM_INCORE): Likewise. (PM_PAGINGOUT): Likewise. (_pager_pagemap_resize): Take a block offset, not a byte-offset. (_pager_mark_next_request_error): Likewise. (_pager_mark_object_error): Likewise. (_pager_lock_object): Likewise. P->interlock is now expected to already be held during the call. * data-return.c (_pager_do_write_request) Remove obsolete function. * data-return.c (_pager_seqnos_memory_object_data_return): Rewritten to work with the new paging interface. * data-request.c (_pager_seqnos_memory_object_data_request): Likewise. * data-unlock.c (_pager_seqnos_memory_object_data_unlock): Likewise. * clean.c (_pager_clean): Use the new call backs in P->ops. * dropweak.c (_pager_real_dropweak): Likewise. * lock-completed.c (_pager_seqnos_memory_object_lock_completed): Handle page offsets and not byte offsets. * lock-object.c (_pager_lock_object): Handle page offsets and not byte offsets. As per the new interface, do not lock P->interlock anymore. * mark-error.c (_pager_mark_next_request_error): Handle page offsets and not byte offsets. Use vm_page_size, not __vm_page_size. Doc fix. (_pager_mark_object_error): Likewise. (pager_get_error): Likewise. * pagemap.c (VMCOPY_BETTER_THAN_MEMCPY): New macro. (_pager_pagemap_resize): Handle page offsets and not byte offsets. Use vm_page_size, not __vm_page_size. Allocate the minimum number of pages. Check the result of mmap against MAP_FAILED, not -1. Use mempcy, not bcopy. Use vm_copy when appropriate. * object-init.c (_pager_seqnos_memory_object_init): Use vm_page_size, not __vm_page_size. Check the result of malloc. * pager-attr.c (pager_change_attributes): Correct a typo. Add an assert. * pager-create.c (pager_create): Comform to the new semantics as outlined in the pager.h changelog entry. * pager-flush.c (pager_flush): Use the new call backs in P->ops. Handle page offsets, not byte offsets. Call the corresponding _some function rather than copy the code. * pager-return.c (pager_return): Likewise. * pager-sync.c (pager_sync): Likewise. * pager-flush.c (pager_flush_some): Handle page offsets and not byte offsets. Lock P->interlock around _pager_lock_object as per the new semantics. * pager-return.c (pager_return_some): Likewise. * pager-sync.c (pager_sync_some: Likewise. * pager-memcpy.c (pager_memcpy): Pass the pager offset, not the byte offset to _pager_get_error. Change OFFSET to an off_t rather than a vm_address_t as per the new semantics. * Makefile (SRCS): Remove offer-pager.c. Add pager-data-read-error.c pager-data-unavailable.c pager-data-unlock.c pager-data-supply.c pager-data-unlock-error.c pager-data-write-error.c. * offer-page.c: Remove obsolete files. * pager-data-read-error.c: New file. * pager-data-supply.c: Likewise. * pager-data-unavailable.c: Likewise. * pager-data-unlock-error.c: Likewise. * pager-data-unlock.c: Likewise. * pager-data-write-error.c: Likewise. Index: Makefile =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/Makefile,v retrieving revision 1.35 diff -u -p -r1.35 Makefile --- Makefile 17 Sep 1996 16:42:27 -0000 1.35 +++ Makefile 10 Apr 2002 21:37:13 -0000 @@ -25,7 +25,10 @@ SRCS = data-request.c data-return.c data pager-create.c pager-flush.c pager-shutdown.c pager-sync.c \ stubs.c seqnos.c demuxer.c chg-compl.c pager-attr.c clean.c \ dropweak.c notify-stubs.c get-upi.c pager-memcpy.c pager-return.c \ - offer-page.c + pager-data-read-error.c pager-data-unavailable.c \ + pager-data-unlock.c pager-data-supply.c pager-data-unlock-error.c \ + pager-data-write-error.c + LCLHDRS = pager.h priv.h installhdrs = pager.h Index: clean.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/clean.c,v retrieving revision 1.4 diff -u -p -r1.4 clean.c --- clean.c 26 Mar 1996 20:50:09 -0000 1.4 +++ clean.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* - Copyright (C) 1995, 1996 Free Software Foundation, Inc. + Copyright (C) 1995, 1996, 2002 Free Software Foundation, Inc. Written by Michael I. Bushnell. This file is part of the GNU Hurd. @@ -46,5 +46,6 @@ _pager_clean (void *arg) mutex_unlock (&p->interlock); } - pager_clear_user_data (p->upi); + if (p->ops->clear_user_data) + p->ops->clear_user_data (p->upi); } Index: data-request.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/data-request.c,v retrieving revision 1.21 diff -u -p -r1.21 data-request.c --- data-request.c 2 Apr 2002 02:06:59 -0000 1.21 +++ data-request.c 10 Apr 2002 21:37:13 -0000 @@ -19,22 +19,33 @@ #include "memory_object_S.h" #include <stdio.h> #include <string.h> +#include <assert.h> /* Implement pagein callback as described in <mach/memory_object.defs>. */ kern_return_t _pager_seqnos_memory_object_data_request (mach_port_t object, mach_port_seqno_t seqno, mach_port_t control, - vm_offset_t offset, + vm_offset_t start_address, vm_size_t length, vm_prot_t access) { - struct pager *p; - short *pm_entry; - int doread, doerror; error_t err; - vm_address_t page; - int write_lock; + struct pager *p; + short *pm_entries; + off_t start; + int npages; + + error_t last_error; + int page_count; + int good_pages; + unsigned char *errors; + + int i; + + DEBUG ("object = %d, seqno = %d,control = %d, start_address = %d, " + "length = %d, access = %d", + object, seqno, control, start_address, length, access); p = ports_lookup_port (0, object, _pager_class); if (!p) @@ -50,12 +61,7 @@ _pager_seqnos_memory_object_data_request printf ("incg data request: wrong control port\n"); goto release_out; } - if (length != __vm_page_size) - { - printf ("incg data request: bad length size %d\n", length); - goto release_out; - } - if (offset % __vm_page_size) + if (start_address & (vm_page_size - 1)) { printf ("incg data request: misaligned request\n"); goto release_out; @@ -68,72 +74,152 @@ _pager_seqnos_memory_object_data_request { printf ("pager in wrong state for read\n"); _pager_release_seqno (p, seqno); + _pager_allow_termination (p); mutex_unlock (&p->interlock); - goto allow_term_out; + ports_port_deref (p); + return 0; } - err = _pager_pagemap_resize (p, offset + length); + start = start_address / vm_page_size; + npages = length / vm_page_size; + + err = _pager_pagemap_resize (p, start + npages); if (err) goto release_out; /* Can't do much about the actual error. */ /* If someone is paging this out right now, the disk contents are - unreliable, so we have to wait. It is too expensive (right now) to - find the data and return it, and then interrupt the write, so we just - mark the page and have the writing thread do m_o_data_supply when it - gets around to it. */ - pm_entry = &p->pagemap[offset / __vm_page_size]; - if (*pm_entry & PM_PAGINGOUT) - { - doread = 0; - *pm_entry |= PM_PAGEINWAIT; - } - else - doread = 1; - - if (*pm_entry & PM_INVALID) - doerror = 1; - else - doerror = 0; - - *pm_entry |= PM_INCORE; - - if (PM_NEXTERROR (*pm_entry) != PAGE_NOERR && (access & VM_PROT_WRITE)) - { - memory_object_data_error (control, offset, length, - _pager_page_errors[PM_NEXTERROR (*pm_entry)]); - _pager_mark_object_error (p, offset, length, - _pager_page_errors[PM_NEXTERROR (*pm_entry)]); - *pm_entry = SET_PM_NEXTERROR (*pm_entry, PAGE_NOERR); - doread = 0; + unreliable, so we have to wait. It is too expensive (right now) + to find the data and return it, and then interrupt the write, so + we just wait for the write to finish and then reread the + requested pages from disk. */ + + pm_entries = &p->pagemap[start]; + + retry: + for (i = 0; i < npages; i ++) + if (pm_entries[i] & PM_PAGINGOUT) + { + pm_entries[i] |= PM_WRITEWAIT; + condition_wait (&p->wakeup, &p->interlock); + goto retry; + } + + last_error = 0; + page_count = 0; + good_pages = 0; + + for (i = 0; i < npages; i ++) + { + error_t err; + + pm_entries[i] |= PM_INCORE; + + if (pm_entries[i] & PM_INVALID) + /* Data on disk was marked bad. */ + err = PAGE_EIO; + else if (PM_NEXTERROR (pm_entries[i]) != PAGE_NOERR + && (access & VM_PROT_WRITE)) + /* In the process of a request, flush, error. */ + err = PM_NEXTERROR (pm_entries[i]); + else + { + good_pages ++; + + if (last_error) + /* This is the start of a good range. Flush the pending + error. */ + err = 0; + else + continue; + } + + if (err == last_error) + page_count ++; + else + { + if (last_error) + { + off_t range_start = start + i - page_count; + + /* Record the bad pages. */ + if (i == good_pages + page_count + 1) + { + errors = alloca (sizeof (*errors) * npages); + memset (errors, 0, sizeof (*errors) * npages); + } + memset (errors + i - page_count, ~0, + sizeof (*errors) * page_count); + + /* Tell the kernel about it. */ + memory_object_data_error (control, range_start * vm_page_size, + page_count * vm_page_size, + _pager_page_errors[last_error]); + _pager_mark_object_error (p, range_start, page_count, + _pager_page_errors[last_error]); + } + + last_error = err; + page_count = 1; + } + } + + if (last_error) + { + off_t range_start = start + i - page_count; + + /* Record the bad pages. */ + if (good_pages != 0) + memset (errors + i - page_count, ~0, + sizeof (*errors) * page_count); + + memory_object_data_error (control, range_start * vm_page_size, + page_count * vm_page_size, + _pager_page_errors[last_error]); + _pager_mark_object_error (p, range_start, page_count, + _pager_page_errors[last_error]); } /* Let someone else in. */ _pager_release_seqno (p, seqno); mutex_unlock (&p->interlock); - if (!doread) - goto allow_term_out; - if (doerror) - goto error_read; - - err = pager_read_page (p->upi, offset, &page, &write_lock); - if (err) - goto error_read; - - memory_object_data_supply (p->memobjcntl, offset, page, length, 1, - write_lock ? VM_PROT_WRITE : VM_PROT_NONE, 0, - MACH_PORT_NULL); - mutex_lock (&p->interlock); - _pager_mark_object_error (p, offset, length, 0); - _pager_allow_termination (p); - mutex_unlock (&p->interlock); - ports_port_deref (p); - return 0; + if (good_pages == npages) + /* All of the pages are good. Optimize the following loop + away. */ + p->ops->read (p, (struct user_pager_info *) &p->upi, start, npages); + else if (good_pages > 0) + { + page_count = 1; + last_error = errors[0]; + + for (i = 1; i < npages; i ++) + { + if (last_error == errors[i]) + page_count ++; + else if (last_error) + /* Already reported the error, just reset the counters. */ + { + page_count = 1; + last_error = 0; + } + else + { + /* Read the pages and give them to the kernel. */ + p->ops->read (p, (struct user_pager_info *) &p->upi, + start + i - page_count, page_count); + + last_error = errors[i]; + } + } + + if (last_error == 0) + p->ops->read (p, (struct user_pager_info *) &p->upi, + start + i - page_count, page_count); + } + else + /* No good data. */ + ; - error_read: - memory_object_data_error (p->memobjcntl, offset, length, EIO); - _pager_mark_object_error (p, offset, length, EIO); - allow_term_out: mutex_lock (&p->interlock); _pager_allow_termination (p); mutex_unlock (&p->interlock); Index: data-return.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/data-return.c,v retrieving revision 1.21 diff -u -p -r1.21 data-return.c --- data-return.c 25 Jul 2000 19:40:27 -0000 1.21 +++ data-return.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Implementation of memory_object_data_return for pager library - Copyright (C) 1994, 1995, 1996, 1999, 2000 Free Software Foundation + Copyright (C) 1994,95,96,99, 2000,02 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -21,51 +21,46 @@ #include <string.h> #include <assert.h> -/* Worker function used by _pager_seqnos_memory_object_data_return - and _pager_seqnos_memory_object_data_initialize. All args are - as for _pager_seqnos_memory_object_data_return; the additional - INITIALIZING arg identifies which function is calling us. */ +/* Implement pageout call back as described by <mach/memory_object.defs>. */ kern_return_t -_pager_do_write_request (mach_port_t object, - mach_port_seqno_t seqno, - mach_port_t control, - vm_offset_t offset, - pointer_t data, - vm_size_t length, - int dirty, - int kcopy, - int initializing) +_pager_seqnos_memory_object_data_return (mach_port_t object, + mach_port_seqno_t seqno, + mach_port_t control, + vm_offset_t start_address, + pointer_t data, + vm_size_t length, + int dirty, + int kcopy) { struct pager *p; short *pm_entries; - int npages, i; - error_t *pagerrs; + off_t start; + int npages; + int i; struct lock_request *lr; struct lock_list {struct lock_request *lr; struct lock_list *next;} *lock_list, *ll; int wakeup; - int omitdata = 0; + DEBUG ("object = %d, seqno = %d, control = %d, start_address = %d, " + "length = %d, dirty = %d, kcopy", + object, seqno, control, start_address, length, dirty, kcopy); + p = ports_lookup_port (0, object, _pager_class); if (!p) return EOPNOTSUPP; - /* Acquire the right to meddle with the pagemap */ + /* Acquire the right to meddle with the pagemap. */ mutex_lock (&p->interlock); _pager_wait_for_seqno (p, seqno); - - /* sanity checks -- we don't do multi-page requests yet. */ + + /* sanity checks. */ if (control != p->memobjcntl) { printf ("incg data return: wrong control port\n"); goto release_out; } - if (length % __vm_page_size) - { - printf ("incg data return: bad length size %d\n", length); - goto release_out; - } - if (offset % __vm_page_size) + if (start_address & (vm_page_size - 1)) { printf ("incg data return: misaligned request\n"); goto release_out; @@ -80,15 +75,15 @@ _pager_do_write_request (mach_port_t obj goto release_out; } - npages = length / __vm_page_size; - pagerrs = alloca (npages * sizeof (error_t)); + start = start_address / vm_page_size; + npages = length / vm_page_size; _pager_block_termination (p); /* until we are done with the pagemap when the write completes. */ - _pager_pagemap_resize (p, offset + length); + _pager_pagemap_resize (p, start + npages); - pm_entries = &p->pagemap[offset / __vm_page_size]; + pm_entries = &p->pagemap[start]; /* Make sure there are no other in-progress writes for any of these pages before we begin. This imposes a little more serialization @@ -103,36 +98,24 @@ _pager_do_write_request (mach_port_t obj condition_wait (&p->wakeup, &p->interlock); goto retry; } - - /* Mark these pages as being paged out. */ - if (initializing) + + for (i = 0; i < npages; i++) { - assert (npages <= 32); - for (i = 0; i < npages; i++) - { - if (pm_entries[i] & PM_INIT) - omitdata |= 1 << i; - else - pm_entries[i] |= PM_PAGINGOUT | PM_INIT; - } - } - else - for (i = 0; i < npages; i++) pm_entries[i] |= PM_PAGINGOUT | PM_INIT; - if (!kcopy) - for (i = 0; i < npages; i++) - pm_entries[i] &= ~PM_INCORE; + if (!kcopy) + pm_entries[i] &= ~PM_INCORE; + } /* If this write occurs while a lock is pending, record it. We have to keep this list because a lock request might come in while we do the I/O; in that case there would be a new entry on p->lock_requests and we must - make sure we don't decrement it. So we have to keep + make sure we don't decrement it. So, we have to keep track independently of which lock requests we incremented. */ lock_list = 0; for (lr = p->lock_requests; lr; lr = lr->next) - if (offset < lr->end && offset + length >= lr->start) + if (start < lr->end && start + npages >= lr->start) { ll = alloca (sizeof (struct lock_list)); ll->lr = lr; @@ -145,50 +128,20 @@ _pager_do_write_request (mach_port_t obj _pager_release_seqno (p, seqno); mutex_unlock (&p->interlock); - /* This is inefficient; we should send all the pages to the device at once - but until the pager library interface is changed, this will have to do. */ - - for (i = 0; i < npages; i++) - if (!(omitdata & (1 << i))) - pagerrs[i] = pager_write_page (p->upi, - offset + (vm_page_size * i), - data + (vm_page_size * i)); + p->ops->write (p, (struct user_pager_info *) &p->upi, start, + npages, (void *) data, 1); /* Acquire the right to meddle with the pagemap */ mutex_lock (&p->interlock); - _pager_pagemap_resize (p, offset + length); - pm_entries = &p->pagemap[offset / __vm_page_size]; + pm_entries = &p->pagemap[start]; wakeup = 0; - for (i = 0; i < npages; i++) + + for (i = 0; i < npages; i ++) { - if (omitdata & (1 << i)) - continue; - if (pm_entries[i] & PM_WRITEWAIT) wakeup = 1; - - if (pagerrs[i] && ! (pm_entries[i] & PM_PAGEINWAIT)) - /* The only thing we can do here is mark the page, and give - errors from now on when it is to be read. This is - imperfect, because if all users go away, the pagemap will - be freed, and this information lost. Oh well. It's still - better than Un*x. Of course, if we are about to hand this - data to the kernel, the error isn't a problem, hence the - check for pageinwait. */ - pm_entries[i] |= PM_INVALID; - - if (pm_entries[i] & PM_PAGEINWAIT) - memory_object_data_supply (p->memobjcntl, - offset + (vm_page_size * i), - data + (vm_page_size * i), - vm_page_size, 1, - VM_PROT_NONE, 0, MACH_PORT_NULL); - else - munmap ((caddr_t) (data + (vm_page_size * i)), - vm_page_size); - - pm_entries[i] &= ~(PM_PAGINGOUT | PM_PAGEINWAIT | PM_WRITEWAIT); + pm_entries[i] &= ~(PM_PAGINGOUT | PM_WRITEWAIT); } for (ll = lock_list; ll; ll = ll->next) @@ -199,9 +152,7 @@ _pager_do_write_request (mach_port_t obj condition_broadcast (&p->wakeup); _pager_allow_termination (p); - mutex_unlock (&p->interlock); - ports_port_deref (p); return 0; @@ -212,17 +163,3 @@ _pager_do_write_request (mach_port_t obj return 0; } -/* Implement pageout call back as described by <mach/memory_object.defs>. */ -kern_return_t -_pager_seqnos_memory_object_data_return (mach_port_t object, - mach_port_seqno_t seqno, - mach_port_t control, - vm_offset_t offset, - pointer_t data, - vm_size_t length, - int dirty, - int kcopy) -{ - return _pager_do_write_request (object, seqno, control, offset, data, - length, dirty, kcopy, 0); -} Index: data-unlock.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/data-unlock.c,v retrieving revision 1.11 diff -u -p -r1.11 data-unlock.c --- data-unlock.c 20 Jun 1995 17:47:21 -0000 1.11 +++ data-unlock.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Implementation of memory_object_data_unlock for pager library - Copyright (C) 1994, 1995 Free Software Foundation + Copyright (C) 1994,95, 2002 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -25,12 +25,11 @@ kern_return_t _pager_seqnos_memory_object_data_unlock (mach_port_t object, mach_port_seqno_t seqno, mach_port_t control, - vm_offset_t offset, + vm_offset_t start, vm_size_t length, vm_prot_t access) { struct pager *p; - volatile int err; p = ports_lookup_port (0, object, _pager_class); if (!p) @@ -58,31 +57,15 @@ _pager_seqnos_memory_object_data_unlock printf ("incg data unlock: not unlock writes\n"); goto out; } - if (offset % __vm_page_size) + if (start & (vm_page_size - 1)) { printf ("incg data unlock: misaligned request\n"); goto out; } - if (length != __vm_page_size) - { - printf ("incg data unlock: bad length size %d\n", length); - goto out; - } - err = pager_unlock_page (p->upi, offset); - - if (!err) - /* We can go ahead and release the lock. */ - _pager_lock_object (p, offset, length, MEMORY_OBJECT_RETURN_NONE, 0, - VM_PROT_NONE, 0); - else - { - /* Flush the page, and set a bit so that m_o_data_request knows - to issue an error. */ - _pager_lock_object (p, offset, length, MEMORY_OBJECT_RETURN_NONE, 1, - VM_PROT_WRITE, 1); - _pager_mark_next_request_error (p, offset, length, err); - } + p->ops->unlock (p, (struct user_pager_info *) &p->upi, + start / vm_page_size, length / vm_page_size); + out: ports_port_deref (p); return 0; Index: dropweak.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/dropweak.c,v retrieving revision 1.1 diff -u -p -r1.1 dropweak.c --- dropweak.c 20 Jun 1995 17:15:49 -0000 1.1 +++ dropweak.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* - Copyright (C) 1995 Free Software Foundation, Inc. + Copyright (C) 1995, 2002 Free Software Foundation, Inc. Written by Michael I. Bushnell. This file is part of the GNU Hurd. @@ -25,5 +25,6 @@ _pager_real_dropweak (void *arg) { struct pager *p = arg; - pager_dropweak (p->upi); + if (p->ops->dropweak) + p->ops->dropweak (p->upi); } Index: lock-completed.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/lock-completed.c,v retrieving revision 1.20 diff -u -p -r1.20 lock-completed.c --- lock-completed.c 26 Mar 1996 21:25:14 -0000 1.20 +++ lock-completed.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Implementation of memory_object_lock_completed for pager library - Copyright (C) 1994, 1995, 1996 Free Software Foundation + Copyright (C) 1994,95,96, 2002 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -26,12 +26,13 @@ kern_return_t _pager_seqnos_memory_object_lock_completed (mach_port_t object, mach_port_seqno_t seqno, mach_port_t control, - vm_offset_t offset, + vm_offset_t start_address, vm_size_t length) { error_t err = 0; struct pager *p; struct lock_request *lr; + off_t start, npages; p = ports_lookup_port (0, object, _pager_class); if (!p) @@ -49,8 +50,11 @@ _pager_seqnos_memory_object_lock_complet mach_port_deallocate (mach_task_self (), control); + start = start_address / vm_page_size; + npages = length / vm_page_size; + for (lr = p->lock_requests; lr; lr = lr->next) - if (lr->start == offset && lr->end == offset + length) + if (lr->start == start && lr->end == start + npages) { if (lr->locks_pending) --lr->locks_pending; Index: lock-object.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/lock-object.c,v retrieving revision 1.16 diff -u -p -r1.16 lock-object.c --- lock-object.c 25 Jul 2000 19:40:27 -0000 1.16 +++ lock-object.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Synchronous wrapper for memory_object_lock_request - Copyright (C) 1993, 1994, 1996, 1997, 2000 Free Software Foundation + Copyright (C) 1993,94,96,97, 2000,02 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -17,14 +17,15 @@ #include "priv.h" -/* Request a lock from the kernel on pager P. Parameters OFFSET, - SIZE, SHOULD_RETURN, SHOULD_FLUSH, and LOCK_VALUE are as for - memory_object_lock_request. If SYNC is set, then wait for the - operation to fully complete before returning. */ +/* Request a lock from the kernel on pager P starting at page START + for COUNT pages. Parameters SHOULD_RETURN, SHOULD_FLUSH, and + LOCK_VALUE are as for memory_object_lock_request. If SYNC is set, + then wait for the operation to fully complete before returning. + This must be called with P->interlock help. */ void _pager_lock_object (struct pager *p, - vm_offset_t offset, - vm_size_t size, + off_t start, + off_t count, int should_return, int should_flush, vm_prot_t lock_value, @@ -33,17 +34,13 @@ _pager_lock_object (struct pager *p, int i; struct lock_request *lr = 0; - mutex_lock (&p->interlock); if (p->pager_state != NORMAL) - { - mutex_unlock (&p->interlock); - return; - } + return; if (sync) { for (lr = p->lock_requests; lr; lr = lr->next) - if (lr->start == offset && lr->end == offset + size) + if (lr->start == start && lr->end == start + count) { lr->locks_pending++; lr->threads_waiting++; @@ -52,8 +49,10 @@ _pager_lock_object (struct pager *p, if (!lr) { lr = malloc (sizeof (struct lock_request)); - lr->start = offset; - lr->end = offset + size; + if (! lr) + return; + lr->start = start; + lr->end = start + count; lr->pending_writes = 0; lr->locks_pending = 1; lr->threads_waiting = 1; @@ -65,7 +64,8 @@ _pager_lock_object (struct pager *p, } } - memory_object_lock_request (p->memobjcntl, offset, size, should_return, + memory_object_lock_request (p->memobjcntl, start * vm_page_size, + count * vm_page_size, should_return, should_flush, lock_value, sync ? p->port.port_right : MACH_PORT_NULL); @@ -84,22 +84,14 @@ _pager_lock_object (struct pager *p, if (should_flush) { - vm_offset_t pm_offs = offset / __vm_page_size; + short *pm_entries; + + _pager_pagemap_resize (p, start + count); - _pager_pagemap_resize (p, offset + size); - if (p->pagemapsize > pm_offs) - { - short *pm_entries = &p->pagemap[pm_offs]; - vm_offset_t bound = size / vm_page_size; - - if (bound > p->pagemapsize) - bound = p->pagemapsize; - - for (i = 0; i < bound; i++) - pm_entries[i] &= ~PM_INCORE; - } + pm_entries = &p->pagemap[start]; + + for (i = 0; i < count; i++) + pm_entries[i] &= ~PM_INCORE; } } - - mutex_unlock (&p->interlock); } Index: mark-error.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/mark-error.c,v retrieving revision 1.8 diff -u -p -r1.8 mark-error.c --- mark-error.c 25 Jul 2000 19:40:27 -0000 1.8 +++ mark-error.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Recording errors for pager library - Copyright (C) 1994, 1997 Free Software Foundation + Copyright (C) 1994, 1997, 2002 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -20,23 +20,21 @@ int _pager_page_errors[] = {KERN_SUCCESS, ENOSPC, EIO, EDQUOT}; -/* Some error has happened indicating that the page cannot be written. - (Usually this is ENOSPC or EDQOUT.) On the next pagein which - requests write access, return the error to the kernel. (This is +/* Some error has happened indicating that the page cannot be written. + (Usually this is ENOSPC or EDQOUT). On the next pagein which + requests write access, return the error to the kernel. (This is screwy because of the rules associated with m_o_lock_request.) - Currently the only errors permitted are ENOSPC, EIO, and EDQUOT. */ + Currently the only errors permitted are ENOSPC, EIO, and EDQUOT. + PAGER->interlock must be held during this call. */ void _pager_mark_next_request_error(struct pager *pager, - vm_address_t offset, - vm_size_t length, + off_t start, + off_t count, error_t error) { int page_error; short *p; - offset /= __vm_page_size; - length /= __vm_page_size; - switch (error) { case 0: @@ -54,27 +52,26 @@ _pager_mark_next_request_error(struct pa break; } - for (p = pager->pagemap + offset; p < pager->pagemap + offset + length; p++) + for (p = pager->pagemap + start; p < pager->pagemap + start + count; p++) *p = SET_PM_NEXTERROR (*p, page_error); } -/* We are returning a pager error to the kernel. Write down - in the pager what that error was so that the exception handling - routines can find out. (This is only necessary because the - XP interface is not completely implemented in the kernel.) - Again, only ENOSPC, EIO, and EDQUOT are permitted. */ +/* We are returning a pager error to the kernel. Write down in the + pager what that error was so that the exception handling routines + can find out. (This is only necessary because the XP interface is + not completely implemented in the kernel.) Again, only ENOSPC, + EIO, and EDQUOT are permitted. PAGER->interlock must be held + during this call. _pager_pagemap_resize should have been + called. */ void _pager_mark_object_error(struct pager *pager, - vm_address_t offset, - vm_size_t length, + off_t start, + off_t count, error_t error) { int page_error = 0; short *p; - offset /= __vm_page_size; - length /= __vm_page_size; - switch (error) { case 0: @@ -92,29 +89,27 @@ _pager_mark_object_error(struct pager *p break; } - for (p = pager->pagemap + offset; p < pager->pagemap + offset + length; p++) + for (p = pager->pagemap + start; p < pager->pagemap + start + count; p++) *p = SET_PM_ERROR (*p, page_error); } /* Tell us what the error (set with mark_object_error) for - pager P is on page ADDR. */ + pager P is on page PAGE. */ error_t -pager_get_error (struct pager *p, vm_address_t addr) +pager_get_error (struct pager *p, off_t page) { error_t err; mutex_lock (&p->interlock); - addr /= vm_page_size; - - /* If there really is no error for ADDR, we should be able to exted the + /* If there really is no error for ADDR, we should be able to extend the pagemap table; otherwise, if some previous operation failed because it couldn't extend the table, this attempt will *probably* (heh) fail for the same reason. */ - err = _pager_pagemap_resize (p, addr); + err = _pager_pagemap_resize (p, page + 1); if (! err) - err = _pager_page_errors[PM_ERROR(p->pagemap[addr])]; + err = _pager_page_errors[PM_ERROR(p->pagemap[page])]; mutex_unlock (&p->interlock); Index: object-init.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/object-init.c,v retrieving revision 1.15 diff -u -p -r1.15 object-init.c --- object-init.c 21 Feb 1996 20:17:26 -0000 1.15 +++ object-init.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Implementation of memory_object_init for pager library - Copyright (C) 1994, 1995, 1996 Free Software Foundation + Copyright (C) 1994,95,96,2001 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -37,7 +37,7 @@ _pager_seqnos_memory_object_init (mach_p mutex_lock (&p->interlock); _pager_wait_for_seqno (p, seqno); - if (pagesize != __vm_page_size) + if (pagesize != vm_page_size) { printf ("incg init: bad page size"); goto out; @@ -47,6 +47,9 @@ _pager_seqnos_memory_object_init (mach_p { #ifdef KERNEL_INIT_RACE struct pending_init *i = malloc (sizeof (struct pending_init)); + if (! i) + goto out; + printf ("pager out-of-sequence init\n"); i->control = control; i->name = name; Index: offer-page.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/offer-page.c,v retrieving revision 1.7 diff -u -p -r1.7 offer-page.c --- offer-page.c 25 Jul 2000 19:40:27 -0000 1.7 +++ offer-page.c 10 Apr 2002 21:37:13 -0000 @@ -1,52 +0,0 @@ -/* Wrapper for unsolicited memory_object_data_supply - Copyright (C) 1996, 1997, 2000 Free Software Foundation, Inc. - Written by Michael I. Bushnell, p/BSG. - - This file is part of the GNU Hurd. - - The GNU Hurd is free software; you can redistribute it and/or - modify it under the terms of the GNU General Public License as - published by the Free Software Foundation; either version 2, or (at - your option) any later version. - - The GNU Hurd is distributed in the hope that it will be useful, but - WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA. */ - - -#include "priv.h" - -void -pager_offer_page (struct pager *p, - int precious, - int writelock, - vm_offset_t offset, - vm_address_t buf) -{ - mutex_lock (&p->interlock); - - if (_pager_pagemap_resize (p, offset + vm_page_size)) - { - short *pm_entry = &p->pagemap[offset / vm_page_size]; - - while (*pm_entry & PM_INCORE) - { - mutex_unlock (&p->interlock); - pager_flush_some (p, offset, vm_page_size, 1); - mutex_lock (&p->interlock); - } - *pm_entry |= PM_INCORE; - - memory_object_data_supply (p->memobjcntl, offset, buf, vm_page_size, 0, - writelock ? VM_PROT_WRITE : VM_PROT_NONE, - precious, MACH_PORT_NULL); - } - - mutex_unlock (&p->interlock); -} - Index: pagemap.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/pagemap.c,v retrieving revision 1.8 diff -u -p -r1.8 pagemap.c --- pagemap.c 25 Jul 2000 19:40:27 -0000 1.8 +++ pagemap.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Pagemap manipulation for pager library - Copyright (C) 1994, 1997, 1999, 2000 Free Software Foundation + Copyright (C) 1994,97,99,2000,02 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -17,31 +17,53 @@ #include "priv.h" #include <string.h> +#include <assert.h> + +/* Start using vm_copy over memcpy when we have at least two + pages. */ +#define VMCOPY_BETTER_THAN_MEMCPY (vm_page_size * 2) -/* Grow the pagemap of pager P as necessary to deal with address OFF */ +/* Grow the pagemap of pager P as necessary to deal with page address + OFF - 1. */ error_t -_pager_pagemap_resize (struct pager *p, vm_address_t off) +_pager_pagemap_resize (struct pager *p, off_t off) { error_t err = 0; - - off /= __vm_page_size; - if (p->pagemapsize < off) + assert (((p->pagemapsize * sizeof (*p->pagemap)) + & (vm_page_size - 1)) == 0); + + if (p->pagemapsize <= off) { void *newaddr; - int newsize = round_page (off); + int newsize = round_page (off * sizeof (*p->pagemap)); - newaddr = mmap (0, newsize * sizeof (*p->pagemap), - PROT_READ|PROT_WRITE, MAP_ANON, 0, 0); - err = (newaddr == (void *) -1) ? errno : 0; - if (! err) + newaddr = mmap (0, newsize, PROT_READ|PROT_WRITE, MAP_ANON, 0, 0); + if (newaddr == MAP_FAILED) + err = errno; + else { - bcopy (p->pagemap, newaddr, p->pagemapsize * sizeof (*p->pagemap)); - munmap (p->pagemap, p->pagemapsize * sizeof (*p->pagemap)); + int oldsize = p->pagemapsize * sizeof (*p->pagemap); + + if (oldsize > 0) + { + if (oldsize >= VMCOPY_BETTER_THAN_MEMCPY) + { + err = vm_copy (mach_task_self (), + (vm_address_t) p->pagemap, oldsize, + (vm_address_t) newaddr); + assert_perror (err); + } + else + memcpy (newaddr, p->pagemap, oldsize); + + munmap (p->pagemap, oldsize); + } + p->pagemap = newaddr; - p->pagemapsize = newsize; + p->pagemapsize = newsize / sizeof (*p->pagemap); } } - + return err; } Index: pager-attr.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/pager-attr.c,v retrieving revision 1.4 diff -u -p -r1.4 pager-attr.c --- pager-attr.c 6 Mar 1996 23:10:30 -0000 1.4 +++ pager-attr.c 10 Apr 2002 21:37:13 -0000 @@ -21,7 +21,7 @@ /* Change the attributes of the memory object underlying pager P. Args MAY_CACHE and COPY_STRATEGY are as for memory_object_change_atributes. Wait for the kernel to report completion - off WAIT is set.*/ + if WAIT is set.*/ void pager_change_attributes (struct pager *p, boolean_t may_cache, @@ -65,6 +65,7 @@ pager_change_attributes (struct pager *p if (!ar) { ar = malloc (sizeof (struct attribute_request)); + assert (ar); ar->may_cache = may_cache; ar->copy_strategy = copy_strategy; ar->attrs_pending = 1; @@ -75,7 +76,7 @@ pager_change_attributes (struct pager *p ar->prevp = &p->attribute_requests; p->attribute_requests = ar; } - } + } memory_object_change_attributes (p->memobjcntl, may_cache, copy_strategy, wait ? p->port.port_right : MACH_PORT_NULL); Index: pager-create.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/pager-create.c,v retrieving revision 1.18 diff -u -p -r1.18 pager-create.c --- pager-create.c 9 May 1996 16:47:42 -0000 1.18 +++ pager-create.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Pager creation - Copyright (C) 1994, 1995, 1996 Free Software Foundation + Copyright (C) 1994,95,96,2001,02 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -19,18 +19,21 @@ /* Create and return a new pager with user info UPI. */ struct pager * -pager_create (struct user_pager_info *upi, - struct port_bucket *bucket, - boolean_t may_cache, - memory_object_copy_strategy_t copy_strategy) +pager_create (struct pager_ops *ops, + size_t upi_size, + struct port_bucket *bucket, + boolean_t may_cache, + memory_object_copy_strategy_t copy_strategy) { + error_t err; struct pager *p; - errno = ports_create_port (_pager_class, bucket, sizeof (struct pager), &p); - if (errno) + err = ports_create_port (_pager_class, bucket, + sizeof (struct pager) + upi_size, &p); + if (err) return 0; - p->upi = upi; + p->ops = ops; p->pager_state = NOTINIT; mutex_init (&p->interlock); condition_init (&p->wakeup); Index: pager-data-read-error.c =================================================================== RCS file: pager-data-read-error.c diff -N pager-data-read-error.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ pager-data-read-error.c 10 Apr 2002 21:37:13 -0000 @@ -0,0 +1,37 @@ +/* Indicate an error reading data. + Copyright (C) 2002 Free Software Foundation, Inc. + Written by Neal H Walfield + + This file is part of the GNU Hurd. + + The GNU Hurd is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2, or (at + your option) any later version. + + The GNU Hurd is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA. */ + + +#include "priv.h" + +void +pager_data_read_error (struct pager *pager, + off_t start, off_t npages, + error_t err) +{ + memory_object_data_error (pager->memobjcntl, start * vm_page_size, + npages * vm_page_size, err); + + mutex_lock (&pager->interlock); + _pager_pagemap_resize (pager, start + npages); + _pager_mark_object_error (pager, start, npages, err); + mutex_unlock (&pager->interlock); +} + Index: pager-data-supply.c =================================================================== RCS file: pager-data-supply.c diff -N pager-data-supply.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ pager-data-supply.c 10 Apr 2002 21:37:13 -0000 @@ -0,0 +1,42 @@ +/* Supply data to the kernel. + Copyright (C) 2002 Free Software Foundation, Inc. + Written by Neal H Walfield + + This file is part of the GNU Hurd. + + The GNU Hurd is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2, or (at + your option) any later version. + + The GNU Hurd is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA. */ + + +#include "priv.h" + +void +pager_data_supply (struct pager *pager, + int precious, int readonly, + off_t start, off_t npages, + void *buf, int dealloc) +{ + memory_object_data_supply (pager->memobjcntl, start * vm_page_size, + (vm_address_t) buf, npages * vm_page_size, + dealloc, + readonly ? VM_PROT_WRITE : VM_PROT_NONE, + precious, MACH_PORT_NULL); + + mutex_lock (&pager->interlock); + _pager_pagemap_resize (pager, start + npages); + _pager_mark_object_error (pager, start, npages, 0); + mutex_unlock (&pager->interlock); + +} + Index: pager-data-unavailable.c =================================================================== RCS file: pager-data-unavailable.c diff -N pager-data-unavailable.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ pager-data-unavailable.c 10 Apr 2002 21:37:13 -0000 @@ -0,0 +1,36 @@ +/* Indicate that there is no allocated data for a given range. + Copyright (C) 2002 Free Software Foundation, Inc. + Written by Neal H Walfield. + + This file is part of the GNU Hurd. + + The GNU Hurd is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2, or (at + your option) any later version. + + The GNU Hurd is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA. */ + + +#include "priv.h" + +void +pager_data_unavailable (struct pager *pager, + off_t start, off_t npages) +{ + memory_object_data_unavailable (pager->memobjcntl, + start * vm_page_size, + npages * vm_page_size); + + mutex_lock (&pager->interlock); + _pager_pagemap_resize (pager, start + npages); + _pager_mark_object_error (pager, start, npages, 0); + mutex_unlock (&pager->interlock); +} Index: pager-data-unlock-error.c =================================================================== RCS file: pager-data-unlock-error.c diff -N pager-data-unlock-error.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ pager-data-unlock-error.c 10 Apr 2002 21:37:13 -0000 @@ -0,0 +1,41 @@ +/* Indicate an error unlocking some pages. + Copyright (C) 2002 Free Software Foundation, Inc. + Written by Neal H Walfield + + This file is part of the GNU Hurd. + + The GNU Hurd is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2, or (at + your option) any later version. + + The GNU Hurd is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA. */ + + +#include "priv.h" + +void +pager_data_unlock_error (struct pager *pager, + off_t start, off_t npages, + error_t err) +{ + mutex_lock (&pager->interlock); + + /* Flush the range and set a bit so that + m_o_data_request knows to issue an error. */ + _pager_lock_object (pager, start, npages, + MEMORY_OBJECT_RETURN_NONE, 1, + VM_PROT_WRITE, 1); + + _pager_pagemap_resize (pager, start + npages); + _pager_mark_next_request_error (pager, start, npages, err); + + mutex_unlock (&pager->interlock); +} Index: pager-data-unlock.c =================================================================== RCS file: pager-data-unlock.c diff -N pager-data-unlock.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ pager-data-unlock.c 10 Apr 2002 21:37:13 -0000 @@ -0,0 +1,34 @@ +/* Mark pages as writable. + Copyright (C) 2002 Free Software Foundation, Inc. + Written by Neal H Walfield + + This file is part of the GNU Hurd. + + The GNU Hurd is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2, or (at + your option) any later version. + + The GNU Hurd is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA. */ + + +#include "priv.h" + +void +pager_data_unlock (struct pager *pager, + off_t start, off_t count) +{ + mutex_lock (&pager->interlock); + _pager_lock_object (pager, start, count, + MEMORY_OBJECT_RETURN_NONE, 0, + VM_PROT_NONE, 0); + mutex_unlock (&pager->interlock); +} + Index: pager-data-write-error.c =================================================================== RCS file: pager-data-write-error.c diff -N pager-data-write-error.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ pager-data-write-error.c 10 Apr 2002 21:37:13 -0000 @@ -0,0 +1,45 @@ +/* Indicate an error while writing data. + Copyright (C) 2002 Free Software Foundation, Inc. + Written by Neal H Walfield + + This file is part of the GNU Hurd. + + The GNU Hurd is free software; you can redistribute it and/or + modify it under the terms of the GNU General Public License as + published by the Free Software Foundation; either version 2, or (at + your option) any later version. + + The GNU Hurd is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111, USA. */ + + +#include "priv.h" + +void +pager_data_write_error (struct pager *pager, + off_t start, off_t npages, + error_t err) +{ + int i; + short *pm_entries; + + memory_object_data_error (pager->memobjcntl, start * vm_page_size, + npages * vm_page_size, err); + + mutex_lock (&pager->interlock); + _pager_pagemap_resize (pager, start + npages); + _pager_mark_object_error (pager, start, npages, err); + + pm_entries = &pager->pagemap[start]; + for (i = 0; i < npages; i ++) + pm_entries[i] |= PM_INVALID; + + mutex_unlock (&pager->interlock); +} + Index: pager-flush.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/pager-flush.c,v retrieving revision 1.5 diff -u -p -r1.5 pager-flush.c --- pager-flush.c 18 Mar 1996 18:10:29 -0000 1.5 +++ pager-flush.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Functions for flushing data - Copyright (C) 1994, 1996 Free Software Foundation + Copyright (C) 1994,96,2002 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -22,24 +22,24 @@ void pager_flush (struct pager *p, int wait) { - vm_address_t offset; - vm_size_t len; + off_t start, end; - pager_report_extent (p->upi, &offset, &len); - - _pager_lock_object (p, offset, len, MEMORY_OBJECT_RETURN_NONE, 1, - VM_PROT_NO_CHANGE, wait); + p->ops->report_extent ((struct user_pager_info *) p->upi, &start, &end); + + pager_flush_some (p, start, end - start, wait); } -/* Have the kernel write back some pages of a pager from OFFSET to - OFFSET+SIZE; if WAIT is set, then wait for them to be finally +/* Have the kernel write back some pages of a pager from START to + START+COUNT; if WAIT is set, then wait for them to be finally written before returning. */ void -pager_flush_some (struct pager *p, vm_address_t offset, - vm_size_t size, int wait) +pager_flush_some (struct pager *p, off_t start, + off_t count, int wait) { - _pager_lock_object (p, offset, size, MEMORY_OBJECT_RETURN_NONE, 1, + mutex_lock (&p->interlock); + _pager_lock_object (p, start, count, MEMORY_OBJECT_RETURN_NONE, 1, VM_PROT_NO_CHANGE, wait); + mutex_unlock (&p->interlock); } Index: pager-memcpy.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/pager-memcpy.c,v retrieving revision 1.8 diff -u -p -r1.8 pager-memcpy.c --- pager-memcpy.c 10 Apr 2002 04:09:58 -0000 1.8 +++ pager-memcpy.c 10 Apr 2002 21:37:13 -0000 @@ -36,7 +36,7 @@ if there is no fault, returns 0 and *SIZE will be unchanged. */ error_t pager_memcpy (struct pager *pager, memory_object_t memobj, - vm_offset_t offset, void *other, size_t *size, + off_t offset, void *other, size_t *size, vm_prot_t prot) { error_t err; Index: pager-return.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/pager-return.c,v retrieving revision 1.2 diff -u -p -r1.2 pager-return.c --- pager-return.c 26 Mar 1996 21:04:05 -0000 1.2 +++ pager-return.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* - Copyright (C) 1996 Free Software Foundation, Inc. + Copyright (C) 1996, 2002 Free Software Foundation, Inc. Written by Michael I. Bushnell, p/BSG. This file is part of the GNU Hurd. @@ -25,20 +25,19 @@ void pager_return (struct pager *p, int wait) { - vm_address_t offset; - vm_size_t len; + off_t start, end; - pager_report_extent (p->upi, &offset, &len); + p->ops->report_extent ((struct user_pager_info *) &p->upi, &start, &end); - _pager_lock_object (p, offset, len, MEMORY_OBJECT_RETURN_ALL, 1, - VM_PROT_NO_CHANGE, wait); + pager_return_some (p, start, end - start, wait); } void -pager_return_some (struct pager *p, vm_address_t offset, - vm_size_t size, int wait) +pager_return_some (struct pager *p, off_t start, off_t count, int wait) { - _pager_lock_object (p, offset, size, MEMORY_OBJECT_RETURN_ALL, 1, + mutex_lock (&p->interlock); + _pager_lock_object (p, start, count, MEMORY_OBJECT_RETURN_ALL, 1, VM_PROT_NO_CHANGE, wait); + mutex_unlock (&p->interlock); } Index: pager-sync.c =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/pager-sync.c,v retrieving revision 1.6 diff -u -p -r1.6 pager-sync.c --- pager-sync.c 26 Mar 1996 21:12:14 -0000 1.6 +++ pager-sync.c 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Functions for sync. - Copyright (C) 1994, 1996 Free Software Foundation + Copyright (C) 1994, 1996, 2002 Free Software Foundation This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -23,23 +23,23 @@ void pager_sync (struct pager *p, int wait) { - vm_address_t offset; - vm_size_t len; + off_t start, end; - pager_report_extent (p->upi, &offset, &len); - - _pager_lock_object (p, offset, len, MEMORY_OBJECT_RETURN_ALL, 0, - VM_PROT_NO_CHANGE, wait); + p->ops->report_extent ((struct user_pager_info *) p->upi, &start, &end); + + pager_sync_some (p, start, end - start, wait); } /* Have the kernel write back some pages of a pager; if WAIT is set, then wait for them to be finally written before returning. */ void -pager_sync_some (struct pager *p, vm_address_t offset, - vm_size_t size, int wait) +pager_sync_some (struct pager *p, off_t start, off_t count, + int wait) { - _pager_lock_object (p, offset, size, MEMORY_OBJECT_RETURN_ALL, 0, + mutex_lock (&p->interlock); + _pager_lock_object (p, start, count, MEMORY_OBJECT_RETURN_ALL, 0, VM_PROT_NO_CHANGE, wait); + mutex_unlock (&p->interlock); } Index: pager.h =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/pager.h,v retrieving revision 1.20 diff -u -p -r1.20 pager.h --- pager.h 3 Jul 1999 23:51:02 -0000 1.20 +++ pager.h 10 Apr 2002 21:37:13 -0000 @@ -1,5 +1,5 @@ /* Definitions for multi-threaded pager library - Copyright (C) 1994, 1995, 1996, 1997, 1999 Free Software Foundation, Inc. + Copyright (C) 1994,95,96,97,99, 2002 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -21,10 +21,67 @@ #include <hurd/ports.h> -/* This declaration exists to place struct user_pager_info in the proper - scope. */ +/* These declarations exists to place struct user_pager_info and + struct pager in the proper scope. */ struct user_pager_info; +struct pager; +struct pager_ops +{ + /* Read from PAGER's backing store, starting at page START, NPAGES + pages. + + The data is to be provided using either pager_data_supply or + pager_data_unavailable. + + If an error is encountered reading any pages, it is to be + reported using pager_data_read_error. + + For each indicated page, the callee *must* call exactly one of + the above methods; the pager library will not rerequest + pages. */ + error_t (*read)(struct pager *pager, + struct user_pager_info *upi, + off_t start, off_t npages); + + /* Synchronously write to PAGER's backing store the NPAGES pages + pointed to be BUF starting at page START. + + If DEALLOC is set, BUF must be deallocate be the callee. + + If an error is encountered while writing the pages to the backing + store, it must be reported using pager_data_write_error. */ + error_t (*write)(struct pager *pager, + struct user_pager_info *upi, + off_t start, off_t npages, + void *buf, int dealloc); + + /* The NPAGES pages, starting at page START, should be made writable. + + Success is to be indicated using pager_data_unlock; errors using + pager_data_unlock_error. */ + error_t (*unlock)(struct pager *pager, + struct user_pager_info *upi, + off_t start, off_t npages); + + /* Report the first (normally zero) and last valid pages that the + pager will accept and store them in START and *END + respectively. */ + error_t (*report_extent)(struct user_pager_info *upi, + off_t *start, off_t *end); + + /* The user may define this function. If non-NULL, it is called + when a pager is being deallocated after all extant send rights + have been destroyed. */ + void (*clear_user_data)(struct user_pager_info *upi); + + /* This is called when the ports library wants to drop weak + references. The pager library creates no weak references itself. + If the user doesn't either, then it's OK for this function to do + nothing or be set to NULL. */ + void (*dropweak)(struct user_pager_info *upi); +}; + /* This de-muxer function is for use within libports_demuxer. */ /* INP is a message we've received; OUTP will be filled in with a reply message. */ @@ -32,36 +89,50 @@ int pager_demuxer (mach_msg_header_t *in mach_msg_header_t *outp); /* Create a new pager. The pager will have a port created for it - (using libports, in BUCKET) and will be immediately ready - to receive requests. U_PAGER will be provided to later calls to - pager_find_address. The pager will have one user reference - created. MAY_CACHE and COPY_STRATEGY are the original values of - those attributes as for memory_object_ready. Users may create - references to pagers by use of the relevant ports library - functions. On errors, return null and set errno. */ + (using libports, in BUCKET), but associated with the OPS operation + structure and will be immediately ready to receive requests. The + pager will have one user reference created. MAY_CACHE and + COPY_STRATEGY are the original values of those attributes as for + memory_object_ready. Users may create references to pagers by use + of the relevant ports library functions. A block of memory of size + UPI_SIZE for pager state will be allocated and provided to the call + back functions or via pager_get_upi. On errors, null is returned + and sets errno is set. */ struct pager * -pager_create (struct user_pager_info *u_pager, +pager_create (struct pager_ops *ops, + size_t upi_size, struct port_bucket *bucket, boolean_t may_cache, memory_object_copy_strategy_t copy_strategy); /* Return the user_pager_info struct associated with a pager. */ struct user_pager_info * -pager_get_upi (struct pager *p); +pager_get_upi (struct pager *pager); +/* Return the port (receive right) for requests to the pager. It is + absolutely necessary that a new send right be created from this + receive right. */ +mach_port_t +pager_get_port (struct pager *pager); + +/* Return the error code of the last page error for pager PAGER at + page PAGE; this will be deleted when the kernel interface is + fixed. */ +error_t +pager_get_error (struct pager *pager, off_t page); + /* Sync data from pager PAGER to backing store; wait for all the writes to complete iff WAIT is set. */ void pager_sync (struct pager *pager, int wait); -/* Sync some data (starting at START, for LEN bytes) from pager PAGER - to backing store. Wait for all the writes to complete iff WAIT is - set. */ +/* Sync some data (starting at page START, for NPAGES pages) from pager + PAGER to backing store. Wait for all the writes to complete iff + WAIT is set. */ void pager_sync_some (struct pager *pager, - vm_address_t start, - vm_size_t len, + off_t start, off_t npages, int wait); /* Flush data from the kernel for pager PAGER and force any pending @@ -70,13 +141,12 @@ void pager_flush (struct pager *pager, int wait); - -/* Flush some data (starting at START, for LEN bytes) for pager PAGER - from the kernel. Wait for all pages to be flushed iff WAIT is set. */ +/* Flush some data (starting at page START, for NPAGES pages) for pager + PAGER from the kernel. Wait for all pages to be flushed iff WAIT + is set. */ void pager_flush_some (struct pager *pager, - vm_address_t start, - vm_size_t len, + off_t start, off_t npages, int wait); /* Flush data from the kernel for pager PAGER and force any pending @@ -86,43 +156,99 @@ void pager_return (struct pager *pager, int wait); - -/* Flush some data (starting at START, for LEN bytes) for pager PAGER - from the kernel. Wait for all pages to be flushed iff WAIT is set. - Have the kernel write back modifications. */ +/* Flush some data (starting at page START, for NPAGES pages) for pager + PAGER from the kernel. Wait for all pages to be flushed iff WAIT + is set. Have the kernel write back modifications. */ void pager_return_some (struct pager *pager, - vm_address_t start, - vm_size_t len, + off_t start, off_t npages, int wait); - -/* Offer a page of data to the kernel. If PRECIOUS is set, then this - page will be paged out at some future point, otherwise it might be - dropped by the kernel. If the page is currently in core, the - kernel might ignore this call. */ -void -pager_offer_page (struct pager *pager, - int precious, - int writelock, - vm_offset_t page, - vm_address_t buf); - + +/* Offer the NPAGES pages from BUF to the kernel for pager PAGER + starting at page START. If PRECIOUS is set, then the pages will be + paged out at some future point, otherwise they may be dropped with + out notice. IF READONLY is set, this data will be provided read + only to the kernel. In this case, any attempts to write to the + pages will cause the PAGER->UNLOCK method to be called. If DEALLOC + is set, the buffer pointed to by BUF will be deallocated. + + NB: If the data is currently in core, the kernel may ignore this + call. As such, pager_flush_some should be called if the call was + not in response to a PAGER->READ event. + + This function is normally called as a response to the PAGER->READ + method. */ +void +pager_data_supply (struct pager *pager, + int precious, int readonly, + off_t start, off_t npages, + void *buf, int dealloc); + +/* Indicate to the kernel that the NPAGES pages starting at START are + unavailable and should be supplied as anonymous (i.e. zero) + pages. + + This function is normally only called in response to the + PAGER->READ method. */ +void +pager_data_unavailable (struct pager *pager, + off_t start, off_t npages); + +/* Indicate that an error has occured while trying to read the NPAGES + pages starting at page START from pager PAGER's backing store. The + only permissable values for ERROR are: EIO, EDQUOT, and ENOSPC (all + others will be ignored and squashed to EIO). + + This is normally only called in response to the PAGER->READ + method. */ +void +pager_data_read_error (struct pager *pager, + off_t start, off_t npages, + error_t error); + +/* Indicate that an error has occured while trying to write the NPAGES + pages starting at page START to pager PAGER's backing store. The + only permissable values for ERROR are: EIO, EDQUOT, and ENOSPC (all + others will be ignored and squashed to EIO). + + This is normally only called in response to the PAGER->WRITE + method. */ +void +pager_data_write_error (struct pager *pager, + off_t start, off_t npages, + error_t error); + +/* Indicate that the NPAGES pages starting at page START in pager PAGER + have been made writable. + + This is normally only called in response to the PAGER->UNLOCK + method. */ +void +pager_data_unlock (struct pager *pager, + off_t start, off_t npages); + +/* Indicate that an error has occured unlocking (i.e. making writable) + the NPAGES pages starting at page START in pager PAGER. The only + permissable values for ERROR are: EIO, EDQUOT, and ENOSPC (all + others will be ignored and squashed to EIO). + + This is normally only called in response to the PAGER->UNLOCK + method. */ +void +pager_data_unlock_error (struct pager *pager, + off_t start, off_t npages, + error_t error); + /* Change the attributes of the memory object underlying pager PAGER. Args MAY_CACHE and COPY_STRATEGY are as for - memory_object_change_atributes. Wait for the kernel to report completion - off WAIT is set.*/ + memory_object_change_attributes. Wait for the kernel to report + completion if WAIT is set.*/ void pager_change_attributes (struct pager *pager, boolean_t may_cache, memory_object_copy_strategy_t copy_strategy, int wait); -/* Return the port (receive right) for requests to the pager. It is - absolutely necessary that a new send right be created from this - receive right. */ -mach_port_t -pager_get_port (struct pager *pager); - /* Force termination of a pager. After this returns, no more paging requests on the pager will be honored, and the pager will be deallocated. (The actual deallocation might @@ -131,65 +257,16 @@ pager_get_port (struct pager *pager); void pager_shutdown (struct pager *pager); -/* Return the error code of the last page error for pager P at address ADDR; - this will be deleted when the kernel interface is fixed. */ -error_t -pager_get_error (struct pager *p, vm_address_t addr); - -/* Try to copy *SIZE bytes between the region OTHER points to - and the region at OFFSET in the pager indicated by PAGER and MEMOBJ. - If PROT is VM_PROT_READ, copying is from the pager to OTHER; - if PROT contains VM_PROT_WRITE, copying is from OTHER into the pager. - *SIZE is always filled in the actual number of bytes successfully copied. - Returns an error code if the pager-backed memory faults; +/* Try to copy *SIZE bytes between the region OTHER points to and the + region at byte OFFSET in the pager indicated by PAGER and MEMOBJ. + If PROT is VM_PROT_READ, copying is from the pager to OTHER; if + PROT contains VM_PROT_WRITE, copying is from OTHER into the pager. + *SIZE is always filled in the actual number of bytes successfully + copied. Returns an error code if the pager-backed memory faults; if there is no fault, returns 0 and *SIZE will be unchanged. */ error_t pager_memcpy (struct pager *pager, memory_object_t memobj, - vm_offset_t offset, void *other, size_t *size, + off_t offset, void *other, size_t *size, vm_prot_t prot); - -/* The user must define this function. For pager PAGER, read one - page from offset PAGE. Set *BUF to be the address of the page, - and set *WRITE_LOCK if the page must be provided read-only. - The only permissable error returns are EIO, EDQUOT, and ENOSPC. */ -error_t -pager_read_page (struct user_pager_info *pager, - vm_offset_t page, - vm_address_t *buf, - int *write_lock); - -/* The user must define this function. For pager PAGER, synchronously - write one page from BUF to offset PAGE. In addition, mfree - (or equivalent) BUF. The only permissable error returns are EIO, - EDQUOT, and ENOSPC. */ -error_t -pager_write_page (struct user_pager_info *pager, - vm_offset_t page, - vm_address_t buf); - -/* The user must define this function. A page should be made writable. */ -error_t -pager_unlock_page (struct user_pager_info *pager, - vm_offset_t address); - -/* The user must define this function. It should report back (in - *OFFSET and *SIZE the minimum valid address the pager will accept - and the size of the object. */ -error_t -pager_report_extent (struct user_pager_info *pager, - vm_address_t *offset, - vm_size_t *size); - -/* The user must define this function. This is called when a pager is - being deallocated after all extant send rights have been destroyed. */ -void -pager_clear_user_data (struct user_pager_info *pager); - -/* The use must define this function. This will be called when the ports - library wants to drop weak references. The pager library creates no - weak references itself. If the user doesn't either, then it's OK for - this function to do nothing. */ -void -pager_dropweak (struct user_pager_info *p); #endif Index: priv.h =================================================================== RCS file: /cvsroot/hurd/hurd/libpager/priv.h,v retrieving revision 1.23 diff -u -p -r1.23 priv.h --- priv.h 25 Jul 2000 19:40:27 -0000 1.23 +++ priv.h 10 Apr 2002 21:37:14 -0000 @@ -1,5 +1,5 @@ /* Private data for pager library. - Copyright (C) 1994,95,96,97,99, 2000 Free Software Foundation, Inc. + Copyright (C) 1994,95,96,97,99, 2000,02 Free Software Foundation, Inc. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as @@ -21,6 +21,13 @@ #include "pager.h" #include <hurd/ports.h> +#if 0 +#include <stdio.h> +#define DEBUG(fmt, args...) printf (__FUNCTION__ ": " fmt "\n" , ## args) +#else +#define DEBUG(fmt, args...) do { 0; } while (0) +#endif + /* Define this if you think the kernel is sending memory_object_init out of sequence with memory_object_terminate. */ /* #undef KERNEL_INIT_RACE */ @@ -28,7 +35,8 @@ struct pager { struct port_info port; - struct user_pager_info *upi; + + struct pager_ops *ops; enum { @@ -67,6 +75,8 @@ struct pager short *pagemap; int pagemapsize; /* number of elements in PAGEMAP */ + + char upi[0]; }; struct lock_request @@ -108,11 +118,10 @@ extern int _pager_page_errors[]; /* Pagemap format */ /* These are binary state bits */ -#define PM_WRITEWAIT 0x0200 /* queue wakeup once write is done */ -#define PM_INIT 0x0100 /* data has been written */ -#define PM_INCORE 0x0080 /* kernel might have a copy */ -#define PM_PAGINGOUT 0x0040 /* being written to disk */ -#define PM_PAGEINWAIT 0x0020 /* provide data back when write done */ +#define PM_WRITEWAIT 0x0100 /* queue wakeup once write is done */ +#define PM_INIT 0x0080 /* data has been written */ +#define PM_INCORE 0x0040 /* kernel might have a copy */ +#define PM_PAGINGOUT 0x0020 /* being written to disk */ #define PM_INVALID 0x0010 /* data on disk is irrevocably wrong */ /* These take values of enum page_errors */ @@ -135,12 +144,10 @@ void _pager_wait_for_seqno (struct pager void _pager_release_seqno (struct pager *, int); void _pager_block_termination (struct pager *); void _pager_allow_termination (struct pager *); -error_t _pager_pagemap_resize (struct pager *, vm_address_t); -void _pager_mark_next_request_error (struct pager *, vm_address_t, - vm_size_t, error_t); -void _pager_mark_object_error (struct pager *, vm_address_t, - vm_size_t, error_t); -void _pager_lock_object (struct pager *, vm_offset_t, vm_size_t, int, int, +error_t _pager_pagemap_resize (struct pager *, off_t); +void _pager_mark_next_request_error (struct pager *, off_t, off_t, error_t); +void _pager_mark_object_error (struct pager *, off_t, off_t, error_t); +void _pager_lock_object (struct pager *, off_t, off_t, int, int, vm_prot_t, int); void _pager_free_structure (struct pager *); void _pager_clean (void *arg); _______________________________________________ Bug-hurd mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/bug-hurd