I've committed the below to the tree; Kris has performed quite a lot of performance and stability testing, but since he tends to run with specific workloads, I wouldn't be surprised if there are minor (and hopefully quickly corrected) issues reported. If you experience hangs or other problems, please make sure to run with INVARIANTS and WITNESS, which will help with debugging.
This patch represents a significant part of the performance improvements for improved scalability on 7-CURRENT with respect to threaded databases, and is only possible because of the long hours of work Attilio, Kris, John, and others have put in preparing the sxlock optimizations this patch depends on, as well as reviewing and testing the patch.
Please let me know if you experience any problems. Thanks, Robert N M Watson Computer Laboratory University of Cambridge ---------- Forwarded message ---------- Date: Wed, 4 Apr 2007 09:11:34 +0000 (UTC) From: Robert Watson <[EMAIL PROTECTED]> To: [EMAIL PROTECTED], [EMAIL PROTECTED], [email protected] Subject: cvs commit: src/sys/compat/linux linux_file.c src/sys/compat/svr4 svr4_filio.c src/sys/dev/streams streams.c src/sys/fs/devfs devfs_vnops.c src/sys/fs/fdescfs fdesc_vfsops.c fdesc_vnops.c src/sys/fs/fifofs fifo_vnops.c src/sys/fs/unionfs union_subr.c ... rwatson 2007-04-04 09:11:34 UTC FreeBSD src repository Modified files: sys/compat/linux linux_file.c sys/compat/svr4 svr4_filio.c sys/dev/streams streams.c sys/fs/devfs devfs_vnops.c sys/fs/fdescfs fdesc_vfsops.c fdesc_vnops.c sys/fs/fifofs fifo_vnops.c sys/fs/unionfs union_subr.c sys/kern kern_descrip.c kern_event.c kern_fork.c subr_witness.c sys_generic.c uipc_mqueue.c uipc_syscalls.c uipc_usrreq.c vfs_cache.c vfs_lookup.c vfs_mount.c vfs_syscalls.c sys/netsmb smb_dev.c sys/opencrypto cryptodev.c sys/security/audit audit_bsm_klib.c sys/sys filedesc.h Log: Replace custom file descriptor array sleep lock constructed using a mutex and flags with an sxlock. This leads to a significant and measurable performance improvement as a result of access to shared locking for frequent lookup operations, reduced general overhead, and reduced overhead in the event of contention. All of these are imported for threaded applications where simultaneous access to a shared file descriptor array occurs frequently. Kris has reported 2x-4x transaction rate improvements on 8-core MySQL benchmarks; smaller improvements can be expected for many workloads as a result of reduced overhead. - Generally eliminate the distinction between "fast" and regular acquisisition of the filedesc lock; the plan is that they will now all be fast. Change all locking instances to either shared or exclusive locks. - Correct a bug (pointed out by kib) in fdfree() where previously msleep() was called without the mutex held; sx_sleep() is now always called with the sxlock held exclusively. - Universally hold the struct file lock over changes to struct file, rather than the filedesc lock or no lock. Always update the f_ops field last. A further memory barrier is required here in the future (discussed with jhb). - Improve locking and reference management in linux_at(), which fails to properly acquire vnode references before using vnode pointers. Annotate improper use of vn_fullpath(), which will be replaced at a future date. In fcntl(), we conservatively acquire an exclusive lock, even though in some cases a shared lock may be sufficient, which should be revisited. The dropping of the filedesc lock in fdgrowtable() is no longer required as the sxlock can be held over the sleep operation; we should consider removing that (pointed out by attilio). Tested by: kris Discussed with: jhb, kris, attilio, jeff Revision Changes Path 1.103 +17 -4 src/sys/compat/linux/linux_file.c 1.35 +4 -4 src/sys/compat/svr4/svr4_filio.c 1.55 +2 -2 src/sys/dev/streams/streams.c 1.143 +3 -1 src/sys/fs/devfs/devfs_vnops.c 1.56 +2 -2 src/sys/fs/fdescfs/fdesc_vfsops.c 1.104 +5 -5 src/sys/fs/fdescfs/fdesc_vnops.c 1.136 +3 -1 src/sys/fs/fifofs/fifo_vnops.c 1.91 +2 -2 src/sys/fs/unionfs/union_subr.c 1.307 +174 -170 src/sys/kern/kern_descrip.c 1.109 +9 -9 src/sys/kern/kern_event.c 1.270 +2 -2 src/sys/kern/kern_fork.c 1.228 +0 -2 src/sys/kern/subr_witness.c 1.155 +11 -12 src/sys/kern/sys_generic.c 1.21 +10 -11 src/sys/kern/uipc_mqueue.c 1.250 +14 -9 src/sys/kern/uipc_syscalls.c 1.201 +10 -9 src/sys/kern/uipc_usrreq.c 1.108 +4 -4 src/sys/kern/vfs_cache.c 1.100 +2 -2 src/sys/kern/vfs_lookup.c 1.252 +2 -2 src/sys/kern/vfs_mount.c 1.436 +26 -25 src/sys/kern/vfs_syscalls.c 1.32 +3 -3 src/sys/netsmb/smb_dev.c 1.33 +3 -1 src/sys/opencrypto/cryptodev.c 1.6 +2 -2 src/sys/security/audit/audit_bsm_klib.c 1.76 +15 -61 src/sys/sys/filedesc.h _______________________________________________ [email protected] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-performance To unsubscribe, send any mail to "[EMAIL PROTECTED]"
