Re: [RFC] fastindex: parallelize index load

2017-11-08 Thread Ben Peart
This is an RFC because it works but I've not done the code cleanup, added tests, support in the update-index command to add/remove it, etc. As a result, there is no reason to point out all the places I'm not currently following the git coding style. :) I wanted to get feedback on the concept firs

Re: [RFC] fastindex: parallelize index load

2017-11-08 Thread Ben Peart
This is an RFC because it works but I've not done the code cleanup, added tests, support in the update-index command to add/remove it, etc. As a result, there is no reason to point out all the places I'm not currently following the git coding style. :) I wanted to get feedback on the concept firs

[RFC] fastindex: parallelize index load

2017-11-08 Thread Ben Peart
This patch will address the CPU cost of loading the index by adding additional data to the index that will allow us to multi-thread the loading and conversion of cache entries. It accomplishes this by adding an (optional) index extension that is a table of offsets to blocks of cache entries in the