Re: [PATCH v2] Remove __init from padata_do_multithreaded and padata_mt_helper.

2020-10-27 Thread Daniel Jordan
On 10/27/20 12:46 AM, Nico Pache wrote: > On Wed, Jul 08, 2020 at 03:51:40PM -0400, Daniel Jordan wrote: > > (I was away for a while) > > > > On Thu, Jul 02, 2020 at 11:55:48AM -0400, Nico Pache wrote: > > > Allow padata_do_multithreaded function to be called

[PATCH] module: statically initialize init section freeing data

2020-10-08 Thread Daniel Jordan
alloc special flag") Reported-by: Corentin Labbe Tested-by: Corentin Labbe Tested-on: sun50i-h6-pine-h64 Tested-on: imx8mn-ddr4-evk Tested-on: sun50i-a64-bananapi-m64 Signed-off-by: Daniel Jordan --- kernel/module.c | 13 +++-- 1 file changed, 3 insertions(+), 10 deletions(-)

Re: WARNING: at kernel/workqueue.c:1473 __queue_work+0x3b8/0x3d0

2020-10-08 Thread Daniel Jordan
On Wed, Oct 07, 2020 at 09:41:17PM +0200, Corentin Labbe wrote: > I have added CONFIG_FTRACE=y and your second patch. > The boot log can be seen at http://kernel.montjoie.ovh/108789.log > > But it seems the latest dump_stack addition flood a bit. Heh, sorry for making it spew, there wasn't such a

Re: WARNING: at kernel/workqueue.c:1473 __queue_work+0x3b8/0x3d0

2020-10-05 Thread Daniel Jordan
On Thu, Oct 01, 2020 at 07:50:22PM +0200, Corentin Labbe wrote: > On Tue, Mar 03, 2020 at 04:30:17PM -0500, Daniel Jordan wrote: > > Barring other ideas, Corentin, would you be willing to boot with > > > > trace_event=initcall:*,module:* trace_options=stacktrace >

Re: WARNING: at kernel/workqueue.c:1473 __queue_work+0x3b8/0x3d0

2020-09-30 Thread Daniel Jordan
On Fri, Sep 25, 2020 at 08:12:03PM +0200, Corentin Labbe wrote: > On Tue, Mar 03, 2020 at 04:31:11PM -0500, Daniel Jordan wrote: > > On Tue, Mar 03, 2020 at 08:48:19AM +0100, Corentin Labbe wrote: > > > The patch fix the issue. Thanks! > > > > Thanks for trying i

[PATCH] padata: fix possible padata_works_lock deadlock

2020-09-02 Thread Daniel Jordan
obs from a pool") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- kernel/padata.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/kernel/padata.c b/kernel/padata.c index

[PATCH v2] padata: add another maintainer and another list

2020-08-27 Thread Daniel Jordan
At Steffen's request, I'll help maintain padata for the foreseeable future. While at it, let's have patches go to lkml too since the code is now used outside of crypto. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.or

Re: [PATCH] padata: add a reviewer

2020-08-27 Thread Daniel Jordan
On Thu, Aug 27, 2020 at 08:44:09AM +0200, Steffen Klassert wrote: > Please also consider to add yourself as one of the maintainers. Ok, sure! I'll take you up on that.

[PATCH] padata: add a reviewer

2020-08-26 Thread Daniel Jordan
I volunteer to review padata changes for the foreseeable future. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS

[PATCH 5/6] padata: fold padata_alloc_possible() into padata_alloc()

2020-07-14 Thread Daniel Jordan
There's no reason to have two interfaces when there's only one caller. Removing _possible saves text and simplifies future changes. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- Documentatio

[PATCH 4/6] padata: remove effective cpumasks from the instance

2020-07-14 Thread Daniel Jordan
A padata instance has effective cpumasks that store the user-supplied masks ANDed with the online mask, but this middleman is unnecessary. parallel_data keeps the same information around. Removing this saves text and code churn in future changes. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc

[PATCH 6/6] padata: remove padata_parallel_queue

2020-07-14 Thread Daniel Jordan
Only its reorder field is actually used now, so remove the struct and embed @reorder directly in parallel_data. No functional change, just a cleanup. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- include

[PATCH 0/6] padata cleanups

2020-07-14 Thread Daniel Jordan
cgi?p=linux-dmjordan.git;a=shortlog;h=refs/heads/padata-mt-wip-v0.5 Based on v5.8-rc5. As always, feedback is welcome. Daniel Daniel Jordan (6): padata: remove start function padata: remove stop function padata: inline single call of pd_setup_cpumasks() padata: remove effective cpumasks fro

[PATCH 1/6] padata: remove start function

2020-07-14 Thread Daniel Jordan
the future. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- crypto/pcrypt.c| 3 --- include/linux/padata.h | 1 - kernel/padata.c| 26 +- 3 files changed, 1 insert

[PATCH 2/6] padata: remove stop function

2020-07-14 Thread Daniel Jordan
urse means there are also no outstanding jobs. Removing it simplifies padata and saves text. [0] https://lore.kernel.org/linux-crypto/20191119225017.mjrak2fwa5vcc...@gondor.apana.org.au/ Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc:

[PATCH 3/6] padata: inline single call of pd_setup_cpumasks()

2020-07-14 Thread Daniel Jordan
pd_setup_cpumasks() has only one caller. Move its contents inline to prepare for the next cleanup. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- kernel/padata.c | 32 +--- 1

Re: [PATCH v2] Remove __init from padata_do_multithreaded and padata_mt_helper.

2020-07-08 Thread Daniel Jordan
(I was away for a while) On Thu, Jul 02, 2020 at 11:55:48AM -0400, Nico Pache wrote: > Allow padata_do_multithreaded function to be called after bootstrap. The functions are __init because they're currently only needed during boot, and using __init allows the text to be freed once it's over, savi

[PATCH] padata: upgrade smp_mb__after_atomic to smp_mb in padata_do_serial

2020-06-08 Thread Daniel Jordan
padata_do_serial() depends on it. Upgrade the barrier from smp_mb__after_atomic to smp_mb to get correct ordering again. Fixes: 3facced7aeed1 ("padata: remove reorder_objects") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc:

[PATCH v3 7/8] mm: make deferred init's max threads arch-specific

2020-05-27 Thread Daniel Jordan
Using padata during deferred init has only been tested on x86, so for now limit it to this architecture. If another arch wants this, it can find the max thread limit that's best for it and override deferred_page_init_max_threads(). Signed-off-by: Daniel Jordan Tested-by: Josh Tri

[PATCH v3 1/8] padata: remove exit routine

2020-05-27 Thread Daniel Jordan
padata_driver_exit() is unnecessary because padata isn't built as a module and doesn't exit. padata's init routine will soon allocate memory, so getting rid of the exit function now avoids pointless code to free it. Signed-off-by: Daniel Jordan Tested-by: Josh Triplett --- kern

[PATCH v3 0/8] padata: parallelize deferred page init

2020-05-27 Thread Daniel Jordan
s://static.sched.com/hosted_files/kvmforum2019/66/VMM-fast-restart_kvmforum2019.pdf https://www.youtube.com/watch?v=pBsHnf93tcQ https://lore.kernel.org/linux-mm/1588812129-8596-1-git-send-email-anthony.yzn...@oracle.com/ [2] https://lore.kernel.org/linux-mm/20181105165558.11698-1-daniel.m.jor...@ora

[PATCH v3 3/8] padata: allocate work structures for parallel jobs from a pool

2020-05-27 Thread Daniel Jordan
er makes sense, so remove it. If the global pool is exhausted, a parallel job is run in the current task instead to throttle a system trying to do too much in parallel. Signed-off-by: Daniel Jordan Tested-by: Josh Triplett --- include/linux/padata.h | 8 +-- ker

[PATCH v3 8/8] padata: document multithreaded jobs

2020-05-27 Thread Daniel Jordan
Add Documentation for multithreaded jobs. Signed-off-by: Daniel Jordan Tested-by: Josh Triplett --- Documentation/core-api/padata.rst | 41 +++ 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/Documentation/core-api/padata.rst b/Documentation/core

[PATCH v3 5/8] mm: don't track number of pages during deferred initialization

2020-05-27 Thread Daniel Jordan
page count can vary, just remove it. The boot message now looks like node 0 deferred pages initialised in 97ms Signed-off-by: Daniel Jordan Suggested-by: Alexander Duyck --- mm/page_alloc.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm

[PATCH v3 2/8] padata: initialize earlier

2020-05-27 Thread Daniel Jordan
padata will soon initialize the system's struct pages in parallel, so it needs to be ready by page_alloc_init_late(). The error return from padata_driver_init() triggers an initcall warning, so add a warning to padata_init() to avoid silent failure. Signed-off-by: Daniel Jordan Tested-by:

[PATCH v3 6/8] mm: parallelize deferred_init_memmap()

2020-05-27 Thread Daniel Jordan
/sbin/init as init process With this patch series: [0.231435] node 1 initialised, 24189223 pages in 32ms [0.236718] node 0 initialised, 23398907 pages in 36ms [1] https://static.sched.com/hosted_files/kvmforum2019/66/VMM-fast-restart_kvmforum2019.pdf Signed-off-by: Daniel Jord

[PATCH v3 4/8] padata: add basic support for multithreaded jobs

2020-05-27 Thread Daniel Jordan
tashin and Steve Sistare. Signed-off-by: Daniel Jordan Tested-by: Josh Triplett --- include/linux/padata.h | 29 kernel/padata.c| 152 - 2 files changed, 178 insertions(+), 3 deletions(-) diff --git a/include/linux/padata.h b/include/linu

Re: [PATCH v2 5/7] mm: parallelize deferred_init_memmap()

2020-05-21 Thread Daniel Jordan
On Thu, May 21, 2020 at 09:46:35AM -0700, Alexander Duyck wrote: > It is more about not bothering with the extra tracking. We don't > really need it and having it doesn't really add much in the way of > value. Yeah, it can probably go. > > > > @@ -1863,11 +1892,32 @@ static int __init deferred_in

[stable-4.4 5/5] padata: purge get_cpu and reorder_via_wq from padata_do_serial

2020-05-21 Thread Daniel Jordan
[ Upstream commit 065cf577135a4977931c7a1e1edf442bfd9773dd] With the removal of the padata timer, padata_do_serial no longer needs special CPU handling, so remove it. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker

[stable-4.4 4/5] padata: initialize pd->cpu with effective cpumask

2020-05-21 Thread Daniel Jordan
ot;padata: Replace delayed timer with immediate workqueue in padata_reorder") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org Signed-off-by: Herbert Xu Signed-off-by: Daniel Jordan --- kernel/padata.c

[stable-4.4 3/5] padata: Replace delayed timer with immediate workqueue in padata_reorder

2020-05-21 Thread Daniel Jordan
equent patch will fix it by replacing it with a ref counting scheme. Signed-off-by: Herbert Xu [dj: - adjust context - corrected setup_timer -> timer_setup to delete hunk - skip padata_flush_queues() hunk, function already removed in 4.4] Signed-off-by: Daniel Jordan --- in

[stable-4.4 2/5] sched/fair, cpumask: Export for_each_cpu_wrap()

2020-05-21 Thread Daniel Jordan
o the cpumask interface, 4.4 doesn't have them in the scheduler] Signed-off-by: Daniel Jordan --- include/linux/cpumask.h | 17 + lib/cpumask.c | 32 2 files changed, 49 insertions(+) diff --git a/include/linux/cpumask.h b/inc

[stable-4.4 1/5] padata: set cpu_index of unused CPUs to -1

2020-05-21 Thread Daniel Jordan
initializing the cpu_index member of all per-cpu parallel queues. Use -1 for unused ones. Signed-off-by: Mathias Krause Signed-off-by: Herbert Xu Signed-off-by: Daniel Jordan --- kernel/padata.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/padata.c b/

[stable-4.9 4/4] padata: purge get_cpu and reorder_via_wq from padata_do_serial

2020-05-21 Thread Daniel Jordan
[ Upstream commit 065cf577135a4977931c7a1e1edf442bfd9773dd ] With the removal of the padata timer, padata_do_serial no longer needs special CPU handling, so remove it. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker

[stable-4.9 1/4] padata: set cpu_index of unused CPUs to -1

2020-05-21 Thread Daniel Jordan
initializing the cpu_index member of all per-cpu parallel queues. Use -1 for unused ones. Signed-off-by: Mathias Krause Signed-off-by: Herbert Xu Signed-off-by: Daniel Jordan --- kernel/padata.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/padata.c b/

[stable-4.9 3/4] padata: initialize pd->cpu with effective cpumask

2020-05-21 Thread Daniel Jordan
ot;padata: Replace delayed timer with immediate workqueue in padata_reorder") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org Signed-off-by: Herbert Xu Signed-off-by: Daniel Jordan --- kernel/padata.c

[stable-4.14 2/4] padata: Replace delayed timer with immediate workqueue in padata_reorder

2020-05-21 Thread Daniel Jordan
equent patch will fix it by replacing it with a ref counting scheme. Signed-off-by: Herbert Xu [dj: - adjust context - corrected setup_timer -> timer_setup to delete hunk - skip padata_flush_queues() hunk, function already removed in 4.14] Signed-off-by: Daniel Jordan --- in

[stable-4.9 2/4] padata: Replace delayed timer with immediate workqueue in padata_reorder

2020-05-21 Thread Daniel Jordan
equent patch will fix it by replacing it with a ref counting scheme. Signed-off-by: Herbert Xu [dj: - adjust context - corrected setup_timer -> timer_setup to delete hunk - skip padata_flush_queues() hunk, function already removed in 4.9] Signed-off-by: Daniel Jordan --- in

[stable-4.14 4/4] padata: purge get_cpu and reorder_via_wq from padata_do_serial

2020-05-21 Thread Daniel Jordan
[ Upstream commit 065cf577135a4977931c7a1e1edf442bfd9773dd ] With the removal of the padata timer, padata_do_serial no longer needs special CPU handling, so remove it. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker

[stable-4.14 1/4] padata: set cpu_index of unused CPUs to -1

2020-05-21 Thread Daniel Jordan
initializing the cpu_index member of all per-cpu parallel queues. Use -1 for unused ones. Signed-off-by: Mathias Krause Signed-off-by: Herbert Xu Signed-off-by: Daniel Jordan --- kernel/padata.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/kernel/padata.c b/

[stable-4.14 3/4] padata: initialize pd->cpu with effective cpumask

2020-05-21 Thread Daniel Jordan
ot;padata: Replace delayed timer with immediate workqueue in padata_reorder") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org Signed-off-by: Herbert Xu Signed-off-by: Daniel Jordan --- kernel/padata.c

[stable-4.19 2/3] padata: initialize pd->cpu with effective cpumask

2020-05-21 Thread Daniel Jordan
ot;padata: Replace delayed timer with immediate workqueue in padata_reorder") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org Signed-off-by: Herbert Xu Signed-off-by: Daniel Jordan --- kernel/padata.c

[stable-4.19 1/3] padata: Replace delayed timer with immediate workqueue in padata_reorder

2020-05-21 Thread Daniel Jordan
equent patch will fix it by replacing it with a ref counting scheme. Signed-off-by: Herbert Xu [dj: - adjust context - corrected setup_timer -> timer_setup to delete hunk - skip padata_flush_queues() hunk, function already removed in 4.19] Signed-off-by: Daniel Jordan --- in

[stable-4.19 3/3] padata: purge get_cpu and reorder_via_wq from padata_do_serial

2020-05-21 Thread Daniel Jordan
[ Upstream commit 065cf577135a4977931c7a1e1edf442bfd9773dd ] With the removal of the padata timer, padata_do_serial no longer needs special CPU handling, so remove it. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-crypto@vger.kernel.org Cc: linux-ker

Re: [PATCH v2 5/7] mm: parallelize deferred_init_memmap()

2020-05-21 Thread Daniel Jordan
On Thu, May 21, 2020 at 08:00:31AM -0700, Alexander Duyck wrote: > So I was thinking about my suggestion further and the loop at the end > isn't quite correct as I believe it could lead to gaps. The loop on > the end should probably be: > for_each_free_mem_pfn_range_in_zone_from(i,

Re: [PATCH v2 5/7] mm: parallelize deferred_init_memmap()

2020-05-21 Thread Daniel Jordan
On Wed, May 20, 2020 at 06:29:32PM -0700, Alexander Duyck wrote: > On Wed, May 20, 2020 at 11:27 AM Daniel Jordan > > @@ -1814,16 +1815,44 @@ deferred_init_maxorder(u64 *i, struct zone *zone, > > unsigned long *start_pfn, > > return nr_pages; > > } &

Re: Backporting "padata: Remove broken queue flushing"

2020-05-21 Thread Daniel Jordan
On Thu, May 21, 2020 at 10:00:46AM +0200, Greg Kroah-Hartman wrote: > but these: > > > [3.16-4.19] 6fc4dbcf0276 padata: Replace delayed timer with immediate > > workqueue in padata_reorder > > [3.16-4.19] ec9c7d19336e padata: initialize pd->cpu with effective cpumask > > [3.16-4.19] 065cf577135a

[PATCH v2 1/7] padata: remove exit routine

2020-05-20 Thread Daniel Jordan
padata_driver_exit() is unnecessary because padata isn't built as a module and doesn't exit. padata's init routine will soon allocate memory, so getting rid of the exit function now avoids pointless code to free it. Signed-off-by: Daniel Jordan --- kernel/padata.c | 6 -- 1

[PATCH v2 0/7] padata: parallelize deferred page init

2020-05-20 Thread Daniel Jordan
ux-mm/1588812129-8596-1-git-send-email-anthony.yzn...@oracle.com/ [2] https://lore.kernel.org/linux-mm/20181105165558.11698-1-daniel.m.jor...@oracle.com/ Daniel Jordan (7): padata: remove exit routine padata: initialize earlier padata: allocate work structures for parallel jobs from a pool padat

[PATCH v2 5/7] mm: parallelize deferred_init_memmap()

2020-05-20 Thread Daniel Jordan
6/VMM-fast-restart_kvmforum2019.pdf Signed-off-by: Daniel Jordan --- mm/Kconfig | 6 ++--- mm/page_alloc.c | 60 - 2 files changed, 58 insertions(+), 8 deletions(-) diff --git a/mm/Kconfig b/mm/Kconfig index c1acc34c1c358..04c1da3f9f44c 10064

[PATCH v2 6/7] mm: make deferred init's max threads arch-specific

2020-05-20 Thread Daniel Jordan
Using padata during deferred init has only been tested on x86, so for now limit it to this architecture. If another arch wants this, it can find the max thread limit that's best for it and override deferred_page_init_max_threads(). Signed-off-by: Daniel Jordan --- arch/x86/mm/init_64.c

[PATCH v2 2/7] padata: initialize earlier

2020-05-20 Thread Daniel Jordan
padata will soon initialize the system's struct pages in parallel, so it needs to be ready by page_alloc_init_late(). The error return from padata_driver_init() triggers an initcall warning, so add a warning to padata_init() to avoid silent failure. Signed-off-by: Daniel Jordan --- in

[PATCH v2 7/7] padata: document multithreaded jobs

2020-05-20 Thread Daniel Jordan
Add Documentation for multithreaded jobs. Signed-off-by: Daniel Jordan --- Documentation/core-api/padata.rst | 41 +++ 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/Documentation/core-api/padata.rst b/Documentation/core-api/padata.rst index

[PATCH v2 4/7] padata: add basic support for multithreaded jobs

2020-05-20 Thread Daniel Jordan
tashin and Steve Sistare. Signed-off-by: Daniel Jordan --- include/linux/padata.h | 29 kernel/padata.c| 152 - 2 files changed, 178 insertions(+), 3 deletions(-) diff --git a/include/linux/padata.h b/include/linux/padata.h index 3bf

[PATCH v2 3/7] padata: allocate work structures for parallel jobs from a pool

2020-05-20 Thread Daniel Jordan
er makes sense, so remove it. If the global pool is exhausted, a parallel job is run in the current task instead to throttle a system trying to do too much in parallel. Signed-off-by: Daniel Jordan --- include/linux/padata.h | 8 +-- kernel/padata.c| 118 +++--

Re: Backporting "padata: Remove broken queue flushing"

2020-05-19 Thread Daniel Jordan
Hello Ben, On Tue, May 19, 2020 at 02:53:05PM +0100, Ben Hutchings wrote: > I noticed that commit 07928d9bfc81 "padata: Remove broken queue > flushing" has been backported to most stable branches, but commit > 6fc4dbcf0276 "padata: Replace delayed timer with immediate workqueue in > padata_reorder

Re: [PATCH 5/7] mm: move zone iterator outside of deferred_init_maxorder()

2020-05-07 Thread Daniel Jordan
On Thu, May 07, 2020 at 02:18:42PM -0700, Alexander Duyck wrote: > The idea behind merging ranges it to address possible cases where a > range is broken up such that there is a hole in a max order block as a > result. Gah, yes, you're right, there could be multiple ranges in a max order block, so

Re: [PATCH 5/7] mm: move zone iterator outside of deferred_init_maxorder()

2020-05-07 Thread Daniel Jordan
On Thu, May 07, 2020 at 08:26:26AM -0700, Alexander Duyck wrote: > On Wed, May 6, 2020 at 3:39 PM Daniel Jordan > wrote: > > On Tue, May 05, 2020 at 08:27:52AM -0700, Alexander Duyck wrote: > > > > Maybe it's better to leave deferred_init_maxorder alone and adapt

Re: [PATCH 6/7] mm: parallelize deferred_init_memmap()

2020-05-06 Thread Daniel Jordan
On Wed, May 06, 2020 at 06:43:35PM -0400, Daniel Jordan wrote: > On Wed, May 06, 2020 at 03:36:54PM -0700, Alexander Duyck wrote: > > On Wed, May 6, 2020 at 3:21 PM Daniel Jordan > > wrote: > > > > > > On Tue, May 05, 2020 at 07:55:43AM -0700, Alexander Duyck

Re: [PATCH 6/7] mm: parallelize deferred_init_memmap()

2020-05-06 Thread Daniel Jordan
On Wed, May 06, 2020 at 03:36:54PM -0700, Alexander Duyck wrote: > On Wed, May 6, 2020 at 3:21 PM Daniel Jordan > wrote: > > > > On Tue, May 05, 2020 at 07:55:43AM -0700, Alexander Duyck wrote: > > > One question about this data. What is the power management > &g

Re: [PATCH 5/7] mm: move zone iterator outside of deferred_init_maxorder()

2020-05-06 Thread Daniel Jordan
On Tue, May 05, 2020 at 08:27:52AM -0700, Alexander Duyck wrote: > As it turns out that deferred_free_range will be setting the > migratetype for the page. In a sparse config the migratetype bits are > stored in the section bitmap. So to avoid cacheline bouncing it would > make sense to section ali

Re: [PATCH 6/7] mm: parallelize deferred_init_memmap()

2020-05-06 Thread Daniel Jordan
On Tue, May 05, 2020 at 07:55:43AM -0700, Alexander Duyck wrote: > One question about this data. What is the power management > configuration on the systems when you are running these tests? I'm > just curious if CPU frequency scaling, C states, and turbo are > enabled? Yes, intel_pstate is loaded

Re: [PATCH 6/7] mm: parallelize deferred_init_memmap()

2020-05-04 Thread Daniel Jordan
On Mon, May 04, 2020 at 09:48:44PM -0400, Daniel Jordan wrote: > On Mon, May 04, 2020 at 05:40:19PM -0700, Alexander Duyck wrote: > > On Mon, May 4, 2020 at 4:44 PM Josh Triplett wrote: > > > > > > On May 4, 2020 3:33:58 PM PDT, Alexander Duyck > > > wrote:

Re: [PATCH 6/7] mm: parallelize deferred_init_memmap()

2020-05-04 Thread Daniel Jordan
On Mon, May 04, 2020 at 05:40:19PM -0700, Alexander Duyck wrote: > On Mon, May 4, 2020 at 4:44 PM Josh Triplett wrote: > > > > On May 4, 2020 3:33:58 PM PDT, Alexander Duyck > > wrote: > > >On Thu, Apr 30, 2020 at 1:12 PM Daniel Jordan > > > wrote: > &g

Re: [PATCH 6/7] mm: parallelize deferred_init_memmap()

2020-05-04 Thread Daniel Jordan
On Mon, May 04, 2020 at 03:33:58PM -0700, Alexander Duyck wrote: > On Thu, Apr 30, 2020 at 1:12 PM Daniel Jordan > > @@ -1778,15 +1798,25 @@ static int __init deferred_init_memmap(void *data) > > goto zone_empty; > > > > /* > > -

Re: [PATCH 5/7] mm: move zone iterator outside of deferred_init_maxorder()

2020-05-04 Thread Daniel Jordan
On Mon, May 04, 2020 at 03:10:46PM -0700, Alexander Duyck wrote: > So we cannot stop in the middle of a max order block. That shouldn't > be possible as part of the issue is that the buddy allocator will > attempt to access the buddy for the page which could cause issues if > it tries to merge the

Re: [PATCH 0/7] padata: parallelize deferred page init

2020-04-30 Thread Daniel Jordan
On Thu, Apr 30, 2020 at 06:09:35PM -0700, Josh Triplett wrote: > On Thu, Apr 30, 2020 at 04:11:18PM -0400, Daniel Jordan wrote: > > Sometimes the kernel doesn't take full advantage of system memory > > bandwidth, leading to a single CPU spending excessive time in > > init

Re: [PATCH 0/7] padata: parallelize deferred page init

2020-04-30 Thread Daniel Jordan
On Thu, Apr 30, 2020 at 05:40:59PM -0400, Pavel Tatashin wrote: > On Thu, Apr 30, 2020 at 5:31 PM Andrew Morton > wrote: > > On Thu, 30 Apr 2020 16:11:18 -0400 Daniel Jordan > > wrote: > > > > > Sometimes the kernel doesn't take full advantage of syste

Re: [PATCH 5/7] mm: move zone iterator outside of deferred_init_maxorder()

2020-04-30 Thread Daniel Jordan
Hi Alex, On Thu, Apr 30, 2020 at 02:43:28PM -0700, Alexander Duyck wrote: > On 4/30/2020 1:11 PM, Daniel Jordan wrote: > > padata will soon divide up pfn ranges between threads when parallelizing > > deferred init, and deferred_init_maxorder() complicates that by using an >

[PATCH 3/7] padata: allocate work structures for parallel jobs from a pool

2020-04-30 Thread Daniel Jordan
er makes sense, so remove it. If the global pool is exhausted, a parallel job is run in the current task instead to throttle a system trying to do too much in parallel. Signed-off-by: Daniel Jordan --- include/linux/padata.h | 8 +-- kernel/padata.c| 118 +++--

[PATCH 1/7] padata: remove exit routine

2020-04-30 Thread Daniel Jordan
padata_driver_exit() is unnecessary because padata isn't built as a module and doesn't exit. padata's init routine will soon allocate memory, so getting rid of the exit function now avoids pointless code to free it. Signed-off-by: Daniel Jordan --- kernel/padata.c | 6 -- 1

[PATCH 4/7] padata: add basic support for multithreaded jobs

2020-04-30 Thread Daniel Jordan
tashin and Steve Sistare. Signed-off-by: Daniel Jordan --- include/linux/padata.h | 29 kernel/padata.c| 152 - 2 files changed, 178 insertions(+), 3 deletions(-) diff --git a/include/linux/padata.h b/include/linux/padata.h index 3bf

[PATCH 5/7] mm: move zone iterator outside of deferred_init_maxorder()

2020-04-30 Thread Daniel Jordan
ck in deferred_grow_zone() is removed as well since this alignment is no longer guaranteed. It's not clear what value the alignment provided originally. Signed-off-by: Daniel Jordan --- mm/page_alloc.c | 88 +++-- 1 file changed, 27 insertions(+), 61

[PATCH 0/7] padata: parallelize deferred page init

2020-04-30 Thread Daniel Jordan
or_cancel() (Tejun) - fixed undo problem in VFIO (Alex) The remaining feedback, mainly resource control awareness (cgroup etc), is TODO for later series. [0] https://lore.kernel.org/linux-mm/20181105165558.11698-1-daniel.m.jor...@oracle.com/ Daniel Jordan (7): padata: remove exit routine

[PATCH 7/7] padata: document multithreaded jobs

2020-04-30 Thread Daniel Jordan
Add Documentation for multithreaded jobs. Signed-off-by: Daniel Jordan --- Documentation/core-api/padata.rst | 41 +++ 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/Documentation/core-api/padata.rst b/Documentation/core-api/padata.rst index

[PATCH 6/7] mm: parallelize deferred_init_memmap()

2020-04-30 Thread Daniel Jordan
init speedup time_ms (stdev)speedup time_ms (stdev) base --656.3 ( 7.1) -- 57.3 ( 1.5) test 8.6%599.7 ( 5.9) 62.8% 21.3 ( 1.2) Signed-off-by: Daniel Jordan --- mm/Kconfig | 6 +++--- mm/page_al

[PATCH 2/7] padata: initialize earlier

2020-04-30 Thread Daniel Jordan
padata will soon initialize the system's struct pages in parallel, so it needs to be ready by page_alloc_init_late(). The error return from padata_driver_init() triggers an initcall warning, so add a warning to padata_init() to avoid silent failure. Signed-off-by: Daniel Jordan --- in

Re: [PATCH v2 1/5] padata: make flushing work with async users

2019-09-18 Thread Daniel Jordan
On Thu, Sep 05, 2019 at 06:37:56PM -0400, Daniel Jordan wrote: > On Thu, Sep 05, 2019 at 02:17:35PM +1000, Herbert Xu wrote: > > I don't think waiting is an option. In a pathological case the > > hardware may not return at all. We cannot and should not hold off > > CP

[PATCH v3 9/9] padata: remove cpu_index from the parallel_queue

2019-09-05 Thread Daniel Jordan
With the removal of the ENODATA case from padata_get_next, the cpu_index field is no longer useful, so it can go away. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: Tejun Heo Cc: linux-crypto@vger.kernel.org Cc: linux-ker

[PATCH v3 5/9] pcrypt: remove padata cpumask notifier

2019-09-05 Thread Daniel Jordan
Now that padata_do_parallel takes care of finding an alternate callback CPU, there's no need for pcrypt's callback cpumask, so remove it and the notifier callback that keeps it in sync. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Pete

[PATCH v3 4/9] padata: make padata_do_parallel find alternate callback CPU

2019-09-05 Thread Daniel Jordan
out nested acquisition of the CPU hotplug lock later in the series. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: Tejun Heo Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- crypto/pcrypt.c

[PATCH v3 1/9] padata: allocate workqueue internally

2019-09-05 Thread Daniel Jordan
Move workqueue allocation inside of padata to prepare for further changes to how padata uses workqueues. Guarantees the workqueue is created with max_active=1, which padata relies on to work correctly. No functional change. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert

[PATCH v3 0/9] padata: use unbound workqueues for parallel jobs

2019-09-05 Thread Daniel Jordan
series when built with and without CONFIG_PADATA/CONFIG_CRYPTO_PCRYPT. v2: https://lore.kernel.org/linux-crypto/20190829173038.21040-1-daniel.m.jor...@oracle.com/ v1: https://lore.kernel.org/linux-crypto/20190813005224.30779-1-daniel.m.jor...@oracle.com/ RFC: https://lore.kernel.org/lkml/20190725212505.1505

[PATCH v3 3/9] workqueue: require CPU hotplug read exclusion for apply_workqueue_attrs

2019-09-05 Thread Daniel Jordan
lock already held. Signed-off-by: Daniel Jordan Acked-by: Tejun Heo Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- kernel/workqueue.c | 19 ++- 1 file changed, 14 insertions

[PATCH v3 6/9] padata, pcrypt: take CPU hotplug lock internally in padata_alloc_possible

2019-09-05 Thread Daniel Jordan
_lock get_online_cpus // recursive rwsem acquisition Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: Tejun Heo Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- crypto/pcrypt.c | 4 kernel/padata.c

[PATCH v3 8/9] padata: unbind parallel jobs from specific CPUs

2019-09-05 Thread Daniel Jordan
t bound to specific CPUs. The EINPROGRESS case takes care of the scenario where a parallel job is potentially running on the same CPU as padata_find_next, and with only one error code left, just use NULL instead. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: Steff

[PATCH v3 7/9] padata: use separate workqueues for parallel and serial work

2019-09-05 Thread Daniel Jordan
padata currently uses one per-CPU workqueue per instance for all work. Prepare for running parallel jobs on an unbound workqueue by introducing dedicated workqueues for parallel and serial work. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc

[PATCH v3 2/9] workqueue: unconfine alloc/apply/free_workqueue_attrs()

2019-09-05 Thread Daniel Jordan
padata will use these these interfaces in a later patch, so unconfine them. Signed-off-by: Daniel Jordan Acked-by: Tejun Heo Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- include/linux

Re: [PATCH v2 0/9] padata: use unbound workqueues for parallel jobs

2019-09-05 Thread Daniel Jordan
On Thu, Sep 05, 2019 at 02:35:48PM +1000, Herbert Xu wrote: > On Thu, Aug 29, 2019 at 01:30:29PM -0400, Daniel Jordan wrote: > > Hello, > > > > Everything in the Testing section has been rerun after the suggestion > > from Herbert last round. Thanks again to St

Re: [PATCH v2 1/5] padata: make flushing work with async users

2019-09-05 Thread Daniel Jordan
On Thu, Sep 05, 2019 at 02:17:35PM +1000, Herbert Xu wrote: > On Wed, Aug 28, 2019 at 06:14:21PM -0400, Daniel Jordan wrote: > > > > @@ -453,24 +456,15 @@ static void padata_free_pd(struct parallel_data *pd) > > /* Flush all objects out of the padata queues

[PATCH v2 1/9] padata: allocate workqueue internally

2019-08-29 Thread Daniel Jordan
Move workqueue allocation inside of padata to prepare for further changes to how padata uses workqueues. Guarantees the workqueue is created with max_active=1, which padata relies on to work correctly. No functional change. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert

[PATCH v2 9/9] padata: remove cpu_index from the parallel_queue

2019-08-29 Thread Daniel Jordan
With the removal of the ENODATA case from padata_get_next, the cpu_index field is no longer useful, so it can go away. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: Tejun Heo Cc: linux-crypto@vger.kernel.org Cc: linux-ker

[PATCH v2 6/9] padata, pcrypt: take CPU hotplug lock internally in padata_alloc_possible

2019-08-29 Thread Daniel Jordan
_lock get_online_cpus // recursive rwsem acquisition Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: Tejun Heo Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- crypto/pcrypt.c | 4 kernel/padata.c

[PATCH v2 8/9] padata: unbind parallel jobs from specific CPUs

2019-08-29 Thread Daniel Jordan
t bound to specific CPUs. The EINPROGRESS case takes care of the scenario where a parallel job is potentially running on the same CPU as padata_find_next, and with only one error code left, just use NULL instead. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: Steff

[PATCH v2 5/9] pcrypt: remove padata cpumask notifier

2019-08-29 Thread Daniel Jordan
Now that padata_do_parallel takes care of finding an alternate callback CPU, there's no need for pcrypt's callback cpumask, so remove it and the notifier callback that keeps it in sync. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Pete

[PATCH v2 2/9] workqueue: unconfine alloc/apply/free_workqueue_attrs()

2019-08-29 Thread Daniel Jordan
padata will use these these interfaces in a later patch, so unconfine them. Signed-off-by: Daniel Jordan Acked-by: Tejun Heo Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- include/linux

[PATCH v2 4/9] padata: make padata_do_parallel find alternate callback CPU

2019-08-29 Thread Daniel Jordan
out nested acquisition of the CPU hotplug lock later in the series. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: Tejun Heo Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- crypto/pcrypt.c

[PATCH v2 3/9] workqueue: require CPU hotplug read exclusion for apply_workqueue_attrs

2019-08-29 Thread Daniel Jordan
lock already held. Signed-off-by: Daniel Jordan Acked-by: Tejun Heo Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc: Peter Zijlstra Cc: linux-crypto@vger.kernel.org Cc: linux-ker...@vger.kernel.org --- kernel/workqueue.c | 19 ++- 1 file changed, 14 insertions

[PATCH v2 0/9] padata: use unbound workqueues for parallel jobs

2019-08-29 Thread Daniel Jordan
kernel.org/linux-crypto/20190828221425.22701-1-daniel.m.jor...@oracle.com/ Daniel Jordan (9): padata: allocate workqueue internally workqueue: unconfine alloc/apply/free_workqueue_attrs() workqueue: require CPU hotplug read exclusion for apply_workqueue_attrs padata: make padata_do_parallel find

[PATCH v2 7/9] padata: use separate workqueues for parallel and serial work

2019-08-29 Thread Daniel Jordan
padata currently uses one per-CPU workqueue per instance for all work. Prepare for running parallel jobs on an unbound workqueue by introducing dedicated workqueues for parallel and serial work. Signed-off-by: Daniel Jordan Acked-by: Steffen Klassert Cc: Herbert Xu Cc: Lai Jiangshan Cc

  1   2   >