branch: externals/el-job
commit 0d3dc5b3ca8abb116c33a6ab430a7d1197725468
Author: Martin Edström <meedst...@runbox.eu>
Commit: Martin Edström <meedst...@runbox.eu>
Docs & comments
---
README.org | 26 +++++++++++++++++---------
el-job.el | 21 +++++++++++++++------
2 files changed, 32 insertions(+), 15 deletions(-)
diff --git a/README.org b/README.org
index 19dc4d2e9e..9b0621b9ea 100644
--- a/README.org
+++ b/README.org
@@ -42,27 +42,35 @@ the default one. It is reimplemented in native code,
reducing GC churn.
To undo this change, set 'fast-read-process-output' to nil.
#+end_src
-** News 2.1.0
+
+** News 2.4
+- Jobs must now have =:inputs=. If =:inputs= nil and there was nothing
queued, =el-job-launch= will no-op and return the symbol =inputs-were-empty=.
+
+** News 2.3
+- Some renames to follow Elisp convention
+ - =el-job:timestamps= and friends now =el-job-timestamps=.
+
+** News 2.1
- DROP SUPPORT Emacs 28
- It likely has not been working for a while anyway. Maybe works on the
[[https://github.com/meedstrom/el-job/tree/v0.3][v0.3 branch]], from 0.3.26+.
-** News 2.0.0
+** News 2.0
- Jobs must now have =:id= (no more anonymous jobs).
- Pruned many code paths.
-** News 1.1.0
+** News 1.1
- Changed internals so that all builds of Emacs can be expected to perform
similarly well.
-** News 1.0.0
+** News 1.0
- No longer keeps processes alive forever. All jobs are kept alive for up to
30 seconds of disuse, then reaped.
- Pruned many code paths.
- Many arguments changed, and a few were removed. Consult the docstring of
=el-job-launch= again.
** Limitations
-2. The return value from the =:funcall-per-input= function must always be a
list with a fixed length, where the elements are also lists.
+1. The return value from the =:funcall-per-input= function must always be a
list with a fixed length, where the elements are also lists.
- For example, org-node passes =:funcall-per-input
#'org-node-parser--scan-file= to el-job, and if you look in
[[https://github.com/meedstrom/org-node/blob/main/org-node-parser.el][org-node-parser.el]]
for the defun of =org-node-parser--scan-file=, this is its final return value:
+ For example, org-node passes =:funcall-per-input
#'org-node-parser--scan-file= to el-job, and if you look in
[[https://github.com/meedstrom/org-node/blob/main/org-node-parser.el][org-node-parser.el]]
for the defun of =org-node-parser--scan-file=, it always returns a list of 6
items:
#+begin_src elisp
(list (if missing-file (list missing-file)) ; List of 0 or 1 item
@@ -73,10 +81,10 @@ To undo this change, set 'fast-read-process-output' to nil.
(if problem (list problem)))) ; List of 0 or 1 item
#+end_src
- It may seem clunky to return lists of only one item, but you could consider
it a minor expense in exchange for simpler library code.
+ It may look clunky to return sub-lists of only one item, but you could
consider it a minor expense in exchange for simpler library code.
-3. Some data types cannot be exchanged with the children: those whose printed
form look like =#<...>=. For example, =#<buffer notes.org>=, =#<obarray
n=94311>=, =#<marker at 3102 in README.org>=.
+2. Some data types cannot be exchanged with the children: those whose printed
form look like =#<...>=. For example, =#<buffer notes.org>=, =#<obarray
n=94311>=, =#<marker at 3102 in README.org>=.
IIUC, this sort of data only has meaning within the current process -- so
even if you could send it, it would not be usable by the recipient anyway.
- In days past, hash tables also looked like that when printed, but not since
Emacs 25 or so: they now look like =#s(hash-table data ...)=, which works fine
to read back.
+3. For now, this library tends to be applicable only to a narrow set of
use-cases, since you can only pass one =:inputs= list which would tend to
contain a single kind of thing, e.g. it could be a list of files to visit, to
be split between child processes. In many potential use-cases, you'd actually
want multiple input lists and split them differently, and that's not supported
yet.
diff --git a/el-job.el b/el-job.el
index 7cb0f6b06d..6bb46f984d 100644
--- a/el-job.el
+++ b/el-job.el
@@ -115,9 +115,10 @@ an .eln anyway, without your having to recompile on save."
(symbol-function (cdr elem))))))
;; FIXME: comp sometimes deletes old eln during
;; recompilation, but does not load the new eln,
- ;; at least not in a way that updates
- ;; `load-history'. Current workaround is return
- ;; nil, ie fall back on FILE; not ideal.
+ ;; at least not in a way that updates the
+ ;; `native-comp-unit-file'. Current workaround
+ ;; is return nil, ie fall back on FILE; not
+ ;; ideal.
(when (file-exists-p eln)
eln))))
file)))
@@ -170,7 +171,7 @@ find the correct file."
`(lambda (&rest _) ,elc)))
(when (native-comp-available-p)
;; FIXME: Guix overrides `comp-el-to-eln-rel-filename' to
- ;; output filenames with NO HASH! So compiling now can result
+ ;; output filenames with no hash! So compiling now can result
;; in an .eln in ~/.emacs.d/ that will always take precedence
;; over the one shipped by Guix. If we want to cover for that,
;; it'd be safer to compile into /tmp with a filename based on
@@ -180,7 +181,7 @@ find the correct file."
(native-compile-async (list loaded)))
;; Native comp may take a while, so build and return .elc this
;; time. We should not pick a preexisting .elc from `load-path'
- ;; if Emacs is now running interpreted code, since that currently
+ ;; if Emacs is running interpreted code now, since that currently
;; running code is likely newer.
(if (or (file-newer-than-file-p elc loaded)
(byte-compile-file loaded))
@@ -382,6 +383,8 @@ The subprocesses do not inherit `load-path', it is the
current Emacs
process that locates files \(by inspecting `load-history', via
`el-job--ensure-compiled-lib'), then gives them to each subprocess.
+Tip: To let them inherit `load-path' anyway, add it to INJECT-VARS.
+
INPUTS is a list that will be split by up to the output of
`num-processors', and this determines how many subprocesses will spawn.
@@ -395,7 +398,8 @@ If INPUTS returns nil, do nothing and return the symbol
The subprocesses have no access to current Emacs state. The only way
they can affect current state, is if FUNCALL-PER-INPUT returns data,
-which is then handled by CALLBACK function in the current Emacs.
+which is then handled by CALLBACK function in the current Emacs, as
+described earlier.
Emacs stays responsive to user input up until all subprocesses finish,
which is when their results are merged and CALLBACK is executed.
@@ -698,6 +702,11 @@ more input in the queue."
;; Somehow, it took 700 lines of code to get here.
(when .callback
(funcall .callback (el-job--zip-all .result-sets) job))
+ ;; REVIEW: Either
+ ;; 1. get rid of this code path (see comments in `el-job-launch' re.
+ ;; queued inputs); or
+ ;; 2. consider if we need to check length of .queued-inputs to maybe
+ ;; respawn all processes (see what `el-job-launch' does).
(when .queued-inputs
(el-job--exec-workload job))))))
