Björn Kettunen <[email protected]> writes: > Ihor Radchenko <[email protected]> writes: > >> Björn Kettunen <[email protected]> writes: >> >>>> I think I got something. I wrote a function that merges table with >>>> matching file-names as keys based of the description in >>>> org-clock-get-table-data. >>>> >>>> The result is very similar to what file-with-archives for a single file >>>> did before the patch. >>>> I.e. headings from main file and archive are not merged. >>>> >>>> >>>> I haven't updated the documentation or removed any of the commented out >>>> code. >>>> >>>> I'm mostly looking for feedback on how the merge table data function is >>>> called >>>> and on the function itself. >> >> I see several issues. For example, you are assuming that Org files >> always have .org extension. That's not necessarily the case. > > OK what's the best approach to this? Maybe using org-agenda-file-regexp? > >> Also, you are trying to merge everything at the same time, from a full >> list of tables. I think you can do it much simpler. >> Instead of (setq files (org-add-archive-files files)), you can call >> org-add-archive-files one by one. That way, you will have readily >> available list of archive file name that you can group. Merging within >> the same group will be much, much easier - just sum up total file time, >> and append the data. > > I think you're right when it comes to the heuristics here, it's easier > and more reliable that way but the merging of the table data itself is > largely the same. The reason why I merged the data this way was that > the table data is gathered further below. > > The merge table data function can largely stay the > same but the determination of what has to be merged should be adjusted.
Here's my updated patch. I added a small helper function to reuse the same path whenever the scope is cons files or file-with-archives.
>From 6753f2ceadd84a45b8e929b17d32293b9a8f2cf1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Bj=C3=B6rn=20Kettunen?= <[email protected]> Date: Tue, 3 Mar 2026 22:47:12 +0200 Subject: [PATCH 1/2] org-clock: clock-report refactor list and function scope * lisp/org-clock.el (org-dblock-write:clocktable): Expand files in directories if any of the entries in scope is a directory. Just like in org-agenda-files. Function scope is now evaluated before any other scope and can return a scope by itself. (org-clock-merge-table-data): A new function which can merge clocktable data for a list of table belonging to a file. (org-clock-get-table-data-with-archives): A new function to get table data for a file with it's archives. * lisp/org.el (org-file-list-expand): (org-agenda-files): Refactor file expansion into separate function. * doc/org-manual.org: Document. * etc/ORG-NEWS: Announce --- doc/org-manual.org | 20 ++++----- etc/ORG-NEWS | 23 ++++++++++ lisp/org-clock.el | 109 ++++++++++++++++++++++++++++++++------------- lisp/org.el | 20 ++++++--- 4 files changed, 124 insertions(+), 48 deletions(-) diff --git a/doc/org-manual.org b/doc/org-manual.org index 904e1270d..d282f419a 100644 --- a/doc/org-manual.org +++ b/doc/org-manual.org @@ -7103,16 +7103,16 @@ *** The clock table The scope to consider. This can be any of the following: - | =nil= | the current buffer or narrowed region | - | =file= | the full current buffer | - | =subtree= | the subtree where the clocktable is located | - | =treeN= | the surrounding level N tree, for example =tree3= | - | =tree= | the surrounding level 1 tree | - | =agenda= | all agenda files | - | =("file" ...)= | scan these files | - | =FUNCTION= | scan files returned by calling {{{var(FUNCTION)}}} with no argument | - | =file-with-archives= | current file and its archives | - | =agenda-with-archives= | all agenda files, including archives | + | =nil= | the current buffer or narrowed region | + | =file= | the full current buffer | + | =subtree= | the subtree where the clocktable is located | + | =treeN= | the surrounding level N tree, for example =tree3= | + | =tree= | the surrounding level 1 tree | + | =agenda= | all agenda files | + | =file-with-archives= | current file and its archives | + | =agenda-with-archives= | all agenda files, including archives | + | =([scope] "file" "dir" "...)= | scan these files or files in directories, scope can be file or file-with-archives | + | f=FUNCTION= | call {{{var(FUNCTION)}}} with no argument process any scope it returns | - =:block= :: diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index 2dcd86aee..a20dc685e 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -86,6 +86,20 @@ This tangles the block to four files: - /backup/config.el - /backup/backup.el +*** Clocktable options =:scope function= and =:scope (file)= have changed. + +- ~function~ + Now called before any scope is processed. + It can now also return any other scope by itself which is then + processed after. + +- ~(file)~ + Can now also include directories which are resolved just like in + ~org-agenda-files~. + The first entry of this list can also be either ~file~ or ~file-archives~. + If this is the case then all following entries are considered + like the ~file~ or ~file-with-archives~ scope. + ** New and changed options # Changes dealing with changing default values of customizations, @@ -106,6 +120,15 @@ variable. Given the completed and total number of tasks, format the percent cookie =[N%]=. +*** New function ~org-agenda-directory-files-recursively~ + +Expand list of flies according to ~org-agenda-file-regexp~. + +*** New function ~org-clock-merge-table-data~ + +Takes a list of clocktable data tables, merge them all +under file or the file name of the first table. + ** Removed or renamed functions and variables ** Miscellaneous diff --git a/lisp/org-clock.el b/lisp/org-clock.el index ce2d23a9b..ba9fd77fa 100644 --- a/lisp/org-clock.el +++ b/lisp/org-clock.el @@ -2677,22 +2677,21 @@ (defun org-dblock-write:clocktable (params) (catch 'exit (let* ((scope (plist-get params :scope)) (base-buffer (org-base-buffer (current-buffer))) + (scope (or (and (functionp scope) + (funcall scope)) + scope)) (files (pcase scope - (`agenda + ((or `agenda `agenda-with-archives) (org-agenda-files t)) - (`agenda-with-archives - (org-add-archive-files (org-agenda-files t))) - (`file-with-archives - (let ((base-file (buffer-file-name base-buffer))) - (and base-file - (org-add-archive-files (list base-file))))) - ((or `nil `file `subtree `tree + ((or `file-with-archives) + (list (buffer-file-name base-buffer))) + ((or `nil `subtree `tree `file (and (pred symbolp) (guard (string-match "\\`tree\\([0-9]+\\)\\'" (symbol-name scope))))) base-buffer) - ((pred functionp) (funcall scope)) ((pred consp) scope) + ((pred stringp) scope) ;; To not break previous function calls here (_ (user-error "Unknown scope: %S" scope)))) (block (plist-get params :block)) (ts (plist-get params :tstart)) @@ -2704,7 +2703,23 @@ (defun org-dblock-write:clocktable (params) (formatter (or (plist-get params :formatter) org-clock-clocktable-formatter 'org-clocktable-write-default)) - cc) + (multifile + ;; Even though `file-with-archives' can consist of + ;; multiple files, we consider this is one extended file + ;; instead. + (and (not hide-files) + (consp files) + (not (eq scope 'file-with-archives)))) + cc) + + (when (consp files) + (when-let* ((cons-scope (car files)) + (cons-scope (and (symbolp cons-scope) + cons-scope))) + (setq scope cons-scope) + (setq files (cdr files))) + (setq files (org-agenda-directory-files-recursively files))) + ;; Check if we need to do steps (when block ;; Get the range text for the header @@ -2718,20 +2733,22 @@ (defun org-dblock-write:clocktable (params) (org-clocktable-steps params) (throw 'exit nil)) - (org-agenda-prepare-buffers (if (consp files) files (list files))) + (unless (consp files) + (org-agenda-prepare-buffers (list files))) (let ((origin (point)) (tables - (if (consp files) - (mapcar (lambda (file) - (with-current-buffer (find-buffer-visiting file) - (save-excursion - (save-restriction - (org-clock-get-table-data file params))))) - files) + (if (consp files) + (if (eq scope 'file-with-archives) + (mapcar (lambda (file) + (org-clock-get-table-data-with-archives + file params)) files) + (mapcar (lambda (file) + (org-clock--get-table-data1 file params)) + files)) ;; Get the right restriction for the scope. (save-restriction - (cond + (cond ((not scope)) ;use the restriction as it is now ((eq scope 'file) (widen)) ((eq scope 'subtree) (org-narrow-to-subtree)) @@ -2739,25 +2756,18 @@ (defun org-dblock-write:clocktable (params) (while (org-up-heading-safe)) (org-narrow-to-subtree)) ((and (symbolp scope) - (string-match "\\`tree\\([0-9]+\\)\\'" + (string-match "\\`tree\\([0-9]+\\)\\'" (symbol-name scope))) (let ((level (string-to-number - (match-string 1 (symbol-name scope))))) + (match-string 1 (symbol-name scope))))) (catch 'exit (while (org-up-heading-safe) - (looking-at org-outline-regexp) - (when (<= (org-reduced-level (funcall outline-level)) + (looking-at org-outline-regexp) + (when (<= (org-reduced-level (funcall outline-level)) level) (throw 'exit nil)))) (org-narrow-to-subtree)))) - (list (org-clock-get-table-data nil params))))) - (multifile - ;; Even though `file-with-archives' can consist of - ;; multiple files, we consider this is one extended file - ;; instead. - (and (not hide-files) - (consp files) - (not (eq scope 'file-with-archives))))) + (list (org-clock-get-table-data nil params)))))) (funcall formatter origin @@ -3112,6 +3122,43 @@ (defun org-clocktable-steps (params) (setq start next)) (end-of-line 0)))) +(defun org-clock--get-table-data1 (file params) + "Get clocktable-data for FILE with PARAMS." + (org-agenda-prepare-buffers (list file)) + (with-current-buffer + (find-buffer-visiting file) + (save-excursion + (save-restriction + (org-clock-get-table-data file params))))) + +(defun org-clock-get-table-data-with-archives (file params) + "Get clocktable data with archives for FILE with parameters PARAMS. +If file-only don't add-archives" + (let* ((file-plus-archives (org-add-archive-files (list file))) + (tables (mapcar (lambda (file) + (org-clock--get-table-data1 file params)) + file-plus-archives))) + (org-clock-merge-table-data tables file))) + +(defun org-clock-merge-table-data (tables &optional file) + "Merge table data for TABLES for FILE. +When FILE isn't given assume FILE as the file of the first table. +TABLES is list of table data returned in the format returned by +`org-clock-get-table-data'. +Returns the same tables but with each table merged." + (let* ((file (or file (caaar tables))) + ;; Make sure that the first element of the first table + ;; was a file-name. + (file (and (stringp file) file)) + (total-time 0) entries) + (while-let ((table (pop tables))) + (incf total-time (nth 1 table)) + (when-let* ((new-entries (car (nthcdr 2 table)))) + + (setq entries (append new-entries entries)))) + (list file total-time + entries ))) + (defun org-clock-get-table-data (file params) "Get the clocktable data for file FILE, with parameters PARAMS. FILE is only for identification - this function assumes that diff --git a/lisp/org.el b/lisp/org.el index fc51d4ba3..05607bd2c 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -15977,6 +15977,18 @@ (defun org-switchb (&optional arg) (mapcar #'list (mapcar #'buffer-name blist)) nil t)))) + +(defun org-agenda-directory-files-recursively (files) + "Expand list of FILES according to `org-agenda-file-regexp'." + (apply 'append + (mapcar (lambda (f) + (if (file-directory-p f) + (directory-files + f t org-agenda-file-regexp) + (list (expand-file-name f org-directory)))) + files))) + + (defun org-agenda-files (&optional unrestricted archives) "Get the list of agenda files. Optional UNRESTRICTED means return the full list even if a restriction @@ -15990,13 +16002,7 @@ (defun org-agenda-files (&optional unrestricted archives) ((stringp org-agenda-files) (org-read-agenda-file-list)) ((listp org-agenda-files) org-agenda-files) (t (error "Invalid value of `org-agenda-files'"))))) - (setq files (apply 'append - (mapcar (lambda (f) - (if (file-directory-p f) - (directory-files - f t org-agenda-file-regexp) - (list (expand-file-name f org-directory)))) - files))) + (setq files (org-agenda-directory-files-recursively files)) (when org-agenda-skip-unavailable-files (setq files (delq nil (mapcar (lambda (file) -- 2.53.0
