Processing and visualising ORG task state changes

I use Org mode for all my project planning and task organisation. It integrates nicely with other Emacs functionality, like contact management and e-mail, and it fits my personal workflow quite well.

If you are already an Org user you might want to fast forward to section Retrieving State Change Data. For all of you who are not Org users yet, but are interested anyways, I have prepared a little example to prime you with some context. This is not an Org tutorial. There are lots of good ones available, this is only one of them.

Let's imagine the one thing you like to forget quite often is the weekly sit-down meeting. Why not let Org assist you with that?

As prerequisites you need to install Org mode and have at least the following stanza in your configuration: 

(setq org-agenda-files '("taskpool.org"))
(setq org-todo-keywords
  '((sequence "TODO(t)" "OPEN(o!)" "|" "DONE(d!)" "CANCELLED(c!)")))
(setq org-log-into-drawer t)

The simplest thing you can do is to create a file named taskpool.org and create a task for the sit-down meeting: 

** TODO Sit-down Meeting
    SCHEDULED: <2016-04-18 Mon 10:00 ++1w>

That basically tells Org that you want to have the task scheduled on 2016-04-18 at 10 o'clock, and from then on every following monday at the same time. The task should show up in your agenda (M-x org-agenda a):

agenda.png

As you can see, the task is scheduled every monday at 10 o'clock.

Now assume it is monday and meeting time. Position the point on the task and hit C-c C-t o. That 'opens' the task, you actively work on it. After the meeting you hit C-c C-t d and the task gets closed, meaning that you finished working on this task. To mark a cancelled meeting you can hit C-c C-t c.

After the meeting the taskpool.org file looks as follows:

taskpool1.png

As you can see, Org has added a lot of information to the task. First you will notice that the state of the task is TODO again, not DONE as expected. That is because it is a recurring task. That is also reflected by the changed schedule date.

For the rest of this text the most interesting part is the LOGBOOK drawer. It holds an entry for every state change of the task. This opens the way for all kinds of evaluations.

Retrieving State Change Data

The first step is to make the textual LOGBOOK data available as a data structure. I have implemented a lisp function (it is very similar to org-habit-parse-todo), that retrieves all log entries of a given task and returns them as a list: 

(defun hmw/org-task-retrieve-logs(&optional date-formatter pom)...)

Each list entry is a 3-tupel, consisting of the old task state, the reached task state and a timestamp. An exception is the first list entry, it is not a 3-tupel, but a string holding the task's heading. For the meeting example the list looks like this: 

("TODO Sit-down Meeting"
 ("TODO" "OPEN" "2016-04-18 Mon 10:01")
 ("OPEN" "DONE" "2016-04-18 Mon 10:15"))

The function's optional argument date-formatter can be used to format the timestamp. An example is the following code: 

(defun hmw/date-formatter(d)
  (org-date-to-gregorian d))
(hmw/org-task-retrieve-logs 'hmw/date-formatter)

This changes the returned list as follows: 

("TODO Sit-down Meeting"
 ("TODO" "OPEN" (4 18 2016))
 ("OPEN" "DONE" (4 18 2016)))

Now, with that data structure at hands we can perform the evaluations. In the next sections I present two examples - exporting to a CSV file and generating a graphical calendar view. To make this a bit more interesting I added a few more state changes to the LOGBOOK drawer.

CSV Export

An easy and general approach is to export the data to a CSV file. This allows for post-processing in your favourite third-party tools.

The entry point for generating CSV files is the following function: 

(defun hmw/org-task-dump-logs-as-csv(csv-file-name &optional with-header)...)

Called on a task the function generates a CSV file of the task's state changes and saves it under the filename csv-file-name. If called with the prefix argument, an additional CSV header line is generated.

To execute the function, place the point somewhere on the Sit-down Meeting task and hit C-u M-x hmw/org-task-dump-logs-as-csv and answer the questions shown in the minibuffer. The content of the CSV file generated from the extended meeting example looks like follows: 

FromState,ToState,Timestamp
TODO,OPEN,2016-04-18 Mon 10:01
OPEN,DONE,2016-04-18 Mon 10:15
TODO,OPEN,2016-04-25 Mon 10:01
OPEN,DONE,2016-04-25 Mon 10:15
TODO,OPEN,2016-05-02 Mon 10:01
OPEN,DONE,2016-05-02 Mon 10:15
TODO,OPEN,2016-05-09 Mon 10:01
OPEN,DONE,2016-05-09 Mon 10:15
TODO,OPEN,2016-05-16 Mon 10:01
OPEN,DONE,2016-05-16 Mon 10:15
TODO,CANCELLED,2016-05-23 Mon 10:01
TODO,OPEN,2016-05-30 Mon 10:01
OPEN,DONE,2016-05-30 Mon 10:15
TODO,OPEN,2016-06-06 Mon 10:01
TODO,OPEN,2016-06-13 Mon 10:01
OPEN,DONE,2016-06-13 Mon 10:15
TODO,OPEN,2016-06-20 Mon 10:01
OPEN,DONE,2016-06-20 Mon 10:15
TODO,CANCELLED,2016-06-27 Mon 11:01

Graphical Calendar View

To get an intuitive feeling of how the data is dispersed over a period of time, a graphical view is often helpful. For this approach I use some makeshift SVG exporter functions to draw a calendar view. The different task state changes are represented by coloured boxes.

The entry point is the following function: 

(defun hmw/org-task-generate-calendar-view(svg-file-name year)...)

Called on a task the function generates a SVG file visualising the task's state changes and saves it under the filename svg-file-name. The calendar view is generated for a given single year.

Again, place the point somewhere on the Sit-down Meeting task and hit M-x hmw/org-task-generate-calendar-view and answer the questions shown in the minibuffer. The generated image for the extended meeting example looks like follows:

meeting.png

At first glance, this graphical view reveals some properties that are hidden in the CSV dump. First, the meeting took place very regularly, except for two times, when it was cancelled. And one time, signalled by the blue box, you started the task, but forgot to finish it.

This approach is not perfect yet, but I think it opens a way to make more use out of the logged state data. The next steps would be to brush up the code, make some of the hardcoded variables adjustable and implement a few more options, like automatically open the created files in Emacs etc.

Feedback

Feel free to contact me (pkgs@c0t0d0s0.de) and tell me your opinion. Do you find this useful? Perhaps you have some feature requests? Your opinion is greatly appreciated.

License

All Emacs lisp files on this page are released under the GNU General Public License, Version 2.

Impressum