Processing org-roam to export every film review I wrote in 2024
Table of Contents
1. Intro
I watch a lot of films. A lot of films (or movies - the word is interchangeable here). Last year I watched exactly 252 movies. That's roughly 1 movie every 0.69 days. Nice.
In an effort to practice writing (as well as just not forgetting what my thoughts were on a given film, which tended to happen a lot prior to this), I write reviews for every film I watch. This leaves me with a lot of reviews written. 56k words, which works out to around 222 words per movie for you number freaks at home.
Because there's so much, I thought it'd be nice to spit out a big doc at the end of the year in order to organise everything for somewhere more permanent than social media, like this blog.
By the end, I ended up with Every Film I Watched in 2024.
2. What data we working with?
With every film I watch, I create an org-roam node with a heading to layout the review.
2.1. Films with one review
This looks something like this:
:PROPERTIES: :ID: 52813152-7c5b-44bf-a361-e90a1690204a :END: #+title: Eraserhead (1977) #+filetags: :movie:review: #+startup: content * Description :PROPERTIES: :VISIBILITY: all :END: 1st [[id:752c4954-f4b9-4d8e-ad2a-68790beaed13][David Lynch]] [[id:8124ef7e-f2b5-4e3c-bedb-6ba5d36c3952][movie]]. Much more experimental and visually surreal compared to all of his films after. * Review :PROPERTIES: :DATE: [2024-03-16 Sat 05:52] :MASTODON_POST_URL: https://mastodon.gamedev.place/@MenacingMecha/112103976959275394 :END: It's not my favourite Lynch, but there's a lot here to love. Also a little less sure on specific specifics with this watch, but as we're firmly in the experimental space, that's less important than ever. Biggest thing to mention is just how different this is to every other Lynch film; his style didn't really take form until [[id:a7010ff9-ef02-438d-bcec-4f4e5184b174][Blue Velvet (1986)]], so here you get a much more "standard" visual-focused surrealism, that is very lite on dialogue. It's not really a negative, but it means it's a lot harder to recommend as an early Lynch film, because of just how different it is. As experimental films go, it's very accessible. It's only 90 minutes, the narrative is extremely clear, and it's surprisingly riff-able, meaning it can function as a fun hangout movie. Compare this to Inland Empire (2006), which is 2h48, comprised almost entirely of unintelligible scenes and requires you to sit silently in a dark room uninterrupted to not break the magic. ** Cut autistic reading TK
The important areas here are the filetags, review body, and the DATE property of the review. Only the text directly under the Review headline gets processed, any subsequent or child headlines are cut. This allows me to cut incomplete thoughts or private in-jokes that I'd still like to keep, but not include in a public review (in Eraserhead's case, incomplete thought).
2.2. Films with multiple reviews
Handling films with multiple reviews is a little more hacky. Instead of a "Review" headline, I instead have a "Reviews" headline, making sure to include the year in the 2nd level heading.
:PROPERTIES: :ID: 63d61837-eb3b-4092-81be-0b9fb91f9437 :END: #+title: Gremlins (1984) #+filetags: :review:movie: #+startup: content * Description :PROPERTIES: :VISIBILITY: all :END: Joe Dante little creature Christmas [[id:8124ef7e-f2b5-4e3c-bedb-6ba5d36c3952][movie]]. Prequel to Gremlins 2: The New Batch (1990). * Reviews ** 1 - 2023 :PROPERTIES: :DATE: [2023-12-22 Fri 02:50] :MASTODON_POST_URL: https://mastodon.gamedev.place/@MenacingMecha/111621886220783937 :END: ... ** 2 - 2024 :PROPERTIES: :DATE: [2024-12-27 Fri 04:00] :MASTODON_POST_URL: https://mastodon.gamedev.place/@MenacingMecha/113722782376856024 :END: ...
Very important to note here that I currently don't have a solution for multiple reviews in the same year, but I'll cross that bridge when I come to it. I think the only instance this could've happened was with Twin Peaks: Fire Walk with Me (1992), but the second rewatch of that year was close enough to the first that I didn't feel the need to write a review.
3. How we processing this?
We have two distinct steps to this:
- Filtering the
org-roamnodes into just the nodes we're interested in, and sorting them by review date. - Processing that list of the nodes we're interested in, and writing that to a new org document.
As both of these tasks take a non-inconsequential amount of time to compute, I'd like to cache the results of the first step to waste less time, as the filtered list changes much less frequently. Because of this, I'm running both steps as org-babel blocks.
3.1. Filtering & Sorting
#+NAME: movie-date-map
#+begin_src elisp :results table :colnames '(TITLE "REVIEW DATE") :cache yes
(let* ((nodes (org-roam-ql-nodes '(tags "movie" "review")))
(film-date-map (mapcar (lambda (x)
(list (format "[[id:%s][%s]]"
(org-roam-node-id x)
(org-roam-node-title x))
(with-temp-buffer (insert-file-contents (cdr (assoc "FILE" (org-roam-node-properties x))) nil)
(cond ((search-forward "\* Reviews" nil t)
(search-forward "2024"))
((search-forward "\* Review" nil t))
(t (message "No event timestamp on current line.")))
(org-mode)
(org-entry-get nil "DATE"))))
nodes))
(filtered-map (seq-filter (lambda (x)
(let ((date (nth 1 x)))
(if date
(string-match-p "2024-" date)
nil)))
film-date-map))
(sorted-map (seq-sort (lambda (a b)
(< (org-time-string-to-seconds (nth 1 a))
(org-time-string-to-seconds (nth 1 b))))
filtered-map)))
sorted-map)
#+end_src
#+RESULTS[e0e061b0f377cad45b98b60193f4370ad22de0c3]: movie-date-map
| TITLE | REVIEW DATE |
|---------------------------------------------------------+------------------------|
| [[id:0f02d620-3bdf-45bc-937a-94f01c232c7a][The Wicked City (1992)]] | [2024-01-02 Tue 04:02] |
| [[id:d2e2800e-9abd-44df-8ba2-4b009f4d15f3][Are We There Yet (2005)]] | [2024-01-02 Tue 04:32] |
| [[id:50a377f6-6cf3-4249-90d1-073f7b9fa443][An Adventure in Space and Time (2013)]] | [2024-01-05 Fri 03:23] |
| [[id:205446f4-9190-496c-b611-4e51d8f344b3][Twin Peaks: Fire Walk with Me (1992)]] | [2024-01-11 Thu 01:50] |
etc...
3.2. Mapping
#+begin_src elisp :results silent :var movie-date-map=movie-date-map
(defun get-review-text-from-filepath (path)
(with-temp-buffer
(insert-file-contents path nil)
(let ((film-title (org-get-title))
(bt (buffer-string)))
;; go to review body
(cond ((string-match "\* Reviews" bt)
(progn (search-forward "\* Reviews")
(search-forward "2024")))
((string-match "\* Review" bt)
(search-forward "\* Review"))
(t (message "No event timestamp on current line.")))
(search-forward "\:END\:")
(delete-region (point-min) (point)) ; delete everything pre-review
(insert (concat "* " film-title))
(org-mode)
(org-set-property "CUSTOM_ID" film-title)
(org-next-visible-heading 1)
(delete-region (point) (point-max)) ; delete everything post main review block (no cut sections, etc.)
(end-of-buffer)
(newline)
;; remove all links - source: https://emacs.stackexchange.com/a/10714
(goto-char (point-min))
(while (re-search-forward org-link-bracket-re nil t)
(replace-match (match-string-no-properties
(if (match-end 2) 2 1))))
;; replace all .mp4 links with html video
(goto-char (point-min))
(while (re-search-forward "http.+\.mp4" nil t)
(replace-match "#+begin_export html
<video controls>
<source src=\"\\&\" type=\"video/mp4\">
Your browser does not support the video tag.
</video>
,#+end_export")))
(buffer-string)))
(defun get-ids-from-map (map)
(mapcar (lambda (i) (car i))
map))
(defun get-review-text-from-id (id)
(let ((path (with-temp-buffer
(org-link-open-from-string id)
(buffer-file-name))))
(get-review-text-from-filepath path)))
(let ((wc (current-window-configuration))
(doc-path "~/2024-movies.org"))
(with-temp-file doc-path
(insert "#+TITLE: Every Film I Watched in 2024
,#+AUTHOR: MenacingMecha
,#+OPTIONS: toc:nil
,*CONTENT WARNING:*
I watch a very large variety of films. Everything from Critereon classics to pervert VHS trash. I frequently like to go out of my comfort zone and take huge gambles, or sometimes even just dive into things I know will appal me. As such, expect a lot of dark and/or poorly aged content.
Unless otherwise specified, I won't include any spoilers for things that I think would harm the viewing experience to know going in, but you might have a different line for how much is acceptable.
,#+TOC: headlines 1
")
(->> (get-ids-from-map movie-date-map)
(mapcar 'get-review-text-from-id)
(mapcar 'insert)))
(set-window-configuration wc)
(find-file-other-window doc-path))
#+end_src
Some important extra steps here:
- Links to other
orgid's are stripped, as they are not properly exported. - Links to videos are substituted with HTML video tags, causing them to be embedded.
From this, I have the org file I can throw into my org-publish blog content folder, ending up with Every Film I Watched in 2024.
4. Issues & Next Steps
4.1. Film Posters
Currently there are no film posters included alongside the titles, leading to an overly text-heavy output. While I always include film posters in my Mastodon posts, I left it too late in the year before I realised I didn't have a good workflow for including these.
While there's not much I can do for 2024's doc (at least not without an unreasonable amount of hassle), I've adressed this for next year by also adding a :POSTER: property to the review headline, with a URL link I can paste under the title headline when processing the end of year doc.
4.2. Spoiler Sections
Additionally, while fairly rare, I do sometimes get into major spoilers for a particular film. It would be nice if I could display these as an opt-in blocks. IIRC there's easy HTML-only ways to achieve this, so it shouldn't be too tricky.