Zippers And CljFri May 8, 2020
So recently, I had to use
zippers at work. Specifically, the Clojure implementation. There were some close-to-arbitrary transformations I needed to do with some close-to-arbitrary trees and it turned out that
zippers were more efficient than the alternatives1.
Using them this way, combined with the general state of the world and my free time, finally tipped me into doing some more Common Lisp development. Before, I go any further, let me be clear about something.
I Like Clojure
Its logo is up top in the language bar, I was one of the inaugural members of the Toronto Clojure User Group, I recommend it as a first lisp you should learn, and have for about six years now. I'm also painfully aware of the shortcomings of Common Lisp, and make no excuses for them.
- I don't like the JVM. It's slow as balls, its' deployment options are less than ideal for my purposes, its' error system is at best useless, and Clojure without it is unlikely.
- Clojurescript build incompatiblities are, if anything, worse2.
- I don't like the underlying licensing decisions.
These are deep reasons to stay away. They're not the sort of thing I can paper over with a library or two. Fixing them would mean a superhuman amount of work poured into the underlying technical and social infrastructure, and I'm not into it. I wouldn't be into it even if the community was interested in heading that way, and near as I can tell, they're not particularly.
Whether or not I think you should learn Clojure as your first3 lisp, it definitely wasn't my first lisp. The more uniform, mostly-better-thought-out interface, lack of historical baggage and functional data structures are not enough to pull me all the way over.
It is enough for me to start plotting a smash-and-grab of as much of the stuff I like as I can carry. Which is exactly what
clj represents. As of this writing, it defines and exports exactly four symbols:
->>. This is a tiny beginning of the list, and I fully plan to put something more substantial together using
test-utils and possibly
optima. Stay tuned to that repo if you're interested, but it's not the focus today.
The thing that percipitated this thought was having used the Clojure Zipper implementation. So, obviously, this is something I want next time I need to manipulate trees in Common Lisp. The paper is here, and unless you have a terminal phobia of datastructures4, you should go read it. It's six pages, they're light, and one of them taken up by the intro and references.
The operations defined in the paper are
delete. There's a few conveniences defined for the Clojure version, and I've implemented some of my own stuff too. Lets go through the main file in almost-literate style.
First up, we have constructors.
(defstruct path (left) (path) (right)) (defstruct loc (node) (path) (fn-branch?) (fn-children) (fn-make-node)) ;;;;;;;;;; Constructors (defun zipper (branch? children make-node root) (make-loc :node root :fn-branch? branch? :fn-children children :fn-make-node make-node)) (defmethod make-zipper ((thing list)) (zipper #'listp #'identity (lambda (node children) (declare (ignore node)) children) thing)) (defun make-node (zipper children) (funcall (loc-fn-make-node zipper) zipper children))
You can see influence from both
clojure.zip and the paper here. I'm taking the lead from the paper by explicitly separating the
path triple our from the
loc definition. However, I'm not explicitly defining my own
type tree the way that Huet does. Instead, I'm going to be dealing with assorted
lisp trees. These could be implemented as
hashes, or any number of other formats. I'm going to implement a few type-distpatching built-ins, including the
make-zipper list method above, but the basic
zipper function just needs to take an interface as input in the form of
make-node arguments. This is the same solution that the Clojure implementation went with, and I see no reason to go a different way. The only material difference is that theirs uses the Clojure
metadata system, while I explicitly define slots in the
Now that we can construct, we need to be able to select.
;;;;;;;;;; Selectors (defun branch? (zipper) (funcall (loc-fn-branch? zipper) (loc-node zipper))) (defun children (zipper) (funcall (loc-fn-children zipper) (loc-node zipper))) (defun node (zipper) (loc-node zipper)) (defun path (zipper) (loc-path zipper)) (defun lefts (zipper) (when (loc-path zipper) (reverse (path-left (loc-path zipper))))) (defun rights (zipper) (when (loc-path zipper) (path-right (loc-path zipper))))
The basic navigation is four functions;
;;;;;;;;;; Navigation ;;;;;;;;;;;;;;; Basic navigation (defun down (zipper) (when (children zipper) (let ((fresh (copy-loc zipper))) (setf (loc-node fresh) (first (children zipper)) (loc-path fresh) (make-path :left nil :path (loc-path zipper) :right (rest (children zipper)))) fresh))) (defun up (zipper) (when (path zipper) (let ((fresh (copy-loc zipper))) (setf (loc-node fresh) (make-node zipper (append (reverse (path-left (path zipper))) (cons (loc-node zipper) (path-right (path zipper))))) (loc-path fresh) (path-path (path zipper))) fresh))) (defun left (zipper) (when (and (path zipper) (path-left (path zipper))) (let ((fresh (copy-loc zipper))) (setf (loc-node fresh) (first (path-left (path zipper))) (loc-path fresh) (make-path :left (rest (path-left (path zipper))) :path (path-path (path zipper)) :right (cons (loc-node zipper) (path-right (path zipper))))) fresh))) (defun right (zipper) (when (and (path zipper) (path-right (path zipper))) (let ((fresh (copy-loc zipper))) (setf (loc-node fresh) (first (path-right (path zipper))) (loc-path fresh) (make-path :left (cons (loc-node zipper) (path-left (path zipper))) :path (path-path (path zipper)) :right (rest (path-right (path zipper))))) fresh)))
The main difference between this and the paper is that I've chosen
nil as my
Top representation, which lets me pull the trick of using
when to check for the presence of a
path, and its' non-
Top-ness at the same time.
The bad news is that since Common Lisp doesn't have pervasive functional data structures, I have to explicitly copy
locs while moving through a tree. The good news is that the copy is fairly light weight. Effectively, I'm copying out a set of 5 pointers, and could get that down to 3 by defining an intermediate struct.
Which I probably should do. Note to self.
Out of those, we get three compound navigation functions. With more probably coming soon. Specifically, I found
find useful for the work I did. It's easily externally definable, but would be even easier to bundle along. The ones I've already implemented are
Each of these involve an intermediate call to
;;;;;;;;;;;;;;; Compound navigation (defun root (zipper) (if-let (z (while zipper #'up)) (node z))) (defun leftmost (zipper) (while zipper #'left)) (defun rightmost (zipper) (while zipper #'right))
while. Which isn't a generic
macro; it's a function defined in
As you can see, all it does is repeatedly call a given function on a
... (defun until (zipper f) (let ((z zipper)) (loop for next = (funcall f z) while next when next do (setf z next)) z)) ...
zipperand return the last non-
node, so this doesn't run into the usual Common Lisp conflict of "Did you fail to find a thing, or find the element
That's the traversals done. Next up, we've got modification, without which this library is fairly useless. The basics are
delete and the
;;;;;;;;;; Modification (defun replace (zipper node) (let ((fresh (copy-loc zipper))) (setf (loc-node fresh) node) fresh)) (defun delete (zipper) (when (path zipper) (let ((fresh (copy-loc zipper)) (fresh-path (copy-path (loc-path zipper)))) (cond ((rights zipper) (setf (loc-node fresh) (pop (path-right fresh-path)) (loc-path fresh) fresh-path)) ((lefts zipper) (setf (loc-node fresh) (pop (path-left fresh-path)) (loc-path fresh) fresh-path)) (t (setf (loc-path fresh) (path-path fresh-path)))) fresh))) (defun insert-child (zipper node) (replace zipper (make-node zipper (cond ((not (branch? zipper)) (list node (node zipper))) ((children zipper) (cons node (children zipper))) (t (list node)))))) (defun append-child (zipper node) (replace zipper (make-node zipper (cond ((not (branch? zipper)) (list (node zipper) node)) ((children zipper) (append (children zipper) (list node))) (t (list node)))))) (defun insert-left (zipper node) (let ((fresh (copy-loc zipper)) (fresh-path (copy-path (loc-path zipper)))) (push node (path-left fresh-path)) (setf (loc-path fresh) fresh-path) fresh)) (defun insert-right (zipper node) (let ((fresh (copy-loc zipper)) (fresh-path (copy-path (loc-path zipper)))) (push node (path-right fresh-path)) (setf (loc-path fresh) fresh-path) fresh))
The paper defines an
insert_down function. It fails on a Leaf node, and otherwise inserts a singleton branch at the given location. The
append child functions above also insert nodes at a lower level at the current
loc. They give you a choice about whether to insert the new node as the leftmost or rightmost child, and additionally succeed on Leaf nodes by including the leaf value as a child of the new branch.
There are, thus far, three compound modification functions;
(defun edit (zipper f &rest args) (replace zipper (apply f (node zipper) args))) (defun splice-left (zipper node-list) (reduce #'insert-left node-list :initial-value zipper)) (defun splice-right (zipper node-list) (reduce #'insert-right (reverse node-list) :initial-value zipper))
edit takes a function instead of a new node, and replaces the node at
loc with the result of running that function on the existing node. The
splice-* twins are fairly self-explanatory; they're like
insert-right, but work on multiple nodes rather than single ones.
I haven't yet implemented
remove because these might relate to the different representation of the traversal
end? state. The reason for this seems to be that
remove assume a depth-first traversal. The reason I'm being weasely here is that I haven't thought about it hard enough to be sure that the
end? marker is really necessary. It also seems odd to privilege depth-first over breadth-first traversals; ideally, I think you'd want to be able to support either. Possibly interchangeably.
That wraps it up for this edition. My immediate intention is to do more work on the
clj libraries, as well as that game I mentioned last time. Ideally, I'd like to up my blogging output too. Probably not to the same volume as I had at my peak, but it was definitely helpful to keep some sort of written journal around for a while. The current state of the world is, hopefully, going to make it easy for me to get more programming time in. All things considered, I'd count that as a win.
- Although admittedly, it does require me to explain the concept of
zippers to a few other people for maintenance purposes. So ironically, this adds complexity despite being much more technically elegant than other options.↩
- There's a reason that
langnostic.jsis a raw JS file, rather than compiled from
clojurescriptsource, and that reason is like 90% that the compilation process is nontrivial to set up.↩
- "First", not "only". You can probably make educated guesses about which other ones I think you should learn.↩
- In which case, why are you here? This blog could kill you accidentally with an errant click or two. You should probably just go do something else.↩