The first codebase I admired was Ost. When I found Ost, I was spending a lot of my job hours debugging weird Resque issues, and I was ready to believe that Redis-backed queues in Ruby were one of the Hard Problems in Computer Science. There was only one reason I could believe that Ost had handled the problem in one little file, and that reason was that I could read and understand the whole little file.

Another codebase I admire on similar grounds is sql-migrate-up, which a friend of mine made when we needed to run migrations in a new codebase. The state of the art on that team was Alembic; we had a whole wiki page explaining how to figure out how to create a new migration that would actually run and run at the right time. Fundamentally, all we needed to do was run some SQL files in order. You can get most of the way there with an ASCIIbetically sortable naming scheme, a function that runs SQL, and a for loop, and that’s pretty much what sql-migrate-up gave us.¹

Then there’s this Slack CLI, which manages the double-whammy of interacting with a big web API in a simple way and also doing it in clear and concise bash.

My point is, sometimes there’s a big difference between the essential complexity of a thing and the complexity we’re conditioned to expect of that thing. It’s like when you order something online that comes in a tiny box, but that tiny box is shipped inside of a much, much larger box where most of the space is taken up not by your thing but instead by peanuts or air plastics or sometimes, oddly, nothing at all.

So anyway, I wanted to put some writing on the Internet.

I didn’t want a capital-P Platform. I just want to write some Markdown or whatever and have it show up on a website. This is the job of a static site generator! So I set out to learn what had happened in the 15-or-so years since I last set up a static site with Jekyll² on GitHub Pages. My completely unorganized³ research process led me to luasmith, which looked pretty good but didn’t support a feature I needed⁴; coleslaw, which of course I would have tried out except that its footnote support used an especially nonstandard syntax; and Hugo. I threw my content at Hugo and it threw it back with nicely rendered footnotes! It also gave me a whole directory structure with a theme and archetypes and stuff, and when I went to the docs to learn what an archetype is, I found myself learning about its security model, and that’s when I heard myself think: wait a minute, this has gotta be easy, right?

I don’t mean that as a knock against Hugo. It seems good and well thought out. Every software author should certainly consider their security model! But I just wanted to write Markdown and get HTML out. I didn’t need a theme, or tooling around creating posts, or to consider how I trust my content author.⁵

So I consulted my Basic Internet Hipster guidebook and did what it told me: pop open a pack of parens, fire up a fun Lisp, and get to work. Thus I brought yet another static site generator, unfortunately named igbew⁶, into this world.

So it was easy, right?

Well, yeah, sure. Fundamentally, here’s what a static site generator needs to do:

1. Find your content input files and render them to HTML files in an output directory.
2. Copy static files to the output directory.
3. Generate some metadata files (e.g., feeds, a sitemap), maybe.

As with just about anything, you can find a lot of complexity in the details, and probably not in the details you were expecting.

In my case, I used lunamark to render Markdown to HTML, which otherwise would have been the most work.⁷ Lunamark also has a built-in mechanism for running Lua code to set metadata inside Markdown, which gave me a convenient solution to that problem that I had just been pretending wouldn’t exist.

I used etlua for templating, which was convenient. Now that I’m writing this, I realize I could have used one of those fancy libraries for writing HTML or XML as S-expressions, and that would have been more fun and flexible.⁸

I thought that when I finally got around to making the feeds and index pages, I’d be able to do some sort of really sick code thing where I modeled each index or feed as a coroutine and processed posts as they were read. And I did, but then I realized that that was actually a stupid idea and I could just use one coroutine to accumulate all the posts and then render a few different templates with them in the end.

What standard library?

In the end, the bulk of the work I had to do was not interesting stuff that addressed the essential complexity of static site generation. Instead, it was making stupid versions of things that would be standard library functions in other languages. This isn’t inherently a bad thing, it’s just a thing I forgot about Lua: it has almost nothing built in. I started to write my find-posts function, and I was like, “how do you list the contents of a directory, again?”, and then I remembered that the answer is “you add a dependency that has a function that can iterate over one directory, and then you write your own recursive wrapper around that.”⁹ I made my own little stupid path manipulation library that is very much platform-dependent¹⁰ and also an inconsistent mess where some paths are strings and some are sequences of strings, depending mainly on what’s convenient where a particular thing is used; sometimes I even cheat and rely on my path garbage being so stupid that I can mix the two approaches and have it conveniently do what I want.

This was perhaps not how I envisioned the fun of a short little fun project. But I do appreciate having to occasionally remember how basic things work. I’ve also been planning to do something more in Fennel and Lua, so any way to stretch these particular muscles was worthwhile.

What types?

A more surprising thing for me was the anxiety I felt not having type checking. I’ve written my share of Lisp, and I spent my formative years on things like Perl, Ruby, and JavaScript¹¹. True, I’ve spent my last several professional years writing TypeScript and Swift, and I like those languages and their type systems a lot! But I didn’t think of myself as a convert, just a person who appreciates the different strengths of different tools. “Go with the grain of the tool,” I’ve been heard to say. So I was not expecting to feel alarmed and adrift when I started passing things around and there was no compiler to tell me if I was passing the wrong things to the wrong places!

There is a baseline level of chaos that a type system insulates you from that I had forgotten about. I was surprised when I saw nil in a rendered template or triggered a runtime error by trying to concatenate a table to a string. Not only am I used to the compiler stopping me when I tell it to do these things, that constant feedback loop with the compiler has become less like an active check and more like a passive sense. This is why my reaction to not having it was more like feeling confused than feeling inconvenienced.

I kept going, and the feeling went away. But I was surprised by how I realized I was compensating: shorter functions, better names, and docstrings. These are all aspects of how I’ve always written Lisp, and really, how I write code in general. What surprised me was experiencing them as tools that addressed the same need as a type system: giving me tools and information to prevent stupid mistakes. Shorter functions lend themselves better to composition and have a smaller surface area for passing in the wrong kind of thing. Docstrings give you an opportunity to think through exactly what a thing is doing and why, which in turn gives you an opportunity to see that you’ve made the wrong abstraction or you’re doing something stupid or you’re tying together a bunch of things that shouldn’t inherently be tied together. Naming is both a stronger version of docstrings—if you can’t describe something clearly and concisely enough to name it, then that thing itself might not be clear or concise enough—and the part of the type system that communicates to the human reader rather than the compiler: I might not be able to tell the compiler if path is one string or an array of strings, but telling the human reader that it’s called path-string or path-components is still a good first line of defense against passing in the wrong thing.

There is probably something to be said (by a smarter person) about how Lisps make this all easier, too. I often use mind mapping software to think things through, and the reason that works well for me is that it lets me find the structure of my thoughts but in an unordered, undirected way. Programming in a Lisp feels similar: I tend to feel like I fling around parentheses, attacking random parts of random problems, until the shapes of my fundamental problem and solution reveal themselves and I discover I’m mostly done. That also describes how I work in other languages, but the feeling is different. It’s the same as the difference between mind mapping and just making a list of ideas: one turns exploring the structure and creating the structure into the same act, while the other gives you an arbitrary inflexible grouping of stuff that you’ll have to tear down and rearrange later.

I tried to come up with a deep and clever reason why this might be, but probably it’s just the harmony of a few simple, obvious things:

• Since everything is fundamentally just a list, the only barrier to snipping out some piece of code and promoting it to its own function is just making you have the right stuff in scope.
• It’s actually just unwieldy to do a lot of stuff in one chunk of code. As long as you’re not doing a lot of mutative, imperative, effectful stuff¹², a function that does a lot has to introduce a lot of scopes and bindings and conditionals and loops and very quickly starts to just look weird.
• Introducing a new binding is explicitly introducing a new scope, and every nested let is just enough friction to make you think hmm, I wonder if this is actually its own function.

Maybe if there is something deep way of summarizing all that, it’s: Lispy languages get out of your way enough that what you see is what you’re doing, and when you see that you’re doing something goofy, they don’t add friction to fixing it.

So anyway

So anyway, I wrote a static site generator. I’m not here to say you should use it,¹³ but because I did it, I am now here saying something, so that’s fun.

1. If you’re of this world, you might be thinking “why were you running Alembic, a Python tool, in a TypeScript codebase in the first place?”. We weren’t and we wouldn’t have, but the TypeScript options we looked at didn’t feel better. ↩

2. Which is, honestly, probably still Just Fine™. ↩

3. For example, it wasn’t until right now while writing this that I added that footnote about Jekyll probably being Just Fine™ and thought to myself, self, did you even consider Jekyll? So I went and checked out what’s going on with Jekyll these days, and here’s how Jekyll still describes itself: “Jekyll takes your content, renders Markdown and Liquid templates, and spits out a complete, static website.” Interesting how much that sounds like what I wanted! ↩

4. Footnotes. It was footnotes. Obviously I need footnotes. ↩

5. Even Hugo’s security model is not enough to protect me from myself. ↩

6. Because I am that lowest form of human life, a compulsive pun-maker, this name sprung fully formed into my head. I don’t know how to pronounce “igbew”; it stands for “it’s gotta be easy write”, which, no matter how I punctuate it, seems designed to injure the reader, directly in the eye’s. I’m sorry. The upside is that there’s nothing else named “igbew”, and since I am on the record as saying that all software should be named with UUIDs to prevent ambiguity, I am happy with this outcome. ↩

7. I would not have written my own thing if I had had to write a Markdown parser. I wanted to have fun. ↩

8. I certainly wouldn’t have been able to work so hard to make sure my HTML and XML were properly indented. Like a weirdo. ↩

9. How do you make a directory? Also with a dependency. How do you copy a file? Read it all and then write it all somewhere else. Removing a directory? The builtin os.remove does actually handle that, as long as the directory is empty. (How do you empty a directory? See “how do you list the contents of a directory?”) ↩

10. Apologies to the nonexistent Windows subset of my nonexistent potential user base. ↩

11. How? Ew. ↩

12. This depends on your Lisp. It’s why I haven’t yet learned to get along with Common Lisp. ↩

13. Really, you probably shouldn’t. I’ve heard Jekyll and Hugo are pretty good! And there are so many others. It’s like competing standards, except without either standards or competition. ↩