Hello again world

For much of my adult life, I've spent every spare minute (and some that were not really spare) thinking about The Problem. Different people have different ideas about what The Problem is (we could call that The Meta Problem but I think that position has been filled); for me at that time, The Problem was The Problem with Humans and Computers.

About six months ago, I turned my attention to music, thinking that I would come back to The Problem after I'd recorded a few songs. As part of that hiatus (or so I considered it), I also went on a complete media blackout. I stopped reading or listening to any contemporaneous reporting about the world, popularly known as The News, including tech news, which I had followed closely for at least fifteen years.

Now, I am already pretty old-fashioned. I love the quotation of Don Knuth (and apparently so does the Computer History Museum), where he says,

Email is a wonderful thing for those people whose role in life is to be on top of things, but not for me: my role is to be on the bottom of things.

—Donald Knuth

I would apply this statement to current events generally. The things that interest me most are the longest-lasting ones, which necessarily means that I often find myself looking backwards. I pay more attention to history than to the state of the art.

But this season, I stopped reading history, too. This year I have read nothing except The Essential Rumi, over and over.

And something happened.

I don't want to go back.

Go back to what? Go back to volunteering my time on an intractable and not-terribly-well-defined problem, when there are truly useful things I could be doing.

Since I can remember, some project or other has dominated my attention and absorbed my energy. And it would not be unfair to say that all I have to show for this devotion is a little philosophy about what we're doing wrong in software. And something I call a knowledge graph interpreter which I've never been able to satisfactorily explain to anyone.

Meanwhile on an actual website that I built, the listen along feature for 23 plays has been coming soon for almost ten years now, putatively because I got tired of building new bespoke tools for every little thing.

Well, Don Knuth I am not, and my yak shave is coming to a close.

I did not begin this HTML file with a definite idea of what I had to say. But I think what I have to say is this.

I just want to scratch my own itches. The idea that what I'm really working on is a platform that would be of use to other people in the future is both unsupportable by the facts and overly demanding of the process.

So what are my own itches?

I want to finish the listen along on willshake.net for the Living Shakespeare recordings.

I want to rebuild my wife's web site so I can move it off of .NET and onto another host.

I want to see all of my computer activity chronologically.

That's it, really. I already have a day job, and I've taken up performing in the evening.

With all that said, there is still a problem. The problem is that I know what happens. Things fall apart. The datacenter cannot hold. If you made something that exists only in a running computer, then someone else can break it (if it doesn't die first). This has always been a problem with software, but as we add more and more layers and connections, it gets worse.

Do you find this surprising? Has it entered our collective understanding yet, that bytes in a computer system deteriorate faster than material things? Whether people talk about the right to die or the right to be preserved online, they're generally talking about data, and preserving data is a hard problem. The related problem of preserving systems (that can interpret the data) is even harder.

Let's face it: I'm starting over again. It was inevitable. But you're never really starting over, if you reflect on what was good and bad and learn from it. So here we go.

Good: I still believe that

Bad: I have found that

Actually I think it all comes down to that. In order to maximize portability, I have assumed the least-capable environment: a standalone HTML file opened locally as such. The biggest problem in this environment arises from web security protocols: without an http or https origin, you can't make most kinds of network requests. The most important consequence of this is that you have no way to reliably save your work.

Let me put this in terms of a real project, albeit one that's been dormant for a long time. Gitlab, where willshake has lived for a while now, puts its repository usage at 35.3 MiB. That's about the same as the uncompressed content without the revision history. However, it does not include all of the assets that ultimately end up on the web site, many of which are sourced from elsewhere (chiefly Wikimedia, the Internet Archive, and Project Gutenberg). Those comprise several gigabytes of additional stuff, and whether those gigabytes belong with this project proper is one of the questions I've grappled with.

You see, if I really believe that process & product MUST NOT be separated, then when I hand you something that I call willshake, I can't mean just the web site, but everything used to produce it. But everything is doing a lot of work there. In practice, I could get away with schlepping around something that weighs 35MB (about one third of which is the complete works of Shakespeare), but when we get to 3 and 4GB (mostly owing to pictures and audio recordings), well that puts us on the wrong side of some operations. It's more than I can email you, let alone cram into an HTML file. So it's strictly less portable.

Anyway, I maintain that I should be able to give you a file called willshake.html that lets you do everything that you can do on the web site and also gives you everything you need to revise the site from your computer just as I do and also lets you provide a willshake.html of which the same things are true.

Are you starting to see why I'm still in this rabbit hole?

Clearly I have to trim back these ambitions a bit, or at least redefine the word everything. I've already acknowledged that I can't practically distribute arbitrary media assets with a standalone HTML file (or even a ZIP file if the target is a small device), and of course the web gives you plenty of ways of dealing with this. There's something going on here that I have yet to untangle. It's not the media assets that make the problem difficult: it's the tools.

Oh and on that point, one more takeaway from previous work:

Yes, my beloved Tup is just too esoteric to take on as a dependency.

All of this might seem like a lot of fuss over hypotheticals, but consider. If your life depended on being able to open and use a file that someone created five years ago, would you rather see that it has a .html or an .exe extension? What about fifteen years ago? Twenty five? Well, it works going forward, too.

This is taking too long. As the timestamps show, I have been drawn away from writing at various points into other things like a cinnamon roll and an at-home commercial that my kids were shooting. And of course a fair amount of staring into space.

So yeah, the detour is over. I'm going to rebuild willshake again. These pages will document my journey. And the yak can start growing out its hair again.