Enthusiastically Shaving the Yak
For fun, I’m working on a macOS app written in Swift that requires a database of some kind.
And, for fun, I’ve decided that I want this database to be synced between clients of this app. I’ve always been a frontend-kind-of-person, and the interesting part of this for me is the app building part, so I want to spend as little time thinking about the backend as possible.
In the past two days, I’ve spent the most time thinking about the backend of my project than I ever anticipated. I’ve stayed up way later than I should have, spending way more time than I should have on a solution that’s way too complex than I need. Yet, it works! And it’s mine! For free! The journey of getting to that point has been the most unexpectedly fun part of this project so far. So, I wanted to do a lil’ write up along with documenting something that I saw a lot of YouTube tutorials and other guides miss with this specific setup.
The Yak’s Got More Hair Then I Thought
Convex is a backend solution built entirely on turning TypeScript into systems. The solution uses code that’s defined in your project and is synced with wherever you’re running the server. It handles relational databases, remote functions, file storage, and scheduled tasks among other things. Super neat, especially if you’re a web developer writing TypeScript!
I’ve never been into web development, it’s never been that fun for me. I’ve always hated that whatever I make always feels and behaves ever-so-slightly different than something that would have been developed natively for the platform. And so, my project’s a Swift app. Convex has an open source Swift package though, along with a couple other solutions for other languages, so no problem. Still, I’m stuck with defining my data layer in two places in two languages, though. It’s a trade-off I’m willing to make if it means I don’t have to write a line of SQL.
And, as previously mentioned, Convex can be self-hosted! Their documentation guides you through using Docker Compose, so that’s what I went with. I’ve toyed with Docker in the past for game server-related things, but always found it was too much work to be worth it. I’ve fully committed here, though. I followed their stellar docs and got a Convex install setup running using OrbStack. It exposes three ports: one for the backend, one to let third-party integrations access the backend, and a super easy-to-use dashboard. Done, right?
Far From It
If I want to connect to my instance I have to use the computer’s hostname and specify the port manually. Disgusting! Eww! What I want is a nice, clean hostname that has no numbers stuck on the end. On top of that, I also want the ability to authorize separate users so different people can use the app with their own data. Who cares if I’m the only person on Earth that’ll ever use this app, I want this! So, now I need an auth service and a reverse proxy to separate traffic between everything.
Hours of Googling led me to Traefik and authentik. Both of these have pretty detailed documentation and guides. How hard could it be?
It took me two days, four YouTube videos, and about three sessions using the finest LLMs available through OpenCode Go, but eventually I got Traefik working. Just Traefik. It was borderline psychosis-inducing following all of these resources to-the-letter and then still running into issues. I came pretty close to doing what would have been the reasonable thing from the beginning and just using the ugly hostname-port combo, but I’m not doing this project to be reasonable.
If you haven’t played Portal 2 for some reason, skip the next paragraph and go play it!
In Portal 2, after Wheatley takes over Aperture Science you eventually learn that any AI that controls the facility gains the insatiable urge to test, like an itch that you can’t scratch. This drives the AI mad, and the plot ensues.
Whenever I get home from my big jobby-job and I haven’t scratched the code itch, I satisfy it with some dumb one-off personal project that only I will ever use or have use for. The projects that grip me the most are the ones that have the highest ratio of unreasonable-but-interesting things to work on. I might be the only person on the planet that’s interested in having an app for managing Minecraft servers on macOS, but I want it! It’s fun to work on things that don’t have value to anyone but yourself, because the bespoke-ness gives it value.
So, I chose the hard path with this project. Making a SwiftData, Core Data, or even a JSON-backed app would have been easy. I wouldn’t have made it, though. The struggle is the whole reason I care about it. All of that effort, the lost sleep, it’s all worth it when I eventually feel like God’s gift to programming for figuring out what one config option I was missing and everything clicks. If LLMs end up owning the world in five years I don’t want my critical thinking and problem solving skills to atrophy. I can say that I made this absolutely dumb and pointless project with my bare hands.
db
On the off-chance that you too, dear reader, are also trying to host your own Convex instance in a Docker container behind a Traefik reverse proxy on macOS, then you should know that this one line cost me about half a day’s worth of sanity.
"traefik.http.services.your-service-name.loadbalancer.server.url=http://localhost:6791"
This line in your Labels section of your services’ Docker Compose file will save so much headache. I swear I watched so many different videos on Traefik and never saw one talk about this. Traefik seems to automatically generate a load balancer service for your routes, and this line tells Traefik that the traffic for the service should point at that URL. When you self-host Convex, it expects traffic at a certain location - in this example, the dashboard expects traffic at port 6791. This applies to any port you’d want to map a route to.
All of the tutorials I watched talked about putting nginx or other web hosts behind the proxy, which assumes traffic at port 80. This is kind of the default case here, so I never saw any examples that pertained to my specific use case. This line was the missing link, and after that everything magically started working.
There’s a tension in my writing above between going full “AI bad” and getting value out of using LLMs to solve problems. As I said, I eventually used GLM-5 through OpenCode to figure out the handful of things I was getting wrong in my config files. Yet, I proclaim that I’ve crafted this “with my bare hands”! Hypocrite?!
I think there’s things that LLMs are extremely transformative for. Code, for example. I also think that there’s an inexcusable and inescapable amount of social and ethical implications of a technology that’s essentially a universal information multiplier. It’s the perfect excuse for vampiric venture capital-funded husks to push their selfish, anti-social and late-stage-capitalistic agendas. I think there’s a world where this technology can be used equitably for the benefit of everyone, but I don’t think Sam Altman or Dario Amodei are gonna be the ones to do it.