My dashboards live in git now, not in a database
I run Grafana on my home cluster to draw graphs of things I care about. For a long time I built those dashboards the way everybody does at first: click around in the web interface, drag panels, fiddle with queries, until it looks right. And then, periodically, lose the whole thing. A reinstall, a wiped volume, a "let me just recreate this environment," and the dashboard I spent an hour tuning was gone, because it only ever lived inside Grafana's own database, which is to say nowhere I was actually keeping safe.
The fix is to stop thinking of a dashboard as a thing you draw and start thinking of it as a thing you commit. A dashboard, underneath the pretty interface, is just a chunk of structured text describing the panels and their queries. So it can live in a file. And if it lives in a file, it lives in git, alongside everything else that defines my setup, and it comes back exactly the same after any disaster, because reproducing it is just applying the file again.
#How the wiring works
Grafana has a feature for this that's a little buried but exactly what you want. You point it at a folder and tell it "any dashboard file you find in here, load it." Then you put your dashboard file in that folder. On my cluster, the folder is filled from a config object that I keep in git, and the dashboard text sits right inside it. Grafana starts up, reads the folder, and there's my dashboard, identical to last time, no clicking required.
The same trick already handles the connection to my data. The bit that tells Grafana "here's the database to draw graphs from" is also just a file it reads on startup, not something I set up by hand in the UI. So the entire thing, the data connection and the dashboards both, is defined in text I keep, and a fresh Grafana with nothing in it becomes my Grafana, fully furnished, the moment it reads those files.
#The honest tradeoff
I'll be straight about the catch, because there is one. Building a dashboard by clicking is pleasant. You see it change as you drag things. Hand-writing the file is fiddlier, and the structured text is verbose and a little finicky about its details. So the workflow that actually works isn't "write the whole thing by hand from a blank file." It's "build it in the UI until it looks right, then export the text and commit that." Click to design, commit to keep. You get the nice drawing experience and the durable file, and you stop pretending you're going to remember the clicks.
There's a small bonus that I didn't expect to care about and now do: because the dashboard is a file in git, its history is right there. I can see when I added a panel, diff two versions, revert a change I didn't like. A dashboard I drew in a UI has no history. It's just whatever it is right now. A dashboard in git is a thing with a past, and that turns out to matter the first time you go "wait, this graph used to show something different, what did I change."
#The pattern under the pattern
This isn't really about Grafana. It's the same move I keep making with everything in the homelab, and the same move that separates a setup you can rebuild from one you can only nurse along. Anything that defines how your system behaves, the dashboards, the connections, the schedules, the policies, wants to live in a file you keep, not in some application's private database where it's invisible and unbackupable and gone the moment that application has a bad day.
The test I use now is simple and a little brutal: if this machine caught fire, could I rebuild this from things I'm keeping in git, or would I be reconstructing it from memory by clicking around? Every time the answer is "from memory," that's a thing that isn't really mine yet. It's a sandcastle. Moving the dashboards into git was a small chore that turned an hour of clicking into a file I'll never lose, and the relief of "I can't lose this anymore" is worth more than the clicking ever cost me.