summaryrefslogtreecommitdiffstats
path: root/blog/homelab.html
blob: f335ad9e6e8f994724e7e5f1281fed9bcaa4c1e1 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
<!DOCTYPE html>
<html lang="en">
	<head>
		<meta charset="UTF-8">
		<title>Homelab | Skullheadx</title>
		<meta name="description" content="A brief tour of my homelab hardware and NixOS + OpenBSD setup.">
		<meta name="viewport" content="width=device-width, initial-scale=1.0">
		<link rel="stylesheet" type="text/css" href="../blog.css">

		<script>

			// By Oskar Wickström
			// Licensed under the MIT License (https://github.com/owickstrom/the-proportional-web/blob/main/LICENSE.md)
			
			window.addEventListener("load", () => {
				const asides = [...document.querySelectorAll("aside")];
				asides.forEach((aside, i) => {
					const anchor = aside.previousElementSibling;
					const name = `--paragraph-before-aside-${i}`;
					anchor.style.anchorName = name;
					aside.style.positionAnchor = name;
					});
					});
		</script>

		<link rel="icon" href="../favicon.ico">
	</head>
	<body>
		<nav class="main-nav">
			<a href="../index.html">Main Page</a>
			<a href="./index.html">All Posts</a>
			<a href="../tech/index.html">Technology Posts</a>
		</nav>
		<main class="blog">
			<header id="top">
				<h1 class="title">A Brief Tour of My Homelab</h1>
				<p class="subtitle">Authored by <cite><a href="../about.html">Skullheadx</a></cite>
					on <time datetime="2026-08-31">August 31st, 2026</time>.</p>
				<!-- <p>Updated: <time datetime="2026-08-27">August 27, 2026</time>.</p> -->
			</header>

			<nav id="TOC" role="doc-toc">
				<h2 id="toc-title">Contents</h2>
				<ul class="incremental">
					<li><a href="#overview" id="toc-overview">Overview</a></li>
					<li><a href="#router" id="toc-router"><span class="toc-section-number">1</span> Router</a>
						<ul class="incremental">
							<li><a href="#router-front" id="toc-router-front"><span class="toc-section-number">1.1</span> Front</a></li>
							<li><a href="#router-back" id="toc-router-back"><span class="toc-section-number">1.2</span> Back</a></li>
							<li><a href="#router-fetch" id="toc-router-fetch"><span class="toc-section-number">1.3</span> Fetch</a></li>
							<li><a href="#router-inside" id="toc-router-inside"><span class="toc-section-number">1.4</span> Inside</a></li>
							<li><a href="#router-services" id="toc-router-services"><span class="toc-section-number">1.5</span> Services</a>
							<ul class="incremental">
								<li><a href="#router-services-pf" id="toc-router-services-pf"><span class="toc-section-number">1.5.1</span> pf</a></li>
								<li><a href="#router-services-relayd" id="toc-router-services-relayd"><span class="toc-section-number">1.5.2</span> relayd</a></li>
								<li><a href="#router-services-acme" id="toc-router-services-acme"><span class="toc-section-number">1.5.3</span> acme-client</a></li>
								<li><a href="#router-services-httpd" id="toc-router-services-httpd"><span class="toc-section-number">1.5.4</span> httpd</a></li>
							</ul></li>
						</ul>
					</li>
					<li><a href="#icon" id="toc-icon"><span class="toc-section-number">2</span> Icon</a>
						<ul class="incremental">
							<li><a href="#icon-side" id="toc-icon-side"><span class="toc-section-number">2.1</span> Side</a></li>
							<li><a href="#icon-back" id="toc-icon-back"><span class="toc-section-number">2.2</span> Back</a></li>
							<li><a href="#icon-fetch" id="toc-icon-fetch"><span class="toc-section-number">2.3</span> Fetch</a></li>
							<li><a href="#icon-inside" id="toc-icon-inside"><span class="toc-section-number">2.4</span> Inside</a></li>
							<li><a href="#icon-services" id="toc-icon-services"><span class="toc-section-number">2.5</span> Services</a>
							<ul class="incremental">
								<li><a href="#icon-services-nginx" id="toc-icon-services-nginx"><span class="toc-section-number">2.5.1</span> nginx</a></li>
								<li><a href="#icon-services-cdn" id="toc-icon-services-cdn"><span class="toc-section-number">2.5.2</span> CDN</a></li>
								<li><a href="#icon-services-cgit" id="toc-icon-services-cgit"><span class="toc-section-number">2.5.3</span> cgit</a></li>
								<li><a href="#icon-services-git-daemon" id="toc-icon-services-git-daemon"><span class="toc-section-number">2.5.4</span> git daemon</a></li>
								<li><a href="#icon-services-soju" id="toc-icon-services-soju"><span class="toc-section-number">2.5.5</span> soju</a></li>
								<li><a href="#icon-services-nfs" id="toc-icon-services-nfs"><span class="toc-section-number">2.5.6</span> NFS</a></li>
								<li><a href="#icon-services-nix-remote" id="toc-icon-services-nix-remote"><span class="toc-section-number">2.5.7</span> Nix remote builder</a></li>
								<li><a href="#icon-services-nix-serve" id="toc-icon-services-nix-serve"><span class="toc-section-number">2.5.8</span> Nix-serve-ng</a></li>
								<li><a href="#icon-services-dnsmasq" id="toc-icon-services-dnsmasq"><span class="toc-section-number">2.5.9</span> dnsmasq</a></li>
							</ul></li>
							<li><a href="#icon-observability" id="toc-icon-observability"><span class="toc-section-number">2.6</span> Observability</a>
							<ul class="incremental">
								<li><a href="#icon-observability-collectd"    id="toc-icon-observability-collectd"><span class="toc-section-number">2.6.1</span> collectd</a></li>
								<li><a href="#icon-observability-smartd"      id="toc-icon-observability-smartd"><span class="toc-section-number">2.6.2</span> smartd</a></li>
								<li><a href="#icon-observability-upsd-upsmon" id="toc-icon-observability-upsd-upsmon"><span class="toc-section-number">2.6.3</span> upsd+upsmon</a></li>
							</ul></li>
							<li><a href="#icon-notes" id="toc-icon-notes"><span class="toc-section-number">2.7</span> Notes</a></li>
						</ul>
					</li>
					<li><a href="#ups" id="toc-ups"><span class="toc-section-number">3</span> UPS</a>
						<ul class="incremental">
							<li><a href="#ups-front" id="toc-ups-front"><span class="toc-section-number">3.1</span> Front</a></li>
						</ul>
					</li>
					<li><a href="#akesis" id="toc-akesis"><span class="toc-section-number">4</span> Akesis (VPS)</a>
						<ul class="incremental">
							<li><a href="#akesis-fetch" id="toc-askesis-fetch"><span class="toc-section-number">4.1</span> Fetch</a></li>
							<li><a href="#akesis-services" id="toc-askesis-services"><span class="toc-section-number">4.2</span> Services</a>
							<ul class="incremental">
								<li><a href="#akesis-services-pf"     id="toc-askesis-services-pf"><span class="toc-section-number">4.2.1</span> pf</a></li>
								<li><a href="#akesis-services-relayd" id="toc-askesis-services-relayd"><span class="toc-section-number">4.2.2</span> relayd</a></li>
								<li><a href="#akesis-services-acme"   id="toc-askesis-services-acme"><span class="toc-section-number">4.2.3</span> acme-client</a></li>
								<li><a href="#akesis-services-httpd"  id="toc-askesis-services-httpd"><span class="toc-section-number">4.2.4</span> httpd</a></li>
							</ul></li>

						</ul>
					</li>
				</ul>
			</nav>

			<h1 id="overview">Overview</h1>
			<p>I started this homelab project as an experiment to learn more about servers, and as of <time datetime="2026-08-31">August 31st, 2026</time>, I'm now hosting a <a href="https://git.skullheadx.com">public git web frontend</a>, git daemon, IRC bouncer, NFS, CDN, <a href="https://nix-cache.skullheadx.com/nix-cache-info">Nix bin cache</a>, and more. This page is intended to be living documentation of my setup as it evolves to suit my ever-growing compute requirements.</p>
			<figure>
				<img fetchpriority="high" width="720" alt="The black and blue small box is my home router running OpenBSD, and below it is my NixOS box that runs most services." src="../blog/homelab_setup.webp">
				<figcaption>I feel like this photo depicts a liminal space. Luckily, this is just my basement. The black and blue small box is my home <a href="#router">router</a> running <a href="https://openbsd.org">OpenBSD</a>, and below it is my <a href="https://nixos.org">NixOS</a> box that runs most services. The <a href="#ups">UPS</a> is located behind the monitor.</figcaption>
			</figure>
			<p>My core goals when it came to designing my network was that it would be highly efficient, simple to maintain, and cheap. I run <a href="https://openbsd.org">OpenBSD</a> on my <a href="#akesis">VPS (<span class="canonical-name">akesis</span>)</a>, and <a href="#router"><span class="canonical-name">router</span></a> at home; <a href="https://nixos.org">NixOS</a> is running bare metal on the desktop computer (<a href="#icon"><span class="canonical-name">icon</span></a>) in the above image.</p>
			<p>All my DNS records point at the two static IPv4 addresses I have assigned to <a href="#askesis"><span class="canonical-name">askesis</span></a>. The firewall, <code>pf</code>, and <code>relayd</code> handle redirects to the appropriate locations. Here is where the traffic diverges into two paths:</p>
			<h2 id="static-web-hosting">Static Web Hosting on <span class="canonical-name">askesis</span></h2>
			<p>I run a static web hosting business, <a href="https://montgomery.systems">Montgomery Systems</a>, which uses <code>httpd</code> to serve sites for local businesses and individuals. Also this blog is hosted there!</p>
			<h2 id="home-router">Home <span class="canonical-name">Router</span></h2>
			<p>The rest of the traffic is sent to the home <a href="#router"><span class="canonical-name">router</span></a>, which also runs <code>pf</code>, and <code>relayd</code> to route traffic to <a href="#icon"><span class="canonical-name">Icon</span></a>. TLS termination is also handled here.</p>
			<h2 id="home-icon"><span class="canonical-name">Icon</span></h2>
			<p>On <a href="#icon"><span class="canonical-name">Icon</span></a>, a variety of services run as <code>systemd</code> services listening on various ports. Some things like my git web frontend, CDN, and Nix bin cache live behind Nginx.</p>


			<h1 id="router">Router</h1>
			<p>This <a href="https://www.barraguard.com/ng-firewall-f12.asp"><span class="canonical-name">Barracuda Cloudgen Firewall F12</span></a> is the firewall to the public internet at home. My <a href="#akesis">VPS</a> has a bidirectional remapping of a static IPv4 to this box over wireguard so that I can route things directly to my house. I mostly just use it for TLS termination so that I can have use a secure connection at home and outside thanks to split horizon DNS.</p>
			<figure id="router-front">
			<img fetchpriority="high" width="720" alt="front of a small rectangular router with lights shining" src="../blog/homelab_router_front.webp">
			</figure>
			<figure id="router-back">
			<img fetchpriority="high" width="720"  alt="back of the router with many ethernet ports" src="../blog/homelab_router_back.webp">
				<figcaption><code>em1</code> is port 4, and <code>em2</code> is port 5. The port ordering is just weird.</figcaption>
			</figure>
			<p>In the future, I plan to move the homelab to be directly plugged into this router, but I wanted <a href="#icon"><span class="canonical-name">icon</span></a> to be easily found via the LAN IP assigned by the ISP modem, which I do not have admin access to. I am considering upgrading my internet so that I can just use this router instead.</p>
			<figure id="router-fetch">
			<img fetchpriority="high" width="720"  alt="specifications of the router running OpenBSD" src="../blog/homelab_router_fetch.webp">
				<figcaption><code>fastfetch</code> run over serial from <a href="#icon"><span class="canonical-name">icon</span></a></figcaption>
			</figure>
			<figure id="router-inside">
			<img fetchpriority="high" width="720"  alt="internal hardware of the router showing a heatsink, SSD and motherboard." src="../blog/homelab_router_inside.webp">
				<figcaption>At the end of the day, a router is just a computer that does networking. I specifically chose this box because it's fanless (hence the giant heatsink), and it's x86 architecture which is better supported by OpenBSD.</figcaption>
				<figcaption>The top cover is held in place by two screws and slides off easily. Very easy to service and repair!</figcaption>
			</figure>
			<h2 id="router-services">Services</h2>
			<h3 id="router-services-pf">pf</h3>
			<h3 id="router-services-relayd">relayd</h3>
			<p>Here's my config in case you need an example to figure out how <code>relayd</code> works.</p>
			<h4><code>/etc/relayd.conf</code></h4>
			<pre><code>
local="127.0.0.1"
homelab="192.168.1.215"
vps="10.0.0.1"

table &lt;local&gt; { $local }
table &lt;homelab_git&gt; { $homelab }
table &lt;homelab_nix_cache&gt; { $homelab }
table &lt;homelab_irc&gt; { $homelab }
table &lt;homelab_cdn&gt; { $homelab }

http protocol "tls_filter" {
        tls keypair "git.skullheadx.com"
        tls keypair "git.montgomery.systems"

        block

        match request header "Host" value "git.skullheadx.com" tag "to_homelab_git"
        match request header "Host" value "nix-cache.skullheadx.com" tag "to_homelab_nix_cache"
        match request header "Host" value "cdn.skullheadx.com" tag "to_homelab_cdn"

        match request header "Host" value "git.montgomery.systems" tag "to_homelab_git"

        match response header remove "Server"

        match request header append "X-Forwarded-For" value "$REMOTE_ADDR"
        match request header append "X-Forwarded-By" value "$SERVER_ADDR:$SERVER_PORT"

        pass request tagged "to_homelab_git" forward to &lt;homelab_git&gt;
        pass request tagged "to_homelab_nix_cache" forward to &lt;homelab_nix_cache&gt;
        pass request tagged "to_homelab_cdn" forward to &lt;homelab_cdn&gt;
}

relay "tlsproxy" {
        listen on 0.0.0.0 port 443 tls
        protocol "tls_filter"
        forward to &lt;local&gt; port 80
        forward to &lt;homelab_git&gt; port 8080
        forward to &lt;homelab_nix_cache&gt; port 5000
        forward to &lt;homelab_cdn&gt; port 8081
}

tcp protocol "ircs" {
        tls { keypair git.skullheadx.com }
}

relay ircs {
        listen on 0.0.0.0 port 6697 tls
        protocol ircs

        forward to <homelab_irc> port 6667
}</code></pre>
			<aside>
				This config covers the services that need to be redirected to my homelab on specific ports depending on the <code>Host</code> header. It also takes care of IRC TLS termination.
			</aside>
<h3 id="router-services-acme">1.5.3 acme-client</h3>
<h3 id="router-services-httpd">1.5.4 httpd</h3>
			<p>This config mainly redirects HTTP to HTTPS. I also have the .well-known for CA cert renewal.</p>
			<h4><code>/etc/httpd.conf</code></h4>
			<pre><code>server "git.skullheadx.com" {
        listen on * port 80
        alias "irc.skullheadx.com"
        alias "nix-cache.skullheadx.com"
        alias "mail.skullheadx.com"
        alias "cdn.skullheadx.com"

        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }

        location "*" {
                block return 301 "https://$HTTP_HOST$REQUEST_URI"
        }
}

server "git.montgomery.systems" {
        listen on * port 80

        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }

        location "*" {
                block return 301 "https://$HTTP_HOST$REQUEST_URI"
        }
}
				</code></pre>

			<h1 id="icon">Icon</h1>
			<p>This <span class="canonical-name">HP Compaq 8200 Elite CMT PC</span> is a 10 year old desktop computer transformed into the power house of my homelab running NixOS. My nix flake can be found <a href="https://git.skullheadx.com/nixos/">here</a>, and I've written about my <a href="../blog/suckless_nix.html">suckless style</a>, and <a href="../blog/3-advantages-nixos.html">3 advantages of using NixOS</a> on this blog.</p>

			<figure id="icon-side">
			<img fetchpriority="high" width="720"  alt="close up photo of the router sitting on top of icon" src="../blog/homelab_icon_side_and_router_front.webp">
			</figure>
			<figure id="icon-back">
			<img fetchpriority="high" width="720"  alt="backside of the computer, featuring many ports and cables" src="../blog/homelab_icon_back.webp">
				<figcaption>Genuine serial, parallel, and VGA ports! There's also room for a CD tray on the front (not depicted).</figcaption>
			</figure>
			<figure id="icon-fetch">
			<img fetchpriority="high" width="720"  alt="specifications of computer" src="../blog/homelab_icon_fetch.webp">
				<figcaption>Due for a RAM upgrade once prices settle down.</figcaption>
			</figure>
			<p>The limited specs of the system means I can't deploy bloated programs without running into severe performance problems, so I try to Keep It Simple, Stupid. I may start to separate different services like NFS into their own servers because they have different uptime SLA requirements, and it would take a bit of load off of this main box.</p>
			<figure id="icon-inside">
			<img fetchpriority="high" width="720"  alt="motherboard, CPU, SSD, HDD inside the computer" src="../blog/homelab_icon_inside.webp">
			</figure>
<h2 id="icon-services">Services</h2>
<h3 id="icon-services-nginx">nginx</h3>
<h3 id="icon-services-cdn">CDN</h3>
<h3 id="icon-services-cgit">cgit</h3>
<h3 id="icon-services-git-daemon">git daemon</h3>
<h3 id="icon-services-soju">soju</h3>
<h3 id="icon-services-nfs">NFS</h3>
<h3 id="icon-services-nix-remote">Nix remote builder</h3>
<h3 id="icon-services-nix-serve">Nix-serve-ng</h3>
<h3 id="icon-services-dnsmasq">dnsmasq</h3>
<h2 id="icon-observability">Observability</h2>
<h3 id="icon-observability-collectd">collectd</h3>
<h3 id="icon-observability-smartd">smartd</h3>
<h3 id="icon-observability-upsd-upsmon">upsd+upsmon</h3>
<h2 id="icon-notes">Notes</h2>


			<h1 id="ups">UPS</h1>
			<p>The <a href="https://www.cyberpowersystems.com/product/ups/pfc-sinewave/cp850pfclcd/"><span class="canonical-name">CyberPower PFC Sinewave CP850PFCLCD</span></a> Uninterruptible Power Supply (UPS) ensures that blackout and brownouts don't corrupt my systems. It is able to switch from line power to battery in under 8ms. The batteries can be replaced with the <a href="https://www.cyberpowersystems.com/product/ups/replacement-batteries/rb1280a/"><span class="canonical-name">RB1280A</span></a> or other compatible <a href="https://batteryclerk.ca/products/cyberpower-cp850pfclcd-12v-9ah-ups-battery">third party batteries</a>. Here's a <a href="https://www.cyberpowersystems.com/resources/cp850avrlcd-um/">link to the user manual</a>.</p>
			<figure id="ups-front">
			<img fetchpriority="high" width="720"  alt="front of a UPS" src="/blog/homelab_ups.webp">
				<figcaption>My equipment doesn't use a lot of power, so it can survive quite a while on this tiny UPS.</figcaption>
			</figure>
			<p>The status of the UPS and it's batteries can be monitored using <span class="canonical-name">upsd</span> and <span class="canonical-name">upsmon</span> to gracefully shutdown if power doesn't come back online before the battery runs out. This is still pending TODO on the homelab :/</p>

			<h1 id="akesis">Askesis (VPS)</h1>
			<p>Here's the VPS I got from <a href="https://hosthatch.com">HostHatch</a> for 1 vCPU, 2 GB RAM, and 10 G of storage for $4 USD/mo. At least they let me get an additional static IPv4 for $2 USD/mo after contacting support. I chose them because an online acquaintance recommended them for their reputable IPs for email deliverability.</p>
<aside><a href="https://cloud.hosthatch.com/a/5382">HostHatch referral code</a></aside>
			<figure id="akesis-fetch">
			<img fetchpriority="high" width="720"  alt="specifications of VPS" src="/blog/homelab_akesis_fetch.webp">
				<figcaption>I used a stackable file system mount on <code>/var/www</code> to make better use of my limited disk space. I learned how to do this from the <a href="https://nostarch.com/download/samples/ao2e_ch8.pdf">sample chapter of Absolute OpenBSD 2e</a> which sold me on the author's technical expertise and teaching ability.</figcaption>
			</figure>
			<p>I mostly use the base packages on this system such as <code>pf</code>, <code>relayd</code>, <code>httpd</code>, and <code>acme-client</code>. This has served me very well because it's high performance at low cost!</p>
<h2 id="akesis-services">Services</h2>
<h3 id="akesis-services-pf">pf</h3>
<h3 id="akesis-services-relayd">relayd</h3>
<h3 id="akesis-services-acme">acme-client</h3>
<h3 id="akesis-services-httpd">httpd</h3>

		</main>
		<footer class="blog-footer">
			<p>Return to <a href="#top">top?</a></p>
			<a href="https://skullheadx.com">https://skullheadx.com</a>
		</footer>
	</body>
</html>