f86d065005
Add ZT management-only network
2022-02-13 11:05:25 -06:00
5efeaa18e9
Polish up checks
2022-02-13 10:11:45 -06:00
eeb1afb31b
Revert "Lock gulagbot to 4.1.8"
...
This reverts commit 4be2e313fd
2022-02-11 13:57:31 -06:00
4be2e313fd
Lock gulagbot to 4.1.8
...
Awaiting resolution of some bugs
2022-02-10 23:52:36 -06:00
d48f04dbe0
Update Forge to 36.2.26
...
This fixes a bug with Forge with a missing class on latest Java
2022-02-09 17:16:49 -06:00
750f7cad15
Change some Stalin settings
2022-02-04 23:04:36 -06:00
cd805fa79f
Purge mainline from framework laptop
2022-02-03 13:28:54 -06:00
0ed99bec3e
Re-add mem_sleep_default to lap-fw-diy-1's kernel params
...
This reverts commit f345e64b79
2022-02-03 13:25:44 -06:00
6c5bea19f1
Add snd-hda-intel args to kernel cmdline
...
I encountered a bug with the new kernels which appears to have caused my audio to dip out.
2022-02-03 10:29:18 -06:00
f345e64b79
Revert "Add mem_sleep_default to lap-fw-diy-1's kernel params"
...
This reverts commit 3c632ce64c
2022-02-03 08:58:27 -06:00
0963554422
Add mainline kernel update utility
...
>=5.14 is required for the Framework to function fully. Since this is just one machine, I'm okay with not completely automating this process.
2022-02-03 08:10:00 -06:00
50add7b8b9
Disable i915.enable_psr= (Re-enable PSR)
...
This comes in tandem with the next commit, which adds a utility for using the mainline kernel
2022-02-03 08:09:30 -06:00
3c632ce64c
Add mem_sleep_default to lap-fw-diy-1's kernel params
...
Apparently, it doesn't suspend to ram at the hardware level by default, which is strange and dumb.
2022-02-03 07:57:21 -06:00
46a567c46a
Make memory checks a lot tighter
...
Alright look, I design my systems with 10% overhead, not 20%
2022-02-02 07:41:22 -06:00
714e78b4d7
Test an older version of the E6 pack with a different set of performance mods
...
I've heard Performant is mostly placebo, and we're finally at a point where we can thoroughly test and see how it runs compared to the suite of performance mods shipped by the pack normally.
2022-01-29 21:17:28 -06:00
c49d4cefad
Provision a new mc serber
2022-01-28 18:37:07 -06:00
aef8f31c87
Change Nagios Matrix channel
2022-01-27 14:31:58 -06:00
662d94a948
Fix incorrect group for Zerotier checks
2022-01-24 18:07:53 -06:00
3810b96a38
Rework Zerotier role to allow for arbitrary adds and deletes
2022-01-23 17:17:43 -06:00
53ffaf52c4
Reorganize playbooks to modularize Zerotier enrollment
2022-01-23 16:37:56 -06:00
964d3228e2
Add some more correct settings for jenkins
2022-01-23 08:47:26 -06:00
c8793cc240
Recalibrate Stalin to be less... totalitarian
2022-01-22 19:30:35 -06:00
5e9c137b34
Decom peertube, add jenkins
2022-01-22 09:48:47 -06:00
380781b1eb
Re-enable NFS mountpoint for desktops
2022-01-21 22:24:03 -06:00
269a7ec784
Enroll key for lap-fw-diy-1, give lap-s76-lemp9-1 some nicer Grub configs for dualboot
2022-01-21 17:18:56 -06:00
1a4f59b71a
Add a new laptop, rebase zerotier repo
2022-01-21 16:55:10 -06:00
e4805658dc
Spin up a Terraria server
2022-01-12 15:14:30 -06:00
8984f71aee
Upgrade Netbox
2022-01-12 14:23:26 -06:00
fe00d12377
Add some opt-in docker checks
2022-01-11 18:30:27 -06:00
fb465a666e
Add temperature sensors to Pis
2022-01-11 00:16:54 -06:00
4a1c5ea8e1
Use new-style checks for the temperature command
2022-01-11 00:00:31 -06:00
0140784111
Add hostgroup for device types
2022-01-10 23:54:03 -06:00
8517e842b2
Add checks for the R720's thermal monitors, also add those thermal monitors
2022-01-10 22:14:07 -06:00
8a3f13a939
Give ansible play check a bit more wiggle room, missing a single day isn't that bad
2022-01-09 11:32:16 -06:00
c59781b47e
Define a timeperiod for Nagios and assign it to our matrix notify user
2022-01-04 18:00:55 -06:00
e0b385a1c5
Fix accidentally removing a ton of checks from each host
2022-01-03 13:18:14 -06:00
e2d738ba40
Enroll vm-scan-1 in backups
2021-12-30 10:37:50 -06:00
9662bb0ff8
Ensure we actually add our new user to sudoers
2021-12-30 10:12:44 -06:00
111f1cdef6
Configure a scanner user on all machines
2021-12-30 10:04:31 -06:00
1cff8a6aa8
Deploy GVM to a box at home
2021-12-30 09:34:45 -06:00
ed64fc0a9a
Backup etc on desktops
2021-12-30 07:48:15 -06:00
db78f7eaf6
Separate HOSTALIAS from SERVICEDESC more properly on Nagios
2021-12-26 22:56:03 -06:00
8612eec85e
Create an Ansible contactgroup in Nagios and tie it to all alerts, enroll our Matrix user in that group
2021-12-24 16:56:06 -06:00
1791c40465
Working on Matrix integration for Nagios
2021-12-24 16:47:21 -06:00
c6c57fce6c
Change alert destination email address
2021-12-24 15:36:18 -06:00
09f33966ac
Disable memory checks for machines running ZFS
...
I give up. I'll circle back on this later
2021-12-24 15:32:53 -06:00
aa493348d3
Add another Minecraft server and some related checks
2021-12-24 14:53:09 -06:00
22863e66e7
Upgrade Nextcloud 22 -> 23
2021-12-24 12:16:44 -06:00
d7c3f97797
Set up ddclient
2021-12-21 11:16:40 -06:00
5e7b8bb881
Add a Minecraft server *at home*
2021-12-20 17:24:11 -06:00
9b64cf8a00
Modularize sanitization cronjobs
2021-12-16 08:11:17 -06:00
6b218b02f9
Add a cronjob to Syncthing to clean up :Zone.Identifier files
2021-12-16 07:44:01 -06:00
911d236c84
Implement a sanitize rule for syncthing
2021-12-15 22:19:05 -06:00
060aa14df3
Fix incorrect dir for cp2077 screenshots
2021-12-15 21:12:24 -06:00
e93124e556
Add more directories to sort out, make the jobs run in parallel at 5AM
2021-12-15 21:10:20 -06:00
58196e3f24
Genericize that cronjob syntax for future endeavors
2021-12-15 20:56:45 -06:00
640e2e0efe
Add a cronjob for a specific bug I'm working around with Syncthing
2021-12-15 20:41:14 -06:00
5031833f39
Remove Package Updates check
...
It's just pointless noise to be honest, it's way too loud. Perhaps a proper patch management solution would be in order?
2021-12-15 20:06:12 -06:00
72697a3953
Move check_disk to those restricted checks, also exclude AppImage loopback mounts
2021-12-15 19:57:20 -06:00
54a4f1539b
Add some sudo rules to nagios-checker so it can start doing restricted checks
2021-12-15 19:57:08 -06:00
4b626dc6be
Implement communication with Nagios when rebooting boxes
...
One step closer to that full automation goal
2021-12-15 19:32:19 -06:00
000d711d7a
Update gulagbot DB IP
2021-12-12 09:54:15 -06:00
31018efeb1
Expose Jellyfin over 192.168.*
2021-12-11 21:50:20 -06:00
e0ce07c4dc
Restart Jellyfin unless stopped
2021-12-11 16:01:21 -06:00
9aab2d6557
Tune the transaction limits for that check we just added
2021-12-09 16:23:59 -06:00
273d83be64
Add a check for old, uncommited PostgreSQL transactions
...
Sometimes reading the blogs of developers whose software you use is worth it
2021-12-09 16:17:49 -06:00
fcffd834a0
Move Nagios into its own role
...
It was getting way too big
2021-12-08 21:34:32 -06:00
a71071b321
Spin up a SL server
2021-12-01 22:34:46 -06:00
386b190130
Add vm-desktop-1 to list of workstations
2021-12-01 07:31:38 -06:00
e85d81ef38
Drop logs for lr.cowfee.moe
2021-11-30 14:11:27 -06:00
558709ce6f
Deploy libreddit
2021-11-29 23:33:56 -06:00
fab7be68c5
Tune thresholds for monitoring the age of ansible-last-run
2021-11-29 22:15:09 -06:00
1952f72c89
Add a check for the last ansible run on a given machine
2021-11-25 16:41:17 -06:00
5b12eb5af2
Add a cleanup task to touch a file upon completion of site.yml
...
This playbook *should* assure that we have a file we can use for checking when the last full play was. It being in a playbook at the tail end of site.yml is paramount, since site.yml dying will cause alarms to be set off.
2021-11-25 16:19:01 -06:00
ce37a7fec3
Rename a bunch of minecraft tasks to prevent ambiguity
2021-11-25 13:30:21 -06:00
6c4b1c701b
Fix some unquoted number variables causing the gulagbot task to fail
2021-11-25 13:20:23 -06:00
84cd7888f1
Shut down hexxit2
2021-11-25 13:19:45 -06:00
12f33d9ffc
Put Syncthing behind an ingress container
2021-11-24 10:43:29 -06:00
3d9ec54467
Nevermind, guess we scope it out even *higher*
2021-11-24 07:43:18 -06:00
733d1006be
Adjust mountpoint for NFS to allow for access to higher dirs like syncthing
2021-11-24 07:41:05 -06:00
811d0bd2d2
Add some params for a new version of gulagbot
2021-11-22 14:12:13 -06:00
decd4b452b
Add transaction logging to home DB
2021-11-21 19:57:33 -06:00
1c4bf65db4
Add a test DB for gulagbot
2021-11-21 17:01:58 -06:00
d67bc370ac
Split nagios-checkhttp into nagios-checkhttp{,s}
2021-11-21 13:59:05 -06:00
7976630ad7
Add a box for Syncthing
2021-11-20 18:29:31 -06:00
ee5f8ffe92
Make a box to move Stalin back home where he belongs
2021-11-19 20:38:20 -06:00
cff68a2a73
Reorganize Jellyfin to reflect usage of Nvidia Container Toolkit, configure hw accel
2021-11-18 19:16:30 -06:00
05a7f19bfe
Update backup dirs for PMX to reflect some PCIe passthrough stuff I'm doing
2021-11-18 16:51:32 -06:00
9680705689
Update NFS mounts for desktops
2021-11-15 11:07:11 -06:00
bca5c1993d
Add a bunch of unit checks for Proxmox boxes
2021-11-11 14:00:25 -06:00
5794379da4
Add some backup configuration for PMX hosts
2021-11-11 13:54:11 -06:00
77084ebc49
Reorganize media dirs, add jellyfin to vm-media-1
2021-11-11 00:23:53 -06:00
b0b71abf6a
Reorganize NFS mounts so that they don't contain SQLite DBs
2021-11-10 19:29:06 -06:00
57b1cf03ca
Set nfs to rw on local connections
2021-11-10 19:14:02 -06:00
7f7a0fd2ba
Run setup tasks before roles on vm-media-1
2021-11-10 19:11:10 -06:00
f3b12234c0
Start reorganizing to have pi-media-1 split into vms
2021-11-09 20:53:59 -06:00
cb6581b708
Add home db playbook
2021-11-08 16:44:04 -06:00
8c213fe693
Ensure hexxit2 backups aren't getting tarred in
2021-11-08 10:04:15 -06:00
c5d39db270
Actually implement device roles in Nagios
2021-11-07 08:55:05 -06:00
f250936fe9
Disable some relatively standard checks on hypervisors, since they're special
2021-11-07 08:38:27 -06:00
f07cb9e35c
Disable docker checks for machines that don't have docker
2021-11-07 08:36:16 -06:00
4efb757c43
Download Hexxit from 9iron
2021-11-07 07:52:33 -06:00
f53726c68a
Add lag goggles to hexxy
2021-11-06 16:25:44 -05:00
0edbac0520
Fix typo on no-docker tag
2021-11-06 15:56:35 -05:00
635c8c1bf4
Move motd configuration to Ubuntu machines and only Ubuntu machines
2021-11-06 15:53:57 -05:00
ea2e98e6ae
Add Hexxit server, removing the tmod one
2021-11-06 14:24:57 -05:00
cec0a5c3f8
Add Ardour to desktops
2021-10-25 19:37:40 -05:00
7bbc291cf8
Edit hostnames on workstations to reflect their actual ones
2021-10-25 19:10:25 -05:00
003b13fa84
Update Gulagbot to latest
2021-10-20 12:29:53 -05:00
7e7030c613
Fix syntax on cronjob (hopefully)
2021-10-17 11:08:31 -05:00
7b624d431a
Change backup cronjob up a bit for Terraria
2021-10-17 10:53:51 -05:00
07647e5ee6
Add check to devices to ensure they can ping themselves over DNS
2021-10-15 19:17:48 -05:00
fd55782766
Overhaul DNS names for machines
2021-10-15 19:03:55 -05:00
ba228984c1
Add local backups for Terraria Fargo
2021-10-14 22:57:24 -05:00
ed1c59662c
Deploy a new box with Fargo
2021-10-13 12:24:27 -05:00
e5441bcc2e
Update to Nextcloud 22
2021-10-12 15:59:30 -05:00
b15fdd96f5
Install imagemagick on Nextcloud to make a big warning triangle go away
2021-10-12 15:29:06 -05:00
5bc39e7f48
Fix being unable to access said share
2021-10-07 10:40:31 -05:00
cf60d672b7
Add Samba to pi-media-1
...
[that was easy]
2021-10-07 10:39:20 -05:00
4f07856028
Modularize contact definitions
2021-10-06 15:30:12 -05:00
37c55b9cb2
Change templating behavior of certain sections of the Nagios config
2021-10-06 15:13:55 -05:00
309bfd8694
Stop notifying on warnings for package updates
2021-10-06 15:12:41 -05:00
e85104c9fd
Add DNS resolution check
2021-10-05 10:28:06 -05:00
bc1b927298
Use check_packages to check for package updates on Debian systems
2021-10-05 10:16:22 -05:00
c9808bb171
Revert "Add stale library check"
...
This reverts commit 0beef5617b
2021-10-03 23:54:32 -05:00
0beef5617b
Add stale library check
2021-10-03 22:39:43 -05:00
1e1946d8e0
Add memory checks to hosts
2021-10-03 22:26:37 -05:00
da3f0a24f4
Add CPU Utilization check, nerf CPU Load check
2021-10-03 15:50:25 -05:00
bfab992eb8
Add check for unapplied package updates
2021-10-03 15:47:28 -05:00
3e20928e14
Add health endpoint to exposed endpoints on matrix.desu.ltd
2021-10-03 11:51:34 -05:00
7669234df9
Allow the addition of custom checks based on config_context, add roles to hostgroups
2021-10-03 11:48:53 -05:00
18655b7d62
Bump thresholds for PSQL connection check
2021-10-03 11:04:09 -05:00
42e2a3bd22
Fix client URL for Matrix being completely wrong. I guess.
2021-10-02 22:57:10 -05:00
c12d37cad2
Work on putting Element in place
2021-10-02 22:50:25 -05:00
7337fb49ed
Narrow down the pass locations for Matrix to just server endpoints
2021-10-02 22:11:10 -05:00
e05d4a379b
Add basic Synapse server configuration
2021-10-02 22:03:22 -05:00
aceba8407b
Add DB configuration for Synapse
2021-10-02 22:03:05 -05:00
d06fc65af9
Correct errors in nginx configuration
2021-10-02 21:53:44 -05:00
e6b2c8b0a6
Configure web1.desu.ltd for Matrix delegation
...
Big things a comin
2021-10-02 21:46:32 -05:00
a7aa38a8e9
Add automatic reboots to main playbook
2021-10-01 09:10:02 -05:00
90da5ad3b1
Hardlock gulagbot to 2.4.0
...
I BROKE IT
AAA
2021-09-29 20:16:45 -05:00
2baffca0f5
Add configuration for Home Assistant
2021-09-27 17:18:30 -05:00
27bb55bf22
Convert pi-media-1 to ingress role
2021-09-27 15:12:20 -05:00
9039a75d3c
Add note to replace Nagios with naemon(?)
2021-09-26 10:40:35 -05:00
1c1c8e41ae
Null-mount nsca on Nagios image
...
God DAMN the log spam from this thing I'm not using is fucking ridiculous
2021-09-26 10:27:33 -05:00
427014f2ae
Sanitize tag hostgroups in nagios with the tag- prefix
...
Stumbled across an issue where I can't have a Netbox tag that's just 'ansible'
2021-09-26 10:23:46 -05:00
0c8aa0a90f
Add test DB for gulagbot on Linode
2021-09-26 08:03:37 -05:00
7779db30ad
Use ansible_managed where possible
2021-09-24 20:48:41 -05:00
87f9c6ceb3
Rename swap checks to be agnostic of underlying tech
2021-09-24 13:25:21 -05:00
fb006b0cd3
Add playbook and Netbox tag to run the ansible role on a host
2021-09-24 13:03:21 -05:00
8ecc7bfbb6
Modularize Netbox into several containers with workers n stuff
2021-09-23 22:09:38 -05:00
8d59a1b201
Rework mounts for netbox container
2021-09-23 21:39:10 -05:00
81988a50fd
Remove defunct deb link for raspberry pi imager
2021-09-23 21:32:36 -05:00
fdeb143a56
Apply mitigation for netbox-community/netbox-docker#586 , update Netbox
2021-09-21 14:49:34 -05:00
f7b5c475d5
Add device_roles_bastion play
2021-09-19 21:49:15 -05:00
fe5eb5c14d
Convert role invocations to use the full namespace of the role
2021-09-18 16:10:54 -05:00
68eb7e5422
Pin Netbox to 3.0.1 since apparently the container's broken
2021-09-18 08:52:05 -05:00
6382a81f47
Remove some extraneous backup locations on web1
2021-09-18 07:27:59 -05:00
31a2371fa1
Simplify task includes
2021-09-18 07:23:03 -05:00
9b79068380
Allow for the definition of a singular proxy_pass on ingress_servers to simplify configuration
2021-09-18 07:19:26 -05:00
60bfe91947
Add role for ingress controller, move configuration into it and its data structures
2021-09-18 00:04:05 -05:00
37150bf7d1
Remove polkit.service check
...
Apparently it's completely normal behavior for this service to be not running on a fresh boot
2021-09-14 19:40:53 -05:00
0f1fbf4fea
Allow 30 second timeouts on check_by_ssh
2021-09-14 17:26:47 -05:00
ac702380b1
Add git to the tags for monitoring-scripts
2021-09-14 17:22:50 -05:00
b4f564cade
Fix mountpoints and NFS exports for media
2021-09-13 13:59:27 -05:00
3f3c7b8392
Decom the K8s cluster, roll all its jobs into one singular machine
2021-09-13 13:50:22 -05:00
e49ebc583f
Upgrade Netbox to 3.0
2021-09-12 15:07:31 -05:00
e405d7bf79
Add some directives to make Nextcloud stop throwing 413s
2021-09-11 10:36:22 -05:00
3f8ecbd8f5
Fix my borked pgsql connection pooling check
2021-09-07 17:08:18 -05:00
4bf02aedd3
Add even more checks for zerotier and psql
2021-09-07 16:11:11 -05:00
3cf9b94cea
Add a quick service check for postgresql
2021-09-07 15:29:26 -05:00
b349015913
Add a ton more checks for things
2021-09-07 15:00:43 -05:00
92f26b7a0c
Add check for atd
2021-09-07 14:55:00 -05:00
c362effe2a
Remove NRPE
2021-09-07 14:33:45 -05:00
bad192e93e
Refactor Nagios checks into check_by_ssh instead of NRPE
...
I was never particularly fond of having a random one-off daemon doing my RCE. Sure, it offers some protection, but limiting my exposure to the open internet is far more ideal.
I have tremendously more trust in the OpenSSH project than I do in Nagios. And for that reason, I'll be deprecating NRPE and shredding config files once these plays clean up
2021-09-07 14:27:23 -05:00
b38bb4bf62
Fix improper tagging on NRPE role
2021-09-07 13:41:21 -05:00
1ca062d6ea
Modularize declaration of Nagios commands
2021-09-07 13:37:06 -05:00
2a7d343ef1
Move SSH check into YAML declaration of services
2021-09-07 13:29:19 -05:00
8e845b5f4e
Modularize out all our service checks
...
I want them in DATA STRUCTURES God dammit. Get them out of the config file.
2021-09-06 19:43:54 -05:00
d3e51301bb
Remove deprecated SNMP service checks
2021-09-06 19:23:54 -05:00
fc2b3cb7b3
Rename Nagios config to more appropriately reflect its role
2021-09-06 19:13:15 -05:00
360238fdd4
Ensure we're on a version of Netbox with secrets support
...
*sigh*
Guess I gotta set up a vault or something now.
2021-09-01 19:25:31 -05:00
c299e505cf
Add Nextcloud auto app update cronjob
2021-08-29 23:55:56 -05:00
4bea6c2168
Add _netdev to args for pi-storage-1 mount
2021-08-29 16:43:55 -05:00
a6a8cd8590
Figure out how custom_apps works with Nextcloud
2021-08-28 11:01:44 -05:00
579b2fa296
Move "all" configuration into its own playbook
2021-08-26 02:39:17 -05:00
62b6a93b65
Discard cron output again
2021-08-24 21:22:11 -05:00
20e73e6fcf
I'm fucking stupid? Don't put the TTY flag on things that aren't TTYs
2021-08-24 21:21:53 -05:00
89e86efafc
Log output of Nextcloud cron to file for debugging
2021-08-24 21:16:44 -05:00
45098866e3
Add some stuff for MOVIE NIGHT WIT DA BOIS
2021-08-24 16:58:45 -05:00
2cef4b1992
Fix incorrect mountpoint for srv
2021-08-24 13:28:53 -05:00
6a938ea6b3
Add Nagios user to pi-storage-1
2021-08-24 12:40:32 -05:00
3b133782c9
Have pi-storage-1 psql listen on localhost
2021-08-24 12:29:44 -05:00
f6004def4a
Add system-wide cronjob for Nextcloud cron
...
Guess that's not containerized, huh
2021-08-24 12:25:29 -05:00
145dcfe3fb
Add Redis for Nextcloud, plus some config tuning
2021-08-24 01:12:12 -05:00
fc6739907e
Remove unnecessary tasks and var files
2021-08-24 00:44:17 -05:00
e49b8e26a0
Fix srv.9iron.club using a mountpoint that didn't make sense
2021-08-24 00:39:42 -05:00
54eeb4a643
God damn can I stop forgetting random small shit please
2021-08-24 00:35:41 -05:00
d8bf31b144
Add rewrite for www.9iron.club to nginx config
2021-08-24 00:32:33 -05:00
1fb222fb15
Move web1 over to a containerized setup, containerize Nextcloud
2021-08-24 00:31:11 -05:00
a6cc1ecece
Move ansible_pull vars to the relevant playbook
2021-08-23 23:25:02 -05:00
38b52a5e4a
Make said playbook executable
2021-08-23 23:22:17 -05:00
5486f26c76
Move S76 configuration to its own playbook with a couple of tasks
2021-08-23 23:21:58 -05:00
02dd6cd553
Reorganize ALL of the playbooks
2021-08-23 20:28:18 -05:00
a2a5f6eedc
Begin a refactor of playbook naming and organization
2021-08-23 20:20:59 -05:00
7f8a06180d
Rename desktops from tags_desktop to device_roles_workstation
...
I already have the role so I may as well
2021-08-23 20:05:27 -05:00
535509db0a
Fix open quotes on NRPE config
2021-08-23 18:23:30 -05:00
26c776a7db
Add check_pgsql monitors
2021-08-23 18:18:53 -05:00
406adc20b9
Tune psql1 a bit to avoid more Bleromer outages
2021-08-22 10:45:23 -05:00
7d3e8b5a86
Specify backup dirs per-desktop
2021-08-20 15:07:27 -05:00
227f5828cd
Oh right the disk check is here
2021-08-18 23:45:21 -05:00
888353910d
Add checks for reboot-required
2021-08-18 23:01:26 -05:00
c031124246
Tighten the thresholds for disk warnings a bit more
2021-08-18 22:37:22 -05:00
ea8e205b42
Rename a bunch of checks to be shorter
2021-08-18 22:28:41 -05:00
5efa094fdc
Back up Pi k8s nodes in some rudimentary way
2021-08-18 19:15:52 -05:00
94edbeafd9
Add checks for some common Systemd units
2021-08-18 19:05:16 -05:00
051fee73d3
Clone a new monitoring-scripts repo to hosts with NRPE installed
2021-08-18 18:16:43 -05:00
812b6dff77
Destroy old MC servers
2021-08-16 00:02:37 -05:00
f8951d61a4
Tag swap monitoring separately from other NRPE checks
2021-08-15 15:36:25 -05:00
beb8cad9ed
Fix swap being way too lenient when there's no swap space at all
2021-08-15 15:28:34 -05:00
668ef3a75f
Fix up some checks, add the swap check
2021-08-15 15:21:46 -05:00
9f4727b6c9
Tweak checks to make them more better
2021-08-15 15:18:52 -05:00
4d0b005181
Add NRPE role, provision checks for it
2021-08-15 14:24:35 -05:00
2918a3348b
Polish up our SNMP checks and playbooks
2021-08-15 13:09:04 -05:00
c745de9309
Reorder args on TCP checks to better match the natural sorting order of the iterations
2021-08-15 02:44:42 -05:00
94f6d45d07
Fix HTTP checks that redirect to TLS connections failing
2021-08-15 02:43:59 -05:00
ce77c104a6
Fix typo in docker-prune playbook
2021-08-15 02:30:20 -05:00
9ab0f62442
Genericize manually-defined checks into tagged Netbox services
2021-08-15 02:29:56 -05:00
73abab9607
Add docker-prune playbook
2021-08-15 00:59:08 -05:00
be7fa959ea
Switch to a regex match for that SNMP check
2021-08-08 15:34:11 -05:00
da432c0dcc
Make our Nagios SNMP user, apply some changes to its container, and spin up some barebones checks
2021-08-08 14:46:58 -05:00
a254910cdc
Testing some SNMP stuff
2021-08-08 13:36:52 -05:00
caadf375f2
Add basic site checks
2021-08-08 12:35:35 -05:00
be7d1a24d6
Auto-restart nagios when its config changes
2021-08-08 12:15:10 -05:00
4c2bfb996c
Actually add that Nagios template
2021-08-08 02:24:59 -05:00
e968d4a7cf
Fix up that jank config and make it actually totally usable
2021-08-08 01:20:48 -05:00
5d5cab59eb
Add Nagios and some dysfunctional templating code
2021-08-08 00:28:25 -05:00
691a934297
Genericize the inclusion of libraspberrypi-bin
2021-08-07 17:23:15 -05:00
d68e3430a8
Modularize zerotier as well
2021-08-07 17:14:28 -05:00
759df2f593
Allow for dynamic tagging of ansible-pull hosts
2021-08-07 17:09:20 -05:00
07ea9806da
Docker goes on everything
2021-08-07 16:55:28 -05:00
38f70d0fca
Unify motd definition
2021-08-07 16:52:19 -05:00
30dd4ff8dc
Divide webservices into task files
2021-08-07 16:49:24 -05:00
69f3edcf2b
Clean out deprecated k8s garbage
2021-08-07 16:31:36 -05:00
55304ac4d9
Rename pistorage to tags_pistorage
2021-08-07 12:16:07 -05:00
340da1926e
Move gameservers into task files
2021-08-07 12:08:29 -05:00
d6328146b3
Add nfs-common to common role
2021-08-07 11:52:03 -05:00
51b9ee745f
Spin botaniapack2 back up
2021-08-03 16:04:10 -05:00
de8c216dbf
Fix incorrect tag naming for k8s Pis
2021-08-02 11:00:37 -05:00
c831a76517
Remove netbox DB creds from pi-storage-1
2021-08-01 23:07:44 -05:00
ccad04d544
Attempt to pivot to a Netbox-based inventory solution
2021-08-01 22:28:39 -05:00
3051d8a299
Provision (read: move) Netbox to desu.ltd
2021-08-01 21:35:41 -05:00
ba321f0e91
Add Terraform stuff
2021-08-01 19:36:39 -05:00
ce034d28b0
Also, only those hosts that use ansible-pull should pull the repo to /etc/ansible
2021-08-01 15:07:02 -05:00
79220bef76
Disable ansible-pull for most systems
...
From now on, nightly plays triggered by GitLab will be standard.
Desktops will still use an ansible-pull configuration as there is
no guarantee that they will be available at 1AM.
2021-08-01 15:00:05 -05:00
1ef715f52e
Disable botaniapack2
2021-08-01 12:32:24 -05:00
e8c3ce3d13
Configure Docker per-machine
2021-07-27 17:41:27 -05:00
df86a7bf38
Also get rid of telegram
2021-07-27 17:23:40 -05:00
c6de02ec1b
Also remove cap_sys_ptrace on Python
2021-07-27 17:17:07 -05:00
2380276b10
Remove random liblzo2 dllmap
2021-07-27 17:16:41 -05:00
b851ca7f35
Disable GRUB provisioning on dsk-ryzen-0
2021-07-27 17:08:19 -05:00
33394ca2b7
Separate packages out based on machines, removing a ton from the defaults
2021-07-27 16:45:56 -05:00
9f48999cbb
Switch to a map that we actually have
2021-07-26 21:58:31 -05:00
14e5c69c77
Add our fancy fun Prop Hunt serber
2021-07-18 23:21:00 -05:00
0b7e67ef2d
Mount server.properties
2021-07-18 18:43:40 -05:00
3544aedffe
Reconfigure gameservers to serve Minecraft and nothing else
2021-07-15 00:35:32 -05:00
4f71ac2035
Fix redirects for 9iron.club not going to... 9iron.club
2021-07-13 12:34:01 -05:00
387cbe26c9
Mount a volume for modpacks
2021-07-13 12:26:29 -05:00
bde93782f9
Retire the Valhelsia serb
2021-07-13 12:22:22 -05:00
53388bb063
Update CSS on launch
2021-07-03 22:19:43 -05:00
805bcc739e
Remove deprecated 9iron varfiles
2021-07-01 12:06:30 -05:00
517073bfee
Remove some deprecated Gitea lines
2021-07-01 12:04:42 -05:00
0c4846ce16
Remove clone of otwstudios.org
2021-07-01 12:03:51 -05:00
4de517abb0
Decom 9iron heads
2021-06-20 21:47:57 -05:00
0eafe43b7f
Clearing up some lint complaints
2021-06-20 19:15:15 -05:00
7fee7bdcf1
Use module_defaults on web1
2021-06-18 12:39:21 -05:00
7817ed5e44
Redirect www. -> . for consistency
2021-06-18 11:37:29 -05:00
7653af202a
Reconfigure LE certs
2021-06-18 11:36:03 -05:00
d0af712d89
Put 9iron back on web1
2021-06-18 11:31:50 -05:00
ac8aa9a049
Add server.cfg to docker mount
2021-06-16 01:51:29 -05:00
1af1698d82
CSS LET'S GO
2021-06-15 23:21:18 -05:00
c028e5a3fe
Turn scpsl back on
2021-06-15 22:42:38 -05:00
0311009a68
Remove mention of vars/desultd-gitea.yml
2021-06-14 17:26:01 -05:00
a208913890
Back up our jank shell script
...
ew
2021-06-14 17:16:48 -05:00
32d2cd2f43
Move Gitea to a container
...
Getting there
2021-06-14 17:16:26 -05:00
40ec82cf47
Move starbound to the beefier box
2021-06-11 12:20:26 -05:00
992fad17cf
Disable setting DB extents
2021-06-11 01:07:02 -05:00
ea44b7ace2
Stop SCPSL container
2021-06-10 01:55:00 -05:00
1afbc806b2
How about starbound
2021-06-08 23:18:26 -05:00
fecdbfd48c
Deploy Peertube
2021-06-07 02:52:11 -05:00
3be959ee6a
Configure nginx after every other container
2021-06-07 00:46:05 -05:00
d97e81f9f2
Employ defaults on web2
2021-06-07 00:45:52 -05:00
bb349c634f
Migrate Pleroma into a Docker container
...
This greatly simplifies configuration on web2, oddly enough
2021-06-03 20:34:48 -05:00
07e666b978
Get ready for a Peertube install
2021-06-03 19:11:50 -05:00
da6dbd4748
Add pi-storage-2
2021-05-25 15:35:50 -05:00
ab82a9038c
Upgrade to OpenJDK11 for Minecraft
2021-05-24 22:25:19 -05:00
a0fd419c79
Correct the timezone in the Minecraft container
2021-05-21 11:49:27 -05:00
178fc54a0a
Tune memory requirements for valhelsia
...
Having issues with it swapping out and have no evidence of it ever GC thrashing
2021-05-17 14:17:37 -05:00
da0a37d461
Generate directory indexes on srv.9iron.club
2021-05-14 10:14:12 -05:00
cf8ef7c3db
Don't backup the on-disk snapshots, we don't need it lol
2021-05-14 09:51:32 -05:00
772ed5e62e
Add logs dir to Docker mounts on valhelsia 3
2021-05-12 23:35:43 -05:00
e3264e3035
Bump container version for Minecraft image
2021-05-12 23:09:20 -05:00
fce5a132dc
Expose MC backups to host
2021-05-12 18:51:32 -05:00
7989ade3f5
Build like everything to set up a Minecraft server
2021-05-11 22:28:19 -05:00
c9656638be
Add a Minecraft serber
2021-05-11 21:16:29 -05:00
ba62f3318d
Cleanup
2021-05-04 14:24:53 -05:00
7484dce678
Update backup script to do it all, allowing for directory AND DB backups
2021-05-04 13:38:39 -05:00
26c45ff080
Add Firefly DB to Pi cluster
2021-05-02 15:25:14 -05:00
225a6b08e4
Switch to directory backups for pi-storage-1
...
Just got burned :c
2021-05-01 19:44:28 -05:00
b85120cd5a
SCPSL
2021-05-01 01:02:42 -05:00
054e462589
Whoops, this kills the keepalived
2021-04-29 11:52:28 -05:00
1d0d1f7eb2
Add a public VIP to the k8s cluster
2021-04-29 11:48:22 -05:00
18c08bcbec
Decom netdata
2021-04-17 21:46:33 -05:00
110bdd4bfb
Decom game1
2021-04-17 21:46:20 -05:00
a3c052d13b
Move desultd to its own container
2021-04-08 14:31:29 -05:00
8d2350da9c
DEPLOY STALIN
2021-04-03 03:40:29 -05:00
80c67ac679
Start initializing gulagbot
2021-04-02 16:31:48 -05:00
055a6bd602
Update tmodloader
2021-03-31 22:18:40 -05:00
45187b069c
Switch tmodloader containers
2021-03-30 19:37:31 -05:00
94c16d6df8
More nodes
2021-03-26 21:58:55 -05:00
8d544165a0
Enroll the previous pimps into the k8s cluster
2021-03-26 20:38:33 -05:00
3d10288d6d
Deploy 9iron container to production
2021-03-25 12:59:40 -05:00
93d69ee303
Remove extraneous package tasks for python3-docker
2021-03-25 12:55:57 -05:00
48ecd2e27f
Deploy docker to web1.desu.ltd
...
Also adds docker libs to all hosts
2021-03-25 12:51:47 -05:00
5deed42849
Pass more VIPs around for DNS usage
2021-03-24 17:50:55 -05:00
d1252592f1
Modularize keepalived configs
2021-03-24 17:49:25 -05:00
23ce4ca27b
Enable Terraria
2021-03-22 21:33:06 -05:00
7007e0ab92
Move Terraria to game2
2021-03-21 21:06:52 -05:00
5cddcf236d
Oh hey that needs to exist
2021-03-16 21:59:06 -05:00
d8446b1bb2
Provision game2.desu.ltd for SCPSL and other things
2021-03-16 21:25:01 -05:00
c0693a1b41
Back up Postgres instead of a huge amount of data overnight
2021-03-15 08:41:18 -05:00
aef451260c
Backup Docker data on game1.desu.ltd
2021-03-13 16:21:40 -06:00
feedb56ca3
And this is to go even further wider
2021-03-11 20:45:24 -06:00
51ac72f086
Wider perms for the NFS share
2021-03-11 20:37:58 -06:00
3480547d35
Install keepalived on pi k8s nodes
2021-03-11 10:25:41 -06:00
00f1b6016d
Fix incorrect user for Pleroma DB
2021-03-11 09:39:04 -06:00
e1d625391e
Allow DB Pi to listen on ZT
2021-03-11 09:11:48 -06:00
9216ed876d
Allow DB connections over ZT
2021-03-10 13:34:27 -06:00
4b52155164
Add netbox DB
2021-03-10 13:29:59 -06:00
8fcffe650b
Rename Pleroma DB
...
Because apparently DB names with dashes are completely unsupported
2021-03-10 11:29:49 -06:00
a9235f0e2d
Add quick MOTD for docker containers
2021-03-10 10:41:40 -06:00
171d769d17
Mount up NFS over ZT on desktop machines
2021-03-09 15:50:04 -06:00
25ce121337
Allow ZT clients to read from k8s pvs
2021-03-09 15:41:54 -06:00
3df725e186
Friendship ended with Minecraft
...
Now Terraria is my new Best Friend
2021-03-06 20:51:16 -06:00
7e595e7e89
Pull images
2021-03-05 01:27:26 -06:00
1d317bcb28
Add docker to the MOTD of game1
2021-03-05 01:16:01 -06:00
1107764d16
Deploy a Docker container for SCPSL
2021-03-05 00:56:03 -06:00
d3126738fc
Enroll pi-storage-1 in the backups it needs
2021-03-02 04:55:47 -06:00
16a1f583fe
Add another directory to NFS server
2021-03-01 15:12:19 -06:00
975cd7280c
Actually have Postgres listen over the network
...
GOD D
2021-03-01 12:29:21 -06:00
