Commit Graph

670 Commits

Author SHA1 Message Date
6db274a959 Remove Docker CPU/Memory checks
Unnecessary, methinks
2022-05-15 15:39:53 -05:00
df55507c9d Privilege up Octoprint so it can actually access the ttydevs it needs to 2022-05-10 21:38:24 -05:00
4942c42422 Simplify the Octoprint stack since it requires websockets and stuff apparently and I really don't want to mess with that plus it's an embedded device so who really cares all that much 2022-05-10 07:00:24 -05:00
90ad046d98 Other minor configuration fixes to get the Octoprint stack up and running 2022-05-10 06:56:53 -05:00
243e1b079f Fix not creating the appropriate docker network ahead of time 2022-05-10 06:48:06 -05:00
2944fc79b7 Add Octoprint, accessible via octoprint.local.desu.ltd 2022-05-10 06:43:52 -05:00
1792066f49 Skip adding interfaces to DNS that don't have an associated IP address 2022-05-10 05:52:11 -05:00
128b3bd982 Add some fancy schmancy unicode to our monitoring alert scripts 2022-05-02 15:46:41 -05:00
2c410a1a7c Remove a deprecated entry for movienight task include 2022-05-02 15:21:59 -05:00
7e40ad28ca Remove libreddit
Not sadge
2022-05-02 15:20:52 -05:00
73a5208d11 Remove movienight
sadge
2022-05-02 15:19:56 -05:00
9ffbadf9ed Add a check for executables in /tmp 2022-04-27 17:58:00 -05:00
4ce44caac0 Switch to Jellyfin unstable
To try to fix syncplay or whatever
2022-04-26 21:19:29 -05:00
85326fa6fc Fix incorrect allowed_hosts setting for ara 2022-04-23 11:16:12 -05:00
d483eaa07b Add ara and its related secrets 2022-04-23 10:54:26 -05:00
e91c0e7710 Move from Jackett to Prowlarr, add Lidarr 2022-04-18 21:13:01 -05:00
67f5a6565a Readd a platform for Ubuntu Jammy 2022-04-18 11:45:48 -05:00
7e6428b0e0 Revert a ton of the changes I just made because they were based on a false premise 2022-04-18 11:44:42 -05:00
73560ce6f6 Gather facts on prod_web.yml
You know that might be why my Nagios configuration fails
2022-04-18 11:34:46 -05:00
d8fe08f879 Put everything back in site.yml again
Originally, I had broken out the other site_*.yml files in an attempt to
organize the individual portions of site.yml out, as it was growing long.
But now I have a solution to its length and its consistency that doesn't
involve using multiple playbooks, so I may as well merge everything back
in again.
2022-04-18 11:23:35 -05:00
f0646fb946 Rename cleanup.yml to site_post, since it now has all its jobs 2022-04-18 11:18:34 -05:00
0d4d3235c8 Add a glob for device type configuration
Apparently these playbooks weren't being called at all, which is really concerning
2022-04-18 11:17:09 -05:00
8dc66ab53a Remove deprecated vars files 2022-04-18 11:16:02 -05:00
eaca75bc7a Add a platform for Ubuntu Jammy 2022-04-18 11:15:31 -05:00
143852c4e8 Move tags_autoreboot into cleanup.yml
This avoids it being picked up during site_pre
2022-04-18 11:15:16 -05:00
5e877ad117 Remove site_common.yml due to overlap with site_pre 2022-04-18 11:13:48 -05:00
a682edf3d3 Totally deprecate scan configuration 2022-04-18 11:13:07 -05:00
7a2c05b131 Modularize site_pre.yml 2022-04-18 11:12:48 -05:00
cfd4aa12b1 Configure the desu.ltd root A record
You can tell that the site serves basically nothing by the fact that I didn't notice this until now.
2022-04-18 11:08:07 -05:00
261cec3bc7 Switch Element to actually track a dockerhub project that exists 2022-04-14 22:41:54 -05:00
60a11942b7 Set riot-web to latest instead of develop 2022-04-14 22:39:51 -05:00
ac4fa98e2b Have gulagbot connect to its DB via DNS
Additionally, I configured a static IP for this machine
This should eliminate the circumstance where gulagbot fails to connect to its DB after a reboot
2022-04-11 12:28:43 -05:00
ce9948147f Don't gather facts on local DNS plays 2022-04-11 12:27:53 -05:00
e531f2cd1a Update pghost for gulagbot 2022-04-10 19:23:17 -05:00
a0c7caf032 Wrench down ddns-route53 update times
Since it skips all external queries if the IP hasn't changed, we can make this much tighter. The only bound is local compute power, and there's plenty of that for something this tiny.
2022-04-01 15:08:44 -05:00
af2ec7c6cd Add link to docs for ddns-route53 2022-04-01 15:02:52 -05:00
050112a62a Add a route53 DDNS client courtesy of some guy, set it up to update a record occasionally 2022-04-01 14:59:26 -05:00
82bfa04e75 Reroute traffic to media apps over personal ZT net 2022-03-30 20:59:06 -05:00
f64c0de901 Increase timeout on ansible-last-run 2022-03-26 19:57:37 -05:00
0d16877d93 Disable libreddit
I have no interest in maintaining a tool that I don't use
that attracts DMCA notices.

Fuck that.
2022-03-24 11:49:37 -05:00
5d2c207be6 Append permissions to NFS mountpoint on workstations
This fixes a lint error
2022-03-07 10:50:18 -06:00
28b34833ab Rework GitLab pipelines 2022-03-07 10:46:02 -06:00
d27554bffb Give the Nagios bullshit it's own play I guess 2022-03-04 21:29:24 -06:00
068966fec2 Reorganize and correct Ubuntu platforms 2022-02-23 21:21:19 -06:00
6707ce48bd Add automatic DNS configuration
I am amazeballs
2022-02-23 17:30:10 -06:00
2d07432a45 Remove Scanner configuration 2022-02-23 16:02:33 -06:00
4114012363 Fix incorrect Netbox media mount 2022-02-21 19:49:49 -06:00
5c7e43dd27 Bump reboot timeout
APPARENTLY
2022-02-21 19:11:29 -06:00
9caef14ee7 Exclude loopback disks in check_disk
Apparently we just now encountered an issue with those that Snap caused. Thanks, Snap.
2022-02-18 19:16:01 -06:00
3dfd06cbdb Disable a ton of fact gathering -- I anticipate this will fail at CI 2022-02-15 15:19:34 -06:00
8652b1a290 Alter gulagbot DB connection settings 2022-02-14 00:45:32 -06:00
07aee85a22 Remove references to NFS over management IP
Except on desktops, because they're supposed to do that
2022-02-13 18:29:42 -06:00
797abf0a52 Alter permission schemes across hosts to account for ZT changes 2022-02-13 13:41:20 -06:00
ba6eae9d97 Add avahi to the management network 2022-02-13 13:03:41 -06:00
f86d065005 Add ZT management-only network 2022-02-13 11:05:25 -06:00
5efeaa18e9 Polish up checks 2022-02-13 10:11:45 -06:00
eeb1afb31b Revert "Lock gulagbot to 4.1.8"
This reverts commit 4be2e313fd.
2022-02-11 13:57:31 -06:00
4be2e313fd Lock gulagbot to 4.1.8
Awaiting resolution of some bugs
2022-02-10 23:52:36 -06:00
d48f04dbe0 Update Forge to 36.2.26
This fixes a bug with Forge with a missing class on latest Java
2022-02-09 17:16:49 -06:00
750f7cad15 Change some Stalin settings 2022-02-04 23:04:36 -06:00
cd805fa79f Purge mainline from framework laptop 2022-02-03 13:28:54 -06:00
0ed99bec3e Re-add mem_sleep_default to lap-fw-diy-1's kernel params
This reverts commit f345e64b79.

Apparently my issue was with the out-of-tree kernel, not this particular
configuration. After some testing, I don't need the new kernel anyway.
2022-02-03 13:25:44 -06:00
6c5bea19f1 Add snd-hda-intel args to kernel cmdline
I encountered a bug with the new kernels which appears to have caused my audio to dip out.
2022-02-03 10:29:18 -06:00
f345e64b79 Revert "Add mem_sleep_default to lap-fw-diy-1's kernel params"
This reverts commit 3c632ce64c.

I'm experirencing serious issues with this mode of sleep, so disabling
it for now.
2022-02-03 08:58:27 -06:00
0963554422 Add mainline kernel update utility
>=5.14 is required for the Framework to function fully. Since this is just one machine, I'm okay with not completely automating this process.
2022-02-03 08:10:00 -06:00
50add7b8b9 Disable i915.enable_psr= (Re-enable PSR)
This comes in tandem with the next commit, which adds a utility for using the mainline kernel
2022-02-03 08:09:30 -06:00
3c632ce64c Add mem_sleep_default to lap-fw-diy-1's kernel params
Apparently, it doesn't suspend to ram at the hardware level by default, which is strange and dumb.
2022-02-03 07:57:21 -06:00
46a567c46a Make memory checks a lot tighter
Alright look, I design my systems with 10% overhead, not 20%
2022-02-02 07:41:22 -06:00
714e78b4d7 Test an older version of the E6 pack with a different set of performance mods
I've heard Performant is mostly placebo, and we're finally at a point where we can thoroughly test and see how it runs compared to the suite of performance mods shipped by the pack normally.
2022-01-29 21:17:28 -06:00
c49d4cefad Provision a new mc serber 2022-01-28 18:37:07 -06:00
aef8f31c87 Change Nagios Matrix channel 2022-01-27 14:31:58 -06:00
662d94a948 Fix incorrect group for Zerotier checks 2022-01-24 18:07:53 -06:00
3810b96a38 Rework Zerotier role to allow for arbitrary adds and deletes 2022-01-23 17:17:43 -06:00
53ffaf52c4 Reorganize playbooks to modularize Zerotier enrollment 2022-01-23 16:37:56 -06:00
964d3228e2 Add some more correct settings for jenkins 2022-01-23 08:47:26 -06:00
c8793cc240 Recalibrate Stalin to be less... totalitarian 2022-01-22 19:30:35 -06:00
5e9c137b34 Decom peertube, add jenkins 2022-01-22 09:48:47 -06:00
380781b1eb Re-enable NFS mountpoint for desktops 2022-01-21 22:24:03 -06:00
269a7ec784 Enroll key for lap-fw-diy-1, give lap-s76-lemp9-1 some nicer Grub configs for dualboot 2022-01-21 17:18:56 -06:00
1a4f59b71a Add a new laptop, rebase zerotier repo 2022-01-21 16:55:10 -06:00
e4805658dc Spin up a Terraria server 2022-01-12 15:14:30 -06:00
8984f71aee Upgrade Netbox 2022-01-12 14:23:26 -06:00
fe00d12377 Add some opt-in docker checks 2022-01-11 18:30:27 -06:00
fb465a666e Add temperature sensors to Pis 2022-01-11 00:16:54 -06:00
4a1c5ea8e1 Use new-style checks for the temperature command 2022-01-11 00:00:31 -06:00
0140784111 Add hostgroup for device types 2022-01-10 23:54:03 -06:00
8517e842b2 Add checks for the R720's thermal monitors, also add those thermal monitors 2022-01-10 22:14:07 -06:00
8a3f13a939 Give ansible play check a bit more wiggle room, missing a single day isn't that bad 2022-01-09 11:32:16 -06:00
c59781b47e Define a timeperiod for Nagios and assign it to our matrix notify user 2022-01-04 18:00:55 -06:00
e0b385a1c5 Fix accidentally removing a ton of checks from each host 2022-01-03 13:18:14 -06:00
e2d738ba40 Enroll vm-scan-1 in backups 2021-12-30 10:37:50 -06:00
9662bb0ff8 Ensure we actually add our new user to sudoers 2021-12-30 10:12:44 -06:00
111f1cdef6 Configure a scanner user on all machines 2021-12-30 10:04:31 -06:00
1cff8a6aa8 Deploy GVM to a box at home 2021-12-30 09:34:45 -06:00
ed64fc0a9a Backup etc on desktops 2021-12-30 07:48:15 -06:00
db78f7eaf6 Separate HOSTALIAS from SERVICEDESC more properly on Nagios 2021-12-26 22:56:03 -06:00
8612eec85e Create an Ansible contactgroup in Nagios and tie it to all alerts, enroll our Matrix user in that group 2021-12-24 16:56:06 -06:00
1791c40465 Working on Matrix integration for Nagios 2021-12-24 16:47:21 -06:00
c6c57fce6c Change alert destination email address 2021-12-24 15:36:18 -06:00
09f33966ac Disable memory checks for machines running ZFS
I give up. I'll circle back on this later
2021-12-24 15:32:53 -06:00
aa493348d3 Add another Minecraft server and some related checks 2021-12-24 14:53:09 -06:00
22863e66e7 Upgrade Nextcloud 22 -> 23 2021-12-24 12:16:44 -06:00
d7c3f97797 Set up ddclient 2021-12-21 11:16:40 -06:00
5e7b8bb881 Add a Minecraft server *at home* 2021-12-20 17:24:11 -06:00
9b64cf8a00 Modularize sanitization cronjobs 2021-12-16 08:11:17 -06:00
6b218b02f9 Add a cronjob to Syncthing to clean up :Zone.Identifier files 2021-12-16 07:44:01 -06:00
911d236c84 Implement a sanitize rule for syncthing 2021-12-15 22:19:05 -06:00
060aa14df3 Fix incorrect dir for cp2077 screenshots 2021-12-15 21:12:24 -06:00
e93124e556 Add more directories to sort out, make the jobs run in parallel at 5AM 2021-12-15 21:10:20 -06:00
58196e3f24 Genericize that cronjob syntax for future endeavors 2021-12-15 20:56:45 -06:00
640e2e0efe Add a cronjob for a specific bug I'm working around with Syncthing 2021-12-15 20:41:14 -06:00
5031833f39 Remove Package Updates check
It's just pointless noise to be honest, it's way too loud. Perhaps a proper patch management solution would be in order?
2021-12-15 20:06:12 -06:00
72697a3953 Move check_disk to those restricted checks, also exclude AppImage loopback mounts 2021-12-15 19:57:20 -06:00
54a4f1539b Add some sudo rules to nagios-checker so it can start doing restricted checks 2021-12-15 19:57:08 -06:00
4b626dc6be Implement communication with Nagios when rebooting boxes
One step closer to that full automation goal
2021-12-15 19:32:19 -06:00
000d711d7a Update gulagbot DB IP 2021-12-12 09:54:15 -06:00
31018efeb1 Expose Jellyfin over 192.168.* 2021-12-11 21:50:20 -06:00
e0ce07c4dc Restart Jellyfin unless stopped 2021-12-11 16:01:21 -06:00
9aab2d6557 Tune the transaction limits for that check we just added 2021-12-09 16:23:59 -06:00
273d83be64 Add a check for old, uncommited PostgreSQL transactions
Sometimes reading the blogs of developers whose software you use is worth it
2021-12-09 16:17:49 -06:00
fcffd834a0 Move Nagios into its own role
It was getting way too big
2021-12-08 21:34:32 -06:00
a71071b321 Spin up a SL server 2021-12-01 22:34:46 -06:00
386b190130 Add vm-desktop-1 to list of workstations 2021-12-01 07:31:38 -06:00
e85d81ef38 Drop logs for lr.cowfee.moe 2021-11-30 14:11:27 -06:00
558709ce6f Deploy libreddit 2021-11-29 23:33:56 -06:00
fab7be68c5 Tune thresholds for monitoring the age of ansible-last-run 2021-11-29 22:15:09 -06:00
1952f72c89 Add a check for the last ansible run on a given machine 2021-11-25 16:41:17 -06:00
5b12eb5af2 Add a cleanup task to touch a file upon completion of site.yml
This playbook *should* assure that we have a file we can use for checking when the last full play was. It being in a playbook at the tail end of site.yml is paramount, since site.yml dying will cause alarms to be set off.
2021-11-25 16:19:01 -06:00
ce37a7fec3 Rename a bunch of minecraft tasks to prevent ambiguity 2021-11-25 13:30:21 -06:00
6c4b1c701b Fix some unquoted number variables causing the gulagbot task to fail 2021-11-25 13:20:23 -06:00
84cd7888f1 Shut down hexxit2 2021-11-25 13:19:45 -06:00
12f33d9ffc Put Syncthing behind an ingress container 2021-11-24 10:43:29 -06:00
3d9ec54467 Nevermind, guess we scope it out even *higher* 2021-11-24 07:43:18 -06:00
733d1006be Adjust mountpoint for NFS to allow for access to higher dirs like syncthing 2021-11-24 07:41:05 -06:00
811d0bd2d2 Add some params for a new version of gulagbot 2021-11-22 14:12:13 -06:00
decd4b452b Add transaction logging to home DB 2021-11-21 19:57:33 -06:00
1c4bf65db4 Add a test DB for gulagbot 2021-11-21 17:01:58 -06:00
d67bc370ac Split nagios-checkhttp into nagios-checkhttp{,s} 2021-11-21 13:59:05 -06:00
7976630ad7 Add a box for Syncthing 2021-11-20 18:29:31 -06:00
ee5f8ffe92 Make a box to move Stalin back home where he belongs 2021-11-19 20:38:20 -06:00
cff68a2a73 Reorganize Jellyfin to reflect usage of Nvidia Container Toolkit, configure hw accel 2021-11-18 19:16:30 -06:00
05a7f19bfe Update backup dirs for PMX to reflect some PCIe passthrough stuff I'm doing 2021-11-18 16:51:32 -06:00
9680705689 Update NFS mounts for desktops 2021-11-15 11:07:11 -06:00
bca5c1993d Add a bunch of unit checks for Proxmox boxes 2021-11-11 14:00:25 -06:00
5794379da4 Add some backup configuration for PMX hosts 2021-11-11 13:54:11 -06:00
77084ebc49 Reorganize media dirs, add jellyfin to vm-media-1 2021-11-11 00:23:53 -06:00
b0b71abf6a Reorganize NFS mounts so that they don't contain SQLite DBs 2021-11-10 19:29:06 -06:00
57b1cf03ca Set nfs to rw on local connections 2021-11-10 19:14:02 -06:00
7f7a0fd2ba Run setup tasks before roles on vm-media-1 2021-11-10 19:11:10 -06:00
f3b12234c0 Start reorganizing to have pi-media-1 split into vms 2021-11-09 20:53:59 -06:00