the great migration from truenas to fedora and all its collatoral
All checks were successful
Reese's Arch Toolbox / build-and-push-arch-toolbox (push) Successful in 24m47s

This commit is contained in:
2025-04-08 12:40:42 -04:00
parent 9a3382862d
commit 9417e711a9
54 changed files with 1533 additions and 519 deletions

View File

@@ -1,22 +1,4 @@
arch:
hosts:
gamebox:
fedora:
hosts:
3dserver:
nextcloud:
kube:
yellow:
ubuntu:
hosts:
unifi-external:
raspbian:
hosts:
pivpn:
unmanaged:
hosts:
driveripper:
homeassistant:

View File

@@ -0,0 +1,12 @@
# AWS CLI
## Install
```bash
# Run as root
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
unzip awscliv2.zip && \
./aws/install && \
rm -f ./awscliv2.zip && \
rm -rf ./aws
```

View File

@@ -1,17 +1,24 @@
# AWS Credentials
Note: this requires the AWS CLI. See [AWS CLI](/cloud/graduated/aws_cli/aws_cli.md)
## Credential Generation
```bash
export AWS_USERNAME=
aws iam create-user --user-name $AWS_USERNAME
aws iam create-access-key --user-name $AWS_USERNAME
# Allow updating reeseapps
aws iam attach-user-policy --user-name $AWS_USERNAME --policy-arn $(cat cloud/graduated/aws_iam/secrets/update-reeseapps-iam-policy-arn)
# Allow updating reeselink
aws iam attach-user-policy --user-name $AWS_USERNAME --policy-arn $(cat cloud/graduated/aws_iam/secrets/update-reeselink-iam-policy-arn)
# Allow sending emails
aws iam attach-user-policy --user-name $AWS_USERNAME --policy-arn $(cat cloud/graduated/aws_iam/secrets/ses-send-email-policy-arn)
# Create credentials (run aws configure on the machine that needs these to input them manually)
aws iam create-access-key --user-name $AWS_USERNAME
```
## AWS Certbot Route53 Policies
@@ -54,4 +61,7 @@ aws iam create-policy --policy-name update-reeselink --policy-document file://cl
# Allow updating route53 records for reeseapps.com
aws iam create-policy --policy-name update-reeseapps --policy-document file://cloud/graduated/aws_iam/secrets/route53_reeseapps_policy.json
# Allow sending emails
aws iam create-policy --policy-name send-email --policy-document file://cloud/graduated/aws_iam/secrets/ses_allow_send_policy.json
```

View File

@@ -0,0 +1,240 @@
## Laptop Specs
Operating System: Fedora Linux 41
KDE Plasma Version: 6.3.4
KDE Frameworks Version: 6.12.0
Qt Version: 6.8.2
Kernel Version: 6.13.9-200.fc41.x86_64 (64-bit)
Graphics Platform: Wayland
Processors: 16 × AMD Ryzen 9 7940HS w/ Radeon 780M Graphics
Memory: 98.9 GB of RAM
Graphics Processor: AMD Radeon 780M
Manufacturer: Framework
Product Name: Laptop 16 (AMD Ryzen 7040 Series)
System Version: AJ
## The Problem
Hey everyone, I've had no luck searching for this issue online. I might be looking in the wrong places so please point me to existing posts if there's already a topic in flight on this, or another, forum.
My Framework 16's dGPU disconnects after waking from sleep. It won't show up in monitoring software (Mission Center from Flatpak) or MangoHud in games. It's also not being detected by steam games or ollama - both will only detect and use the internal gpu. I'm pretty sure it shows up in `lscpi` (shown below in dGPU disconnected state).
No bios settings have made a difference. This happens both on-battery and charging with the framework charger and a 240 watt Delta charger. Power profiles don't make a difference either.
I have secure boot on. I don't use hibernate. I have a couple custom udev rules that prevent usb devices from waking the laptop in sleep (shown below).
Looking for anything to try, thanks for the help!
## Details
### lspci
```bash
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Root Complex
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Phoenix IOMMU
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Dummy Host Bridge
00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Phoenix GPP Bridge
00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Dummy Host Bridge
00:02.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Phoenix GPP Bridge
00:02.4 PCI bridge: Advanced Micro Devices, Inc. [AMD] Phoenix GPP Bridge
00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Dummy Host Bridge
00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 19h USB4/Thunderbolt PCIe tunnel
00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Dummy Host Bridge
00:04.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 19h USB4/Thunderbolt PCIe tunnel
00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Dummy Host Bridge
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Internal GPP Bridge to Bus [C:A]
00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Internal GPP Bridge to Bus [C:A]
00:08.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Internal GPP Bridge to Bus [C:A]
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 71)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Data Fabric; Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Data Fabric; Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Data Fabric; Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Data Fabric; Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Data Fabric; Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Data Fabric; Function 5
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Data Fabric; Function 6
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Phoenix Data Fabric; Function 7
01:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch (rev 12)
02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch (rev 12)
03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 33 [Radeon RX 7600/7600 XT/7600M XT/7600S/7700S / PRO W7600] (rev c1)
03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 HDMI/DP Audio
04:00.0 Network controller: Intel Corporation Wi-Fi 6E(802.11ax) AX210/AX1675* 2x2 [Typhoon Peak] (rev 1a)
05:00.0 Non-Volatile memory controller: Seagate Technology PLC E18 PCIe SSD (rev 01)
c4:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Phoenix1 (rev c1)
c4:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller
c4:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Phoenix CCP/PSP 3.0 Device
c4:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15b9
c4:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15ba
c4:00.5 Multimedia controller: Advanced Micro Devices, Inc. [AMD] ACP/ACP3X/ACP6x Audio Coprocessor (rev 63)
c4:00.6 Audio device: Advanced Micro Devices, Inc. [AMD] Family 17h/19h/1ah HD Audio Controller
c5:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Phoenix Dummy Function
c5:00.1 Signal processing controller: Advanced Micro Devices, Inc. [AMD] AMD IPU Device
c6:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Phoenix Dummy Function
c6:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15c0
c6:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Device 15c1
c6:00.5 USB controller: Advanced Micro Devices, Inc. [AMD] Pink Sardine USB4/Thunderbolt NHI controller #1
c6:00.6 USB controller: Advanced Micro Devices, Inc. [AMD] Pink Sardine USB4/Thunderbolt NHI controller #2
```
### UDev Rules
```conf
ACTION=="add", SUBSYSTEM=="acpi", DRIVERS=="button", ATTRS{hid}=="PNP0C0D", ATTR{power/wakeup}="disabled"
ACTION=="add", SUBSYSTEM=="serio", DRIVERS=="atkbd", ATTR{power/wakeup}="disabled"
ACTION=="add", SUBSYSTEM=="i2c", DRIVERS=="i2c_hid_acpi", ATTRS{name}=="PIXA3854:00", ATTR{power/wakeup}="disabled"
ACTION=="add", SUBSYSTEM=="usb", DRIVERS=="usb", ATTR{power/wakeup}="disabled"
```
### Dmesg
```bash
[27962.377892] CPU: 11 UID: 0 PID: 187288 Comm: kworker/11:6 Tainted: G W 6.13.9-200.fc41.x86_64 #1
[27962.377896] Tainted: [W]=WARN
[27962.377898] Hardware name: Framework Laptop 16 (AMD Ryzen 7040 Series)/FRANMZCP09, BIOS 03.05 11/13/2024
[27962.377901] Workqueue: pm pm_runtime_work
[27962.377906] RIP: 0010:amdgpu_irq_put+0x46/0x70 [amdgpu]
[27962.378233] Code: c0 74 33 48 8b 4e 10 48 83 39 00 74 29 89 d1 48 8d 04 88 8b 08 85 c9 74 11 f0 ff 08 74 07 31 c0 e9 0a a3 c6 fa e9 1a fd ff ff <0f> 0b b8 ea ff ff ff e9 f9 a2 c6 fa b8 ea ff ff ff e9 ef a2 c6 fa
[27962.378237] RSP: 0018:ffffb598a9667c98 EFLAGS: 00010246
[27962.378241] RAX: ffff9a9fe45a0ea8 RBX: ffff9a9fdb880000 RCX: 0000000000000000
[27962.378244] RDX: 0000000000000000 RSI: ffff9a9fdb8a5560 RDI: ffff9a9fdb880000
[27962.378246] RBP: ffff9a9fdb8c55d0 R08: 0000000000000000 R09: 0000000001195b5a
[27962.378249] R10: ffffb598a9667c48 R11: 0000000000000000 R12: 0000000000000006
[27962.378251] R13: ffff9a9fdb880000 R14: 0000000000000001 R15: ffff9aa5dcb2b040
[27962.378254] FS: 0000000000000000(0000) GS:ffff9ab67ff80000(0000) knlGS:0000000000000000
[27962.378257] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[27962.378260] CR2: 00001f4402539014 CR3: 000000161c82c000 CR4: 0000000000f50ef0
[27962.378263] PKRU: 55555554
[27962.378265] Call Trace:
[27962.378268] <TASK>
[27962.378270] ? srso_alias_return_thunk+0x5/0xfbef5
[27962.378274] ? show_trace_log_lvl+0x255/0x2f0
[27962.378280] ? show_trace_log_lvl+0x255/0x2f0
[27962.378289] ? gfx_v11_0_hw_fini+0x41/0xf0 [amdgpu]
[27962.378597] ? amdgpu_irq_put+0x46/0x70 [amdgpu]
[27962.378906] ? __warn.cold+0x93/0xfa
[27962.378912] ? amdgpu_irq_put+0x46/0x70 [amdgpu]
[27962.379216] ? report_bug+0xff/0x140
[27962.379222] ? handle_bug+0x58/0x90
[27962.379226] ? exc_invalid_op+0x17/0x70
[27962.379230] ? asm_exc_invalid_op+0x1a/0x20
[27962.379239] ? amdgpu_irq_put+0x46/0x70 [amdgpu]
[27962.379549] ? srso_alias_return_thunk+0x5/0xfbef5
[27962.379553] gfx_v11_0_hw_fini+0x41/0xf0 [amdgpu]
[27962.379879] gfx_v11_0_suspend+0xe/0x20 [amdgpu]
[27962.380210] amdgpu_ip_block_suspend+0x24/0x40 [amdgpu]
[27962.380523] amdgpu_device_ip_suspend_phase2+0x125/0x340 [amdgpu]
[27962.380830] amdgpu_device_suspend+0xcf/0x170 [amdgpu]
[27962.381154] amdgpu_pmops_runtime_suspend+0xb9/0x1a0 [amdgpu]
[27962.381488] pci_pm_runtime_suspend+0x67/0x1a0
[27962.381494] ? __pfx_pci_pm_runtime_suspend+0x10/0x10
[27962.381499] __rpm_callback+0x41/0x170
[27962.381503] ? __pfx_pci_pm_runtime_suspend+0x10/0x10
[27962.381508] rpm_callback+0x55/0x60
[27962.381512] ? __pfx_pci_pm_runtime_suspend+0x10/0x10
[27962.381516] rpm_suspend+0xe6/0x5f0
[27962.381520] ? srso_alias_return_thunk+0x5/0xfbef5
[27962.381523] ? finish_task_switch.isra.0+0x99/0x2c0
[27962.381531] pm_runtime_work+0x98/0xb0
[27962.381535] process_one_work+0x176/0x330
[27962.381541] worker_thread+0x252/0x390
[27962.381547] ? __pfx_worker_thread+0x10/0x10
[27962.381551] kthread+0xcf/0x100
[27962.381557] ? __pfx_kthread+0x10/0x10
[27962.381562] ret_from_fork+0x31/0x50
[27962.381568] ? __pfx_kthread+0x10/0x10
[27962.381573] ret_from_fork_asm+0x1a/0x30
[27962.381583] </TASK>
[27962.381587] ---[ end trace 0000000000000000 ]---
[27965.093916] amdgpu 0000:03:00.0: amdgpu: MES failed to respond to msg=REMOVE_QUEUE
[27965.093923] [drm:amdgpu_mes_unmap_legacy_queue [amdgpu]] *ERROR* failed to unmap legacy queue
[27967.860541] amdgpu 0000:03:00.0: amdgpu: MES failed to respond to msg=REMOVE_QUEUE
[27967.860554] [drm:amdgpu_mes_unmap_legacy_queue [amdgpu]] *ERROR* failed to unmap legacy queue
[27970.792292] amdgpu 0000:03:00.0: amdgpu: MES failed to respond to msg=REMOVE_QUEUE
[27970.792306] [drm:amdgpu_mes_unmap_legacy_queue [amdgpu]] *ERROR* failed to unmap legacy queue
[27973.717776] amdgpu 0000:03:00.0: amdgpu: MES failed to respond to msg=REMOVE_QUEUE
[27973.717790] [drm:amdgpu_mes_unmap_legacy_queue [amdgpu]] *ERROR* failed to unmap legacy queue
[27976.642661] amdgpu 0000:03:00.0: amdgpu: MES failed to respond to msg=REMOVE_QUEUE
[27976.642670] [drm:amdgpu_mes_unmap_legacy_queue [amdgpu]] *ERROR* failed to unmap legacy queue
[27979.503724] amdgpu 0000:03:00.0: amdgpu: MES failed to respond to msg=REMOVE_QUEUE
[27979.503737] [drm:amdgpu_mes_unmap_legacy_queue [amdgpu]] *ERROR* failed to unmap legacy queue
[27982.355092] amdgpu 0000:03:00.0: amdgpu: MES failed to respond to msg=REMOVE_QUEUE
[27982.355102] [drm:amdgpu_mes_unmap_legacy_queue [amdgpu]] *ERROR* failed to unmap legacy queue
[27985.020353] amdgpu 0000:03:00.0: amdgpu: MES failed to respond to msg=REMOVE_QUEUE
[27985.020365] [drm:amdgpu_mes_unmap_legacy_queue [amdgpu]] *ERROR* failed to unmap legacy queue
[27987.689301] amdgpu 0000:03:00.0: amdgpu: MES failed to respond to msg=REMOVE_QUEUE
[27987.689308] [drm:amdgpu_mes_unmap_legacy_queue [amdgpu]] *ERROR* failed to unmap legacy queue
[27987.916382] [drm:gfx_v11_0_hw_fini [amdgpu]] *ERROR* failed to halt cp gfx
[27987.916872] ------------[ cut here ]------------
[27987.916873] WARNING: CPU: 11 PID: 187288 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:631 amdgpu_irq_put+0x46/0x70 [amdgpu]
[27987.917063] Modules linked in: ib_core uinput overlay rfcomm snd_seq_dummy snd_hrtimer nft_reject_ipv6 nft_masq nft_reject_ipv4 act_csum cls_u32 sch_htb nf_nat_tftp nf_conntrack_tftp bridge stp llc nf_conntrack_netbios_ns nf_conntrack_broadcast nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables qrtr uhid bnep snd_hda_codec_realtek snd_hda_codec_generic snd_hda_scodec_component snd_hda_codec_hdmi sunrpc binfmt_misc vfat fat snd_sof_amd_acp70 snd_sof_amd_acp63 snd_sof_amd_vangogh snd_sof_amd_rembrandt snd_sof_amd_renoir snd_sof_amd_acp snd_sof_pci iwlmvm snd_sof_xtensa_dsp snd_sof cdc_mbim cdc_wdm cdc_ncm cdc_ether usbnet snd_sof_utils mac80211 snd_pci_ps snd_soc_acpi_amd_match snd_amd_sdw_acpi soundwire_amd soundwire_generic_allocation soundwire_bus snd_soc_sdca amd_atl intel_rapl_msr intel_rapl_common snd_soc_core libarc4 snd_hda_intel edac_mce_amd snd_intel_dspcfg snd_compress
[27987.917127] cros_usbpd_charger leds_cros_ec gpio_cros_ec cros_charge_control led_class_multicolor cros_ec_chardev ac97_bus cros_ec_hwmon cros_ec_sysfs cros_usbpd_logger cros_usbpd_notify snd_intel_sdw_acpi snd_pcm_dmaengine snd_hda_codec kvm_amd hid_sensor_als snd_rpl_pci_acp6x snd_acp_pci hid_sensor_trigger cros_ec_dev snd_hda_core hid_sensor_iio_common snd_acp_legacy_common kvm btusb iwlwifi industrialio_triggered_buffer snd_pci_acp6x btrtl btintel snd_hwdep kfifo_buf btbcm snd_seq btmtk cros_ec_lpcs spd5118 cros_ec snd_seq_device industrialio rapl cfg80211 wmi_bmof bluetooth snd_pcm snd_pci_acp5x pcspkr snd_rn_pci_acp3x snd_timer r8152 snd_acp_config thunderbolt mii k10temp snd_soc_acpi i2c_piix4 snd rfkill i2c_smbus snd_pci_acp3x amd_pmf soundcore amdtee joydev amd_sfh tee platform_profile amd_pmc loop nfnetlink zram lz4hc_compress lz4_compress dm_crypt hid_logitech_hidpp hid_logitech_dj typec_displayport amdgpu amdxcp drm_exec gpu_sched drm_panel_backlight_quirks drm_buddy drm_ttm_helper ttm nvme i2c_algo_bit
[27987.917196] drm_suballoc_helper crct10dif_pclmul crc32_pclmul drm_display_helper crc32c_intel nvme_core polyval_clmulni polyval_generic ghash_clmulni_intel video hid_multitouch sha512_ssse3 ucsi_acpi hid_sensor_hub typec_ucsi sha256_ssse3 sha1_ssse3 cec sp5100_tco typec nvme_auth wmi i2c_hid_acpi i2c_hid fuse i2c_dev
[27987.917222] CPU: 11 UID: 0 PID: 187288 Comm: kworker/11:6 Tainted: G W 6.13.9-200.fc41.x86_64 #1
[27987.917226] Tainted: [W]=WARN
[27987.917227] Hardware name: Framework Laptop 16 (AMD Ryzen 7040 Series)/FRANMZCP09, BIOS 03.05 11/13/2024
[27987.917229] Workqueue: pm pm_runtime_work
[27987.917235] RIP: 0010:amdgpu_irq_put+0x46/0x70 [amdgpu]
[27987.917403] Code: c0 74 33 48 8b 4e 10 48 83 39 00 74 29 89 d1 48 8d 04 88 8b 08 85 c9 74 11 f0 ff 08 74 07 31 c0 e9 0a a3 c6 fa e9 1a fd ff ff <0f> 0b b8 ea ff ff ff e9 f9 a2 c6 fa b8 ea ff ff ff e9 ef a2 c6 fa
[27987.917405] RSP: 0018:ffffb598a9667c68 EFLAGS: 00010246
[27987.917408] RAX: ffff9a9fe45a0508 RBX: ffff9a9fc7c27000 RCX: 0000000000000000
[27987.917410] RDX: 0000000000000000 RSI: ffff9a9fc7c27008 RDI: ffff9a9fdb880000
[27987.917411] RBP: ffff9a9fdb880000 R08: ffffffffbc873d28 R09: ffff9a9fc04020e8
[27987.917412] R10: ffffffffffffffff R11: 0000000000000000 R12: 0000000000000004
[27987.917413] R13: ffff9a9fdb880000 R14: 0000000000000001 R15: ffff9aa5dcb2b040
[27987.917415] FS: 0000000000000000(0000) GS:ffff9ab67ff80000(0000) knlGS:0000000000000000
[27987.917416] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[27987.917418] CR2: 00001f4402539014 CR3: 000000161c82c000 CR4: 0000000000f50ef0
[27987.917419] PKRU: 55555554
[27987.917420] Call Trace:
[27987.917423] <TASK>
[27987.917425] ? srso_alias_return_thunk+0x5/0xfbef5
[27987.917429] ? show_trace_log_lvl+0x255/0x2f0
[27987.917435] ? show_trace_log_lvl+0x255/0x2f0
[27987.917439] ? smu_smc_hw_cleanup+0x68/0xa0 [amdgpu]
[27987.917633] ? amdgpu_irq_put+0x46/0x70 [amdgpu]
[27987.917785] ? __warn.cold+0x93/0xfa
[27987.917789] ? amdgpu_irq_put+0x46/0x70 [amdgpu]
[27987.917930] ? report_bug+0xff/0x140
[27987.917934] ? handle_bug+0x58/0x90
[27987.917937] ? exc_invalid_op+0x17/0x70
[27987.917939] ? asm_exc_invalid_op+0x1a/0x20
[27987.917945] ? amdgpu_irq_put+0x46/0x70 [amdgpu]
[27987.918084] ? srso_alias_return_thunk+0x5/0xfbef5
[27987.918087] smu_smc_hw_cleanup+0x68/0xa0 [amdgpu]
[27987.918231] smu_suspend+0x77/0xe0 [amdgpu]
[27987.918344] amdgpu_ip_block_suspend+0x24/0x40 [amdgpu]
[27987.918461] amdgpu_device_ip_suspend_phase2+0x125/0x340 [amdgpu]
[27987.918568] amdgpu_device_suspend+0xcf/0x170 [amdgpu]
[27987.918674] amdgpu_pmops_runtime_suspend+0xb9/0x1a0 [amdgpu]
[27987.918778] pci_pm_runtime_suspend+0x67/0x1a0
[27987.918782] ? __pfx_pci_pm_runtime_suspend+0x10/0x10
[27987.918784] __rpm_callback+0x41/0x170
[27987.918787] ? __pfx_pci_pm_runtime_suspend+0x10/0x10
[27987.918790] rpm_callback+0x55/0x60
[27987.918792] ? __pfx_pci_pm_runtime_suspend+0x10/0x10
[27987.918794] rpm_suspend+0xe6/0x5f0
[27987.918795] ? srso_alias_return_thunk+0x5/0xfbef5
[27987.918797] ? finish_task_switch.isra.0+0x99/0x2c0
[27987.918802] pm_runtime_work+0x98/0xb0
[27987.918805] process_one_work+0x176/0x330
[27987.918808] worker_thread+0x252/0x390
[27987.918811] ? __pfx_worker_thread+0x10/0x10
[27987.918813] kthread+0xcf/0x100
[27987.918816] ? __pfx_kthread+0x10/0x10
[27987.918818] ret_from_fork+0x31/0x50
[27987.918821] ? __pfx_kthread+0x10/0x10
[27987.918823] ret_from_fork_asm+0x1a/0x30
[27987.918828] </TASK>
[27987.918829] ---[ end trace 0000000000000000 ]---
[27987.918833] amdgpu 0000:03:00.0: amdgpu: Fail to disable thermal alert!
[27987.918836] amdgpu 0000:03:00.0: amdgpu: suspend of IP block <smu> failed -22
[27987.918851] amdgpu 0000:03:00.0: amdgpu: SMU: response:0xFFFFFFFF for index:46 param:0x00000000 message:PrepareMp1ForUnload?
[27987.918853] amdgpu 0000:03:00.0: amdgpu: [PrepareMp1] Failed!
[27987.918855] [drm:amdgpu_device_ip_suspend_phase2 [amdgpu]] *ERROR* SMC failed to set mp1 state 2, -121
[28270.864511] amdgpu 0000:03:00.0: amdgpu: PSP is resuming...
[28271.140289] amdgpu 0000:03:00.0: amdgpu: PSP create ring failed!
[28271.140299] amdgpu 0000:03:00.0: amdgpu: PSP resume failed
[28271.140303] amdgpu 0000:03:00.0: amdgpu: resume of IP block <psp> failed -62
[28271.140309] amdgpu 0000:03:00.0: amdgpu: amdgpu_device_ip_resume failed (-62).
```

View File

@@ -0,0 +1 @@
# Framework Laptop 16

View File

@@ -22,6 +22,9 @@
5. Press and hold the zwave button until the light turns solid red, release and it should flash red
1. (OR) Enter programming pin on lock -> 0 (this may take a few attempts, don't click the pair button)
If the lock ever disconnects you can safely delete it from home assistant and re-interview. It will
set back up with the correct entity IDs and automations/dashboards will work just fine.
### Philips Hue Lights
1. I configure all philips hue lights through zigbee directly connected to HA

View File

@@ -10,10 +10,10 @@ COMMAND_PREFIX=""
export DEBUG_TAG="debug-$(date +"%s")"
docker image pull docker.io/archlinux:latest
podman image pull podman.io/archlinux:latest
# Run the build for the CPU image
docker build \
podman build \
-t gitea.reeseapps.com/services/arch-toolbox:$DEBUG_TAG \
-f ./infrastructure/graduated/distoolbox/arch-toolbox.containerfile \
--target cpu \
@@ -22,7 +22,7 @@ docker build \
./infrastructure/graduated/distoolbox
# Run the build for the AMD gpu image
docker build \
podman build \
-t gitea.reeseapps.com/services/arch-toolbox-amdgpu:$DEBUG_TAG \
-f ./infrastructure/graduated/distoolbox/arch-toolbox.containerfile \
--target amdgpu \

View File

@@ -26,7 +26,7 @@ RUN pacman -Syu --noconfirm
# Install a variety of commonly used tools and utilities using Pacman.
RUN pacman -S --noconfirm \
# A powerful shell with syntax highlighting, autosuggestions, and more.
zsh grml-zsh-config zsh-syntax-highlighting zsh-autosuggestions \
zsh \
# Utility to find which packages own files or directories in the system.
pkgfile \
# Advanced text editor for code editing and other tasks.
@@ -106,12 +106,18 @@ RUN pacman -S --noconfirm \
# Reattach to running processes
reptyr \
# Netcat, for basic tcp/udp operations
openbsd-netcat
openbsd-netcat \
# 7zip support
7zip
########################
##### Extra Apps #####
########################
# Install oh-my-zsh
RUN sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"
# Install AWS CLI version 2.
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
unzip -qq awscliv2.zip && \
@@ -127,9 +133,6 @@ RUN curl -L https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -o
##### COPIES #####
####################
# Copy the zshrc.local configuration file to the container.
COPY arch-toolbox-supporting-files/zshrc /etc/zsh/zshrc.local
# Copy tmux.conf to configure tmux in the container.
COPY arch-toolbox-supporting-files/arch-toolbox-tmux.conf /etc/tmux.conf

View File

@@ -9,35 +9,10 @@ Distrobox? Toolbox? Whatever you want.
- [Aliases using Distrobox](#aliases-using-distrobox)
- [Aliases using Toolbox](#aliases-using-toolbox)
- [Building Reese's Arch Toolbox](#building-reeses-arch-toolbox)
- [Distrobox Notes](#distrobox-notes)
## Reese's Arch Distoolbox
I have a custom arch image based on the default arch-toolbox image. It offers:
- zsh with many completions installed/enabled
- vim
- nslookup
- iperf3
- kubectl
- helm
- nethogs
- python, pip, and pipx
- ansible
- aws cli
- podman (connected automatically to the host machine via the unix socket)
- tmux
- ffmpeg
- wine
- podman
- unzip
- bat
- btop
- jq
- yq
- imagemagick
- code
- make, gcc
### Using Reese's Arch Toolbox
Head to <https://gitea.reeseapps.com/services/-/packages> and pick the CPU
@@ -164,3 +139,12 @@ In vscode you can set this as your default build task for homelab and trigger it
]
}
```
## Distrobox Notes
<https://github.com/89luca89/distrobox/blob/main/docs/usage/distrobox-create.md>
```bash
# --init creates a separate systemd environment. You won't be able to access the system's host processes.
distrobox create -i docker.io/library/debian --name test-debian
```

View File

@@ -10,6 +10,7 @@
- [Network](#network)
- [Hostname](#hostname)
- [VLAN Setup with nmcli](#vlan-setup-with-nmcli)
- [GPU Support in Distrobox](#gpu-support-in-distrobox)
## TPM2 Luks Decryption
@@ -147,3 +148,17 @@ nmcli conn
export NMCLI_DEVICE=enp195s0f4u1u3
nmcli connection add type VLAN con-name $NMCLI_DEVICE.2 dev $NMCLI_DEVICE id 2
```
## GPU Support in Distrobox
Fix for `error="failed to check permission on /dev/kfd: open /dev/kfd: invalid argument"`
```bash
# You have to create the video and render group to /etc/group before you can use it
sudo grep -E '^video:' /usr/lib/group | sudo tee -a /etc/group
sudo grep -E '^render:' /usr/lib/group | sudo tee -a /etc/group
sudo usermod -aG video $USER
sudo usermod -aG render $USER
```
Logout and log back in to adopt new groups.

View File

@@ -2,17 +2,42 @@
- [Fedora Server](#fedora-server)
- [Installation](#installation)
- [Resize logical volume](#resize-logical-volume)
- [Setup SSH](#setup-ssh)
- [DNF](#dnf)
- [Fail2Ban](#fail2ban)
- [BTRFS Parent Volumes](#btrfs-parent-volumes)
- [BTRFS Snapshots](#btrfs-snapshots)
- [TPM2 Luks Decryption](#tpm2-luks-decryption)
- [Change your password](#change-your-password)
- [Automatic Updates](#automatic-updates)
- [Disable Swap](#disable-swap)
- [Selinux](#selinux)
- [Monitoring](#monitoring)
- [Disk Usage](#disk-usage)
- [Disk Wear](#disk-wear)
- [Common Storage Mounts](#common-storage-mounts)
- [Network Bridge](#network-bridge)
- [Virtualization](#virtualization)
- [QEMU Images](#qemu-images)
- [Firewalld](#firewalld)
- [Backups](#backups)
- [Quick Backup](#quick-backup)
- [Regular Backup to an NFS Share](#regular-backup-to-an-nfs-share)
- [Optional Steps](#optional-steps)
- [Docker with Podman as Runtime](#docker-with-podman-as-runtime)
- [Extras](#extras)
- [Vanilla Docker](#vanilla-docker)
- [Extra Software](#extra-software)
- [Disable Swap](#disable-swap)
- [Disable Selinux](#disable-selinux)
- [Downgrading Kernel](#downgrading-kernel)
- [Resize logical volume](#resize-logical-volume)
- [Create XFS LVM](#create-xfs-lvm)
- [LVM Thin Provisioning](#lvm-thin-provisioning)
- [Set eui64 on network interface](#set-eui64-on-network-interface)
- [Install and Enable Cockpit](#install-and-enable-cockpit)
- [Troubleshooting](#troubleshooting)
- [Cockpit Terminal Unusable or Weird Colors](#cockpit-terminal-unusable-or-weird-colors)
- [Chroot into a mounted disk](#chroot-into-a-mounted-disk)
- [Resize Last Partition to Fill Available Space](#resize-last-partition-to-fill-available-space)
- [LUKS performance](#luks-performance)
<https://docs.fedoraproject.org/en-US/fedora-server/installation/postinstallation-tasks/#_manage_system_updates>
@@ -23,22 +48,27 @@ and the operator will store information about each server.
## Installation
1. Create an administrator. We'll give ssh root access later, but this gives you a cockpit user.
2. Ensure IPV6 connection is set to "eui64".
3. Set hostname
## Resize logical volume
```bash
# Replace /dev/sda2 with whatever your disks are
# This assumes xfs
pvresize /dev/sda2
lvextend /dev/mapper/root -l+100%FREE
xfs_growfs -d /dev/mapper/root
```
1. Configure network first
1. Set a hostname
2. Disable ipv6 privacy extensions
2. Software Selection
1. Headless Management
3. User Creation
1. Set a simple password, we'll change it later
4. Disk partitioning
1. Select manual (blivet) partitioning
2. Create a 1GB EFI system partition and mount it at `/boot/efi`
3. Create a 1GB ext4 partition and mount it at `/boot`
4. Create a btrfs volume with the remaining data and name it something unqiue, do not mount it
5. Create a btrfs subvolume called "root" and mount it at `/`
6. Create any other btrfs subvolumes you might need
5. Take note of the ipv4 and ipv6 address. Update any DNS records at this time.
6. Install and reboot
## Setup SSH
In this setup we'll allow ssh to the root user via key and keep the admin user for cockpit.
On the operator:
```bash
@@ -47,7 +77,11 @@ ssh-keygen -t rsa -b 4096 -C ducoterra@"$SSH_HOST".reeselink.com -f ~/.ssh/id_"$
# Note: If you get "too many authentication failures" it's likely because you have too many private
# keys in your ~/.ssh directory. Use `-o PubkeyAuthentication` to fix it.
ssh-copy-id -o PubkeyAuthentication=no -i ~/.ssh/id_"$SSH_HOST"_rsa.pub ducoterra@"$SSH_HOST".reeselink.com
ssh-copy-id -o PubkeyAuthentication=no -i ~/.ssh/id_"$SSH_HOST"_rsa.pub ducoterra@${SSH_HOST}.reeselink.com
ssh -i ~/.ssh/id_"$SSH_HOST"_rsa ducoterra@${SSH_HOST}.reeselink.com
# Copy authorized_keys to root
sudo cp ~/.ssh/authorized_keys /root/.ssh/authorized_keys
exit
cat <<EOF >> ~/.ssh/config
@@ -61,26 +95,17 @@ Host ${SSH_HOST}
KeepAlive yes
IdentityFile ~/.ssh/id_${SSH_HOST}_rsa
EOF
```
On the server:
```bash
# Copy authorized_keys to root
sudo cp ~/.ssh/authorized_keys /root/.ssh/authorized_keys
# Change your password
passwd
sudo su -
ssh ${SSH_HOST}
# Disable password auth
echo "PasswordAuthentication no" > /etc/ssh/sshd_config.d/01-prohibit-password.conf
echo '%wheel ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/01-nopasswd-wheel
systemctl restart sshd
```
On the operator:
# OPTIONAL: Disable sudo password
echo '%wheel ALL=(ALL) NOPASSWD: ALL' > /etc/sudoers.d/01-nopasswd-wheel
exit
```bash
# Test if you can SSH with a password
ssh -o PubkeyAuthentication=no ducoterra@${SSH_HOST}.reeselink.com
@@ -95,7 +120,9 @@ Configure dnf to use the fastest mirror:
```bash
echo 'fastestmirror=1' >> /etc/dnf/dnf.conf
dnf clean all
dnf update
dnf update --refresh -y
# libdnf5 is required for ansible to work
dnf install -y glances tmux vim python3-libdnf5
```
## Fail2Ban
@@ -103,6 +130,9 @@ dnf update
On the server:
```bash
# Run tmux session
tmux
dnf install -y fail2ban
# Setup initial rules
@@ -123,45 +153,198 @@ enabled = true
EOF
systemctl enable fail2ban --now
# OPTIONAL: follow logs
tail -f /var/log/fail2ban.log
```
## BTRFS Parent Volumes
In `/etc/fstab`, add the parent volumes for your disks mounted with subvolid=5 at `/btrfs` so you can see
all subvolumes.
```conf
UUID=64beedac-c0c9-48bf-a3ae-7707df6ebc97 /btrfs/3dserver-root btrfs subvolid=5,compress=zstd:1,x-systemd.device-timeout=0 0 0
UUID=3c76b83f-7547-4c18-b08f-9e7902022b8d /btrfs/3dserver-data btrfs subvolid=5,compress=zstd:1,x-systemd.device-timeout=0 0 0
```
```bash
systemctl daemon-reload
mount -a --mkdir
```
## BTRFS Snapshots
<https://en.opensuse.org/openSUSE:Snapper_Tutorial>
Note: This requires you set up the mount point at `/btrfs` correctly! See [above](#btrfs-parent-volumes).
We'll be using snapper, a tool for automating and controlling snapshot behavior.
```bash
dnf install snapper dnf-plugin-snapper
# Allow selinux management
semanage permissive -a snapperd_t
# Note, if you mess something up you can run snapper -c root delete-config to delete
# System configs are stored in /etc/sysconfig/snapper as well as /etc/snapper
snapper -c root create-config /
snapper -c data create-config /path/to/other/data
# Enable automatic snapshots
systemctl enable --now snapper-timeline.timer
# Enable automatic cleanup
systemctl enable --now snapper-cleanup.timer
# Enable snapshots on boot
systemctl enable --now snapper-boot.timer
# List snapshots
snapper -c root list
# Create snapshot manually
snapper -c root create --description "test snapshot"
# Delete first snapshot
snapper -c root delete 1
```
## TPM2 Luks Decryption
Mostly taken from here:
<https://gist.github.com/jdoss/777e8b52c8d88eb87467935769c98a95>
PCR reference for `--tpm2-pcrs` args
```text
0: System firmware executable
2: Kernel
4: Bootloader
7: Secure boot state
8: Cmdline
9: Initrd
```
Note, if your threat vector is people trying to get data off your old disks after throwing them
away, you can set `--tpm2-pcrs=""`. Someone could gain access to your encrypted partition if they
can access your machine physically by manipulating the boot parameters but you're guaranteed to
unlock despite updates and upgrades.
Basic commands:
```bash
# Run tmux session
tmux
# Show tpm2 devices
systemd-cryptenroll --tpm2-device=list
# Show crypto luks block devices
blkid -t TYPE=crypto_LUKS
# Enroll the tpm2 device with systemd-cryptenroll
systemd-cryptenroll /dev/nvme0n1p3 --tpm2-device=auto --tpm2-pcrs=""
####################
##### OPTIONAL #####
####################
# If you have lots of devices to decrypt (like a btrfs raid array), use these commands.
# Get all crypto luks partitions
blkid | grep crypto_LUKS
# List them all space-separated and drop the '/dev'
LUKS_DEVS="nvme0n1p4 nvme1n1p1 nvme2n1p1 nvme3n1p1 nvme5n1p1 nvme4n1p1 nvme6n1p1"
# Check that your list is good
for dev in $LUKS_DEVS; do echo will enroll /dev/$dev; done
# Enroll
for dev in $LUKS_DEVS; do \
echo "Enrolling /dev/$dev"; \
systemd-cryptenroll /dev/$dev --tpm2-device=auto --tpm2-pcrs=""; \
done
########################
##### END OPTIONAL #####
########################
# Reenroll
systemd-cryptenroll /dev/nvme0n1p3 --wipe-slot=tpm2 --tpm2-device=auto --tpm2-pcrs=""
# Append to command line args
echo "add_dracutmodules+=\" tpm2-tss \"" | tee /etc/dracut.conf.d/tpm2.conf
dracut -f
```
Finally, `vim /etc/default/grub` and add `rd.luks.options=tpm2-device=auto` to GRUB_CMDLINE_LINUX
```bash
grub2-mkconfig -o /boot/grub2/grub.cfg
reboot
```
## Change your password
In Cockpit navigate to Accounts -> user -> Set password
## Automatic Updates
On the server:
In Cockpit navigate to software updates -> automatic updates -> install -> security updates only
## Monitoring
In Cockpit: Overview -> View metrics and history -> Install PCP Support -> Metrics settings -> Turn on Collect Metrics
### Disk Usage
TODO
### Disk Wear
TODO
## Common Storage Mounts
Note: mount these before you install the relavant package!
1. For virtual machines: `/var/lib/libvirt`
2. For podman: `/var/lib/containers`
3. For docker: `/var/lib/docker`
## Network Bridge
Networking -> Add bridge -> add network interface and save
```bash
dnf install dnf-automatic -y
systemctl enable --now dnf-automatic.timer
nmcli connection modify bridge0 ipv6.addr-gen-mode eui64
```
Edit the configuration to only do security updates.
## Disable Swap
## Virtualization
```bash
swapoff -a
zramctl --reset /dev/zram0
dnf -y remove zram-generator-defaults
dnf group install --with-optional virtualization
systemctl enable --now libvirtd
```
## Selinux
Install the cockpit machines application.
By default selinux will be enforcing. You can set it to permissive with
### QEMU Images
```bash
setenforce 0
# Grow an image to 2TB
qemu-img resize nextcloud_aio-fcfgp.qcow2 2T
```
And then make it permanent by editing `/etc/selinux/config` and inserting `SELINUX=permissive`.
```bash
# Convert OVA to img
qemu-img convert -f vmdk -O raw in.vmdk out.img
# Convert qcow2 to img
qemu-img convert -f qcow2 -O raw in.raw out.img
```
## Firewalld
Set the default firewalld zone to `public`
```bash
firewall-cmd --set-default-zone=public
# Note, you probably don't have to do this. Check Cockpit Network -> Firewall
# firewall-cmd --set-default-zone=public
```
Firewalld will be on and blocking by default. You can check the zone and allowed ports with:
@@ -178,7 +361,50 @@ firewall-cmd --permanent --zone=public --add-port=9090/tcp
firewall-cmd --reload
```
## Docker with Podman as Runtime
## Backups
### Quick Backup
```bash
rsync -av / \
--exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} \
/mnt/root/
```
### Regular Backup to an NFS Share
1. Create a new dataset called "<server>_backup"
2. Remove "other" read/exec permissions from the dataset
3. Create a new NFS share for that dataset with maproot user and group set to root
4. Mount the NFS share to your server at `/backup`
5. Copy the following script into /root/backup.sh
```bash
#!/bin/bash
BACKUP_PATH="/backup"
EXCLUDE_DIR='{"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"}'
SOURCE_DIR="/"
rsync -aAXv ${SOURCE_DIR} --exclude=${EXCLUDE_DIR} ${BACKUP_PATH}
if [ $? -eq 0 ]; then
echo "Backup completed successfully"
else
echo "Some error occurred during backup"
fi
```
6. `chmod +x /root/backup.sh`
7. `crontab -e`
```cron
0 2 * * * bash /root/backup.sh >> /root/backup.log
```
## Optional Steps
### Docker with Podman as Runtime
Note, you'll need to ssh into the server as the user in order to start the user's systemd session.
@@ -191,9 +417,18 @@ docker context create podman --docker host=unix://$XDG_RUNTIME_DIR/podman/podman
docker context use podman
```
## Extras
### Vanilla Docker
On the server:
<https://docs.docker.com/engine/install/fedora/>
```bash
dnf -y install dnf-plugins-core
dnf-3 config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
dnf install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
systemctl enable --now docker
```
### Extra Software
```bash
# Set vim as the default editor
@@ -202,38 +437,30 @@ dnf install -y vim-default-editor --allowerasing
# Install glances for system monitoring
dnf install -y glances
# Install zsh with autocomplete and suggestions
dnf install -y zsh zsh-autosuggestions zsh-syntax-highlighting
cat <<EOF > ~/.zshrc
# History
HISTFILE=~/.zsh_history
HISTSIZE=10000
SAVEHIST=10000
setopt appendhistory
# Basic settings
autoload bashcompinit && bashcompinit
autoload -U compinit; compinit
zstyle ':completion:*' menu select
# Prompt settings
autoload -Uz promptinit
promptinit
prompt redhat
PROMPT_EOL_MARK=
# Syntax Highlighting
source /usr/share/zsh-syntax-highlighting/zsh-syntax-highlighting.zsh
source /usr/share/zsh-autosuggestions/zsh-autosuggestions.zsh
### Custom Commands and Aliases ###
EOF
# ZSH
dnf install -y zsh
chsh -s $(which zsh) && chsh -s $(which zsh) ducoterra
```
## Downgrading Kernel
### Disable Swap
```bash
swapoff -a
zramctl --reset /dev/zram0
dnf -y remove zram-generator-defaults
```
### Disable Selinux
By default selinux will be enforcing. You can set it to permissive with
```bash
setenforce 0
```
And then make it permanent by editing `/etc/selinux/config` and inserting `SELINUX=permissive`.
### Downgrading Kernel
```bash
dnf install koji
@@ -243,3 +470,153 @@ cd $(mktemp -d) && koji download-build --arch=x86_64 --arch=noarch kernel-6.11.3
reboot
```
### Resize logical volume
```bash
# Replace /dev/sda2 with whatever your disks are
# This assumes xfs
pvresize /dev/sda2
lvextend /dev/mapper/root -l+100%FREE
xfs_growfs -d /dev/mapper/root
```
### Create XFS LVM
<https://www.linuxtechi.com/how-to-create-lvm-partition-in-linux/>
<https://www.golinuxcloud.com/lvcreate-command-in-linux/#How_to_install_lvcreate>
If you get the error "Not creating system devices file due to existing VGs."
Run `vgimportdevices -a` and check `/etc/lvm/devices/system.devices`
1. Create a new partition for the Physical Volume (fdisk)
```bash
# Create the physical volume
pvcreate /dev/vda4
# Create the volume group (vgcreate <vg_name> <pv>)
vgcreate nextcloud_data /dev/vda4
# Create the logical volume (lvcreate -L <Size-of-LV> -n <LV-Name> <VG-Name>)
# Or lvcreate -l 100%FREE
lvcreate -l 100%FREE -n nextcloud_data_vol nextcloud_data
# list the PV, VG, LV
pvs
vgs
lvs
# Format lv
mkfs.btrfs /dev/nextcloud_data/nextcloud_data_vol
```
### LVM Thin Provisioning
<https://linuxconfig.org/introduction-to-lvm-thin-provisioning>
If you get the error "Not creating system devices file due to existing VGs."
Run `vgimportdevices -a` and check `/etc/lvm/devices/system.devices`
Thin provisioning allows you to overprovision your storage drives to make the filesystem
think it has more data than it does.
```bash
# Create the physical volume
pvcreate /dev/vda4
# Create the volume group
vgcreate vg0 /dev/vda4
# Create the thin pool - the volume with real data that will hold our thing volumes with fake data
lvcreate -l 100%FREE -T vg0/thinpool
# Create the thin volumes with fake data
lvcreate -T -V 2T vg0/thinpool -n local-path-provisioner
lvcreate -T -V 2T vg0/thinpool -n docker-data
# Format the fake volumes
mkfs.xfs /dev/mapper/vg0-local--path--provisioner
mkfs.xfs /dev/mapper/vg0-docker--data
```
### Set eui64 on network interface
```bash
nmcli connection modify Wired\ connection\ 1 ipv6.addr-gen-mode eui64
```
### Install and Enable Cockpit
<https://cockpit-project.org/running>
```bash
dnf install cockpit
systemctl enable --now cockpit.socket
firewall-cmd --add-service=cockpit
firewall-cmd --add-service=cockpit --permanent
```
## Troubleshooting
### Cockpit Terminal Unusable or Weird Colors
Make sure you give canvas access to the browser (especially in librewolf)
### Chroot into a mounted disk
This lets you run grub2-mkconfig among other things.
```bash
# Mount root
mount /dev/mapper/vg0-root /mnt
# Mount proc, sys, and dev
mount -t proc /proc proc/
mount --rbind /sys sys/
mount --rbind /dev dev/
# Mount boot and efi
mount /dev/vdb2 /mnt/boot
mount /dev/vdb1 /mnt/boot/efi
chroot /mnt
```
### Resize Last Partition to Fill Available Space
```bash
parted /dev/vdb
# to resize /dev/vdb3 to fill 100% of the disk, for example
resizepart 3 100%
quit
# Resize the physical volume to match the partition
pvresize /dev/vdb3
```
### LUKS performance
```bash
cryptsetup benchmark
```
Should output something like:
```bash
# Algorithm | Key | Encryption | Decryption
aes-cbc 128b 1409.1 MiB/s 3627.9 MiB/s
serpent-cbc 128b 146.5 MiB/s 981.4 MiB/s
twofish-cbc 128b 289.8 MiB/s 613.3 MiB/s
aes-cbc 256b 1100.2 MiB/s 3448.2 MiB/s
serpent-cbc 256b 150.3 MiB/s 982.1 MiB/s
twofish-cbc 256b 294.3 MiB/s 590.8 MiB/s
aes-xts 256b 4423.5 MiB/s 4561.2 MiB/s
serpent-xts 256b 874.9 MiB/s 883.7 MiB/s
twofish-xts 256b 557.8 MiB/s 559.4 MiB/s
aes-xts 512b 4551.2 MiB/s 4669.6 MiB/s
serpent-xts 512b 890.8 MiB/s 860.5 MiB/s
twofish-xts 512b 557.5 MiB/s 564.2 MiB/s
```
Which will tell you how fast you can theoretically write/read to encrypted drives.
The default encryption used by most modern operating systems is AES-XTS.
You can see your system's cipher and key with `cryptsetup luksDump /dev/nvme0n1p1 | grep -i cipher`

View File

@@ -0,0 +1,36 @@
# Proxmox
## Install
Download the Proxmox Virtual Environment ISO
<https://www.proxmox.com/en/downloads/proxmox-virtual-environment>
```bash
dd bs=1M conv=fdatasync if=./proxmox-ve_*.iso of=/dev/XYZ
```
## First Boot
1. Change root password
## SSH
SSH into your instance as normal and add ssh keys to ~/.ssh/authorized_keys
## ZFS Import Encrypted Dataset
1. Create a new file with the raw key contents of your encrypted dataset
2. `zfs load-key -L file:///root/enc0.vms.key enc0/vms`
3. `zfs list -o name,keystatus enc0/vms` Checks for lock status ("unavailable" means locked)
## ZFS Encryption
## VMs with zvols
```bash
# Get VM ID
qm list
export VM_ID=100
qm set $VM_ID --virtiodisk0 /dev/zvol/enc0/vms/,format=raw
```

View File

@@ -23,6 +23,8 @@
- [Scale](#scale)
- [ARC Limit](#arc-limit)
- [Certs](#certs)
- [Let's Encrypt](#lets-encrypt)
- [Self-signed CA](#self-signed-ca)
- [Testing](#testing)
- [iperf](#iperf)
- [disk](#disk)
@@ -34,6 +36,8 @@
- [UPS Monitoring](#ups-monitoring)
- [ZFS Size Data](#zfs-size-data)
- [ZFS Rename](#zfs-rename)
- [ISCSI](#iscsi)
- [Base Name](#base-name)
## Bios settings
@@ -307,8 +311,16 @@ setfacl -b -R /mnt/enc0/smb/media
### Converting zvol to qcow2
```bash
dd if=/dev/zvol/enc1/vms/unifi-e373f of=unifi.raw
# Convert zvol to raw
dd status=progress bs=1M if=/dev/zvol/enc0/vms/nextcloud-fi7tkq of=/mnt/enc0/vms/qcow2/nextcloud-fi7tkq.raw
# Convert raw to qcow
qemu-img convert -f raw -O qcow2 unifi.raw unifi.qcow2
# Convert in batch
# Convert zvol to raw
for FILE in $(ls /dev/zvol/enc0/vms); do dd status=progress bs=1M if=/dev/zvol/enc0/vms/$FILE of=/mnt/enc0/vms/qcow2/$FILE.raw; done
# Convert raw to qcow
for FILE in $(ls /dev/zvol/enc0/vms); do echo "qemu-img convert -f raw -O qcow2 /mnt/enc0/vms/qcow2/$FILE.raw /mnt/enc0/vms/qcow2/$FILE.qcow2"; done
```
### Converting qcow2 to zvol
@@ -354,6 +366,24 @@ Set `When` to `Post Init`.
## Certs
### Let's Encrypt
<https://www.truenas.com/docs/scale/22.12/scaletutorials/credentials/certificates/settingupletsencryptcertificates/>
Note, for all "Name" fields use your domain with all "." replaced with "-"
Examaple: `driveripper.reeselink.com` becomes `driveripper-reeselink-com`
1. Go to Credentials > Certificates and click ADD in the ACME DNS-Authenticators widget
2. Generate credentials for your domain via [AWS IAM](/cloud/graduated/aws_iam/aws_iam.md)
3. Click ADD in the Certificate Signing Requests widget
1. Remember, only the SAN is required
4. Click the wrench icon next to the new CSR
1. In "Identifier" use `domain-something-com-le-stage` for staging and `domain-something-com-le-prod` for prod
5. System -> General Settings -> GUI Settings -> GUI SSL Certificate -> `domain-something-com-le-prod`
### Self-signed CA
<https://raymondc.net/2018/02/28/using-freenas-as-your-ca.html>
1. Create a new Root certificate (CAs -> ADD -> Internal CA)
@@ -535,3 +565,13 @@ Make sure you unshare any connected shares, otherwise you'll get
```bash
zfs rename enc0/something enc0/something_else
```
## ISCSI
### Base Name
<https://datatracker.ietf.org/doc/html/rfc3721.html#section-1.1>
| iqn | . | year-month of domain registration | . | reversed domain | : | unique string
iqn.2022-01.com.reeselink:driveripper

View File

@@ -6,6 +6,8 @@
- [Automatic Updates](#automatic-updates)
- [Disable Swap](#disable-swap)
- [Extras](#extras)
- [Troubleshooting](#troubleshooting)
- [nmcli device unmanaged](#nmcli-device-unmanaged)
Note these instructions differentiate between an `operator` and a `server`. The operator can be
any machine that configure the server. A pipeline, laptop, dedicated server, etc. are all options.
@@ -145,3 +147,19 @@ chsh -s $(which zsh) && chsh -s $(which zsh) ducoterra
apt install -y cockpit
systemctl enable --now cockpit
```
## Troubleshooting
### nmcli device unmanaged
Ubuntu installs a config file that sets most devices unmanaged:
/usr/lib/NetworkManager/conf.d/10-globally-managed-devices.conf:
[keyfile]
unmanaged-devices=*,except:type:wifi,except:type:gsm,except:type:cdma
To disable this, You can create a blank file with the same name in /etc:
sudo touch /etc/NetworkManager/conf.d/10-globally-managed-devices.conf
sudo systemctl restart NetworkManager

View File

@@ -0,0 +1,15 @@
provider:
name: aws
env:
- name: AWS_SHARED_CREDENTIALS_FILE
value: /etc/aws/credentials/externaldns-credentials
- name: AWS_DEFAULT_REGION
value: us-east-1 # change to region where EKS is installed
extraVolumes:
- name: aws-credentials
secret:
secretName: external-dns # In this example, the secret will have the data stored in a key named `my_credentials`
extraVolumeMounts:
- name: aws-credentials
mountPath: /etc/aws/credentials
readOnly: true

View File

@@ -59,16 +59,16 @@ spec:
claimName: {{ .Release.Name }}-cache
- name: movies
nfs:
server: driveripper.reeselink.com
path: /mnt/enc0/media/Movies
server: driveripper-lab.reeselink.com
path: /mnt/enc0/smb/media/Movies
readOnly: true
- name: shows
nfs:
server: driveripper.reeselink.com
path: /mnt/enc0/media/Shows
server: driveripper-lab.reeselink.com
path: /mnt/enc0/smb/media/Shows
readOnly: true
- name: videos
nfs:
server: driveripper.reeselink.com
path: /mnt/enc0/media/Videos
server: driveripper-lab.reeselink.com
path: /mnt/enc0/smb/media/Videos
readOnly: true

View File

@@ -3,11 +3,8 @@
value: |-
{
"storageClassConfigs": {
"ssd": {
"sharedFileSystemPath": "/opt/local-path-provisioner/ssd"
},
"hdd": {
"sharedFileSystemPath": "/opt/local-path-provisioner/hdd"
"local-path": {
"sharedFileSystemPath": "/opt/local-path-provisioner"
}
}
}

View File

@@ -1,3 +0,0 @@
- op: replace # action
path: /metadata/name # resource we want to change
value: hdd # value we want to use for patching

View File

@@ -1,15 +1,10 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# Creates local path storage and a storage class
# Patched by ConfigMap-patch.yaml and namespace-patch.yaml
resources:
- local-path-storage.yaml
- ssd-storage.yaml
patches:
- target:
group: storage.k8s.io
version: v1
kind: StorageClass
name: local-path
path: StorageClass-hdd-patch.yaml
- target:
group: ""
version: v1

View File

@@ -0,0 +1,24 @@
# Local Path Provisioner Install
1. `mkdir /var/lib/rancher/k3s/storage`
2. Edit fstab to mount your drive to `/var/lib/rancher/k3s/storage`
3. `systemctl daemon-reload`
4. `mount -a`
<https://github.com/rancher/local-path-provisioner/tree/master/deploy/chart/local-path-provisioner>
```bash
# Download the updated template from github
kubectl kustomize "github.com/rancher/local-path-provisioner/deploy?ref=v0.0.31" > kubernetes/graduated/local-path-provisioner/local-path-storage.yaml
# Apply customizations (ssd/hdd storage, read write many support)
kubectl kustomize kubernetes/graduated/local-path-provisioner | kubectl apply -f -
# Create test pod
kubectl apply -f systemd/graduated/k3s/tests/local-storage-test.yaml
kubectl get pod -n default
# Exec in and test - storage will be mounted at /storage
kubectl exec -it -n default <local-storage-test> -- bash
kubectl delete -f systemd/graduated/k3s/tests/local-storage-test.yaml
```

View File

@@ -176,7 +176,7 @@ spec:
fieldPath: metadata.namespace
- name: CONFIG_MOUNT_PATH
value: /etc/config/
image: rancher/local-path-provisioner:v0.0.28
image: rancher/local-path-provisioner:v0.0.31
imagePullPolicy: IfNotPresent
name: local-path-provisioner
volumeMounts:

View File

@@ -1,9 +0,0 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: ssd
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: rancher.io/local-path
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

View File

@@ -1,22 +1,13 @@
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: internal
namespace: kube-system
spec:
addresses:
- 2603:6013:3140:103::4-2603:6013:3140:103:ffff:ffff:ffff:ffff
- 10.4.0.4-10.4.255.255
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: external
name: unifi-pool
namespace: kube-system
spec:
addresses:
- 2603:6013:3140:104::4-2603:6013:3140:104:ffff:ffff:ffff:ffff
- 10.5.0.4-10.5.255.255
- 2603:6013:3140:105:10:5:0:10-2603:6013:3140:105:10:5:0:210
- 10.5.0.10-10.5.0.210
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
@@ -25,5 +16,4 @@ metadata:
namespace: kube-system
spec:
ipAddressPools:
- external
- internal
- unifi-pool

View File

@@ -17,7 +17,7 @@ spec:
- name: snapdrop
image: {{ .Values.snapdrop.image }}
ports:
- containerPort: 80
- containerPort: 3000
name: http
envFrom:
- configMapRef:

View File

@@ -1,3 +1,3 @@
snapdrop:
image: linuxserver/snapdrop:latest
image: linuxserver/pairdrop:latest
domain: snapdrop.reeseapps.com

3
kubernetes/kubernetes.md Normal file
View File

@@ -0,0 +1,3 @@
# Kubernetes
See [k3s](/systemd/graduated/k3s/k3s.md)

View File

@@ -0,0 +1,63 @@
# Caddy Reverse Proxy
## Install Caddy
As root
```bash
mkdir /etc/caddy
vim /etc/caddy/Caddyfile
```
Caddy will automatically provision certificates if the server DNS points to the correct IP
and is accessible on the ports specifified. All you need to do is put `https` in the caddy conf.
```conf
# Nextcloud
https://nextcloud.reeseapps.com:443 {
reverse_proxy podman.reeselink.com:11000
}
https://nextcloud.reeseapps.com:8443 {
reverse_proxy podman.reeselink.com:11001 {
transport http {
tls_insecure_skip_verify
}
}
}
# Gitea
https://gitea.reeseapps.com:443 {
reverse_proxy podman.reeselink.com:3000
}
```
```bash
vim /etc/containers/systemd/caddy.container
```
```conf
[Unit]
Description=Caddy
[Container]
AddCapability=NET_ADMIN
ContainerName=caddy
Image=docker.io/caddy:2
Network=host
SecurityLabelDisable=true
Volume=/etc/caddy:/etc/caddy
Volume=caddy_data:/data
Volume=caddy_config:/config
[Service]
Restart=always
[Install]
WantedBy=default.target
```
```bash
systemctl daemon-reload
systemctl start caddy
```

View File

@@ -0,0 +1,45 @@
version: "3"
networks:
gitea:
services:
gitea:
image: docker.gitea.com/gitea:1.23.7
container_name: gitea
environment:
- USER_UID=1000
- USER_GID=1000
- GITEA__database__DB_TYPE=postgres
- GITEA__database__HOST=postgres:5432
- GITEA__database__NAME=gitea
- GITEA__database__USER=gitea
- GITEA__database__PASSWD=gitea
restart: always
networks:
- gitea
volumes:
- /home/gitea/gitea:/data
- /etc/localtime:/etc/localtime:ro
ports:
- "3000:3000"
- "2222:22"
depends_on:
- postgres
security_opt:
- label=disable
postgres:
image: docker.io/library/postgres:15
container_name: postgres
restart: always
environment:
- POSTGRES_USER=gitea
- POSTGRES_PASSWORD=gitea
- POSTGRES_DB=gitea
networks:
- gitea
volumes:
- /home/gitea/postgres:/var/lib/postgresql/data
security_opt:
- label=disable

View File

@@ -0,0 +1,110 @@
# Gitea
## Gitea on Rootless Podman
### Create the gitea user
```bash
useradd gitea
su - gitea
ssh-keygen
exit
cp ~/.ssh/authorized_keys /home/gitea/.ssh/authorized_keys
chown gitea:gitea /home/gitea/.ssh/authorized_keys
loginctl enable-linger $(id -u gitea)
```
SSH into the server as gitea
```bash
systemctl --user enable podman-restart
systemctl --user enable --now podman.socket
mkdir -p ~/.config/containers/systemd
mkdir data config postgres
```
### Convert Compose to Quadlet
```bash
# Run this in Homelab, not on the serrver.
mkdir quadlets
# Generate the systemd service
podman run \
--security-opt label=disable \
--rm \
-v $(pwd):/compose \
-v $(pwd)/quadlets:/quadlets \
quay.io/k9withabone/podlet \
-f /quadlets \
-i \
--overwrite \
compose /compose/compose.yaml
# Copy the files to the server
scp -r quadlets/. gitea:~/.config/containers/systemd/
```
### Install Quadlets
The first user you register will be the admin
```bash
ssh gitea
systemctl --user daemon-reload
systemctl --user start gitea postgres
```
## Gitea Runners
<https://docs.gitea.com/next/usage/actions/act-runner/#install-with-the-docker-image>
### Firewall Rules
Since our runner will be contacting our public IP, we need to add a firewall rule to allow
traffic from our DMZ network to our DMZ network. Do this in Unifi or whatever equivalent
you have.
### Install
```bash
touch config.yaml
export GITEA_TOKEN=
docker run \
-v /var/run/docker.sock:/var/run/docker.sock \
-e GITEA_INSTANCE_URL=https://gitea.reeseapps.com \
-e GITEA_RUNNER_REGISTRATION_TOKEN=$GITEA_TOKEN \
-e GITEA_RUNNER_NAME=gitea_runner \
--restart always \
--name gitea_runner \
-d docker.io/gitea/act_runner:latest
```
### Cache Cleanup
Each org or project with a package registry will have its own cleanup rules. For example,
services -> settings -> Packages -> Add Cleanup Rule will allow you to create a cleanup
rule for packages stored under the "services" org. These cleanup rules should run automatically.
On the other hand, the docker builder cache will balloon out of control over time. The gitea
docker runner is handled outside of Gitea's context, so you'll need to clean it up yourself.
```bash
# Check used system resources
docker system df
```
You should run something like this on a schedule:
```bash
# Prune the builder cache
docker builder prune -a
```
To run it every day at midnight: `crontab -e`
```bash
0 0 * * * yes | docker builder prune -a
```

View File

@@ -0,0 +1,19 @@
[Unit]
Requires=postgres.service
[Container]
ContainerName=gitea
Environment=USER_UID=1000 USER_GID=1000 GITEA__database__DB_TYPE=postgres GITEA__database__HOST=postgres:5432 GITEA__database__NAME=gitea GITEA__database__USER=gitea GITEA__database__PASSWD=gitea
Image=docker.gitea.com/gitea:1.23.7
Network=gitea.network
PublishPort=3000:3000
PublishPort=2222:22
SecurityLabelDisable=true
Volume=/home/gitea/gitea:/data
Volume=/etc/localtime:/etc/localtime:ro
[Service]
Restart=always
[Install]
WantedBy=default.target

View File

@@ -0,0 +1,4 @@
[Network]
[Install]
WantedBy=default.target

View File

@@ -0,0 +1,13 @@
[Container]
ContainerName=postgres
Environment=POSTGRES_USER=gitea POSTGRES_PASSWORD=gitea POSTGRES_DB=gitea
Image=docker.io/library/postgres:15
Network=gitea.network
SecurityLabelDisable=true
Volume=/home/gitea/postgres:/var/lib/postgresql/data
[Service]
Restart=always
[Install]
WantedBy=default.target

View File

@@ -4,9 +4,10 @@
- [Install with Rootless Podman](#install-with-rootless-podman)
- [Create the nextcloud user](#create-the-nextcloud-user)
- [Install Podman](#install-podman)
- [Install Caddy](#install-caddy)
- [Create the container autostart service](#create-the-container-autostart-service)
- [Install Nextcloud](#install-nextcloud)
- [Install Caddy](#install-caddy)
- [Firewall](#firewall)
- [Backups](#backups)
- [Maintenace Mode](#maintenace-mode)
- [Trusted Proxy](#trusted-proxy)
@@ -32,21 +33,84 @@ This has been tested working on Fedora 41 with selinux and firewalld enabled.
### Create the nextcloud user
```bash
sudo useradd nextcloud
sudo loginctl enable-linger $(id -u nextcloud)
useradd nextcloud
su - nextcloud
ssh-keygen
exit
cp ~/.ssh/authorized_keys /home/nextcloud/.ssh/authorized_keys
chown nextcloud:nextcloud /home/nextcloud/.ssh/authorized_keys
loginctl enable-linger $(id -u nextcloud)
```
### Install Podman
```bash
# As admin user
sudo dnf install podman
# As root user
dnf install podman
# Now SSH into the server as the nextcloud user
systemctl --user enable podman-restart
systemctl --user enable --now podman.socket
```
### Create the container autostart service
As the nextcloud user.
`systemctl --user edit podman-restart.service`
```conf
[Service]
ExecStart=
ExecStart=/usr/bin/podman $LOGGING start --all --filter restart-policy=always --filter restart-policy=unless-stopped
ExecStop=
ExecStop=/bin/sh -c '/usr/bin/podman $LOGGING stop $(/usr/bin/podman container ls --filter restart-policy=always --filter restart-policy=unless-stopped -q)'
```
```bash
systemctl --user daemon-reload
```
### Install Nextcloud
`mkdir -p ~/.config/containers/systemd`
`vim ~/.config/containers/systemd/nextcloud-aio-mastercontainer.container`
```conf
[Unit]
Description=Nextcloud AIO Master Container
Documentation=https://github.com/nextcloud/all-in-one/blob/main/docker-rootless.md
After=local-fs.target
Requires=podman.socket
[Container]
ContainerName=nextcloud-aio-mastercontainer
Image=docker.io/nextcloud/all-in-one:latest
PublishPort=0.0.0.0:11001:8080
Volume=nextcloud_aio_mastercontainer:/mnt/docker-aio-config
Volume=/run/user/1001/podman/podman.sock:/var/run/docker.sock:Z
Network=bridge
SecurityLabelDisable=true
Environment=APACHE_PORT=11000
Environment=APACHE_IP_BINDING=0.0.0.0
Environment=WATCHTOWER_DOCKER_SOCKET_PATH=/run/user/1001/podman/podman.sock
Environment=NEXTCLOUD_DATADIR="/home/nextcloud/nextcloud_data"
Environment=SKIP_DOMAIN_VALIDATION=true
[Service]
Restart=always
[Install]
WantedBy=multi-user.target default.target
```
```bash
systemctl --user daemon-reload
systemctl --user start nextcloud-aio-mastercontainer
```
### Install Caddy
As root
@@ -84,7 +148,7 @@ Description=Caddy
[Container]
AddCapability=NET_ADMIN
ContainerName=caddy
Image=caddy
Image=docker.io/caddy:2
Network=host
SecurityLabelDisable=true
Volume=/etc/caddy:/etc/caddy
@@ -103,62 +167,10 @@ systemctl daemon-reload
systemctl start caddy
```
### Create the container autostart service
As the nextcloud user.
### Firewall
`systemctl --user edit podman-restart.service`
```conf
[Service]
ExecStart=
ExecStart=/usr/bin/podman $LOGGING start --all --filter restart-policy=always --filter restart-policy=unless-stopped
ExecStop=
ExecStop=/bin/sh -c '/usr/bin/podman $LOGGING stop $(/usr/bin/podman container ls --filter restart-policy=always --filter restart-policy=unless-stopped -q)'
```
```bash
systemctl --user daemon-reload
systemctl --user enable podman-restart
```
### Install Nextcloud
`vim ~/.config/containers/systemd/nextcloud-aio-mastercontainer.container`
```conf
[Unit]
Description=Nextcloud AIO Master Container
Documentation=https://github.com/nextcloud/all-in-one/blob/main/docker-rootless.md
After=local-fs.target
Requires=podman.socket
[Container]
ContainerName=nextcloud-aio-mastercontainer
Image=docker.io/nextcloud/all-in-one:latest
PublishPort=127.0.0.1:11001:8080
Volume=nextcloud_aio_mastercontainer:/mnt/docker-aio-config
Volume=/run/user/1001/podman/podman.sock:/var/run/docker.sock:Z
Network=bridge
SecurityLabelDisable=true
Environment=APACHE_PORT=11000
Environment=APACHE_IP_BINDING=127.0.0.1
Environment=WATCHTOWER_DOCKER_SOCKET_PATH=/run/user/1001/podman/podman.sock
Environment=NEXTCLOUD_DATADIR="/home/nextcloud/nextcloud_data"
Environment=SKIP_DOMAIN_VALIDATION=true
[Service]
Restart=always
[Install]
WantedBy=multi-user.target default.target
```
```bash
systemctl --user daemon-reload
systemctl --user start nextcloud-aio-mastercontainer
```
Allow traffic to 11000 from your reverse proxy
## Backups
@@ -250,7 +262,6 @@ Sometimes this is caused by a broken app or twofactor. try:
./occ app:disable integration_openai
```
## Freezing after working for a bit
### Out of disk space

View File

@@ -0,0 +1,20 @@
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
stream {
server {
listen 2222;
listen [::]:2222;
proxy_pass podman.reeselink.com:2222;
}
}

View File

@@ -0,0 +1,37 @@
# Ngnix
## TCP Stream Proxy
```bash
# Get the initial configuration
podman run --rm --entrypoint=cat docker.io/nginx /etc/nginx/nginx.conf > nginx.conf
scp nginx.conf 3dserver:/etc/nginx/nginx.conf
```
```bash
vim /etc/containers/systemd/nginx.container
```
```conf
[Unit]
Description=Nginx
[Container]
AddCapability=NET_ADMIN
ContainerName=nginx
Image=docker.io/nginx
Network=host
SecurityLabelDisable=true
Volume=/etc/nginx:/etc/nginx
[Service]
Restart=always
[Install]
WantedBy=default.target
```
```bash
systemctl daemon-reload
systemctl start nginx
```

View File

@@ -1,14 +0,0 @@
# DDNS Service
Since we occasionally need an ipv4 address we'll make one.
This creates and keeps updated `ipv4.reeselink.com`.
This requires the aws cli to be installed on each node with credentials that can modify
records in route53.
<https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html>
```bash
ansible-playbook -i ansible/inventory.yaml systemd/graduated/ddns/install_ddns.yaml
```

View File

@@ -0,0 +1,15 @@
# DDNS Service
Since we occasionally need an ipv4 address we'll make one.
This creates and keeps updated ipv4 records for reeseapps.com and reeselink.com
as specified in vars.yaml
**NOTE**: This requires the aws cli to be installed on each node with
credentials that can modify records in route53. See
[aws_iam](/cloud/graduated/aws_iam/aws_iam.md) and
[aws_cli](/cloud/graduated/aws_cli/aws_cli.md)
```bash
ansible-playbook -i ansible/inventory.yaml systemd/graduated/ddns/install_ddns.yaml
```

View File

@@ -1,20 +1,29 @@
#!/bin/bash
# Get public IP address (there are many ways to do it, I picked this way)
PUBLIC_IP=$(curl -4 ifconfig.me)
PUBLIC_IPV4=$(curl -4 ifconfig.me)
PUBLIC_IPV6=$(curl -6 ifconfig.me)
# Update reeselink records
cat /etc/ddns/reeselink_record_template.json \
| jq '.Changes[0].ResourceRecordSet.ResourceRecords[0].Value = "'$PUBLIC_IP'"' \
> /etc/ddns/reeselink_record.json
cat /etc/ddns/ipv4_reeselink_record_template.json \
| jq '.Changes[0].ResourceRecordSet.ResourceRecords[0].Value = "'$PUBLIC_IPV4'"' \
> /etc/ddns/ipv4_reeselink_record.json
cat /etc/ddns/ipv6_reeselink_record_template.json \
| jq '.Changes[0].ResourceRecordSet.ResourceRecords[0].Value = "'$PUBLIC_IPV6'"' \
> /etc/ddns/ipv6_reeselink_record.json
# Update reeseapps records
cat /etc/ddns/reeseapps_record_template.json \
| jq '.Changes[].ResourceRecordSet.ResourceRecords[0].Value = "'$PUBLIC_IP'"' \
> /etc/ddns/reeseapps_record.json
cat /etc/ddns/ipv4_reeseapps_record_template.json \
| jq '.Changes[].ResourceRecordSet.ResourceRecords[0].Value = "'$PUBLIC_IPV4'"' \
> /etc/ddns/ipv4_reeseapps_record.json
cat /etc/ddns/ipv6_reeseapps_record_template.json \
| jq '.Changes[].ResourceRecordSet.ResourceRecords[0].Value = "'$PUBLIC_IPV6'"' \
> /etc/ddns/ipv6_reeseapps_record.json
# Update reeselink records
aws route53 change-resource-record-sets --hosted-zone-id {{ reeselink_zone_id }} --change-batch file:///etc/ddns/reeselink_record.json
aws route53 change-resource-record-sets --hosted-zone-id {{ reeselink_zone_id }} --change-batch file:///etc/ddns/ipv4_reeselink_record.json
aws route53 change-resource-record-sets --hosted-zone-id {{ reeselink_zone_id }} --change-batch file:///etc/ddns/ipv6_reeselink_record.json
# Update reeseapps records
aws route53 change-resource-record-sets --hosted-zone-id {{ reeseapps_zone_id }} --change-batch file:///etc/ddns/reeseapps_record.json
aws route53 change-resource-record-sets --hosted-zone-id {{ reeseapps_zone_id }} --change-batch file:///etc/ddns/ipv4_reeseapps_record.json
aws route53 change-resource-record-sets --hosted-zone-id {{ reeseapps_zone_id }} --change-batch file:///etc/ddns/ipv6_reeseapps_record.json

View File

@@ -1,5 +1,5 @@
- name: Update nginx stream configuration
hosts: yellow
- name: Create DDNS Service
hosts: 3dserver
vars_files:
- vars.yaml
- secrets/secret_vars.yaml
@@ -27,17 +27,31 @@
path: /etc/ddns
state: directory
mode: '0755'
- name: Copy reeseapps_record_template.json
- name: Copy IPv4 reeseapps_record_template.json
template:
src: secrets/reeseapps_record_template.json
dest: /etc/ddns/reeseapps_record_template.json
src: ipv4_reeseapps_record_template.json.j2
dest: /etc/ddns/ipv4_reeseapps_record_template.json
owner: root
group: root
mode: '0644'
- name: Copy reeselink_record_template.json
- name: Copy IPv4 reeselink_record_template.json
template:
src: secrets/reeselink_record_template.json
dest: /etc/ddns/reeselink_record_template.json
src: ipv4_reeselink_record_template.json.j2
dest: /etc/ddns/ipv4_reeselink_record_template.json
owner: root
group: root
mode: '0644'
- name: Copy IPv6 reeseapps_record_template.json
template:
src: ipv6_reeseapps_record_template.json.j2
dest: /etc/ddns/ipv6_reeseapps_record_template.json
owner: root
group: root
mode: '0644'
- name: Copy IPv6 reeselink_record_template.json
template:
src: ipv6_reeselink_record_template.json.j2
dest: /etc/ddns/ipv6_reeselink_record_template.json
owner: root
group: root
mode: '0644'

View File

@@ -0,0 +1,20 @@
{
"Comment": "Update Public IPV4 Address",
"Changes": [
{%- for item in records.reeseapps %}
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "{{ item }}.reeseapps.com",
"Type": "A",
"TTL": 300,
"ResourceRecords": [
{
"Value": ""
}
]
}
}{{ ", " if not loop.last else "" }}
{%- endfor %}
]
}

View File

@@ -0,0 +1,20 @@
{
"Comment": "Update Public IPV4 Address",
"Changes": [
{%- for item in records.reeselink %}
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "{{ item }}.reeselink.com",
"Type": "A",
"TTL": 300,
"ResourceRecords": [
{
"Value": ""
}
]
}
}{{ ", " if not loop.last else "" }}
{%- endfor %}
]
}

View File

@@ -0,0 +1,20 @@
{
"Comment": "Update Public IPV4 Address",
"Changes": [
{%- for item in records.reeseapps %}
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "{{ item }}.reeseapps.com",
"Type": "AAAA",
"TTL": 300,
"ResourceRecords": [
{
"Value": ""
}
]
}
}{{ ", " if not loop.last else "" }}
{%- endfor %}
]
}

View File

@@ -0,0 +1,20 @@
{
"Comment": "Update Public IPV4 Address",
"Changes": [
{%- for item in records.reeselink %}
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "{{ item }}.reeselink.com",
"Type": "AAAA",
"TTL": 300,
"ResourceRecords": [
{
"Value": ""
}
]
}
}{{ ", " if not loop.last else "" }}
{%- endfor %}
]
}

View File

@@ -1,19 +0,0 @@
{
"Comment": "Update Public IPV4 Address",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "ipv4.myhost.com",
"Type": "A",
"TTL": 300,
"ResourceRecords": [
{
"Value": ""
}
]
}
}
]
}

View File

@@ -0,0 +1,7 @@
records:
reeseapps:
- nextcloud
- gitea
- git
reeselink:
- ipv4

View File

@@ -2,12 +2,12 @@
- [K3S](#k3s)
- [Guide](#guide)
- [Disable Firewalld](#disable-firewalld)
- [Firewalld](#firewalld)
- [Set SELinux to Permissive](#set-selinux-to-permissive)
- [Install K3S (Single Node)](#install-k3s-single-node)
- [Dual Stack IPv6 Support](#dual-stack-ipv6-support)
- [Single Stack IPv4](#single-stack-ipv4)
- [Kube Credentials](#kube-credentials)
- [Storage](#storage)
- [Coredns](#coredns)
- [Metal LB](#metal-lb)
- [VLAN Setup](#vlan-setup)
- [Installation](#installation)
@@ -32,47 +32,56 @@
7. Install longhorn storage for automatic PVC creation and management
8. Set up automatic database backups
## Disable Firewalld
## Firewalld
<https://docs.k3s.io/advanced#red-hat-enterprise-linux--centos--fedora>
```bash
firewall-cmd --permanent --zone=public --add-port=6443/tcp # apiserver
firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16 # pods
firewall-cmd --permanent --zone=trusted --add-source=fd02:c91e:56f4::/56 # pods
firewall-cmd --permanent --zone=trusted --add-source=10.43.0.0/16 # services
firewall-cmd --permanent --zone=trusted --add-source=fd02:c91e:56f5::/112 # services
Disable firewalld. You could add rules for each service but every time you open a port
from a container you'd need to run a firewalld rule.
You can disable firewalld from the web interface.
firewall-cmd --reload
```
## Set SELinux to Permissive
K3S is more than capable of running with SELinux set to enforcing. We won't be doing
that, however. We'll set it to permissive and you can reenable it once you've added all
the rules you need to keep your services running.
Set SELinux to permissive by editing `/etc/selinux/config`
SELINUX=permissive
Make sure to add `--selinux` to your install script.
## Install K3S (Single Node)
### Dual Stack IPv6 Support
```bash
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.30.2+k3s2 sh -s - \
"--cluster-init" \
"--flannel-ipv6-masq" \
"--disable" \
"traefik" \
"--disable" \
"servicelb" \
"--disable" \
"coredns" \
"--disable" \
"local-storage" \
"--tls-san" \
"kube.reeselink.com" \
"--cluster-cidr" \
"10.42.0.0/16,fd02:c91e:56f4::/56" \
"--service-cidr" \
"10.43.0.0/16,fd02:c91e:56f5::/112" \
"--cluster-dns" \
"fd02:c91e:56f5::10"
curl -sfL https://get.k3s.io | sh -s - \
"--disable" \
"traefik" \
"--disable" \
"servicelb" \
"--tls-san" \
"k3s.reeselink.com" \
"--flannel-ipv6-masq" \
--kubelet-arg="node-ip=::" \
"--cluster-cidr" \
"10.42.0.0/16,fd02:c91e:56f4::/56" \
"--service-cidr" \
"10.43.0.0/16,fd02:c91e:56f5::/112" \
"--cluster-dns" \
"fd02:c91e:56f5::10" \
--selinux
```
### Single Stack IPv4
```bash
curl -sfL https://get.k3s.io | sh -s - \
"--disable" \
"traefik" \
"--disable" \
"servicelb" \
"--tls-san" \
"k3s.reeselink.com" \
--selinux
```
## Kube Credentials
@@ -80,54 +89,21 @@ curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.30.2+k3s2 sh -s - \
On the operator
```bash
export KUBE_SERVER_ADDRESS="https://k3s.reeselink.com:6443"
# Copy the kube config down
scp kube:/etc/rancher/k3s/k3s.yaml ~/.kube/admin-kube-config
# Edit the server to match the remote address.
```
## Storage
1. `mkdir /var/lib/rancher/k3s/storage`
2. Edit fstab to mount your drive to `/var/lib/rancher/k3s/storage`
3. `systemctl daemon-reload`
4. `mount -a`
<https://github.com/rancher/local-path-provisioner/tree/master/deploy/chart/local-path-provisioner>
```bash
# Download the updated template from github
kubectl kustomize "github.com/rancher/local-path-provisioner/deploy?ref=v0.0.28" > kubernetes/incubating/local-path-provisioner/local-path-storage.yaml
# Apply customizations (ssd/hdd storage, read write many support)
kubectl kustomize kubernetes/incubating/local-path-provisioner/local-path-provisioner | kubectl apply -f -
# Create test pod
kubectl apply -f infrastructure/graduated/k3s/tests/local-storage-test.yaml
```
## Coredns
1. Edit `kubernetes/graduated/coredns/values.yaml` to ensure the forward nameserver is correct.
```bash
# Install CoreDNS
helm upgrade --install \
--namespace=kube-system \
--values kubernetes/graduated/coredns/values.yaml \
coredns coredns/coredns
# Test DNS works
kubectl run -it --rm \
--restart=Never \
--image=infoblox/dnstools:latest \
dnstools
ssh k3s cat /etc/rancher/k3s/k3s.yaml | \
yq -y ".clusters[0].cluster.server = \"${KUBE_SERVER_ADDRESS}\"" > \
~/.kube/admin-kube-config
```
## Metal LB
### VLAN Setup
I would remove firewalld to get this working. VLAN IPv6 traffic doesn't work for some
reason and there aren't good docs yet. Your router firewall will suffice, just be sure
to configure those rules correctly.
Before working with Metallb you'll need at least one available VLAN. On Unifi equipment
this is accomplished by creating a new network. Don't assign it to anything.
@@ -168,16 +144,18 @@ address, effectively moving the IP to another node. This isn't really "load bala
```bash
helm repo add metallb https://metallb.github.io/metallb
helm repo update
# Install metallb
helm upgrade --install metallb \
--namespace kube-system \
metallb/metallb
--namespace kube-system \
metallb/metallb
```
MetalLB doesn't know what IP addresses are available for it to allocate so we'll have
to provide it with a list. The `metallb-addresspool.yaml` has one IP address (we'll get to
IP address sharing in a second) which is an unassigned IP address not allocated to any of our
nodes. Note if you have many public IPs which all point to the same router or virtual network
you can list them. We're only going to use one because we want to port forward from our router.
MetalLB doesn't know what IP addresses are available for it to allocate so
we'll have to provide it with a list. The
[metallb-addresspool.yaml](/kubernetes/graduated/metallb/addresspool.yaml) has
the configuration for our available pools. Note these should match the VLAN you
created above.
```bash
# create the metallb allocation pool
@@ -189,13 +167,18 @@ You'll need to annotate your service as follows if you want an external IP:
```yaml
metadata:
annotations:
metallb.universe.tf/address-pool: "external"
# or
metallb.universe.tf/address-pool: "internal"
metallb.universe.tf/address-pool: "unifi-pool"
spec:
ipFamilyPolicy: SingleStack
ipFamilyPolicy: PreferDualStack
ipFamilies:
- IPv6
- IPv4
```
Then test with
```bash
kubectl apply -f systemd/graduated/k3s/tests/metallb-test.yaml
```
## External DNS
@@ -211,22 +194,24 @@ aws iam create-user --user-name "externaldns"
aws iam attach-user-policy --user-name "externaldns" --policy-arn arn:aws:iam::892236928704:policy/update-reeseapps
aws iam attach-user-policy --user-name "externaldns" --policy-arn arn:aws:iam::892236928704:policy/update-reeselink
SECRET_ACCESS_KEY=$(aws iam create-access-key --user-name "externaldns")
ACCESS_KEY_ID=$(echo $SECRET_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId')
GENERATED_ACCESS_KEY=$(aws iam create-access-key --user-name "externaldns")
ACCESS_KEY_ID=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.AccessKeyId')
SECRET_ACCESS_KEY=$(echo $GENERATED_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey')
cat <<-EOF > secrets/externaldns-credentials
[default]
aws_access_key_id = $(echo $ACCESS_KEY_ID)
aws_secret_access_key = $(echo $SECRET_ACCESS_KEY | jq -r '.AccessKey.SecretAccessKey')
aws_access_key_id = $ACCESS_KEY_ID
aws_secret_access_key = $SECRET_ACCESS_KEY
EOF
kubectl create secret generic external-dns \
--namespace kube-system --from-file secrets/externaldns-credentials
kubectl apply -f kubernetes/graduated/external-dns/sa.yaml
kubectl apply -f kubernetes/graduated/external-dns/deploy.yaml
helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/
helm upgrade --install external-dns external-dns/external-dns \
--values kubernetes/graduated/external-dns/values.yaml \
--namespace kube-system
```
### Annotation

View File

@@ -1,71 +0,0 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: ssd-test
namespace: default
spec:
storageClassName: ssd
accessModes:
- ReadWriteMany
resources:
requests:
storage: 8Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: hdd-test
namespace: default
spec:
storageClassName: hdd
accessModes:
- ReadWriteMany
resources:
requests:
storage: 8Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: local-storage-test
namespace: default
spec:
selector:
matchLabels:
app: local-storage-test
template:
metadata:
labels:
app: local-storage-test
spec:
containers:
- image: debian
command:
- bash
- -c
- 'sleep infinity'
name: local-storage-test
volumeMounts:
- mountPath: /ssd
name: ssd
- mountPath: /hdd
name: hdd
resources:
limits:
memory: "4Gi"
cpu: "2"
requests:
memory: "1Mi"
cpu: "1m"
restartPolicy: Always
volumes:
- name: hdd
persistentVolumeClaim:
claimName: hdd-test
- name: ssd
persistentVolumeClaim:
claimName: ssd-test

View File

@@ -36,9 +36,12 @@ metadata:
name: ingress-nginx-demo-1
namespace: default
annotations:
metallb.universe.tf/allow-shared-ip: "production"
metallb.universe.tf/address-pool: production
metallb.universe.tf/address-pool: "unifi-pool"
spec:
ipFamilyPolicy: PreferDualStack
ipFamilies:
- IPv6
- IPv4
type: LoadBalancer
ports:
- name: http
@@ -47,55 +50,3 @@ spec:
targetPort: 80
selector:
app.kubernetes.io/name: ingress-nginx-demo-1
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ingress-nginx-demo-2
namespace: default
spec:
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx-demo-2
strategy:
type: Recreate
template:
metadata:
labels:
app.kubernetes.io/name: ingress-nginx-demo-2
spec:
containers:
- name: httpd
image: httpd
ports:
- containerPort: 80
name: http
resources:
requests:
memory: "100Mi"
cpu: "1m"
limits:
memory: "256Mi"
cpu: "1"
---
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx-demo-2
namespace: default
annotations:
metallb.universe.tf/allow-shared-ip: "production"
metallb.universe.tf/address-pool: production
spec:
type: LoadBalancer
ports:
- name: http
protocol: TCP
port: 8002
targetPort: 80
selector:
app.kubernetes.io/name: ingress-nginx-demo-2

View File

@@ -1,5 +1,7 @@
# IPv4 Proxy
**DEPRECATED** Replaced by [Caddy](/podman/graduated/caddy/caddy.md)
This project aims to serve those without an IPv6 ISP by forwarding IPv4 requests to the
correct destination. This is accomplished by SSL preread and port mapping. This service
is intended only for publicly accessible services.

0
yq Normal file
View File