QC: SysOps/SecOps

Quality Control Platform
1

SysOps/SecOps public discussion for the DLN Quality Control Community Project

Objectives
Updates
Community suggestions
Chit chat
& more…

2

Completed

  • Initialize enviroments (@Michael @Ulfnic)
    • Plan/communicate requests (@Ulfnic)
    • Set up sub-domains (@Michael)
    • Initialize hosting enviroments (@Michael)
  • Collab server (@Ulfnic)
    • Initial setup
    • Deploy Mumble (@Ulfnic)
    • Deploy NextCloud (@Ulfnic)
      • Apply and enforce HTTPS (@Ulfnic)
      • Accounts (@Ulfnic)
        • Set up QC group (@Ulfnic)
        • Create secure policy for password delivery (@Ulfnic)
        • Create accounts & deliver passwords (@Ulfnic)
      • Create initial shared directory structure (@Ulfnic)
      • Install requested collab addons (@Ulfnic)
        • U2F and TOTP 2FA
        • Draw io (@Ulfnic)
          • Showcase usage examples for website wireframes and UML (@Ulfnic)
        • Deck (@Ulfnic)
    • Production of initial setup guidebook (@Ulfnic)
  • Initialize SysOps/SecOps (@Shaderoit @Ulfnic)
3

Objectives

  • Further security improvement to collab server guidebook (@Shaderoit)
  • Use test server to create collab server candidate
    • Re-implementation of Mumble deployment
      • Containerization
      • Hardening
      • Load test
      • Guidebook
    • Re-implementation of NextCloud deployment
      • Upgrade to NextCloud 20
      • Containerization
      • Hardening
      • Guidebook
    • Backup procedure
      • Method
        • Encrypted at rest prior to send
      • Organize destination
      • Sheduling
      • Guidebook
    • Transition to Ansible
    • End level boss!
      • Server rebuild from scratch via Ansible
      • Data recovery, exclusively using guidebooks
4

Great progress and next steps looking sensible and promising :ok_hand:.

Q… What exactly are we classing as “sysops”?

And Q2… If we want to get a CI and some preprod/prod environments created for our apps (frontend, backend, database, whatever!) who would like to be involved?
Assuming a solution built on Ansible + GitLab CI would fit us best at the moment so expertise in those areas would be great. Either way, will all be config-as-code so anyone can review and feedback :+1:

5

Thank you :slight_smile:

Given it’s a small team definitions are going to be pretty flexible but I see our SysOps as anything that creates and maintains systems for DevOps.

If DevOps has a need SysOps will make it happen. @Shaderoit (who’s been doing a lot for SecOps behind the scenes which sadly can’t make the checklist in much detail) has also offered some assistance/advisory with SysOps. Both @kobberholm and Marius have a lot of scalable database experience so i’d probably hand those decision to them as long as everyone’s comfortable with the choice. I’ll assist creation/maintenance of the choice.

I want guide books for community consumption because YAML Playbooks are hard to read for the uninitiated. That said, the Playbooks will inform what’s in the guidebooks and it’d be good to share them both.

In future: @kobberholm/Marius will have preferences on how the backend enviroment is set up so I need to work with them, if nothing else to make sure it’s fully documented and present in a Playbook.

There’s been a lot of rapid changes and SSH has been a great quickie solution but i’d agree transitioning to Ansible is high priority. There’s plenty to do on the collab server and a lot of it will be used for preprod/prod, we’d just need to know when DevOps wants that GitLab CI/CD ready.

6

Bellow is an outline of the planned infrastructure, thoughts and opinions welcome before I go and try implementing any of this.
As an overview, all of this would be using GitLab CI/CD and their Docker Container repository to build and publish the app with Ansible controlling the actual update/deployment of the dev server.

CI Pipeline Plan

  1. Run any tests (tbc)
  2. Build docker container
  1. Publish to privte docker registry (Gitlab)
    -GitLab Container Registry | GitLab
  2. Deploy to PREPROD (devel.destinationlinux.network) using Ansible
  1. Smoke test deployment
  2. Repeat process with manual release for PROD

Mostly academic so far although I can’t see it being too difficult to implement.
Main thinking would be around:

  • Ensuring we handle keys/permissions securely for Ansible and the container repository
  • Securing Docker (ports, user, etc). Could/should we use something like podman?
  • Exposing ports
  • Handling secure env_vars (i.e. db_pass, api_token, etc.) on the servers (not a requirement, yet…)

Aim is to get a minimal viable/secure implementation we can build on over time as we require any more ‘complex’ deployment.

Will do more investigation as I can and if no big blockers will crack on trying to get something implemented over the next week or so.

1 Like
7

I passed this on to @Shaderoit, he may post but from his reaction i’d say this is a go. I’d also be glad for podman.