0

I'm working on hiring for a Site Reliability Engineering role and I want to create a "broken" VM for them to log into and debug.

What sort of scenarios would you suggest I put in place to accurately test an applicant's knowledge of Linux command line and basic web server (LAMP) functionality?

We run an Ubuntu environment on AWS.

Jeremy Wilson
  • 311
  • 1
  • 3
  • 12
  • 3
    How about a real problem you've had in the past? – Michael Hampton Oct 27 '20 at 21:21
  • I was thinking more on a practical, specific level - like, is there a way to corrupt the filesystem subtly, or misconfigure something that isn't obvious at first glance and allows nginx to run but not work, or things like that. – Jeremy Wilson Oct 28 '20 at 01:02
  • break dns in a discoverable way while running stuff that needs resolution (certificates with *ocsp-must-staple* come to mind) – anx Oct 28 '20 at 12:18

1 Answers1

3

edit replace SELinux with AppArmor for Ubuntu

Here are a few suggestion that should be easy to set up in a VM (and I ran into over the years):

  • set invalid POSIX permissions for content the web server should deliver.
  • set invalid SELinux labels for content.
  • break the connection between web server and a backend server (e.g. with IPTables or SELinux)
  • as @anx suggested break DNS with /etc/hosts entries or invalid nameservers
  • create an invalid virtual host / SNI config, making the web server reply with an incorrect vhosts' content.
  • create a resource exhaustion scenario invoking the OOM-Killer.
  • Block inbound connections with firewalld/iptables/nftables/SELinux.
  • Create a config missing content - e.g. missing user and homedirectory, missing content, misspelled ip addresses or directory names, etc.
  • Block outbound connections and serve an app from the webserver that needs them.
  • Stop and disable needed services - e.g. the DB server.
  • Scramble the app's db user's password or introduce a typo.
  • Remove grants for the app db user.
  • Fill up the filesystem.
  • Introduce a failure state in the application depending on text encoding and collation (e.g. jumbled up characters, missing data).
fuero
  • 9,591
  • 1
  • 35
  • 40