Terraspace support for sops

Terraspace
1

Hi!

First of all, thank you for introducing Terraspace as a new framework for TF, I definitively love it! Now, I’m not 100% sure this is the right place for raising a discussion about features, but…

In our use case scenario, we use sops - https://github.com/mozilla/sops in combination with AWS KMS to encrypt secrets.

I wonder if this was already considered as a feature to support in Terraspace, if not, I would like to submit the feature request or suggestion.

If I’m in the wrong place, please forgive me and just point me in the right direction.

Thanks!

2

Thanks for the kind words.

RE: sops - https://github.com/mozilla/sops

Haven’t heard of it. Took a quick look. It looks like you use sops mynewtestfile.yaml and it creates a file with key values and the values are encrypted via the tool.

Unsure how it would work yet with Terraspace. Maybe a secrets helper that points to a file that can grab the secret value by calling sops somehow. Wondering if that’s what you’re thinking. Open to suggestions and open to reviewing and considering PRs. No sweat either way of course. Thanks!

3

Hi @tung , thanks for such quick reply

Yes, this a use case scenario when TFC isn’t used, in our case the backend is AWS S3 and we can’t feed sensitive values like in TFC Sensitive Environment variables or using AWS SecretsManager or ParametersStore.

Now, here is an example using sops with tg:

 db_creds         = yamldecode(sops_decrypt_file("${get_terragrunt_dir()}/../../rds/data-creds.sops.yaml"))
datadog_creds    = yamldecode(sops_decrypt_file("${get_terragrunt_dir()}/../../rds/datad-creds.sops.yaml"))

once the file has been decryped by sops, then their values can be parsed as locals

master_user            = local.db_creds.username
master_password    = local.db_creds.password
datadog_user          = local.datadog_creds.username
datadog_password  = local.datadog_creds.password

Now, sops supports several of major cloud providers and is quite useful for startups too when using gpg encryption.

I do believe a helper can do it, after reading TS documentation, I believe it can be something like:

user = "<%= sops_secret("demo-:ENV-<.key.user>", "<path-to-yaml-encrypted-file>.sops.yaml" %>"
pass = "<%= sops_secret("demo-:ENV-<.key.pass>", "<path-to-yaml-encrypted-file>.sops.yaml") %>"

And translated to a command, it will be:

sops -d <path-to-yaml-encrypted-file>.sops.yaml | yq e '<.key.user>'
sops -d <path-to-yaml-encrypted-file>.sops.yaml | yq e '<.key.pass>'

Sadly I lack the knowledge to get it done in Ruby, but if you have a document to build TS helpers, I can give it a try. It will take me some time but… I’m more than happy to help.

Best, hope it helps.

4

Thinking first would start off with a project-level helper. Here’s docs:

Generate the helper:

terraspace new helper sops --type project

Some possible starter code. Note: Haven’t tested this. Just showing an example of how it could possibly work.

config/helpers/sops_helpers.rb

module Terraspace::Project::SopsHelpers
  def sops_secret(name, options={})
    # logic to grab sops_secret from the file
    # would probably add some logic to check that sops and yq are installed
    # and print out a user-friendly message to tell the user to install them if it's not available
    `sops -d #{sops_file(options[:file])} | yq e "#{name}"`.strip
  end

  def sops_file(file=nil)
    file || ENV['SOPS_FILE'] || "config/sops.yaml"
  end
end

So thinking that mabye method interface could be something like

sops_secret("demo-:ENV-<.key.user>")

Generally set the path with an env variable for now. If this is package as a library, then maybe a config/app.rb option eventually? You can override the file on a per method call basis. But generally, it has a default so the code is DRYer.

sops_secret("demo-:ENV-<.key.user>", file: "<path-to-yaml-encrypted-file>.sops.yaml" %>"
  • One of the reasons thinking to start with a custom project-level helper for now is this helps you to get it done now.
  • Another reason, is that unsure where the sops_secret helper would live right now: a separate plugin? terraspace core itself? Or maybe even a new concept like an extension? Unsure :thinking:
1 Like