Thoughts on SSM Parameter Store for configuration

Jets
1

When using the serverless framework it is considered good practice to use SSM Parameter Store for configuration (https://serverless.com/framework/docs/providers/aws/guide/variables/#reference-variables-using-the-ssm-parameter-store).

I feel like using SSM would be preferable for secret credentials to the current Dotenv system since there isn’t any obvious solution for how to sync gitignored .env files across a large number of developers. Also it doesn’t provide a solution for sharing the same configuration across multiple Ruby on Jets projects which can come in handy.

As I see it there might be two approaches for implementation:

  1. Do it like serverless - interpolate configuration at deploy time before setting lambda function environment variables
  2. Look up the parameter store at run time - this has added benefits such as allowing to change parameters / rotate credentials without re-deploying all related functions

Number 1 is probably fairly straight forward to implement while number 2 would imply some deeper thought into configuration management / caching of variables etc but could result in a more seamless experience.

An article on this subject and how it relates to serverless can be found here: https://hackernoon.com/you-should-use-ssm-parameter-store-over-lambda-env-variables-5197fc6ea45b (the author concludes that it would have been preferable if serverless would have allowed run time lookup instead of deploy time)

Do you have any thoughts on this in general? If we had build time / deploy time hooks it might be possible to add SSM support as a plugin but I really think this should be a core feature of Ruby on Jets considering it’s a core service in the AWS ecosystem. It would also be more in line with how deployment in most frameworks is currently done where you expect remote configuration to live server side and be decoupled from the local environment.

1 Like
2

A quick proof of concept of variable interpolation via Jets::Dotenv.load! monkey patch (assumes aws-sdk-ssm gem being required):

class Jets::Dotenv
  SSM_VARIABLE_REGEXP = /\$\{ssm:([a-zA-Z0-9_.\-\/]+)}/

  def load!
    variables = ::Dotenv.load(*dotenv_files)
    interpolate_ssm_variables(variables)
  end

  private

  def interpolate_ssm_variables(variables)
    interpolated_variables = variables.map do |key, value|
      interpolated_value = value.gsub(SSM_VARIABLE_REGEXP) do |match|
        fetch_ssm_value($1)
      end
      [key, interpolated_value]
    end

    interpolated_variables.each do |key, value|
      ENV[key] = value
    end

    interpolated_variables.to_h
  end

  def fetch_ssm_value(name)
    response = ssm.get_parameter(name: name, with_decryption: true)
    response.parameter.value
  rescue Aws::SSM::Errors::ParameterNotFound
    abort "Error loading .env variables. No parameter matching #{name} found on AWS SSM.".color(:red)
  end

  def ssm
    @ssm ||= Aws::SSM::Client.new
  end
end

This allows .env.staging and friends to look like:

SECRET_KEY_BASE='${ssm:/app-name/secret-key-base}'
DATABASE_URL='${ssm:/app-name/database-url}'
MY_SERVICE_URL='https://${ssm:/my-service/username}:${ssm:/my-service/password}@${ssm:/my-service/url}'

Since they no longer contain any secrets they can now be committed to source control and bypass the issue of manually synchronising the .env files between developers.

I could see some opportunities to establish conventions if going down this route. Such as allowing something like prefixing with a dot to automatically interpolate the app name (similar to how I18n.t ".my-key" works in Rails views). For example: ssm:.parameter-name could be expanded to ssm:/app-name/parameter-name. This could encourage good and consistent naming for parameters.

Note that the MY_SERVICE_URL is just a convoluted example of interpolating multiple parameters into a single ENV variable. I don’t have a real world use-case for this personally but seemed like something which would be good to support.

In this POC I just copied the interpolation syntax used by serverless, which I guess also happens to be the one used by bash, which is why we have to wrap .env variables in single quotes to avoid the $ sign disappearing. This doesn’t feel ideal so maybe another interpolation syntax would be preferable.

Finally if making a production ready version of this concept the interpolation algorithm should be optimized by using ssm.get_parameters instead of get_parameter to fetch all parameters in a single request.

3

RE: Do you have any thoughts on this in general?

Yup. Wanted to add, it’s been a time thing. Ultimately, would like to see both ssm parameter store and aws secrets manager supported.

Was also thinking it would happen at runtime as part of the Lambda Execution Context so it only gets called once upon bootup. And was thinking it would result in setting ENV variables. So from the user’s perspective, it’s just an ENV variable that is available.

However, think your POC implementation is a great win and an improvement to the simple dotenv files system now. Would be happy to review and consider the PR.

Note, compile time vs runtime has it’s pros and cons. There are some advantages doing it at compile time. Example: If param store ever goes down the lambda functions will still work on a new cold start. A disadvantage is that the deploying machine needs access to parameter store. There are probably other tradeoffs.

RE: interpolation syntax used by serverless, which I guess also happens to be the one used by bash, which is why we have to wrap .env variables in single quotes to avoid the $ sign disappearing. This doesn’t feel ideal so maybe another interpolation syntax would be preferable.

The bash notation does look a little weird but honestly don’t really have a much better suggestion right now. Maybe make the regexp so both ${var} and #{var} work :thinking:

RE: I could see some opportunities to establish conventions if going down this route.

Yes, conventions would be awesome. One convention to have is the app and JETS_ENV. So maybe:

VAR=${ssm:parameter-name}

Results in:

VAR=${ssm:/app-name/JETS_ENV/parameter-name}

And if you want to break away from the convention, then start the ssm key with a /:

VAR=${ssm:/app-name/production/parameter-name} # breaks away from convention
VAR=${ssm:/app-name/base/parameter-name} # breaks away from convention

The app-name/JETS_ENV namespace is important because it allows usage of the get_parameters_by_path call which I believe is more efficient than looping through all the pages of parameters with get_parameters. See Organizing Parameters into Hierarchies

4

PR created: https://github.com/tongueroo/jets/pull/233

Due to time constraints I’ve kept the code fairly identical to the one posted above but added specs. It will be easier to discuss and improve the code on Github if necessary.

1 Like