Terraspace Test: Test for failures

Hey,

I have a question about the testing framework. We’ve been researching rspec and managed to set up a nice little testing framework that works for our use-cases. The only thing missing right now is the ability to test for failures. As none of us are ruby developers we mainly work by googling, but in this case we can’t seem to find a working solution.

Basically the idea is to test that specific inputs (a specific tfvars file for example) contains values that trigger validation checks in terraform of some kind. Think validation of a variable for emails that checks if there is an @ in the value, but we specify a value without. The expectation would be that “terraspace.up” fails and that we can test for this and continue.

Unfortunately it seems that I can’t find a way to actually continue after terraspace.up runs into an issue, even if we have other tests afterwards that would run another terraspace.up with working input.

From what I understand, terraspace simply exits with 1 if there was any terraform failure in the plan process. rspec-terraspace doesn’t do anything specific to handle failures either, so from my understanding of ruby we should receive a SystemExit exception in our main_spec.rb file.

I tried to encapsulate it with “begin/rescue”, but that doesn’t seem to help - the failed test may return “success”, but no other test will be evaluated afterwards.

this is a small snippet of what I am trying at the moment:

let(:status) do
    begin
      terraspace.up(@module_name)
      true
    rescue
      false
    end
  end

it "deployment" do
    expect(status).to eq test[:expect_deployment]
  end

This will detect a failure, but will still not continue to other ‘it’ further down the file. If we test for success (and terraspace.up doesn’t fail), other examples are tested as usual.

Is there anything we are missing or is there simply no way for us to test for failure with the system? We feel it’s a very important missing feature if that would be the case.

Any help is much appreciated!

Emanuel