Bundle exec may solve this

Terraspace
1

Given that probably the best solution is always to prepend bundle exec inside terraspace project directory, we get a strange behavior with terraspace on the ruby/gem side.

On devops machines we install rbenv and terraspace 0.6.2. On these machines we use ruby “runtime” only for terraspace. No other software or developer touch ruby/gem environment.

Initially terraspace command works perfectly without the need for bundle exec, then suddenly we get errors like this:

You have already activated google-apis-core 0.3.0, but your Gemfile requires google-apis-core 0.2.1. Prepending `bundle exec` to your command may solve this.

solved prepending “bundle exec” to terraspace commands from now on

  1. Why terraspace inconsistently change gem bundle/environment introducing this error and forcing to use bundle exec ?

  2. What is the best practice to simplifying and standardizing for all developers the terraspace command ? Script wrapper ?

note: we read terraspace 0.5.0 and above no longer requires a shim Shim Wrapper

2

@Roxyrob Sounds like your Terraspace project needs it’s Gemfile.lock updated. Run the following in your Terraspace project and commit the changes:

bundle update
3

@ellisio,
bundle update worked. Thank you.

Not so clear what made terraspace project Gemfile.lock inconsistent by itself !?

1 Like
4

@Roxyrob On your build machines, are you installing latest? The new version of 0.6.2 installed globally probably differs from the Gemfile version, which may have been pinned to an older version.

I’m not a pro on Ruby, but I literally experienced this early today and that is about as far as I got into digging into the issue.

5

@ellisio Yes 0.6.2 but was the first version installed on a clean machine and for some days terraspace worked fine without prefixing bundle exec or running bundle update. Only after some days, apparently with no changes to Ruby environment (rbenv) this issue started.

6

@Roxyrob The only thing I can think of is that the project was initialized with an older version of Terraspace. When you run terraspace new project ... it initializes a Gemfile with the same version of terraspace at your global CLI level. Check it out:

❯ terraspace new project demo --plugin google --examples
=> Creating new project called demo.
      create  demo
      create  demo/.gitignore
      create  demo/Gemfile
      create  demo/README.md
      create  demo/Terrafile
      create  demo/config/app.rb
       exist  demo
      create  demo/config/terraform/backend.tf
      create  demo/config/terraform/provider.tf
=> Creating test for new module: example
      create  demo/app/modules/example
      create  demo/app/modules/example/main.tf
      create  demo/app/modules/example/outputs.tf
      create  demo/app/modules/example/variables.tf
=> Creating new stack called demo.
      create  demo/app/stacks/demo
      create  demo/app/stacks/demo/main.tf
      create  demo/app/stacks/demo/outputs.tf
      create  demo/app/stacks/demo/variables.tf
=> Installing dependencies with: bundle install

❯ cat demo/Gemfile
source "https://rubygems.org"

gem "terraspace", '~> 0.6.2'
gem "rspec-terraspace"
gem "terraspace_plugin_google"

So if your engineer had Terraspace 0.5.x installed on their machine to initialize the project, and your build machine has Terraspace 0.6.2, you’ll see this behavior.

This reminds me a lot of the early days of Yarn for Node projects…

7

@ellisio It doesn’t seems like my environment story but thank you for the example. The next same issue arise I’ll go deep on Gemfile contents.

8

@Roxyrob looks like the above is just an example. By working through that example, with 0.6.2 installed, a project I have for part of our infrastructure started experiencing this, but with a different Gem.

It’s almost like the terraspace CLI tool is updating global gem versions when running terraspace new project. Then when running, is complaining about a version mismatch in the project’s lock file.

Since I’m not versed in Ruby, I can’t explain why this is happening.

Why is the global terraspace CLI even concerned about the project’s Gem lock file?

@tung any insight on why this might be?

9

@Roxyrob Looks like if I would just RTFM some more, I could have sent you this link. It explains everything: https://terraspace.cloud/docs/misc/multiple-terraspace/

10

@ellisio Already read this link. It explain how to have multiple terraspace versions (e.g. for testing/compatibility purpose - not mentioning bundle update to re-sync ).

On my devel machine is present single version terraspace. I cannot see how this help understanding our issue.

11

The short answer is: dunno.

Longer answer: This is where a little ruby and bundler knowledge would help to at least better understand what’s going on. Guessing a gem install ... resulted in google-apis-core-0.2.1 being installed at some point. Think this could have been previous to the gem install terraspace command or maybe after. Inspected a Gemfile.lock in a newly generated project:

google-apis-iamcredentials_v1 (0.2.0)
 google-apis-core (~> 0.1)
google-apis-storage_v1 (0.3.0)
 google-apis-core (~> 0.1)

It could have been another gem upstream that was installed on the devops machines a while back or afterwards. My understanding is that when gem install, bundle install, or bundle update runs, a dependency tree calculation runs. The commands can actually calculate different dependency trees. If there’s no current Gemfile.lock, then the bundle install calculation is freer to decide whether it can reuse existing gems that are already installed on the system.

The devil-are-in-the-details, it may also depend on ruby version, bundler version, etc. The net result is that the Gemfile.lock can possibly not reflect dependencies available on the system. Think this is what probably happened. At least, that’s an attempted explanation.

In general, dependency resolution is a tough CS problem because different trees can satisfy the same constraints. As @ellisio mentioned, it surfaces elsewhere. Additional examples:

  • ruby bundler and Gemfile.lock
  • node, yarn, npm, yarn.lock, package.json
  • python and requirements.txt often with pyenv

Found that there are nuances and quirks to each language that require some familiarity with the dependency system to understand. Maybe that’s why it feels like node and yarn deja vu. :joy:

One nuance is that a gem install terraspace won’t update the Gemfile.lock. But remember that both gem install and bundle update perform a dependency tree calculation. The tree gets resolved successfully, but it may be slightly different. IE: gem install can use the existing google-apis-core-0.2.1, but bundle update can update it to google-apis-core-0.3.0 Different developers from all over the world work on these different pieces of the ruby ecosystem. So can see how commands can exhibit slightly different behavior. For most cases, and what others do in practice, is just run bundle update and move on.

Also, terraspace actually has some error handling for bundle require errors already. But for some reason, the handle_error method is not invoking. It’s defined here https://github.com/boltops-tools/terraspace/blob/a3d4310364efb4de2cfa0093fa1f3c5398b7d4a7/lib/terraspace/bundle.rb#L31 Again, maybe different bundler versions raise different exceptions. Unsure. There’s not enough context.

Anyway, updated the docs:

Also, note, this comes from a good place. Am hoping you bring a little more positive energy to the community. @ellisio is doing a great job trying to help. He’s using his own time, free of charge. There will always be ways to improve software and Terraspace. Constructiveness helps motivate me and others on this forum to help. Additionally, it fosters a healthy community. Again, this comes from a good place.

One thing that may help others help you in the future with these type of issues is providing a step-by-step reproduction. Saying nothing was done is often not the best way to elicit help. It’s more likely something did changed, we just don’t know what, and there’s unfortunately not enough context. Examples are also helpful. Letting others know ruby version, bundler versions, etc is helpful. The forums are a little more free form than the github issues to foster discussion, but there are so many things you can still do in the request to help.

And sometimes software mysteries are simply left unresolved. Life’s short. Restarting my computer and moving on rings a bell :bell:

Lastly, terraspace is it’s in early days. :grinning_face_with_smiling_eyes: Terraspace was literally released a few months ago. :tada: Regardless, been getting very positive feedback and many thank yous from appreciative users impressed with the initial feature set. Will continue to harden terraspace. Wrapping my head around a few pretty cool future ideas for this year. Good things to come.:rocket:

12

It’s always interesting to go deep on each DevOps tool and language internals but it is not always feasible. DevOps is an amazing “maze”, a recursive trip. One knows where he starts but not always where he lands :upside_down_face:

IMHO also if terraspace is in its early steps it’s a very promising solution to make terraform DRY and incresing a lot flexibility.

Said that I think every discussion and feedback foster tools and its docs improvement. As far as possible normally I provide repeatable test case of an issue to make context as clear as possible. I’ll do that too as soon as I get this issue again.

Thank you all

13

Ran into this issue and identified a cause of it. When using Terraspace generators, your local ruby/bundle configuration is updated. So in the scenario of,

  1. I ran a Terraspace generator to setup a new test project (–example), played around, went to sleep.
  2. A few days later, I ran a Terraspace generator to setup a second new test project (–example).
  3. A few days later, I returned to the original project (1.), and received this error.

The dependencies were updated during step 2 which no longer matched the configuration from step 1. I’ve yet to run into this issue since, but I haven’t used generators again.

Hope this helps anyone stumbling across this in the future.

1 Like
14

Hi @nswarner2,
I can confirm steps depicted in your issue description but how do you solved this in the first test project ?

Can be solved without using bundle exec if possible since all projects for me use same configuration ?

15

In this spec case solved with bundle update.

Step1. running

terraspace new project test1

Step2. working on project test1
Step3. after a while runing

terraspace new project test2

Step4. running terraspace -v in project test2 return

terraspace -v
0.6.6

Step5. running terraspace -v in project test1 return

terraspace -v
You have already activated i18n 1.8.10, but your Gemfile requires i18n 1.8.9.
Prepending `bundle exec` to your command may solve this.

In Gemfile not i18n version was specified.
In Gemfile.lock I found:

GEM
  remote: https://rubygems.org/
  specs:
    activesupport (6.1.3.1)
      concurrent-ruby (~> 1.0, >= 1.0.2)
      i18n (>= 1.6, < 2)
   ...
   ...
    i18n (1.8.9)
      concurrent-ruby (~> 1.0)
   ...

Solved gem inconsistency version executing

bundle update

Fetching gem metadata from https://rubygems.org/........
Resolving dependencies....
Using rake 13.0.3
Using concurrent-ruby 1.1.8
Using i18n 1.8.10 (was 1.8.9)
...
...

terraspace -v works again on project test1 and continue to work on project test2, without using bindle exec

I use ruby only for terraspace so my ruby environment is simple enough not to make things too complicated. Not sure this solution works for others.

16

Updated: Released Terraspace 1.0.0.

Improved the terraspace wrapper generated by the standalone installers. Details here:

You won’t have to remember to type bundle exec.

In addition, added a friendly usage message with instructions on how to resolve if the error occurs. PR: https://github.com/boltops-tools/terraspace/pull/177 The standalone installer’s new shim, as of terraspace 1.0.0 includes this smarter shim already.

Also, remove terraspace_plugins_* out of terraspace core. https://github.com/boltops-tools/terraspace/pull/173 This should also mitigate gem dependency graph resolution issues.

Believe this should resolve the issues. :face_with_monocle:

1 Like