Encountering error with Terraspace when run from Azure DevOps Microsoft Agent

Terraspace
1

Hi BoltOps Community,

We are running Terraspace in Azure DevOps pipeline using the Microsoft-hosted agents (using ubuntu-latest). With the MS-hosted agent, everytime the pipeline runs, it starts with a “blank” VM, and we have to install Terraspace (which we’ve embedded to our process). This is how we’re installing Terraspace on the agent:

sudo gem install terraspace
sudo gem install terraspace_plugin_aws
sudo gem install bundler # This appears to be a dependency

Interestingly, it was still working around 8AM AEST today and it stopped working around 10AM AEST today. While the installation itself does not appear to error, when terraspace is invoked, it produces an error. I don’t believe this is a result of the new version of terraspace (0.6.20) - as we have pipelines running using terraspace 0.6.20 before we encountered the error.

This is the output of doing the installation (as per the installation steps above)

**Gem install terraspace output:**
Successfully installed concurrent-ruby-1.1.9
Successfully installed i18n-1.8.11
Successfully installed tzinfo-2.0.4
Successfully installed activesupport-7.0.0
Successfully installed text-table-1.2.4
Successfully installed zeitwerk-2.5.1
Successfully installed cli-format-0.2.0
Successfully installed deep_merge-1.2.1
Successfully installed rainbow-3.0.0
Successfully installed dsl_evaluator-0.1.3
Building native extensions. This could take a while...
unable to convert "\xDB" from ASCII-8BIT to UTF-8 for lib/google/protobuf/descriptor_pb.rb, skipping
Successfully installed eventmachine-1.2.7
Successfully installed eventmachine-tail-0.6.5
Successfully installed graph-2.10.0
Successfully installed rhcl-0.1.0
Successfully installed hcl_parser-0.2.1
Successfully installed tilt-2.0.10
Successfully installed render_me_pretty-0.8.3
Successfully installed nokogiri-1.12.5-x86_64-linux
Successfully installed thor-1.1.0
Successfully installed terraspace-bundler-0.4.3
Successfully installed tty-tree-0.4.0
Successfully installed aws-sdk-dynamodb-1.69.0
Successfully installed aws-sdk-secretsmanager-1.54.0
Successfully installed aws-sdk-ssm-1.126.0
Successfully installed aws_data-0.1.1
Successfully installed s3-secure-0.5.1
Successfully installed terraspace_plugin_aws-0.3.3
Successfully installed connection_pool-2.2.5
Successfully installed net-http-persistent-4.0.1
Successfully installed azure-storage-common-2.0.4
Successfully installed azure-storage-blob-2.0.3
Successfully installed azure_info-0.1.2
Successfully installed timeliness-0.3.10
Successfully installed ms_rest-0.7.6
Successfully installed ms_rest_azure-0.12.0
Successfully installed azure_mgmt_resources-0.18.2
Successfully installed azure_mgmt_storage-0.23.0
Successfully installed terraspace_plugin_azurerm-0.3.2
Successfully installed gcp_data-0.2.0
Successfully installed google-apis-cloudresourcemanager_v1-0.18.0
Successfully installed google-cloud-resource_manager-0.36.2
Successfully installed google-protobuf-3.19.1-x86_64-linux
Successfully installed googleapis-common-protos-types-1.3.0
Successfully installed grpc-1.42.0-x86_64-linux
Successfully installed googleapis-common-protos-1.3.12
Successfully installed gapic-common-0.7.0
Successfully installed grpc-google-iam-v1-1.0.0
Successfully installed google-cloud-secret_manager-v1-0.11.2
Successfully installed google-cloud-secret_manager-v1beta1-0.8.4
Successfully installed google-cloud-secret_manager-1.1.2
Successfully installed terraspace_plugin_google-0.3.2
Successfully installed rspec-support-3.10.3
Successfully installed rspec-core-3.10.1
Successfully installed diff-lcs-1.4.4
Successfully installed rspec-expectations-3.10.1
Successfully installed rspec-mocks-3.10.2
Successfully installed rspec-3.10.0
Successfully installed rspec-terraspace-0.3.0
Successfully installed terraspace-0.6.19
Parsing documentation for concurrent-ruby-1.1.9
Installing ri documentation for concurrent-ruby-1.1.9
Parsing documentation for i18n-1.8.11
Installing ri documentation for i18n-1.8.11
Parsing documentation for tzinfo-2.0.4
Installing ri documentation for tzinfo-2.0.4
Parsing documentation for activesupport-7.0.0
Installing ri documentation for activesupport-7.0.0
Parsing documentation for text-table-1.2.4
Installing ri documentation for text-table-1.2.4
Parsing documentation for zeitwerk-2.5.1
Installing ri documentation for zeitwerk-2.5.1
Parsing documentation for cli-format-0.2.0
Installing ri documentation for cli-format-0.2.0
Parsing documentation for deep_merge-1.2.1
Installing ri documentation for deep_merge-1.2.1
Parsing documentation for rainbow-3.0.0
Installing ri documentation for rainbow-3.0.0
Parsing documentation for dsl_evaluator-0.1.3
Installing ri documentation for dsl_evaluator-0.1.3
Parsing documentation for eventmachine-1.2.7
Installing ri documentation for eventmachine-1.2.7
Parsing documentation for eventmachine-tail-0.6.5
Installing ri documentation for eventmachine-tail-0.6.5
Parsing documentation for graph-2.10.0
Installing ri documentation for graph-2.10.0
Parsing documentation for rhcl-0.1.0
Installing ri documentation for rhcl-0.1.0
Parsing documentation for hcl_parser-0.2.1
Installing ri documentation for hcl_parser-0.2.1
Parsing documentation for tilt-2.0.10
Installing ri documentation for tilt-2.0.10
Parsing documentation for render_me_pretty-0.8.3
Installing ri documentation for render_me_pretty-0.8.3
Parsing documentation for nokogiri-1.12.5-x86_64-linux
Installing ri documentation for nokogiri-1.12.5-x86_64-linux
Parsing documentation for thor-1.1.0
Installing ri documentation for thor-1.1.0
Parsing documentation for terraspace-bundler-0.4.3
Installing ri documentation for terraspace-bundler-0.4.3
Parsing documentation for tty-tree-0.4.0
Installing ri documentation for tty-tree-0.4.0
Parsing documentation for aws-sdk-dynamodb-1.69.0
Installing ri documentation for aws-sdk-dynamodb-1.69.0
Parsing documentation for aws-sdk-secretsmanager-1.54.0
Installing ri documentation for aws-sdk-secretsmanager-1.54.0
Parsing documentation for aws-sdk-ssm-1.126.0
Installing ri documentation for aws-sdk-ssm-1.126.0
Parsing documentation for aws_data-0.1.1
Installing ri documentation for aws_data-0.1.1
Parsing documentation for s3-secure-0.5.1
Installing ri documentation for s3-secure-0.5.1
Parsing documentation for terraspace_plugin_aws-0.3.3
Installing ri documentation for terraspace_plugin_aws-0.3.3
Parsing documentation for connection_pool-2.2.5
Installing ri documentation for connection_pool-2.2.5
Parsing documentation for net-http-persistent-4.0.1
Installing ri documentation for net-http-persistent-4.0.1
Parsing documentation for azure-storage-common-2.0.4
Installing ri documentation for azure-storage-common-2.0.4
Parsing documentation for azure-storage-blob-2.0.3
Installing ri documentation for azure-storage-blob-2.0.3
Parsing documentation for azure_info-0.1.2
Installing ri documentation for azure_info-0.1.2
Parsing documentation for timeliness-0.3.10
Installing ri documentation for timeliness-0.3.10
Parsing documentation for ms_rest-0.7.6
Installing ri documentation for ms_rest-0.7.6
Parsing documentation for ms_rest_azure-0.12.0
Installing ri documentation for ms_rest_azure-0.12.0
Parsing documentation for azure_mgmt_resources-0.18.2
Installing ri documentation for azure_mgmt_resources-0.18.2
Parsing documentation for azure_mgmt_storage-0.23.0
Installing ri documentation for azure_mgmt_storage-0.23.0
Parsing documentation for terraspace_plugin_azurerm-0.3.2
Installing ri documentation for terraspace_plugin_azurerm-0.3.2
Parsing documentation for gcp_data-0.2.0
Installing ri documentation for gcp_data-0.2.0
Parsing documentation for google-apis-cloudresourcemanager_v1-0.18.0
Installing ri documentation for google-apis-cloudresourcemanager_v1-0.18.0
Parsing documentation for google-cloud-resource_manager-0.36.2
Installing ri documentation for google-cloud-resource_manager-0.36.2
Parsing documentation for google-protobuf-3.19.1-x86_64-linux
Installing ri documentation for google-protobuf-3.19.1-x86_64-linux
Parsing documentation for googleapis-common-protos-types-1.3.0
Installing ri documentation for googleapis-common-protos-types-1.3.0
Parsing documentation for grpc-1.42.0-x86_64-linux
Installing ri documentation for grpc-1.42.0-x86_64-linux
Parsing documentation for googleapis-common-protos-1.3.12
Installing ri documentation for googleapis-common-protos-1.3.12
Parsing documentation for gapic-common-0.7.0
Installing ri documentation for gapic-common-0.7.0
Parsing documentation for grpc-google-iam-v1-1.0.0
Installing ri documentation for grpc-google-iam-v1-1.0.0
Parsing documentation for google-cloud-secret_manager-v1-0.11.2
Installing ri documentation for google-cloud-secret_manager-v1-0.11.2
Parsing documentation for google-cloud-secret_manager-v1beta1-0.8.4
Installing ri documentation for google-cloud-secret_manager-v1beta1-0.8.4
Parsing documentation for google-cloud-secret_manager-1.1.2
Installing ri documentation for google-cloud-secret_manager-1.1.2
Parsing documentation for terraspace_plugin_google-0.3.2
Installing ri documentation for terraspace_plugin_google-0.3.2
Parsing documentation for rspec-support-3.10.3
Installing ri documentation for rspec-support-3.10.3
Parsing documentation for rspec-core-3.10.1
Installing ri documentation for rspec-core-3.10.1
Parsing documentation for diff-lcs-1.4.4
Installing ri documentation for diff-lcs-1.4.4
Parsing documentation for rspec-expectations-3.10.1
Installing ri documentation for rspec-expectations-3.10.1
Parsing documentation for rspec-mocks-3.10.2
Installing ri documentation for rspec-mocks-3.10.2
Parsing documentation for rspec-3.10.0
Installing ri documentation for rspec-3.10.0
Parsing documentation for rspec-terraspace-0.3.0
Installing ri documentation for rspec-terraspace-0.3.0
Parsing documentation for terraspace-0.6.19
Installing ri documentation for terraspace-0.6.19
Done installing documentation for concurrent-ruby, i18n, tzinfo, activesupport, text-table, zeitwerk, cli-format, deep_merge, rainbow, dsl_evaluator, eventmachine, eventmachine-tail, graph, rhcl, hcl_parser, tilt, render_me_pretty, nokogiri, thor, terraspace-bundler, tty-tree, aws-sdk-dynamodb, aws-sdk-secretsmanager, aws-sdk-ssm, aws_data, s3-secure, terraspace_plugin_aws, connection_pool, net-http-persistent, azure-storage-common, azure-storage-blob, azure_info, timeliness, ms_rest, ms_rest_azure, azure_mgmt_resources, azure_mgmt_storage, terraspace_plugin_azurerm, gcp_data, google-apis-cloudresourcemanager_v1, google-cloud-resource_manager, google-protobuf, googleapis-common-protos-types, grpc, googleapis-common-protos, gapic-common, grpc-google-iam-v1, google-cloud-secret_manager-v1, google-cloud-secret_manager-v1beta1, google-cloud-secret_manager, terraspace_plugin_google, rspec-support, rspec-core, diff-lcs, rspec-expectations, rspec-mocks, rspec, rspec-terraspace, terraspace after 69 seconds
59 gems installed


**Gem install terraspace_plugin_aws output:**
Successfully installed terraspace_plugin_aws-0.3.3
Parsing documentation for terraspace_plugin_aws-0.3.3
Done installing documentation for terraspace_plugin_aws after 0 seconds
1 gem installed


**Gem install bundler output:**
Successfully installed bundler-2.2.33
Parsing documentation for bundler-2.2.33
Installing ri documentation for bundler-2.2.33
Done installing documentation for bundler after 1 seconds
1 gem installed

But this is what we’re getting when terraspace is invoked. E.g. running terraspace version

/var/lib/gems/2.7.0/gems/activesupport-7.0.0/lib/active_support/xml_mini.rb:184:in `current_thread_backend': uninitialized constant ActiveSupport::XmlMini::IsolatedExecutionState (NameError)
	from /var/lib/gems/2.7.0/gems/activesupport-7.0.0/lib/active_support/xml_mini.rb:103:in `backend='
	from /var/lib/gems/2.7.0/gems/activesupport-7.0.0/lib/active_support/xml_mini.rb:201:in `<module:ActiveSupport>'
	from /var/lib/gems/2.7.0/gems/activesupport-7.0.0/lib/active_support/xml_mini.rb:11:in `<top (required)>'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.1/lib/zeitwerk/kernel.rb:35:in `require'
	from /var/lib/gems/2.7.0/gems/activesupport-7.0.0/lib/active_support/core_ext/hash/conversions.rb:3:in `<top (required)>'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.1/lib/zeitwerk/kernel.rb:35:in `require'
	from /var/lib/gems/2.7.0/gems/activesupport-7.0.0/lib/active_support/core_ext/hash.rb:3:in `<top (required)>'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.1/lib/zeitwerk/kernel.rb:35:in `require'
	from /var/lib/gems/2.7.0/gems/terraspace-0.6.19/lib/terraspace.rb:10:in `<top (required)>'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
	from /var/lib/gems/2.7.0/gems/terraspace-0.6.19/exe/terraspace:11:in `<top (required)>'
	from /usr/local/bin/terraspace:23:in `load'
	from /usr/local/bin/terraspace:23:in `<main>'

I’ve also tried installing an older version of terraspace using sudo gem install terraspace -v 0.6.19, and it still produces the same/similar error. I suspect it might be something that’s changed on the MS-hosted agent side, but I’m not sure. However, I was wondering if the error message might mean anything to anyone, and there is a solution for it?

Many thanks,
James

2

Hi again,

I’ve compared the output of previous successful installation vs the recent installation (which have been failing), and I noticed that where the installation was successful, when running sudo gem install, it installed this particular version of activesupport
Successfully installed activesupport-6.1.4.4

But on the install that was failing, it had
Successfully installed activesupport-7.0.0

I wonder if it’s the new version of activesupport that is causing the installation to fail?

P.S. Doing sudo gem install terraspace -v 0.6.19 in recent runs are also installing activesupport-7.0.0 (i.e. the change is likely to be unrelated to the version change in terraspace, but perhaps related to something that’s changed in the MS-hosted agents)

Many thanks,
James

3

Hello,

Another update from me… I tried to see if I could use the shim wrapper to call terraspace (after the install), but it didn’t work (because the config/app.rb doesn’t get created by the installation). I’ve tried to work around this by manually creating the config/app.rb and calling terraspace with the shim wrapper (e.g. terraspace version). Again, this didn’t work because it Could not locate Gemfile or .bundle/ directory (which I normally run e.g. terraspace new project myproject --plugin aws to automatically generate them)

Cheers,
James

4

Thanks for the constructive reports as always. :+1:

Looks like activesupport 7 added an isolated_execution_state component that is causing issues. It’s a bummer. It’s probably pretty hard for activesupport developers to account for this. Details of how I addressed this are in this PR:

Give terraspace 0.6.21 a try and hope that fixes the issue. :crossed_fingers:

1 Like
5

Hi Tung,

What a champion and the miracle worker as always :smiley:

Thanks for the quick reply and turnaround. I’ll give it a quick whirl and will let you know shortly

Cheers,
James

1 Like
6

You’re welcome Tung!

A quick update… Can confirm that 0.6.21 has resolved the issue. Thanks again for resolving the issue so quickly :smiley:

Wishing you and your loved ones a Merry Xmas and Happy New Year!

Cheers,
James

1 Like
7

Hi Tung,

Sorry to be reviving an old thread, but I’ve encountered an error with Terraspace again when running it in Azure DevOps Microsoft hosted agent. I wonder if it’s something that’s been updated in the MS hosted agent (e.g. a ruby upgrade) that might be causing the installation of Terraspace to fail.

This is an output of an installation of Terraspace that is now failing (installation steps is exactly the same as the first post in this thread)

##[section]Starting: Terraspace : install
==============================================================================
Task         : Bash
Description  : Run a Bash script on macOS, Linux, or Windows
Version      : 3.195.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
========================== Starting Command Output ===========================
[command]/usr/bin/bash /home/vsts/work/_temp/9a987806-4296-454b-85f2-9a714f360b61.sh
Installing latest version of terraspace
Successfully installed concurrent-ruby-1.1.9
Successfully installed i18n-1.8.11
Successfully installed tzinfo-2.0.4
Successfully installed activesupport-7.0.1
Successfully installed text-table-1.2.4
Successfully installed zeitwerk-2.5.3
Successfully installed cli-format-0.2.1
Successfully installed deep_merge-1.2.2
Successfully installed rainbow-3.1.0
Successfully installed dsl_evaluator-0.1.3
Building native extensions. This could take a while...
Successfully installed eventmachine-1.2.7
Successfully installed eventmachine-tail-0.6.5
Successfully installed graph-2.10.0
Successfully installed rhcl-0.1.0
Successfully installed hcl_parser-0.2.1
Successfully installed tilt-2.0.10
Successfully installed render_me_pretty-0.8.4
Successfully installed nokogiri-1.13.0-x86_64-linux
Successfully installed thor-1.2.1
Successfully installed terraspace-bundler-0.4.4
Successfully installed tty-tree-0.4.0
Successfully installed rspec-support-3.10.3
Successfully installed rspec-core-3.10.1
Successfully installed diff-lcs-1.5.0
Successfully installed rspec-expectations-3.10.1
Successfully installed rspec-mocks-3.10.2
Successfully installed rspec-3.10.0
Successfully installed rspec-terraspace-0.3.1
Successfully installed terraspace-1.0.0
Parsing documentation for concurrent-ruby-1.1.9
Installing ri documentation for concurrent-ruby-1.1.9
Parsing documentation for i18n-1.8.11
Installing ri documentation for i18n-1.8.11
Parsing documentation for tzinfo-2.0.4
Installing ri documentation for tzinfo-2.0.4
Parsing documentation for activesupport-7.0.1
Installing ri documentation for activesupport-7.0.1
Parsing documentation for text-table-1.2.4
Installing ri documentation for text-table-1.2.4
Parsing documentation for zeitwerk-2.5.3
Installing ri documentation for zeitwerk-2.5.3
Parsing documentation for cli-format-0.2.1
Installing ri documentation for cli-format-0.2.1
Parsing documentation for deep_merge-1.2.2
Installing ri documentation for deep_merge-1.2.2
Parsing documentation for rainbow-3.1.0
Installing ri documentation for rainbow-3.1.0
Parsing documentation for dsl_evaluator-0.1.3
Installing ri documentation for dsl_evaluator-0.1.3
Parsing documentation for eventmachine-1.2.7
Installing ri documentation for eventmachine-1.2.7
Parsing documentation for eventmachine-tail-0.6.5
Installing ri documentation for eventmachine-tail-0.6.5
Parsing documentation for graph-2.10.0
Installing ri documentation for graph-2.10.0
Parsing documentation for rhcl-0.1.0
Installing ri documentation for rhcl-0.1.0
Parsing documentation for hcl_parser-0.2.1
Installing ri documentation for hcl_parser-0.2.1
Parsing documentation for tilt-2.0.10
Installing ri documentation for tilt-2.0.10
Parsing documentation for render_me_pretty-0.8.4
Installing ri documentation for render_me_pretty-0.8.4
Parsing documentation for nokogiri-1.13.0-x86_64-linux
Installing ri documentation for nokogiri-1.13.0-x86_64-linux
Parsing documentation for thor-1.2.1
Installing ri documentation for thor-1.2.1
Parsing documentation for terraspace-bundler-0.4.4
Installing ri documentation for terraspace-bundler-0.4.4
Parsing documentation for tty-tree-0.4.0
Installing ri documentation for tty-tree-0.4.0
Parsing documentation for rspec-support-3.10.3
Installing ri documentation for rspec-support-3.10.3
Parsing documentation for rspec-core-3.10.1
Installing ri documentation for rspec-core-3.10.1
Parsing documentation for diff-lcs-1.5.0
Installing ri documentation for diff-lcs-1.5.0
Parsing documentation for rspec-expectations-3.10.1
Installing ri documentation for rspec-expectations-3.10.1
Parsing documentation for rspec-mocks-3.10.2
Installing ri documentation for rspec-mocks-3.10.2
Parsing documentation for rspec-3.10.0
Installing ri documentation for rspec-3.10.0
Parsing documentation for rspec-terraspace-0.3.1
Installing ri documentation for rspec-terraspace-0.3.1
Parsing documentation for terraspace-1.0.0
Installing ri documentation for terraspace-1.0.0
Done installing documentation for concurrent-ruby, i18n, tzinfo, activesupport, text-table, zeitwerk, cli-format, deep_merge, rainbow, dsl_evaluator, eventmachine, eventmachine-tail, graph, rhcl, hcl_parser, tilt, render_me_pretty, nokogiri, thor, terraspace-bundler, tty-tree, rspec-support, rspec-core, diff-lcs, rspec-expectations, rspec-mocks, rspec, rspec-terraspace, terraspace after 27 seconds
29 gems installed
Successfully installed aws-sdk-core-3.125.2
Successfully installed aws-sdk-dynamodb-1.70.0
Successfully installed aws-sdk-secretsmanager-1.55.0
Successfully installed aws-sdk-ssm-1.127.0
Successfully installed aws_data-0.1.1
Successfully installed s3-secure-0.6.1
Successfully installed terraspace_plugin_aws-0.3.6
Parsing documentation for aws-sdk-core-3.125.2
Installing ri documentation for aws-sdk-core-3.125.2
Parsing documentation for aws-sdk-dynamodb-1.70.0
Installing ri documentation for aws-sdk-dynamodb-1.70.0
Parsing documentation for aws-sdk-secretsmanager-1.55.0
Installing ri documentation for aws-sdk-secretsmanager-1.55.0
Parsing documentation for aws-sdk-ssm-1.127.0
Installing ri documentation for aws-sdk-ssm-1.127.0
Parsing documentation for aws_data-0.1.1
Installing ri documentation for aws_data-0.1.1
Parsing documentation for s3-secure-0.6.1
Installing ri documentation for s3-secure-0.6.1
Parsing documentation for terraspace_plugin_aws-0.3.6
Installing ri documentation for terraspace_plugin_aws-0.3.6
Done installing documentation for aws-sdk-core, aws-sdk-dynamodb, aws-sdk-secretsmanager, aws-sdk-ssm, aws_data, s3-secure, terraspace_plugin_aws after 9 seconds
7 gems installed
Successfully installed bundler-2.3.4
Parsing documentation for bundler-2.3.4
Installing ri documentation for bundler-2.3.4
Done installing documentation for bundler after 0 seconds
1 gem installed

/var/lib/gems/2.7.0/gems/zeitwerk-2.5.3/lib/zeitwerk/loader.rb:475:in `block (3 levels) in raise_if_conflicting_directory': loader (Zeitwerk::Error)

#<Zeitwerk::Loader:0x000055cecd5be4a0
 @autoloaded_dirs=[],
 @autoloads={},
 @collapse_dirs=#<Set: {}>,
 @collapse_glob_patterns=#<Set: {}>,
 @eager_load_exclusions=#<Set: {}>,
 @eager_loaded=false,
 @ignored_glob_patterns=#<Set: {}>,
 @ignored_paths=#<Set: {}>,
 @inflector=#<Terraspace::Autoloader::Inflector:0x000055cecd5be040>,
 @initialized_at=2022-01-11 08:57:58.54685738 +0000,
 @lazy_subdirs={},
 @logger=nil,
 @mutex=#<Thread::Mutex:0x000055cecd5be090>,
 @mutex2=#<Thread::Mutex:0x000055cecd5be068>,
 @on_load_callbacks={},
 @on_setup_callbacks=[],
 @on_unload_callbacks={},
 @reloading_enabled=false,
 @root_dirs={},
 @setup=false,
 @tag="e40079",
 @to_unload={}>


wants to manage directory /var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib, which is already managed by

#<Zeitwerk::Loader:0x000055cecd006800
 @autoloaded_dirs=[],
 @autoloads=
  {"/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/templates"=>
    [Object, :Templates],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/util.rb"=>
    [Terraspace, :Util],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/terraform"=>
    [Terraspace, :Terraform],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/dependency"=>
    [Terraspace, :Dependency],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/ext"=>
    [Terraspace, :Ext],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/core.rb"=>
    [Terraspace, :Core],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/layering.rb"=>
    [Terraspace, :Layering],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/all"=>
    [Terraspace, :All],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/version.rb"=>
    [Terraspace, :VERSION],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/cli.rb"=>
    [Terraspace, :CLI],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/shell.rb"=>
    [Terraspace, :Shell],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/plugin.rb"=>
    [Terraspace, :Plugin],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/logger.rb"=>
    [Terraspace, :Logger],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/compiler"=>
    [Terraspace, :Compiler],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/command.rb"=>
    [Terraspace, :Command],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/app.rb"=>
    [Terraspace, :App],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/builder.rb"=>
    [Terraspace, :Builder],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/booter.rb"=>
    [Terraspace, :Booter],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/autodetect.rb"=>
    [Terraspace, :Autodetect],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/tester.rb"=>
    [Terraspace, :Tester],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/mod.rb"=>
    [Terraspace, :Mod],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/hooks"=>
    [Terraspace, :Hooks],
   "/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/seeder.rb"=>
    [Terraspace, :Seeder]},
 @collapse_dirs=#<Set: {}>,
 @collapse_glob_patterns=#<Set: {}>,
 @eager_load_exclusions=#<Set: {}>,
 @eager_loaded=false,
 @ignored_glob_patterns=
  #<Set: {"/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/ext.rb"}>,
 @ignored_paths=
  #<Set: {"/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/ext.rb"}>,
 @inflector=
  #<Terraspace::Autoloader::Inflector:0x000055cecd006260 @overrides={}>,
 @initialized_at=2022-01-11 08:57:58.005192738 +0000,
 @lazy_subdirs=
  {"Templates"=>["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/templates"],
   "Terraspace::Terraform"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/terraform"],
   "Terraspace::Dependency"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/dependency"],
   "Terraspace::Ext"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/ext"],
   "Terraspace::All"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/all"],
   "Terraspace::Builder"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/builder"],
   "Terraspace::Mod"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/mod"],
   "Terraspace::App"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/app"],
   "Terraspace::Shell"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/shell"],
   "Terraspace::Compiler"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/compiler"],
   "Terraspace::Plugin"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/plugin"],
   "Terraspace::Logger"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/logger"],
   "Terraspace::Tester"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/tester"],
   "Terraspace::CLI"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/cli"],
   "Terraspace::Seeder"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/seeder"],
   "Terraspace::Util"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/util"],
   "Terraspace::Hooks"=>
    ["/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/hooks"]},
 @logger=nil,
 @mutex=#<Thread::Mutex:0x000055cecd0062b0>,
 @mutex2=#<Thread::Mutex:0x000055cecd006288>,
 @on_load_callbacks={},
 @on_setup_callbacks=[],
 @on_unload_callbacks={},
 @reloading_enabled=false,
 @root_dirs={"/var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib"=>Object},
 @setup=true,
 @tag="82a586",
 @to_unload={}>

	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.3/lib/zeitwerk/loader.rb:469:in `each'
	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.3/lib/zeitwerk/loader.rb:469:in `block (2 levels) in raise_if_conflicting_directory'
	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.3/lib/zeitwerk/loader.rb:464:in `each'
	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.3/lib/zeitwerk/loader.rb:464:in `block in raise_if_conflicting_directory'
	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.3/lib/zeitwerk/loader.rb:463:in `synchronize'
	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.3/lib/zeitwerk/loader.rb:463:in `raise_if_conflicting_directory'
	from /var/lib/gems/2.7.0/gems/zeitwerk-2.5.3/lib/zeitwerk/loader/config.rb:115:in `push_dir'
	from /var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace/autoloader.rb:27:in `setup'
	from /var/lib/gems/2.7.0/gems/terraspace-1.0.0/lib/terraspace.rb:6:in `<top (required)>'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
	from /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
	from /var/lib/gems/2.7.0/gems/terraspace-1.0.0/exe/terraspace:11:in `<top (required)>'
	from /usr/local/bin/terraspace:23:in `load'
	from /usr/local/bin/terraspace:23:in `<main>'
Installed Terraspace 
##[section]Finishing: Terraspace : install

This is an output of a previously working Terraspace installation

##[section]Starting: Terraspace : install
==============================================================================
Task         : Bash
Description  : Run a Bash script on macOS, Linux, or Windows
Version      : 3.195.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
Generating script.
========================== Starting Command Output ===========================
[command]/usr/bin/bash /home/vsts/work/_temp/c5dbf078-0015-4175-a614-d7dd9ae7e347.sh
Installing latest version of terraspace
Successfully installed concurrent-ruby-1.1.9
Successfully installed i18n-1.8.11
Successfully installed tzinfo-2.0.4
Successfully installed activesupport-7.0.1
Successfully installed text-table-1.2.4
Successfully installed zeitwerk-2.5.3
Successfully installed cli-format-0.2.1
Successfully installed deep_merge-1.2.2
Successfully installed rainbow-3.0.0
Successfully installed dsl_evaluator-0.1.3
Building native extensions. This could take a while...
Successfully installed eventmachine-1.2.7
Successfully installed eventmachine-tail-0.6.5
Successfully installed graph-2.10.0
Successfully installed rhcl-0.1.0
Successfully installed hcl_parser-0.2.1
Successfully installed tilt-2.0.10
Successfully installed render_me_pretty-0.8.4
Successfully installed nokogiri-1.13.0-x86_64-linux
Successfully installed thor-1.2.1
Successfully installed terraspace-bundler-0.4.4
Successfully installed tty-tree-0.4.0
Successfully installed rspec-support-3.10.3
Successfully installed rspec-core-3.10.1
Successfully installed diff-lcs-1.5.0
Successfully installed rspec-expectations-3.10.1
Successfully installed rspec-mocks-3.10.2
Successfully installed rspec-3.10.0
Successfully installed rspec-terraspace-0.3.1
Successfully installed terraspace-1.0.0
Parsing documentation for concurrent-ruby-1.1.9
Installing ri documentation for concurrent-ruby-1.1.9
Parsing documentation for i18n-1.8.11
Installing ri documentation for i18n-1.8.11
Parsing documentation for tzinfo-2.0.4
Installing ri documentation for tzinfo-2.0.4
Parsing documentation for activesupport-7.0.1
Installing ri documentation for activesupport-7.0.1
Parsing documentation for text-table-1.2.4
Installing ri documentation for text-table-1.2.4
Parsing documentation for zeitwerk-2.5.3
Installing ri documentation for zeitwerk-2.5.3
Parsing documentation for cli-format-0.2.1
Installing ri documentation for cli-format-0.2.1
Parsing documentation for deep_merge-1.2.2
Installing ri documentation for deep_merge-1.2.2
Parsing documentation for rainbow-3.0.0
Installing ri documentation for rainbow-3.0.0
Parsing documentation for dsl_evaluator-0.1.3
Installing ri documentation for dsl_evaluator-0.1.3
Parsing documentation for eventmachine-1.2.7
Installing ri documentation for eventmachine-1.2.7
Parsing documentation for eventmachine-tail-0.6.5
Installing ri documentation for eventmachine-tail-0.6.5
Parsing documentation for graph-2.10.0
Installing ri documentation for graph-2.10.0
Parsing documentation for rhcl-0.1.0
Installing ri documentation for rhcl-0.1.0
Parsing documentation for hcl_parser-0.2.1
Installing ri documentation for hcl_parser-0.2.1
Parsing documentation for tilt-2.0.10
Installing ri documentation for tilt-2.0.10
Parsing documentation for render_me_pretty-0.8.4
Installing ri documentation for render_me_pretty-0.8.4
Parsing documentation for nokogiri-1.13.0-x86_64-linux
Installing ri documentation for nokogiri-1.13.0-x86_64-linux
Parsing documentation for thor-1.2.1
Installing ri documentation for thor-1.2.1
Parsing documentation for terraspace-bundler-0.4.4
Installing ri documentation for terraspace-bundler-0.4.4
Parsing documentation for tty-tree-0.4.0
Installing ri documentation for tty-tree-0.4.0
Parsing documentation for rspec-support-3.10.3
Installing ri documentation for rspec-support-3.10.3
Parsing documentation for rspec-core-3.10.1
Installing ri documentation for rspec-core-3.10.1
Parsing documentation for diff-lcs-1.5.0
Installing ri documentation for diff-lcs-1.5.0
Parsing documentation for rspec-expectations-3.10.1
Installing ri documentation for rspec-expectations-3.10.1
Parsing documentation for rspec-mocks-3.10.2
Installing ri documentation for rspec-mocks-3.10.2
Parsing documentation for rspec-3.10.0
Installing ri documentation for rspec-3.10.0
Parsing documentation for rspec-terraspace-0.3.1
Installing ri documentation for rspec-terraspace-0.3.1
Parsing documentation for terraspace-1.0.0
Installing ri documentation for terraspace-1.0.0
Done installing documentation for concurrent-ruby, i18n, tzinfo, activesupport, text-table, zeitwerk, cli-format, deep_merge, rainbow, dsl_evaluator, eventmachine, eventmachine-tail, graph, rhcl, hcl_parser, tilt, render_me_pretty, nokogiri, thor, terraspace-bundler, tty-tree, rspec-support, rspec-core, diff-lcs, rspec-expectations, rspec-mocks, rspec, rspec-terraspace, terraspace after 23 seconds
29 gems installed
Successfully installed aws-sdk-core-3.125.2
Successfully installed aws-sdk-dynamodb-1.70.0
Successfully installed aws-sdk-secretsmanager-1.55.0
Successfully installed aws-sdk-ssm-1.127.0
Successfully installed aws_data-0.1.1
Successfully installed s3-secure-0.6.1
Successfully installed terraspace_plugin_aws-0.3.6
Parsing documentation for aws-sdk-core-3.125.2
Installing ri documentation for aws-sdk-core-3.125.2
Parsing documentation for aws-sdk-dynamodb-1.70.0
Installing ri documentation for aws-sdk-dynamodb-1.70.0
Parsing documentation for aws-sdk-secretsmanager-1.55.0
Installing ri documentation for aws-sdk-secretsmanager-1.55.0
Parsing documentation for aws-sdk-ssm-1.127.0
Installing ri documentation for aws-sdk-ssm-1.127.0
Parsing documentation for aws_data-0.1.1
Installing ri documentation for aws_data-0.1.1
Parsing documentation for s3-secure-0.6.1
Installing ri documentation for s3-secure-0.6.1
Parsing documentation for terraspace_plugin_aws-0.3.6
Installing ri documentation for terraspace_plugin_aws-0.3.6
Done installing documentation for aws-sdk-core, aws-sdk-dynamodb, aws-sdk-secretsmanager, aws-sdk-ssm, aws_data, s3-secure, terraspace_plugin_aws after 7 seconds
7 gems installed
Successfully installed bundler-2.3.4
Parsing documentation for bundler-2.3.4
Installing ri documentation for bundler-2.3.4
Done installing documentation for bundler after 0 seconds
1 gem installed

Installed Terraspace 1.0.0
##[section]Finishing: Terraspace : install

I’ve done a comparison on both outputs and there is a discrepancy between the failed install (Successfully installed rainbow-3.1.0) vs the successful install (Successfully installed rainbow-3.0.0) - though the error in the failed install seemed to be related to zeitwerk-2.5.3 (which is exactly the same version on both installs).

Appreciate your time and help on this (as always!)

Many thanks,
James

8

Bummer. Unsure why it used to work before and now it does not. It seems environmental. Tricky to figure this one out without an actual full-reproduction. These are guesses here and shots in the dark.

Unsure what the install process is. Saw gem install terraspace above, and am guessing that apt-get install ruby ruby-dev -y is being used. This is based on the path /var/lib/gems/2.7.0 in the output. It looks like a system-level ruby has been installed and used.

Usually, do not use system-level ruby provided by OS installers like apt-get because they’re typically out-of-date and not well maintained. Totally get why folks use apt-get though, it’s easy and fast, and one less thing to deal with. Just pointing out the cons.

Dug into this a bit and tried a few things.

  1. Docker container with ubuntu. Ran into issues with the system ruby and openssl with system ruby. Didn’t think that rabbit hole would be worth it, so moved on.
  2. Microsoft-hosted agents. Dug into this a little bit also and actual set up a new account. Looks like it takes a few days even to get approved for parallelism and run jobs. :man_facepalming:t2: Will probably not return to this.

Here are some suggestions to try and possibly help.

  1. Try the standalone installer. https://terraspace.cloud/docs/install/standalone/ubuntu/ It includes pre-compiled ruby and will be decently fast to install as part of a pipeline. Though, generally, prefer installing terraspace as a gem for control, maybe the standalone installer is good here.
  2. Instead of using the system ruby, use something like rvm to install ruby and then install terraspace. With rvm, some ruby versions will use pre-compiled rubies. Test to see which versions are available (showing a test below) The reason am actually considering rvm here instead of rbenv as suggested per docs https://terraspace.cloud/docs/install/ruby/ is because speed is a consideration for pipelines. Dont think rbenv uses precompiled binaries.

Example of rvm install using a pre-compiled binaries:

root@501d335a0934:/# rvm install 3.0
Searching for binary rubies, this might take some time.
Found remote file https://rvm_io.global.ssl.fastly.net/binaries/ubuntu/20.04/x86_64/ruby-3.0.0.tar.bz2
1 Like
9

Hi Tung,

Really appreciate the time and effort you’ve put into looking into this for us. Interestingly enough, it’s now managed to resolve itself (without us doing anything) - so as suspected, it must’ve been something in the MS agent that was causing the issue (though I don’t know what it is).

Ruby is installed on the MS agent by default, and is not something that we install ourselves. When we run the pipeline, it spins us a fresh ubuntu server (with pre-installed software) and we literally only run the following commands for the terraspace install

sudo gem install terraspace
sudo gem install terraspace_plugin_aws
sudo gem install bundler # This appears to be a dependency

(we’ve left the gem install bundler there as we found that it was previously required. i.e. we’ve not checked if that’s still required with the new version of terraspace).

We’ll give the standalone installer a try too (for comparison or something to keep in our back pocket - as the current terraspace install steps takes just a little over a minute to complete)

Many thanks,
James

10

Cool. Glad to hear it worked out.

Yup. bundler was added as a dependency a while ago. https://github.com/boltops-tools/terraspace/blob/f1507ff9ccb5daee7ee4eb64639bc5049a9b612b/terraspace.gemspec#L22 So no longer needed to be installed separately on new systems.

Noticed that ri documentation is being installed above. You can dramatically speed up the install process by using

sudo gem install terraspace --no-document
sudo gem install terraspace_plugin_aws --no-document

If you don’t want to remember to the --no-document option, you can create a ~/.gemrc file and then install the gems.

cat << EOF > ~/.gemrc
---
:backtrace: false
:bulk_threshold: 1000
:sources:
- https://rubygems.org
:update_sources: true
:verbose: true
benchmark: false
install: "--no-document"
update: "--no-document"
EOF
sudo gem install terraspace
sudo gem install terraspace_plugin_aws
1 Like
11

Just a quick follow-up from my previous post…

It turns out that the standalone installation does not work on MS hosted agents (and I just remembered that we’ve tried this in the past too). i.e. when we do

sudo echo "deb https://apt.boltops.com stable main" > /etc/apt/sources.list.d/boltops.list

we get

/home/vsts/work/_temp/ebf836f3-1174-4b2d-b244-a206e65491c0.sh: line 2: /etc/apt/sources.list.d/boltops.list: Permission denied

therefore, we end up with

E: Unable to locate package terraspace

So I guess we’ll be sticking with the gem install :slight_smile:

Cheers,
James