Keep getting random UTF-8 errors at the end of the terraspace execution

Just as a terraspace all up is finishing for me I periodically get a random UTF-8 encoding issue show up which causes terraspace to exit abnormally. This doesn’t happen on all runs, and sometimes on different environments being deployed in the CI/CD pipeline (gitlab cloud). The interesting thing here is that the apply has completed succesfully.

The error looks like the below.

I, [2021-06-23T17:24:19.760230 #13573]  INFO -- : Running: terraspace up kubernetes_gitops Logs: log/up/kubernetes_gitops.log
/opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/cli/logs/concern.rb:21:in `match': invalid byte sequence in UTF-8 (ArgumentError)
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/cli/logs/concern.rb:21:in `match'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/cli/logs/concern.rb:21:in `pid'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/cli/logs/concern.rb:7:in `block in readlines'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/cli/logs/concern.rb:6:in `each'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/cli/logs/concern.rb:6:in `find'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/cli/logs/concern.rb:6:in `readlines'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/summary.rb:15:in `run'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:100:in `block in summarize'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:94:in `each'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:94:in `summarize'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:56:in `deploy_batch'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:34:in `block in deploy_batches'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:31:in `each'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:31:in `each_with_index'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:31:in `deploy_batches'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:15:in `block in run'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:165:in `time_took'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/all/runner.rb:11:in `run'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/cli/all.rb:60:in `up'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/thor-1.1.0/lib/thor/command.rb:27:in `run'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/thor-1.1.0/lib/thor/invocation.rb:127:in `invoke_command'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/thor-1.1.0/lib/thor.rb:392:in `dispatch'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/command.rb:59:in `dispatch'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/thor-1.1.0/lib/thor/invocation.rb:116:in `invoke'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/thor-1.1.0/lib/thor.rb:243:in `block in subcommand'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/thor-1.1.0/lib/thor/command.rb:27:in `run'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/thor-1.1.0/lib/thor/invocation.rb:127:in `invoke_command'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/thor-1.1.0/lib/thor.rb:392:in `dispatch'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/lib/terraspace/command.rb:59:in `dispatch'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/thor-1.1.0/lib/thor/base.rb:485:in `start'
	from /opt/terraspace/embedded/lib/ruby/gems/2.7.0/gems/terraspace-0.6.11/exe/terraspace:14:in `<top (required)>'
	from /opt/terraspace/embedded/bin/terraspace:23:in `load'
	from /opt/terraspace/embedded/bin/terraspace:23:in `<main>'
Running after_script

The docker image being used has as an explicit step at the start the following

RUN set -eux \
    && apt-get update -qq \
    && apt-get install -qq -y --no-install-recommends --no-install-suggests \
      locales \
      sed \
# fix up the locale for ruby's i18n
    && sed -i 's/# \(en_US\.UTF-8 .*\)/\1/g' /etc/locale.gen \
    && locale-gen \
    && export LC_ALL="en_US.UTF-8"

To ensure that the image has UTF-8 selected by default for its locale.

Any ideas on how to resolve this?

To debug i’ve added to app.rb

require 'io/console'

Terraspace.configure do |config|
  $stdout.sync = true
  logger = Logger.new($stdout)
  logger.level = :info
  config.logger = logger
  config.test_framework = "rspec"
end

and to all the environment specific configs in config/envs/

Terraspace.configure do |config|
  config.logger.level = :debug
end

Now to note, the problem also shows up without any of the above configuration of the terraspace configuration. Was noticing this behaviour then, and the above helps debug many pieces.
(also, i’m not a Ruby programmer, so everything is a partially understood hack. Much more comfortable in Go and Python)

It’s been some time since I saw this UTF-8 error, especially when using the Terraspace Docker image (I use the Centos flavor)… the way I got around it is similar to what you did, except I utilized this env var, LANG=C.UTF-8