Potential issue regarding EOL character?

HoLengZai · September 22, 2022, 9:55am

I’m using WSL to use terraspace (I read on github that it’s now compatible with Windows too?)
I got an issue with logger.rb line 17

      out = line.force_encoding('UTF-8') =~ /\n$/ ? line : "#{line}\n"

when I run:
terraspace all up
(it “up” 3 stacks : an azure resource group, an azure virtual machine scale set, an azure virtual scale set extension)

Wondering if it’s related about using VSCode Windows to editing the files but use terraspace on WSL?
However, if I run the same command right after that issue, I got an error that my state is locked… so I use force_unlock with the id provided on the log file then I run the terraspace all up command again and I got no issue/error and the vmss and the vmss extension stacks are deployed without any issue (for the azure resource group it has been already created on the first attempt, so I got no change on the 2nd and 3rd attempt).

The weird thing is… I don’t have this issue all the time. Like I have multiple env (so different tfvars) thanks to the terraspace layering mechanism, so for other env, I don’t have any issue and the tfvars are the same except the name of the resources.

Do you have any idea of my issue, please?

Here are the logs when I got an error with UTF-8 format (so on the first attempt):

[2022-09-22T17:14:02 #9716 terraspace up vmss_linux]: e[31mException ArgumentError: invalid byte sequence in UTF-8e[0m
[2022-09-22T17:14:02 #9716 terraspace up vmss_linux]: /home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/logger.rb:17:in `format_message'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/3.1.0/logger.rb:476:in `add'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/3.1.0/logger.rb:529:in `info'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:99:in `handle_stdout'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:67:in `block (3 levels) in handle_streams'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:65:in `each'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:65:in `block (2 levels) in handle_streams'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:60:in `each'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:60:in `block in handle_streams'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/3.1.0/timeout.rb:107:in `block in timeout'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/3.1.0/timeout.rb:36:in `block in catch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/3.1.0/timeout.rb:36:in `catch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/3.1.0/timeout.rb:36:in `catch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/3.1.0/timeout.rb:123:in `timeout'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:53:in `handle_streams'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:42:in `block in popen3'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/3.1.0/open3.rb:228:in `popen_run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/3.1.0/open3.rb:103:in `popen3'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:36:in `popen3'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:26:in `shell'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/shell.rb:17:in `run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/terraform/runner.rb:53:in `block in terraform'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/hooks/builder.rb:25:in `run_hooks'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/hooks/concern.rb:6:in `run_hooks'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/terraform/runner.rb:50:in `terraform'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/terraform/runner.rb:15:in `block in run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/terraform/runner.rb:97:in `time_took'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/terraform/runner.rb:14:in `run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/cli/commander.rb:12:in `run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/cli/plan.rb:43:in `plan_only'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/cli/up.rb:44:in `plan'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/cli/up.rb:21:in `perform'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/cli/up.rb:11:in `run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:128:in `run_terraspace'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:62:in `deploy_stack'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:46:in `block (3 levels) in deploy_batch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:45:in `fork'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:45:in `block (2 levels) in deploy_batch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:43:in `each'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:43:in `block in deploy_batch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:42:in `each'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:42:in `each_slice'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:42:in `deploy_batch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:35:in `block in deploy_batches'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:33:in `each'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:33:in `each_with_index'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:33:in `deploy_batches'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:16:in `block in run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:188:in `time_took'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/all/runner.rb:12:in `run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/cli/all.rb:62:in `up'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/command.rb:76:in `dispatch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:116:in `invoke'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:243:in `block in subcommand'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/command.rb:76:in `dispatch'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/lib/terraspace/cli/concern.rb:65:in `start'
/home/my_account/.rbenv/versions/3.1.2/lib/ruby/gems/3.1.0/gems/terraspace-2.2.2/exe/terraspace:7:in `<top (required)>'
/home/my_account/.rbenv/versions/3.1.2/bin/terraspace:25:in `load'
/home/my_account/.rbenv/versions/3.1.2/bin/terraspace:25:in `<main>'

Thanks !

tung · September 27, 2022, 1:02am

Bummer. Guessing the output from terraform sometimes contains non-utf8 characters.

Think may have to add a big rescue in that area of the code and print out the output for debugging and the re-raising the error. Unsure when will get to but wanted to provide some thoughts.

sfxandy · February 24, 2023, 4:55pm

What would be really awesome is if terraspace all up allowed you to pass through the -no-color options as I find this resolves the issue when applying stacks individually

tung · February 24, 2023, 7:48pm

Sure. Will probably make this an configurable option with the default to no-color as mentioned also in Terraspace up <layer> crashes with error Exception ArgumentError: invalid byte sequence in UTF-8 - #4 by tung Thanks for the post.

tomaec · September 9, 2024, 7:09am

Was there any decision made around this>?
I also have a use case where it would be useful to use the -no-color option on all plan

tomaec · September 10, 2024, 12:44pm

come to find that using below var before commands works for my particular issue
TF_CLI_ARGS="-no-color"