GitHub Actions JSON::ParserError

Terraspace
#1

Hi, thanks for creating this framework. It’s been fantastic so far. I’m in the process of setting up Terraspace deployments in CI using GitHub Actions and am running into issues that only manifest on the GitHub runner but works fine locally.

I have 2 stacks, infra and backstage and the latter depends on the first. The base.tfvars for that stack looks like this:

# Required variables:
db_name = <%= output("infra.db_name") %>

# Optional variables:
namespace = "default"

The outputs.tf for infra looks like this:

output "db_name" {
  description = "SQL database configuration"
  value       = module.sql_db.name
}

When running in GHA, I’m running into this error:

Bundle complete! 5 Gemfile dependencies, 89 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Building .terraspace-cache/us-central1/prd/stacks/backstage
JSON::ParserError: unexpected token at 'command]/home/runner/work/_temp/40a8b396-95f5-4924-a790-2f3b0684087f/terraform-bin state pull

<lot of JSON text>

Error evaluating ERB template around line 2 of: /home/runner/work/rs-backstage-web/rs-backstage-web/deployment/app/stacks/backstage/tfvars/base.tfvars:
1 # Required variables:
2 db_name = <%= output("infra.db_name") %>
3 
4 # Optional variables:
5 namespace = "default"

Original backtrace (last 8 lines):
/opt/hostedtoolcache/Ruby/3.2.0/x64/lib/ruby/3.2.0/json/common.rb:216:in `parse'
/opt/hostedtoolcache/Ruby/3.2.0/x64/lib/ruby/3.2.0/json/common.rb:216:in `parse'
/opt/hostedtoolcache/Ruby/3.2.0/x64/lib/ruby/3.2.0/json/common.rb:569:in `load'
/opt/hostedtoolcache/Ruby/3.2.0/x64/lib/ruby/gems/3.2.0/gems/terraspace-2.2.3/lib/terraspace/terraform/remote_state/fetcher.rb:120:in `read_statefile_outputs'
/opt/hostedtoolcache/Ruby/3.2.0/x64/lib/ruby/gems/3.2.0/gems/terraspace-2.2.3/lib/terraspace/terraform/remote_state/fetcher.rb:108:in `load'
/opt/hostedtoolcache/Ruby/3.2.0/x64/lib/ruby/gems/3.2.0/gems/memoist-0.16.2/lib/memoist.rb:169:in `load'
/opt/hostedtoolcache/Ruby/3.2.0/x64/lib/ruby/gems/3.2.0/gems/terraspace-2.2.3/lib/terraspace/terraform/remote_state/fetcher.rb:18:in `run'
/opt/hostedtoolcache/Ruby/3.2.0/x64/lib/ruby/gems/3.2.0/gems/terraspace-2.2.3/lib/terraspace/terraform/remote_state/fetcher.rb:23:in `output'

I have the same versions locally and running these commands never yields this failure. The GHA steps in short looks roughly like this:

      # Provision the infrastructure
      - name: Provision Infra
        run: |-
          make infra-up

      ...

      # Deploy the services to the GKE cluster
      - name: Deploy
        run: |-
          make apply

Any help or guidance would be greatly appreciated as I’m at a bit of a loss trying to figure out why this doesn’t work on GHA.

1 Like
#2

Another quick note, I was able to reproduce locally using act and printing insecure secrets loaded the full json into irb with JSON[json_str] and it didn’t have any issues either but still no idea what is triggering this only in this setup.

#3

I have encountered this same issue before in linux and this is probably the OS you are using since you referenced Github Actions. I was able to circumvent this issue by hardcoding the values in the tfvars. So instead of relying on output(“infra.db_name”), you set the hardcoded value directly in tfvars. This is not ideal since you cannot use any dependencies essentially within linux.

#4

Ok so I solved the issue but I still don’t know why this is happening exactly. Debugging in the Docker container itself I noticed some odd things like the terraform command was under a bin folder but was actually a script wrapper on the binary.

The error message sort of indicated terraform-bin in the beginning which looked suspicious. So important thing to note here is I didn’t use the recommended .github/bin/install script from the docs as it felt incomplete given that brew was not on linux by default. But I decided to go that route and set up env paths manually myself instead of using the official Terraform GitHub Action.

After doing this and wiring things up, it succeeded in building and running Terraspace. So please note that the official action seems to be incompatible with terraspace atm, but not sure why. Thanks and hope this helps anyone else stuck on similar issues. Hopefully someone can root cause and open a bug against Hashicorp or fix on this side though.

1 Like
#5

Hi @saheljalal I am facing the same issue. Could you explain in more details how you fixed this problem. Currently I am using the install script running on macos-latest. My terraspace project is located in a subdirectory called terraspace in the repo, which is part of larger IaC code base. Thanks in advance.

My modified install script

#!/bin/bash
# install terraform

brew install tfenv
tfenv install latest
tfenv use latest
terraform --version

# install terraspace
BUNDLE_GEMFILE="terraspace/Gemfile" bundle install
BUNDLE_GEMFILE="terraspace/Gemfile" bundle exec terraspace new shim
BUNDLE_GEMFILE="terraspace/Gemfile" bundle exec terraspace --version

My terraspace plan step

  • name: Terraspace Plan
    if: ${{github.event.inputs.isApply}} == false
    run: |
    cd ./terraspace
    terraspace plan ${{ inputs.name }}

Terraspace plan error

/Users/runner/hostedtoolcache/Ruby/3.1.4/x64/lib/ruby/gems/3.1.0/gems/terraspace-2.2.7/exe/terraspace:7:in `<top (required)>'
/Users/runner/hostedtoolcache/Ruby/3.1.4/x64/bin/terraspace:25:in `load'
/Users/runner/hostedtoolcache/Ruby/3.1.4/x64/bin/terraspace:25:in `<main>'
                                   ^^^^^^^^^^^^^^^^^^^^
Error evaluating ERB template around line 1 of: /Users/runner/work/ideation-machine-infra/ideation-machine-infra/terraspace/app/stacks/plt-network-hub/tfvars/base.tfvars:
1 logs = <%= output("plt-logs.outputs") %>
2 dns  = <%= output("plt-dns.outputs") %>
Original backtrace (last 8 lines):