Nested Stack Folders

Thanks for such a cool tool. Just wondering if terraspace supports nested folders in stacks. In app/stacks I want to group resources but still have separate state. Example, app/stacks/network which has a vpc module, security groups module etc. I notice if I do terraspace up network/vpc it plans correctly but not sure if this is supported/recommended.

Haven’t really tested nested folders much yet. So currently wouldn’t recommend it. Would like to test nested structures a little more, though. It’s a matter of time. :stopwatch:

Just from my cursory attempt, looks like the concern would be in the dependency graph. This looks strange and not sure its working as it should

vpc_id = <%= output('network/vpc.vpc_id') %>

The generators seem to work, and the up/plan work. I haven’t taken the leap testing a real apply. Will let you know what I find.

Is the alternative to just create more projects? Like have a network project. But then how do I get the state from that project?

Again thanks for a much needed tool in this space.

RE: alternatives

In short: Don’t believe there are ideal alternatives right now.

Longer answer: Like you mentioned, can see trying both of these approaches:

  1. multiple projects
  2. nested folders

Both are not quite there though. Think would need to update terraspace. Here are more thoughts to help foster that process.

1. multiple projects

The projects would be decoupled. From one perspective, this is a good thing. For example, this can be useful for teams to operate independently. IE: a network team and application-specific teams. From a security perspective, the isolation provides a guardrail.

From another perspective, it’s a bad thing. We have added the complexity of multiple projects. The mono-repo is nice at the beginning when there’s not as much code. It’s not always worth it to distribute that complexity to multiple terraspace projects yet.

One of the complexities of multiple projects is getting state. We are unable to get the state from project1 in project2 with the current output helper without triggering a detected dependency. There’s been some discussion about this. Something like:

output('vpc.vpc_id', env: 'common', dependent: false) 

And maybe dependent: false is automatically set when env doesn’t match the current set one

output('vpc.vpc_id', env: 'common')

That’s maybe how that interface would work. Maybe even a different method entirely. Unsure.

In the post, also mentioned maybe creating a helper in project2 that will look up the state from project1. At least that reduces some repetition. Think it would be better as a built-in terraspace helper, though. IE: output('vpc.vpc_id', env: 'common'). Would have to dig more into it to figure out the interface.

2. nested folders

RE: Just from my cursory attempt, looks like the concern would be in the dependency graph.

Bummer. Yup, the dependency graph likely doesn’t consider nested folders yet. Guessing there are some other devilish details that would have to test and fix. So the other approach here is to also test and add support for nested folders. Would like to also see this. Will consider PRs. Of course no sweat either way. :ok_hand:

Summary

Currently, would lean on multiple terraspace projects and defining custom helpers. Some more thoughts about Multiple Terraspace Projects here However, am unsure. Can understand that many will prefer nested folders and having more of a mono-repo, particularly at the beginning. Would like to improve both here.

Would love to have nested stacks. Is there any update on this?

I have a services directory within my stacks directory that holds multiple nested ECS services each with its own directory.

Would this be possible with the current Terraspace release?

No updates on it. There’s a open GitHub Issue https://github.com/boltops-tools/terraspace/issues/214

Unsure when will get to it. Will welcome and review PRs. No sweat either way of course. :+1: Thanks.