Stacks and the Configured Guardrails

Overview

Turbot provides a mechanism for managing resource configuration using Terraform. Turbot Stack and Configured controls allow you to define the configuration for a resource or set of resources using standard Terraform HCL. Turbot can apply your configuration whenever resources change, enforcing your standards and preventing configuration drift.

  • A Stack is a set of resources configured via a terraform source from Turbot.
  • A resource can configure itself using terraform via its Configured control.
A resource does not need to be configured by a Turbot stack or configured policy to be managed by other Turbot guardrails.

Stacks

A Turbot Stack is a set of resources managed by Turbot using Terraform. Turbot uses stacks to deploy, configure, and manage sets of related resources.

Each stack has a single Source policy that specifies the Terraform configuration source code. When the stack control runs, resources will be created, modified or deleted per the Source.

Stacks are responsible for the creation and deletion of multiple resources. Once created, the resources are responsible for updating themselves via the Configured control. Changes to the underlying resources will trigger the resource's Configured control to run. The Configured control will use the Source from the parent stack to re-apply its configuration, keeping it configured per the policy.

After being created by the AWS > Turbot > Event Handlers stack, the configuration of a the turbot_aws_api_handler SNS Topic is managed by its AWS > SNS > Topic > Configured control. If the topic is modified, the AWS > SNS > Topic > Configured control will re-apply its configuration using the terfafrom source defined in the AWS > Turbot > Event Handlers > Source policy

Stack Terraform Version

Turbot Stacks supports up to the most recent Terraform 0.15 version, which is 0.15.5. Using the policy family * > * > Terraform Version, users can define which version of Terraform to use. The policy Turbot > Stack Terraform Version [Default] can be used to define the Terraform version across all stack policies.

Notes:

  • While the policy allows administrators to specify any version of Terraform, only the "oldest" minor version for each patch is valid. I.e. 0.11.* and 0.11.12 are valid values, but 0.11.10 is not. In general, it is recommended to use * in conjunction with the patch version.
  • Turbot supports the different syntax of each version, but does NOT support the use of new features or modules.

Supported values for the policy * > * > Terraform Version:

  • 0.11.* or 0.11.14
  • 0.12.* or 0.12.28
  • 0.13.* or 0.13.0-beta3
  • 0.14.* or 0.14.11
  • 0.15.* or 0.15.5
  • * This value will use the latest version of Terraform.

Turbot-Defined Stacks

Turbot provides pre-defined stacks to assist with common setup and configuration tasks. Turbot-defined stacks manage common resources required to operate Turbot, as well as resources used as containers or defaults for other controls.

Turbot-defined stacks typically appear under {provider} > Turbot in the policy type hierarchy.

Turbot-defined stacks include:
  • GCP > Turbot > Event Handlers
  • Azure > Turbot > Event Handlers
  • AWS > Turbot > Event Handlers
  • AWS > Turbot > Audit Trail
  • AWS > Turbot > Logging
  • AWS > Turbot > OS Management
  • AWS > Turbot > Service Roles

The Source policy is read-only for Turbot-defined stacks, and is generated by Turbot. Sub-policies of the stack allow you to change common parameters, providing a simple interface for customizing resources created by the stack. Modifying a stack policy will result in changes in the stack source policy, which in turn will run the stack to apply the changes.

The AWS > Turbot > Logging > Bucket stack creates and manages S3 buckets used by multiple AWS services for logging. This stack is configured per the AWS > Turbot > Logging > Bucket > Source> policy, which is generated using policies that allow you to set options to customime the bucket name, tags, and other properties:
  • AWS > Turbot > Logging > Bucket > Default Encryption
  • AWS > Turbot > Logging > Bucket > Name
  • AWS > Turbot > Logging > Bucket > Name > Prefix
  • AWS > Turbot > Logging > Bucket > Regions
  • AWS > Turbot > Logging > Bucket > Tags
  • AWS > Turbot > Logging > Bucket > Versioning

User-Defined Stacks

User-defined stacks are custom Turbot stacks built entirely from user-defined Terraform source. With user-defined stacks, you can describe your configuration using standard Terraform syntax, and let Turbot manage applying the configuration, re-applying the configuration in response to changes on the resources or policies, and automatically configuring new resources as they are added.

Stack Polices

Stack behavior is controlled by the Stack policy and sub-policies.

Policy Description
Stack Determine whether to run the stack in check mode, enforce mode, or skip
Stack > Source The Terraform configuration source code that should be applied
Stack > Variables .tfvar-style variable overrides
Stack > Secret Variables .tfvar-style variable overrides for sensitive variables

The Stack policy determines what action the control will take:

Value Description
Skip The control will not run
Check: Configured A Terraform plan will be generated. If the planned configuration does not match the current configuration, the control will alarm.
Enforce: Configured A Terraform plan will be generated. If the planned configuration does not match the current configuration, the control will apply the configuration.

The Stack > Source policy contains the Terraform configuration code that should be applied.

Note that the stack expects to continue to manage any resources that were created in the stack - if you delete a resource from the Terraform configuration in the Source policy, the stack control will destroy the resource. For example, if you wish to destroy all the objects created by the stack, set the Source policy to {}, and leave the Stack policy set to Enforce: Configured.

The Stack > Variables policy can contain variable definitions in Terraform HCL, in the same way that they would use a .tfvars file.

Like Stack > Variables, the Stack > Secret Variables policy can contain variable definitions. This policy will be marked secret in Turbot, and is meant for parameters that are sensitive or confidential.

The Variables and Secret Variables policies are merged into a single set of variables that are passed as a tfvars file to Terraform by the stack control.

The Variables and Secret Variables are not required, however separating the variables from the configuration will simplify using stacks in Turbot:

  • As a best practice, you should only enter an immediate value in the Stack > Source. If calculated policies are required to get input data for the stack, the Stack > Variables should use a calculated policies to get the data and pass it in as Terraform variables.

    • This makes the source easily testable outside of Turbot, as it is not a calculated policy
    • Rendering the input variables in nunjucks is much simpler than rendering the whole Terraform source
  • Using map or object variables allows you to create a map policy in the Variables with configuration information that can be used in all child resource stacks. If a new item is added, the variables can be updated without updating the terraform configuration.

User-Defined Stack Controls

Turbot typically provides custom stack policies at the account/project/subscription for managing global resources:

  • AWS > Account > Stack
  • GCP > Project > Stack
  • Azure > Subscription > Stack

Stacks also target a Region or Resources Group, allowing you to manage locally scoped resources as well:

  • AWS > Region > Stack
  • GCP > Region > Stack
  • Azure > Resource Group > Stack

Turbot also provides service-level stacks for some services. This allows you to organize and separate your stack configurations by the types of resources that they manage. The services stacks target the region or resource group for regional services, and the "global" region for global services (Route53, IAM, etc).

  • AWS > VPC > Stack
  • AWS > SSM > Stack

Example - standard IAM service roles and users

Many organizations rely on 3rd party software or SaaS products that requires IAM users or roles to access their accounts. Performance monitoring tools are an example -- typically, an IAM cross-account role has to be created in all AWS accounts in the enterprise to allow the tool access.

You can user Turbot Stacks to simplify the creation and management of these roles across all of your AWS Accounts. Using a Stack targeting AWS accounts, simply define the configuration for the IAM role using Terraform. Turbot will create it in all your AWS accounts. If the vendor adds new features that require additional access for the role, you can simply modify the Stack > Source policy, and Turbot will deploy the changes. If you add new AWS Accounts, Turbot automatically runs your stack, making it consistent and compliant with your standards.

  1. Enter the Terraform configuration in the AWS > Account > Stack > Source policy. For example:
resource "aws_iam_role" "monitoring_role" {
  name = "my_monitoring_role"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "12345678"
        }
      }
    }
  ]
}
EOF
}

resource "aws_iam_policy" "monitoring_policy" {
  name = "my_monitoring_policy"

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "apigateway:GET",
        "autoscaling:Describe*",
        "budgets:ViewBudget",
        "cloudfront:GetDistributionConfig",
        "cloudfront:ListDistributions",
        "cloudtrail:DescribeTrails",
        "cloudtrail:GetTrailStatus",
        "cloudwatch:Describe*",
        "cloudwatch:Get*",
        "cloudwatch:List*",
        "codedeploy:List*",
        "codedeploy:BatchGet*",
        "directconnect:Describe*",
        "dynamodb:List*",
        "dynamodb:Describe*",
        "ec2:Describe*",
        "ecs:Describe*",
        "ecs:List*",
        "elasticache:Describe*",
        "elasticache:List*",
        "elasticfilesystem:DescribeFileSystems",
        "elasticfilesystem:DescribeTags",
        "elasticloadbalancing:Describe*",
        "elasticmapreduce:List*",
        "elasticmapreduce:Describe*",
        "es:ListTags",
        "es:ListDomainNames",
        "es:DescribeElasticsearchDomains",
        "health:DescribeEvents",
        "health:DescribeEventDetails",
        "health:DescribeAffectedEntities",
        "kinesis:List*",
        "kinesis:Describe*",
        "lambda:AddPermission",
        "lambda:GetPolicy",
        "lambda:List*",
        "lambda:RemovePermission",
        "logs:TestMetricFilter",
        "logs:PutSubscriptionFilter",
        "logs:DeleteSubscriptionFilter",
        "logs:DescribeSubscriptionFilters",
        "rds:Describe*",
        "rds:List*",
        "redshift:DescribeClusters",
        "redshift:DescribeLoggingStatus",
        "route53:List*",
        "s3:GetBucketLogging",
        "s3:GetBucketLocation",
        "s3:GetBucketNotification",
        "s3:GetBucketTagging",
        "s3:ListAllMyBuckets",
        "s3:PutBucketNotification",
        "ses:Get*",
        "sns:List*",
        "sns:Publish",
        "sqs:ListQueues",
        "support:*",
        "tag:GetResources",
        "tag:GetTagKeys",
        "tag:GetTagValues",
        "xray:BatchGetTraces",
        "xray:GetTraceSummaries"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}
EOF
}

resource "aws_iam_role_policy_attachment" "monitoring_policy_attach" {
  role       = "${aws_iam_role.monitoring_role.name}"
  policy_arn = "${aws_iam_policy.monitoring_policy.arn}"
}
  1. Set the AWS > Account > Stack policy to Enforce: Configured

Turbot will apply the Terraform source, creating the custom role and policy, and then assigning the policy to the role.

Configured Control

Every resource in Turbot can manage its own configuration in its own Configured control. For resources that are configured by a stack, it is the Configured control that keeps the resource configured per the stack source. Resources that are not managed by a stack can define their own configuration in their own Configuration > Source.

Every resource in Turbot can manage its own configuration in its own Configured policy:
  • AWS > EC2 > Instance > Configured
  • AWS > VPC > Security Group > Configured
  • AWS > DynamoDB > Table > Configured

Resources that are not part of a stack can define their own Terraform configuration using the standard Configured policies:

Policy Description
{resource} > Configured Determine how to configure this resource. Note that If the resource is managed by another stack, then the Skip/Check/Enforce values here are ignored and inherit from the stack that owns it
{resource} > Configured > Source The Terraform source used to configure this resource
{resource} > Configured > Precedence An ordered list of who is allowed to claim this resource. A stack cannot claim this resource if it is already claimed by another stack at a higher level of precedence.

The Configured policy determines the overall behavior of the control. Note that resources that are part of a stack are controled by the corresponding Stack policy, and the Configured policy setting will have no effect.

Value Description
Skip (unless claimed by a stack) Do not manage this resource via the configured control (unless it is managed by another stack)
Check: Per Configured > Source (unless claimed by a stack) Check the resource configuration against the Source policy, and alarm if it is not configured correctly (unless it is managed by another stack)
Enforce: Per Configured > Source (unless claimed by a stack) Apply the configuration in theSource policy (unless it is managed by another stack)

The Configured > Source policy should specify the Terraform source used to configure this resource.

The Configured > Precedence policy allows you to define what stack may own this resource. A resource can only be configured by a single stack (at a time), and the claim precedence defines the rules for determining who can configure the resource.

Resource Claiming

Turbot determines the Terraform resource state dynamically. This allows Turbot to manage existing resources via Terraform, even if they were not created by Turbot! While this is an extremely powerful capability, this may not always be the desired behavior - sometimes you may NOT want to Turbot to manage a resource in a Terraform stack if it already exists. Claim Precedence allows you to define whether a resource can be claimed by a stack, and the order of priority for ownership.

The Precedence policy defines an ordered list of who is allowed to claim a resource. The list is ordered by precedence from highest to lowest. - A stack cannot claim a resource if it is already claimed by another stack at a higher level of precedence. If this occurs the stack that tries to claim it should become invalid - The stack will not be able to create or delete resources - Although the stack is invalid, the individual resources will be able to continue to manage themselves using the stack source.

  • If the stack that attempts to claim the resource does not match any item in the list, it cannot claim it. Note that this implies that an empty list means the resource cannot be claimed by any stack. If this occurs the stack that tries to claim it should become invalid.
  • The list is comprised of Turbot control type references, as well as special constants.

    • The following constants are available:

      • current - The current stack (usually the one that created it)
      • source - The Configured > Source sub-policy
    • The references will allow wildcards following standard bash globbing.
    • Note that '*' matches any character except '/'; you should use '**' to match recursively:

      @turbot/turbot-iam#/control/types/**
      @turbot/aws-vpc**
      @turbot/**
    • Using glob wildcards, it is possible that the current (owning) stack, and the new stack attempting to claim the resource BOTH match the same rule. When this occurs, the new stack will not be able to claim the resource

      • Using this pessimistic claim model prevents fighting - If the new stack was allowed to claim the resource, the next time the old stack ran, it would also be able to claim it, and the two stacks would fight infinitely.
      • This means that ** by itself is the same as current when a resource has already been claimed -- ["**"] is identical to ["current","**"]
      • The stack attempting to claim the resource will become invalid at this point, providing a notification to the user of the conflict
      • Example: Consider a precedence list as follows:

        @turbot/**
        @myCompany/**
        current
        source
      • my resource is currently configured by: @turbot/aws-vpc#/control/types/my-stack
      • another stack, @turbot/aws-ec2#/control/types/my-other-stack, attempts to claim this resource
      • In this case, both items are matched by the first entry: turbot/**. The current stack wins - @turbot/aws-ec2#/control/types/my-other-stack cannot claim the resource

Claim Precedence Examples

  • No one can claim me:

    [] #empty list
  • Anyone can claim me if I have never been claimed:

      - **
  • I can be configured via Source, but not claimed via a stack

    - source
  • Turbot wins, but custom stacks are allowed:

    - @turbot/**
    - current
    - source
    - **
  • Turbot wins, but only if not already claimed:

    - current
    - @turbot/**
    - source
    - **
  • Can only be configured by Turbot, or as a custom (non-configured resource):

    - @turbot/**
  • Internal controls > Turbot > Custom:

    - @mycompany/**
    - @turbot/**
    - current
    - source
    - **