Turbot Enterprise (TE) Releases

v5.24.3 [2020-07-10]

Enterprise

  • Installation of a control or policy type during mod install could fail if the underlying metadata was incorrect. This is now handled gracefully.
  • Requires: TEF v1.22.0, TED v1.9.1

v5.24.2 [2020-07-08]

Enterprise

  • AWS credential generation for control runs did not work properly for AWS IAM roles using a path. Now it does.
  • JWT decoding errors now return unauthorized instead of internal error.
  • Reduced logging noise during mod installation.
  • Requires: TEF v1.22.0, TED v1.9.1

v5.24.1 [2020-07-07]

Enterprise

  • Improved error handling and comptability for workspace installation and upgrades with regard to TED versions that no longer store database roles.
  • Requires: TEF v1.22.0, TED v1.9.1

5.24.0 [2020-07-06]

What's new?

  • Our AWS credentials are now fully partition and region aware, opening the door for our AWS mods to work across partitions (e.g. manage GovCloud or China from Commercial) and with newer regions (e.g. Hong Kong, Middle East).
  • HTTP requests are now automatically redirected to HTTPS at the load balancer level, avoiding awkward timeouts when users go old school.
  • Retrying controls and calculated policies in Error or TBD is very helpful to cleanup after a variety of issues. We've expanded our retries to try more in the first hour, and with backoff, all the way out to 56 days.
  • Previously, the stack converted the Terraform configuration to JSON format before executing Terraform. It now uses the original configuration without any conversion or alteration. Note: Some invalid Terraform syntax (e.g. unquoted variable type strings) previously handled by Turbot will now fail since it's not handled by Terraform natively.

Bug fixes

  • Resource deletion through external events could lose the actor information in some cases. We now track it through and report it properly in the CMDB.
  • Policy settings are only valid on the target resource type for the policy (e.g. AWS > SQS > Queue) and any resources above it in the hierarchy (e.g. region, account, folder, Turbot). We were checking this in the UI, but not in the API level, making it possible - even though useless - to set policies on invalid resource types (e.g. set an AWS policy on an Azure subscription). We now properly prevent saving invalid policies.
  • Smart retention actions were timing out in large environments, particularly when it had not been enabled before. We've optimized these queries, so smart retention is faster and more reliable.
  • Fixed the character casing of allowIdpInitiatedSso in SAML directory queries.

Enterprise

  • Incoming events are now handled by a separate service to general API traffic, making it easier to handle surges and providing a smoother user experience under extreme load.
  • Improved performance of type installation during mod upgrades.
  • Improved performance of control handling, with particular focus on discovery which can have large runs of identical resource upsert queries. This improves the overall load profile, particularly while importing very large collections of resources.
  • Expanded error logging in the hive manager, which is used to perform database migrations during version upgrades. If things go wrong, the underlying error should now be clearer.
  • The connectivity checker Lambda function is used to check the current permissions and network access in the installation account. We've expanded the logging and made the timeout optional for more information and flexibility, particularly against network timeouts.
  • SNS topics used to send commands to Mod Lambda functions are now setup with encryption enabled during mod install. This will be enabled as mods are ugpraded or reinstalled.
  • Requires: TEF v1.22.0, TED v1.9.1

5.23.2 [2020-06-30]

Bug fixes

  • Automatically triggers any controls or calculated policies that may have missed their trigger due to the resource update defect added in v5.23.0 and fixed in v5.23.1.

Enterprise

5.23.1 [2020-06-30]

Bug fixes

  • A change in v5.23.0 broke automatic running of dependencies when a resource is updated. We've also expanded our testing to cover this case.

Enterprise

5.23.0 [2020-06-29]

What's new?

  • Filter results can now be sorted by any path in the data object. For example, sort:$.QueueUrl or sort:$.Tags.environment. Null values are returned last.
  • GraphQL process(id:"1234") queries now return the last version of the process, even if it has been terminated. Which is what users expected all along.

Bug fixes

  • Deletion of fundamental Turbot identities (e.g. Turbot, or Unidentified User) is now blocked.
  • Filter queries with resource:undefined, while not expected, should be handled gracefully. Now they are.

Enterprise

  • Resource creation is approximately 4 times faster for the typical case (e.g. AWS S3 Bucket), and even better for resources with a large number of controls or policies (e.g. Azure Subscription). Primarily this was achieved by redesigning our model for dependency checks and triggers.
  • Mod installation is a significant operation, and tied to the number of resources affected. This release breaks up the process into separate steps per control type and policy type. Install now takes a little longer, but is more robust for very large workspaces.
  • API containers use AWS credentials for access to various services (the task role). Sometimes, retrieving these credentials from the metadata service has a timeout, leading to unexpected access denied errors. We've added logging for this case and increased the backoff / retry settings.
  • Low level functions in the database for JSON updates and differences relied on plv8. This is convenient, and performant, but makes the database more prone to unexpected (and particularly evil) crashes under load. We've rewritten these functions into native sql and plpgsql to improve stability.
  • Control and policy updates were overly aggressive in locking their dependency data, slowing down those operations. We've streamlined our approach, which should smooth control running at scale.
  • Updated our API server keep alive timeouts to match those expected by the load balancer, which should prevent nasty 502 errors that were very visible to users but both random and very quiet in our logs.
  • Fixed a crash when attempting to retrieve mods from the registry without valid credentials.
  • Any policy full text search data broken in v5.22.0 will be repaired by this upgrade.
  • Added a maintenance job to cleanup old Lambda functions created through local mod development.
  • Requires: TEF v1.21.0, TED v1.9.1

5.22.3 [2020-06-24]

Enterprise

  • Stale processes are cleaned up after 45 mins. Instead of giving up, we'll now schedule them to be retried.
  • Requires: TEF v1.21.0, TED v1.9.1

5.22.2 [2020-06-22]

Enterprise

  • Worker Lambda functions now have a dead letter queue. We already have a DLQ for the SQS queue feeding them, but wanted to ensure we capture the (theoretical, but unlikely) case where a message is successfully taken from SQS but not successfully processed by the Lambda function.
  • Made cleanup of temporary directories more resilient against unforeseen errors.
  • Requires: TEF v1.21.0, TED v1.9.1

5.22.1 [2020-06-22]

Enterprise

  • Resource create operations are now about 35% faster for a typical cloud resource. Another step smoothing the import experience at enterprsie scale.
  • Worker Lambda functions could run out of disk space if used for a number of large commands. We've fixed the temp data cleanup to cover this case.
  • Our efforts to expand the search data for policy settings to include the value ended up removing other important data (e.g. the title) during updates.
  • Requires: TEF v1.21.0, TED v1.9.1

5.22.0 [2020-06-19]

What's new?

  • Terraform stacks managed by Turbot were limited to 1-1 relationships between Terraform resources and Turbot CMDB resources. We now support a variety of relationships making them cleaner and more flexible. For example, a single aws_security_group_rule resource can map to several AWS > VPC > Security Group Rule resources in Turbot. Also, association resources (which exist only in Terraform) like aws_vpc_dhcp_options_association can now be managed inside existing Turbot resources (VPC or DHCP Options) without the need for an extra (and confusing) association resource.
  • Full text search of policy values will now also match on the actual value of the policy, not just it's resource and policy type titles.

Bug fixes

  • If there is a Terraform error during a stack run, any resources which were successfully created are claimed and upserted. This reduces problems with duplicate/unclaimed resources when the stack re-runs.

Enterprise

  • External messages are received in Turbot via our webhook. During upgrades, there can be a delay in the change of DNS from the old version to the new version, causing events to be received by the old version. Rather than processing them in the wrong place (since the workspace has been upgraded), we redirect many of these requests to the new version. Unfortunately, a defect meant that for external events we were redirecting them to the new version but not updating their webhook signature, so the new version would reject the event. This has been fixed so events will flow smoothly even if the workspace DNS does not point to the correct workspace version endpoint.
  • Controls and policies make decisions and send commands back to the Turbot handler like "upsert resource" or "set control to OK". In most cases, we'd group these updates into a single transaction for completeness. But, the size of the command set is not easily controlled (consider discovery of thousands of resources from a provider where paging is not supported) and resulted in very large transactions, creating risk and load. Mods are designed for idempotency, so this transaction had limited value. So, we now break the commands up into single operations and apply them in order - reducing conflicts and risk.
  • One database function had a crash condition that could slip through, causing the ugliest of database segmentation faults. We've tracked it down and now handle it without panic.
  • Our database connection pools in API and workers were not properly handling unexpected errors from the database (e.g. a crash), causing the API and worker process to also crash. We now catch and log these errors properly.
  • Workspace and mod installation create a small number of resources like Lambda functions, SNS topics and SSM parameters. These now inherit custom tags added in the TEF stack, supporting enterprises with very specific tagging requirements for their Turbot primary account.
  • Each Turbot Enterprise version installs minimal IAM policies and roles specific to its requirements. Some customers prefer more control over IAM management, so we now support BYO-IAM with parameters for all IAM entities required in the Turbot primary account.
  • Improved performance of control and policy value dependency management.
  • Terraform stack creation used to trigger a stack run for each newly configured resource - which creates unnecessary load and work. We've optimized this flow to realize the resource is new and properly configured, avoiding those runs.
  • Requires: TEF v1.21.0, TED v1.9.1

5.21.1 [2020-06-11]

Bug fixes

  • If two processes attempted to update the same resource in parallel, we'd see constraint errors in some cases. Specifically, process A starts, process B starts, process B gets lock first and updates resource, then process A (blocked behind B) now get the lock. Process A would fail with a timestamp constraint error. This is now fixed, and high throughput situations are running much smoother (e.g stacks, mod install).

Enterprise

5.21.0 [2020-06-10]

Active Directory / LDAP Integration

  • Connect Turbot to your Active Directory or LDAP system. Use your existing identity management system to manage all Turbot and cloud access.
  • Continously synchronize users and group memberships (including nested groups) into Turbot. Automatically disable access when users leave. Update group memberships on login. Ensure instant access and permissions for new Turbot users.
  • Seamlessly search your internal directory and assign permissions in Turbot. Build exact least-privilege models by combining your internal groups with the Turbot resource hierarchy for permissions.
  • Works seamlessly with SAML, allowing trusted authentication combined with instant and continuous synchronization.
  • Automatically integrates with Turbot's permission expiration and temporary elevation capabilities. Also works with our full stack cloud IAM model for per service permissions. For example, grant App Team A from LDAP the AWS/Admin permission until the end of the week (their setup period).

What's new?

  • Controls can now access detailed information about the maintenance mode, including the type of trigger and if the change window is currently open. They can also use turbot.set("nextRun", "CHANGE_WINDOW") to deliberate target actions to the next available change window.
  • Turbot generates temporary AWS credentials for each mod run. Those credentials are now based on the STS endpoint for the target resource - e.g. if running a control for an EC2 instance in ap-southeast-2 we'll create credentials using that region.
  • We now block the @turbot/turbot and @turbot/turbot-iam mods from being uninstalled. They are our heart and soul.

Bug fixes

  • Concurrent updates to the same resource were not properly sequenced using locks. Now they are, and conflicts have been reduced.
  • Directory create dropdown was blocked in the UI by a bad overlay. It's now visible.
  • upsertResource was incorrectly validating against the full (updated) data rather than the request data, which could lead to validation warnings if the update schema specifically required existing fields to be excluded. We now validate against request data.
  • Incoming events must be evaluated against the webhook secrets. If we cannot retrieve those secrets we'll now immediately stop execution.
  • Mods can choose their behavior under various maintenance mode conditions (e.g. CMDB controls should always run). If it is changed by a mod author we'll now properly update this setting during mod update.

Enterprise

  • Controls running in a container (e.g. stacks) are passed temporary data via an S3 object. These objects are now deleted immediately on process termination, rather than waiting for daily S3 lifecycle jobs.
  • IAM permissions for ECS tasks to access ECR images have been further tightened to only the specific resources required. When it comes to IAM policies we are minimalists.
  • Requires: TEF v1.19.1, TED v1.9.1

5.20.3 [2020-06-11]

Bug fixes

Enterprise

5.20.2 [2020-06-04]

Enterprise

  • We added checks in v5.20.0 that a Lambda function is installed and available before it will be run as part of a control or action. These checks work well for controls (the primary case) but have been disabled for actions (which were hanging).
  • Requires: TEF v1.19.1, TED v1.9.1

5.20.1 [2020-05-28]

Enterprise

  • Turbot is frequently throttled by the SSM service because the Worker Lambda fetches our feature flags from SSM at startup. Turbot now reads the flags from an environment variable instead.
  • Requires: TEF v1.19.1, TED v1.9.1

5.20.0 [2020-05-28]

Security

  • Since v5.17.0, permission checks were not properly checking the disabled status of inherited groups. Consider user A in group X which is in group Y, where group Y was granted Turbot/Admin permission. If group Y is enabled, then group X and user A inherit the Turbot/Admin permission. But, if group Y is disabled then it's permissions should no longer be available to group X or user A. We now correctly check the group disabled flag for inherited groups as part of permission evaluation. (Note that the disabled flag check was always working correctly for users, directories and direct groups like X above; just not indirect groups like Y.)
  • A low-level policy setting updated from being an exception (required under required) to an orphan (recommended under required) would not properly re-evaluate policy values using the (now orphaned) setting. Consider this sequence: 1. Set AWS > S3 > Bucket > Approved as Required to be Check: Approved on AWS Account 1234. 2. Set an exception as Required to be Skip on my-bucket. 3. At this point, the effective value is Skip for my-bucket. 4. Update the policy from Required to Recommended as Skip on my-bucket, making this policy setting an orphan and ineffective. 5. At this point, the effective value should be Check: Approved on my-bucket; but because of this defect, it was not re-evaluated so remained as Skip. This release properly handles this scenario, and triggers affected policy values to be re-evaluated to match their true effective setting (instead of the orphaned setting).

What's new?

  • GraphQL queries to get a resource will return an error if the resource is not found. We now support resource(id:"{aka}", options: {notFound: RETURN_NULL}) for cases where execution should continue either way.
  • The test resource browse dropdown in the calculated policy editor will now automatically select the current resource by default.
  • Improved performance and scalability of large scale changes to policy values (e.g. during mod install) and dependency trigger checks.

Bug fixes

  • Update policy setting mutation was not blocking operations on read-only policies. Now read-only means read-only and updates will return an error.
  • When executing a large number of upsert commands in parallel (e.g. import or stack control) it's possible for the same resource to get created twice in parallel (e.g. stack record from Turbot competing with a real-time event from the cloud provider). We now protect against these conflicts, smoothing large operations.
  • Changes to Terraform state information in the CMDB was not properly triggering dependencies, causing controls to be re-triggered. Stacks are now smoother and more accurate.
  • Control Installed control will no longer fail if Mod URL has expired. It will stop the current process and most importantly it will not change the current state of the control. Previously if the Mod was installed > 6 days the Control Installed state will be set to OK.
  • Create child resource should validate the AWS Account ID on blur. It's not an error until they are done entering the ID.
  • Login buttons for cloud accounts were not appearing if the user only had permission for a specific service (e.g. AWS/S3/Admin). Now they do.
  • Filter queries like policyTypeId:undefined would crash. Since the request is not specific, we'll now ignore this invalid input and continue the query.
  • Feature flags now support backoff and properly stop the process if retriving them fails.

Enterprise

  • Runnables like controls and calculated policies use Lambda functions for function execution. By default, for security & stability, these functions run outside the VPC. For organizations wishing to inspect and control all network traffic, we now support running of these Lambda functions inside the VPC.
  • Executing a runnable before it's Lambda function is installed and active would result in an error, which is noisy (particularly when using the slower Lambda in VPC model). We now leave the control in TBD state and automatically retry.
  • Requires: TEF v1.19.1, TED v1.9.1

5.19.5 [2020-05-22]

Enterprise

  • S3 process logs would fail to save in a small set of cases for the small set of customers using a fixed process log bucket name.

5.19.4 [2020-05-21]

Enterprise

  • Stacks were not properly running for AWS Gov Cloud environments since our change to ECS optimized Amazon Linux 2 in TEF v1.17.0. We relied on the AWS_REGION environment variable, which is no longer published in that newer image. This release fixes our environment check.
  • Requires: TEF v1.18.1, TED v1.8.0

5.19.3 [2020-05-20]

Enterprise

  • The new maintenance container was not properly built in our production artifacts. This release fixes that build.
  • Requires: TEF v1.18.1, TED v1.8.0

5.19.2 [2020-05-19]

Enterprise

  • Turbot is designed to support multiple workspaces as subdomains, using a repeatable format. But, workspaces may also be configured using a custom alternate URL. This release fixes an issue with the way we tracked and managed that alternate URL, ensuring it was used in all cases for logging and routing.
  • Requires: TEF v1.18.1, TED v1.8.0

5.19.1 [2020-05-15]

Bug fixes

  • Fixed error when deleting a grant from the permissions list in the UI.

Enterprise

5.19.0 [2020-05-14]

What's new?

  • Azure client key is now hidden as a secret in the UI during subscription import.
  • Calculated policy modal now shows the test resource as a link, for convenient viewing of the available data.
  • SAML directory setup will now automatically turn on group synchronization (per policies).

Bug fixes

  • Resources list was not immediately updating when browsing the hierarchy. Now it does.
  • The process dialog should only display the Terminate button if you actually have permission to terminate the Turbot process.
  • Controls with deferred actions based on the Maintenance Window were not properly re-run if the control had changed state in the first run. Now they do.
  • Smart Folder breadcrumb fixed with the correct destination links.

Enterprise

  • A new maintenance container has been added to perform general cleanup duties, such as cleaning and migration process logs into our new TED-based S3 buckets.
  • All process logs are now stored in TED-based S3 buckets with improved naming, lifecycle and encryption controls.
  • Support for TEF Flags, giving us more flexibility to innovate and gradually deploy features.
  • Dashboard response time metrics now track the maximum instead of average.
  • API container scaling capacity min and max has been fixed to work correctly.
  • Requires: TEF v1.18.1, TED v1.8.0

5.18.1 [2020-05-06]

Bug fixes

  • Turbot maintains a complex set of dependencies between controls and policies. One of the more complex cases to track is the use of lists (e.g. resources(filter:"")), which change as items are added, changed or removed. Our structure here was too slow, causing timeouts in large scale operations in large workspaces. We've optimized the structure and queries so operations like mod install and account import should be faster and more reliable.

Enterprise

5.18.0 [2020-05-05]

Bug fixes

  • Using Azure credentials from US Government Azure AD requires a specific authentication context endpoint (for some tenants).

Enterprise

  • Support for "Allow Self-Signed Certificates" parameter from TEF, which enables Turbot to work in environments where a self-signed proxy is used to access external services like Azure or AWS.
  • Requires: TEF v1.16.0, TED v1.7.0

5.17.1 [2020-05-01]

Enterprise

  • We track various logs in S3 (e.g. process, input queries). We've updated this storage with better key prefixes for data management and lifecycle targeting. (Primarily a change in v5.17.0, but immediately tweaked in v5.17.1 before release.)

5.17.0 [2020-05-01]

What's new?

  • SAML group synchronization. Turbot can now detect group memberships during SAML login by users and automatically represent those groups and relationships in the Turbot IAM model. Grant permissions to the group, and then any user new to Turbot will automatically get the appropriate permissions on login.
  • Error messages from controls are now prominently displayed as the reason and details, making problems easier to diagnose without digging through logs.
  • We now show directory information while granting permissions to a profile or group profile.

Bug fixes

  • Calculated policies in TBD or Error were not automatically retried in all cases. You should see less policies getting stuck now, especially during account import.
  • Each control and policy waits in TBD until all policies it depends on are in an OK state. We were not triggering the waiting policy to re-run when it's dependency moved to OK. Now we do.
  • Mod versions must now be valid semantic version format. We admired the creativity, and love potatoes, but vegetables are not easy to digest as version numbers.
  • Process logs list would crash if passed an invalid log level. Obviously that was an overreaction, so now we just return a bad request error.
  • Concurrent events could cause a database deadlock when both trying to mark the same control as due to re-run.
  • Immediately after a mod update the UI would show two versions as currently installed. And now, there is one.
  • Policies set to 0 were not displaying the value in the control detail page.

Enterprise

  • As scale increases, audit trail logging could fail due to conflicts when writing. Log stream names are now unique for each workspace, version, container combination.
  • API health timeout increased to 30 seconds. Our previous setting was too aggressive and would cause unnecessary web server replacements.

5.16.0 [2020-04-24]

What's new?

  • The external role ID is now optional (but still recommended) when importing an AWS account.
  • Resource creation and updates via Terraform of GraphQL now prevent setting an invalid parent type - e.g. an Azure subscription should not be a child of a directory. Previously we allowed it but logged a warning.
  • Reviewed and optimized smart folder performance, increasing reliability and scale. Smart folders let you do more (e.g. policy settings) over a wider scope (e.g. many accounts) faster (e.g. single operation).
  • Filters now have sort:rank to order results by their full text search rank.
  • Mod runnable functions now default to using nodejs v12 (up from v10).
  • Use mode:node in controlSummariesByResourceType to see control data organized by resource type (e.g. AWS > S3 > Bucket).

Bug fixes

  • Controls and calculated policies are automatically triggered by changes to their input. Because the input is calculated before the run, a type could not depend on or be triggered by resources it created. Complex controls like Terraform stacks need this capability. So, types may now specify that their dependencies should be recalculated after a run. TLDR - complex controls will be triggered more reliably.
  • Turbot automatically calculates dependencies for every control and calculated policies, including filtered lists. This was working correctly for filters like resource:<id>, but not correctly handling cases like resource:<aka>. We now match and automatically trigger for these AKA cases as well.
  • Large scale deletions would sometimes fail with a conflict error caused by our efforts to track the original actor through many levels of events. Our brave developers have returned from deep in the events jungle with a fix.
  • Smart folders in the policy detail page could show as duplicates in complex configurations. One is enough.

Enterprise

  • Usage reporting is now incorporated to assist with billing. The data is aggregated by control type and does not include customer specific metadata (e.g. resource names are not reported). Opt-out is available when appropriate.
  • Ongoing cleanup of error messages and logging to reduce noise (e.g. S3 getObject errors for the new log locations) and improve traceability.

5.15.0 [2020-04-15]

Warning

  • Requires TEF v1.13.0 and TED v1.7.0. Please upgrade in order: TEF, TED, then TE.
  • Turbot > Maintenance policies (added in v5.14.0) have been renamed to Turbot > Change Window. Please upgrade @turbot/turbot to v5.11.0 to access these policies.

What's new?

  • Resource statistics now support mode:node to aggregate by specific nodes rather than the default (mode:lca) which rolls the data up to a common ancestor. For example, resourceSummariesByResourceType(filter:"mode:node").
  • New GraphQL query resourceSummariesBySmartFolder, to aggregate resources by smart folder.
  • Expanded logging to assist with troubleshooting: GraphQL errors are now shown in the process log, and errors in event handling will log the full payload.
  • Added flexibility to filters: filter resource categories by resourceType:, filter resource types by resourceCategory:, filter control categories by controlType:.
  • Notification filters now support exact matching with resourceId:{aka}, controlTypeId:{aka}, etc.
  • Favorites are now sorted by title on the home page.

Bug fixes

  • Policy evaluation was not triggered correctly when multiple smart folders were attached above a resource. We've expanded our testing and fixed a number of edge cases.
  • Mod updates that add targets to an existing policy type should create policy values for all instances of the new target resource types.
  • It should be possible to grant multiple custom roles to the same user on the same resource.
  • Smart folders defined in the UI were incorrectly limited to 64 character titles. Express yourself.
  • Policy detail page should support large number of smart folder attachments.

Enterprise

  • Added CloudWatch alarms to the dashboard to monitor queue health and unhealthy hosts.

5.14.6 [2020-04-14]

Bug fixes

  • Stack execution was broken in the build of v5.14.4. They will now run correctly again.

5.14.5 [2020-04-12]

Enterprise

  • Short term internal caching of policy type information was corrupted in some cases. This fix will reduce not found errors.

5.14.4 [2020-04-08]

Enterprise

  • Process data in S3 was not working correctly in multi-region installations with randomized bucket names.

5.14.3 [2020-04-08]

Bug fixes

  • Optimization work in v5.14.0 broke the display of resource summaries for users that are not granted Turbot/Metadata at Turbot level specifically. Which wasn't exactly optimal, so it's been fixed.

5.14.2 [2020-04-08]

Bug fixes

  • Terraform stacks run by Turbot can claim existing resources, automatically bringing them under management. We discovered a long standing bug where we were doing the hard work to claim, but not actually saving the claim information - meaning that claims didn't work effectively.

5.14.1 [2020-04-07]

Bug fixes

  • Eradicated various edge cases causing the API container to crash, increasing overall stability.

5.14.0 [2020-04-07]

Warning

  • Use @turbot/turbot v5.6.0 to access policies related to new features.

Security

  • Since v5.0.0, Turbot access keys created by a Local Directory user would still successfully authenticate even after the user was made inactive or deleted. With this fix, those invalid keys will now correctly fail authentication. No further action is required.

What's new?

  • Use Turbot > Maintenance policies to define the period of time when Turbot is permitted to apply changes to resources. (Note: Renamed to Turbot > Change Window in v5.15.0.)
  • Use Resource Type interfaces in filter queries, e.g. resourceType:'@turbot/turbot#/resource/interfaces/grants' resourceTypeLevel:self. Easily query resources from any mod that implements the interface.
  • Aggregation queries have always automatically calculated the longest common ancestor (mode:lca), grouping results appropriately (e.g. AWS). A new mode:node will group results by their specific node (e.g. AWS > S3 > Bucket).
  • The policy setting modal now asks for confirmation on cancel or close if you have unsaved work.
  • Notifications displayed on the resource detail page can now be filtered by type. More filtering, less scrolling.
  • Hovering a resource type in the left navigation bar will now show the URI.
  • Clicking a section (e.g. alarm) of the control summary chart now filters the entire chart to that state.
  • Use Turbot > Workspace > Retention > Debug Log Retention to automatically cleanup old debug logs from RDS.

Bug fixes

  • Custom Role permissions will now show the actual name of the custom role in the UI, an important capability when you have more than one.
  • The test resource is now optional in the calculated policy editor, so it works even if you have no existing resources of the required type.

Enterprise

  • Use of a database read replica is no longer required in region where the primary database resides. This provides a significant cost saving, in many cases the read replica has low utilization anyway. A read replica is still required in additional regions, ensuring data is available for faster failover.
  • The events webhook API will no longer log to the Audit Trail. It was very noisy, expensive for ingest and adds little value compared to the audit trail of user actions.
  • Process data is now stored in S3 instead of RDS. This reduces database growth by about 40% from v5.12.x and earlier.
  • Further clarified and reduced IAM permissions granted to various functions in the Turbot core.

5.13.0 [2020-03-27]

Warning

  • Requires TEF v1.9.0 or later.

What's new?

  • Performance improvements for deleting large collections of resources (e.g. an account or project).

Bug fixes

  • The Turbot root resource should not allow editing or smart folder attachments. So, now they are blocked in the UI.
  • Invalid dependencies in mod definitions should not crash the mod installation process.

Enterprise

  • Process logs are now stored in S3 instead of RDS. This reduces database growth by about 20% from v5.12.x and earlier.
  • Optimized dependency matching, particularly for new fields at the root level of an object. This reduces unnecessary control runs and policy calculations, a specific example being the upcoming account alias field for AWS.

5.12.2 [2020-03-23]

Enterprise

  • Revert performance improvements around policy value updates, they were triggering too often.

5.12.1 [2020-03-20]

Bug fixes

  • Mods with more than 300 types were failing during installation.

Enterprise

  • Creating indexes on the notification table may timeout for large workspaces.

5.12.0 [2020-03-19]

What's new?

  • Expanded filter support for array data. Match a specific index (e.g. 0) with a query like $.EncryptionAlgorithms.0:SYMMETRIC_DEFAULT or any item in the array using a splat (.*) like $.Policy.Statement.*.Action:'kms:*'.
  • Optimized queries for notifications and action history.

Bug fixes

  • Improved database error handling to prevent hard crashes.
  • Filter searches with quoted strings were not parsed correctly, leading to errors and bad results.
  • Controls and calculated policies were not being properly triggered by changes related to the new filter types of resourceId:, resourceTypeId:, etc.

5.11.0 [2020-03-12]

Warning

  • Requires TEF v1.6.0 or later.

What's new?

  • Control filters support state:active, a simpler way of asking for state:alarm,invalid,error,ok.
  • Simplified the home page controls chart to only show active controls.
  • Improved error messages when the GraphQL input to a control fails, making troubleshooting easier.

Bug fixes

  • In some cases, automatic retry of controls and calculated policies could toggle back and forth from Error to TBD indefinitely. It will now backoff and stop as expected.
  • Permissions should not be grantable on smart folders.

Enterprise

  • Improved performance of resource upserts.

5.10.0 [2020-03-06]

What's new?

  • Control Summary now shows active controls by default, reducing the noise and complexity from lower priority controls in Skipped or TBD state.
  • Search controls by keywords in their type, reason or resource details. For example, s3 bucket tags.

Bug fixes

  • Policy detail page was not showing settings on attached smart folders. Now it does.
  • Editing an existing policy setting with a recommended precedence would show it as required precedence in the editor.
  • During a workspace upgrade events may briefly be sent to the old version. They now queue for retry on the new version.

v5.9.1 [2020-03-06]

Enterprise

  • Fixed: Our API Gateway proxy used by some customers was broken in v5.9.0, stripping the URL passed through to the Turbot server too aggressively.

v5.9.0 [2020-03-03]

Warning

  • Turbot/Owner now includes Turbot/Admin rights.

What's new?

  • Turbot/Owner is now defined as Turbot/Admin plus perrmission management. Previously Turbot/Owner was Turbot/Metadata plus permission management. This simplifies our permission management model and aligns better with user expectations. It does reduce our inherent segregation of duties, but Turbot/Owner could always have granted themselves Turbot/Admin rights anyway.
  • Notification filters now support full text search of the resource details. Makes it easy to get the full history of deleted resources e.g. i-abcd12341. Simplified type matching with collective matches like notificationType:resource, and specific matches like notificationType:controlDeleted. Support for tags: and $.{field}: has also been added.
  • Expanded exact matching in control, policy value and policy setting filters to include resourceTypeId:{aka}, controlTypeId:{aka}, policyTypeId:{aka}, resourceCategoryId:{aka} and controlCategoryId:{aka}. The existing fuzzy match filters (e.g. controlType:{fuzzyAka} remain unchanged and more convenient.
  • Google login now redirects to the originally requested URL after successful authentication. Doing our bit to reduce your tabs.
  • The webhook GraphQL query needs a resource for context, so we now accept one as an argument.
  • The calculated policy editor now supports the full range of nunjucks filters. Enter and test your calculated policies with filters for json, yaml, alphanum, date, hex, pascalCase, snakeCase and camelCase.
  • Terminate "stuck" processes from the UI.

Bug fixes

  • Switching between policy settings and values in the UI will now preserve your search query.
  • Scrolling through children in the left navigation bar was failing after 2 pages of data. You can now scroll in support of all your children.

v5.8.6 [2020-02-19]

Bug fixes

  • Policy dependencies widget on the policy detail page was entering an infinite retry loop in some cases.

v5.8.5 [2020-02-15]

Bug fixes

  • New GraphQL queries for user and group data related to upcoming features were accidentally released early. They've been removed for now, please enjoy the sense of anticipation.

v5.8.4 [2020-02-14]

Bug fixes

  • Infinite scrolling in the resources pane of the left nav was broken for long lists in v5.8.0. It now scrolls smoothly again.

v5.8.3 [2020-02-13]

Bug fixes

  • We tried to remove unsafe_event to tighten Content Security Policies in the browser. But CodeMirror editors require it, so we've allowed it again for now.

v5.8.2 [2020-02-13]

Bug fixes

  • Mod resources can store metadata. This is currently called metadata (duh). Previously it was turbot.metadata and then turbot.custom, but those are now deprecated. This fix restores part of their functionality cleaned up in v5.8.0 that is still used by some older mods.

v5.8.1 [2020-02-13]

Bug fixes

  • Errors during mod install should set the state to error.

v5.8.0 [2020-02-13]

What's new?

  • Controls in Error or TBD state will now be automatically rerun after approximately 5 mins, 1 hr, 4 hrs, 1 day and 3 days (final). This should automatically clear the vast majority of "stuck" controls.
  • Specific GraphQL mutations for managing Turbot IAM objects like directories and profiles (e.g. createSamlDirectory). These are simpler to use, consistent with terraform and allow tighter validation of relationships.
  • Policy setting filters now support is:exception, is:orphan, is:expired, is:active, is:required, is:recommended. As usual, they work with "and" queries is:exception is:expired, "or" queries is:exception,orphan and negations !is:expired.
  • Policy value filters now support is:calculated and of course !is:calculated.
  • Expanded exact matching in resource list filters to include resourceTypeId:{aka}, controlTypeId:{aka}, policyTypeId:{aka}, resourceCategoryId:{aka} and controlCategoryId:{aka}. The existing fuzzy match filters (e.g. controlType:{fuzzyAka} remain unchanged and more convenient.
  • Login redirect to Azure Government subscriptions via the UI.

Bug fixes

  • Calculated policies producing object data (e.g. a tags template) were having the new value merged with the old value. It should have been replacing the entire object with the new value.
  • Clicking a specific state bar (e.g. Error) for a specific row (e.g. us-east-1) in the Controls Summary Chart should filter to both the correct data (i.e. us-east-1) and the desired state (i.e. Error).

v5.7.2 [2020-02-07]

Bug fixes

  • Mod installations that update a policy type will recalculate policy values for that type (ensuring they are up to date). In some cases, this process would briefly clear the policy value before setting it again. Generally you'd never notice, but this could trigger unexpected control or policy changes.

Enterprise

  • Some complex JSON operations are performed deep in the database layer. When they are good, they are very very good. When they are bad, they are now caught and logged for later review.

v5.7.1 [2020-02-07]

Skipped due to technical difficulties.

v5.7.0 [2020-02-06]

What's new?

  • Faster browsing experience through all filter pages. We feel your need for speed.
  • Filter to an exact resource ID or AKA using resourceId:{aka}. The existing resource:{fuzzyAka} is simpler and super intuitive, but can have multiple matches since it does a fuzzy match (partial, case insensitive) on resource AKAs.
  • New users now have a default favorite (Turbot root) and a clear warning if they have been added with no permissions.
  • Controls summary added to the home page.
  • Automatically run Policies and Controls on a set interval (e.g. daily). This interval can be defined on the type itself, or customized using the Interval policy.

Bug fixes

  • GraphQL query resourceVersion should check permissions on the specific version, not on the (potentially deleted) item.

v5.6.1 [2020-01-30]

Bug fixes

  • Listing resources at the Turbot level should not accidentally include searchable user profile information. You may need to see those profiles, but not all the time.

v5.6.0 [2020-01-30]

What's new?

  • Resource and control filter queries are a lot faster. The sort of speed improvement that should be noticed by everyone, not just it's loving parents.
  • Searching for a Turbot ID (e.g. 12345) will now find the resource with that ID. Just as you'd expect it to.

Bug fixes

  • Smart folders should be blocked from being attached to smart folders.

Enterprise

  • Improved logging of AWS SNS subscription confirmation requests, providing more information to help debug during installation into complex custom networking environments.

v5.5.0 [2020-01-22]

What's new?

  • Turbot Directory type. Optionally allow authentication into your workspace by users registered at turbot.com. Eventually this will be the default directory for new workspaces, making setup easier.
  • Improved performance of the Permissions tab.
  • Updated GraphQL documentation for many object and input types.

Bug fixes

  • Smart folder detachment was not triggering policy values to be recalculated. Now it does.
  • Imagine smart folder X is created under a parent resource A. To prevent cycles, X may be attached to any descendant of A, but not to A or its ancestors. After enjoying a number of "chicken or the egg" jokes, we're now correctly blocking X from being attached directly to its parent A.
  • GraphQL queries from runnables (controls and calculated policies) may use resource { data } to get the full object information. This is rare and not great style, but we will now fulfil the query (instead of passive aggressively always returning null).
  • Historic activity records for deleted resources should not silently eat their dropdown (hamburger) menus.

Enterprise

  • Requires Turbot Enterprise Foundation v1.3.0 or later.
  • Optimized resource creation and updates, smoothing database utilization.

v5.4.1 [2020-01-15]

Enterprise

  • Turbot containers need the ability to create CloudWatch Log streams. Our least privilege improvements in v5.4.0 went too far to less than least in this case.

v5.4.0 [2020-01-14]

What's new?

  • Easily delete any resource (and it's descendants) from the left navigation bar in the UI.
  • Faster loading of counts into tab headings (e.g. number of resources).
  • Documentation for GraphQL policy types and data.
  • Simplified the policy setting create and update modal by moving the precedence input into the advanced section. Most users create policies in Required mode, and using Recommended is really only for more advanced policy designs.
  • Summary charts have been removed from the policies tab. Enjoy the cleaner look with a focus on policy list data.

Bug fixes

  • Resource inserts were actually slowed by work we'd done to improve concurrency. That has been recitified, so larger activities (e.g. cloud account import) should now run faster and more reliably as they did before.

Enterprise

  • Reduce scope of CloudWatch Logging permissions granted to Fargate when executing Turbot tasks.

v5.3.0 [2020-01-09]

What's new?

  • Users can now subscribe to notifications for changes to resources, controls and more.

Bug fixes

  • Commands sent from mod controls back to Turbot may be split into multiple messages due to size limits. Occasionally we receive them out of order, and were hanging in these cases. You will see a lot less processes getting "stuck" now.
  • Large scale resource upserts triggered by complex Terraform stack runs revealed two edge cases - we were not always saving the terraform information (which causes unnecessary rework) and we were not always safe against concurrent inserts. Both are now fixed.
  • Events with errors are sent to a dead letter queue for cleanup. Our handler was not correctly logging these errors. Hopefully there won't be many of these errors, but at least now they are visible when they exist.

v5.2.0 [2020-01-09]

What's new?

  • Only show Revoke All permissions in the UI if the user has Turbot/Owner permission at the Turbot root level.

Bug fixes

  • The GraphiQL component for Developers now displays scrollbars as that team originally intended, instead of being forcibly removed by our overzealous CSS.
  • Toggling between the policy settings and values lists in the Policies tab was clearing the search query. You worked hard on that query, so we now keep it as you switch.

v5.1.1 [2020-01-08]

Bug fixes

  • When terraform stacks are run inside Turbot we automatically update the state information from the CMDB. It's important this information is correct for smooth stack operation on the next run. We detect failures on future runs and then try to automatically update the resource to clear the problem. Before this fix we did all the hard work to detect the problem, but were not actually saving that back to the CMDB to fix it.

v5.1.0 [2020-01-07]

Security

  • Since v5.0.0 Turbot has allowed a user to setup a notification to receive information about changes to a resource. Those updates were not properly filtered based on the permissions of the user, which may cause a workspace user to receive notifications for unintended resources in the same workspace. Notification matches for resources now correctly limit results based on the permissions of the subscriber.

What’s new?

  • Easily rearrange the resource hierarchy using the new "Move" option in the left navigation menu.
  • Performance improvements for mod installation and exploring policy detail pages.
  • Documentation for GraphQL root queries.
  • Filter resources by turbot metadata using queries like $.turbot.id:1234.
  • AWS IAM operations (console login, credentials) have been added to the GraphQL API (and removed from the REST API). The Turbot CLI uses these capabilities to make cross-account access easy.

Bug fixes

  • Deleting large blocks of resources was unreliable due to timeouts and conflicts from ongoing background changes. While delete was mostly used because we didn’t support moves (added above), it will now work reliably when needed.
  • Changes to the type hierarchy (e.g. resource types, control types) made in new mod versions were not applied properly in the workspace. Now they do, as they should.

Enterprise

  • Efficiency improvements in backend event handling and data storage - reducing event flow, improving next task selection, improving mod installation performance, and reducing noise in notification data.
  • Security groups (e.g. load balancer, outbound internet access) are now defined in TE, making them specific and immutable to each version while allowing them to evolve over time (just like our other other serverless infrastructure). Custom security groups can still be defined in TEF if you prefer full control.
  • Turbot now uses the AWS RDS bundled 2015 and 2019 root certificate, allowing TED managed RDS Instances to be upgraded to the new certificate.

v5.0.0 [2019-12-18]

  • Folders, discoverable resources
  • IDs (not URNs)
  • Resource types & categories
  • Control types & categories
  • Policy types & categories
  • Permission types & levels
  • Terraform stacks with CMDB
  • Standard control types: approved, active, configured, cmdb, discovery, tags
  • Webhook events
  • Statistics & aggregation
  • Change history - resources, policies, controls, grants, etc
  • GraphQL
  • Filters
  • Dependencies / Dependents
  • Related policies / controls
  • Calculated policies
  • Tags / title
  • Multi-region
  • Serverless