Aggregating connections in Turbot Pipes
In a workspace, all connections of the same type are automatically aggregated. Now you also can define aggregators for subsets of connections.
A Steampipe aggregator bundles multiple connections for a plugin. For example, when you're using the AWS plugin, Turbot Pipes automatically creates an aggregator called aws_all. When you write a query like select * from aws_all.aws_s3_bucket, Steampipe runs against all the connections bundled by the aggregator, and does so concurrently.
A default aggregation
Here's a workspace with four connections: two for Dunder-Mifflin's Scranton office, and two for the New York office.
Turbot Pipes has automatically created the aws_all aggregator. Querying aws_all taps all four bundled connections.
A custom aggregation
Now let's add an aggregator for just the Scranton office. We'll use the new Create aggregator feature.
We'll use wildcards to match just the Scranton connections.
Now we can query aws_all for all four connections, or aws_scranton for just the two Scranton connections.
We could similarly bundle the New York connections as aws_newyork. And, of course, we can still query the Scranton and New York connections individually.
The power of aggregation
With Turbot Pipes aggregators you can group your connections together in varying contexts. Give it a try and share your experiences with us in the #pipes channel of our Slack community!