Connect your GCP Organizations with Pipes

You asked, we delivered. Now you can import your entire GCP organization, with just a few clicks, as a set of Pipes connections. This new integration dramatically simplifies connecting GCP resources to Pipes, and enables you to query and analyze your cloud infrastructure more thoroughly than ever before.

Get ready to import

Here's the GCP structure we'll import.

To create a new GCP Integration, select GCP from the integration options.

In Setup Integration, you can accept the default handle for the integration, GCP, or choose your own.

To authenticate, drop the key file for a service account with Viewer permissions for the folder hierarchy you'll import. Then test to verify the credentials work.

Set permissions

Now choose which workspaces — all, some, or none — are permitted to use the connections.

Note the separation of concerns: Pipes administrators make connections available to workspaces, and workspace owners decide, on the workspace's Advanced / Connections page, which (if any) to activate for query.

Import folders and projects

Now click Create Integration to import the discovered folders and projects. Pipes performs the import, and creates a tree that matches your GCP structure, so delegation of permissions works the same way in Pipes as it does in GCP.

Add connections to a workspace

Subject to the connections they are permitted to see, workspace owners can now use any folder that was imported into Pipes to add its corresponding set of connections to a workspace. In this example, we choose the Pipeling Scale Testing Small folder which contains 15 projects.

The workspace owner uses the Add to schema button to make the corresponding connections available for query.

When a Pipes connection folder contains two or more connections, Pipes automatically creates an aggregator for them. Here's all_gcp which aggregates the 15 connections acquired from the Pipeling Scale Testing Small GCP folder.

Query the connections that were imported and added

Now, in the query pane, you can select and query any of the aggregated connections, or use the aggregator to query all of them. Here we query the aggregator.

And that's it! We've gone from a standing start to a queryable set of imported GCP connections in just minutes.

Try it yourself!

This new GCP integration addresses a common challenge faced by our users with large GCP footprints. It would be time-consuming to do this by hand or with Terraform scripting. Now you can bring your whole GCP organization into Pipes quickly, easily, and automatically. And when folders or projects change, Pipes syncs the changes.

We're thrilled to offer this enhanced GCP support and can't wait to see how it accelerates your cloud operations and security workflows. As always, we welcome your feedback and we're to help as you explore this powerful new capability.

See it in action