Google Cloud Storage integration

Google-cloud-storage is Google's object storage service, where teams keep media, backups, logs, and datasets at any scale. The NoClick Google-cloud-storage node works against the Storage API, so a single workflow can create and delete buckets, copy and compose objects, bulk restore deleted files, and organize everything into folders and managed folders. It also covers the administrative side: bucket and object ACLs, default object ACLs, HMAC keys, notification configs, anywhere caches, and even bucket relocation. Drop it onto the canvas to turn storage housekeeping into steps that run on their own.

What you can automate with Google Cloud Storage

Google Cloud Storage operations

Triggers1

Buckets10

IAM6

Objects14

Notifications4

Projects6

Operations4

Bucket ACLs6

Default Object ACLs6

Object ACLs6

Folders6

Managed Folders7

Anywhere Cache7

Storage Intelligence6

Templates using Google Cloud Storage

Cloud Storage Object Watch

Watches every object landing in your ingest bucket and tells the data channel when the overnight partner drop is late, far smaller than usual, or written under the wrong prefix. Data deliveries fail quietly. The partner's job exports zero rows, or lands at 05:10 instead of 04:00, or begins writing to a path with the date in a different order, and you find out at ten in the morning when a dashboard is wrong and nobody can say for how long. This agent checks each object as it arrives against the prefixes, sizes and timings you describe, compares it with the same prefix on the days before, and puts one line in the data channel when something is off. ## What people use it for - **The overnight drop, checked overnight** - A partner file due at four in the morning is read at four in the morning. A missing or half sized delivery reaches the channel before the first ingest job has run on top of it. - **Zero bytes is not a delivery** - An empty file uploads successfully, satisfies every existence check ever written, and breaks whatever reads it next. The size goes into the note beside what that same path weighed on the days around it. - **A prefix nobody agreed to** - When an export starts writing one directory across, or with the date arranged differently, downstream jobs read an empty prefix and report no problem at all. Every object gets its path compared with what the rules expect. - **It only ever reads** - No object is moved, rewritten or deleted, no lifecycle rule or bucket setting is changed, and nothing upstream is rerun. Reprocessing remains a decision for the people who own the pipeline. ## Before you fork **What do I connect, and what permission does it need?** A Google Cloud service account that can list objects and read object metadata on the bucket, the object notification wired into this workflow, plus a Slack channel the data team reads. List and read is the whole requirement, since it never writes, and granting nothing beyond that is the cleanest way to keep it harmless. What a good delivery looks like goes in as a variable, in your own words. **Our loads write hundreds of part files a night, is that hundreds of runs?** Yes, one run per object, which is why most teams scope the trigger instead of watching a whole bucket. Point it at the prefixes where a late or missing delivery actually costs you something, or at the manifest object that completes a drop, and let the parts land unwatched. A bucket taking twelve deliveries a day costs about twelve short runs. **Can it tell me whether the data inside is right?** No, and it states which checks it made rather than implying more. It sees path, name, size, content type and write time, plus every neighbouring object, so it catches late, missing, empty, oddly sized, badly named and misfiled deliveries. Row counts, schema drift and duplicated batches live inside the file, which puts them in your pipeline's own validation step.

80804 nodes

Questions

Yes. The Google-cloud-storage node has an On New Object trigger that starts a workflow whenever an object appears in a bucket. From there you can copy the object, compose it with others, or fan the event out to any other node in the same run.

It creates and deletes buckets, and it builds structure with create folder, create managed folder, and delete folder, including a recursive delete for clearing an entire tree at once. Wire these to a form, a schedule, or a trigger so new projects get their storage laid out without manual setup.

It sets bucket ACLs, default object ACLs, and individual object ACL entries, and it can delete any of those when access needs to be pulled back. It also creates and deletes HMAC keys, which is how you rotate the credentials that apps use to talk to your buckets.

Yes. The node includes a bulk restore objects operation that brings back files deleted within the bucket's soft-delete window. You can run it on demand or on a schedule to undo an accidental mass delete.

Add the Google-cloud-storage node and connect your Google Cloud credentials in the node's credential panel. Once connected, every operation runs against your own project, so buckets, objects, and keys are all created under your account.