googlecloudstorage

README

---
title: "Google Cloud Storage"
lang: "en-US"
draft: false
description: "Learn about how to set up a VDP Google Cloud Storage component https://github.com/instill-ai/instill-core"
---

The Google Cloud Storage component is a data component that allows users to upload data to Google's Cloud Storage.
It can carry out the following tasks:

- [Upload](#upload)
- [Read Objects](#read-objects)
- [Create Bucket](#create-bucket)



## Release Stage

`Alpha`



## Configuration

The component configuration is defined and maintained [here](https://github.com/instill-ai/component/blob/main/data/googlecloudstorage/v0/config/definition.json).




## Setup




In order to communicate with Google, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
withing the component's `setup` block, or you can create a **Connection** from
the [**Integration Settings**](https://www.instill.tech/docs/vdp/integration)
page and reference the whole `setup` as `setup:
${connection.<my-connection-id>}`.

| Field | Field ID | Type | Note |
| :--- | :--- | :--- | :--- |
| JSON Key File contents (required) | `json-key` | string | Contents of the JSON key file with access to the bucket. |




## Supported Tasks

### Upload

Upload data to Google Cloud Storage.


| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_UPLOAD` |
| Bucket Name (required) | `bucket-name` | string | Name of the bucket to be used for object storage |
| Object Name (required) | `object-name` | string | The name of the object to be created |
| Data (required) | `data` | string | The data to be saved in the object |



| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Authenticated URL (optional) | `authenticated-url` | string | Only users granted permission can access the object with this link |
| Gsutil URI (optional) | `gsutil-uri` | string | File path to this resource in Cloud Storage |
| Public Access (optional) | `public-access` | boolean | Whether the object is publicly accessible |
| Public URL (optional) | `public-url` | string | Anyone with this link can access the object on the public Internet |
| Upload Status | `status` | string | Status of the upload operation |






### Read Objects

Read objects from Google Cloud Storage.


| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_READ_OBJECTS` |
| Bucket Name (required) | `bucket-name` | string | Name of the bucket to be used for object storage |
| Delimiter | `delimiter` | string | The delimiter to use when listing objects |
| Prefix | `prefix` | string | The prefix to use when listing objects |
| Versions | `versions` | boolean | Whether to list all versions of an object |
| Start Offset | `start-offset` | string | The offset to start listing objects from |
| End Offset | `end-offset` | string | The offset to stop listing objects at |
| Include Trailing Delimiter | `include-trailing-delimiter` | boolean | Whether to include the delimiter in the object name |
| Match Glob | `match-glob` | string | Whether to match objects using a glob pattern |
| Include Folders As Prefixes | `include-folders-as-prefixes` | boolean | Whether to include folders as prefixes |



| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Audio Objects (optional) | `audio-objects` | array[object] | The audio objects in the bucket |
| Document Objects (optional) | `document-objects` | array[object] | The document objects in the bucket |
| Image Objects (optional) | `image-objects` | array[object] | The image objects in the bucket |
| Text Objects (optional) | `text-objects` | array[object] | The text objects in the bucket |
| Video Objects (optional) | `video-objects` | array[object] | The video objects in the bucket |






### Create Bucket

Read objects from Google Cloud Storage.


| Input | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Task ID (required) | `task` | string | `TASK_CREATE_BUCKET` |
| Bucket Name (required) | `bucket-name` | string | Name of the bucket to be used for object storage |
| Project ID (required) | `project-id` | string | The project ID to create the bucket in |
| Location | `location` | string | The location to create the bucket in |



| Output | ID | Type | Description |
| :--- | :--- | :--- | :--- |
| Result (optional) | `result` | string | The result of the operation |
| Bucket Name (optional) | `name` | string | The name of the bucket |
| Location (optional) | `location` | string | The location of the bucket |

Documentation

Index

• func Init(bc base.Component) *component
• func NewClient(jsonKey string) (*storage.Client, error)
• type Attributes
• type AudioObject
• type CreateBucketInput
• type CreateBucketOutput
• type DocumentObject
• type ImageObject
• type ReadInput
• type ReadOutput
• type TextObject
• type VideoObject

Functions

func Init

func Init(bc base.Component) *component

func NewClient

func NewClient(jsonKey string) (*storage.Client, error)

Types

type Attributes

type Attributes struct {
	Name               string            `json:"name"`
	ContentType        string            `json:"content-type"`
	ContentLanguage    string            `json:"content-language"`
	Owner              string            `json:"owner"`
	Size               int64             `json:"size"`
	ContentEncoding    string            `json:"content-encoding"`
	ContentDisposition string            `json:"content-disposition"`
	MD5                []byte            `json:"md5"`
	MediaLink          string            `json:"media-link"`
	Metadata           map[string]string `json:"metadata"`
	StorageClass       string            `json:"storage-class"`
}

type AudioObject

type AudioObject struct {
	Data       string     `json:"data"`
	Attributes Attributes `json:"attributes"`
}

type CreateBucketInput

type CreateBucketInput struct {
	BucketName string `json:"bucket-name"`
	ProjectID  string `json:"project-id"`
	Location   string `json:"location"`
}

type CreateBucketOutput

type CreateBucketOutput struct {
	Result   string `json:"result"`
	Name     string `json:"name"`
	Location string `json:"location"`
}

type DocumentObject

type DocumentObject struct {
	Data       string     `json:"data"`
	Attributes Attributes `json:"attributes"`
}

type ImageObject

type ImageObject struct {
	Data       string     `json:"data"`
	Attributes Attributes `json:"attributes"`
}

type ReadInput

type ReadInput struct {
	BucketName               string
	Delimiter                string
	Prefix                   string
	Versions                 bool
	StartOffset              string
	EndOffset                string
	IncludeTrailingDelimiter bool
	MatchGlob                string
	IncludeFoldersAsPrefixes bool
}

type ReadOutput

type ReadOutput struct {
	TextObjects     []TextObject     `json:"text-objects"`
	ImageObjects    []ImageObject    `json:"image-objects"`
	DocumentObjects []DocumentObject `json:"document-objects"`
	AudioObjects    []AudioObject    `json:"audio-objects"`
	VideoObjects    []VideoObject    `json:"video-objects"`
}

type TextObject

type TextObject struct {
	Data       string     `json:"data"`
	Attributes Attributes `json:"attributes"`
}

type VideoObject

type VideoObject struct {
	Data       string     `json:"data"`
	Attributes Attributes `json:"attributes"`
}