Pub/Sub Sink Node
The Pub/Sub Sink node is not generally available in Fleak Data Workflow. The workflow builder offers it only to accounts where it has been enabled. For the connectors that are generally available, see Data Assets.
This page is retained for accounts where it is enabled.
Quick Reference
In Destination, choose Asset Type and Asset Name, then open Configure Pipeline Settings. The fields listed here are the complete set of editable inputs for this Destination type, subject to the stated visibility conditions.
- Asset Type: Choose a Data Asset; UI selection determines the sink type on Save.
- Asset Name: Choose a Data Asset; reference to the selected data asset.
- Batch Size: Configure Pipeline Settings.
- Ordering Key Expression: Configure Pipeline Settings.
- Additional Properties: Configure Pipeline Settings; static message attributes as key-value rows; expand a row to edit Key and Value.
Destination has no Encoding selector; its encoding value is fixed by the panel. The asset owns Topic; the GCP connection profile owns Project ID and authentication. Asset Information → Edit shows Topic, but Save rejects a changed topic: create a replacement asset for another topic, select it in Asset Name, then Save. To change Project ID or credentials, edit the GCP profile in Settings → Connection Profiles.
Configuration
Configuration keys below are relative to the node configuration. Use the field name in the panel when following instructions in the builder; required markers such as an asterisk are not part of the name.
Destination panel
Choose Asset Type and Asset Name first. Type-specific inputs appear in Configure Pipeline Settings after an asset is selected; an asset error can prevent that section from appearing. A missing or unavailable asset is a reference problem, not proof that the connector does not exist.
The fields listed here are the complete set of editable inputs for this Destination type, subject to the stated visibility conditions.
| Field in the panel | Configuration key | Where to edit / when shown |
|---|---|---|
| Asset Type | UI only; no separate config key | Choose a Data Asset; UI selection determines the sink type on Save |
| Asset Name | dataAssetId | Choose a Data Asset; reference to the selected data asset |
| Batch Size | batchSize | Configure Pipeline Settings; required to save Destination — Maximum number of messages buffered before a publish is issued (1–1000). Default: 100. Required: Yes. |
| Ordering Key Expression | orderingKeyExpression | Configure Pipeline Settings — FEEL expression that returns the ordering key for each message (e.g. $.userId). Pub/Sub preserves ordering for messages sharing a key when message ordering is enabled on the subscription. Required: No. |
| Additional Properties | additionalProperties.* | Configure Pipeline Settings; static message attributes as key-value rows; expand a row to edit Key and Value |
Destination has no Encoding selector; its encoding value is fixed by the panel. The asset owns Topic; the GCP connection profile owns Project ID and authentication. Asset Information → Edit shows Topic, but Save rejects a changed topic: create a replacement asset for another topic, select it in Asset Name, then Save. To change Project ID or credentials, edit the GCP profile in Settings → Connection Profiles.
Legacy panel
The legacy panel has no Additional Properties editor.
Use this section only when the workflow opens the legacy editor. Asset-managed connection values can be read-only; see Managing Data Assets for available edits and the replacement-asset flow.
| Field in the panel | Configuration key | Where to edit / when shown |
|---|---|---|
| Google Pub/Sub Integration | integrationId | Legacy panel; read-only when dataAssetId is set, because the asset/profile supplies this value — Selects the GCP project ID and credential used for this topic. Required without a data asset; an asset-managed configuration can supply this value. |
| Topic | topic | Legacy panel; read-only when dataAssetId is set, because the asset/profile supplies this value — Pub/Sub topic short ID (e.g. my-topic). The project comes from the integration. Required without a data asset; an asset-managed configuration can supply this value. |
| Encoding | encodingType | Legacy panel — Format used to serialize each record. Options: CSV, JSON Object, JSON Array, JSON Lines. More info here. Required without a data asset; an asset-managed configuration can supply this value. |
| Batch Size | batchSize | Legacy panel — Maximum number of messages buffered before a publish is issued (1–1000). Default: 100. Required: No. |
| Ordering Key Expression | orderingKeyExpression | Legacy panel — FEEL expression that returns the ordering key for each message (e.g. $.userId). Pub/Sub preserves ordering for messages sharing a key when message ordering is enabled on the subscription. Required: No. |
Overview
The Pub/Sub Sink node publishes each record from your workflow as a message to a Google Cloud Pub/Sub topic. Messages are buffered and flushed in batches for throughput, and an optional ordering-key expression lets you route related messages in sequence. In Destination, project identity and credentials come from the selected asset/profile. An unmanaged legacy node selects Google Pub/Sub Integration directly.
Setting behavior
Related Nodes
- Pub/Sub Source: Consume messages from a Pub/Sub subscription
- Kafka Sink: Publish records to a Kafka topic