Skip to main content

Connection Profiles

Connection Profiles store the credentials and endpoint for one external platform, so source and sink nodes authenticate without credentials being entered on each node.

Overview​

A connection profile answers how Fleak authenticates against an external platform, and where that platform lives — a broker address, a workspace host, a region. It does not name the individual topic, queue, table, or folder you read from or write to. That is a Data Asset.

Together the three layers divide as follows:

LayerAnswersExample
Connection ProfileWhich platform, and what credentialsAn AWS access key pair plus us-east-1
Data AssetWhich location on that platform, and the format of the data thereThe queue https://sqs.us-east-1.amazonaws.com/123456789012/orders, holding JSON objects
NodeHow the workflow reads or writes itBatch size, flush interval

One profile backs many assets, and one asset backs many workflows. Rotating a credential is a single edit on the profile rather than a change to every node that uses it.

note

Connection profiles replace what earlier versions of Fleak called Integrations. The settings page is now Settings → Connection Profiles.

Managing Connection Profiles​

Accessing Connection Profiles​

Click your username in the top left of the Fleak dashboard, select Settings, then Connection Profiles in the settings sidebar (/settings/connection-profiles). Data Assets is a separate item directly in the main sidebar, not a Settings page.

Creating a Connection Profile​

Click Add New, choose the platform in Add Connection Profile, then fill in the fields for that platform. Each platform's fields are listed in Supported Platforms below. Every profile also takes a name, which is what you select by when configuring a data asset or a node.

You can also create a profile without leaving the workflow builder, while configuring a node or a data asset.

Ownership and Sharing​

A profile has a single owner, the person who created it. Sharing it with your organization makes it selectable by other members; leaving it unshared keeps it private to you.

Secrets are masked server-side for anyone who is not the owner. A non-owner sees a profile's secret fields as •••••••• and cannot read the stored value back, whether through the UI or the API. Non-secret configuration — a region, a host URL, a Datadog site — stays visible, so a shared profile is still usable and auditable without exposing the credential.

Editing​

Open the profile, then use Edit in its Configuration card (available to the profile owner) and Save. Profile fields control the platform connection; this changes the profile itself. To change an asset's profile association, use its separate Connection Profile → Edit card in Data Assets. Asset-owned target and property edits are described in Managing Data Assets.

Fleak asks you to confirm on save:

Update this connection profile? Saving updates this connection profile. Running deployments keep the current settings until you redeploy them. New runs use the updated settings.

A running deployment keeps its current credentials and endpoint until you redeploy it. Runs that start later use the updated values.

Deleting​

A profile that data assets or workflows still reference cannot be deleted until those references are removed.

Supported Platforms​

The platforms below back the connectors that are generally available. Accounts with additional connectors enabled see further platforms on the settings page; those are not generally available, and the ones documented here are under Platforms Not Generally Available.

PlatformUsed by
AWSS3 Sink, File System Source, SQS Sink, SQS Source, S3 Event Notifications Source
Apache KafkaKafka Sink, Kafka Source
DatabricksDatabricks Sink, Databricks Zerobus Sink
DatadogDatadog Logs Sink
Microsoft Sentinel WorkspaceMicrosoft Sentinel Sink
SnowflakeSnowflake Sink
Splunk HECSplunk HEC Sink

AWS S3​

S3 assets use the shared AWS profile (type AWS), which also serves SQS assets.

FieldDescriptionRequired
RegionThe default AWS region for this connection profile, e.g. us-east-1. For an S3 asset, the bucket region detected by Fleak takes precedence over this default; this field does not change the bucket's region.Yes
AWS Access Key IDThe access key ID of the IAM user Fleak authenticates as.Yes
AWS Secret Access KeyThe matching secret access key. Stored encrypted and masked from non-owners.Yes

Required IAM permissions:

  • Profile validation uses AWS STS and needs no S3 permission at all.
  • s3:ListBucket on the bucket ARN — required by S3 Folder source nodes (File System Source) to list objects.
  • s3:GetObject on the object or prefix ARN — required by S3 Folder source nodes (File System Source) to read objects.
  • s3:PutObject on the destination object or prefix ARN — required by S3 Folder sink nodes.
  • s3:AbortMultipartUpload on the destination ARN — used by the default batch/multipart sink only to clean up a failed upload so incomplete parts do not remain billable. This is not s3:DeleteObject.
  • s3:ListAllMyBuckets on * — optional, used only for automatic bucket discovery. You can type a bucket name manually if this is denied.
  • For buckets or objects with default SSE-KMS encryption, grant kms:Decrypt for reads, kms:GenerateDataKey for writes, and both for multipart writes, on the KMS key ARN. Fleak has no KMS field in this form.

AWS SQS​

SQS assets use the same AWS profile (type AWS); AWS SQS is an asset family, not a separate profile type.

FieldDescriptionRequired
RegionThe default AWS region for this connection profile, e.g. us-west-2. An SQS asset uses its own region, normally derived from its Queue URL or explicitly set on the asset; that asset region takes precedence over this default.Yes
AWS Access Key IDThe access key ID of the IAM user Fleak authenticates as.Yes
AWS Secret Access KeyThe matching secret access key.Yes

Apache Kafka​

FieldDescriptionRequired
Broker/Bootstrap AddressComma-separated bootstrap servers, e.g. broker1:9092,broker2:9092.Yes
Authentication TypePLAINTEXT, SSL, SASL_PLAINTEXT, or SASL_SSL. Defaults to SASL_SSL.Yes
SASL MechanismPLAIN, SCRAM-SHA-256, or SCRAM-SHA-512. Shown only for the SASL protocols. Defaults to PLAIN.Yes, for SASL
UsernameSASL username.No
PasswordSASL password.No
Additional Connection PropertiesFree-form Kafka client properties for anything the fields above do not cover, e.g. ssl.* or client.id. Found under Show advanced options.No

The authentication type and SASL mechanism are written into the profile's Kafka client properties on save. Because the profile owns the security configuration, a node whose connection comes from a profile cannot override security.protocol, sasl.mechanism, sasl.jaas.config, or the SSL password properties on itself.

Databricks​

FieldDescriptionRequired
Host URLYour workspace URL, e.g. https://dbc-xxxx.cloud.databricks.com.Yes
Client IDThe service principal's application ID.Yes
Client SecretThe service principal's OAuth secret.Yes
Zerobus Ingest EndpointUnder + Show advanced options when creating the profile; used by Zerobus table assets.Required for Zerobus writes; optional for other Databricks assets

Use OAuth M2M (machine-to-machine) credentials from a Databricks service principal, not a personal access token. M2M tokens refresh automatically, which is what keeps a 24/7 pipeline authenticated.

Databricks Zerobus​

Zerobus uses the shared Databricks profile (type DATABRICKS), not a separate profile type. Its Host URL, Client ID, and Client Secret authenticate to the workspace. Configure Zerobus Ingest Endpoint under + Show advanced options when creating a profile, or in Configuration → Edit on an existing profile. A profile without that endpoint can serve other Databricks assets, but cannot supply the endpoint required for Zerobus writes.

Datadog​

FieldDescriptionRequired
SiteYour Datadog region domain — the host in your Datadog URL, e.g. us3.datadoghq.com. Enter the domain only, without https:// or a path. US1 accounts use datadoghq.com.Yes
API KeyA Datadog API key from Organization Settings → API Keys. This is the API key, not an Application key.Yes

The site determines the intake endpoint, so it decides both where and how logs are delivered.

Microsoft Sentinel Workspace​

Directory (Tenant) ID, Application (Client) ID, Client Secret, Log Analytics Workspace ID, and Azure Cloud (PUBLIC | US_GOV | CHINA) belong to the Microsoft Sentinel Workspace profile (type SENTINEL). Tenant ID is a profile setting, not an asset setting. A Sentinel DCR stream asset separately owns Data Collection Endpoint, Immutable DCR ID, and Stream Name. In Data Assets → Asset Information → Edit, existing Immutable Dcr Id and Stream Name properties can be changed with Save → Save Changes. Save rejects a changed Data Collection Endpoint even though its input is shown; use a replacement asset for another endpoint.

Snowflake​

The Snowflake profile owns Account Identifier, Authentication Method, Username, the credential for that authentication method, and optional Role. Role is under + Show advanced options when creating a profile; on an existing profile use Configuration → Edit. The Snowflake data asset owns Database, Schema, Table, and Warehouse. Warehouse is not a profile setting, and Role is not an asset setting. Existing Database, Schema, and Warehouse property inputs can be changed in Data Assets → Asset Information → Edit, then Save → Save Changes. A missing property key cannot be added there. Save rejects a changed Table location even though its input is shown; use a replacement asset for another table.

Splunk HEC​

The Splunk HEC profile (type SPLUNK_HEC) holds the HTTP Event Collector token for Splunk HEC Endpoint assets, and nothing else. The asset owns the HEC URL, Index, Verify SSL Certificates and Format.

FieldDescriptionRequired
HEC TokenThe HTTP Event Collector token from Splunk. Stored encrypted and masked from non-owners.Yes

Create the profile like any other: Add New, then Splunk HEC in Add Connection Profile. You can also create it inline while creating the asset: in Connect to Splunk, open Select a Connection Profile and choose + Add new connection profile. Splunk HEC profiles are listed in Settings → Connection Profiles. To rotate the token, the owner opens the profile, uses Edit in Configuration, then Save → Save Changes; running deployments keep the current token until you redeploy them. The owner can delete the profile from More actions, once no asset or workflow uses it (see Deleting).

The API Key profiles of Splunk HEC Endpoint assets created before this profile type existed were converted to Splunk HEC profiles in place, keeping their name and token. No action is needed.

A Splunk connection profile (Splunk URL, Validate SSL Certificates, Username, Password) is a different profile type. A Splunk HEC Endpoint asset cannot use it.

Platforms Not Generally Available​

caution

The platforms in this section back connectors that are not generally available. They appear in Add Connection Profile only for accounts where those connectors have been enabled. Each connector's own page states its availability.

PlatformUsed by
GCPPub/Sub Sink
Splunk ObservabilitySignalFx Sink
XSIAMXSIAM Sink

GCP​

The GCP profile owns Project ID and authentication. A Pub/Sub asset owns its Topic, which is selected or entered during asset creation. Changing Project ID on the profile does not make the existing asset's Topic editable; use the replacement-asset flow to choose another topic.

Splunk Observability​

A SignalFx ingest asset requires a Splunk Observability profile (type SIGNALFX), with Realm and Access Token. A generic API Key profile is not compatible. The asset creation form displays Realm read-only from the profile and stores it as the asset location; Default Dimensions belong to the asset. Edit Realm or Access Token on the profile. Existing dimension values can be changed in the asset's Asset Information → Edit card, but it has no add-key control. Its Realm input cannot save a changed realm. To switch the asset to another compatible profile, use Connection Profile → Edit → Save → Save Changes; the asset derives its Realm from that profile.

XSIAM​

The XSIAM profile (type XSIAM) holds the API key for XSIAM Sink datasets, and nothing else. The asset owns the Endpoint URL.

FieldDescriptionRequired
API KeyThe XSIAM API key, sent verbatim as the Authorization header.Yes

Create the profile like any other: Add New, then XSIAM in Add Connection Profile. You can also create it inline while creating the dataset: in Connect to XSIAM, open Select a Connection Profile and choose + Add new connection profile. XSIAM profiles are listed in Settings → Connection Profiles. To rotate the key, the owner opens the profile, uses Edit in Configuration, then Save → Save Changes; running deployments keep the current key until you redeploy them. Rotating the key keeps the same credential, so older XSIAM Sink nodes that select the credential directly keep working. The owner can delete the profile from More actions, once no asset or workflow uses it (see Deleting).

The API Key profiles of XSIAM datasets created before this change were converted to XSIAM profiles in place, keeping their key. They keep their name; a (API key) suffix is added only if you already had an XSIAM profile with that name. No action is needed.