SQS Source Node
Quick Reference
- Data Asset: Select an SQS queue asset; queue URL, region and authentication belong to the asset/profile.
- Message Encoding: Defines how each message body should be interpreted. Options: CSV, JSON Object, JSON Array, JSON Lines, String Line, Text, XML. More info here. Default:
JSON Object. - Max Messages Per Poll: How many messages to fetch in each request. Maximum allowed by SQS is 10. Default:
10. - Wait Time (seconds): Seconds to wait for new messages before returning. Setting this above 0 enables long polling, which reduces empty responses and costs. Maximum is 20 seconds. Default:
20. - Visibility Timeout (seconds): Seconds that a received message stays hidden from other consumers while your workflow processes it. If processing takes longer than this, the message may be received again. Default:
30.
Configuration
Configuration keys below are relative to the node configuration. Use the field name in the panel when following instructions in the builder; required markers such as an asterisk are not part of the name.
| Field in the panel | Configuration key | Where to edit / when shown |
|---|---|---|
| Data Asset | dataAssetId | Select an SQS queue asset; queue URL, region and authentication belong to the asset/profile |
| Message Encoding | encodingType | Node panel — Defines how each message body should be interpreted. Options: CSV, JSON Object, JSON Array, JSON Lines, String Line, Text, XML. More info here. Default: JSON Object. Required: Yes. |
| Max Messages Per Poll | maxNumberOfMessages | Node panel — How many messages to fetch in each request. Maximum allowed by SQS is 10. Default: 10. Required: No. |
| Wait Time (seconds) | waitTimeSeconds | Node panel — Seconds to wait for new messages before returning. Setting this above 0 enables long polling, which reduces empty responses and costs. Maximum is 20 seconds. Default: 20. Required: No. |
| Visibility Timeout (seconds) | visibilityTimeoutSeconds | Node panel — Seconds that a received message stays hidden from other consumers while your workflow processes it. If processing takes longer than this, the message may be received again. Default: 30. Required: No. |
Overview
The SQS Source node allows your workflow to continuously read messages from an Amazon SQS queue. It acts as the starting point of your pipeline, polling the queue for new messages and passing each one downstream for processing. Messages are automatically removed from the queue after they are successfully processed.
Both standard and FIFO queues are supported.
The queue is defined by an AWS SQS Queue data asset, which supplies the queue URL and region, and links the AWS SQS connection profile holding the credentials. The node configures only how messages are decoded and polled.
Setting behavior
The queue URL and region are supplied by the data asset. The node has a Data Asset selector, not separate queue URL or region inputs.
To update the region, open an asset you own in Data Assets → Asset Information → Edit. If the Region input is present, change it while leaving Queue URL unchanged, then Save → Save Changes.
The Queue URL input is visible in the asset edit form, but Save rejects a changed URL because the queue location is fixed at creation. To use another queue, create or choose a replacement SQS queue asset for that URL, select it in the source node's Data Asset selector, and Save the node. Re-deploy the workflow for either change to affect an active deployment.
Related Nodes
- SQS Sink: Send processed records to an SQS queue
- Kafka Source: Consume streaming data from Kafka
- S3 Event Notifications Source: Read S3 objects announced on an SQS queue
- Data Assets: Defining the source queue
- Connection Profiles: AWS SQS credentials