Skip to main content

Filter Node

The filter node allows you to selectively process events in your ZephFlow pipeline based on specific conditions. It evaluates each incoming event against an expression written in the Fleak Eval Expression Language. Events that evaluate to true are passed to downstream nodes, otherwise are dropped from the pipeline.

For more details of using FEEL, please refer to FEEL Reference

Key Features​

  • Conditional Processing: Apply downstream transformations only to events that match specific criteria
  • Data Validation: Filter out invalid or incomplete events early in your pipeline
  • Conditional Branching: Use multiple filter nodes to create branched pipelines for different event types
  • Performance Optimization: Reduce load on downstream processors by dropping unnecessary events early

Attaching Filter Node Using SDK​

ZephFlow flow = ZephFlow.startFlow();
flow.filter("your FEEL filter condition");

Basic Usage​

# Keep only events where the status field equals "success"
$.status == "success"

# Keep only events with response time under 500ms
$.response_time < 500

# Keep only events that have a non-empty items array
size_of($.items) > 0

# Complex conditions can be combined with logical operators
$.status == "success" and $.response_time < 500

# Non boolean evaluation result will cause event to be dropped
dict(value=$.status)

Examples​

Basic Filtering​

# Process only error events
$.level == "ERROR"

# Filter out null values
$.user_id != null

Complex Conditions​

# Process only high-priority errors
$.level == "ERROR" and $.priority > 3

# Process events from specific sources OR with high severity
$.source == "payment-gateway" or $.severity == "critical"

# Process events with complex nested conditions
($.type == "transaction" and $.amount > 1000) or ($.type == "login" and str_contains($.ip, "192.168."))

Using Functions​

# Filter based on array content
size_of($.tags) > 0

# Filter based on string content
str_contains(lower($.message), "error")

# Filter using case expressions
case(
    $.status == 200 => true,
    $.retry_count < 3 => true,
    _ => false
)

Type-Specific Filtering​

# Numeric comparisons
$.age >= 18 and $.age <= 65

# String operations
str_contains(lower($.user_agent), "mozilla")

# Timestamp filtering
ts_str_to_epoch($.timestamp, "yyyy-MM-dd") > 1609459200000  # After January 1, 2021 12:00:00 AM UTC
  • eval: Transform events using the same expression language
  • assertion: Validate that events meet specific conditions using FEEL