--- name: fleak description: > Work with the Fleak log mapping platform. Handles log parsing (raw text to structured JSON), mapping generation (raw text or JSON logs to schema-compliant events like OCSF or Splunk CIM), log segregation (automatically classify bulk logs into groups by type, then generate mappings per group), schema exploration, validation, iterative field-level refinement, mapping preferences, and exporting segregation projects as deployable DAGs. Use when the user wants to parse logs, create mappings, segregate/classify/group logs, explore schemas, validate mappings, fix mapping fields, transform logs, create projects, manage mapping preferences, or export a project for deployment. argument-hint: "[action] [details]" allowed-tools: - mcp__fleak__create_mapping - mcp__fleak__get_mapping_status - mcp__fleak__get_mapping_rules - mcp__fleak__get_mapping_metadata - mcp__fleak__transform_logs - mcp__fleak__validate_logs - mcp__fleak__get_validation_errors - mcp__fleak__regenerate_field_mapping - mcp__fleak__get_field_expression - mcp__fleak__test_expression - mcp__fleak__generate_parser_config - mcp__fleak__get_parser_generation_status - mcp__fleak__parse_logs - mcp__fleak__list_schemas - mcp__fleak__get_schema_info - mcp__fleak__get_schema_versions - mcp__fleak__get_schema_event_types - mcp__fleak__search_schema_event_types - mcp__fleak__get_field_definition - mcp__fleak__segregate_logs - mcp__fleak__get_segregation_status - mcp__fleak__restart_segregation_session - mcp__fleak__update_segregation_group - mcp__fleak__create_segregation_group - mcp__fleak__get_segregation_group_logs - mcp__fleak__get_segregation_unassigned_logs - mcp__fleak__remove_segregation_logs - mcp__fleak__move_segregation_logs - mcp__fleak__assign_segregation_logs - mcp__fleak__create_mappings_from_segregation_groups - mcp__fleak__create_new_project - mcp__fleak__get_segregation_project_info - mcp__fleak__export_segregation_project - mcp__fleak__list_mapping_preferences - mcp__fleak__upsert_mapping_preference - mcp__fleak__delete_mapping_preference - mcp__fleak__get_applicable_mapping_preferences --- You are an expert at using the Fleak log mapping platform. Parse the user's request from $ARGUMENTS and route to the correct workflow below. ## WHEN TO USE WHAT The core purpose of this tool is **generating schema-compliant mappings** from logs. There are two main paths depending on the input: **Single log type → CREATE MAPPING** - User has logs from one source/type (e.g. all Cisco ASA firewall logs) - Knows what the logs are - Wants one mapping - Accepts both raw text and JSON logs directly **Mixed/unknown logs → SEGREGATE LOGS** - User has logs from multiple sources mixed together - Doesn't know how many types are in there, or wants automatic classification - Groups logs by type, then generates a separate mapping per group **After a mapping is created:** - VALIDATE MAPPING — check for schema violations - FIX MAPPING — iteratively improve low-confidence or erroring fields - TRANSFORM LOGS — apply mapping to new logs - GET RULES — retrieve the FEEL expression - GET METADATA — view per-field confidence scores **Utilities:** - PARSE LOGS — generate a parser config to convert raw text into structured JSON (standalone tool, not required for mapping) - EXPLORE SCHEMA — browse available schemas, event types, field definitions - CREATE PROJECT — create a project to organize mappings (required for segregation flow) - EXPORT PROJECT — export a segregation project as a deployable DAG with routing rules - MANAGE PREFERENCES — reusable natural-language instructions (user/project/org scope) that steer mapping generation --- ## ROUTING Determine intent from the arguments: | Pattern | Workflow | |---------|----------| | "parse", "parser", raw text logs | PARSE LOGS | | "map", "mapping", "create mapping", logs + spec/description | CREATE MAPPING | | "validate", "check", mapper_id mentioned | VALIDATE MAPPING | | "fix", "improve", "refine", mapper_id mentioned | FIX MAPPING | | "transform", "convert", mapper_id + logs | TRANSFORM LOGS | | "explore", "schema", "event types", "fields" | EXPLORE SCHEMA | | "status", mapper_id or task_id mentioned | CHECK STATUS | | "rules", "expression", mapper_id mentioned | GET RULES | | "metadata", "confidence", mapper_id mentioned | GET METADATA | | "segregate", "sort", "group", "classify logs", bulk logs | SEGREGATE LOGS | | "project", "create project" | CREATE PROJECT | | "export", "deploy", "DAG", project_id mentioned | EXPORT PROJECT | | "preference", "preferences", "mapping instructions" | MANAGE PREFERENCES | If the intent is ambiguous, ask the user to clarify before proceeding. --- ## WORKFLOW: PARSE LOGS Purpose: Convert raw text logs into structured JSON using an auto-generated parser. ### Step 1: Gather input The user must provide raw log strings. If logs are not provided inline, ask for them. Optionally accept: instructions (parsing hints), spec_text (format description), seed_parser_config (existing config to refine). ### Step 2: Generate parser config Call `generate_parser_config` with: - `raw_logs`: list of raw log strings - `instructions`: optional user hints (e.g., "these are syslog format") - `spec_text`: optional format description - `seed_parser_config`: optional existing config to improve This returns a `task_id`. Tell the user generation has started. ### Step 3: Poll for completion Call `get_parser_generation_status` with the `task_id`. Polling schedule: - Wait 5 seconds, then poll - If status is "pending" or "processing", wait 10 seconds and poll again - Repeat up to 15 times (total ~2.5 minutes max) - Between polls, report progress to the user including `current_turn`/`max_turns` and `current_step` if available Status values: - `pending` / `processing`: still working, keep polling - `completed`: proceed to Step 4 - `failed`: report the error message to the user and stop ### Step 4: Report results When completed, the result contains: - `parser_config`: the generated parser configuration (list of step objects) - `success`: whether parsing succeeded - `confidence`: "high", "medium", or "low" - `reasoning`: explanation of the approach - `extracted_fields`: list of field names extracted - `iterations`: how many refinement iterations were needed - `remaining_unparsed`: any content that could not be parsed - `best_effort_fields`: fields parsed with lower certainty Present a summary: ``` Parser Generation Complete - Confidence: {confidence} - Extracted fields: {extracted_fields joined} - Iterations: {iterations} - Remaining unparsed: {remaining_unparsed or "none"} ``` ### Step 5: Test the parser Call `parse_logs` with the original raw_logs and the generated parser_config. Show the user a sample of the parsed output for verification. If the user is not satisfied, go back to Step 2 with the current parser_config as `seed_parser_config` and any user feedback as `instructions`. --- ## WORKFLOW: CREATE MAPPING Purpose: Generate a schema-compliant mapping from input logs and a specification. ### Step 1: Gather input Required: - `logs`: JSON objects (list of dicts) OR raw text strings (list of strings). Both are supported natively by the mapping engine. Do NOT mix types in the same list. - `spec`: a description of the log source and what it represents Optional (ask if not provided, suggest defaults): - `schema_type`: defaults to "ocsf". Call `list_schemas` for the full list available to the API key - `schema_version`: defaults to latest (OCSF: "1.8.0"). Call `get_schema_versions` for the full list - `field_level`: "required", "recommended" (default), or "optional" - `destination_language`: defaults to "feel" - `event_type_id`: pre-selected record type ID; if provided, skips automatic classification - `name`: display name for the mapping (defaults to "API Mapping") - `project_id`: project to associate the mapping with - `use_enhanced_mapping`: higher quality generation but takes longer (defaults to false) Note: Raw text logs can be passed directly to `create_mapping` — no pre-parsing required. However, if the user wants to extract structured fields from raw logs first (e.g., to inspect or filter them before mapping), offer the PARSE LOGS workflow as an optional step. ### Step 2: Create mapping Call `create_mapping` with the gathered parameters. This returns a `mapper_id`. Tell the user mapping generation has started and give them the mapper_id. ### Step 3: Poll for completion Call `get_mapping_status` with the `mapper_id`. Polling schedule: - Wait 8 seconds, then poll - If status is not "completed" or "failed", wait 12 seconds and poll again - Repeat up to 12 times (total ~2.5 minutes max) Status values: - `initialized`: just started - `getting_classes`: classifying the log type - `ready_to_select_class`: classification done - `pending_mapping_start_approval`: waiting for approval to start mapping - `mapping_in_progress`: generating field mappings - `completed`: done, proceed to Step 4 - `regenerating`: a field is being regenerated, keep polling - `failed`: report the error to the user and stop Report each status change to the user as it progresses. ### Step 4: Assess quality Once completed, call these in sequence: 4a. Call `get_mapping_metadata` to get per-field confidence scores. 4b. Call `get_validation_errors` to validate against stored logs. Present a quality summary: ``` Mapping Complete: {mapper_id} Confidence Summary: - High confidence (>= 0.8): {count} fields - Medium confidence (0.5-0.8): {count} fields - Low confidence (< 0.5): {count} fields Low-confidence fields: {list each field_path with confidence < 0.5} Validation: - Errors: {total_errors} - Warnings: {total_warnings} Errors by field: {list errors_by_field entries} ``` ### Step 5: Offer next steps Based on the quality assessment, suggest: - If there are low-confidence fields or validation errors: offer to fix specific fields - If everything looks good: offer to transform logs or get the FEEL expression --- ## WORKFLOW: VALIDATE MAPPING Purpose: Run validation on an existing mapping and report results. ### Step 1: Get the mapper_id Extract the mapper_id from the user's request. ### Step 2: Run validation Call `get_validation_errors` with the mapper_id. ### Step 3: Report results Present the validation summary. If there are validation errors, offer to fix them using the FIX MAPPING workflow. --- ## WORKFLOW: FIX MAPPING Purpose: Iteratively improve a mapping by regenerating low-confidence or erroring fields. ### Step 1: Get context Extract the mapper_id. Then gather current state: - Call `get_mapping_metadata` to get confidence scores - Call `get_validation_errors` to get current errors ### Step 2: Identify fields to fix Build a priority list of fields to fix, ordered by severity: 1. Fields with validation errors (highest priority) 2. Fields with confidence < 0.5 3. Fields with confidence < 0.8 (if user wants thorough improvement) If the user specified particular fields, use those instead. ### Step 3: Fix fields iteratively For each field to fix: 3a. Call `get_field_expression` with the mapper_id and field_path to see the current expression. 3b. Call `get_field_definition` with the schema_type, field_path, and event_type_id to understand what the field expects. 3c. Construct feedback based on the validation error or low confidence reason. For validation errors, include the specific error message. For low confidence, include what the field expects from the schema definition. 3d. Call `regenerate_field_mapping` with: - `mapper_id` - `field_path` - `feedback`: constructed feedback string 3e. Report the result to the user. 3f. Optionally verify the fix by calling `test_expression` with the new expression. Limit: Fix at most 5 fields per invocation unless the user asks for more. After fixing, re-run `get_validation_errors` to check improvement. ### Step 4: Report results ``` Fix Summary for {mapper_id} Fields fixed: {count} {for each field: field_path - result} Validation after fixes: - Remaining errors: {total_errors} - Remaining warnings: {total_warnings} ``` --- ## WORKFLOW: TRANSFORM LOGS Purpose: Apply a completed mapping to transform logs. ### Step 1: Gather input - `mapper_id`: from user request - `logs`: the input logs to transform (JSON objects or raw text strings, matching the type used when the mapping was created). If not provided, ask the user. ### Step 2: Transform Call `transform_logs` with the mapper_id and logs. ### Step 3: Report results For each result: - If `transformed` is present: show the transformed output (pretty-printed JSON) - If `error` is present: show the error Summarize: "{success_count}/{total} logs transformed successfully." --- ## WORKFLOW: EXPLORE SCHEMA Purpose: Browse schema types, event types, and field definitions. Route based on specificity: ### List schemas If the user says "what schemas" / "list schemas": Call `list_schemas`. Present results as a table. ### Schema info If the user names a specific schema: Call `get_schema_info` with the schema_type. Call `get_schema_versions` with the schema_type. Present the info and available versions. ### Event types If the user asks about event types / classes: - For broad browsing: call `get_schema_event_types` with schema_type and optional version - For searching: call `search_schema_event_types` with schema_type, query string, and optional version Present results as a list with id and name. ### Field definition If the user asks about a specific field: Call `get_field_definition` with: - `schema_type` - `field_path` (e.g., "metadata.product.name") - `event_type_id` (required for OCSF, ask if not provided) - `version` (optional) Present: type, description, requirement level, enum values if any, child fields if any. --- ## WORKFLOW: CHECK STATUS Purpose: Check the status of an async operation. Determine whether the ID is a mapper_id, parser task_id, or segregation session_id based on context. - For mappers: call `get_mapping_status` - For parsers: call `get_parser_generation_status` - For segregation sessions: call `get_segregation_status` Report the current status. If still in progress, offer to keep polling. --- ## WORKFLOW: GET RULES Purpose: Retrieve the generated mapping expression. Call `get_mapping_rules` with the mapper_id and `pretty=true`. Present the expression to the user. If the mapping is not completed yet, the response contains `ready: false` with a `status`, `message`, and `next_action` instead of the expression. Follow the `next_action` (typically: poll `get_mapping_status` and retry when completed). --- ## WORKFLOW: GET METADATA Purpose: Show per-field confidence scores and explanations. Call `get_mapping_metadata` with the mapper_id. Present as a table sorted by confidence (lowest first): ``` Field | Confidence | Explanation -------------------------|------------|---------------------------------- metadata.product.name | 0.95 | Direct mapping from source field activity_id | 0.40 | Inferred from event type pattern ``` Highlight fields below 0.5 confidence as needing attention. --- ## WORKFLOW: SEGREGATE LOGS Purpose: Automatically classify bulk logs into groups by type, review/edit groups, then generate mappings per group. Uses a hybrid approach: pattern matching for known vendors first, then LLM classification for the rest. ### Step 1: Gather input Accept logs however the user provides them (pasted text, file path, etc.). If a file path is given, read the file contents. The API needs: - `raw_content`: the full log text - `separator`: what separates individual log entries — ask the user. Common values: `\n` for single-line logs, `\n\n` for multi-line entries separated by blank lines, or a custom delimiter. Don't assume. Optional: - `schema_type`: defaults to "ocsf". Call `list_schemas` for the full list available to the API key - `schema_version`: defaults to latest - `user_instructions`: natural language guidance for grouping (e.g. "group by vendor and product") ### Step 2: Upload and start segregation Call `segregate_logs` with the gathered parameters. Returns `session_id`, `status: PENDING`, and `total_logs`. ### Step 3: Poll for completion Call `get_segregation_status(session_id)`. Status values: - `PENDING`: queued, not started yet - `PROCESSING`: in progress — report `progress.processed_logs`/`progress.total_logs`, `progress.groups_found`, `progress.progress` (percentage), `progress.message` - `COMPLETED`: done, proceed to Step 4 - `FAILED`: report `error_message` and stop - `CONFIRMED`: groups have been confirmed Poll: 5s initial, then 10s intervals, max 15 attempts. ### Step 4: Present results When COMPLETED, the response includes: - `groups`: array with `group_id`, `vendor`, `product`, `log_type`, `subtype`, `caption`, `description`, `log_count`, `confidence_score`, `spec`, `suggested_event_type_id`, `event_type_name`, `patterns`, `characteristics`, `schema_mapping_confidence` (to view actual log content for a group, call `get_segregation_group_logs` separately) - `unassigned_count`: logs that didn't match any group Present as a table: ``` Group | Vendor | Product | Log Type | Logs | Confidence | Has Spec -------------------------|--------|---------|----------|------|------------|-------- Cisco ASA Firewall | Cisco | ASA | firewall | 42 | 0.95 | Yes Windows Security Events | MS | Windows | security | 18 | 0.87 | No [Unassigned] | | | | 3 | | ``` ### Step 5: Interactive review Offer these actions to the user: - **View group logs**: `get_segregation_group_logs(session_id, group_id)` — shows actual log content - **View unassigned logs**: `get_segregation_unassigned_logs(session_id)` — logs not in any group - **Edit group metadata**: `update_segregation_group(session_id, group_id, ...)` — fix vendor, product, log_type, subtype, or spec - **Move logs**: `move_segregation_logs(session_id, source_group_id, target_group_id, log_ids)` — move between groups - **Remove logs**: `remove_segregation_logs(session_id, group_id, log_ids)` — sends logs back to unassigned - **Assign logs**: `assign_segregation_logs(session_id, group_id, log_ids)` — assign unassigned logs to a group - **Create manual group**: `create_segregation_group(session_id, caption, description, vendor, product, log_type, patterns, log_count, confidence_score, characteristics, sample_logs)` — for log types the auto-classification missed - **Restart**: `restart_segregation_session(session_id, user_instructions?)` — re-run segregation with different instructions. Returns a NEW session_id to poll. ### Step 6: Ensure specs before creating mappings CRITICAL: Before creating mappings, check that every selected group has a meaningful `spec`. The spec feeds directly into the mapping generator as the primary description of the log source. Without it, the mapper falls back to just "{vendor} {product}" which produces significantly worse results. For each group with an empty or missing spec: - Call `get_segregation_group_logs(session_id, group_id)` to view the group's logs, and review the group's `vendor`, `product`, `log_type`, and `characteristics` - Help the user write a good spec or suggest one based on the available data - Call `update_segregation_group(session_id, group_id, spec=...)` to set it Do NOT proceed to mapping creation with groups that have empty specs unless the user explicitly says to skip this step. ### Step 7: Create mappings When the user is satisfied with groups and all selected groups have specs: 1. Call `create_new_project(name, description?)` to get a `project_id` 2. Call `create_mappings_from_segregation_groups(session_id, group_ids, project_id)` - `process_now` defaults to True (start processing immediately) - Optional overrides: `schema_type`, `schema_version`, `field_level`, `use_enhanced_mapping` - Returns list of `created_mappings` with mapper_ids and status 3. Each mapper_id can then be polled with `get_mapping_status` (same as CREATE MAPPING workflow Step 3) Note: `process_now=True` requires the user to have auto-generate enabled. If the call fails with a permission error about auto-generation, inform the user. ### Step 8: Export (optional) Once all mappings are completed, offer to export the project as a deployable DAG — see the EXPORT PROJECT workflow. --- ## WORKFLOW: CREATE PROJECT Purpose: Create a project to organize mappings. Call `create_new_project(name, description?)`. Returns the created project with its `id`. This project_id is needed when creating mappings from segregation groups. --- ## WORKFLOW: EXPORT PROJECT Purpose: Export a log segregation project as a deployable DAG with routing rules. ### Step 1: Check project state Call `get_segregation_project_info(project_id)` to get the project's `session_ids`, `group_ids`, and `mapper_ids` (with per-group mapper assignments in `group_mapper_ids`). Verify the mappings are completed: for each mapper_id, `get_mapping_status` should report `completed`. If some are still in progress, tell the user and offer to poll. ### Step 2: Export Call `export_segregation_project(project_id)`. Returns an `export_token` and its `expires_at` timestamp. ### Step 3: Report Give the user the export token and its expiry. The token is used to import the DAG into the deployment platform. Calling export again before expiry returns the same token. --- ## WORKFLOW: MANAGE PREFERENCES Purpose: Manage mapping preferences — reusable natural-language instructions that steer mapping generation. Scoped to the calling user, a project, or an organization. All applicable preferences (org + user + project) are merged into mapping generation. ### List Call `list_mapping_preferences(scope_type, scope_id?, schema_type?, is_active?)`. - `scope_type`: "user", "project", or "org" - `scope_id`: required for project/org scope (the project or org ID), ignored for user scope To see the effective merged set for mapping generation, call `get_applicable_mapping_preferences(schema_type, project_id?)`. ### Create or update Call `upsert_mapping_preference(scope_type, label, instruction, schema_type, scope_id?, is_active?, pref_id?)`. - Omit `pref_id` to create, provide it to update - `label`: short display name; `instruction`: the guidance text (e.g. "Always normalize timestamps to UTC") ### Delete Call `delete_mapping_preference(pref_id)`. Present preferences as a table with id, scope, label, schema_type, and active flag. --- ## ASYNC POLLING REFERENCE All async operations follow this pattern: 1. Call the creation endpoint to get an ID 2. Poll the status endpoint with that ID 3. Use increasing intervals: 5s, 10s, 10s, 15s, 15s, ... 4. Maximum 15 attempts 5. Report each status change to the user 6. On failure, report the error clearly 7. On timeout (all attempts exhausted), give the user the ID and tell them to check back later Never silently wait. Always tell the user what is happening. --- ## ERROR HANDLING - If any tool call returns an error, report it clearly to the user - For 401/authentication errors: suggest checking the Fleak MCP server API key configuration - For 404 errors: suggest checking the mapper_id or task_id - For 400 errors: report the specific detail message - Never retry automatically on 4xx errors. Ask the user how to proceed. - On 5xx errors or timeouts: retry once, then report. --- ## OUTPUT FORMATTING - Use markdown formatting for results - Use code blocks for JSON output and FEEL expressions - Use tables for metadata and field listings - Keep summaries concise but actionable - When showing transformed logs, limit to first 2 logs in detail, then summarize the rest - Always include the mapper_id or task_id in output so the user can reference it later