Skip to main content

Mapping Text Logs

Quick Demo​

Watch this short video to see how to create mappings from text logs:

What are Text Logs?​

Text logs are unstructured log entries that need to be parsed before they can be mapped to the OCSF schema. These logs typically come from:

  • System logs
  • Application logs
  • Security device logs
  • Network device logs
  • Custom application logs

Creating a Mapping from Text Logs​

To create a new mapping from text logs:

  1. Click the "New Mapping" button from your project dashboard
  2. Select "From Text Log" from the options presented
New Mapping Options

Working with Text Logs​

When working with text logs, you have two options for input:

  1. Paste Text Logs: Directly paste your text logs into the input area
  2. Upload Text File: Upload a file containing your text logs
Text Log Input Interface

Text Log Requirements​

For successful mapping, your text logs should:

  • Be consistent in format
  • Include all relevant security event information
  • Be in a readable format (not binary or encoded)

Supported Parser Types​

The app currently supports these parser types for text logs:

  • Windows Multiline
  • Grok
  • CEF (Common Event Format)

Parser Configuration​

The parser configuration screen allows you to:

  1. Select the appropriate log parser type
  2. Configure parser-specific options (like Grok patterns)
  3. View the raw log and parsed JSON side by side to verify correct parsing
  4. Adjust configurations as needed
Parser Configuration Interface

For Grok patterns, you can specify expressions to extract fields from unstructured text. The interface provides a preview of both the raw log and the resulting parsed JSON structure.

Syslog Header Parsing​

Many security logs include a syslog header. If your logs contain a syslog header:

  1. Check the "Include syslog parser" checkbox
  2. Select the syslog components present in your logs (Timestamp, Device, Application, etc.)
  3. Arrange them in the correct order using the Components Order interface
  4. If using Timestamp, provide the timestamp pattern in the appropriate format
  5. Optionally specify a Message Body Delimiter (default is whitespace)
Syslog Configuration Interface

Generating Mapping Rules​

After configuring your log parsing, you have two options for creating mapping rules:

Mapping Generation Options
  1. Analyze Logs: Let the AI generate mapping rules automatically
  2. Edit Directly: Create mapping rules manually in the editor

AI-Generated Mappings​

For AI-generated mappings:

  1. Provide clear additional context about your logs. This helps the AI understand the log format and generate accurate mappings.
  2. Click "Analyze Logs" to begin the process
  3. The AI will first present several OCSF class options that might match your log type
  4. Select the most appropriate class for your logs
OCSF Class Selection
info

The quality of the additional context greatly determines the AI accuracy. Make sure you provide clear and comprehensive explanation or specs about the sample logs. Formatting the context is NOT important.

Editing Mapping Rules​

The mapping editor presents three main tabs:

  1. Rules: Displays and allows editing of the mapping rules
  2. Logs: Shows sample logs and their mapped output
  3. Parser Config: Allows editing of the parser configuration

Rules Editor​

Mapping Rules Editor

The Rules tab displays mapping rules written in Fleak Eval Expression Language (FEEL). You can edit these rules in two modes:

  • Visual Mode: Click on fields to edit them through a user-friendly interface
  • Code Mode: Edit the FEEL expression directly in the code editor

The editor provides access to field definitions, documentation, and generation assistance for individual fields.

Logs Tab​

Mapping Rules Editor Logs Tab

In the Logs tab, you can:

  • Add or remove sample log entries using the "+" icon
  • View the mapping result for each sample log
  • Toggle between individual log view and table view using the icon next to log navigation

Parser Config Tab​

The Parser Config tab allows you to review and modify the parser configuration that was created during the mapping setup.

Best Practices for Text Mapping​

  1. Provide Diverse Samples: Include multiple variations of your SAME TYPE text logs to ensure comprehensive mapping
  2. Check Format Consistency: Ensure log format is consistent across all samples
  3. Include Context: When using AI generation, provide clear explanations of your log format
  4. Verify Results: Always review the mapping output for sample logs
  5. Test Before Production: Validate the mapping with a representative sample set

Downloading and Using the Mapping​

Once you've completed your mapping configuration:

  1. Click the "Download Expression" button
  2. The mapping configuration can be directly used with ZephFlow
  3. For implementation guidance, refer to the ZephFlow Cisco ASA to OCSF tutorial

Need Help?​

If you need assistance with text mapping:

  1. Check the Templates Guide for pre-configured mappings
  2. Review the JSON Mapping Guide for working with structured logs
  3. Contact support for assistance with specific log formats