Custom Lookup Tables

Custom lookup tables are reference-data files you upload to enrich log events during ingestion. Common use cases include correlating user IDs with department information, adding asset metadata based on hostnames, or attaching GeoIP context to IP addresses.

Two file formats are supported. The format is auto-detected from the file's contents at upload time:

• CSV — best for general-purpose reference data with arbitrary columns (user directories, asset inventories, network mappings, etc.).
• MMDB (MaxMind DB) — a binary format optimized for fast IP-range lookups. Use this when your reference data is keyed by IP and you want subnet matching (typical for GeoIP datasets like IPinfo Lite).

Creating Lookup Tables

1

Create a new lookup table

• Navigate to Library → Lookup Tables
• Click Upload Table

2

Upload your file

• Upload your CSV or MMDB file (drag-and-drop or browse). The format is detected automatically from the file's contents.

3

Provide details

• Provide a name (allowed characters: [A-Za-z0-9_-]) and optional description

4

Finish

• Click Create

File Format Requirements

• CSV: the first row must contain column headers.
• MMDB: must be a valid MaxMind DB binary file.
• The uploaded file must be smaller than 100MB.

Updating Lookup Tables

Lookup tables can be updated by uploading a new file. The updated data will be used for all subsequent ingestion, but does not retroactively modify previously-indexed logs. The replacement file must use the same format as the original.

Using Lookup Tables

Custom lookup tables can be referenced in custom transformations using VRL functions to add contextual data to your log events during ingestion. The same VRL functions work for both CSV and MMDB tables; the difference is in the match criteria you supply.

• CSV tables match on any column you specify (typically by exact value).
• MMDB tables match on the single key "ip" and accept any IPv4 or IPv6 address as a string. The MMDB format performs subnet/range lookups internally.

VRL Functions

get_enrichment_table_record - Returns exactly one matching record (or errors if zero or multiple matches found).

get_enrichment_table_record.vrl

get_enrichment_table_record(
  "table_name",                     # Lookup table name
  {"column_name": .field_to_match}, # Match criteria; N.B. for MMDB lookup tables, must be only {"ip": <IPv4 or IPv6 string>}
  ["column1", "column2"],           # Optional: columns to return (returns all if omitted)
  true                              # Optional: case sensitive (default: true)
)

find_enrichment_table_records - Returns an array of all matching records

find_enrichment_table_records.vrl

find_enrichment_table_records(
  "table_name",                     # Lookup table name
  {"column_name": .field_to_match}, # Match criteria
  ["column1", "column2"],           # Optional: columns to return (returns all if omitted)
  true                              # Optional: case sensitive (default: true)
)

Parameters

• Lookup table name: The name of the lookup table to lookup the values in.
• Match criteria: an object where keys are CSV column names and values are a vrl expression that you want to match against. You can specify multiple columns to match on - all criteria must be satisfied (AND operation).

  • Single column match:

    • {"user_id": .user_id}
    • Match where user_id column equals .user_id
  • Multiple column match:

    • {"hostname": .host, "environment": "production"}
    • Match where hostname AND environment both match
• Columns to return: (Optional) Under normal operation the entire record is returned as a VRL object. This allows for smaller returned objects, with only the specified keys.
• Case sensitive: (Optional, default true) Controls if match criteria is case sensitive or not.

Examples

Enrich user ID with department info:

enrich_user.vrl

# Lookup table: user_directory.csv with columns: user_id, name, department
user_info, err = get_enrichment_table_record(
  "user_directory", {"user_id": .user_id}, ["name", "department"]
)

if err == null {
  .user_name = user_info.name
  .department = user_info.department
}

Find all assets for a given hostname:

find_assets.vrl

# Lookup table: assets.csv with a hostname column (and presumably other columns)
assets = find_enrichment_table_records("assets", {"hostname": .host})
.matching_assets = assets
.asset_count = length(assets)

Look up an IP in an MMDB table:

enrich_ip_from_mmdb.vrl

# Lookup table: an MMDB file (e.g. ipinfo_lite). Match on the "ip" key — the
# MMDB format handles subnet matching internally.
record, err = get_enrichment_table_record("ipinfo_lite", {"ip": .source.ip})
if err == null {
  .source.geo.country_iso_code = record.country_code
  .source.geo.country_name = record.country
}