A LogQL wildcard alternation that silently matches nothing in Loki
Notes. First seen on a Loki-backed detection rule set, reproduced on throwaway Loki 3.4.1 and 3.7.7 with synthetic datasets, root cause traced in source. Re-tests changed the story twice: “~20x slower” did not hold, and the trigger is a shared literal prefix, not missing parentheses. Corrections welcome.
At 10,000 feet
Setting: Grafana Loki alerting rules using LogQL regex line filters
(|~ "..."), commonly wildcard alternations like argv=.*chmod|argv=.*chown.
Problem: when the branches share a literal prefix (argv=) followed by
.*, Loki silently drops the .* and matches only the contiguous literal —
argv=chmod yes, argv=/usr/bin/chmod no. No error or warning; the rule never
fires. Confirmed on 3.4.1 through the current 3.7.7.
What this note establishes (measured, reproduced): (1) the correctness bug
with its minimal trigger A.*B|A.*C, traced to Loki’s line-filter simplifier;
(2) a conditional perf cliff — a correct wildcard alternation ran ~6x slower
than pure-literal, but only when its implied literal is common in the stream.
Takeaway: never share a literal prefix followed by .* across alternation
branches. Filter on a cheap literal first, then the precise regex
(|= "argv=" |~ "argv=.*(?:chmod|chown)") — fixes both problems.
Prior art: Loki’s literal prefilter is documented; these failure edges are not
Grafana’s 2020 “300x faster” writeup describes the optimization itself: a
pure-literal alternation like err|panic becomes a bytes.Contains prescan.
Credit goes there. New here are the failure edges — silent .* deletion and
the conditional perf penalty — in neither that writeup, Loki’s LogQL docs, nor
(as of 2026-08-31) any Loki GitHub issue.
The correctness bug: .* silently deleted after a shared prefix (measured)
460-line synthetic set, 110 true matches (ground truth confirmed with Python
re and Go regexp — this is Loki’s filter layer, not regex semantics):
{job="x"} |~ `argv=.*chmod|argv=.*chown` → 10 of 110 (only contiguous "argv=chmod" lines)
{job="x"} |~ `arg.*chmod|arg.*chown` → 0 of 110 (cleanest zero-result repro)
{job="x"} |~ `(argv=.*chmod)|(argv=.*chown)` → 110 of 110 (capturing parens mask it)
{job="x"} |~ `argv=.*(?:chmod|chown)` → 110 of 110 (different parse shape, fine)
{job="x"} |~ `(?:argv=.*chmod)|(?:argv=.*chown)` → 10 of 110 (non-capturing groups do NOT mask it)
{job="x"} |~ `argv=.*chmod` → 60 of 60 (single branch, fine)
{job="x"} |~ `usr.*chmod|bin.*chown` → correct (no shared prefix, fine)
{job="x"} |~ `argv=.+chmod|argv=.+chown` → correct (`.+` unaffected)
Root cause (measured, from source). Go’s parser factors the shared prefix
to argv=(?:.*chmod|.*chown) (Go’s own regexp.MatchString still matches
correctly — the bug is Loki’s). In pkg/logql/log/filter.go, simplifyConcat
carries baseLiteral = "argv=" into each branch, where the .* node is
skipped:
if sub.Op == syntax.OpStar && sub.Sub[0].Op == syntax.OpAnyCharNotNL {
continue
}
yielding a wrong contains("argv=chmod") filter. Skipping .* is valid for a
standalone unanchored filter; concatenating across it while carrying a base
literal is unsound. The region is byte-identical on main and
release-3.7.x, and unit tests cover shared prefixes only among plain
literals — an unintended edge case. A rule written the natural way never fires
and never errors.
The perf cliff is real but conditional (measured, reproduced)
Correct (capturing-parenthesized) forms over 1M lines, median of 5 runs:
| Query shape | Median | Ratio |
|---|---|---|
pure-literal alternation chmod\|chown\|chattr |
0.678 s | 1x |
wildcard alternation (argv=.*chmod)\|... |
4.124 s | ~6x slower |
| two-stage: literal filter, then the regex | 0.651 s | recovers fully |
The ~6x appeared only with argv= in most lines; at ~5% of lines the wildcard
form was not slower (0.77 s vs 0.69 s — Go’s regexp itself rejects at
bytes.Index speed). The original ~19x field figure did not reproduce; ~6x is
the controlled number, the direction the robust result.
The fix
{job="x"} |= `argv=` |~ `argv=.*(?:chmod|chown|chattr)`
The |= stage discards most lines before the regex runs, and the suffix-group
form parses to a shape the simplifier handles — on the million-line set it
matched pure-literal speed with the correct count. (If you must keep branch
alternation, use capturing parens; (?:...) does not help.)
Validation (2026-08-31, Loki 3.4.1 and 3.7.7)
Throwaway single-binary Docker instances, filesystem storage, full-window
query_range:
- Round 1 (3.4.1, 1M-line auditd-style stream, 500 planted matches): the
natural rule returned 0 of 500; probes showed the effective filter was
contains("argv=chmod"). Correct forms returned 500. Timings as tabled. - Round 2 (3.4.1 and 3.7.7, 460-line set): identical wrong counts on both
versions (10 of 110; 0 for
arg.*chmod|arg.*chown) — not fixed upstream. Shared prefix +.*is the isolated trigger; no shared prefix,.+,\S*, or single branch are correct; capturing parens mask it (they block Go’s prefix factoring),(?:...)does not. - Not documented, not reported: LogQL docs describe
|~only as Go RE2, “not fully anchored”; a GitHub issue search found no existing report.
Verdict: reproduced end-to-end on all current Loki versions, mechanism traced in source.
Licensed under the site footer’s CC BY 4.0. If this saved you a debugging session, the optional thanks link in the footer is appreciated, no obligation.