Regular Expressions: The Parts Worth Knowing
Greedy versus lazy, capture groups, and the catastrophic backtracking that takes a server down.
A regular expression describes a pattern of text. Most of the syntax is learnable in an afternoon; the parts that actually cause problems are greediness, anchoring, and what happens when a pattern meets input designed to hurt it.
The core
| Pattern | Matches |
|---|---|
. | Any character except newline |
\d \w \s | Digit, word character, whitespace |
* + ? | Zero or more, one or more, zero or one |
{2,5} | Between two and five |
^ $ | Start and end of string (or line, in multiline mode) |
[abc] / [^abc] | One of these / anything but these |
(...) / (?:...) | Capturing group / grouping without capturing |
Greedy by default
Quantifiers take as much as they can and give back only when forced. Against <b>one</b> and <b>two</b>, the pattern <b>.*</b> matches the entire string, not the first tag — .* ran to the end and backtracked to the last </b>.
Adding ? makes a quantifier lazy: <b>.*?</b> stops at the first match. A better habit still is to exclude the terminator — [^<]* — which cannot overshoot at all and runs faster.
Anchoring is a security property
An unanchored pattern matches anywhere. Validating with \d{3}-\d{4} accepts garbage 555-1234 more garbage, because you asked whether the string contains that shape, not whether it is that shape.
For validation, anchor both ends: ^\d{3}-\d{4}$. And in multiline mode $ matches before a newline, so ^\w+$ against "admin\nrm -rf" can pass — use \A and \z where the language offers them.
Catastrophic backtracking
This is the one that causes outages. Nested quantifiers over overlapping character sets create exponentially many ways to match, and a near-miss forces the engine to try them all.
^(a+)+$ against "aaaaaaaaaaaaaaaaaaaaaaaaaaX"
Roughly 226 paths before the engine concludes it cannot match. Add a few characters and it is hours. A single request can pin a CPU core — the class of bug is called ReDoS, and it has taken down real services.
Avoid it by never nesting quantifiers over the same characters, preferring precise character classes to .*, and applying a timeout where the language supports one. Go's RE2 and Rust's regex crate use a different algorithm with linear-time guarantees and no backtracking at all.
What not to parse
HTML and nested structures generally. Regular expressions cannot count arbitrary nesting — that is not a limitation of any implementation, it is what "regular" means. Use a parser. The same applies to a surprising degree to email: the RFC-complete pattern is thousands of characters, and the practical validation is "contains an @, then send a confirmation".
Habits that pay
- Use non-capturing groups
(?:...)unless you need the capture. - Name your captures —
(?<year>\d{4})— so a reordered pattern does not silently change what index means what. - Use verbose mode with comments for anything non-trivial; a regex is written once and read for years.
- Test against input designed to break it, not just input you expect.