Repetition, Hygiene, and Macro Patterns

Repetition is what makes macro_rules! worth learning. A function takes a fixed number of typed arguments; a rule can accept a list of any length and generate code per element, because the repetition operators *, +, and ? wrap any matcher fragment in $( ... ) with an optional separator token.

macro_rules! max {
    ($first:expr $(,)?) => { $first };
    ($first:expr $(, $rest:expr)* $(,)?) => {{
        let mut m = $first;
        $( if $rest > m { m = $rest; } )*
        m
    }};
}

assert_eq!(max!(3, 9, 4), 9);
assert_eq!(max!(7,), 7);   // trailing comma accepted by the $(,)? idiom

Read the second arm from the inside out:

• $(, $rest:expr)* matches zero or more repetitions of "a comma followed by an expression". Inside a repetition, the metavariable $rest holds every element that was consumed — one binding per iteration, not one binding total.
• The transcriber's $( if $rest > m { m = $rest; } )* walks that same list and emits one if per element. Separators in the transcriber are your choice: here the statements separate themselves, and the reference explicitly allows changing them — a comma-separated list in the matcher can be re-emitted semicolon-separated with $( $i );*.
• $(,)? is the trailing-comma idiom: ? means "zero or one", so it silently swallows a final comma and makes both max!(7) and max!(7,) compile.

The three operators, precisely:

The separator may be any token that is neither a delimiter nor a repetition operator; , and ; are the conventions. Follow-set rules apply inside repetitions too: the separator must be able to follow the contents, the contents must be able to follow themselves, and whatever comes after the whole repetition must be able to follow either the contents or (for */?) whatever preceded it. That is why $( $e:expr ),* is accepted and $( $e:expr + )* is not.

Transcription enforces two hard rules, and both report errors against your definition rather than the call site:

1. Nesting must mirror the matcher, exactly. If $x was captured inside one $( ... )* in the matcher, it must appear inside exactly one repetition in the transcriber. Writing => { $x } (unwrapped), $( $( $x )* )* (double-wrapped), or $( $x )+ when the matcher used * are all rejected — the compiler needs to know how many iterations to emit, and only a mirrored nesting answers that.
2. Every transcriber repetition must contain at least one metavariable, and all metavariables sharing one repetition must have bound the same number of items. So ( $( $i:ident ),* ; $( $j:ident ),* ) => { $( ($i, $j) ),* } zips two lists pairwise and rejects (a, b, c; d, e) — mismatched counts are a caller error you get for free.

Nesting repetitions builds two-dimensional output: $( $($row),* );* re-emits a table row by row from a flat input, and $( ($($inner:tt)*) ),* walks grouped subtrees. Together with $(,)?, repetition is the mechanism behind every variadic macro you use — vec![a, b, c], format!("{}-{}", k, v), and matches!(x, A | B) all need "any number of arguments, one unit of code each", which no function signature can express. The previous topic covered rule anatomy in macro_rules! Fundamentals; the rest of this topic covers what repetition cannot do — and what hygiene does to the identifiers your expansions create.

Write a macro that introduces a temporary variable and two things surprise you: the caller cannot see it, and the macro cannot see the caller's. Both come from syntax hygiene — every identifier the transcriber writes literally carries the context of the macro's definition site. macro_rules! has mixed-site hygiene, a precise split:

let x = 1;

macro_rules! check {
    () => {
        assert_eq!(x, 1);   // literal `x` in the template
    };
}

{
    let x = 2;
    check!();               // still 1 — the definition-site binding wins
}

Generated locals are private to one expansion. A macro that defines a variable cannot hand it to its caller, and two invocations never share state:

macro_rules! define_tmp { () => { let tmp = 41; } }
macro_rules! use_tmp    { () => { println!("{}", tmp + 1); } }

// define_tmp!();   // creates `tmp`
// use_tmp!();      // error: cannot find value `tmp` in this scope

That kills the classical C-macro bug — a parameter silently redefining an outer variable, or a generated name shadowing the caller's — and it means a macro API must return values rather than leak bindings. The standard library depends on this: core::pin::pin! creates its temporary with a match binding whose name is safe precisely because expansions own their identifiers:

macro_rules! pin {
    ($value:expr $(,)?) => {
        match $value {
            __pin => unsafe { core::pin::Pin::new_unchecked(&mut __pin) }
        }
    };
}

The match trick is what keeps the borrowed temporary alive until the end of the enclosing statement; hygiene is what guarantees __pin cannot collide with anything the caller wrote. (Captured fragments keep the caller's context, so dbg!(x) still resolves your x — hygiene governs the identifiers the template types out, not the ones it captured.)

Generated paths resolve at the call site. Functions, types, and macros written into the template are looked up where the macro is used, which is a hazard for an exported macro: the consumer may have renamed your crate in Cargo.toml or never imported your helper macro. The fix is $crate, the one metavariable the engine fills in with the defining crate's real path:

#[macro_export]
macro_rules! helped {
    () => { $crate::helper!() };   // survives renaming and missing imports
}

$crate expands to a path that is valid at the invocation — the current crate from inside, the dependency's real name from outside — and non-macro items need the full path: $crate::inner::foo(). Two caveats interviewers like: $crate does not grant visibility (foo must still be pub or the consumer gets "is private" errors, which is why a macro can compile locally and fail downstream), and #[macro_export(local_inner_macros)] is the legacy shim that rewrites bare helper calls to $crate::helper!() for pre-2018 code; new macros write $crate:: explicitly.

Practical rules: capture the names that must refer to caller entities ($fn_name:ident, $ty:ty) instead of hardcoding them, let hygiene own the names you invent, and export only what you also make importable. Hygiene prevents accidental capture; $crate handles the one thing hygiene deliberately leaves open — path resolution at the site of use. The next chapter covers the pattern for inputs that capturing and hygiene cannot express at all: processing a token stream one tree at a time — and when the input is an annotated item rather than a token list, that is the job of Attribute Macros.

Some inputs defeat every matcher no matter how deeply you nest repetitions. Three shapes in particular: counting (how many arguments arrived?), sequential processing without separators (scan a = 1 AND b = 2 token by token and rewrite AND), and accumulating state while walking a list. Recursive muncher patterns solve all three with one trick — match a prefix of the token stream, do a single unit of work, then re-invoke the macro on the remainder.

The skeleton has three parts: a public entry rule that opens the machine, internal rules tagged with a marker token, and a base case.

macro_rules! sql_where {
    // entry: start with an empty accumulator
    ($($tt:tt)*) => { sql_where!(@scan [] $($tt)*); };

    // base case: nothing left, emit what we built
    (@scan [$($out:tt)*]) => { $($out)* };

    // specific rule first: rewrite AND to &&
    (@scan [$($out:tt)*] AND $($rest:tt)*) => {
        sql_where!(@scan [$($out)* &&] $($rest)*)
    };

    // fallback: copy one token tree, keep going
    (@scan [$($out:tt)*] $one:tt $($rest:tt)*) => {
        sql_where!(@scan [$($out)* $one] $($rest)*)
    };
}

// sql_where!(a = 1 AND b = 2)  expands to  a = 1 && b = 2

Mechanics worth naming:

• $one:tt peels exactly one token tree — a, =, 1, or an entire (group) — so each invocation advances a known distance. That unambiguous "where does the next element start?" is precisely what matchers lack when no separator exists.
• Arm order becomes control flow. The AND arm must precede the generic $one:tt arm or the fallback swallows it. Rules are still first-match-wins; munching just makes the ordering behave like a match on tokens.
• State travels in the arguments. [$($out:tt)*] is a bracketed accumulator re-passed on every call, and @scan separates internal rules from the public entry. Any token works as the marker — @ is idiomatic because it reads as non-Rust syntax — but nothing stops a caller from invoking @scan directly, so treat internal rules as private by convention and document the entry point.
• Counting is the same machine with arithmetic in the base case:

macro_rules! count_exprs {
    () => { 0usize };
    ($head:expr $(, $tail:expr)* $(,)?) => { 1usize + count_exprs!($($tail),*) };
}
const N: usize = count_exprs!(a, b, c, d);   // 4 — a const-evaluable expression

Each call peels one element; the expansion is 1 + 1 + 1 + 1 + 0, which type-checks and const-evaluates like any other expression.

Errors surface at the expansion site, so design the machine to fail usefully. When a muncher runs out of rules, the caller sees "no rules expected this token" pointed at your internal @scan arms — a message about your implementation. The fix is a catch-all arm that can only fire once the specific arms have failed:

macro_rules! props {
    // base case: nothing left
    () => {};
    // specific arm: a well-formed pair, then recurse on the rest
    ($k:ident => $v:expr, $($rest:tt)*) => {
        set(stringify!($k), &$v); props!($($rest)*)
    };
    // catch-all: anything else is a caller mistake
    ($($bad:tt)+) => {
        compile_error!("props!: expected `name => value,` pairs")
    };
}

Because the specific arm demands a real =>-shaped pair, input like foo = 1, falls through to the catch-all and the compiler reports your sentence at the caller's invocation line instead of complaining about internal rules. That is the payoff of the "errors point at the call site" rule from macro_rules! Fundamentals: a good macro converts every reachable failure mode into compile_error! text naming the fix.

One cost to remember: munching is one expansion per token. A 300-token input means 300 recursive expansions against the default #![recursion_limit = "128"], so deep inputs die with recursion limit reached while expanding. Raise the limit at the crate root only when the recursion is genuinely linear; otherwise prefer a single * repetition (one expansion regardless of length) and reserve munching for inputs that no matcher can describe. When the input is not a token list at all but an annotated item — a whole function or type you want to rewrite — declarative matching has reached its limit: that is the job of Attribute Macros.