Skip to content
advanced Phase 16 · Systems Projects and Interview Preparation

CLI Tool Capstone

Build a command-line search or parsing tool with clap, iterators, and error types.

1h 30m
0 problems
Topic Progress 0%

Modeling the Whole CLI in One Derive Struct: Flags, Options, and Subcommands

Build logscan, a tool that searches log files: logscan error app.log, logscan --level warn --since 24h scan app.log out/, logscan stats app.log. Every flag you add by hand with std::env::args() is a string match, a parse, a help string, and a bug waiting to happen. clap's derive API turns the CLI's grammar into the type system: one struct per argument group, one enum per subcommand, and the parser, validation, and --help output all generate from it.

use clap::{Parser, Subcommand, ValueEnum};

#[derive(Parser)]
#[command(name = "logscan", version, about = "Search and summarize log files")]
struct Cli {
    /// Level threshold to report
    #[arg(long, value_enum, default_value_t = Level::Info)]
    level: Level,
    #[command(subcommand)]
    command: Command,
}

#[derive(ValueEnum, Clone, Copy, Debug, PartialEq)]
enum Level { Error, Warn, Info, Debug }

#[derive(Subcommand)]
enum Command {
    /// Search files for lines at or above the level
    Scan {
        /// Files to search; omit to read stdin
        files: Vec<std::path::PathBuf>,
        /// Case-insensitive substring filter
        #[arg(long)]
        contains: Option<String>,
    },
    /// Print per-level counts for a file
    Stats { file: std::path::PathBuf },
}

fn main() -> anyhow::Result<()> {
    let cli = Cli::parse();
    match cli.command {
        Command::Scan { files, contains } => run_scan(cli.level, files, contains.as_deref()),
        Command::Stats { file } => run_stats(cli.level, file),
    }
}

Read the decisions, because interviewers ask why each one exists. Level is a ValueEnum, not a string: clap validates --level error at the parse boundary and the compiler guarantees no other values exist inside run_scan. PathBuf, not String, for file arguments — the OS hands you paths that may not be valid UTF-8, and PathBuf is the type that admits them. Vec<PathBuf> with no required attribute makes zero or many files legal in one field, and clap reads stdin when the list is empty (your code decides; clap only parses). contains: Option<String> distinguishes flag absent from flag present with a value — a distinction raw args() matching gets wrong constantly. #[command(version, about)] wires --version and the about line from Cargo.toml metadata into --help for free.

Cli::parse() is the boundary that earns its keep: on a bad flag it prints a usage error and exits with code 2 before your main logic exists, so run_scan can assume a well-formed world. Note what main returns: anyhow::Result<()> — in a binary crate, anyhow is the right error type because there is exactly one consumer (the user at the terminal) and context over a typed chain beats exhaustive matching at top level. The typed story belongs inside the library, and the next chapter builds it.

Tests belong next to the grammar while it is still cheap. Unit-test the parser by constructing argv explicitly:

#[cfg(test)]
mod tests {
    use super::*;
    use clap::CommandFactory;

    #[test]
    fn cli_definition_is_consistent() {
        Cli::command().debug_assert();   // arg ids, groups, conflicts
    }

    #[test]
    fn parses_scan_with_level_and_filter() {
        let cli = Cli::try_parse_from([
            "logscan", "--level", "warn", "scan", "a.log", "b.log", "--contains", "timeout",
        ]).unwrap();
        assert_eq!(cli.level, Level::Warn);
        match cli.command {
            Command::Scan { files, contains } => {
                assert_eq!(files.len(), 2);
                assert_eq!(contains.as_deref(), Some("timeout"));
            }
            _ => panic!("expected Scan"),
        }
    }
}

Cli::command().debug_assert() is the sleeper hit: it verifies the entire derived definition (unique ids, valid conflicts, well-formed groups) in one test you never have to update. try_parse_from takes any iterator of strings, so parser tests are pure functions — no process-global state, no subprocess. That property is what the Async and Concurrency Interview Drills topic predicted about good boundaries: parse at the edge, then pass typed values inward, and the core becomes trivially testable — which is exactly where the iterators go.

Streaming a Million-Line File with BufRead, Iterators, and a Typed Error Enum

The scan core must handle a 2 GB log without loading it into memory, filter lines by level and substring, and report both kinds of failure correctly: an unreadable file is an IO problem, a malformed line is a parse problem. Two mechanisms do the work: BufRead lines as an iterator, and a custom error enum that keeps the two failure families distinct.

Start with the line source. File::open yields a File, which implements Read; wrapping it in BufRead's BufReader adds an 8 KB buffer so each next_line() is a cheap memcpy instead of a syscall. Then treat lines as a stream:

use std::fs::File;
use std::io::{self, BufRead, BufReader};
use std::path::Path;

fn scan_file(path: &Path) -> Result<Vec<ScanHit>, ScanError> {
    let file = File::open(path)?;                       // IO error, propagated
    let reader = BufReader::new(file);
    let hits = reader
        .lines()
        .enumerate()
        .map(|(i, line)| (i + 1, line?))               // per-line IO error → ScanError
        .filter_map(|(lineno, line)| match parse_line(&line) {
            Ok(entry) if entry.level >= threshold => Some(Ok(ScanHit { lineno, entry })),
            Ok(_) => None,
            Err(cause) => Some(Err(ScanError::Parse { source_line: lineno, cause })),
        })
        .collect::<Result<Vec<_>, _>>()?;               // stop at first parse error
    Ok(hits)
}

Two adapter details worth narrating. lines() yields io::Result<String>, so the line? inside map converts a mid-file read failure into your error type instead of silently truncating the stream — a silent truncation is the bug most people ship first. And collect::<Result<Vec<_>, _>>() is the idiomatic "any error aborts the pipeline" move: the iterator already threads Result through, and collect collapses it. For a tool that should count bad lines instead of dying, swap the collect for a fold that tallies Errs — same stream, different policy. Note the memory profile: one 8 KB buffer plus the hits you chose to keep; a million-line file never materializes as one Vec<String>.

Now the error type. Inside the library crate, thiserror derives the Display and From plumbing for an enum whose variants are your failure taxonomy:

use thiserror::Error;

#[derive(Error, Debug)]
pub enum ScanError {
    #[error("cannot open {path}: {source}")]
    Open {
        path: std::path::PathBuf,
        #[source]
        source: io::Error,
    },
    #[error("read error in {path} after line {line}: {source}")]
    Read {
        path: std::path::PathBuf,
        line: usize,
        #[source]
        source: io::Error,
    },
    #[error("unparseable entry at {path}:{source_line}: {cause}")]
    Parse {
        path: std::path::PathBuf,
        source_line: usize,
        cause: LogParseError,
    },
}

The design rules this encodes, which is what an interviewer actually probes. First, variants separate the failure domains: Open/Read mean the filesystem failed (permissions, disk, deleted file); Parse means the input was bad — and only the caller knows whether that aborts the run or increments a counter. Collapsing both into Box<dyn Error> or one String loses exactly the information your main needs to choose an exit code. Second, #[source] preserves the underlying io::Error for {:?} chains while #[error(...)] gives users a one-line message that names the file and line — parse errors must tell the user where to look in their data, which is why source_line is a field, not a log message. Third, #[from] on the variants you want implicit conversion for makes every ? in the streaming code above just work. The binary's main then matches once: Parse prints the location and exits 1, Open/Read exits 1 with the OS message, Ok exits 0 — the error enum is the exit-code table. The full thiserror/anyhow rationale — and why the split flips between library and binary crates — is covered in Anyhow and Thiserror; this capstone applies it to a real taxonomy. The last piece before shipping is proving the whole pipeline from the outside, covered next.

Proving It Works: assert_cmd Integration Tests, Fixtures, and a Tagged Release

Unit tests cover parse_line and the iterator pipeline. What they cannot prove is the contract a user actually experiences: exit codes, stderr messages, --help, stdin handling. That is the job of integration tests that run the built binary as a subprocess — the assert_cmd + predicates combination that most polished Rust CLIs ship.

use assert_cmd::Command;
use predicates::prelude::*;

#[test]
fn scan_reports_matches_and_exits_zero() {
    Command::cargo_bin("logscan").unwrap()
        .args(["--level", "error", "scan", "tests/fixtures/sample.log"])
        .assert()
        .success()
        .stdout(predicate::str::contains("app.log:14").from_env());
}

#[test]
fn missing_file_exits_one_with_context() {
    Command::cargo_bin("logscan").unwrap()
        .args(["scan", "no/such/file.log"])
        .assert()
        .failure()
        .code(1)
        .stderr(predicate::str::contains("cannot open")
                 .and(predicate::str::contains("no/such/file.log")));
}

#[test]
fn bad_level_is_a_usage_error() {
    Command::cargo_bin("logscan").unwrap()
        .args(["--level", "fatal", "scan", "x.log"])
        .assert()
        .failure()
        .code(2)                       // clap's usage-exit code, not yours
        .stderr(predicate::str::contains("possible values: error, warn, info, debug"));
}

Command::cargo_bin builds the binary through cargo and spawns it; predicates turns assertions into composable matchers. Note what each test asserts and what it refuses to assert: the third test does not call into your scan code at all — it proves that clap rejects fatal before your main logic is reachable, pinning the boundary between the generated parser and your hand-written core. That is the layering paying for itself in the test suite: parser behavior is proven by construction (debug_assert, try_parse_from), behavior users see is proven from outside, and the core is proven by unit tests with direct calls. Three properties make this suite worth its minutes. It is black-box: exactly what a user types and sees, including clap's own messages — which is how you notice that --level fatal exits 2 with the possible-values list for free, something no unit test of your own code could verify. It is fixture-driven: tests/fixtures/sample.log is a checked-in 30-line log with known hits, one malformed line, and a CRLF line — a miniature of every input class, so a regression in lines() handling shows up as a named failing test, not a user report. And it forces the exit-code discipline from the previous chapter into a written contract: 0 success, 1 runtime error, 2 usage error.

Before tagging, the checklist — the production baseline from Production Crate Baseline now pays off directly: what began as four commands in a generic CI template is now four gates standing in front of a specific artifact that users will download, and each one maps to a failure you have already seen happen in this project's own history. Run them in this order, cheapest first, so the fastest feedback comes earliest:

cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features              # unit + integration + doc tests
cargo audit                            # advisories in Cargo.lock
cargo dist plan                        # what the tag will publish

Then the release itself, automated by cargo-dist (configured in that same baseline topic): a v0.1.0 tag triggers the workflow, which cross-builds logscan for your target list, archives each with checksums, and drafts the GitHub Release with install instructions. Two finishing touches that separate a repo from a tool: shell completions generated by clap (Cli::command().generate_completions_to into an included asset or a completions hidden subcommand) and a --json output mode on stats so the tool composes in pipelines.

How to present this capstone in an interview: walk the architecture as a funnel — clap parses strings into typed values once, the library core consumes only typed values and returns Result<_, ScanError>, and main alone decides exit codes. Then name the two tests that prove each boundary (debug_assert for the grammar, missing_file_exits_one_with_context for the error contract). That framing shows a reviewer's instinct — boundaries tested from outside, errors typed inside — which is the real deliverable. The capstone that exercises the same instincts under threads and channels is Concurrent Service Capstone.

Quiz

1. In clap derive, why model a file argument as `files: Vec<PathBuf>` instead of `args: Vec<String>` parsed later?

Question 1 options

2. While scanning a 2 GB file, `reader.lines()` yields `Err(io::Error)` for line 84,201. What does `.map(|(i, line)| (i + 1, line?))` followed by `collect::<Result<Vec<_>, _>>()` do?

Question 2 options

3. Your thiserror enum has `Open { path, #[source] source: io::Error }` and `Parse { path, source_line, cause }`. What does the variant split buy you over a single `Error(String)`?

Question 3 options

4. An assert_cmd integration test runs `logscan --level fatal scan x.log` and asserts `.failure().code(2)`. Why 2, and what does this test protect?

Question 4 options

Flashcards

Question

How does clap derive model a CLI, and what does Cli::parse() guarantee about the boundary?

Answer

One #[derive(Parser)] struct per argument group (fields = flags/options/args, attributes like #[arg(long, value_enum, default_value_t)] configure them), #[derive(Subcommand)] enums for subcommands, and ValueEnum for closed option sets. Cli::parse() validates everything and exits code 2 with usage on failure, so main's logic can assume typed, valid input; try_parse_from returns Result for tests.

Question

Why wrap File in BufReader and treat .lines() as an iterator?

Answer

BufReader adds an 8 KB buffer so each next_line() is a memcpy, not a syscall — a multi-GB file streams in fixed memory. lines() yields io::Result<String> items that compose with map/filter_map/collect, so per-line IO errors can be propagated (line?) or handled (count them) as policy — and the stream must never be allowed to silently truncate.

Question

Design rule for a CLI's thiserror enum: why separate Parse from Open/Read?

Answer

Variants encode failure domains so callers choose policy by matching: Parse (bad user data) can be counted per line or reported with path:line, while Open/Read (filesystem) abort with the OS message. #[source] keeps the underlying io::Error for chains; #[error("cannot open {path}: {source}")] gives a user-fixable message. An untyped String error erases exactly this decision information.

Question

What do assert_cmd + predicates test that unit tests cannot?

Answer

The black-box user contract: the built binary run as a subprocess with real argv, asserting exit codes (0 success, 1 runtime error, 2 clap usage error), stderr/stdout via predicates, and clap's generated messages (e.g. possible-values lists). Combined with checked-in fixtures (known hits, a malformed line, CRLF), they pin boundary behavior no internal test can see.

Question

The capstone's architecture funnel — state it in one breath.

Answer

clap parses strings into typed values once at the edge; the library core consumes only typed values and returns Result<_, ScanError>; main alone maps errors to exit codes (0/1/2). Boundaries are tested from outside (debug_assert for the grammar, assert_cmd for the contract) and typed inside (thiserror variants, unit tests for parse_line and the adapter pipeline).

Revision Notes

Key Takeaways

  • 1. Modeling argv as derive structs moves the CLI grammar into types: ValueEnum closes the option set, PathBuf admits non-UTF-8 paths, and Cli::parse() exits code 2 before your logic sees bad input
  • 2. BufReader + lines() streams a multi-gigabyte file in 8 KB of memory; the iterator's io::Result items let you choose fail-fast (collect::<Result>) or count-and-continue per line
  • 3. A thiserror enum whose variants separate parse failures from filesystem failures is the exit-code table: main matches once, users get path:line context, the OS error survives as a source
  • 4. Integration tests with assert_cmd + predicates cover the contract unit tests cannot: real subprocess, real argv, exit codes, and clap-generated messages against checked-in fixtures
  • 5. The capstone architecture is a funnel — typed parse at the edge, typed core, exit codes only in main — and the release is the fmt/clippy/test/audit gate plus a cargo-dist tag

Interview Tips

  • • "Walk me through your CLI's structure" — the funnel: clap derive at the edge, library core with typed inputs and a thiserror enum, main as the only place exit codes are chosen; it shows you test boundaries from outside and type them inside
  • • "How would you handle a malformed line mid-file?" — policy lives in the adapter chain: filter_map returning Some(Err(...)) with path:line, collect deciding fail-fast, or a fold counting bad lines; naming both policies shows the iterator is a design surface, not boilerplate
  • • "Why thiserror here but anyhow in main?" — the library needs matchable variants (parse vs IO = different caller behavior); the binary has one consumer and prints context, which anyhow does with less code — and the From chain makes every ? free
  • • "How do you know the tool works end to end?" — name assert_cmd asserting exit codes and stderr against fixtures, plus Cli::command().debug_assert() for the grammar; ending with the cargo-dist tagged release closes the production story

Cheat Sheet

CLI Tool Capstone (logscan)

Derive skeleton:

#[derive(Parser)]
#[command(name = "logscan", version, about = "...")]
struct Cli {
    #[arg(long, value_enum, default_value_t = Level::Info)] level: Level,
    #[command(subcommand)] command: Command,
}
#[derive(Subcommand)] enum Command {
    Scan { files: Vec<PathBuf>, #[arg(long)] contains: Option<String> },
    Stats { file: PathBuf },
}

Cli::command().debug_assert() in a test validates the whole grammar.

Streaming core: BufReader::new(File::open(p)?) → .lines().enumerate().map(|(i,l)| (i+1, l?)).filter_map(...).collect::<Result<Vec<_>,_>>()? — fixed memory, fail-fast or count-bad-lines by policy.

Error taxonomy (thiserror): Open { path, #[source] source: io::Error }, Read { path, line, source }, Parse { path, source_line, cause }. Variants = policy at the call site; Display names file + line so users can fix input.

Exit codes: 0 success · 1 runtime error (Open/Read/Parse) · 2 clap usage error.

Test layers: unit (parse_line, adapters, try_parse_from) · integration (assert_cmd: exit codes, stderr predicates, fixtures with known hits + one malformed line) · release gate: fmt check, clippy -D warnings, test, cargo audit, cargo dist plan, then tag.

Continue Learning