Extract fields

Pull typed values, key and value pairs, and the columns of a table out of text.

One typed value

Name the atom for the kind: \E an email address, \N a number, \I an IP address, \U a URL, \T a timestamp, \V a version; the token atoms lists them all.

$ trex scan '\E' --text 'ping bob@x.com please'
[5..14] "bob@x.com"

let pat = trex::parse(r"\E").expect("valid pattern");
let text = "ping bob@x.com please";
let found: Vec<&str> = trex::scan(&pat, text.as_bytes()).iter().map(|s| &text[s.range()]).collect();
assert_eq!(found, ["bob@x.com"]);

Key and value pairs

Bind each side to a register. A literal = is written "=", since a bare = is the back-reference operator.

$ trex scan '\W:k "=" \Q:v' --text 'name = "bob" city = "nyc"'
[0..12] "name = \"bob\""  captures: k="name", v="\"bob\""
[13..25] "city = \"nyc\""  captures: k="city", v="\"nyc\""

let pat = trex::parse(r#"\W:k "=" \Q:v"#).expect("valid pattern");
let text = br#"name = "bob" city = "nyc""#;
let pairs: Vec<(&[u8], &[u8])> = trex::captures(&pat, text, &trex::scan(&pat, text))
    .iter()
    .map(|m| (m.group("k", text).expect("k is bound"), m.group("v", text).expect("v is bound")))
    .collect();
assert_eq!(pairs, [(&b"name"[..], &br#""bob""#[..]), (&b"city"[..], &br#""nyc""#[..])]);

--json writes each match’s registers under captures.

The columns of a table

Write one row as a pattern, naming the columns to keep. ^ starts it at a line’s first token, and the header row does not match, since age is no number:

$ cat people.csv
name,age,score
alice,30,95
bob,25,88

$ trex scan '^ \W:name "," \N "," \N:score' people.csv --format '${name} ${score}'
alice 95
bob 88

let pat = trex::parse(r#"^ \W:name "," \N "," \N:score"#).expect("valid pattern");
let csv = b"name,age,score\nalice,30,95\nbob,25,88\n";
let rows: Vec<(&[u8], &[u8])> = trex::captures(&pat, csv, &trex::scan(&pat, csv))
    .iter()
    .map(|m| (m.group("name", csv).expect("bound"), m.group("score", csv).expect("bound")))
    .collect();
assert_eq!(rows, [(&b"alice"[..], &b"95"[..]), (&b"bob"[..], &b"88"[..])]);

For lines whose layouts differ, build the pattern from marked examples instead of writing it.

Search a tree of files Index a tree