Originally built Liqe to enable Roarr log filtering via cli. I have since been polishing this project as a hobby/intellectual exercise. I've seen it being adopted by various CLI and web applications that require advanced search. To my knowledge, it is currently the most complete Lucene-like syntax parser and serializer in JavaScript, as well as a compatible in-memory search engine.
Liqe use cases include:
parsing search queries
serializing parsed queries
searching JSON documents using the Liqe query language (LQL)
Note that the Liqe AST is treated as a public API, i.e., one could implement their own search mechanism that uses Liqe query language (LQL).
Liqe uses Liqe Query Language (LQL), which is heavily inspired by Lucene but extends it in various ways that allow a more powerful search experience.
Liqe syntax cheat sheet
# search for "foo" term anywhere in the document (case insensitive)foo# search for "foo" term anywhere in the document (case sensitive)'foo'"foo"# search for "foo" term in `name` fieldname:foo# search for "foo" term in `full name` field'full name':foo"full name":foo# search for "foo" term in `first` field, member of `name`, i.e.# matches {name: {first: 'foo'}}name.first:foo# search using regexname:/foo/name:/foo/o# search using wildcardname:foo*barname:foo?bar# boolean searchmember:truemember:false# null searchmember:null# search for age =, >, >=, <, <=height:=100height:>100height:>=100height:<100height:<=100# search for height in range (inclusive, exclusive)height:[100 TO 200]height:{100 TO 200}# boolean operatorsname:foo AND height:=100name:foo OR name:bar# unary operatorsNOT foo-fooNOT foo:bar-foo:barname:foo AND NOT (bio:bar OR bio:baz)# implicit AND boolean operatorname:foo height:=100# groupingname:foo AND (bio:bar OR bio:baz)
Keyword matching
Search for word "foo" in any field (case insensitive).
foo
Search for word "foo" in the name field.
name:foo
Search for name field values matching /foo/i regex.
name:/foo/i
Search for name field values matching f*o wildcard pattern.
name:f*o
Search for name field values matching f?o wildcard pattern.
name:f?o
Search for phrase "foo bar" in the name field (case sensitive).
name:"foo bar"
Number matching
Search for value equal to 100 in the height field.
height:=100
Search for value greater than 100 in the height field.
height:>100
Search for value greater than or equal to 100 in the height field.
height:>=100
Range matching
Search for value greater or equal to 100 and lower or equal to 200 in the height field.
height:[100 TO 200]
Search for value greater than 100 and lower than 200 in the height field.
height:{100 TO 200}
Ranges also work with non-numeric values whose lexical ordering matches their chronological ordering: ISO 8601 dates, UTC date-times, and 24-hour times. The boundaries must be quoted, and inclusive ([ ]) and exclusive ({ }) brackets behave the same as for numbers.
Search for dates from the start to the end of 2020, inclusive of both boundaries.
created:["2020-01-01" TO "2020-12-31"]
Search for dates strictly between the two boundaries, exclusive.
created:{"2020-01-01" TO "2020-12-31"}
UTC date-times and 24-hour times use the same syntax (and the same bracket rules).
created:["2020-01-01T00:00:00.000Z" TO "2021-01-01T00:00:00.000Z"]startTime:["09:00" TO "17:00"]
Only these canonical, big-endian formats are accepted, because that is what makes lexical ordering equal chronological ordering. A range whose boundaries are anything else — free text, DD-MM-YYYY, timezone offsets, lowercase z, or mismatched precision — throws a TypeError.
Wildcard matching
Search for any word that starts with "foo" in the name field.
name:foo*
Search for any word that starts with "foo" and ends with "bar" in the name field.
name:foo*bar
Search for any word that starts with "foo" in the name field, followed by a single arbitrary character.
name:foo?
Search for any word that starts with "foo", followed by a single arbitrary character and immediately ends with "bar" in the name field.
name:foo?bar
Boolean operators
Search for phrase "foo bar" in the name field AND the phrase "quick fox" in the bio field.
name:"foo bar" AND bio:"quick fox"
Search for either the phrase "foo bar" in the name field AND the phrase "quick fox" in the bio field, or the word "fox" in the name field.
(name:"foo bar" AND bio:"quick fox") OR name:fox
Serializer
Serializer allows to convert Liqe tokens back to the original search query.
import { type BooleanOperatorToken, type ComparisonOperatorToken, type EmptyExpression, type FieldToken, type ImplicitBooleanOperatorToken, type ImplicitFieldToken, type LiteralExpressionToken, type LogicalExpressionToken, type RangeExpressionToken, type RegexExpressionToken, type TagToken, type UnaryOperatorToken,} from 'liqe';
There are 11 AST tokens that describe a parsed Liqe query.
If you are building a serializer, then you must implement all of them for the complete coverage of all possible query inputs. Refer to the built-in serializer for an example.
Utilities
import { isSafeUnquotedExpression,} from 'liqe';/** * Determines if an expression requires quotes. * Use this if you need to programmatically manipulate the AST * before using a serializer to convert the query back to text. */isSafeUnquotedExpression(expression: string): boolean;
Consider using highlight-words package to highlight Liqe matches.
Development
Compiling Parser
If you are going to modify parser, then use npm run watch to run compiler in watch mode.
Benchmarking Changes
Before making any changes, capture the current benchmark on your machine using npm run benchmark. Run benchmark again after making any changes. Before committing changes, ensure that performance is not negatively impacted.