When trying to use the new regex functions (match/test/sub/capture/etc)
in a JQ that was compiled without the ONIGURAMA regex library,
report an informative error message instead of a 'not defined' error.
Before:
$ echo '"foo"' | ./jq-old 'test("foo")'
jq: error: test/1 is not defined at <top-level>, line 1:
test("foo")
jq: 1 compile error
After:
$ echo '"foo"' | ./jq 'test("foo")'
jq: error: jq was compiled without ONIGURAMA regex libary. match/test/sub and related functions are not available.
When escaping string (e.g. for `@tsv` or `@csv` outputs),
escape NULs as '\0'.
Existing behaviour, unchanged by this patch:
$ echo '"a\u0000b"' | ./jq '.'
"a\u0000b"
$ echo '"a\u0000b"' | ./jq -r '.' | od -a
0000000 a nul b nl
0000004
When using `@tsv`, escape NUL to `\0`:
$ echo '"a\u0000b"' | ./jq -r '[.]|@tsv'
a\0b
$ echo '"a\u0000b"' | ./jq '[.]|@tsv'
"a\\0b"
When using '@tsv' output format with '-r' option,
escape \r, \n and \\ in addition to \t.
Example:
$ printf '{"id":"hello\\ttab\\nworld","x":43 }' | jq .
{
"id": "hello\ttab\nworld",
"x": 43
}
Before: newlines are not escaped, generating invalid TSV output:
$ printf '{"id":"hello\\ttab\\nworld","x":43 }' \
| ./jq-old -r '[.id,.x] | @tsv'
hello\ttab
world 43
After: newlines are properly escaped:
$ printf '{"id":"hello\\ttab\\nworld","x":43 }' \
| ./jq-new -r '[.id, .x] | @tsv'
hello\ttab\nworld 43
Before: backslashes themselves are not escaped, so there's no way to
distinguish between escaped characters and 'real' backslashes
(in the example below, there is should not be newline, despite the
output containing "\n".
$ printf '{"x":"hello\\ttab\\\\new world"}' \
| ./jq-old -r '[.x]|@tsv'
hello\ttab\new world
After: backslashes are escaped:
$ printf '{"x":"hello\\ttab\\\\new world"}' \
| ./jq-new -r '[.x]|@tsv'
hello\ttab\\new world
Tests won't pass if built without Oniguruma. We don't have a way to
make a test optional yet. That will come later. For now the ability to
reduce build-time dependencies could really help some users.
To import a module now use:
# Import module.jq file:
import "relative/path/to/module" as foo;
# Use the module's defs as foo::<def-name>
To import a JSON file:
# Read file.json:
import "relative/path/to/file" as $foo;
#
# Use as $foo::foo
Using `-L` now drops the builtin library path and appends the requested
path to the empty array (or the result of an earlier `-L`).
Support for the `$JQ_LIBRARY_PATH` environment variable has been
removed.
This is to fix the problem where `break` is dynamic, not lexical.
With this it should be possible to do this sort of thing:
label $break | inputs | if ... then $break|error else . end
This is a backwards-incompatible change for master, but the previous
`break` hadn't shipped yet.
Still needed:
- testing
Streaming means that outputs are produced as soon as possible. With the
`foreach` syntax one can write programs which reduce portions of the
streaming parse of a large input (reduce into proper JSON values, for
example), and discard the rest, processing incrementally.
This:
$ jq -c --stream .
should produce the same output as this:
$ jq -c '. as $dot | path(..) as $p | $dot | getpath($p) | [$p,.]'
The output of `jq --stream .` should be a sequence of`[[<path>],<leaf>]`
and `[[<path>]]` values. The latter indicate that the array/object at
that path ended.
Scalars and empty arrays and objects are leaf values for this purpose.
For example, a truncated input produces a path as soon as possible, then
later the error:
$ printf '[0,\n'|./jq -c --stream .
[[0],0]
parse error: Unfinished JSON term at EOF at line 3, column 0
$
Nicolas Williams