When reporting an error to the user, add information about the offending
object/value (possibly truncated).
The goal is to give a user some context regarding which input object
caused the runtime error.
Examples:
$ echo '"hello"' | ./jq '-.'
jq: error: string ("hello") cannot be negated
$ echo '"very-long-string"' | ./jq '-.'
jq: error: string ("very-long-...) cannot be negated
$ echo '["1",2]' | ./jq '.|join(",")'
jq: error: string (",") and number (2) cannot be added
$ echo '["1","2",{"a":{"b":{"c":33}}}]' | ./jq '.|join(",")'
jq: error: string (",") and object ({"a":{"b":{...) cannot be added
$ echo '{"a":{"b":{"c":33}}}' | ./jq '.a | @tsv'
jq: error: object ({"b":{"c":33}}) cannot be tsv-formatted, only array
(Fix#754)
With this patch, jq run-time errors printed to stderr will contain the
filename and line of the offending input.
But note that the JSON text parser does (yet) print the input filename
on parse error. A jq run-time error is a jq program error, not
including syntax errors nor JSON text input format errors.
Examples:
With stdin and multiple lines:
$ printf '{"a":43}\n{"a":{"b":66}}\n' | ./jq '.a+1'
44
jq: error (at stdin:2): object and number cannot be added
With multiple files:
$ printf '{"a":43}' > 1.json
$ printf '{"a":"hello"}\n' > 2.json
$ printf '{"a":{"b":66}}\n' > 3.json
$ ./jq '[.a]|@tsv' 1.json 2.json 3.json
"43"
"hello"
jq: error (at 3.json:1): object is not valid in a csv row
With very long lines (spanning multiple `fgets` calls):
$ ( printf '{"a":43}\n' ;
printf '{"a":{"b":[' ; seq 10000 | paste -d, -s | tr -d '\n' ;
printf ']}}\n' ;
printf '{"a":"hello"}\n' ) | ./jq '[.a] | @tsv'
"43"
jq: error (at stdin:2): object is not valid in a csv row
"hello"
With raw input:
$ seq 1000 | ./jq --raw-input 'select(.=="700") | . + 10'
jq: error (at stdin:700): string and number cannot be added
Caveat:
The reported line will be the last line of the (valid) parsed JSON data.
Example:
$ printf '{\n"a":\n"hello"\n\n\n}\n' | ./jq '.a+4'
jq: error (at stdin:6): string and number cannot be added
When trying to use the new regex functions (match/test/sub/capture/etc)
in a JQ that was compiled without the ONIGURAMA regex library,
report an informative error message instead of a 'not defined' error.
Before:
$ echo '"foo"' | ./jq-old 'test("foo")'
jq: error: test/1 is not defined at <top-level>, line 1:
test("foo")
jq: 1 compile error
After:
$ echo '"foo"' | ./jq 'test("foo")'
jq: error: jq was compiled without ONIGURAMA regex libary. match/test/sub and related functions are not available.
When escaping string (e.g. for `@tsv` or `@csv` outputs),
escape NULs as '\0'.
Existing behaviour, unchanged by this patch:
$ echo '"a\u0000b"' | ./jq '.'
"a\u0000b"
$ echo '"a\u0000b"' | ./jq -r '.' | od -a
0000000 a nul b nl
0000004
When using `@tsv`, escape NUL to `\0`:
$ echo '"a\u0000b"' | ./jq -r '[.]|@tsv'
a\0b
$ echo '"a\u0000b"' | ./jq '[.]|@tsv'
"a\\0b"
When using '@tsv' output format with '-r' option,
escape \r, \n and \\ in addition to \t.
Example:
$ printf '{"id":"hello\\ttab\\nworld","x":43 }' | jq .
{
"id": "hello\ttab\nworld",
"x": 43
}
Before: newlines are not escaped, generating invalid TSV output:
$ printf '{"id":"hello\\ttab\\nworld","x":43 }' \
| ./jq-old -r '[.id,.x] | @tsv'
hello\ttab
world 43
After: newlines are properly escaped:
$ printf '{"id":"hello\\ttab\\nworld","x":43 }' \
| ./jq-new -r '[.id, .x] | @tsv'
hello\ttab\nworld 43
Before: backslashes themselves are not escaped, so there's no way to
distinguish between escaped characters and 'real' backslashes
(in the example below, there is should not be newline, despite the
output containing "\n".
$ printf '{"x":"hello\\ttab\\\\new world"}' \
| ./jq-old -r '[.x]|@tsv'
hello\ttab\new world
After: backslashes are escaped:
$ printf '{"x":"hello\\ttab\\\\new world"}' \
| ./jq-new -r '[.x]|@tsv'
hello\ttab\\new world
Detect output errors when the program exits.
Currently:
$ echo '{}' | jq . > /dev/full && echo ok
ok
with the patch:
$ echo '{}' | jq . > /dev/full && echo ok
Error: writing output failed: No space left on device
also apply to hardware/network/other I/O errors.
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
Improve robustness in automated system when using exit code in shell scripts,
by exiting with code 2 if there was any input error (even overriding other
possible error exit codes).
Exit code 2 is already used to indicate system errors.
Without the patch:
$ jq . no-such-file ; echo $?
jq: no-such-file: No such file or directory
0
With the patch:
$ jq . no-such-file ; echo $?
jq: no-such-file: No such file or directory
2
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
With this change, runtime exceptions are propagated to non-zero exit
code of 'jq', allow better scripting and automation. The new exit code
value is 5.
This allows using the shell's and/or operations ('&&' and '||') to
detect input runtime exceptions.
Before:
runtime exceptions are printed to STDERR, but not reported as non-zero exit-code:
$ echo '"hello"' | jq '.|tonumber' ; echo $?
jq: error: Invalid numeric literal at EOF at line 1, column 5 (while parsing 'hello')
0
After:
$ echo '"hello"' | ./jq '.|tonumber' ; echo $?
jq: error: Invalid numeric literal at EOF at line 1, column 5 (while parsing 'hello')
5
Note that there's a subtle interplay when using "-e" option.
The value of the non-zero exit code changes from 4 to 5, but it still
indicates a 'failure' or non-true value.
Before:
$ echo '"hello"' | jq -e '.|tonumber' ; echo $?
jq: error: Invalid numeric literal at EOF at line 1, column 5 (while parsing 'hello')
4
After:
$ echo '"hello"' | ./jq -e '.|tonumber' ; echo $?
jq: error: Invalid numeric literal at EOF at line 1, column 5 (while parsing 'hello')
5
Assaf Gordon