Previously, ltrimstr/rtrimstr would just let the input pass through for
non-string inputs or arguments.
That was happening because, they were leaking the errors returned by
startswith/endswith treating them as if they were jv_false().
The leak was resolved by #2977 for 1.7.1
This patch rewrites ltrimstr and rtrimstr in jq, and makes them not
ignore startswith and endswith errors anymore.
ltrimstr/rtrimstr was ignoring and leaking the error returned by
f_startswith()/f_endswith().
This also means that they just let the input pass through for non-string
inputs or arguments.
Only fix the leak for now; in the next release, #2969 will make them
rethrow the error returned by startswith/endswith.
Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64946
Although #2839 fixed the overflow of exponent subtraction,
there still is possibility of overflow in the `D2U` macro.
This patch fixes the overflow in the `D2U` macro, and also
truncates the maximum digits to `DEC_MAX_DIGITS`.
The decNumber library subtracts the exponents of two numbers,
we make sure to limit the number of digits not to make it overflows.
Since the maximum adjusted exponent is `emax` and the minimum is
`emin - digits + 1`, we follow `emax - (emin - digits + 1) <= INT32_MAX`.
This reuses the existing `block_list_funcs` capability and adds an extra field on the `modulemeta` output, called `defs`, containing that list of functions.
This patch removes the weird behaviour of jv_invalid_with_msg(jv_null())
that returns jv_invalid() (i.e. empty), instead of a boxed jv_null().
The previous behaviour of null|error was obviously unintentional, and
allowing is jv_invalid_with_msg() to return values on which you can't
call jv_invalid_get_msg() is only error prone.
Previously constant folding of zero division (e.x. 1/0) produces a
compile error. This was incorrectly implemented by checking if the
division result is infinite, so produces wrong results compared to the
query where no constant folding is processed (e.x. 1e308/0.1). This
patch delays the operation when the divisor is zero. This makes the
results more consistent, but changes the exit code on zero division from
3 to 5. Also 0/0 now produces the zero division error, not NaN.
This patch also fixes the modulo operation. Previously constant folding
logic does not take care of the % operator, but now it folds if the both
dividend and divisor are safe numbers to cast to the integer type, and
the divisor is not zero. This patch also fixes some code that relies on
undefined cast behaviors in C. The modulo operation produces NaN if
either the dividend or divisor is NaN.
Error on non-number and nan codepoint, would asserd before
Replace negative codepoint and surrogate range with unicode replacement character, would assert before
Fixes#1160
- Add error/0 and mentions null input behavior (close#2231)
- Explain value iterator suffix syntax .foo[] (close#1047)
- Mention array slicing is also zero-based (close#2094)
- Add examples of input and inputs filters (close#2216, close#2470)
- Improve sort_by about multiple values (close#2103, close#2467, close#2474)
- Improve foreach section and simplify examples (close#1148, close#2169)
- Fix recurse/1 document on how it is identical using recurse/2 (close#2036, close#2412)
- Add non-string examples of index/1, rindex/1 (close#1422)
- Simplify the example of truncate_stream/1 (close#1736)
Close#1885, #2140, #2011, #2220, #2485, #2073
Rename the FORK_OPT opcode to TRY_BEGIN, add a TRY_END opcode, and wrap
errors when raising through a TRY_END so that they will not be caught by
the matching TRY_BEGIN.
Now a `try exp catch handler` expression generates code like:
TRY_BEGIN handler
<exp>
TRY_END
JUMP past_handler
handler: <handler>
past_handler:
...
On backtrack through TRY_BEGIN it just backtracks.
If anything past the whole thing raises when <exp> produced a value,
then the TRY_END will catch the error, wrap it in another, and
backtrack. The TRY_BEGIN will see a wrapped error and then it will
unwrap and re-raise the error.
If <exp> raises, then TRY_BEGIN will catch the error and jump to the
handler, but the TRY_BEGIN will not stack_save() in that case, so on
raise/backtrack the TRY_BEGIN will not execute again (nor will the
TRY_END).
To make debug/0 not call an uninitialised function pointer when using
--run-tests or when using a jq_state on which jq_set_debug_cb() has not
been called.
To avoid causing segmentation faults when input/1 is called in a
jq_state on which jq_set_input_cb() has not been called; e.g. the one
used by jq --run-tests.
That segfault could also be fixed in run_jq_tests() by calling:
jq_set_input_cb(jq, NULL, NULL);
But I think it makes sense to just make jq_init() initialise those
values to NULL.
Ref: https://github.com/jqlang/jq/pull/2717#discussion_r1264338841
* Fix memory leak for has(nan)
jv_array_get() used to be responsible of freeing the input array, but
since b5c4c3d67d, it is no longer called
if the key is nan. We need to free it manually to avoid leaking the
array.
* Parse nan in JSON as NaN instead of triggering a parse error
Fixes#2021
n.b.: { $__loc__: "foo" } is still not allowed.
$__loc__ cannot be overwritten, and it is always an object.
Using { $x: "foo" } where $x is not a string is an error, so there is
not much of a point in supporting that.
Previously, bindings were parsed as the combination of two tokens:
'$' IDENT
This meant that using a keyword as variable name (e.g. $then, $label)
did not work.
Attempts were made to allow $keyword to work by adding some '$' Keyword
rules in the parser, but this did not allow $keyword in all places:
jq --arg label foo -n '$label' # ok
jq -n '"foo" as $label | .' # error
Treating $foo as a single token is much simpler, in my opinion.
This patch also changes how LOC is lexed: "$__loc__" instead of as
"__loc__" that gets combined with '$' in the parser.
This patch also disallows having spaces after '$' when recalling a
variable `$ foo' since that was probably just an unintentional side
effect of the implementation, and it was not documented.
Fixes#2675
The primary purpose of this commit is to rectify most problems with
`gsub` (and also `sub` with the `g` option), in particular fix#1425 ('\b'),
fix#2354 (lookahead), and fix#2532 (regex == `"^(?!cd ).*$|^cd "`).
This commit also partly resolves#2148 and resolves#1206 in that
`gsub` no longer loops infinitely; however, because the new `gsub`
depends critically on `match/2`, the behavior when regex == `""` is
sometimes non-standard.
The documentation has been updated to reflect the fact that `sub`
and `gsub` are intended to be regular in the second argument.
Also, `_nwise/1` has been tweaked to take advantage of TCO.
* Try making some GitHub Actions
* GHA: Add OS X, Windows, and Linux w/ scan-build builds
* Add link to discord server
Add link to discord server
* Update Ubuntu to 22.04 for GitHub Actions
* Remove if condition for Ubuntu build
* Fix MacOS OS on GitHub Actions
* Fix python3 package on GitHub Actions
* Disalbe valgrind
Getting failed tests: https://github.com/jqlang/jq/actions/runs/5113866588/jobs/9193542140#step:9:341
* Format file
* Fix macos build
* Fix syntax errors of windows build
https://github.com/jqlang/jq/actions/runs/5114162556/workflow
* Install windows package with choco
* Pin oniguruma to 5a24a49d710a9e3bb8ff11d12e1eae5a9f9df40c
MacOS build [fails](https://github.com/jqlang/jq/actions/runs/5114162555/jobs/9194126677#step:6:3160) due to
```
posix.c:94:3: error: implicit declaration of function 'onig_end' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
onig_end();
```
The current `oniguruma` revision
6fa38f4084b448592888ed9ee43c6e90a46b5f5c (15 Mar 2017) lacks the following explicit declaration in src/onigposix.h:
```
ONIG_EXTERN int onig_end P_((void));
```
This was added to oniguruma in revision 5a24a49d710a9e3bb8ff11d12e1eae5a9f9df40c (8 Sep 2017).
Ref: https://github.com/jqlang/jq/issues/2381
* Revert windows build to use pacman
* Don't zip jq.exe
* Fixing windows build
* Update .gitattriutes for eol on Windows
* Skip workflow_dispatch for windows build
* Clean up builds
* Use LF line endings for all text
Ref: https://github.com/actions/checkout/issues/135
* Fix test that has rounding error
https://github.com/jqlang/jq/pull/2596#issuecomment-1568498993
* Enable CI for all
---------
Co-authored-by: Nicolas Williams <nico@cryptonector.com>
Extend jv_number to use decNumber for storing number literals. Any math
operations on the numbers will truncate them to double precision.
Comparisons when both numbers are literal numbers will compare them
without truncation.
Delay conversion of numbers to doubles until a math operation is performed,
to preserve precision. A literal jv_number will only need conversion to
double once, and will reuse the resultant double on subsequent
conversions.
Outputting literal jv_numbers preserves the original precision.
Add strong pthread requirement to manage contexts/allocations for
converting numbers between their decNumber, string, and double formats.
Emanuele Torre