The primary purpose of this commit is to rectify most problems with
`gsub` (and also `sub` with the `g` option), in particular fix#1425 ('\b'),
fix#2354 (lookahead), and fix#2532 (regex == `"^(?!cd ).*$|^cd "`).
This commit also partly resolves#2148 and resolves#1206 in that
`gsub` no longer loops infinitely; however, because the new `gsub`
depends critically on `match/2`, the behavior when regex == `""` is
sometimes non-standard.
The documentation has been updated to reflect the fact that `sub`
and `gsub` are intended to be regular in the second argument.
Also, `_nwise/1` has been tweaked to take advantage of TCO.
We need to be careful to not grab an extra reference when mutating data
structures because that means we make extra copies. Doing that every
time in `_modify` is really painful, as that function implements `|=`
and all modify-assign operators.
`jv_setpath()` also grabs additional references, so this is not the only
fix needed for the modify-assign operators to not be accidentally
quadratic.
We have to use `LOADVN` in order to make this fix possible, so we should
really byte-code `_modify` rather than jq-code it. However, as a
stop-gap to make this fix easier, I'm adding syntax for referring to a
`$binding` such that you get `LOADVN` instead of `LOADV`.
This syntax is not meant to be used outside jq's internals! Therefore
it is not documented. I promise to break it later, so don't use it!
TBD, but later:
- Optimize `_modify` for the case where `update` outputs more than one
value.
This is trivial: add a `setpath($p; null)` in the middle of `_modify`
before calling `update`. But unfortunately the VM retains a
reference to `value_at_path` for path expression checking, and fixing
that will require more work.
The parser generates a call to error/0 on break. Having that exported
from C directly removes a language dependency on builtin.jq.
In any case, error/0 seems to be the more fundamental operation, seeing
as the old C impementation of error/1 did nothing useful with its input.
The distinction between input/0 and _input/0 has been unnecessary since
a9c613..7dc34b, which moved end-of-input handling into C and replaced
the jq definition of input with `def input: _input;`
As noted by @nicowilliams, `tostream` used `keys`,
which sorts the keys in alphabetical order, instead
of `keys_unsorted`, which preserves the parsing order.
Fixes#1541.
Now that #1313 is fixed, |= no longer outputs null when the RHS update
expression outputs empty.
When a user wants to keep the current value of the LHS they would have
the RHS update expression output `.`, so having `empty` achieve the same
thing would be redundant. The obvious thing to do is to delete the LHS
when the RHS update outputs `empty` (i.e., doesn't output any values).
It's reasonable to think that existing programs won't be broken by this
change, because reduce and |= not handling empty well is clearly a bug.
(Though it's possible that some programs were using empty to quickly
terminate reduce or |=, it's not likely. They should use label/break
instead.)
Prior to this change |= would use the _last_ value output by the RHS
update expression. With this change |= will use the _first_ value
instead. This change _is_ a minor backwards-incompatible change. It
may or may not be acceptable; we'll see. It is a useful change in that
it makes |= faster when the update expression produces multiple values.
first(g) was extracting two values, which, if g is slow, made first(g)
slow. And if the second extraction were to throw an error, then
first(g) would throw that error, which is clearly not the right thing to
do. Besides that, first(g) was allocating garbage, which it no longer
does.
./jq --run-tests < tests/onig.test
has been run successfully:
...
Testing 'gsub("a";"b")' at line number 70
Testing 'gsub( "(.*)"; ""; "x")' at line number 74
...
22 of 22 tests passed (0 malformed)
itchyny