* Create `src/config.h` instead of passing everything on the compiler
command line.
* To reduce the magnitude of this change, add `--include src/config.h`
to CFLAGS instead of adding `#include "src/config.h"` at the top of
each source file. Not all compilers support this, but I don't think
we care about anything other than gcc and clang at this point.
* Rather than generate `src/config_opts.inc`, emit `JQ_CONFIG` directly
into `src/config.h`.
* Rather than generate `src/version.h`, define `JQ_VERSION` as an alias
for the standard `PACKAGE_VERSION`.
In C23 (default C standard used by GCC 15), jv (*fptr)(); has become
equivalent to jv (*fptr)(void); so we can no longer assign builtin
implemenations directly to the fptr member of cfunctions without
generating a compile error.
Since there does not seem to be any straight-forward way to tell
autoconf to force the compiler to use C99 short of explicitly adding
-std=c99 to CFLAGS, it is probably a cleaner solution to just make the
code C23 compatible.
A possible solution could have been to just redeclare cfunction.fptr
as void*, but then the functions' return type would not have been type
checked (e.g. if you tried to add a {printf, "printf", 2}, where printf
is a function that does not return jv, the compiler wouldn't have
complained.)
We were already not typechecking the arguments of the functions, so e.g.
{binop_plus, "_plus", 3}, /* instead of {f_plus, "_plus, 3}, */
{f_setpath, "setpath", 4}, /* instead of {f_setpath, "setpath", 3}, */
compile without errors despite not having the correct prototype.
So I thought of instead improving the situation by redefining
cfunction.fptr as a union of function pointers with the prototypes that
the jq bytecode interpreter can call, and use a macro to add the builtin
functions to function_list using to the arity argument to assign the
implementation function to the appropriate union member.
Now the code won't compile if the wrong arity, or an arity not supported
by the bytecode interpreter (>5 = 1input+4arguments), or a prototype not
jallable by the bytecode interpreter (e.g. binop_plus that doesn't
expect a jq_state* argument).
Also, the code now compiles with gcc -std=c23.
Fixes#3206
When the UPDATE query in a reduce or foreach syntax does not emit
a value, it should simply skip updating the state variable. But
currently it resets the state variable to null.
This comment predates the addition of $ARGS in jq 1.6, so there was no
way to access named arguments with an invalid identifier. Rather than
forbid such names, update the manual to clarify.
When jq is not supplied with a program, it usually exits with an error
and usage text, except for when jq is not connected to a TTY (i.e., not
interactive), in which case a default program of `.` is used. Combined
with `-f`, it then tries to read `.` as the filename.
$ jq
jq - commandline JSON processor [version 1.7.1]
…
$ jq -f
jq - commandline JSON processor [version 1.7.1]
…
$ jq | cat
^C
$ jq -f | cat
jq: Could not open .: It's a directory
Fixes:
* Short options cannot be repeated in the same group:
e.g., `-nn` fails, but `--null-input --null-input` works
* Short options are processed in an arbitrary fixed order, not the order
given by the user: e.g., `-hV`, `-Vh`, `-h -V`, and `--help --version`
are processed as `-h`, but `-V -h` and `--version --help` are
processed as `-V`
* `-L` cannot be a part of a short option group (as the last one):
e.g., `-nLlib` and `-nL lib` fail
I also changed the behavior of limit/2 with negative count to emit an error, like nth/2 does.
Also, I redefined nth/2 using skip/2 to minimize the impact of growth of builtin filters.
This check was redundant and useless since jv2tm() used to require 8
elements, and errored anyway if not enough elements were provided.
After commit bc96146, this check also prevents mktime/0 from accepting
now supported inputs with less than 8 elements.
Ref: https://github.com/jqlang/jq/pull/3070#issuecomment-2299902072
Reported-by: itchyny <itchyny@cybozu.co.jp>
`jq_reset` calls `jv_free` on the `exit_code` and the `error_message` stored on the jq state.
However, it doesn't replace the actual instance of those members. This means that subsequent
calls to `jq_reset` will call `jv_free` again on those members, which in turn may call `free`
on the same pointer multiple times. Freeing the same pointer multiple times is undefined
behavior and can cause heap corruption, which is how I spotted this issue.
In practice, this issue only occurs when using a program that may `halt_error`, because that
is when the `exit_code` and `error_message` are set to values other than `jv_invalid`.
Subsequent attempts to call `jq_start` (which calls `jq_reset` internally) after hitting a
`halt_error` can cause you to run into this issue.
The changes simply reset the `exit_code` and the `error_message` to `jv_invalid` (the initial
value set in `jq_init`) after they are freed.
`-b`/`--binary` should be accepted on all platforms so that they can be used in
code that is meant to be portable.
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
arrays[arrays] is a special case of "INDEX" that actually returns an
array containing the indices in which the array that is being indexed
contains the start of the key array.
So array keys, for array values, are a kind of key that can be "got",
but not "set". jv_setpath() was not freeing the value it "got" from
indexing that key, in case the following "set" on that key failed,
resulting in a leak.
$ ./jq -n '[] | setpath([[1]]; 1)'
jq: error (at <unknown>): Cannot update field at array index of array
=================================================================
==953483==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 272 byte(s) in 1 object(s) allocated from:
#0 0x725f4d4e1359 in __interceptor_malloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:69
#1 0x5ec17b1a7438 in jv_mem_alloc src/jv_alloc.c:141
SUMMARY: AddressSanitizer: 272 byte(s) leaked in 1 allocation(s).
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66061
Now, time functions accept array inputs even if they don't have all the
elements, 0 will be assumed if a value is not present.
Also, jv2tm now properly clamps large number values to a signed 32-bit
integer and rejects nan.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65885
tests/mantest were failing on Raspberry Pi OS 11:
==10205== Conditional jump or move depends on uninitialised value(s)
==10205== at 0x48806F8: checkfail (jq_test.c:58)
==10205== by 0x48806F8: run_jq_tests (jq_test.c:99)
==10205== by 0x4880FCB: jq_testsuite (jq_test.c:39)
==10205== by 0x10B37F: main (main.c:576)
==10205==
==10205== Conditional jump or move depends on uninitialised value(s)
==10205== at 0x4880714: checkfail (jq_test.c:58)
==10205== by 0x4880714: run_jq_tests (jq_test.c:99)
==10205== by 0x4880FCB: jq_testsuite (jq_test.c:39)
==10205== by 0x10B37F: main (main.c:576)
In windows, time_t is a signed 32-bit integer type, so TIME_MAX needs to
be declared as INT32_MAX instead of INT64_MAX.
Also bump NetBSD's strptime to revision 1.65 from 1.63 to fix undefined
behaviour (signed integer overflow) bugs.
Related NetBSD problem report:
https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=58041
Noticed thanks to a compiler warning in the windows build CI.
I declared the d variable as time_t instead of as unsigned to ensure
that the signedness/size of TIME_MAX-d in the sse > TIME_MAX-d check
is always correct, and to prevent -Wsign-compare warnings from
triggering in the windows build.
There was a incorrect else, that caused jq to not ensure that the
argument to strftime/1 is a string when the input is a number; this ends
up calling jv_string_value on a non-string value, which does not work,
and causes an assert failure.
Also fix same bug in strflocaltime/1.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67403
Previously, the tonumber/0 filter parses the input as JSON values, but
this is less-performant on large non-number strings. Parsing the input
string as number literal fixes the performance issue. Also, this fix
changes the filter to reject numbers with white spaces.
Before, arrays would not reset colors after `[` and `,`, but objects
would; it would reset colors twice before `]` and `}`; and some cases of
indentation would have colors applied. Now, colors are reset immediately
after any token that is colored, before any indentation. This makes the
formatting consistent, for the benefit of custom `JQ_COLORS`.
C99 only allows string literals long at most 4095 characters.
jq_builtins was a lot longer than that.
I rewrote all the optional libm error stubs in C so the value of
jq_builtins is not build dependent.
I replaced the command that generates builtin.inc with a POSIX compliant
od|sed command that encodes builtin.jq as a comma delimited list of
octal numbers (that can be embedded in C using a {} literal).
I also added -Woverlength-strings to AM_CFLAGS to verify that the
problem is fixed.
Fixes#1481
itchyny