Revert "Add -I / --online-input for huge top-level arrays"

This reverts commit 77936a594d.

There are too many odd bugs in this mode, and it turns out to be a bad
idea anyways.  Instead, in the future a better option will be to pursue
alternative parsers, such as:

 - streaming parser that outputs only when a new leaf value is added or
   an array/object is opened/closed; options here include whether to
   include a path in each output;

 - parsers for binary JSON encodings (there's a variety of them).

Then one might run jq with a streaming parser and use `reduce` to
coalesce inputs from some depth down (instead of from one level down as
the reverted commit had intended).

Besides, a fully streaming parser is desirable in some cases, therefore
we should have such a thing as an option.

I've explored modifying the current parser to support a streaming
option, but it only makes the code very difficult to follow, which is
one reason that alternate parsers makes sense.  At any rate, this is all
for the future.  For now there's no streaming of individual texts, just
text sequences.
This commit is contained in:
Nicolas Williams committed 2014-06-04 18:15:58 -05:00
1 parent 5fb82d1018
commit ae625d0de7
4 files changed
+12 -32

No files matched your search

+1 -4
View File
@@ -154,7 +154,6 @@ int main(int argc, char* argv[]) {
ninput_files = 0;
int further_args_are_files = 0;
int jq_flags = 0;
jv_parser_flags parser_flags = 0;
jv program_arguments = jv_array();
for (int i=1; i<argc; i++) {
if (further_args_are_files) {
@@ -192,8 +191,6 @@ int main(int argc, char* argv[]) {
options |= FROM_FILE;
} else if (isoption(argv[i], 'e', "exit-status")) {
options |= EXIT_STATUS;
} else if (isoption(argv[i], 'I', "online-input")) {
parser_flags = JV_PARSE_EXPLODE_TOPLEVEL_ARRAY;
} else if (isoption(argv[i], 0, "arg")) {
if (i >= argc - 2) {
fprintf(stderr, "%s: --arg takes two parameters (e.g. -a varname value)\n", progname);
@@ -283,7 +280,7 @@ int main(int argc, char* argv[]) {
slurped = jv_array();
}
}
struct jv_parser* parser = jv_parser_new(parser_flags);
struct jv_parser* parser = jv_parser_new();
char buf[4096];
while (read_more(buf, sizeof(buf))) {
if (options & RAW_INPUT) {