Add Streaming parser (--stream)

Streaming means that outputs are produced as soon as possible.  With the
`foreach` syntax one can write programs which reduce portions of the
streaming parse of a large input (reduce into proper JSON values, for
example), and discard the rest, processing incrementally.

This:

    $ jq -c --stream .

should produce the same output as this:

    $ jq -c '. as $dot | path(..) as $p | $dot | getpath($p) | [$p,.]'

The output of `jq --stream .` should be a sequence of`[[<path>],<leaf>]`
and `[[<path>]]` values.  The latter indicate that the array/object at
that path ended.

Scalars and empty arrays and objects are leaf values for this purpose.

For example, a truncated input produces a path as soon as possible, then
later the error:

    $ printf '[0,\n'|./jq -c --stream .
    [[0],0]
    parse error: Unfinished JSON term at EOF at line 3, column 0
    $

tests/run

@@ -20,7 +20,6 @@ if [ -z "$d" ]; then
     exit 0
 fi
 
-
 ## Test constant folding
 
 # String constant folding (addition only)
@@ -101,14 +100,14 @@ cmp $d/out $d/expected
 # Note that here jq sees no inputs at all but it still succeeds because
 # --seq ignores parse errors
 cat > $d/expected <<EOF
-ignoring parse error: Unfinished string
+ignoring parse error: Unfinished string at EOF at line 1, column 4
 EOF
 printf '"foo'|./jq -ce --seq . > $d/out 2>&1
 cmp $d/out $d/expected
 
 # Numeric values truncated by EOF are ignored
 cat > $d/expected <<EOF
-ignoring parse error: Potentially truncated top-level numeric value
+ignoring parse error: Potentially truncated top-level numeric value at EOF at line 1, column 1
 EOF
 printf '1'|./jq -ce --seq . > $d/out 2>&1
 cmp $d/out $d/expected
@@ -118,6 +117,12 @@ EOF
 printf '1\n'|./jq -ces --seq '. == [1]' >/dev/null 2> $d/out
 cmp $d/out $d/expected
 
+## Test streaming parser
+
+$VALGRIND $Q ./jq -c '. as $d|path(..) as $p|$d|getpath($p)|scalars_or_empty|[$p,.]' < "$PWD/tests/torture/input0.json" > $d/out0
+$VALGRIND $Q ./jq --stream -c '.|select(length==2)' < "$PWD/tests/torture/input0.json" > $d/out1
+diff $d/out0 $d/out1
+
 ## Test library/module system
 
 mods=$PWD/tests/modules

tests/torture/input0.json

@@ -0,0 +1,7 @@
+0 12 true false null [] {} [{}] [[]]
+[0,01,[12,22,[34,[45,56],7]],[]]
+{"a":[1]}
+{"a":[{}]}
+{"a":[{},[],[[]]]}
+{"a":[{"b":[]}]}
+{"a":[{"b":[]},{},[2]]}