Compare commits

..
Author SHA1 Message Date
Nicolas Williams 7f251344b7 Fix make dist (fix #870) 2015-07-26 23:26:30 -05:00
David Tolnay 21309bc6d7 Error on bytecodes longer than 0xFFFF (fix #269)
This lets us use uint16_t for the program counter. JVM
languages have the same limit on method size so it is a
reasonable limit.
2015-07-26 16:19:20 -07:00
David Tolnay 8295baf284 Drop plus from "jq 1.5+" on download page
See https://github.com/stedolan/jq/issues/488#issuecomment-124998113
2015-07-26 09:37:34 -07:00
David Tolnay ec84f28630 News is not THIS important 2015-07-26 01:27:14 -07:00
David Tolnay 701a47c82e Add 1.4 and 1.3 manuals to website (fix #866) 2015-07-26 01:26:13 -07:00
David Tolnay 616f365e5a Suggest --disable-maintainer-mode on download page 2015-07-25 23:44:52 -07:00
David Tolnay ceb2684f2f Update setup.sh script
Adjust what gets cleaned up now that ChangeLog and
config/.gitignore are checked into the repo, and skip
install and distcheck steps.
2015-07-24 23:26:47 -07:00
David Tolnay b9c340a211 Change homepage .com -> .io
The .com redirects to .io, but might as well have the
correct one to begin with.
2015-07-24 22:55:58 -07:00
David Tolnay 3a964e2025 Change AC bug-report parameter to point to GitHub
This affects where `make check` tells you to report failures.
2015-07-24 22:48:03 -07:00
Santiago Lapresta 3d49fc59dd Implement flatten/0 in terms of flatten/1 2015-07-24 21:55:43 -07:00
David Tolnay 16e8d0b1ab detect invalid path expression (fix #862) 2015-07-24 20:23:07 -07:00
pkoppstein 5856a2d2c0 simplify implementation of bsearch to use "until" and avoid "last"; tested 2015-07-24 12:29:20 -07:00
David Tolnay 227f62e013 Rebuild jq.1.prebuilt 2015-07-24 09:01:21 -07:00
Evan Zacks 89244cc3ea fix references to --color-output 2015-07-24 10:18:12 -04:00
Thomas R. Hall 3a43e2ff07 Updated v1.3 filenames
The jq 1.3 filenames for Windows are jq-1.3.exe instead of jq.exe. Updated these so they no longer return a 404.
2015-07-23 22:24:39 -07:00
David Tolnay 81f328f1aa Copy over changes made to gh-pages directly
This includes the following commits from the gh-pages branch:

 - 63e80b8
 - d24a397
 - 9796cd6
 - ee21707
 - ccb0f1d
 - 200e6b3
 - 64134c0
 - 57152d2
 - 8ffbfa3
2015-07-23 21:51:30 -07:00
David Tolnay f5a7bba986 Resolve remaining shift/reduce conflicts involving '?'
By lowering the precedence of rules that should never use
the generic Exp '?' rule.
2015-07-19 09:43:10 -07:00
David Tolnay 66ef8e2c24 Resolve shift/reduce conflict of 'def' vs '|'
This was an important conflict. In the following expression:

    def a: 0; . | a

Bison needs to decide between these two equally valid
parses:

    (def a: 0; .) | a
    def a: 0; (. | a)

For jq we want the second one, because the first results in
"a/0 is not defined". In the current parser the first parse
is a reduce and the second parse is a shift. Since Bison
prefers to shift in shift/reduce conflicts, we accidentally
got the correct behavior.

This commit adds a precedence level FUNCDEF which is lower
precedence than '|', meaning we explicitly choose the
correct parse.

Of course many unit tests already cover this case, but I
added one specifically for it.
2015-07-19 09:42:01 -07:00
David Tolnay 1628bbf95f Clean up trailing whitespace 2015-07-19 09:38:50 -07:00
Nicolas Williams 812703139c Use =, not ==, for shell tests 2015-07-18 21:12:08 -05:00
David Tolnay 54add5ac9a travis has whitelisted libonig-dev
https://github.com/travis-ci/travis-ci/issues/4123
2015-07-10 13:21:00 -07:00
Nicolas Williams f183b57ed5 input_filename should never empty 2015-07-10 12:32:11 -05:00
Nicolas Williams dc4dfdff17 Update AUTHORS 2015-07-10 12:28:21 -05:00
Nicolas Williams f55034d314 Update NEWS 2015-07-10 11:26:29 -05:00
Nicolas Williams 930899e12d Update ChangeLog 2015-07-10 11:12:24 -05:00
Nicolas Williams 579518c78d Use include for import into namespace 2015-07-10 10:19:33 -05:00
Nicolas Williams fa45a5121c Simplify import docs 2015-07-10 10:11:30 -05:00
Nicolas Williams 862566a48f Fix typo in docs 2015-07-10 10:10:28 -05:00
James Andariese f9f4d2d1e5 Dockerfile reorganized
* Remove excess layer creation
  This is an antipattern in Docker.
* Remove build tools once build is complete
  Leaving them around is not necessary and expands the image size 5x
* Make check works with libonig installed
* Compiles statically (needs libonig to be built rather than installed)
* Valgrind doesn't work with the use of TLS in jq so it's disabled -- this
  might be a FIXME situation; I'm not familiar enough with valgrind to
  say whether this is expected
* Make entrypoint be the jq binary so that the image can also be used
  to run jq in environments where you don't want to or can't install
  jq (such as CoreOS).
2015-07-10 07:15:44 -07:00
Mike Fletcher b74379532d Fixing missing D in LIBM_DDD_NO macro name 2015-07-09 14:59:37 -04:00
David Tolnay 1e5e0c9fe2 Make jq.h usable from C++
Previously, with clang++:
jq.h:46:37: error: typedef redefinition with different
      types ('struct jq_util_input_state *' vs 'jq_util_input_state')

With g++:
jq.h:46:37: error: conflicting declaration
      ‘typedef struct jq_util_input_state* jq_util_input_state’

This typedef was added to libjq by commit 0d41447 which was
after the 1.4 release, so although it is a public API, this
is not a backcompat break because it has never been in a
release.

Specifying the "*" at all uses of jq_util_input_state is
slightly tedious, but jq_state already works that way, so at
least it will be consistent.
2015-07-04 15:08:01 -07:00
Nicolas Williams 05899f8b3e Fix Windows build 2015-07-04 13:45:24 -05:00
Nicolas Williams a2a6f2d896 Document math support 2015-07-03 21:19:01 -05:00
Nicolas Williams 1c350c67d7 Remove pow10 for now; CHECK_MATH_FUNC needs work
The CHECK_MATH_FUNC() m4 macro needs work.  It should use
AC_RUN_IFELSE(), not just AC_LINK_IFELSE(), and it should #define
_GNU_SOURCE and/or other such feature macros in the prologue to get
non-standard math functions (alternatvely jq should only support
standard math functions).
2015-07-03 21:13:13 -05:00
David Tolnay 2348b7245e docs and diagram for exec_stack 2015-07-02 23:58:07 -07:00
David Tolnay 6e27de4f74 strftime wrong day-of-week (fix #838) 2015-07-01 20:37:16 -07:00
Nicolas Williams e5dafaea44 Document --run-tests 2015-06-28 13:57:02 -05:00
Nicolas Williams fd4f4f2087 Make --run-tests' jv_test() quiet 2015-06-28 13:39:32 -05:00
Nicolas Williams 6c89186437 Make --run-tests less verbose by default
Adding --debug-trace or --debug-dump-disasm before --run-tests now makes
it verbose.
2015-06-27 23:31:44 -05:00
Nicolas Williams 32c4759e6b Add more basic number tests 2015-06-27 23:15:16 -05:00
Nicolas Williams 6366cfd7a4 Add pow, better libm detection (fix #443) 2015-06-27 23:14:46 -05:00
David Tolnay 9985c01975 gcov exclusions 2015-06-27 14:48:13 -07:00
David Tolnay e6371712ff flag to enable gcov and coveralls 2015-06-27 11:25:01 -07:00
David Tolnay d7e35101c5 add configure option to run tests without valgrind 2015-06-27 11:24:53 -07:00
Nicolas Williams 1defaa7b00 Fix braino in merging the previous commit
There's a mantest that shows how to use `env`.  Well, we need to set the
env var as expected then, else it will fail.
2015-06-27 13:00:41 -05:00
David Tolnay 815b4c9772 get Travis CI working 2015-06-27 12:40:01 -05:00
Nicolas Williams aaf305868c Restore import into caller's namespace 2015-06-26 23:40:37 -05:00
Nicolas Williams 8cc31f0f49 Slight re-phrasing of module docs 2015-06-26 23:38:24 -05:00
Nicolas Williams a011ac6392 Use set -u in tests/setup 2015-06-26 23:36:34 -05:00
Nicolas Williams 505e23124f Fix #830 2015-06-26 21:24:01 -05:00
Nicolas Williams e7f6d1e6d0 Update Adavanced section of manual 2015-06-26 21:19:29 -05:00
Nicolas Williams a220482de6 Document {$var} 2015-06-26 21:18:58 -05:00
Nicolas Williams e16bd17571 Add {$var} . as {$var} syntax (fix #831) 2015-06-26 20:45:14 -05:00
Nicolas Williams 25d47ca08e Add streaming utilities (fix #827) 2015-06-26 20:45:06 -05:00
Nicolas Williams 5108a451ca Alternative implementation of tovalues 2015-06-26 20:17:02 -05:00
Santiago Lapresta ec7647c4ab Add combinations/0 and combinations/1 2015-06-23 10:59:58 -05:00
Nicolas Williams 8aecf82bfe WriteFile() on WIN32 when stdout isatty (fix #824)
Use WriteFile() and bypass stdio IFF stdout isatty and we're on Windows.
2015-06-22 20:47:47 -05:00
Nicolas Williams 10cbb9b580 Fix test failure introduced by previous commit 2015-06-22 18:32:28 -05:00
David Tolnay 4a316fbb5a fix errors flagged by clang static analyzer
builtin.c: bug - free of uninitialized jv
compile.c: missing assertion
jq_test.c: buggy logic / unreachable code
jv.c: missing assertion
jv_alloc.c: false positive - intentional read of uninitialized memory
jv_file.c: dead code
2015-06-20 16:08:56 -05:00
Nicolas Williams 7811ef1e17 Fix #811: use CommandLineToArgvW() and _wfopen() 2015-06-19 18:54:44 -05:00
David Tolnay b04cd6aada fix use after free in f_strptime 2015-06-19 00:34:27 -05:00
David Tolnay 1146b8b84a separate jq, oniguruma, sh, and man tests 2015-06-18 23:55:43 -05:00
Nicolas Williams 292f2d3208 argv[] may not be UTF-8 (fix #811) 2015-06-18 21:31:22 -05:00
Doug Luce 3cbefde376 Add alloca() discovery to configure.ac
The build failed on FreeBSD as there is no alloca.h.  This patch is
lifted from the autoconf documentation.
2015-06-18 19:19:15 -05:00
Nicolas Williams 0b42457929 Fix finites 2015-06-18 10:14:52 -05:00
David Tolnay 51a81c96f1 fix broken tests in manual.yml 2015-06-18 00:06:52 -05:00
Nicolas Williams 6083581fea Fix #802 2015-06-18 00:03:46 -05:00
Nicolas Williams 164b877bfa Add isnormal and related, rename *inf 2015-06-17 23:14:26 -05:00
Nicolas Williams b9c2a326ba Fix #814: raise on div-0, add inf isinf nan isnan 2015-06-17 19:58:55 -05:00
Nicolas Williams bdc1feb50e Fix infinite loop on error in sequence parser 2015-06-17 19:58:49 -05:00
Nicolas Williams 4ef04e5b5c Sequence parser: wait for RS on startup (fix #687)
Per-RFC7464.
2015-06-17 11:24:34 -05:00
Nicolas Williams 0f5345e515 Fix infinite loop on EOF bug 2015-06-17 11:23:58 -05:00
Aaron Peschel c95b4a8ab4 Add Missing Makefile Dependency
main.c requires version.h, but there is no logic marking this dependency
in the Makefile. This commit adds the dependency to the Makefile
template.
2015-06-09 19:01:20 -05:00
Nicolas Williams 520b429ca0 Clarify array destructuring docs 2015-06-09 17:29:48 -05:00
David Tolnay d3343d5113 array and object destructuring (fix #533)
`. as [$i, $j, $k] | ...`
`. as {a: $i, b: $j} | ...`
`. as [[[$i]], {a: $j}] | ...`
`foreach . as [$i, $j, $k] (...)`
`reduce . as {a: $i, b: $j} (...)`
2015-06-09 17:29:48 -05:00
cdnbacon 2e96fb2c9b Remove extension from links
The downloads are available without an extension.
2015-06-08 21:30:40 -05:00
Nicolas Williams 24a5e5b1b1 Add --tab and -indent n options 2015-06-03 20:20:11 -05:00
Nicolas Williams 74ecb88935 Improve usage message 2015-05-29 15:35:30 -05:00
Nicolas Williams 33874eb513 Fixup --slurpfile/argile docs 2015-05-29 15:25:30 -05:00
Nicolas Williams 1de56bc5df Add --slurpfile 2015-05-29 14:24:33 -05:00
Nicolas Williams 8a79f1d0a3 Better handling of stdout errors 2015-05-29 11:50:08 -05:00
Nicolas Williams f4f5d8882d Fix --argfile memleak (fix #799 part 2) 2015-05-29 11:49:45 -05:00
Nicolas Williams 4451c08007 Fix jv_load_file() (partial fix #799) 2015-05-29 11:48:10 -05:00
Nicolas Williams 89c57f7936 Revert "Fix #705 (--argfile weirdness)"
This reverts commit 3d2ab93b11.
2015-05-29 10:38:08 -05:00
Nicolas Williams 6af7857ccd Revert "Better argfile fix (#705, fix #736)"
This reverts commit 4f8567476d.
2015-05-29 10:37:58 -05:00
Nicolas Williams dff96aebcf Add jv_mem_strdup*() and callocs too 2015-05-25 21:20:37 -05:00
Nicolas Williams 8af52aaf52 Add ./configure --enable-all-static instead 2015-05-25 21:19:56 -05:00
Nicolas Williams d66e434fb4 Always add -all-static when --enable-static 2015-05-25 14:14:53 -05:00
Nicolas Williams 3700b457aa Update parser.c 2015-05-25 13:35:23 -05:00
Nicolas Williams 11f084f9a7 Keywords should be OK as object keys (fix #794)
With this change it's now OK to use keywords as object keys like so:

    {if:0}
2015-05-25 13:34:07 -05:00
Nicolas Williams 454a9dc042 Fix Windows build: util.c 2015-05-25 02:06:44 -05:00
Steven Penny ff458803ce Fix #793 2015-05-25 02:05:07 -05:00
Travis Gockel 110e009996 Add wrapping and clamping to jv_array_slice
Fix #716.  Fix #717.
2015-05-21 19:17:16 -05:00
Assaf Gordon 7b6a018dff Print offending object in runtime error messages
When reporting an error to the user, add information about the offending
object/value (possibly truncated).

The goal is to give a user some context regarding which input object
caused the runtime error.

Examples:

    $ echo '"hello"' | ./jq '-.'
    jq: error: string ("hello") cannot be negated

    $ echo '"very-long-string"' | ./jq '-.'
    jq: error: string ("very-long-...) cannot be negated

    $ echo '["1",2]' | ./jq '.|join(",")'
    jq: error: string (",") and number (2) cannot be added

    $ echo '["1","2",{"a":{"b":{"c":33}}}]' | ./jq '.|join(",")'
    jq: error: string (",") and object ({"a":{"b":{...) cannot be added

    $ echo '{"a":{"b":{"c":33}}}' | ./jq '.a | @tsv'
    jq: error: object ({"b":{"c":33}}) cannot be tsv-formatted, only array

(Fix #754)
2015-05-21 18:49:24 -05:00
Nicolas Williams a50e548cc5 Fix error message for @tsv 2015-05-21 18:20:15 -05:00
Nicolas Williams 4b258f7d31 Don't test input_filename/line_number yet 2015-05-21 17:55:27 -05:00
Assaf Gordon d1cb8ee0ad Add filename/line functions to jq (fix #753)
This adds `input_filename` and `input_line_number` built-in functions
for use in, for example, `error` messages.

Example:

    $ printf '{"a":1}\n{"a":2}\n' > 4.json
    $ printf '{"a":"hello"}\n' > 5.json
    $ ./jq '{ "file":input_filename, "line":input_line_number, "value":.a }' 4.json 5.json
    {
      "file": "4.json",
      "line": 1,
      "value": 1
    }
    {
      "file": "4.json",
      "line": 2,
      "value": 2
    }
    {
      "file": "5.json",
      "line": 1,
      "value": "hello"
    }
2015-05-21 00:49:32 -05:00
Nicolas Williams c7f063188f Fix formatting error in manual.yml 2015-05-21 00:11:21 -05:00
Assaf Gordon a6460f4fe2 Report filename:line on runtime errors (fix #752)
With this patch, jq run-time errors printed to stderr will contain the
filename and line of the offending input.

But note that the JSON text parser does (yet) print the input filename
on parse error.  A jq run-time error is a jq program error, not
including syntax errors nor JSON text input format errors.

Examples:

With stdin and multiple lines:

    $ printf '{"a":43}\n{"a":{"b":66}}\n' | ./jq '.a+1'
    44
    jq: error (at stdin:2): object and number cannot be added

With multiple files:

    $ printf '{"a":43}' > 1.json
    $ printf '{"a":"hello"}\n' > 2.json
    $ printf '{"a":{"b":66}}\n' > 3.json
    $ ./jq '[.a]|@tsv' 1.json 2.json 3.json
    "43"
    "hello"
    jq: error (at 3.json:1): object is not valid in a csv row

With very long lines (spanning multiple `fgets` calls):

    $ (  printf '{"a":43}\n' ;
         printf '{"a":{"b":[' ; seq 10000 | paste -d, -s | tr -d '\n' ;
         printf ']}}\n' ;
         printf '{"a":"hello"}\n' ) | ./jq '[.a] | @tsv'
    "43"
    jq: error (at stdin:2): object is not valid in a csv row
    "hello"

With raw input:

    $ seq 1000 | ./jq --raw-input 'select(.=="700") | . + 10'
    jq: error (at stdin:700): string and number cannot be added

Caveat:
The reported line will be the last line of the (valid) parsed JSON data.
Example:

    $ printf '{\n"a":\n"hello"\n\n\n}\n' | ./jq '.a+4'
    jq: error (at stdin:6): string and number cannot be added
2015-05-20 23:38:22 -05:00
Nicolas Williams 2e257563a6 Fix spelling 2015-05-20 10:21:30 -05:00
Nicolas Williams ad0090964b Document gsub/3 2015-05-19 01:19:51 -05:00
Nicolas Williams cbdaeb4062 Fix gsub, add gsub/3 (fix #782) 2015-05-18 23:02:11 -05:00
Nicolas Williams cf1306f469 Add -all-static when --enable-static 2015-05-18 12:07:29 -05:00
Nicolas Williams 7675f7434a Fix --stream assertion 2015-05-11 21:36:47 -05:00
Nicolas Williams 821d6404b6 Add error injection library 2015-05-11 21:36:46 -05:00
Nicolas Williams e3cb1f76cd Report read errors too (and fix #772) 2015-05-11 21:36:26 -05:00
Nicolas Williams ef12efbf8c Fix README 2015-05-02 17:36:37 -05:00
Nicolas Williams adb9b9b8a7 README: send questions to SO and Freenode 2015-05-02 17:34:59 -05:00
Nicolas Williams 07f94f82a2 usage() should check fprintf() result (fix #771) 2015-04-28 10:39:34 -05:00
Nicolas Williams 215b4c0fe1 Use jv_mem_alloc() in compile.c (fix #771) 2015-04-28 10:37:37 -05:00
Nicolas Williams 89963e1370 Fix header guards (fix #770) 2015-04-28 10:36:21 -05:00
Nicolas Williams 5d40f3a7c0 --raw-input wrongly adds NULs (fix #761) 2015-04-24 10:42:19 -05:00
Nicolas Williams 5e0e7a7ebd With inputs builtin, -n and -R can now coexist 2015-04-23 23:43:44 -05:00
Nicolas Williams dad6e42934 --raw-input ought to read NULs (partial fix #760)
We can't know how many bytes fgets() read when we reach EOF and fgets()
didn't see a newline; we can only assume that at least strlen(buf) bytes
were read.  This is quite obnoxious if one wants to use NULs in raw
input, but at least we can make reading "a\0b\0c\0" with no newline
yield "a\0b\0c", losing only the final sequence of NULs.

We can't use getline() either, since it will want to allocate a buffer
big enough for an entire line, and we might not have any newlines in our
input.  A complete fix will have to use getc() or read(), preferably the
latter.
2015-04-23 23:43:44 -05:00
Nicolas Williams 7d938487dd --slurp --raw-input is broken (fix #761) 2015-04-23 18:26:57 -05:00
Nicolas Williams fcfacb932d from_entries is broken (fix #767) 2015-04-23 18:25:32 -05:00
Assaf Gordon 6906d291c5 regex functions: report informative error if not available.
When trying to use the new regex functions (match/test/sub/capture/etc)
in a JQ that was compiled without the ONIGURAMA regex library,
report an informative error message instead of a 'not defined' error.

Before:

    $ echo '"foo"' | ./jq-old 'test("foo")'
    jq: error: test/1 is not defined at <top-level>, line 1:
    test("foo")
    jq: 1 compile error

After:

    $ echo '"foo"' | ./jq 'test("foo")'
    jq: error: jq was compiled without ONIGURAMA regex libary. match/test/sub and related functions are not available.
2015-04-22 16:57:03 -05:00
Andrew O'Brien 943090846f Fixes manual generation with psych
When running `make` I ran into a couple of problems building the manual. While
I'm not entirely sure that this is the root cause, it appears to have been
related to the fact that ruby 2.0 dropped syck completely in favor of psych
(which was introduced in 1.9.2) for YAML processing. I'm currently using ruby
2.1.0p0.

I'm assuming that the fact that the YAML engine was explicitly set to syck in
the Rakefile was an attempt to work around some incompatibility between the two
libraries, so I looked into what would be necessary to get it to work with the
newer one. The changes to `manual.yml` ended up being pretty minor: I ran it
through `iconv` to convert some ISO-8859-1 characters to UTF-8 and added some
quotes in places (apparently you can't start a string value with '`').
2015-04-22 11:01:39 -05:00
Assaf Gordon 2dc8621d05 Handle NUL in escaped-string output
When escaping string (e.g. for `@tsv` or `@csv` outputs),
escape NULs as '\0'.

Existing behaviour, unchanged by this patch:

    $ echo '"a\u0000b"' | ./jq '.'
    "a\u0000b"
    $ echo '"a\u0000b"' | ./jq -r '.' | od -a
    0000000   a nul   b  nl
    0000004

When using `@tsv`, escape NUL to `\0`:

    $ echo '"a\u0000b"' | ./jq -r '[.]|@tsv'
    a\0b
    $ echo '"a\u0000b"' | ./jq '[.]|@tsv'
    "a\\0b"
2015-04-20 16:51:48 -04:00
tal@whatexit.org b6cc00fa71 manual.yml: Clarify how to specify keys with ":" and special chars.
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2015-04-16 11:09:49 -05:00
Assaf Gordon 9e2d9b6419 docs: expand @tsv section - add escape sequences. 2015-04-15 23:24:24 -04:00
Assaf Gordon 3210b29bba @tsv: escape \r, \n, \\
When using '@tsv' output format with '-r' option,
escape \r, \n and \\ in addition to \t.

Example:
    $ printf '{"id":"hello\\ttab\\nworld","x":43 }' | jq .
    {
      "id": "hello\ttab\nworld",
      "x": 43
    }

Before: newlines are not escaped, generating invalid TSV output:

    $ printf '{"id":"hello\\ttab\\nworld","x":43 }' \
        | ./jq-old -r '[.id,.x] | @tsv'
    hello\ttab
    world	43

After: newlines are properly escaped:

    $ printf '{"id":"hello\\ttab\\nworld","x":43 }' \
        | ./jq-new -r '[.id, .x] | @tsv'
    hello\ttab\nworld	43

Before: backslashes themselves are not escaped, so there's no way to
distinguish between escaped characters and 'real' backslashes
(in the example below, there is should not be newline, despite the
output containing "\n".

    $ printf '{"x":"hello\\ttab\\\\new world"}' \
        | ./jq-old -r '[.x]|@tsv'
    hello\ttab\new world

After: backslashes are escaped:

    $ printf '{"x":"hello\\ttab\\\\new world"}' \
        | ./jq-new -r '[.x]|@tsv'
    hello\ttab\\new world
2015-04-15 16:11:58 -04:00
Nicolas Williams ddad9618dc Test fix for mktime 2015-03-31 10:03:35 -05:00
Nicolas Williams 24005287f4 Add $__loc__ (fix #740) 2015-03-30 22:36:04 -05:00
Nicolas Williams 1dcfc2f547 Include filename and lineno in error messages 2015-03-30 15:56:29 -05:00
Nicolas Williams 7d6d4066dd Tweak fix for #719 2015-03-24 15:05:06 -05:00
Assaf Gordon 4c22bda09b detect and report output writing errors
Detect output errors when the program exits.

Currently:
    $ echo '{}' | jq . > /dev/full && echo ok
    ok

with the patch:
    $ echo '{}' | jq . > /dev/full && echo ok
    Error: writing output failed: No space left on device

also apply to hardware/network/other I/O errors.

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2015-03-24 15:01:32 -05:00
Santiago Lapresta 9d62c6fd0c Adds Dockerfile
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2015-03-24 14:59:29 -05:00
Assaf Gordon 5e0db6dd65 partial handling of input errors
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2015-03-24 14:56:46 -05:00
Assaf Gordon 551d875be1 always propagate input errors to exit code
Improve robustness in automated system when using exit code in shell scripts,
by exiting with code 2 if there was any input error (even overriding other
possible error exit codes).
Exit code 2 is already used to indicate system errors.

Without the patch:
   $ jq . no-such-file ; echo $?
   jq: no-such-file: No such file or directory
   0

With the patch:
   $ jq . no-such-file ; echo $?
   jq: no-such-file: No such file or directory
   2

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2015-03-24 14:55:14 -05:00
Nicolas Williams 4f8567476d Better argfile fix (#705, fix #736) 2015-03-24 14:53:35 -05:00
Nicolas Williams ffa2832e33 Drop name-less label/break for now
See #734 and #658.
2015-03-24 01:21:17 -05:00
William Langford 0f7759f831 Fix #735 2015-03-23 22:11:55 -04:00
Nicolas Williams ccfba00178 Add more date builtins 2015-03-09 11:27:58 -05:00
Nico Williams 3e8183fcd5 Merge pull request #720 from agordon/exit-code-5-exceptions
exit with non-zero code on runtime exceptions
2015-03-08 17:09:07 -05:00
Nicolas Williams 0df387f6b5 Automake: jq depends on version.h (fix #721) 2015-03-08 16:52:08 -05:00
Assaf Gordon 4104c4fa1c exit with non-zero code on runtime exceptions
With this change, runtime exceptions are propagated to non-zero exit
code of 'jq', allow better scripting and automation. The new exit code
value is 5.

This allows using the shell's and/or operations ('&&' and '||') to
detect input runtime exceptions.

Before:
runtime exceptions are printed to STDERR, but not reported as non-zero exit-code:
    $ echo '"hello"' | jq '.|tonumber' ; echo $?
    jq: error: Invalid numeric literal at EOF at line 1, column 5 (while parsing 'hello')
    0

After:
    $ echo '"hello"' | ./jq '.|tonumber' ;  echo $?
    jq: error: Invalid numeric literal at EOF at line 1, column 5 (while parsing 'hello')
    5

Note that there's a subtle interplay when using "-e" option.
The value of the non-zero exit code changes from 4 to 5, but it still
indicates a 'failure' or non-true value.

Before:
    $ echo '"hello"' | jq -e '.|tonumber' ;  echo $?
    jq: error: Invalid numeric literal at EOF at line 1, column 5 (while parsing 'hello')
    4

After:
    $ echo '"hello"' | ./jq -e '.|tonumber' ;  echo $?
    jq: error: Invalid numeric literal at EOF at line 1, column 5 (while parsing 'hello')
    5
2015-03-06 18:50:58 -05:00
Nicolas Williams 33a85679b9 Fix #718 2015-03-06 00:18:45 -06:00
Nicolas Williams a4b9552c82 Add date builtins (fix #364)
Windows support for strptime() is missing.  We can add a copy of one
from a BSD later.
2015-03-06 00:14:15 -06:00
Nicolas Williams b82c231900 Remove -i option (#704)
In-place editing should be implemented with builtins for file I/O.
2015-03-05 21:52:02 -06:00
Nicolas Williams 3d2ab93b11 Fix #705 (--argfile weirdness) 2015-02-18 18:38:11 -06:00
Nicolas Williams c86ef36769 Test in-place functionality; fix #704 2015-02-18 18:01:12 -06:00
Nicolas Williams 3b8d08177f Fix #702 2015-02-18 10:21:21 -06:00
Nico Williams 0164ff0bbc Merge pull request #701 from sseemayer/master
Correct automake and autoconf version requirements
2015-02-18 10:00:23 -06:00
Stefan Seemayer a4baac2256 Correct automake and autoconf version requirements 2015-02-18 13:15:44 +01:00
Nicolas Williams 1e5e9f3ef8 Mention --disable-maintainer-mode in bison error 2015-02-17 11:00:07 -06:00
Nico Williams d58d85af9e Merge pull request #699 from hroptatyr/oniguruma-detection
Fix oniguruma detection logic
2015-02-16 10:54:33 -06:00
Sebastian Freundt 10e8d34fd3 Fix oniguruma detection logic
At the moment, a failed oniguruma header check would leave
HAVE_ONIGURUMA set to 1 still, resulting in a compiler error
in builtin.c.
2015-02-16 09:01:08 +00:00
Nicolas Williams ebc090ad15 README tweaks 2015-02-15 18:34:44 -06:00
Nicolas Williams 8cef5a37ab Add --disable-maintainer-mode; make bison optional
Also flex is now optional.

The outputs of flex and bison are now committed.  By default they get
built, but users who want to build from git can now

    ./configure --disable-maintainer-mode

to turn off the dependency on bison and flex.

Maintainers must, of course, commit the bison and/or flex outputs when
they make changes to parser.y and/or lexer.l, respectively.
2015-02-15 18:34:44 -06:00
Nicolas Williams 8afdeee785 Make Oniguruma/regexp optional
Tests won't pass if built without Oniguruma.  We don't have a way to
make a test optional yet.  That will come later.  For now the ability to
reduce build-time dependencies could really help some users.
2015-02-15 18:32:01 -06:00
Nicolas Williams a83a9e0bc2 Reduce number of msg callback typedefs 2015-02-13 15:58:02 -06:00
Nicolas Williams 0d414471bb Refactor moar: move parts of main.c into libjq
This adds utility functions for reading and parsing files that should be
easy to use by other apps, together with jq_start()/jq_next().
2015-02-13 15:58:02 -06:00
Nicolas Williams 91fb3df495 Refactor handling of inputs in main() (fix #667)
Much of this could be in libjq.  Eventually all of the work of reading
from files and looping over `jq_next()` should move into libjq, with
`main()` mostly doing all the command-line option processing.
2015-02-13 15:55:31 -06:00
Nico Williams 373ef71feb Merge pull request #692 from kim-toms/enhance-from_entries
Enhance from_entries to better deal with Amazon AWS Tags
2015-02-10 10:49:35 -06:00
Kim Toms ca78a746e6 Enhance from_entries to better deal with Amazon AWS Tags 2015-02-10 08:33:56 -05:00
Nicolas Williams 1e13e1e06e Fix sequence warnings (fix #686) 2015-01-30 17:21:50 -06:00
Nicolas Williams 42ff8a6959 Usage message for -h should go to stdout 2015-01-30 10:27:46 -06:00
William Langford 2e92c3e568 Merge pull request #678 from isomorphisms/patch-1
readability
2015-01-27 14:15:07 -05:00
i 5389fdb651 readability
Easier to read dashes than hyphens imho.
2015-01-27 13:32:15 -05:00
Nicolas Williams 7a295b30e0 Fix --raw-input 2015-01-20 00:22:24 -06:00
Nico Williams cca3e92854 Merge pull request #669 from joelpurra/join-empty-array-668
Empty arrays join/1 to an empty string, fixes #668 bug introduced by 9760245
2015-01-14 15:50:15 -06:00
Joel Purra 4d05dc55a3 Empty arrays join/1 to an empty string, fixes #668 bug introduced by 9760245 2015-01-14 12:27:55 +01:00
Nicolas Williams aeb52e29d6 Add debug and stderr builtins
And refactor setup of jv dump flags.
2015-01-14 01:39:13 -06:00
Nicolas Williams 97602456e3 join/1: respect empty strings (fix #668) 2015-01-14 01:32:25 -06:00
Nicolas Williams 8b5ff40402 Split on empty sep: fix #552 moar 2015-01-14 01:32:11 -06:00
Nicolas Williams d630597e87 Fix docs for split/0 2015-01-12 10:47:43 -06:00
Nicolas Williams 60d1ecb58a Fix #552 2015-01-12 10:44:44 -06:00
Nicolas Williams d927a2c185 Fix --run-tests arg handling 2015-01-12 10:43:06 -06:00
Nicolas Williams 6e7cf81d74 Look for jq/main.jq for imports 2015-01-02 19:16:43 -06:00
Nicolas Williams 736f1751ee Add comments to tests/run 2015-01-02 19:15:57 -06:00
Nicolas Williams 28aab9ca79 Fix tests/run in from-tar-ball case
Drop useless use of scripts/version
2015-01-02 19:15:11 -06:00
Nicolas Williams 972c048149 Add static build instructions (fix #294) 2015-01-01 16:26:55 -06:00
Nicolas Williams 902aa39fce Fix warning in util.c 2015-01-01 15:15:10 -06:00
Nicolas Williams 22f55aa585 Include missing EXTRA_DIST items; fix #661 2015-01-01 15:14:49 -06:00
Nicolas Williams 157c95b988 Add mkstemp() for mingw build 2015-01-01 03:14:55 -06:00
Nicolas Williams 426913f127 Fix memleak introduced run-tests enhancement 2015-01-01 02:12:08 -06:00
Nicolas Williams c4cc62a053 Add more missing test files 2015-01-01 01:49:40 -06:00
Nicolas Williams 862696a778 Add missing test file 2015-01-01 01:39:50 -06:00
Nicolas Williams 86bc662bd2 Move some module tests into all.test 2014-12-31 20:09:53 -06:00
Nicolas Williams ae7f8d6ab9 Further module system revamp (fix #659)
To import a module now use:

    # Import module.jq file:
    import "relative/path/to/module" as foo;

    # Use the module's defs as foo::<def-name>

To import a JSON file:

    # Read file.json:
    import "relative/path/to/file" as $foo;
    #
    # Use as $foo::foo

Using `-L` now drops the builtin library path and appends the requested
path to the empty array (or the result of an earlier `-L`).

Support for the `$JQ_LIBRARY_PATH` environment variable has been
removed.
2014-12-31 20:09:53 -06:00
Nicolas Williams 7dc34b92af Add label $name | EXP; fix break
This is to fix the problem where `break` is dynamic, not lexical.

With this it should be possible to do this sort of thing:

    label $break | inputs | if ... then $break|error else . end

This is a backwards-incompatible change for master, but the previous
`break` hadn't shipped yet.

Still needed:

 - testing
2014-12-30 11:42:45 -06:00
Nicolas Williams cbfc0d6130 Remove string indexing by string (fix #454)
This turns out to have been a bad idea:

    "foo"|.["o"]

it interacts badly with `path()`.

See #454 for the gory details.
2014-12-30 11:31:52 -06:00
Nicolas Williams c308b2881f Fix parser leak (fuzzing) 2014-12-30 11:31:52 -06:00
Nicolas Williams c959d1fca1 Fix lineno counting in jq_test.c 2014-12-30 11:28:50 -06:00
Nicolas Williams 63511a874f Add support for testing erroneous programs
This will allow moving tests of some tests from tests/run to
tests/all.test.  It will also be useful for more extensive testing of
language features where we want nice error messages, or errors at all,
to be produced, particularly language features where the errors arise at
compile-, codegen-, or link-time rather than parse-time.

This does not check for run-time errors though -- those we can already
check for with try/catch.

    $ jq --run-tests
    %%FAIL
    break
    jq: error: break used outside labeled control structure
    ^D
    1 of 1 tests passed (0 malformed)
    $
2014-12-30 10:55:30 -06:00
Nicolas Williams 90c705f138 Make --run-tests more informative
Print the line number and program text on the same output line where a
failure is reported, for all failures that were already reported on
lines starting with '***'.  This makes poring over test logs with
failures much easier.
2014-12-30 10:55:30 -06:00
Nicolas Williams b67bad82cb Allow resetting of jq err callback
This will be useful for the upcoming test-erroneous-programs improvement
to --run-tests, so we can switch between the default error reporting
method (print to stderr) to a method internal to --run-tests, and back.

The idea is that when testing programs that are expected to compile (and
link), it'd be nice if errors continue going to stderr, while when
testing programs that must fail to compile (or link), the error has to
be captured so it can be compared to the error expected by the test.
2014-12-30 10:52:38 -06:00
Nicolas Williams 6981b46338 Fix streaming bug (fuzzing) 2014-12-30 01:35:22 -06:00
pkoppstein 752e00bc66 transpose/0 for possibly jagged matrices 2014-12-27 18:52:36 -06:00
pkoppstein 00f018a054 bsearch(x) (binary search): builtin.c (tested), with documentation and test case. Always yields an integer (even if input is unsorted); returns (-1 - ix) if x is not in input array. 2014-12-27 18:44:20 -06:00
pkoppstein 56344670f4 ascii_upcase/0 and ascii_downcase/0 2014-12-27 18:38:52 -06:00
Nicolas Williams 5df20f4954 Add debug builtin
And refactor setup of jv dump flags.
2014-12-27 18:25:34 -06:00
Nicolas Williams a9c613e87d Don't force C API users to set input cb 2014-12-27 18:15:10 -06:00
Nicolas Williams 7538ef34be Make jq --run-tests show test line numbers 2014-12-27 18:15:10 -06:00
Nicolas Williams 5c62a4b805 Fix uninitialized struct field 2014-12-27 18:09:28 -06:00
Nicolas Williams 559863ca68 Streaming parser torture tests 2014-12-26 23:18:07 -06:00
Nicolas Williams 845ad5e969 Fuzz JSON parser 2014-12-26 23:05:57 -06:00
Nicolas Williams 243b1f8e5e Better quoting in tests/run 2014-12-26 23:05:56 -06:00
Nicolas Williams 5bfb9781f7 Add Streaming parser (--stream)
Streaming means that outputs are produced as soon as possible.  With the
`foreach` syntax one can write programs which reduce portions of the
streaming parse of a large input (reduce into proper JSON values, for
example), and discard the rest, processing incrementally.

This:

    $ jq -c --stream .

should produce the same output as this:

    $ jq -c '. as $dot | path(..) as $p | $dot | getpath($p) | [$p,.]'

The output of `jq --stream .` should be a sequence of`[[<path>],<leaf>]`
and `[[<path>]]` values.  The latter indicate that the array/object at
that path ended.

Scalars and empty arrays and objects are leaf values for this purpose.

For example, a truncated input produces a path as soon as possible, then
later the error:

    $ printf '[0,\n'|./jq -c --stream .
    [[0],0]
    parse error: Unfinished JSON term at EOF at line 3, column 0
    $
2014-12-26 23:05:56 -06:00
Nicolas Williams 906d2537b9 Allow C-coded functions to empty
Just return a jv_invalid() without a message.
2014-12-26 23:05:56 -06:00
Nicolas Williams bbb287f54b Add BLOCK_8() macro 2014-12-26 23:05:56 -06:00
Nicolas Williams e51e916e31 Fix foreach non-progation of errors
Errors were being re-propagated as "break".

Also add `error/0` builtin, which made this fix easier.
2014-12-26 23:05:20 -06:00
Nicolas Williams 46e3125e26 Allow zero-length buffers in jv_parser_set_buf()
If the caller is at EOF and has no more bytes to feed the parser, how is
the parser to be told about the EOF condition?  A small fix to allow
zero-length buffers in jv_parser_set_buf() fixes this problem (it also
makes it easier to deal with async I/O: feed the parser whatever is
available, including nothing).
2014-12-26 22:56:48 -06:00
Nicolas Williams c02fcc8fef Fix in and inside 2014-12-26 22:56:48 -06:00
Nico Williams 25d4cbd1b3 Merge pull request #624 from Janrain-Colin/patch-1
Doc correction
2014-12-26 22:14:17 -06:00
Nico Williams 3234a7681e Merge pull request #366 from slapresta/inversed-contains
Added `in` command
2014-12-26 22:00:41 -06:00
Nico Williams 36077da66a Merge pull request #628 from slapresta/map-on-objects
Modify map\1 so that it works on objects
2014-12-26 21:50:11 -06:00
Nicolas Williams 9e1fd8cf4d Fix infinite loop in read_more() (fix #656) 2014-12-24 16:49:55 -06:00
Nicolas Williams a55085b662 Fix EOF handling; fix #656 2014-12-24 16:07:36 -06:00
Nico Williams e0f9b6a5cd Merge pull request #651 from eiiches/fix-examples
Fix several errors in the manual
2014-12-24 11:29:29 -06:00
Nicolas Williams 0053aa868c Add @tsv; fix #645 2014-12-24 11:21:18 -06:00
Nico Williams fab2048648 Merge pull request #653 from msabramo/patch-1
.travis.yml: Set sudo false; use containers
2014-12-24 11:20:47 -06:00
Nicolas Williams 56ae88d9d5 Module search revamp for pkg managers
The search path listed in an import directive can now be an array.  The
top-level search path is appended.  Null and empty strings in the path
terminate any search.  The "." in "." and "./*" is replaced with the
directory containing the file doing the import (for command-line
programs this is the current directory, though that may be a bad idea).

No version numbers or anything of the sort are gratuitously added to the
search paths.

All this makes external package managers possible by allowing
dependencies to be installed local to dependents.
2014-12-24 02:31:51 -06:00
Nicolas Williams 1cdcff1c01 Fix bug caused by 8ca07a0 2014-12-24 02:25:11 -06:00
Nicolas Williams 8ca07a08c3 Fix #348: reject unescaped control chars 2014-12-24 00:01:43 -06:00
Nico Williams 2775b90b20 Merge pull request #601 from slapresta/redefine-any-all
Define {any,all}/0,1 in terms of {any,all}/2
2014-12-23 23:37:25 -06:00
Nicolas Williams ae312bd7fe Use __attribute__ __printf__ with GCC 2014-12-23 23:22:57 -06:00
Nicolas Williams 9017b83d4e Make values faster (fix #652) 2014-12-23 23:22:57 -06:00
Nicolas Williams 02cf1831e9 Fix #649 2014-12-23 18:16:21 -06:00
Marc Abramowitz d237de7b30 .travis.yml: Set sudo false; use containers
This is the way to enable Travis's fancy new containers, which should result in faster builds.
2014-12-22 21:37:23 -08:00
Santiago Lapresta 84d3203dd1 Define map_values 2014-12-22 23:05:35 +01:00
Santiago Lapresta a3234034b5 in is now inside, added in as inverse of has 2014-12-22 22:17:53 +01:00
Santiago Lapresta b748eae035 Added in command 2014-12-22 22:14:06 +01:00
Eiichi Sato a6656edc7c Fix examples in manual 2014-12-21 03:26:04 +09:00
Eiichi Sato b4a9ea56ae Fix indents in manual.yml 2014-12-21 03:24:45 +09:00
Eiichi Sato fdbc91e3c4 HTML-escape jq programs in manual 2014-12-21 03:24:41 +09:00
Eiichi Sato bd67bb8273 Fix examples in manual 2014-12-21 03:24:38 +09:00
Nico Williams 54a0470c42 Merge pull request #611 from StevenMaude/tutorial-typo-fixes
Fix typos in tutorial
2014-12-12 17:46:17 -06:00
Nico Williams 3dd790c408 Merge pull request #616 from ianmiell/ianmiell_requirements
Requirements made slightly more complete
2014-12-12 17:45:17 -06:00
Nicolas Williams be11b2768f Add until(cond; next); fix #639 2014-12-12 17:35:59 -06:00
Nicolas Williams 2075bec60f Fix #647, but caused by 89791a0 2014-12-12 17:05:16 -06:00
Nicolas Williams 44c2382402 Add --argjson, fix #648 2014-12-12 16:40:07 -06:00
Nico Williams fd041c2064 Merge pull request #642 from Janrain-Colin/patch-3
fix typo in manual
2014-12-12 16:03:18 -06:00
Colin von Heuring c6495f6632 fix typo in manual 2014-12-06 16:27:43 -08:00
Nicolas Williams 7243989c52 Fix refcount leak, fix #618 2014-11-29 15:24:43 -06:00
Nicolas Williams 46613af64c STOREV/LOADV* should also print refcnts 2014-11-28 20:52:22 -06:00
Nicolas Williams 79000644c6 Enable printing of stack val refcnts 2014-11-28 19:35:29 -06:00
Nicolas Williams b349e7d9a8 Print stack value refcounts when tracing (#636) 2014-11-28 19:26:41 -06:00
Colin von Heuring 6c8882caf3 Doc correction 2014-11-23 11:37:50 -08:00
William Langford 30e00820a7 Fix #600. Add regression test 2014-11-12 20:46:34 -05:00
Ian Miell 0f81e1bcd0 Requirements made slightly more complete: cf https://github.com/ianmiell/shutit/blob/master/library/jq/jq.py 2014-11-11 23:02:31 +00:00
Steven Maude cb376b27f8 Fix typos in tutorial
parrent > parent, url > URL
2014-11-05 10:47:37 +00:00
Santiago Lapresta dc5e77ef8e Define {any,all}/2 independently from {any,all}/0 2014-10-21 13:00:38 +02:00
Santiago Lapresta 72735167f9 Define {any,all}/{0,1} in terms of {any,all}/2 2014-10-20 22:01:25 +02:00
Nicolas Williams 89791a000b Add support for JSON sequence MIME type
Per draft-ietf-json-text-sequence-07 (which soon will be published as an
RFC).
2014-10-12 08:44:40 -05:00
William Langford 3411167c03 Properly call onig_error_code_to_str
Fix #585
2014-10-06 21:48:56 -04:00
pkoppstein 85f0e30c81 fix sub (#586); add gsub/3; add transpose/0.
Signed-off-by: William Langford <wlangfor@gmail.com>
2014-10-06 21:32:07 -04:00
Nicolas Williams 1796a716ea Restore split/1; don't use regexp (fix #576) 2014-10-03 17:58:45 -05:00
Nicolas Williams 07d8d5c5ef Update docs about sort/group/min/max/unique 2014-10-03 17:58:44 -05:00
Nicolas Williams 0b29b44cab from-entries should work with EC2 (fix #592) 2014-10-03 17:47:13 -05:00
Nicolas Williams a0a350f468 Remove sort/1 and group/1
Reserve these for sorting with jq-coded comparators.
2014-10-03 17:47:13 -05:00
Nicolas Williams 4a57b84db0 to_entries should not sort keys (fix #561) 2014-09-30 21:52:30 -05:00
William Langford f7a2af7052 Properly handle when objects cannot be folded
Fix #579.
2014-09-22 20:14:46 -04:00
Nicolas Williams 456bafa82f Drop the jq version directory from search path 2014-08-30 00:58:24 -05:00
Nicolas Williams 7a8e3c759e Never close stdin; allow multiple - arguments 2014-08-30 00:40:03 -05:00
Nicolas Williams b70bea8d34 Handle invalid inputs in argument files (fix #562) 2014-08-30 00:40:03 -05:00
William Langford d177944b75 Properly handle incomplete json when input is file
Fix #562
2014-08-28 21:52:45 -04:00
Nicolas Williams 9bd55487ea C-coded builtins should not assert (fix #553) 2014-08-21 00:12:29 -05:00
Nicolas Williams 23e2e2eab3 Quiet warning about freopen() of stdout 2014-08-20 20:49:30 -05:00
Nicolas Williams 8d2d5e37e5 Drop "any/" in module search; use 1.x-master 2014-08-20 20:48:48 -05:00
Nicolas Williams 287e2c96ed BSD sed lacks -r; fix #551 2014-08-14 03:31:25 -05:00
Nicolas Williams f8d9478517 Document module system
Fix #511.
2014-08-14 03:26:28 -05:00
Nicolas Williams 1ba8c2cfa6 Add module directive, modulemeta builtin
Fix #425.
2014-08-14 03:26:26 -05:00
Nicolas Williams d654724fef Add JV_OBJECT() macro 2014-08-14 03:21:38 -05:00
Nicolas Williams 3375089013 Add JV_ARRAY() macro 2014-08-14 03:21:38 -05:00
Nicolas Williams 9f13afa20f Add jq_report_error() function; use it
Put a stop to fprintf(stderr, ...) where we shouldn't.
2014-08-14 03:21:35 -05:00
Nicolas Williams 908c10928b Fix #549 2014-08-11 14:58:44 -05:00
Nicolas Williams c321c3b86b Constant fold objects 2014-08-09 20:47:03 -05:00
Nicolas Williams f87521183f Fold constant arrays 2014-08-09 20:27:08 -05:00
Nicolas Williams 8a561120c4 More constant folding: null, true, and false
A step towards doing constant folding for arrays and objects.
2014-08-09 19:15:50 -05:00
Nicolas Williams 20ca21cb0d Out of bounds negative array indices should raise 2014-08-09 19:15:50 -05:00
Nicolas Williams 2d1a75f550 .foo[-1] = ... trips assertion (fix #490) 2014-08-09 13:10:41 -05:00
Nicolas Williams 8cddb7c681 Allow any number of jq-coded function arguments 2014-08-09 12:42:39 -05:00
Nicolas Williams 4249bbf0d5 Make nwise private 2014-08-08 19:36:43 -05:00
Nicolas Williams 8a57c27e92 Make regexp builtins and range/3 use #524 too 2014-08-08 19:31:11 -05:00
Nicolas Williams 8706b6b16a Use def f($a): ...; syntax for builtins 2014-08-08 19:23:46 -05:00
Nicolas Williams 32c08d48f3 Add def f($arg): syntax (fix #524) 2014-08-08 19:16:01 -05:00
Nicolas Williams fc52e78ed4 Fix tests for #432 2014-08-08 18:17:58 -05:00
Nicolas Williams eb12f0358e Fix docs YAML (introduced in #432) 2014-08-08 18:01:42 -05:00
pkoppstein a696c6b551 regex filters (#432): scan, splits, split, sub, gsub 2014-08-08 17:00:14 -05:00
Nicolas Williams 0d437e25de Better error msg for bad shell quoting (fix #538) 2014-08-06 17:51:53 -05:00
William Langford 6744664ff4 Actually check version for bison.
Clean up oniguruma checks.
2014-08-06 12:26:34 -04:00
William Langford ef5bd5a6de Merge pull request #532 from wtlangford/top-removal
Remove duplicate TOP insertions, drop unused defs.
2014-08-05 13:41:59 -04:00
Nicolas Williams 2159f9f5e0 Document TCO a bit more 2014-08-05 01:15:33 -05:00
Nicolas Williams 2518b3a34f Fix docs (missing backquotes) 2014-08-05 01:15:33 -05:00
William Langford c3d6d5fd37 Merge pull request #523 from eproxus/patch-1
Add example of selecting object with keys
2014-08-05 00:50:21 -04:00
William Langford a25950a58c Remove duplicate TOP insertions, drop unused defs. 2014-08-05 00:46:10 -04:00
pkoppstein f8fed0f0bc Apply TCO to recurse/1, add recurse/2; tweak docs
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-08-04 21:11:01 -05:00
Adam Lindberg 89545ea9d0 Add example of selecting object with keys
It is not very straight forward how to select objects by their attributes from an array. This example provides the basic use case of how to select some objects based key values.
2014-08-01 12:13:35 +02:00
Nicolas Williams a2f99d43c3 Some builtins are unnecessarily special (fix #521) 2014-08-01 00:20:30 -05:00
Nicolas Williams a7878a8d1a Correct @wtlangford's address 2014-07-30 13:19:46 -05:00
Nicolas Williams c73ad74be0 Add @wtlangford to the AUTHORS list 2014-07-30 13:02:34 -05:00
pkoppstein 53993a9942 Add capture; document regular expression filters
Fix #493.
2014-07-28 19:41:21 -05:00
Nicolas Williams e1b20b48f0 Add first, nth, last (fix #510) 2014-07-28 13:18:58 -05:00
William Langford 14bc1f6526 Fixing crash when a library's dependency could not be loaded.
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-07-27 17:51:48 -05:00
Nicolas Williams 2e2538ccb8 Fold constants (fix #504) 2014-07-27 17:48:49 -05:00
Nicolas Williams ae27178352 Fix #499, #500 2014-07-27 17:41:40 -05:00
William Langford 0f89270d08 Changing color codes to fix #495 2014-07-22 23:05:43 -05:00
Nicolas Williams 10f5a3448d Do fix remaining bison warning (fix #433) 2014-07-22 22:51:11 -05:00
Nicolas Williams 50b85c2911 Rename jq_memmem() to _jq_memmem() 2014-07-22 22:51:11 -05:00
William Langford 38b939688a Added library system with -l, -L, and JQ_LIBRARY_PATH
Created util.[ch] to hold common utilities.
2014-07-22 22:51:11 -05:00
Nicolas Williams 01fc8168e9 Add -i option to edit files in place (fix #105) 2014-07-20 00:11:23 -05:00
Nicolas Williams 3362fb3406 Add -n to short usage msg; fix --arg msg 2014-07-20 00:04:24 -05:00
Nicolas Williams 8c3d503d54 Fix bison empty rule warnings (#433, partial fix) 2014-07-14 16:41:48 -05:00
Nicolas Williams 2bb9fc5dda Fix #484, try/catch syntax has conflicts 2014-07-14 16:41:48 -05:00
Nico Williams ea3e947b50 Merge pull request #483 from else/master
jq 1.4 is in Debian
2014-07-14 09:16:01 -05:00
Simon Elsbrock 538e012e85 jq 1.4 is in Debian
jq 1.4 is in Debian, please see https://tracker.debian.org/pkg/jq
2014-07-14 11:24:47 +02:00
Nico Williams 918ada06dc Merge pull request #482 from mbruggmann/manual-fixes
Fix manual example for `endswith`.
2014-07-13 19:01:57 -05:00
Nicolas Williams cb2737a433 Add regression test for #479 2014-07-13 19:00:37 -05:00
Nicolas Williams c38e5b1b2c Lib defs don't bind to each other; fix #479 again 2014-07-13 18:36:27 -05:00
Marc Bruggmann 089041b090 Fix manual example for endswith. 2014-07-13 19:02:21 +02:00
Nicolas Williams cadf8c3cb0 Revert "Library defs don't bind to each other; fix #479"
This reverts commit 6b6e3f4262.  It was
the wrong fix.
2014-07-13 02:04:27 -05:00
Nicolas Williams 6b6e3f4262 Library defs don't bind to each other; fix #479 2014-07-12 18:38:55 -05:00
Nicolas Williams 6e88d419ba Fix #478 assertion failure 2014-07-10 19:23:02 -05:00
Nico Williams ab4bf05300 Merge pull request #475 from qqshfox/patch-1
Fix examples for `del` in manual
2014-07-09 10:09:28 -05:00
Hanfei Shen ecac7bd3a1 Fix examples for del in manual 2014-07-09 16:37:17 +08:00
Zhiming Wang 1d620c0de3 Fix invalid YAML in manual.yml
Grave accent "`" is a reserved indicator in YAML per
http://www.yaml.org/spec/1.2/spec.html#id2774228, so it can't start a
plain scalar. This commit applies quoting to titles beginning with
grave accents.

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-07-09 01:12:23 -05:00
Zhiming Wang f68815d5ad Add tests/all.trs to .gitignore
tests/all.trs is auto generated during make check.

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-07-09 01:11:41 -05:00
Nicolas Williams 666aeda82c Better document path()'s power; also |= 2014-07-09 01:05:06 -05:00
Nicolas Williams 5be80b81eb Add foreach EXP as $var (INIT; UPDATE) form 2014-07-09 00:43:06 -05:00
Nicolas Williams 10477fc2d4 Make while() handle break 2014-07-09 00:39:25 -05:00
Nicolas Williams d8672e1b2a Make C-coded built-ins take jq_state * argument 2014-07-07 23:37:42 -05:00
Nicolas Williams a68958e5dc error(x) should not tostring its arg; fix #466 2014-07-07 22:26:53 -05:00
Nicolas Williams 824f7df404 limit should use break 2014-07-07 22:26:07 -05:00
Nicolas Williams 4bcff40828 Make any/2 and all/2 efficient using foreach 2014-07-07 22:25:34 -05:00
Nicolas Williams 80a9937e48 exec_stack.h was not using jv_mem_realloc() 2014-07-07 19:33:19 -05:00
Nicolas Williams b963ebaf1d jv_invalid() shouldn't allocate
jv_invalid() should behave like jv_invalid_with_msg(jv_null()), and
neither should allocate memory, because neither ought to need to.
2014-07-07 19:33:18 -05:00
Nicolas Williams eb45372414 jv_show() should be able to display invalid values 2014-07-07 19:33:18 -05:00
Nicolas Williams 539dccae90 Add break builtin for foreach 2014-07-07 19:33:17 -05:00
Nicolas Williams 6458910ba3 Explain foreach's powers a bit more 2014-07-07 17:22:53 -05:00
Nicolas Williams ef9b591215 Document path(path_expression) builtin 2014-07-07 17:13:51 -05:00
Nicolas Williams f845808813 Add missing backtick in manual 2014-07-07 17:13:48 -05:00
Nicolas Williams 25719b5596 $var["foo"]=1 can't work as expected; doc fix #236 2014-07-07 16:56:14 -05:00
Nicolas Williams 8780bc0b8e Better check for lib has only functions (fix #138) 2014-07-07 16:03:32 -05:00
Nicolas Williams 821cd31e67 Add any/N and all/N x N in (1, 2) (fix #455)
Contributed by @pkoppstein.
2014-07-06 20:08:06 -05:00
Nicolas Williams 5a863bf010 Add foreach and limit 2014-07-06 19:40:05 -05:00
William Langford d0ca11d665 Add support for negative indices for .[]; fix #462
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-07-06 01:40:22 -05:00
Nicolas Williams 7d3a44a1e2 Add general ? operator 2014-07-06 01:29:43 -05:00
Nicolas Williams 7fce34292e Add try EXP catch EXP 2014-07-06 01:29:42 -05:00
Nicolas Williams 9113282c27 Document error/1 2014-07-06 00:27:05 -05:00
Nicolas Williams 78a8419428 Add while(cond; update) (fix #314) 2014-07-02 22:05:33 -05:00
Nicolas Williams ff9a61ead2 Add range(init;upto;by) (fix #317) 2014-07-02 22:05:33 -05:00
Nicolas Williams 3a647d3e47 Multi-arity needs better errors (fix #438) 2014-07-02 02:18:10 -05:00
Nicolas Williams 1f17ef3623 Descrive generators, range() with by to manual 2014-07-01 23:32:44 -05:00
Nicolas Williams a6030ab374 Fix typo in manual 2014-07-01 23:32:40 -05:00
William Langford 9f4d7e2fb8 Fixed base64 issue with UTF-8 strings
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-07-01 22:56:13 -05:00
Nicolas Williams ba1acbe961 Teach disassembler about TAIL_CALL_JQ 2014-07-01 22:49:25 -05:00
Nicolas Williams 9deee38dc8 Fix off-by-one in TCO
Now we have the ability to define a generator in jq:

        def for(cond; update):
            def _for:
                if cond then ., (update | _for) else . end;
            _for;

        for(. < 10; . + 1) # generates numbers between `.` and 10

    Running this by hand with --debug-dump-disasm (with a fix for that
    coming up next) we can see that the call to _for is optimized:

        _for:0:
          0000 DUP
          0001 CALL_JQ cond:0^1
          0005 JUMP_F 0022
          0007 POP
          0008 FORK 0012
          0010 JUMP 0020
          0012 CALL_JQ update:1^1
          0016 TAIL_CALL_JQ _for:0^1
          0020 JUMP 0023
          0022 POP
          0023 RET

    And timing this with 1000, 10000, 100000 iterations shows that
    indeed we must be applying TCO; otherwise, without TCO, this gets
    very slow very quickly.
2014-07-01 22:40:40 -05:00
Nicolas Williams 436941d48b TCO to the max!
Close #446.

    Currently tested by disassembling and tracing various recursive jq
    programs by hand under valgrind.  An improved test framework that
    can test for errors and specific bytecode patterns is in
    development.
2014-06-30 23:41:20 -05:00
William Langford e1c5a2f575 Added cross-compilation script to build libjq for iOS.
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-06-30 20:02:19 -05:00
Zhiming Wang a5e9086676 Let @uri produce uppercase hexadecimal digits...
instead of lowercase ones.

According to RFC 3986,

   The uppercase hexadecimal digits 'A' through 'F' are equivalent to
   the lowercase digits 'a' through 'f', respectively.  If two URIs
   differ only in the case of hexadecimal digits used in percent-encoded
   octets, they are equivalent.  For consistency, URI producers and
   normalizers should use uppercase hexadecimal digits for all percent-
   encodings.

See https://github.com/stedolan/jq/issues/451 for details.

Test suite and manual are also updated to reflect this change.

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-06-30 19:52:42 -05:00
Nicolas Williams 1204e328df Add much commentary about CALL_JQ and call frames 2014-06-30 19:40:56 -05:00
Nicolas Williams 3e1baf5916 Get "Try Online" button working (fix #440) 2014-06-24 00:39:00 -05:00
Nicolas Williams beb0d081bb Improve TCO
Instead of checking for self-recursion check that the thing we're
calling is a function and not a closure, therefore the new frame will
have the same env as the current frame.
2014-06-23 19:59:00 -05:00
Nicolas Williams 4b7c701fdd Tail call optimization (close #437) 2014-06-22 00:24:02 -05:00
Nicolas Williams 233e32208c Fix option stacking bug 2014-06-21 18:01:00 -05:00
Nicolas Williams 63e31c2a35 Remove stray fprintf() from last commit 2014-06-20 23:29:53 -05:00
Nicolas Williams f9349becab Allow stacking of short options (fix #346) 2014-06-20 23:26:54 -05:00
Nicolas Williams ff5cf0abc7 Remove extra free of testmode
I added a jv_free() too many while rebasing @wtlangford's contribution.
It could only be triggered by calling `_match_impl` directly.
2014-06-18 21:11:06 -05:00
William Langford 8ff935c01a Added regex support as per issue #164.
jq now depends on oniguruma for regex support.
Modified configure.ac accordingly.

Added valgrind suppression file for oniguruma to prevent one-time and bounded
leaks from causing tests to fail.

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-06-18 21:02:47 -05:00
Nicolas Williams 5d9d1b1020 Partial revamp of scripts/crosscompile
Fix #430.
2014-06-18 16:29:37 -05:00
Nicolas Williams 8725d9fa3e Minor style cleanup in main.c 2014-06-17 19:04:55 -05:00
Nicolas Williams 9ae2d66d21 Remove remnants of --online-input option 2014-06-17 19:02:23 -05:00
Nicolas Williams 0c762925b2 Add -j / --join-output option, similar to -r
Fix #215.
2014-06-17 18:59:35 -05:00
David Haguenauer ad52026550 Fixed typo.
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-06-17 18:47:27 -05:00
Nicolas Williams cf7895876b Rebuild jq.1.prebuilt 2014-06-17 18:17:50 -05:00
Nicolas Williams f902d5fae6 Remove trailing whitespace from manual.yml 2014-06-17 18:17:50 -05:00
Nicolas Williams 98e3ebc378 Use new style in manual for defs with args 2014-06-17 18:17:50 -05:00
Santiago Lapresta dde43f796e Simplified standard library
Close #426.

Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-06-17 18:17:50 -05:00
Nicolas Williams 7ee3b724d1 Fix #419, improve Cannot index message (sortof)
jv_get() doesn't know if it's being called in the context of a jq
program or not, so it can't produce a very useful error message when the
types of the to-be-indexed value and the key don't agree.  For now,
including the key (when it is a short string) in the error message is as
significant an improvement as is easy to make.
2014-06-17 17:00:23 -05:00
Nicolas Williams e151a300d0 Fix #266; make help message more useful 2014-06-17 16:47:12 -05:00
Nicolas Williams f58f771660 Fix #280: from_entries of [] is null, should be {} 2014-06-16 22:40:44 -05:00
Nicolas Williams ea63d5d3c2 No args default w/ tty stdout, not tty stdin #220 2014-06-16 22:34:50 -05:00
Nicolas Williams ea0ffe24f8 Don't declare environ on WIN32 2014-06-16 22:34:37 -05:00
Santiago Lapresta 061a604a2a Added flatten and flatten(x) functions
Fix #415.
2014-06-16 17:03:05 -05:00
Nicolas Williams f741074267 Remove download 1.3 button; reposition try online 2014-06-16 17:01:09 -05:00
Nicolas Williams de5952d67e Add ChangeLog and NEWS files 2014-06-16 15:29:48 -05:00
Nicolas Williams a03f4802ef Add missing contributors to AUTHORS 2014-06-16 15:29:31 -05:00
Nicolas Williams 778ba9f7ab Update AUTHORS 2014-06-16 13:25:50 -05:00
Nicolas Williams 676e0ce38e Fix Win64 jq.exe link on main page 2014-06-16 11:13:55 -05:00
Nicolas Williams 088a42fc53 Allow multiple functions with different arities 2014-06-16 00:05:30 -05:00
Nicolas Williams 63dbac28ad Fix typo; rename jq.1-prebuit to jq.1.prebuilt 2014-06-13 19:04:06 -05:00
Nicolas Williams e9a1de47b4 Add env builtin 2014-06-13 17:51:41 -05:00
Nicolas Williams d14fca0cc6 Document the lambda nature of function args #391 2014-06-13 17:22:43 -05:00
Nicolas Williams bb83813f51 Automatically update jq.1.prebuilt 2014-06-13 17:22:17 -05:00
Nicolas Williams 8225d4e0c5 Add jqplay link to the site 2014-06-13 17:21:56 -05:00
Jingwen Owen Ou d45195ab5c jqplay has a domain now 2014-06-12 18:17:39 -07:00
Nicolas Williams 73bd347fcf Add links to Solaris 11+ executables 2014-06-12 19:26:05 -05:00
Nicolas Williams c9dbe090c5 Include links to 1.3 executables in gh-pages 2014-06-12 17:30:30 -05:00
Nicolas Williams 61cc0d4fb1 Make a better jq.1 when Ruby deps missing 2014-06-12 17:30:09 -05:00
Nico Williams 15c4a7f95f Merge pull request #400 from kdeme/master
Heuristic IEEE754 endianness autoconf detection

Use AC_C_BIGENDIAN, though it's not really the correct approach.  Autoconf ought to have provided a test specifically for IEEE754 endianness, since it can differ from integer/pointer endianness!
2014-06-12 15:55:41 -05:00
Kim De Mey 20e629284f Detect endianness at configuration with Autoconf AC_C_BIGENDIAN feature 2014-06-11 10:37:32 +02:00
Nicolas Williams e73951f3d1 Add lib.h to dist file list 2014-06-09 18:58:46 -05:00
Nicolas Williams fdf843c08b Make the note about shell quoting appear on site 2014-06-09 17:43:17 -05:00
Nicolas Williams cf145ec65e Add note about cmd.exe quoting 2014-06-09 10:22:55 -05:00
Nicolas Williams 4889844d0c Building docs fails on powerpc (#349)
Thanks @shym!!
2014-06-09 09:39:02 -05:00
Nicolas Williams dc7857c585 Update site news 2014-06-08 20:45:06 -05:00
Nicolas Williams d65dbf89c9 Also fix configure.ac to use git describe --tags 2014-06-08 19:36:06 -05:00
Nicolas Williams ed2b3e26fb Fix scripts/version: use git describe --tags ...
After tagging as 1.4 scripts/version was still producing jq-1.3-....
2014-06-08 19:31:21 -05:00
Nicolas Williams 46436bd870 Add test of indices with short input 2014-06-08 02:27:27 -05:00
Nicolas Williams 1dbe9317bc Add indices(s), improve index(s), rindex(s)
Now these deal with arrays as input and `s` being an array or a scalar.
2014-06-08 02:01:44 -05:00
Nicolas Williams ef4f3a54fe Improve index and rindex examples 2014-06-08 02:01:21 -05:00
Nicolas Williams 9af03c2562 Remove reference to getpath from docs 2014-06-08 01:35:03 -05:00
Nicolas Williams c10e5f4e73 Document index and rindex (#389) 2014-06-08 00:00:17 -05:00
Santiago Lapresta 01f676041d Added join function
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2014-06-07 21:53:03 -05:00
Nicolas Williams 6856ab8a69 String * number should be commutative 2014-06-07 21:43:30 -05:00
Nicolas Williams 8212750fe9 Add cross-compilation notes to README
A detailed set of instruction as to how to setup a cross-compilation
environment for OS X and Win32/64 would be nice.
2014-06-04 18:36:30 -05:00
Nicolas Williams 35b6b949e3 Add -j option to scripts/crosscompile 2014-06-04 18:36:30 -05:00
Nicolas Williams d1ea3ab89d Add flags argument to jv_parser_new()
For extensibility.  We might add streaming parser options, even binary
JSON encoding options.
2014-06-04 18:35:30 -05:00
Nicolas Williams ae625d0de7 Revert "Add -I / --online-input for huge top-level arrays"
This reverts commit 77936a594d.

There are too many odd bugs in this mode, and it turns out to be a bad
idea anyways.  Instead, in the future a better option will be to pursue
alternative parsers, such as:

 - streaming parser that outputs only when a new leaf value is added or
   an array/object is opened/closed; options here include whether to
   include a path in each output;

 - parsers for binary JSON encodings (there's a variety of them).

Then one might run jq with a streaming parser and use `reduce` to
coalesce inputs from some depth down (instead of from one level down as
the reverted commit had intended).

Besides, a fully streaming parser is desirable in some cases, therefore
we should have such a thing as an option.

I've explored modifying the current parser to support a streaming
option, but it only makes the code very difficult to follow, which is
one reason that alternate parsers makes sense.  At any rate, this is all
for the future.  For now there's no streaming of individual texts, just
text sequences.
2014-06-04 18:15:58 -05:00
Nicolas Williams 5fb82d1018 Fix tests failures on Windows
And Solaris 8 and 9 too, no doubt.  The problem was that non-standard
vsnprintf()s that return -1 when the buffer is too small were not
properly supported.
2014-06-02 00:56:01 -05:00
Nicolas Williams f9d1b94496 Update .gitignore and config/.gitignore 2014-06-01 22:32:53 -05:00
Nico Williams 854ff5643a Merge pull request #367 from slapresta/del-docs
Documented `del` command
2014-05-20 22:11:01 -05:00
Santiago Lapresta 072ebb911b Documented del command 2014-05-21 01:23:28 +02:00
Nico Williams d46f5d29fa Merge pull request #360 from slapresta/275-unique-by
unique_by(.foo) function
2014-05-12 00:30:55 -05:00
Santiago Lapresta 69aa0003af Added texts/examples to unique_by function 2014-05-12 01:44:53 +02:00
Santiago Lapresta 41a093c1f2 Added unique_by function 2014-05-12 01:37:27 +02:00
Nicolas Williams 022a418894 Make pthread tls configurable for Mingw build
For the Mingw build we don't want to pull in the pthread DLL just
because we can autodetect pthread support.  That would make the jq.exe
binary not self-contained.
2014-04-17 20:17:06 -05:00
Nicolas Williams d056668f4f Add autoconf checks for pthreads; fix #340 2014-04-16 18:42:36 -05:00
Nico Williams 917334d18a Merge pull request #332 from jingweno/master
Add link to jq playground
2014-03-20 23:47:40 -05:00
Jingwen Owen Ou 7a83f1c416 Add link to jqplay 2014-03-20 15:49:59 -07:00
Nico Williams 6f0a9521e1 Merge pull request #176 from pearkes/new-tutorial
Docs: Update the tutorial to use GitHub's API
2014-03-13 00:57:54 -05:00
Nicolas Williams aee5c3ecb0 Fix for #303 in the sources 2014-03-13 00:57:33 -05:00
Nico Williams da0d575649 Merge pull request #278 from polyester/master
work with newer versions of automake
2014-03-13 00:52:58 -05:00
Nicolas Williams 74d2216da2 Test arrays and friend filters 2014-03-13 00:29:53 -05:00
Nicolas Williams 769314a8a2 Document scalars 2014-03-13 00:29:53 -05:00
Nicolas Williams 831be04027 Also add scalars 2014-03-13 00:29:53 -05:00
Nicolas Williams 1a1adf73e8 Document arrays, objects, ... 2014-03-13 00:29:53 -05:00
Santiago Lapresta 68f41097b9 Added arrays and other filters
Arrays, objects, numbers, strings, booleans, nulls, values (non-nulls)
-- these builtins filter out those inputs that don't match the name of
the builtin.

This fixes #322 and #324.
2014-03-13 00:27:56 -05:00
Nicolas Williams 4402c6d806 One more recursive merge test 2014-03-12 23:44:21 -05:00
Filippo Valsorda 2aa8a43c5b Add a recursive object merge strategy and bind it to *
This commit adds a jv_object_merge_recursive function, that performs
recursive object merging, and binds it to multiply when applied to
two objects.

Added docs and tests.

Closes #320
2014-03-08 03:56:05 +01:00
Nicolas Williams 219bb9ecb7 Make libm tests more portable 2014-03-06 21:38:10 -06:00
Nico Williams 88fd2ee4f2 Merge pull request #310 from arodland/master
Make jq --raw-output --unbuffered work
2014-03-03 13:46:38 -06:00
Andrew Rodland 6c3683dfe9 Repair jv_show 2014-02-26 01:43:00 -06:00
Andrew Rodland 36e495da1e Make jq --raw-output --unbuffered work
--unbuffered was only affecting the normal output case, not the --raw-output case. Make the two of them play together.

This also makes sure that the output is flushed *after* printing the newline, so a consumer doesn't lag a line behind.
2014-02-26 01:42:29 -06:00
Nicolas Williams 15faf9dc3b Add cbrt (cube root) 2014-02-21 10:51:16 -06:00
Nicolas Williams 59302fd208 Add missing trig functions and barebones test 2014-02-21 10:49:21 -06:00
Nicolas Williams 24202d1ecf Remove non-standard exp10() 2014-02-21 10:49:10 -06:00
Mike McCabe 336f4e27e6 Initial add of math functions. 2014-02-21 08:46:56 +00:00
Nicolas Williams fe29d3d3fa Add ?, .[]?, and .. operators
Make XPath-like `//a/b` recursive structure traversal easier in jq,
which then becomes:

    ..|.a?.b?

The `?` operator suppresses errors about . not being an array or object.
The `..` operator is equivalent to calling the new `recurse_down`
built-in, which in turn is equivalent to

    recurse(.[]?)

Note that `..a` is not supported; neither is `...a`.  That could be add
added, but it doesn't seem worth the trouble of saving the need to type
a '|'.
2014-02-20 15:33:07 -06:00
Nico Williams 3d33412e9a Merge pull request #300 from lcd047/master
Fix building of the manual
2014-02-19 11:21:04 -06:00
LCD 47 fb9196021e manual.yml: fix indent for the new any and all commands. 2014-02-19 15:10:32 +02:00
Nico Williams b23fae9ee3 Merge pull request #296 from slapresta/all-any
Added `all` and `any` builtins
2014-02-16 22:45:49 -06:00
Santiago Lapresta 8b41415b07 Added all and any builtins 2014-02-17 04:01:32 +01:00
Nicolas Williams 1fa55a3fae Fix cut-n-paste in leaf_paths; doc and test 'em 2014-02-05 23:14:05 -06:00
Nicolas Williams a45c937f80 #285 add and builtins 2014-02-05 15:23:32 -06:00
polyester 6ed3adb091 work with newer versions of automake
when using a newer automake, the autoreconf step fails with warnings:
"linking libtool libraries using a non-POSIX archiver requires 'AM_PROG_AR' in 'configure.ac' "

This happens for instance on ubuntu 13.10. 
Doing just that, adding 'AM_PROG_AR' to configure.ac fixes the problem.
2014-01-25 18:55:58 +01:00
Nicolas Williams 106fdf5114 Fix #201; check that bison accepts --warnings 2014-01-01 23:10:00 -06:00
Nicolas Williams 99f170adc3 Fix version for make dist on master
- Re-remove VERSION file and all traces of it
 - Pass a decent version indicator to AC_INIT (using git describe)

When we tag 1.4 this will produce 1.4 as the version string when
building on the HEAD of that tag.
2013-12-31 17:19:40 -06:00
Joe Littlejohn f60d83c15f Fix rpm build (make rpm)
* Re-add VERSION as it's required for `./setup superclean`
  and `make rpm`.
* Add *.rpm to git ignore, we never want them under version control.
2013-12-27 13:01:14 -06:00
Filippo Giunchedi 36a47bae56 include additional files in jq.spec
this will probably need changing upon SONAME bump
2013-12-27 12:49:15 -06:00
Filippo Giunchedi dc8528af0f fix rpm Makefile target and prerequisites
depend on dist and the specfile, plus use automake's variables
2013-12-27 12:49:15 -06:00
Nicolas Williams e12d0ffd1e Document --version 2013-12-26 18:48:51 -06:00
Nicolas Williams a71138a43a Add jv_dumpf() and jv_show()
jv_dumpf() takes a FILE *.

jv_show() is intended for use in debuggers, so it dumps the jv to stderr
and it does not jv_free() the jv, so it's safe to
"call jv_show(some_jv, -1)" in a debugger.  If flags == -1 then the jv
will be shown pretty-printed and in color.
2013-12-26 18:40:34 -06:00
Nicolas Williams cb976b9a50 Document .foo.bar in manual 2013-12-26 18:25:31 -06:00
Nicolas Williams 060eb39bf9 Document exit numbers 2013-12-26 12:54:07 -06:00
Nicolas Williams ab363e9e50 Normalize errors for -e 2013-12-26 12:54:03 -06:00
Nicolas Williams 41a8ae7557 Fix doc typos (.[foo] wanted to be .["foo"]) 2013-12-25 17:09:27 -06:00
Nicolas Williams 1956abbedf Add note to jq.1 about shell quoting 2013-12-25 17:05:32 -06:00
Stephen Dolan 2003a0440f Merge pull request #251 from phihag/fix-uri-in-manual
Fix @uri example
2013-12-21 11:47:24 -08:00
Stephen Dolan a0274cc46e Merge pull request #252 from phihag/ignore-config-test-driver
Ignore the config/test-driver file
2013-12-21 11:46:53 -08:00
Philipp Hagemeister 3b384740b9 Ignore the config/test-driver file
This file is automatically generated and does not need to be committed.
2013-12-20 11:36:52 +01:00
Philipp Hagemeister 4d40a47eab Fix @uri example
Previously, the @uri example didn't match the actual behavior of the current jq, as exclamation marks do not need to be encoded in URIs.
Replace the example with an input that needs encoding, and is encoded by jq.
2013-12-20 11:32:53 +01:00
Stephen Dolan 18e828f17c Allow negated object values without parens. Fixes #247 2013-12-17 22:28:31 +00:00
Nicolas Williams 0966230ba8 Fix memmem() error 2013-12-17 08:40:03 -06:00
Stephen Dolan 669f4ab4c3 Merge pull request #242 from sieben/travis-ci
Adding a .travis.yml file to use the travis-ci.org
2013-12-15 12:54:17 -08:00
Rémy Léone 3455714410 Adding a .travis.yml file to use the travis-ci.org
From wikipedia:

Travis CI is a hosted, distributed continuous integration service used
to build and test projects hosted at GitHub.

Travis CI is configured by adding a file named .travis.yml, which is a
YAML format text file, to the root directory of the GitHub repository.

Travis CI automatically detects when a commit has been made and pushed
to a GitHub repository that is using Travis CI, and each time this
happens, it will try to build the project and run tests. This includes
commits to all branches, not just to the master branch. When that
process has completed, it will notify a developer in the way it has been
configured to do so — for example, by sending an email containing the
test results (showing success or failure), or by posting a message on an
IRC channel. It can be configured to run the tests on a range of
different machines, with different software installed (such as older
versions of a programming language, to test for compatibility).
2013-12-13 17:14:37 +01:00
Stephen Dolan 32ea50d336 Make the testsuite run on machines without valgrind 2013-12-13 15:22:17 +00:00
Stephen Dolan 5ac6a460fa Format more integers as integers, not scientific notation.
jq is now willing to put up to 15 zeros after an integer before
moving to scientific notation.
2013-12-13 15:22:17 +00:00
Stephen Dolan f147b7fae5 Merge pull request #239 from DRMacIver/overflows
Avoid undefined behaviour with large array indices
2013-12-12 16:38:14 -08:00
Nicolas Williams 474386a1d9 Complete more-arity feature not complete
And test
2013-12-11 14:14:26 -06:00
Stephen Dolan 7fd2c2d08b Merge pull request #235 from DRMacIver/prototypes
some functions were missing proper prototypes
2013-12-10 03:56:02 -08:00
David R. MacIver 5412f3f1c1 convert range bounds to integers in a way that avoids undefined behaviour 2013-12-10 09:20:13 +00:00
David R. MacIver 014b45b4a9 add checking of numeric indices to an array to see if they can reasonably be considered integers. Avoid undefined behaviour if out of bounds 2013-12-10 09:20:12 +00:00
David R. MacIver 30ea390b77 some functions were missing prototypes. Add them 2013-12-10 00:07:08 +00:00
Stephen Dolan 00f244385b Merge pull request #232 from DRMacIver/teardown_on_error
consistent use of goto out in main
2013-12-08 16:41:16 -08:00
David R. MacIver 7b671b95d2 don't need to check for NULL there 2013-12-08 23:52:07 +00:00
Stephen Dolan 1535f23455 Merge pull request #233 from DRMacIver/printf
Fix warnings about vprintf
2013-12-08 15:44:33 -08:00
Stephen Dolan 4ba0c8baef Merge pull request #231 from DRMacIver/fix_mem_leak_in_compile
args to jq_compile_args were not getting freed when there were errors in the compile
2013-12-08 15:17:20 -08:00
Stephen Dolan a7dd3ab793 Clean up string/object interactions in jv. 2013-12-08 22:49:28 +00:00
David R. MacIver f6f7d4d973 These vfprintfs are being used as if they were printfs. Fix that 2013-12-08 20:08:04 +00:00
David R. MacIver f4ae5c883b consistent use of goto out in main 2013-12-08 19:22:18 +00:00
Stephen Dolan 74a14f5de7 Refactor jv structure.
New structure layout is simpler and also faster.  In particular, it's
now small enough to be passed in registers on amd64.
2013-12-08 17:46:23 +00:00
Stephen Dolan fee21c959a Make testsuite not leak when compiled with -DNDEBUG. 2013-12-08 17:44:54 +00:00
David R. MacIver 4b48054f41 test for losing memory on compile errors 2013-12-08 16:19:12 +00:00
David R. MacIver 17a319d120 args to jq_compile_args were not getting freed when there were errors in the compile 2013-12-08 15:28:29 +00:00
Nicolas Williams 3a1dab5396 Fix double-free typo in print_error() 2013-12-06 23:25:06 -06:00
Nicolas Williams dac081e183 Fix manual.yml 2013-12-06 11:58:55 -06:00
Stephen Dolan 489fcdc089 Merge pull request #224 from nicowilliams/features
Features: more arity, string ops
2013-12-04 17:36:08 -08:00
Nicolas Williams 3ba7c97b2a Add self to AUTHORS 2013-12-04 18:21:42 -06:00
Nicolas Williams 54635000c5 Conditionally #define _GNU_SOURCE in compile.c 2013-12-04 18:21:42 -06:00
Nicolas Williams 426edff09c Add tests for string index by string and builtins 2013-12-04 18:21:42 -06:00
Nicolas Williams b37ce20c2a Add index and rindex builtins 2013-12-04 18:21:42 -06:00
Nicolas Williams cac14a531d Add index strings by string; return string indexes
% jq '.[","]'
    "a,bc,def,ghij,klmno"
    [1,4,8,13]
    %
2013-12-04 18:21:42 -06:00
Nicolas Williams ae7a042876 Make length return abs value of numeric inputs 2013-12-04 18:21:42 -06:00
Nicolas Williams eb165459aa Add callback interface for errors
Printing to stderr is not the right answer for a library.
2013-12-04 18:21:41 -06:00
Nicolas Williams 04bc2ef7cf Add jv_string_vfmt() 2013-12-04 18:21:41 -06:00
Nicolas Williams 09104ce531 Document ltrimstr and rtrimstr 2013-12-04 18:21:41 -06:00
Nicolas Williams 9fa2173825 Test ltrimstr and rtrimstr functions 2013-12-04 18:21:41 -06:00
Nicolas Williams 346da34432 Add ltrimstr and rtrimstr functions 2013-12-04 18:21:41 -06:00
Nicolas Williams fa316ac430 Document -u / --unslurp option 2013-12-04 18:21:41 -06:00
Nicolas Williams 77936a594d Add -I / --online-input for huge top-level arrays 2013-12-04 18:21:41 -06:00
Nicolas Williams 4e07eac356 Document -e / --exit-status argument 2013-12-04 18:21:40 -06:00
Nicolas Williams d018e43ddc Add -e | --exit-status CLI option 2013-12-04 18:21:40 -06:00
Nicolas Williams 8cee66e609 Document tojson and fromjson builtins 2013-12-04 18:21:40 -06:00
Nicolas Williams 3c4248192d Test tojson and fromjson 2013-12-04 18:21:40 -06:00
Nicolas Williams 255f8ef5be Add tojson and fromjson builtins 2013-12-04 18:21:40 -06:00
Nicolas Williams 20b32db5af Document split function 2013-12-04 18:21:40 -06:00
Nicolas Williams b634722c8a Document string multiplication and division 2013-12-04 18:21:40 -06:00
Nicolas Williams c9e8581f45 Document string functions and slicing 2013-12-04 18:21:39 -06:00
Nicolas Williams da129f6664 Test string slicing 2013-12-04 18:21:39 -06:00
Nicolas Williams 884e6c7d8b Add string slicing 2013-12-04 18:21:39 -06:00
Nicolas Williams 5aadaa79eb Add tests for string division/splitting 2013-12-04 18:21:39 -06:00
Nicolas Williams cf562961b7 Add string division by string (split on separator) 2013-12-04 18:21:39 -06:00
Nicolas Williams 63bed9bdf1 Test starts/endswith and string multiplication 2013-12-04 18:21:39 -06:00
Nicolas Williams 5989dbdfcf Add string multiplication by number 2013-12-04 18:21:39 -06:00
Nicolas Williams db19a11399 Add startswith/endswith 2013-12-04 18:21:38 -06:00
Nicolas Williams 6c734cb609 Add explode/implode jq functions to match jv API 2013-12-04 18:21:38 -06:00
Nicolas Williams ef41d1cb57 Use uint32_t for codepoint in jv_string_append_codepoint() 2013-12-04 18:21:38 -06:00
Nicolas Williams 08b4a34a8e Add jv string utility functions
jv_string_empty()
        -> return an empty string with given allocated length (for fast
           appends)
    jv_string_append_codepoint
        -> append a single codepoint (int) to the given string
    jv_string_explode
        -> return an array of codepoints making up a string
    jv_string_implode
        -> return the UTF-8 encoding of an array of codepoint numbers
2013-12-04 18:21:38 -06:00
Nicolas Williams 1fc5656762 Support more arguments for defs 2013-12-04 18:21:38 -06:00
Stephen Dolan 91cfc34ae4 Preserve insertion order in objects. Closes #169. 2013-12-04 22:30:39 +00:00
Nicolas Pouillard a0b942a7f5 Add a few more test cases (from the man page)
Signed-off-by: Nicolas Williams <nico@cryptonector.com>
2013-11-30 00:55:26 -06:00
Stephen Dolan 6a401c8262 Add a --unbuffered option. Closes #206 2013-11-08 12:21:45 +00:00
Stephen Dolan d235f32bc9 Merge pull request #210 from Habbie/patch-1
count should be length
2013-11-08 03:43:57 -08:00
Peter van Dijk bdbfa7e32c count should be length
Example refers to a count function, which does not exist. Replacing it with length works.
2013-11-07 16:39:44 +01:00
Stephen Dolan 78976b3504 Merge pull request #193 from ryo1kato/fix-docs
Docs: add description of --from-file option
2013-11-07 04:41:01 -08:00
Stephen Dolan 8d4748c8df Fix a crash on group_by of empty list. Fixes #208. 2013-11-07 12:18:48 +00:00
Ryoichi KATO da723d3e11 Docs: add description of --from-file option 2013-10-16 21:42:29 +09:00
Stephen Dolan 0feeb7d0d4 Merge pull request #188 from Juanxo/patch-1
Fix typo on error message
2013-10-07 06:59:50 -07:00
Juan Guerrero 9e53f36a74 Fix typo on error message 2013-10-06 17:52:04 +02:00
Stephen Dolan 7e7c08da47 Merge pull request #185 from qoelet/master
Add missing -i flag in build instructions
2013-09-20 06:05:42 -07:00
Kenny Shen 2f6b89ef70 Add missing -i flag in build instructions 2013-09-20 10:34:57 +08:00
Stephen Dolan 4e3024cb61 Merge pull request #182 from mdaines/sqrt-builtin
Add builtin for calculating square root
2013-09-14 16:34:52 -07:00
Michael Daines c427bf6a94 Add test showing calculation of standard deviation 2013-09-14 17:48:19 -04:00
Stephen Dolan dae2422fd1 Merge branch 'libjq'
Conflicts:
	Makefile.am
	configure.ac
2013-09-14 19:30:39 +01:00
Stephen Dolan f98e3e6c26 Merge pull request #181 from mdaines/patch-1
Fix typo
2013-09-13 15:24:22 -07:00
Mike Daines fe99532f8b Fix typo 2013-09-13 12:15:52 -04:00
Stephen Dolan 41a4968c90 Merge pull request #171 from joelittlejohn/fix-makefile
Fix Makefile after refactoring of stacks in 05d90517b0
2013-09-13 09:02:42 -07:00
Stephen Dolan 1cdaabf2dd Merge pull request #175 from lipidity/fix-build
Call AM_INIT_AUTOMAKE once only
2013-09-13 09:02:24 -07:00
Michael Daines 82d8253c19 Add sqrt operator 2013-09-11 20:24:41 -04:00
Jack Pearkes c1717d6e12 docs: update the tutorial to use GitHub's API 2013-09-04 11:42:40 +02:00
Ankur f0d1c39b5b Call AM_INIT_AUTOMAKE once only
Fixes build with automake-1.14
2013-09-01 18:58:26 +10:00
Joe Littlejohn 111aea3f0a Fix Makefile after refactoring of stacks in 05d90517b0 2013-08-19 13:40:08 +01:00
Stephen Dolan 37cfc912c1 Remove #includes from jv.h 2013-06-23 14:23:07 +01:00
Stephen Dolan 8c2e228c74 Fix the jv_parser interface. 2013-06-23 12:26:49 +01:00
Stephen Dolan 8630fb02bb Use libtool's built-in symbol exporting rather than a mapfile. 2013-06-23 12:10:58 +01:00
Stephen Dolan 5052ab4e2b Move gen_utf8_tables to scripts 2013-06-23 12:08:22 +01:00
Stephen Dolan a44f739356 Move libtool m4 junk to config/ and delete some autogenerated files. 2013-06-23 12:01:58 +01:00
Stephen Dolan f44ecf3872 Remove Autoconf-generated config.h. 2013-06-23 12:01:58 +01:00
Stephen Dolan 1c9e03f800 Merge branch 'header-cleanup' into libjq
Conflicts:
	Makefile.am
2013-06-22 23:27:16 +01:00
Stephen Dolan 5d9ec83805 Build libjq only once, and link it statically to ./jq
This means ./jq is a real binary rather than a libtool turd.
2013-06-22 20:53:28 +01:00
Stephen Dolan 3a295c659b Fix distcheck.
Update list of files to be distributed.
2013-06-22 20:53:03 +01:00
Stephen Dolan d253414783 Merge pull request #162 from stedolan/utf8-fixes
Utf8 fixes. Closes #161
2013-06-22 09:30:59 -07:00
Stephen Dolan 8b1b503609 Reject all overlong UTF8 sequences. 2013-06-22 13:43:46 +01:00
Stephen Dolan ff48bd6ec5 Fix various UTF8 parsing bugs.
In particular, parse bad UTF8 by replacing the broken bits with U+FFFD
and resychronise correctly after broken sequences.
2013-06-22 13:34:24 +01:00
Stephen Dolan 54b9c9bdb2 Fix example in manual for floor. See #155. 2013-06-22 12:17:28 +01:00
Nicolas Williams 3dcdc582ea Document floor 2013-06-21 15:27:34 -05:00
Nicolas Williams 7c4171d414 Add floor operator 2013-06-21 15:27:34 -05:00
Nicolas Williams 1d839ff9df Document mod 2013-06-21 15:27:34 -05:00
Nicolas Williams 3403d07912 Add mod (and setmod) operators 2013-06-21 15:27:34 -05:00
Nicolas Williams e8643787cf Update .gitignore 2013-06-21 12:41:57 -05:00
Nicolas Williams 298b2a6033 Add libjq autoconf goo 2013-06-21 11:57:12 -05:00
Nicolas Williams c791428905 Quiet setup.sh re: tmp dir 2013-06-21 11:57:11 -05:00
Stephen Dolan dc2fe63e42 Merge pull request #157 from nicowilliams/argfile-fixes
Argfile fixes
2013-06-21 09:40:32 -07:00
Stephen Dolan 7af88962ee Move cfunction invocation code to the interpreter loop. 2013-06-21 12:06:28 +01:00
Nicolas Williams 3216d7683a Fix serious bug in handling of --argfile 2013-06-18 19:05:52 -05:00
Nicolas Williams e4094c8664 Fix leaks in jv_load_file() 2013-06-18 19:05:52 -05:00
Stephen Dolan b49d65a276 Fold opcode.{c,h} into bytecode.{c,h} 2013-06-18 01:36:18 +01:00
Stephen Dolan 824ce76cec Simplify block functions for variables 2013-06-18 01:07:18 +01:00
Stephen Dolan 2a18f97d12 Saner build instructions in README.md
Closes #144
2013-06-18 00:29:05 +01:00
Stephen Dolan 8c4d29ee38 Remove some initialise-to-zero code.
This lets valgrind find more bugs - if a field isn't given a
well-defined value valgrind will now find it instead of seeing it
set to zero with memset.
2013-06-18 00:17:20 +01:00
Stephen Dolan 2c13f4fe72 Merge branch 'stack-refactor-merge' 2013-06-18 00:13:44 +01:00
Stephen Dolan 94931040a8 Merge branch 'stack-refactor'
Conflicts:
	execute.c
2013-06-18 00:06:00 +01:00
Stephen Dolan b78a4cdc93 Merge pull request #154 from nicowilliams/no-fopen-re
Remove accidentally introduced use of fopen "e"
2013-06-17 15:34:01 -07:00
Nicolas Williams da09544bb0 Remove accidentally introduced use of fopen "e" 2013-06-17 17:27:25 -05:00
Stephen Dolan ce17b6bd53 Merge pull request #114 from nicowilliams/nomem_handler
Add jv_nomem_handler()
2013-06-16 06:20:27 -07:00
Nicolas Williams fe03bea191 Remove last remnant of main.h 2013-06-16 08:06:52 -05:00
Nicolas Williams 0ed6347d47 Allow --run-tests to take a file argument 2013-06-15 17:37:15 -05:00
Nicolas Williams 3f86e97f70 Fixup API to get closer to a libjq 2013-06-15 17:37:15 -05:00
Stephen Dolan 69fa279e3c Merge pull request #150 from nicowilliams/jv_load_file
Move slurp_file() into library as jv_load_file()
2013-06-15 05:29:17 -07:00
Nicolas Williams bbf4e05fde Move slurp_file() into library as jv_load_file()
Needed as part of creating a libjq.
2013-06-15 00:08:59 -05:00
Stephen Dolan 05d90517b0 Clean up lots of stack and frame logic.
Move frame defs to execute.c
2013-06-14 22:08:18 +01:00
Stephen Dolan 5f5c1dc5a6 Simplify frame logic. 2013-06-14 01:20:24 +01:00
Stephen Dolan e0524644f8 Unify all stacks. Passes tests, but needs cleanup. 2013-06-13 21:50:32 +01:00
Stephen Dolan 81e2336ace Support ."foo" syntax for accessing fields. See #141. 2013-06-11 22:26:39 +01:00
Stephen Dolan 25bf1a0169 Unify frame and data stacks 2013-06-10 01:17:58 +01:00
Stephen Dolan c8ab67be27 Speed up cached configure (./configure -C) 2013-06-05 16:40:01 +01:00
Stephen Dolan 6c34288ebf Clean up flex lines in build 2013-06-05 13:43:23 +01:00
Stephen Dolan 2ffbb9d6b7 Lex and parse .foo better.
'.as' is now valid, '. foo' is now invalid. See #141.
2013-06-05 13:41:56 +01:00
Stephen Dolan dad608d5c0 Merge pull request #142 from lanthaler/master
Update README.md
2013-06-04 04:13:24 -07:00
Markus Lanthaler 04b939625b Update README.md
Update the link to the documentation. All GitHub pages are now using the github.io domain.
2013-06-04 13:04:42 +02:00
Stephen Dolan 5dac485458 Make jq --version print to stdout, not stderr 2013-06-03 23:22:59 +01:00
Stephen Dolan 2c4ccd122e Better error handling for .foo case in parser. See #141. 2013-06-03 23:12:31 +01:00
Stephen Dolan b90ae078b8 Let the parser rather than the lexer handle invalid characters. 2013-06-03 23:12:31 +01:00
Stephen Dolan cc8761dbde Add command-line option to sort object keys.
Closes #79.
2013-06-03 23:12:31 +01:00
Stephen Dolan 08cc591884 Clean up Makefile.am (distcheck, rebuild version.h less often) 2013-06-03 23:12:31 +01:00
Stephen Dolan 1bc4d04f43 Merge pull request #139 from 13ren/stopwarning
Stop warning on fgets, jq_test.c:42
2013-06-01 07:30:15 -07:00
Brendan Macmillan d4018192a9 Stop warning on fgets, simple version 2013-06-01 11:12:28 +10:00
Brendan Macmillan 19ddbfeca9 Stop warning on fgets, complex version 2013-06-01 11:03:15 +10:00
Stephen Dolan 4fed6763fc Squash a warning on some GCC versions 2013-05-31 12:15:58 +01:00
Stephen Dolan 73446a9cce Support for printing object keys in sorted order.
No command-line option to enable this yet. See #79.
2013-05-29 11:47:12 +01:00
Stephen Dolan dcf1ac0d1f Merge pull request #135 from 13ren/locfile_bugfix
Locfile bugfix
2013-05-29 03:38:15 -07:00
Stephen Dolan 364c58d13f Merge pull request #130 from 13ren/libload_2
libload_2 (update to pull request #126, for issue  #112)
2013-05-29 03:37:36 -07:00
Brendan Macmillan 8cfe11e4e2 Bugfix multiline off-by-one (locfile.c) 2013-05-29 15:23:08 +10:00
Brendan Macmillan 40378a18f7 locfile.h -> locfile.h + locfile.c
clean up includes of a few files
2013-05-29 15:17:01 +10:00
Brendan Macmillan 7fb5d8c973 Hack bugfix for multiline off-by-one (locfile.c) 2013-05-29 15:16:45 +10:00
Brendan Macmillan 6e373942e5 Load library from ~/.jq 2013-05-29 15:05:40 +10:00
Stephen Dolan f0b982795c Make jq --version report an actual git revision.
Closes #129.
2013-05-24 23:35:38 +01:00
Stephen Dolan cbd422d56e Merge pull request #133 from nicowilliams/option_argfile-rebase0
Add --argfile variant of --arg (issue #117)
2013-05-24 04:13:45 -07:00
Nicolas Williams 03598bb5d6 Add --argfile variant of --arg (issue #117)
This is useful when one has a database (in JSON form) to query using jq
input data.

    % echo '{"a":1, "c":5}' > db.json
    % echo '"c"'|./jq --argfile f /tmp/a '$f[.]'
    5
    % echo '"a"'|./jq --argfile f /tmp/a '$f[.]'
    1
    % echo '"b"'|./jq --argfile f /tmp/a '$f[.]'
    null
    %
2013-05-23 13:52:06 -05:00
Stephen Dolan 983a53a643 'make clean' won't delete jq.1 if it can't be rebuilt.
See #131
2013-05-23 19:20:19 +01:00
Stephen Dolan 0535e724a2 jq 1.3 release 2013-05-19 11:48:18 +01:00
Stephen Dolan fd1ac5dd79 EACH need not make a backtrack point on the last iteration 2013-05-17 22:49:08 +01:00
Stephen Dolan 9b91eafac4 Make array construction use LOADVN (slightly faster) 2013-05-16 15:09:59 +01:00
Stephen Dolan bc42812715 Remove the insane "fold" operation, replace with saner "reduce". 2013-05-16 15:07:53 +01:00
Stephen Dolan 8fbee891b3 Add LOADVN opcode.
Does a variable load, but sets the variable to be null afterwards.
2013-05-16 15:02:18 +01:00
Stephen Dolan 919292764a Reverse function. Closes #94. 2013-05-16 14:25:28 +01:00
Stephen Dolan a1e4dfa324 Better debug info for struct bytecode.
--debug-dump-disasm produces more comprehensible output now.
2013-05-16 00:51:17 +01:00
Stephen Dolan f8644c31a3 Only generate code for those builtin functions actually used.
Makes output of --debug-dump-disasm much simpler.
2013-05-15 01:37:15 +01:00
Stephen Dolan 1e2851cdb3 Remove the YIELD opcode (use RET instead) 2013-05-15 01:23:06 +01:00
Stephen Dolan e83e51eb56 'length' function now measures string length in codepoints, not bytes. 2013-05-15 00:37:38 +01:00
Stephen Dolan c496a924ce Bugfix for array slices. 2013-05-15 00:22:44 +01:00
Stephen Dolan 81be37b236 Add the range function 2013-05-14 16:09:10 +01:00
Stephen Dolan 4d540b7102 Fix some documentation bugs 2013-05-13 20:26:44 +01:00
Stephen Dolan 33901b74b1 Array slicing. Closes #2. 2013-05-13 20:16:19 +01:00
Stephen Dolan a47cfa4757 Delete unnecessary _POSIX_SOURCE. Closes #89 2013-05-13 16:41:11 +01:00
Stephen Dolan a625d2821e Add to_entries, from_entries and with_entries functions.
Closes #97.
2013-05-13 16:15:49 +01:00
Stephen Dolan d8706fd460 Fix the //= operator.
Closes #62.
2013-05-13 16:04:30 +01:00
Stephen Dolan e13f24ab45 Better error handling for INSERT opcode
jq 'null | {(.a): 1}' no longer crashes.
2013-05-13 15:42:57 +01:00
Stephen Dolan 34a63246ab Remove the is_backtrack_frame special case hack. 2013-05-13 15:37:57 +01:00
Stephen Dolan 8c708f3c7a Refactor path logic. 2013-05-13 15:00:05 +01:00
Stephen Dolan b0e65d149f Fix a bug triggered by jv_array_slice. 2013-05-13 14:10:39 +01:00
Stephen Dolan e29db8d272 Fix bug with path handling (used in assignments).
Closes #67
2013-05-11 18:10:21 +01:00
Stephen Dolan 7ca5127fcc Merge branch 'autotools' 2013-05-11 15:21:23 +01:00
Stephen Dolan 4b1b9c219a Delete some Autotools junk and update README.
Autotools autogenerated files that change frequently have been
removed, those that don't are still checked in.
2013-05-11 15:15:56 +01:00
Stephen Dolan 4a5405e4b8 Rake task to build website including source dist and binaries. 2013-05-11 14:57:58 +01:00
Stephen Dolan d03149f4df Rake task to build source distribution for website 2013-05-11 14:48:39 +01:00
Stephen Dolan 18cf4ad475 Add the docs to the distributed tarballs 2013-05-09 20:53:51 +01:00
Stephen Dolan ba4bb136bb Fix manpage build. 2013-05-09 20:43:26 +01:00
Stephen Dolan e6494857fc Add some build instructions for how to install doc dependencies.
jq can now build without Ruby, but you won't get a nice manpage.
2013-05-09 20:35:19 +01:00
Stephen Dolan 12d2d4d1f5 Don't build the manpage unless there's a working Ruby setup. 2013-05-09 15:22:38 +01:00
Stephen Dolan d75414bc94 support for out-of-tree tests 2013-05-09 13:40:00 +01:00
Stephen Dolan a592c98461 Fail properly when attempting to build manpage without rake 2013-05-09 13:10:09 +01:00
Stephen Dolan f6089a627a Autotools hackery so that "make dist" builds lexer and parser. 2013-05-09 11:31:17 +01:00
Stephen Dolan 898dc8978b Add a redundant intialisation to squash a gcc warning. 2013-05-09 11:30:49 +01:00
Stephen Dolan d43084086b Make manpage building more robust. 2013-05-09 02:38:57 +01:00
Stephen Dolan 124bda6993 Add a "binaries" target to the website Rakefile. 2013-05-09 02:18:28 +01:00
Stephen Dolan 7fbc50239c Minor docs fix to get rid of Bonsai warnings. 2013-05-09 02:01:01 +01:00
Stephen Dolan 282667bbd4 Script for cross-compiling jq binaries for other platforms. 2013-05-09 01:50:20 +01:00
Stephen Dolan 63b689bf54 Use the version number from configure.ac in the website. 2013-05-08 19:57:29 +01:00
Stephen Dolan c710f91257 Add the parser to BUILT_SOURCES 2013-05-08 19:57:05 +01:00
Stephen Dolan a511d03d85 Configure whether colour is displayed by default directly in source 2013-05-08 19:37:12 +01:00
Stephen Dolan 9d245c88c4 Check in a pile of Autotools junk, including the configure script. 2013-05-08 18:55:10 +01:00
Stephen Dolan 7813f363a2 Merge pull request #115 from nicowilliams/complex
Rename complex to nontrivial for safety (#113)
2013-05-08 07:07:31 -07:00
Stephen Dolan d121420757 Clean up autotools config.
Remove some of @stagrlee's hard work since jq's testsuite is
less silly than when he wrote the autotools config.
2013-05-08 02:42:23 +01:00
Stephen Dolan 1ebb382540 Check in jv_utf8_tables.gen.h, it is unlikely to change. 2013-05-08 01:33:04 +01:00
Stephen Dolan 2363246eea Merge remote-tracking branch 'stagrlee/master' into autotools
Conflicts:
	Makefile
	jq_test.c
2013-05-08 00:52:24 +01:00
Nicolas Williams fb3c124af1 Rename complex to nontrivial for safety (#113) 2013-05-07 18:46:13 -05:00
Stephen Dolan 46814da91a Merge pull request #119 from 13ren/master
tiny typo fix in manual
2013-05-07 03:57:47 -07:00
Nicolas Williams 65deaaacf5 Fix #111: exit(1) when inputs are not valid JSON 2013-05-06 12:36:53 -05:00
Stephen Dolan 74eb914aba Add trailing slashes to various internal doc links.
Fixes #110
2013-05-06 15:03:14 +01:00
Stephen Dolan 5be97463ec Add a --arg option to allow variables to be passed from the cmdline.
Closes #107
2013-05-06 14:21:00 +01:00
Stephen Dolan 48be23233a Add the "has" function. Closes #74. 2013-05-06 13:46:54 +01:00
Stephen Dolan 88a6dc5343 Merge pull request #77 from jkleint/patch-1
Clarify description of `as`
2013-05-05 18:39:59 -07:00
Stephen Dolan fee38afd8b Document behaviour of .[] on objects.
Closes #57.
2013-05-06 02:37:34 +01:00
Stephen Dolan cacda19e75 Add documentation for "type".
Closes #75.
2013-05-06 02:34:02 +01:00
Stephen Dolan 7f979cf1c7 Update gems and use HTTPS to get them. 2013-05-06 01:15:33 +01:00
Stephen Dolan 8bf4a73f3d Fix some bad escaping 2013-05-06 01:14:53 +01:00
Stephen Dolan d04bd897e3 Merge commit 'e45f93f85f22c73025830a6bd0b2807263068996'
See #87
2013-05-05 23:58:24 +01:00
Stephen Dolan 1741d8c161 Remove JQ_DEBUG #define and jq_test binary, simplifying build.
The debugging features previously available via JQ_DEBUG are now
command-line options.
2013-05-05 23:12:10 +01:00
Stephen Dolan c1748fa633 WIP 2013-05-05 22:59:53 +01:00
Stephen Dolan a49402c53a Combine the functionality of jq and jq_test
One binary is much simpler. ./jq --run-tests now runs the tests.
2013-05-05 22:59:53 +01:00
Stephen Dolan 47e015e946 Make a few constant globals actually const.
This means the .data and .bss sections are empty, a good thing
in a shared library (see #98).
2013-05-05 22:59:53 +01:00
Stephen Dolan ab1a97472c Merge pull request #109 from nicowilliams/no_mutable_globals-rebased
Remove globals/make jq_init/next/teardown() an API
2013-05-05 12:29:07 -07:00
Stephen Dolan 74df082283 Merge pull request #93 from shayel/master
Allow manpage to build
2013-05-05 12:13:36 -07:00
Nicolas Williams a136c3b7c9 Build and install libjq 2013-04-28 20:12:49 -05:00
Nicolas Williams 312ab754a2 Remove unnecessary #includes 2013-04-28 20:12:15 -05:00
Nicolas Williams 87e9c64003 Remove globals/make jq_init/next/teardown() an API 2013-04-28 18:46:21 -05:00
Shay Elkin 054e3f3795 Add missing require to Rakefile
Use bundle exec to make manpage
Don't set executable bit on manpage
2013-03-24 14:45:10 +02:00
Simon Elsbrock e45f93f85f fix typo in manual 2013-02-20 22:38:50 +01:00
Lee Thompson dc4d3d8cbe get man pages working 2013-02-04 13:35:42 -06:00
Lee Thompson 1d3d2efc89 add man page to release 2013-02-04 07:01:18 -06:00
Lee Thompson 43fd939f16 rework build to not distribute tests unless --enable-devel is specificied to autoconf 2013-02-03 14:49:03 -06:00
Lee Thompson d8a072ca01 fix email addresses 2013-02-03 00:35:56 -06:00
Lee Thompson f62184855b use AM_SILENT_RULES 2013-02-03 00:09:47 -06:00
Lee Thompson 0370d5b526 fix RPM build process 2013-02-02 23:50:43 -06:00
Lee Thompson 52f906ab43 fix jq_test so that it runs in the distcheck phase of autoconf 2013-02-02 23:05:00 -06:00
Lee Thompson c7725a8d4d merging upstream stedolan changes 2013-02-02 20:39:23 -06:00
Lee Thompson dd70eeb29d use built in macro to RPM 2013-01-29 19:13:22 -06:00
jkleint 115552f3a6 Clarify description of as
It's more of a foreach loop than an identity operation.

Addresses issue #72.
2013-01-25 12:17:56 -05:00
13ren d0cd26cd8e Update docs/content/3.manual/manual.yml
There's a , after map ; should be .
#65
2013-01-13 17:33:01 +11:00
Stephen Dolan 925ec3751f Fix negative number syntax. Add a unary '-' operator.
Closes #63.
2013-01-03 12:53:23 +00:00
Stephen Dolan c013b557a2 Change APPEND opcode to directly modify a variable.
Avoids a big O(n^2) loop in constructing arrays. Fixes #61.
2013-01-03 12:51:33 +00:00
Stephen Dolan fb84541e11 Clean up jv_object_foreach and add jv_array_foreach 2012-12-31 23:27:00 +00:00
Stephen Dolan 38fe6fc0cb Ignore some more stuff. 2012-12-31 23:25:12 +00:00
Stephen Dolan b3bad59dd7 Add the recurse function. See #37. 2012-12-29 22:59:07 +00:00
Stephen Dolan e0cda536f3 Make null + foo return foo, rather than an error.
This also allows 'add' to be implemented in jq rather than C.
2012-12-29 16:52:26 +00:00
Stephen Dolan d5fdf70434 Refactor assignment.
New version is much more elegant and probably slower. Uses
library functions implemented in jq rather than dedicated opcodes.
2012-12-29 16:43:36 +00:00
Stephen Dolan 465a4ec565 Improvements to del(foo).
del(foo,bar) is now very different from del(foo),del(bar).
See #37.
2012-12-29 16:13:06 +00:00
Stephen Dolan 3a5377e183 Fix a bug in URI-encoding of certain non-ASCII characters.
Oh how I hate C casting rules.
2012-12-29 01:37:22 +00:00
Stephen Dolan 79c9c418c6 Path manipulation (path/getpath/setpath/delpath) and docs.
del function should fix #37.
2012-12-28 16:08:29 +00:00
Stephen Dolan f1e23448e7 Support for two-argument functions.
Should probably support n args instead. This is a quick hack.
Also, ';' is a slightly ugly argument separator.
2012-12-28 15:07:27 +00:00
Stephen Dolan 417899f9a0 Fold operation (code/docs/test) 2012-12-28 15:05:34 +00:00
Stephen Dolan 9302b16247 Docs typos. 2012-12-28 14:23:28 +00:00
Stephen Dolan 89e26969ae @foo syntax for encoding of strings into various formats.
Fixes part of #47 and #48.
2012-12-28 01:23:09 +00:00
Stephen Dolan 52db8000c6 Fix an embarassing I/O bug.
Pulling some I/O out to a function meant that buf changed from
"char buf[4096]" to "char* buf", and "sizeof(buf)" got a lot
less interesting. The upshot of this is that jq read input eight
bytes at a time, which is not the fastest.
2012-12-27 01:57:09 +00:00
Stephen Dolan d327772dfc JSON parsing error messages now specify a location of the error.
Should help with #53.
2012-12-27 01:56:23 +00:00
Stephen Dolan cedda2084d Sneaky valgrind trick to detect stack memory issues.
After something is popped from a stack, we overwrite the memory
with uninitialised data (if JQ_DEBUG is on). This means that
valgrind reports use-after-pop as an uninitialised memory error.
2012-12-24 17:11:18 +00:00
Stephen Dolan 0fed03f35a Fix a bad memory leak when using --raw-output 2012-12-23 15:41:20 +00:00
Stephen Dolan 5578ec0f69 Give Windows builds a proper .exe suffix. 2012-12-20 15:12:14 +00:00
Stephen Dolan a05904b226 Version 1.2! 2012-12-20 12:26:15 +00:00
Stephen Dolan c181a9185a Add a few lines to the bottom of the manpage. 2012-12-20 12:25:50 +00:00
Stephen Dolan 6a597b41a6 Update download page and build system to handle more platforms. 2012-12-20 12:17:56 +00:00
Stephen Dolan c85d8d8b4e Don't enable coloured output by default on Windows. 2012-12-20 12:16:22 +00:00
Stephen Dolan dcdb996085 Merge pull request #50 from stesh/master
Add information about installing dev environment on OS X
2012-12-20 03:41:19 -08:00
Stephen Dolan 5968f54f55 Fix a bug in stack reallocation during deep recursion. 2012-12-18 17:36:24 +00:00
Stephen Dolan 8dca3ef10d Print an error message and abort in out-of-memory situations.
Closes #43.

Tested with:

    ulimit -v 5000
    ./jq -n -c 'def f(x): x,f([x,x]); f(0)'
2012-12-18 17:01:23 +00:00
Stephen Dolan 04daafbde3 Add wrappers for malloc/realloc/free. See #43. 2012-12-18 16:53:51 +00:00
Stephen Dolan 75421cbfe3 Add the ability to specify input files on the commandline.
See #21.
2012-12-16 19:46:41 +00:00
Stephen Dolan b0aa62f2b7 Improve manpage introduction, add examples to manpage.
Closes #19
2012-12-16 17:11:40 +00:00
Stephen Dolan 3ae8fb7371 Fix a bug uncovered by tests extracted from manual.
`null | length` should give 0, and now does.
2012-12-16 13:10:48 +00:00
Stephen Dolan 73c657cdc4 Lots of build system and docs improvements, including a manpage.
- Build binaries for multiple platforms
 - Make a manpage out of the manual (see #19)
 - Extract more tests from the documentation
 - Fix a few documentation bugs uncovered by above.
2012-12-16 13:06:03 +00:00
Stephen Dolan 2b48ee4a8f Add a Gemfile to make building the documentation easier.
Use "bundle install" in the docs folder to install ruby dependencies.
2012-12-12 23:18:43 +00:00
Stephen Dolan 8ba2cef913 Delete autogenerated source from the repository.
This was a horrible hack to fix build issues on OS X some time
ago. There are better ways to fix build issues on OS X :)

See #49.
2012-12-10 22:36:01 +00:00
Stephen Dolan dcc93042d3 some words explaining struct inst a little 2012-12-10 22:35:33 +00:00
Stephen Dolan 34ff993059 error() function 2012-12-10 22:30:09 +00:00
Stephen Dolan b660bd2be6 Fix Rack setup so that viewing documentation locally is less painful. 2012-12-10 22:25:11 +00:00
Stephen Shaw 1db51e66bc Remove old OS X 'instructions' 2012-12-07 14:40:29 +00:00
Stephen Shaw c15e51ffb1 Add information about installing dev environment on OS X 2012-12-07 14:31:59 +00:00
Stephen Dolan 84026f0513 Change order of evaluation for certain indexing operations.
'.foo[.bar]' used to parse like '.foo | .bar as $b | .[$b]',
now it parses like '.bar as $b | .foo | .[$b]'.
2012-12-07 00:28:11 +00:00
Lee Thompson a156fb19de get testing working in autoconf 2012-12-04 19:50:10 -06:00
Lee Thompson ca862a594c kill prereq on RPM spec file 2012-12-04 19:00:59 -06:00
Lee Thompson dfa9e61aeb fixes jq_test core dump 2012-12-04 18:12:53 -06:00
Lee Thompson 99353d8c84 remove obsolete line 2012-12-04 16:50:11 -06:00
Stephen Dolan c0a2f1ab47 Add min, max, min_by, max_by functions. 2012-12-04 22:45:03 +00:00
Lee Thompson 76c8706145 fix rpm mv command when rpm stashes the package in a subdirectory 2012-12-04 16:29:01 -06:00
Lee Thompson 277249ffe6 change rpmbuild target to rpm 2012-12-04 14:49:39 -06:00
Stephen Dolan 616e8f9924 Refactor function argument passing into what it always should have been.
Most visible change is that error messages when a function is called
with the wrong number of arguments are much better.
2012-12-04 00:39:21 +00:00
Lee Thompson 00e4feba81 get rpm building from tarball with autoconf 2012-12-03 17:27:40 -06:00
Lee Thompson 26bbed29f7 start work on getting RPM to work with tarball 2012-12-03 16:19:45 -06:00
Stephen Dolan a88d53d2fd Extend {foo} syntax to allow {"foo"} as well.
Useful when "foo" contains unusual characters. Should help with
the issues #7, #38, #40, #42.
2012-12-03 20:31:40 +00:00
Stephen Dolan 5f6a95c7b5 General cleanup - rename a few things, delete dead code. 2012-12-03 20:02:02 +00:00
Stephen Dolan fadeb66365 Add a unique function. 2012-12-03 02:02:12 +00:00
Stephen Dolan c8bbfed041 See, I told you colour was a bad idea! #11
Last commit broke the build, minor fix needed to testsuite.
2012-12-03 02:00:29 +00:00
Stephen Dolan e9c7548b82 Oh alright then, if you insist.
Colo(u)red output for jq. Enabled by default if isatty(stdout).
Closes #11.
2012-12-03 01:21:07 +00:00
Stephen Dolan 67f8ad9435 Ignore a UTF-8 BOM if one appears at the start of a JSON document.
Closes #45.
2012-12-02 23:53:55 +00:00
Stephen Dolan 5b45184a1a Make the -c output more compact. Closes #44. 2012-12-02 23:24:07 +00:00
Stephen Dolan 11965aaa2e sort_by and group_by functions, tests and docs. 2012-12-02 23:22:15 +00:00
Stephen Dolan ed7f95a492 Demote "contains" to a built-in function rather than an operator. 2012-12-02 22:52:38 +00:00
Stephen Dolan 63d10433c9 Fix a horrible scoping/associativity bug.
`1 as $x | . | $x` didn't compile, as it was parsed as

    (1 as $x | .) | $x

where `$x` is undefined, rather than as

    1 as $x | (. | $x)
2012-12-02 22:22:08 +00:00
Stephen Dolan ea9db414ed Clean up implementation of builtin functions. 2012-12-02 22:12:08 +00:00
Stephen Dolan b6f2fbbe62 Move all the includes one place to the left 2012-12-02 21:25:54 +00:00
Stephen Dolan 125e278502 Clean up calls to C functions, unify opcodes 2012-12-02 20:45:55 +00:00
Stephen Dolan 251f221b85 Refactor of function call codegen. Separate codegen for C and jq calls. 2012-12-02 20:45:06 +00:00
Stephen Dolan 9e743d517a Sort function and more general comparison operators. 2012-11-30 20:27:16 +00:00
Lee Thompson ddeec45b2e revert back to make invoking flex, fix a few bugs 2012-11-28 01:08:23 -06:00
Lee Thompson 006357f954 initial attempt at autoconf implementation, removed all generated code from git 2012-11-27 16:02:59 -06:00
Stephen Dolan 2dad2bdd97 Merge remote-tracking branch 'origin/master' 2012-11-26 22:24:04 +00:00
Stephen Dolan d56370f734 Move some higher-level JSON manipulation functions into jv_aux.{c,h} 2012-11-26 22:22:45 +00:00
Stephen Dolan 334a79b704 Define bytecoded builtins more concisely. 2012-11-26 18:53:47 +00:00
Stephen Dolan 4c65847568 Raise a proper error from the lexer on unmatched },),] 2012-11-26 01:40:35 +00:00
Stephen Dolan 4747f8681b Transparent handling for CBV arguments to C functions.
C function arguments closures are inlined before the call.
2012-11-26 01:36:55 +00:00
Stephen Dolan 3e3fe51959 Clean up function creation API and epilogue generation. 2012-11-26 00:39:01 +00:00
Stephen Dolan 924aeda504 Add some preprocessor junk to make codegen cleaner. 2012-11-25 23:49:57 +00:00
Stephen Dolan f19ba32fdd Minor refactor of function call setup to allow for multiple arguments. 2012-11-25 23:07:43 +00:00
Anthony Shortland a33c6f3df9 Fixed to work under CentOS 5.4 2012-10-30 14:30:36 -07:00
Anthony Shortland 04640aeaeb Added initial support for packaging jq as an RPM 2012-10-30 10:26:08 -07:00
Stephen Dolan 0923c79fee Merge pull request #36 from stroan/contains-operator
Implemented contains operator
2012-10-25 11:51:55 -07:00
Stephen Roantree 50bcbc2271 Merge branch 'master' of https://github.com/stedolan/jq into contains-operator
Conflicts:
	lexer.gen.c
	parser.gen.c
	parser.gen.h
	parser.gen.info
	parser.y
2012-10-25 10:02:02 -07:00
Stephen Roantree 1b556315af Remove redundant code from jv_object_contains 2012-10-25 06:29:23 -07:00
Stephen Roantree ecc8998d38 Restructure contains methods to use public jv methods 2012-10-24 18:29:33 -07:00
Stephen Dolan 72691b4909 Fix a bug in string parsing. Closes #35 2012-10-25 00:13:39 +01:00
Stephen Roantree 5e25c2a259 Implemented contains operator 2012-10-24 13:50:26 -07:00
Stephen Dolan 033d9b2fd5 Merge pull request #34 from dgryski/master
Add != (not equal) to list of supported binops
2012-10-24 02:43:54 -07:00
Damian Gryski 0adf4638d1 Update generated lexer.gen.* and parser.gen.* files 2012-10-24 09:31:21 +02:00
Damian Gryski e40778727b Replace yyscan_t with another pointer type that we control.
This prevents the circuluar dependency between parser.gen.h and
lexer.gen.h.  Newer versions of bison add a prototype for yyparse() to
parser.gen.h that include the as-yet-undeclared yyscan_t type.
2012-10-24 09:28:27 +02:00
Stephen Dolan 5a64a29774 Remove .gitignore from tarballs. 2012-10-23 21:58:50 +01:00
Damian Gryski 45b6fc8148 Add some in/equality test cases 2012-10-23 21:12:47 +02:00
Stephen Dolan 8ad3b6f9b9 Merge pull request #22 from alexchamberlain/patch-1
Arch Packaging docs
2012-10-23 11:24:12 -07:00
Damian Gryski 409456f581 Regenerate lex/yacc files 2012-10-23 17:02:19 +02:00
Damian Gryski 553de39b2a Implement 'not equal' (!=) as a binop 2012-10-23 17:01:39 +02:00
Damian Gryski b193bca563 yyscan_t isn't exposed by some versions of flex. 2012-10-23 17:00:42 +02:00
Alex Chamberlain 56aa4cf1df Add docs for installing on Arch Linux. 2012-10-23 14:46:44 +01:00
Stephen Dolan 2a610da2d1 Allow the 'keys' function to take arrays. 2012-10-22 23:31:07 +01:00
Stephen Dolan 93f80b2726 I should probably document select and empty :) (#29, #5) 2012-10-22 23:14:34 +01:00
Stephen Dolan 794d29a430 Docs fixes.
- Mention homebrew on download page (#25)
 - Document 'keys' and string interpolation
2012-10-22 22:56:56 +01:00
Stephen Dolan 3c8f5ea251 Merge pull request #14 from merriam/patch-2
Update docs/content/1.tutorial/default.yml
2012-10-22 14:31:04 -07:00
Stephen Dolan c0f1c7dc1a Delete a badly-thought-out bit of a Makefile. 2012-10-22 19:23:57 +01:00
Stephen Dolan a737a67eda updated autogenerated files 2012-10-22 19:18:35 +01:00
Stephen Dolan 48fb7c61b3 Support a -f option to load from a file, and # comments
This means '#!/usr/local/bin/jq -f' is now a sensible way
to start a file.

Closes #13
2012-10-22 19:18:30 +01:00
Stephen Dolan f2601ab4f0 Fix logic for "slurp" option. Fixes #18 2012-10-22 18:55:56 +01:00
Stephen Dolan 2620e21dcc Make the gen_utf8_tables script Python3 compatible (#20) 2012-10-22 18:47:52 +01:00
Charles Merriam ce3328f11e Update docs/content/1.tutorial/default.yml 2012-10-22 02:43:53 -07:00
Stephen Dolan c53e001973 Version number tracking. 2012-10-21 23:26:31 +01:00
Stephen Dolan 976e4ff711 Fix a mistake in the docs. Fixes #10 2012-10-21 22:16:23 +01:00
Stephen Dolan 6e64cdcbfc Merge pull request #8 from maxme/master
add install and uninstall rules in Makefile
2012-10-21 12:23:22 -07:00
Stephen Dolan 73e479e171 Update website: binaries + version bump. 2012-10-21 20:16:26 +01:00
Maxime Biais 758da604cd add install and uninstall rules in Makefile 2012-10-21 20:55:50 +02:00
Stephen Dolan 95a0074edf Check in a bunch of autogenerated files.
These seem not to build with the version of bison, etc. on OS X.
This is a hack for now, I'll fix the OS X build when I get access
to a mac.
2012-10-21 19:52:16 +01:00
Stephen Dolan 32e1b114df Add a 'keys' function. Fixes #4. 2012-10-20 00:26:37 +01:00
Stephen Dolan afec2544f1 A string interpolation syntax that I don't hate. Also tests.
You can interpolate values into strings using \(this syntax), e.g.

    "best \("str" + "ing") ever"
2012-10-20 00:09:20 +01:00
Stephen Dolan 6828ec9852 Merge pull request #6 from stroan/numeric-comparison-binops
Add <,>,<=,>= binops
2012-10-11 07:47:42 -07:00
Stephen Roantree b3557429eb Make comp binops nonassoc 2012-10-10 23:00:24 +01:00
Stephen Roantree 0618e61e0b Add <,>,<=,>= binops 2012-10-07 22:34:12 +01:00
Stephen Dolan a2643cc0d5 Allow underscores in IDENT tokens. Fixes #3.
IDENT syntax now includes ASCII letters and underscores, so
'.foo_bar' now works. Non-ASCII letters won't work in IDENT
tokens (it's impossible to tell which non-ascii characters are
"letters" without full unicode tables), so '.données' is still
a syntax error (the workaround is '.["données"]', since you can
put anything you like in a string).
2012-09-22 14:03:46 +01:00
Stephen Dolan 359d5f33bd Add a bunch of commandline options for controlling I/O format.
Also docs for the above. Closes #1.
2012-09-20 19:31:14 +01:00
Stephen Dolan fcd7cf4ddd More broken links :( 2012-09-19 10:54:58 +01:00
Stephen Dolan 1f4a5d8d9f Make URLs relative to not break on github-pages 2012-09-19 10:48:23 +01:00
Stephen Dolan 92ab193369 Make the <title>s of the doc pages less messy. 2012-09-19 01:15:49 +01:00
Stephen Dolan fc57cf702c No HTTPS for me, sadly. 2012-09-19 01:12:33 +01:00
Stephen Dolan f51eca20f5 Fix some issues with relative URLs on Github Pages. 2012-09-19 01:10:47 +01:00
Stephen Dolan 9887cb57b2 Add a rake task to compile docs into static html. 2012-09-19 01:03:05 +01:00
Stephen Dolan 040604b7db Use a theme from Bootswatch for docs. 2012-09-19 01:00:25 +01:00
Stephen Dolan 26c7132237 Ensure 'make tarball' creates the required directories. 2012-09-19 00:36:37 +01:00
Stephen Dolan 483f6f2134 Add some release stuff to Makefile. 2012-09-19 00:34:49 +01:00
Stephen Dolan c463877e17 Add binaries and update download docs. 2012-09-19 00:08:02 +01:00
Stephen Dolan 71ab537ee9 Initialise some variables to keep gcc happy. 2012-09-18 23:46:23 +01:00
Stephen Dolan 134b062be7 Fix a parsing bug for \uXXXX escapes (some invalid escapes were accepted).
Found by gcc -O -Wall identifying a use of uninitialised variables.
2012-09-18 23:45:30 +01:00
Stephen Dolan 3898f744dc Add a README 2012-09-18 23:40:19 +01:00
Stephen Dolan 305059ff9f Usage messages if jq is run with no arguments.
Beats segfaulting.
2012-09-18 23:37:46 +01:00
Stephen Dolan 5863160112 Implement the 'add' builtin promised by the docs' examples. 2012-09-18 23:32:24 +01:00
Stephen Dolan fbf7bc835b More docs + docs cleanup 2012-09-18 23:17:27 +01:00
Stephen Dolan 04e85e11de Print Unicode characters unescaped by default. 2012-09-18 22:47:19 +01:00
Stephen Dolan 20e45f363c Separate the tests and the main program. 2012-09-18 22:17:13 +01:00
Stephen Dolan 46af5238ce Add a COPYING file with too many words in it. 2012-09-18 17:59:09 +01:00
Stephen Dolan cf134909fd Documentation. Copious. 2012-09-18 17:51:53 +01:00
Stephen Dolan a4eea165bb Move everything around - delete old Haskell code, clean up build. 2012-09-18 17:44:43 +01:00
Stephen Dolan 25cbab056b Slightly better string interpolation. 2012-09-18 13:22:22 +01:00
Stephen Dolan 8041ce3119 Add the builtin 'select' function. 2012-09-18 12:58:39 +01:00
Stephen Dolan 8f0c91c032 Add a missing file from prev commit 2012-09-18 12:58:29 +01:00
Stephen Dolan 940f8c4270 Builtin functions defined in jq. 2012-09-18 10:17:38 +01:00
Stephen Dolan 5b9bb64a88 -Werror was a little overzealous.
Older versions of gcc warn about random things. -Werror makes the
built fail on such compilers.
2012-09-18 09:43:44 +01:00
Stephen Dolan 7d8d97933f Builtin function 'type'. 2012-09-17 23:06:48 +01:00
Stephen Dolan f13865d7a1 Fix a precedence bug.
a == b or c == d should obviously parse as (a==b) or (c==d).
2012-09-17 22:37:51 +01:00
Stephen Dolan 49cf2e67fe Recursive functions + bugfix for stack reallocation. 2012-09-17 22:32:06 +01:00
Stephen Dolan cbdeddbab8 Make the code compile with warnings-as-errors.
-Wextra found a bona-fide bug: signed/unsigned comparison in a
stack overflow check.
2012-09-17 22:04:58 +01:00
Stephen Dolan eab9c4e587 Fix debugging code in execute.c so that debug jq compiles again. 2012-09-17 21:43:10 +01:00
Stephen Dolan 830610cef8 Make "not" a builtin function rather than syntax.
Fixes a really awkward grammar issue.
2012-09-17 21:08:43 +01:00
Stephen Dolan 297a0679d1 Redefine true and false. Define empty, distict from null.
This might be the most philosophical commit. I shall ponder it.

true, false, null, empty now defined as bytecoded builtins rather
than calls out to C functions.
2012-09-17 20:59:34 +01:00
Stephen Dolan fe33150b7f Merge branch 'master' of github:stedolan/jq
Conflicts:
	c/builtin.c
2012-09-17 20:49:41 +01:00
Stephen Dolan 3db27b01a1 First pass at string interpolation.
Requires a lexer hack, but a surprisingly un-hideous one. The lexer
maintains a stack of bracket characters so that it can match
parens/brackets/braces to tell where a subexp nested in a string
actually ends.
2012-09-17 20:41:57 +01:00
Stephen Dolan 326771f4b4 Add a "tostring" function. 2012-09-17 20:14:07 +01:00
Stephen Dolan 9c15ad2021 Support rendering a JSON value to a string buffer. 2012-09-17 20:11:12 +01:00
Stephen Dolan 84ea129f26 Better support for appending strings in JV. 2012-09-17 19:41:41 +01:00
Stephen Dolan 50ebb036c4 Bind builtin functions in a slightly less ugly way. 2012-09-16 17:08:56 +01:00
Stephen Dolan 539156430a Builtin function 'length', for arrays/objects/strings. 2012-09-16 11:08:42 +01:00
Stephen Dolan cc2fb20ca0 Hrm. Update operators (//=, +=, etc.) aren't very well thought out.
In complex cases, their behaviour is kinda weird. Here's a failing
test for what I think they should do.
2012-09-16 11:07:36 +01:00
Stephen Dolan df195b3187 Add update operators (+=, -=, *=, /= and //=) 2012-09-11 23:55:59 +01:00
Stephen Dolan f6c6ba95ad Make .[] capable of iteration over objects as well as arrays.
Also change an assert to a proper error message if it's given
something silly like a number.
2012-09-11 16:10:55 +01:00
Stephen Dolan d9d6f43407 Pretty-printing of JSON values. 2012-09-11 15:53:37 +01:00
Stephen Dolan 51a44edc63 JSON stream parser.
Allow multiple values on input as concatenated JSON objects,
possibly separated by whitespace.
2012-09-11 14:52:10 +01:00
Stephen Dolan 062a6aa6d7 More error handling - locations of refs to undefined symbols. 2012-09-11 12:24:54 +01:00
Stephen Dolan 4c7bede5a4 Better error handling and messages for invalid index/assign operations. 2012-09-11 10:37:44 +01:00
Stephen Dolan 0ce437ea97 More error handling in the parser.
Add a special case in the lexer to detect unterminated strings.
Add some error recovery in the parser for more copious error spam.
2012-09-11 10:18:18 +01:00
Stephen Dolan ac3f8bcc52 Proper error messages from lexer errors (e.g. bad characters). 2012-09-11 09:13:20 +01:00
Stephen Dolan 95dd64b0f0 Much, much better error reporting from the parser.
Getting proper locations through flex/bison was more effort than
was advertised.
2012-09-11 00:04:47 +01:00
Stephen Dolan 4d47f8f1e1 '==' operator, tests for equality and operator precedence. 2012-09-10 18:08:00 +01:00
Stephen Dolan 3895bbf856 String -> number conversions with "tonumber". 2012-09-10 17:08:13 +01:00
Stephen Dolan aa3ebdfe9b Multiplication and division operators.
It is an error to use these on anything but numbers. Maybe later
implementing python/ruby-style ("a" * 100) might be handy.
2012-09-10 16:57:17 +01:00
Stephen Dolan 6f3abbac62 Subtraction - as expected for numbers, ruby-style set diff for arrays. 2012-09-10 16:49:25 +01:00
Stephen Dolan 65ce73deb4 String concatenation. 2012-09-10 16:16:39 +01:00
Stephen Dolan e718bd50b6 Sensible error messages when a silly addition is performed.
Fix a test to no longer perform an invalid addition.
2012-09-10 16:15:05 +01:00
Stephen Dolan a3e509034b jv_string_fmt (create printf-formatted JSON strings) 2012-09-10 16:01:56 +01:00
Stephen Dolan 0159c0ecc5 Plug a memory leak in jv_object_merge 2012-09-10 15:08:58 +01:00
Stephen Dolan 649cac791a Make jv_invalid() first-class values capable of holding an error. 2012-09-10 15:04:19 +01:00
Stephen Dolan 5c25c90657 When comparing two objects for equality, we don't need to recurse
into e.g. array elements if the two objects are the same array.
2012-09-10 13:06:46 +01:00
Stephen Dolan c2593dc417 Make a main program that doesn't spam debugging info. 2012-09-10 10:23:15 +01:00
Stephen Dolan e258d20ba2 Make the '+' operator merge objects. 2012-09-09 19:17:07 +01:00
Stephen Dolan a4ff3d4670 Bugfix for expanding objects. 2012-09-09 18:09:05 +01:00
Stephen Dolan e6a85737da Print bad number values (Inf,NaN) "correctly". 2012-09-09 15:35:34 +01:00
Stephen Dolan 0dfaebcdcb null and string literals now exist, so add tests 2012-09-06 21:44:46 +01:00
Stephen Dolan 0fd7d4b30c UTF8 coding utilities and unicode escaping in jv_dump()'d strings.
Beyond the Basic Multilingual Plane, dead Cthulhu waits dreaming.
2012-09-06 21:39:00 +01:00
Stephen Dolan 58a9b0cfe1 Make the test harness not segfault when a test fails (!) 2012-09-04 20:56:44 +01:00
Stephen Dolan da5d653de8 Add a Boolean "not" operator.
Not wholly convinced this is a good idea, maybe "not" should be a
builtin function instead? 'not (.a == .b)' vs. '.a == .b | not'
2012-09-04 20:43:40 +01:00
Stephen Dolan 6e6ea50763 Short-circuiting Boolean "and" and "or" operators. 2012-09-04 20:38:59 +01:00
Stephen Dolan c5ab3b2336 Move some unicode handling stuff to a separate file. 2012-09-04 20:38:47 +01:00
Stephen Dolan 328c4a13f1 Add "elif" to if-then-else constructs. 2012-09-04 16:05:24 +01:00
Stephen Dolan 5add2c2ceb if-then-else and defined-or operators 2012-09-04 15:34:34 +01:00
Stephen Dolan 520c7bb15e Fix some confusion between "null" and "invalid". 2012-09-03 17:26:52 +01:00
Stephen Dolan 32e324a283 Fix a memory leak in JV and add more tests 2012-09-03 17:18:10 +01:00
Stephen Dolan eea05ff924 Perl-style autovivification.
.foo.bar = 1 will cause objects to be created if they don't exist.
2012-09-03 16:50:38 +01:00
Stephen Dolan c9dbe1aff8 squash a debug printf 2012-09-03 16:42:58 +01:00
Stephen Dolan a8c3648ca5 JV_KIND_INVALID values to represent failed lookups, etc + various tests. 2012-09-03 16:16:14 +01:00
Stephen Dolan d895d39ba9 Support "null" in JQ programs 2012-09-03 16:14:52 +01:00
Stephen Dolan 8fe9c8a22a Make assignment work again: = and |= operators. 2012-09-03 15:33:59 +01:00
Stephen Dolan d8fad1ed9b Disallow a + before numbers (makes parsing easier, agrees with JSON) 2012-09-03 15:32:50 +01:00
Stephen Dolan cf004c2b3a Modify number formatting so that 1e-3 renders as 0.001, not .001 2012-09-03 13:57:53 +01:00
Stephen Dolan 6041fd7661 Literal strings and better literal numbers. 2012-09-03 13:57:47 +01:00
Stephen Dolan 52487ff812 Remove globals from parser, use explicit structure. 2012-09-03 01:12:42 +01:00
Stephen Dolan 22ffc0fdfc Mark a constant array const in jvp_dtoa.c 2012-09-02 22:24:27 +01:00
Stephen Dolan e3710c0d1f Fix behaviour of stack at program termination. 2012-09-02 22:11:06 +01:00
Stephen Dolan 4f747e6fff Move from Jansson to JV - proper freeing of memory
A few more tests, now passes valgrind.
2012-09-02 21:45:52 +01:00
Stephen Dolan 9da3e0d373 Clean up build a little and add .gitignore. 2012-09-02 17:24:17 +01:00
Stephen Dolan f72c5ffac0 Move from Jansson to JV - interpreter loop
Passes tests, but leaks some memory.
2012-09-02 17:20:13 +01:00
Stephen Dolan 2cb9a6e61d Move from Jansson to JV - everything but the interpreter loop
Passes valgrind --leak-check=full.
2012-09-02 16:31:59 +01:00
Stephen Dolan 22f8aed31e Validation for empty objects
Now (correctly) fails to parse {,}
2012-09-02 00:24:23 +01:00
Stephen Dolan e98624e5a9 First pass at a JSON parser 2012-09-01 19:16:43 +01:00
Stephen Dolan 92654f8204 Fix a memory leak when inserting into an object with an already-present key 2012-09-01 18:24:17 +01:00
Stephen Dolan 66fd702363 Minor changes to dtoa.c to make it easier to work with.
- Change memory allocation to thread a context structure through
   the code rather than using globals (which would require a
   -lpthread and some locking to become threadsafe).

 - Remove a few configuration options (the "private memory" space,
   and K&R style headers).

 - Prefix exported symbols with jvp_ to prevent libc conflicts.

 - Add jvp_dtoa.h

 - Include netlib's g_fmt.c into dtoa.c

 - Add a few not-strictly-necessary initialisations in order to
   compile cleanly with gcc -Wall
2012-09-01 18:01:35 +01:00
Stephen Dolan 9e544c55dc Import a copy of David Gay's dtoa.c 2012-09-01 16:50:25 +01:00
Stephen Dolan f25cb16e6b Fix a bug in lexical scoping, add tests 2012-08-28 18:38:30 +01:00
Stephen Dolan 463b175cdb Reference-counted copy-on-write JSON library.
Comes with tests and, presumably, bugs.
2012-08-28 18:09:43 +01:00
Stephen Dolan dc6cf2fb58 Remove syntax distinction between builtin and user calls 2012-08-27 11:19:42 +01:00
Stephen Dolan 6c8b55793a first pass at assignment 2012-08-27 10:12:02 +01:00
Stephen Dolan a847d2250f 2nd order functions 2012-08-26 14:25:56 +01:00
Stephen Dolan befb103a12 Fix a revolting hack 2012-08-22 19:21:35 +01:00
Stephen Dolan 2086e59ab1 bugfix for backtracking over RET insns, and a truly evil testcase 2012-08-22 19:05:53 +01:00
Stephen Dolan 79d81f4bee Closures over variables 2012-08-21 18:24:38 +01:00
Stephen Dolan c128c2029e First pass at functions + tests 2012-08-21 18:14:13 +01:00
Stephen Dolan dd0d340eba frame layout refactor 2012-08-21 12:35:44 +01:00
Stephen 15ad640202 Stacks grow down now 2012-08-20 15:53:20 +01:00
Stephen fd738bface an addition operator, of sorts 2012-08-16 01:16:08 +01:00
Stephen 2002dc1a2f Most of a C implementation of jq 2012-08-16 01:00:30 +01:00
130 changed files with 45817 additions and 581 deletions

No files matched your search

+2
View File
@@ -0,0 +1,2 @@
.gitattributes export-ignore
.gitignore export-ignore
+47
View File
@@ -0,0 +1,47 @@
*.o
*.a
*.lo
*.la
*.lai
*.so
*.so.*
*~
.*.sw[a-p]
tags
jq
!tests/modules/lib/jq/
jq.1
# Something delightfully recursive happens otherwise
docs/content/2.download/source/*
# Autotools junk
.libs
.deps
libtool
*.log
stamp-h1
config.log
config.status
autom4te.cache
INSTALL
Makefile
jq-*.tar.gz
configure
aclocal.m4
Makefile.in
version.h
.remake-version-h
config.cache
*.rpm
m4/libtool.m4
m4/ltoptions.m4
m4/ltsugar.m4
m4/ltversion.m4
m4/lt~obsolete.m4
tests/*.trs
cscope.in.out
cscope.out
cscope.po.out
+53
View File
@@ -0,0 +1,53 @@
sudo: false
language: c
compiler:
- gcc
- clang
matrix:
include:
- compiler: gcc
env: COVERAGE="--disable-valgrind --enable-gcov"
addons:
apt:
packages:
- libonig-dev
- valgrind
before_install:
- rm lexer.h lexer.c parser.h parser.c
install:
- bundle install --gemfile=docs/Gemfile
- wget http://ftp.debian.org/debian/pool/main/b/bison/bison_3.0.2.dfsg-2_amd64.deb
- ar p bison_3.0.2.dfsg-2_amd64.deb data.tar.xz | tar xJ
- if [ -n "$COVERAGE" ]; then pip install --user cpp-coveralls; fi
before_script:
- autoreconf -i
- ./configure
YACC="usr/bin/bison -y"
$COVERAGE
script:
- make -j4 BISON_PKGDATADIR=$(pwd)/usr/share/bison
- make check -j4
after_script:
- |
if [ -n "$COVERAGE" ]; then
rm -rf .libs usr # don't care about coverage for libjq and bison
coveralls -e lexer.c -e parser.c -e jv_dtoa.c --gcov-options '\-lp'
fi
after_failure:
- cat test-suite.log
- cat tests/*.log
notifications:
email: false
+73
View File
@@ -0,0 +1,73 @@
Created By:
Stephen Dolan <mu@netsoc.tcd.ie>
Maintained by:
Nicolas Williams <nico@cryptonector.com>
William Langford <wlangfor@gmail.com>
David Tolnay <dtolnay@gmail.com>
Contributions by:
Andrew Rodland <andrew@cleverdomain.org> - bug fixes
Ankur <ankz.kothari@gmail.com> - bug fixes
Anthony Shortland <anthony.shortland@me.com> - rpmbuild target
Brendan Macmillan <melbourne.research@gmail.com> - bug fixes, load library from ~/.jq
David R. MacIver <david@drmaciver.com> - bug fixes
David Tolnay <dtolnay@gmail.com> - destructuring, build improvements
Filippo Giunchedi <fgiunchedi@gmail.com> - bug fixes
Filippo Valsorda <filippo.valsorda@gmail.com> - recursive object merge (`*`)
Jack Pearkes <jackpearkes@gmail.com> - update tutorial
Jingwen Owen Ou <jingweno@gmail.com> - jqplay.org and link to it
Joe Littlejohn <joe.littlejohn@nokia.com> - bug fixes
Juan Guerrero <juan.guerrero.lozano@gmail.com> - bug fixes
Kenny Shen <kenny.shen@zalora.com> - doc fixes
Lee Thompson <stagr.lee@gmail.com> - autoconf stuff
Markus Lanthaler <mark_lanthaler@gmx.net> - doc fixes
Michael Daines <michael@mdaines.com> - add sqrt; doc fixes
Mike McCabe <mccabe@archive.org> - math (libm) functions
Nicolas Pouillard <nicolas.pouillard@gmail.com> - add tests
Nicolas Williams <nico@cryptonector.com> - library-fication, autoconf stuff, exception handling, various
Peter van Dijk <peter@7bits.nl> - doc fixes
Philipp Hagemeister <phihag@phihag.de> - doc fixes
polyester <paul@cleanclothes.org> - automake version update
Rémy Léone <remy.leone@gmail.com> - add .travis.yml
Ryoichi KATO <ryo1kato@gmail.com> - doc fixes
Santiago Lapresta <santiago.lapresta@gmail.com> - join, arrays, all, any, other filters
pkoppstein <pkoppstein@gmail.com> - various builtins, improvements
13ren <melbourne.research@gmail.com>
Aaron Peschel <apeschel@zendesk.com>
Adam Lindberg <hello@alind.io>
Alex Chamberlain <alex@alexchamberlain.co.uk>
Andrew O'Brien <obrien.andrew@gmail.com> - docs build
Assaf Gordon <assafgordon@gmail.com> - error handling
cdnbacon <mike@busbud.com>
Charles Merriam <charles.merriam@gmail.com>
Colin von Heuring <colin@janrain.com>
Damian Gryski <damian@gryski.com>
David Haguenauer <ml@kurokatta.org>
Doug Luce <doug@github.con.com> - build
Eiichi Sato <sato.eiichi@gmail.com>
Hanfei Shen <qqshfox@gmail.com>
Ian Miell <ian.miell@gmail.com>
i <isomorphisms@sdf.org>
James Andariese <james.andariese@locationlabs.com> - Dockerfile
jkleint <jkleint@gmail.com>
Joel Purra <code+github@joelpurra.com>
Kim De Mey <kim.demey@gmail.com> - build
Kim Toms <kim.toms@bplglobal.net>
LCD 47 <lcd047@gmail.com>
Lee Thompson <thompson@dtosolutions.com> - rpm
Marc Abramowitz <marc@marc-abramowitz.com>
Marc Bruggmann <marcbr@spotify.com>
Maxime Biais <maxime.biais@gmail.com> - build
Mike Fletcher <john.m.fletcher@bankofamerica.com>
Sebastian Freundt <freundt@ga-group.nl> - build
Shay Elkin <shay@everything.me>
Simon Elsbrock <simon@iodev.org> - Debian
Stefan Seemayer <stefan@seemayer.de>
Stephen Roantree <stroantree@gmail.com>
Stephen Shaw <stesh@netsoc.tcd.ie>
Steven Maude <StevenMaude@users.noreply.github.com>
Steven Penny <svnpenn@gmail.com> - Windows bug fixes
tal@whatexit.org <tal@whatexit.org>
Travis Gockel <travis@gockelhut.com>
Zhiming Wang <zmwangx@gmail.com>
+70
View File
@@ -0,0 +1,70 @@
jq is copyright (C) 2012 Stephen Dolan
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
jq's documentation (everything found under the docs/ subdirectory in
the source tree) is licensed under the Creative Commons CC BY 3.0
license, which can be found at:
http://creativecommons.org/licenses/by/3.0/
The documentation website includes a copy of Twitter's Boostrap and
relies on Bonsai, Liquid templates and various other projects, look
them up for detailed licensing conditions.
jq incorporates David M. Gay's dtoa.c and g_fmt.c, which bear the
following notices:
dtoa.c:
The author of this software is David M. Gay.
Copyright (c) 1991, 2000, 2001 by Lucent Technologies.
Permission to use, copy, modify, and distribute this software for any
purpose without fee is hereby granted, provided that this entire notice
is included in all copies of any software which is or includes a copy
or modification of this software and in all copies of the supporting
documentation for such software.
THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
WARRANTY. IN PARTICULAR, NEITHER THE AUTHOR NOR LUCENT MAKES ANY
REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
g_fmt.c:
The author of this software is David M. Gay.
Copyright (c) 1991, 1996 by Lucent Technologies.
Permission to use, copy, modify, and distribute this software for any
purpose without fee is hereby granted, provided that this entire notice
is included in all copies of any software which is or includes a copy
or modification of this software and in all copies of the supporting
documentation for such software.
THIS SOFTWARE IS BEING PROVIDED "AS IS", WITHOUT ANY EXPRESS OR IMPLIED
WARRANTY. IN PARTICULAR, NEITHER THE AUTHOR NOR LUCENT MAKES ANY
REPRESENTATION OR WARRANTY OF ANY KIND CONCERNING THE MERCHANTABILITY
OF THIS SOFTWARE OR ITS FITNESS FOR ANY PARTICULAR PURPOSE.
+1349
View File
File diff suppressed because it is too large. Load diff
+50
View File
@@ -0,0 +1,50 @@
FROM debian:8
COPY . /app
# get dependencies, build, and remove anything we don't need for running jq.
# valgrind seems to have trouble with pthreads TLS so it's off.
RUN apt-get update && \
apt-get install -y \
build-essential \
autoconf \
libtool \
git \
bison \
flex \
ruby \
wget \
ruby-dev && \
wget http://www.geocities.jp/kosako3/oniguruma/archive/onig-5.9.6.tar.gz && \
sha512sum onig-5.9.6.tar.gz | grep 4a181ea6f0e1a018bbaf6c87e666dfffd1ef4b8f5dcead07fa0b6564a76174e7e01854173924668433ae74c455dbced6a0e1b43e9066f0499b4a57e855e1a2b2 && \
tar zxvf onig-5.9.6.tar.gz && \
(cd onig-5.9.6 && \
./configure --prefix=/usr/local && \
make && \
make install ) && \
gem install bundler && \
(cd /app/docs && bundle install) && \
(cd /app && \
autoreconf -i && \
./configure --disable-valgrind --enable-all-static --prefix=/usr/local && \
make -j8 && \
make check && \
make install && \
make distclean ) && \
(cd onig-5.9.6 && \
make uninstall ) && \
apt-get purge -y \
build-essential \
autoconf \
libtool \
bison \
git \
flex \
ruby \
ruby-dev && \
apt-get autoremove -y && \
rm -rf onig-5.9.6 && \
rm -rf /var/lib/apt/lists/* /var/lib/gems
ENTRYPOINT ["/usr/local/bin/jq"]
-304
View File
@@ -1,304 +0,0 @@
module JQ where
import Text.JSON
import Text.JSON.String
import Data.Maybe
import Data.Char
import Data.List (sortBy,sort,groupBy,partition,intercalate)
import Data.Function (on)
import Data.Ord (comparing)
import Control.Monad
import Control.Monad.Writer
import Control.Monad.List
import Control.Monad.Reader
import qualified Data.Map as M
import qualified Data.Set as S
import Data.Map ((!))
import Debug.Trace
type Path = [Either Int String]
type Program = JSValue -> [(JSValue, Path)]
type Filter = JQ JSValue
newtype Operator = Operator {runOperator:: [Filter] -> Filter}
type JQ = ReaderT (JSValue, M.Map String JSValue, M.Map (String, Int) Operator) (WriterT Path [])
runJQ :: JQ a -> JSValue -> [a]
runJQ prog val = map fst $ runWriterT $ runReaderT prog (val,M.empty,primitives)
(>|) :: JQ JSValue -> JQ a -> JQ a
a >| b = do
val <- a
local (\(v,s,d) -> (val,s,d)) b
setvar name val prog =
local (\(v,s,d) -> (v, M.insert name val s, d)) prog
getvar name = liftM (! name) $ asks (\(v,s,d) -> s)
input = asks (\(v,s,d) -> v)
collect :: JQ a -> JQ [a]
collect prog = liftM (map fst) $ collectPaths prog
collectPaths :: JQ a -> JQ [(a,Path)]
collectPaths prog = do
rd <- ask
return $ runWriterT $ runReaderT prog rd
withDefn :: (String, [String], Filter) -> Filter -> Filter
withDefn (name, formals, body) subexp =
local (\(v,s,d) -> (v,s,M.insert (name,length formals) (Operator func) d)) subexp
where
func args = local (\(v,s,d) -> (v,M.empty,M.fromList (zip (zip formals (repeat 0)) (map (Operator . const) args)) `M.union` d)) body
subexp :: JQ a -> JQ a
subexp = censor $ const []
yieldPaths :: [(a,Path)] -> JQ a
yieldPaths ps = ReaderT $ const $ WriterT ps
insert :: JQ JSValue -> JSValue -> Path -> JQ JSValue
insert replace base [] = replace
insert replace (JSArray values) ((Left n):rest) = do
let array = take (max (n+1) (length values)) (values ++ repeat JSNull)
replacement <- insert replace (array !! n) rest
let (left, (_:right)) = splitAt n array
return $ JSArray $ left ++ [replacement] ++ right
insert replace (JSObject obj) ((Right k):rest) = do
let oldval = maybe JSNull id (lookup k $ fromJSObject obj)
replacement <- insert replace oldval rest
let withoutK = filter ((/= k) . fst) $ fromJSObject obj
return $ JSObject $ toJSObject $ (k, replacement):withoutK
insert replace JSNull p@((Right k):rest) = insert replace (JSObject $ toJSObject []) p
insert replace JSNull p@((Left n):rest) = insert replace (JSArray []) p
insert _ base p = error $ "Cannot insert into " ++ intercalate ", " (map (either show show) p) ++ " of " ++ encode base
eqj a b = JSBool $ a == b
boolj (JSBool false) = False
boolj (JSNull) = False
boolj _ = True
andj a b = JSBool $ boolj a && boolj b
orj a b = JSBool $ boolj a || boolj b
liftp :: (JSValue -> JSValue) -> JQ JSValue
liftp f = liftM f input
idp = undefined
failp t = []
constp :: JSValue -> Program
constp t t' = idp t
anyj :: [JSValue] -> Bool
anyj values = any isTrue values
where
isTrue (JSBool False) = False
isTrue (JSNull) = False
isTrue _ = True
selectp prog = do
match <- collect prog
guard $ anyj match
input
constStr :: String -> JQ JSValue
constStr = return . JSString . toJSString
constInt :: Int -> JQ JSValue
constInt = return . JSRational False . toRational
tr x = trace (show x) x
assignp sel replace = do
paths <- collectPaths sel
t <- input
foldM (\base (val,path) -> insert (return val >| replace) base path) t paths
arrayp prog = liftM JSArray $ collect prog
childp' :: JSValue -> JQ JSValue
childp' (JSArray values) = msum [tell [Left i] >> return v | (v,i) <- zip values [0..]]
childp' (JSObject obj) = msum [tell [Right k] >> return v | (k,v) <- fromJSObject obj]
childp' _ = mzero
childp = input >>= childp'
--findp :: Program -> Program
findp prog = do
found <- collect prog
if anyj found then input else childp >| findp prog
groupp prog = do
list <- input
case list of
JSArray values -> do
marked <- forM values $ \v -> do
m <- collect (return v >| prog)
return (m,v)
msum $
map (return . JSArray . map snd) $
groupBy ((==) `on` fst) $
sortBy (comparing fst) $
marked
_ -> return JSNull
recp prog = do
found <- collectPaths prog
let (roots,subs) = partition (null . snd) found
msum $
[tell p >> return x | (x,p) <- roots] ++
[tell p >> (return x >| recp prog) | (x,p) <- subs]
elsep p1 p2 = do
p1' <- collectPaths p1
if null p1' then p2 else yieldPaths p1'
fullresultp prog = do
res <- collectPaths prog
msum [return $ JSObject $ toJSObject $ [("val",a),("path",JSArray $ map fromPath p)] | (a,p) <- res]
where
fromPath (Left n) = js n
fromPath (Right s) = js s
withArray f (JSArray values) = JSArray $ f values
withArray f x = x
withString f (JSString str) = JSString $ toJSString $ f $ fromJSString str
withString f x = x
{-
callp "select" [p] = selectp p
callp "find" [p] = findp p
callp "sort" [] = liftp (withArray sort)
callp "group" [p] = groupp p
callp "rec" [p] = recp p
callp "empty" [] = mzero
callp "true" [] = return $ JSBool True
callp "false" [] = return $ JSBool False
callp "null" [] = return $ JSNull
callp "count" [] = liftp countj
callp "fullresult" [p] = fullresultp p
callp "uppercase" [] = liftp $ withString $ map toUpper
callp "lowercase" [] = liftp $ withString $ map toLower
-}
primitives = M.fromList [((name,arglen),Operator func) |
(name,arglen,func) <- prim]
where
prim = [("if",1,\[p] -> selectp p),
("find", 1, \[p] -> findp p),
("group", 1, \[p] -> groupp p),
("rec", 1, \[p] -> recp p),
("true", 0, const $ return $ JSBool True),
("false", 0, const $ return $ JSBool False),
("null", 0, const $ return $ JSNull),
("count", 0, const $ liftp countj),
("fullresult", 1, \[p] -> fullresultp p),
("zip", 0, const $ liftp zipj),
("keys", 0, const $ liftp keysj)
]
callp :: String -> [Filter] -> Filter
callp name args = do
(v,s,d) <- ask
runOperator (d ! (name, length args)) args
countj (JSArray v) = js$ length v
countj (JSObject o) = js$ length $ fromJSObject o
countj _ = js$ (1::Int)
lookupj :: JSValue -> JSValue -> JQ JSValue
lookupj (JSArray values) (JSRational _ n) = do
let idx = round n
tell [Left idx]
if idx >= 0 && idx < length values
then return $ values !! idx
else return $ JSNull
lookupj (JSObject obj) (JSString s) = do
tell [Right (fromJSString s)]
case (lookup (fromJSString s) (fromJSObject obj)) of
Just x -> return x
Nothing -> return JSNull
lookupj JSNull (JSRational _ n) = do
tell [Left $ round n]
return JSNull
lookupj JSNull (JSString s) = do
tell [Right (fromJSString s)]
return JSNull
--lookupj v i = error $ "Cannot get element " ++ encode i ++ " of " ++ encode v
lookupj _ _ = mzero
plusj (JSRational f1 n1) (JSRational f2 n2) = JSRational (f1 || f2) (n1 + n2)
plusj (JSString s1) (JSString s2) = JSString $ toJSString (fromJSString s1 ++ fromJSString s2)
plusj (JSArray a1) (JSArray a2) = JSArray $ a1 ++ a2
plusj (JSObject o1) (JSObject o2) = JSObject $ toJSObject $ o1' ++ fromJSObject o2
where
newkeys = map fst $ fromJSObject o2
o1' = filter (not . (`elem` newkeys) . fst) $ fromJSObject o1
keysj (JSArray v) = js [0..length v - 1]
keysj (JSObject obj) = js (map fst $ fromJSObject obj)
keysj _ = JSArray []
zipj jsonValue = result $ tx values ctor
where
(result, values) = extract packed
(packed, ctor) = case jsonValue of
JSArray values -> (values, (\vs' -> JSArray $ [v' | Just v' <- vs']))
JSObject jsObject ->
let object = fromJSObject jsObject
keys = map fst object
values = map snd object
build vs' = [(k,v') | (k,Just v') <- zip keys vs']
in (values, JSObject . toJSObject . build)
_ -> error "only arrays and objects may be zipped"
extract values | all isArray values = (JSArray, [map Just arr | JSArray arr <- values])
| all isObject values =
let objects = [fromJSObject o | JSObject o <- values]
keys = S.toList $ S.fromList [k | obj <- objects, (k,_) <- obj]
values' :: [[Maybe JSValue]]
values' = [[lookup k object | k <- keys] | object <- objects]
result r = JSObject $ toJSObject $ zip keys r
in (result, values')
| otherwise = error "elements of zipped value must be all objects or all arrays"
where
isArray (JSArray a) = True
isArray _ = False
isObject (JSObject o) = True
isObject _ = False
head' [] = Nothing
head' (x:xs) = x
tail' [] = []
tail' (x:xs) = xs
tx values ctor | all null values = []
| otherwise = ctor (map head' values):tx (map tail' values) ctor
js :: JSON a => a -> JSValue
js = showJSON
index s = do
v <- input
lookupj v (js s)
dictp progs = do
liftM (JSObject . toJSObject) $ forM progs $ \(k,v) -> do
JSString k' <- subexp k
v' <- subexp v
return (fromJSString k', v')
-102
View File
@@ -1,102 +0,0 @@
{
module Lexer where
import Control.Monad.Error
}
%wrapper "monadUserState"
$digit = 0-9
$alpha = [a-zA-Z_]
@reserved = "."|"["|"]"|","|":"|"("|")"|"{"|"}"|"|"|"=="|"+"|"="|"$"|"def"|";"|"else"|"and"|"or"|"as"
@ident = $alpha [$alpha $digit]*
@string = \" ($printable)* \"
tokens :-
<0> "#" ($printable # [\n\r])* ;
<0> $white+ ;
<0> @reserved { tok TRes }
<0> @ident { tok TIdent }
<0> $digit+ { tok $ TInt . read }
<0> \" { enterString }
<string> \" { leaveString }
<string> ($printable # [\"\\]) { pushString id }
<string> \\ [\"\\\/] { pushString (drop 1) }
<string> \\ [nrt] { pushString (escape . drop 1) }
--<string> \\ 'u' [0-9a-fA-F]{4}
-- { pushString (parseUnicode . drop 2) }
-- @string { \s -> TString $ init $ tail s}
{
escape :: String -> String
escape "r" = "\r"
escape "n" = "\n"
escape "t" = "\t"
getState :: Alex AlexState
getState = Alex $ \s -> Right (s, s)
getUserState :: Alex AlexUserState
getUserState = liftM alex_ust getState
setUserState :: AlexUserState -> Alex ()
setUserState s' = Alex $ \s -> Right (s{alex_ust = s'}, ())
alexEOF = return $ Nothing
enterString input len = do
setUserState []
alexSetStartCode string
skip input len
pushString f i@(p, _, s) len = do
buf <- getUserState
setUserState (buf ++ [f $ take len s])
skip i len
leaveString input len = do
s <- getUserState
alexSetStartCode 0
return $ Just $ TString $ concat s
tok f (p,_,s) len = return $ Just $ f (take len s)
data Token = TRes String | TString String | TIdent String | TInt Int
instance Show Token where
show (TRes t) = "token " ++ t
show (TString t) = "string " ++ t
show (TIdent t) = "identifier " ++ t
show (TInt t) = "integer " ++ show t
type AlexUserState = [String]
alexInitUserState = undefined
wrapError (Alex scanner) = Alex $ \s -> case scanner s of
Left message -> Left (message ++ " at " ++ showpos (alex_pos s))
where
showpos (AlexPn off line col) = "line " ++ show line ++ ", column " ++ show col
x -> x
scanner = do
tok <- wrapError alexMonadScan
case tok of
Nothing -> do
s <- getState
case alex_scd s of
0 -> return []
string -> alexError "Unterminated string literal"
Just tok -> liftM (tok:) scanner
runLexer :: String -> Either String [Token]
runLexer input = runAlex input scanner
}
-23
View File
@@ -1,23 +0,0 @@
import Parser
import Lexer
import JQ
import Text.JSON
import Text.JSON.String
import PrettyJSON
import System.Environment
import Control.Monad
import System.IO
parseJS :: String -> JSValue
parseJS s = case runGetJSON readJSValue s of
Left err -> error err
Right val -> val
main = do
[program] <- getArgs
stdlib <- openFile "stdlib.jq" ReadMode >>= hGetContents
json <- liftM parseJS $ hGetContents stdin
case runLexer (stdlib ++ program) >>= runParser of
Left err -> putStrLn err
Right program -> mapM_ (putStrLn . show . renderJSON) (runJQ program json)
-8
View File
@@ -1,8 +0,0 @@
jq: *.hs Parser.hs Lexer.hs
ghc *.hs -o jq
Parser.hs: Parser.y
happy -i Parser.y
Lexer.hs: Lexer.x
alex Lexer.x
+196
View File
@@ -0,0 +1,196 @@
### C source files to be built and distributed.
LIBJQ_INCS = jq_parser.h builtin.h bytecode.h compile.h exec_stack.h \
libm.h jv_alloc.h jv_dtoa.h jv_unicode.h locfile.h \
opcode_list.h parser.y jv_utf8_tables.h lexer.l util.h linker.h
LIBJQ_SRC = locfile.c bytecode.c compile.c execute.c builtin.c jv.c \
jv_parse.c jv_print.c jv_dtoa.c jv_unicode.c jv_aux.c jv_file.c \
jv_alloc.c jq_test.c util.c linker.c ${LIBJQ_INCS}
### C build options
AM_CFLAGS = -Wextra -Wall -Wno-missing-field-initializers \
-Wno-unused-parameter -Wno-unused-function
ACLOCAL_AMFLAGS = -I config/m4
### Generating the lexer and parser
# While there is some autoconf macro support for lex/flex, it doesn't support
# header file creation so we'll use good old make
if MAINTAINER_MODE
BUILT_SOURCES = lexer.h lexer.c parser.h parser.c version.h
lexer.c: lexer.l
$(AM_V_LEX) flex -o lexer.c --header-file=lexer.h $<
lexer.h: lexer.c
else
BUILT_SOURCES =
.y.c:
$(AM_V_YACC) echo "NOT building parser.c!"
.l.c:
$(AM_V_LEX) echo "NOT building lexer.c!"
endif
# Tell YACC (bison) autoconf macros that you want a header file created.
# If the --warnings=all fails, you probably have an old version of bison
# OSX ships an old bison, so update with homebrew or macports
AM_YFLAGS = --warnings=all -d
### libjq
lib_LTLIBRARIES = libjq.la
libjq_la_SOURCES = ${LIBJQ_SRC}
libjq_la_LIBADD = -lm
libjq_la_LDFLAGS = -export-symbols-regex '^j[qv]_' -version-info 1:4:0
include_HEADERS = jv.h jq.h
### Running tests under Valgrind
if ENABLE_VALGRIND
NO_VALGRIND =
else
NO_VALGRIND = 1
endif
### Code coverage with gcov
if ENABLE_GCOV
AM_CFLAGS += --coverage --no-inline
endif
### Error injection for testing
if ENABLE_ERROR_INJECTION
lib_LTLIBRARIES += libinject_errors.la
libinject_errors_la_SOURCES = inject_errors.c
libinject_errors_la_LIBADD = -ldl
libinject_errors_la_LDFLAGS = -module
endif
### Building the jq binary
# Remake the version.h header file if, and only if, the git ID has changed
.PHONY: .FORCE
.FORCE:
generate_ver = ver="`{ $(srcdir)/scripts/version || echo '$(VERSION)' ; } | sed 's/.*/\#define JQ_VERSION \"&\"/'`"
.remake-version-h: .FORCE
@ $(generate_ver); test "x`cat version.h 2>/dev/null`" = "x$$ver" || touch .remake-version-h
version.h: .remake-version-h
$(AM_V_GEN) $(generate_ver); echo "$$ver" > $@
main.c: version.h
bin_PROGRAMS = jq
jq_SOURCES = main.c version.h
jq_LDFLAGS = -static-libtool-libs
jq_LDADD = libjq.la -lm
if ENABLE_ALL_STATIC
jq_LDFLAGS += -all-static
endif
### Tests (make check)
TESTS = tests/mantest tests/jqtest tests/onigtest tests/shtest
TESTS_ENVIRONMENT = NO_VALGRIND=$(NO_VALGRIND)
### Building the manpage
# If ENABLE_DOCS is not set, just copy jq.1.default to jq.1
# The real_docs target succeeds (possibly after building jq.1) only if ENABLE_DOCS is set
# Thus, making "dist" depend on "real_docs" ensures that we never build a tarball with
# a stub manpage.
man_MANS = jq.1
.PHONY: real_docs
if ENABLE_DOCS
jq.1: $(srcdir)/docs/content/3.manual/manual.yml
$(AM_V_GEN) ( cd ${abs_srcdir}/docs; '$(BUNDLER)' exec rake manpage ) > $@ || { rm -f $@; false; }
jq.1.prebuilt: jq.1
$(AM_V_GEN) cp $^ $@ || { rm -f $@; false; }
jq.1.default: $(srcdir)/docs/default_manpage.md
$(AM_V_GEN) ( cd ${abs_srcdir}/docs; '$(BUNDLER)' exec rake manpage_default ) > $@ || { rm -f $@; false; }
real_docs: jq.1 jq.1.prebuilt
@if cmp jq.1 $(srcdir)/jq.1.default > /dev/null; then\
rm -f jq.1; $(MAKE) $(AM_MAKEFLAGS) jq.1;\
fi
else
jq.1: $(srcdir)/jq.1.prebuilt
$(AM_V_GEN) cp $^ $@
real_docs:
@echo "Ruby dependencies not found, cannot build manpage." > /dev/stderr
@echo "Follow the instructions in docs/README.md to install them" > /dev/stderr
@echo "and then rerun ./configure" > /dev/stderr
@echo "A pre-built, possibly out-of-date manpage used." > /dev/stderr
false
endif
### Packaging
docs/site.yml: configure.ac
sed 's/^jq_version: .*/jq_version: "$(VERSION)"/' $@ > $@.new
mv $@.new $@
install-binaries: $(BUILT_SOURCES)
$(MAKE) $(AM_MAKEFLAGS) install-exec
# Ensure "make dist" fails when we can't build the real manpage
dist-hook: real_docs
DOC_FILES = docs/content docs/public docs/templates docs/site.yml \
docs/Gemfile docs/Gemfile.lock docs/Rakefile docs/README.md \
docs/default_manpage.md jq.1.default
# setup is only used by distribution developers, not package developers.
# Still, as a matter of allowing patching, its not a bad idea to distribute
# the developer setup script in the tarball.
EXTRA_DIST = $(man_MANS) $(TESTS) $(TEST_LOG_COMPILER) jq.spec \
$(DOC_FILES) scripts/version parser.h parser.c lexer.h \
lexer.c tests/onig.supp tests/torture/input0.json \
tests/modules/.jq tests/modules/a.jq tests/modules/b/b.jq \
tests/modules/c/c.jq tests/modules/c/d.jq \
tests/modules/lib/jq/e/e.jq tests/modules/lib/jq/f.jq \
tests/modules/data.json \
tests/modules/syntaxerror/syntaxerror.jq \
tests/modules/test_bind_order.jq \
tests/modules/test_bind_order0.jq \
tests/modules/test_bind_order1.jq \
tests/modules/test_bind_order2.jq
# README.md is expected in Github projects, good stuff in it, so we'll
# distribute it and install it with the package in the doc directory.
docdir = ${datadir}/doc/${PACKAGE}
dist_doc_DATA = README.md COPYING AUTHORS README
RELEASE ?= 1
rpm: dist jq.spec
@echo "Packaging jq as an RPM ..."
mkdir -p rpm/SOURCES rpm/BUILD rpm/BUILDROOT rpm/RPMS rpm/SPECS
cp jq-$(VERSION).tar.gz rpm/SOURCES/
rpmbuild -tb --define "_topdir ${PWD}/rpm" --define "_prefix /usr" --define "myver $(VERSION)" --define "myrel ${RELEASE}" rpm/SOURCES/jq-$(VERSION).tar.gz
find rpm/RPMS/ -name "*.rpm" -exec mv {} ./ \;
rm -rf rpm
dist-clean-local:
rm -f ${BUILT_SOURCES}
# Not sure why this doesn't get cleaned up automatically, guess
# automake used to man pages which are hand coded?
# 'make clean' doesn't delete the manpage if it can't be rebuilt
clean-local-docs:
if ENABLE_DOCS
rm -f jq.1
endif
clean-local-gcov:
rm -f *.gcno *.gcda *.gcov
clean-local: clean-local-docs clean-local-gcov
rm -f version.h .remake-version-h
.PHONY: clean-local-docs clean-local-gcov
+88
View File
@@ -0,0 +1,88 @@
Release history
* jq version 1.0 was released on Sun Oct 21 2012
* jq version 1.1 was released on Sun Oct 21 2012
* jq version 1.2 was released on Thu Dec 20 2012
* jq version 1.3 was released on Sun May 19 2013
* jq version 1.4 was released on Mon Jun 9 2014
New features in 1.5 since 1.4:
- regular expressions (with Oniguruma)
- a library/module system
- many new builtins
- datetime builtins
- math builtins
- regexp-related builtins
- stream-related builtins (e.g., all/1, any/1)
- minimal I/O builtins (`inputs`, `debug`)
- new syntactic features, including:
- destructuring (`. as [$first, $second] | ...`)
- try/catch, generalized `?` operator, and label/break
- `foreach`
- multiple definitions of a function with different numbers of
arguments
- command-line arguments
- --join-lines / -j for raw output
- --argjson and --slurpfile
- --tab and --indent
- --stream (streaming JSON parser)
- --seq (RFC7464 JSON text sequence)
- --run-tests improvements
- optimizations:
- tail-call optimization
- reduce and foreach no longer leak a reference to .
New features in 1.4 since 1.3:
- command-line arguments
- jq --arg-file variable file
- jq --unbuffered
- jq -e / --exit-status (set exit status based on outputs)
- jq -S / --sort-keys (now jq no longer sorts object keys by
default
- syntax
- .. -> like // in XPath (recursive traversal)
- question mark (e.g., .a?) to suppress errors
- ."foo" syntax (equivalent to .["foo"])
- better error handling for .foo
- added % operator (modulo)
- allow negation without requiring extra parenthesis
- more function arguments (up to six)
- filters:
- any, all
- iterables, arrays, objects, scalars, nulls, booleans, numbers,
strings, values
- string built-ins:
- split
- join (join an array of strings with a given separator string)
- ltrimstr, rtrimstr
- startswith, endswith
- explode, implode
- fromjson, tojson
- index, rindex, indices
- math functions
- floor, sqrt, cbrt, etetera (depends on what's available from libm)
- libjq -- a C API interface to jq's JSON representation and for
running jq programs from C applications
-121
View File
@@ -1,121 +0,0 @@
{
module Parser where
import Lexer
import JQ
import Text.JSON
import Debug.Trace
import Data.List
import Control.Monad.Error
import Control.Monad.Reader
instance Error (Maybe a) where
noMsg = Nothing
strMsg = const Nothing
instance (Error a, Error b) => Error (a, b) where
noMsg = (noMsg, noMsg)
strMsg s = (strMsg s, strMsg s)
}
%name runParser TopLevel
%tokentype { Token }
%monad { Either String }
%error { \t -> throwError $ "parse error: unexpected " ++ if null t then [] else (show $ t) }
%token
'|' { TRes "|" }
'.' { TRes "." }
'[' { TRes "[" }
']' { TRes "]" }
'{' { TRes "{" }
'}' { TRes "}" }
'(' { TRes "(" }
')' { TRes ")" }
'$' { TRes "$" }
'as' { TRes "as" }
',' { TRes "," }
':' { TRes ":" }
'==' { TRes "==" }
'=' { TRes "=" }
'+' { TRes "+" }
'def' { TRes "def" }
';' { TRes ";" }
'else' { TRes "else" }
'and' { TRes "and" }
'or' { TRes "or" }
Ident { TIdent $$ }
String { TString $$ }
Int { TInt $$ }
%left 'else'
%right '|'
%left '='
%left ','
%left 'and' 'or'
%nonassoc '=='
%left '+'
%%
TopLevel
: Defn TopLevel { withDefn $1 $2 }
| Exp { $1 }
Exp
: Exp 'else' Exp { $1 `elsep` $3 }
| Assign '|' Exp { do { v <- snd $1; setvar (fst $1) v $3 } }
| Exp '|' Exp { $1 >| $3 }
| Exp ',' Exp { $1 `mplus` $3 }
| Exp 'and' Exp { liftM2 andj $1 $3 }
| Exp 'or' Exp { liftM2 orj $1 $3 }
| Exp '=' Exp { assignp $1 $3 }
| Exp '==' Exp { liftM2 eqj (subexp $1) (subexp $3) }
| Exp '+' Exp { liftM2 plusj (subexp $1) (subexp $3) }
| Term { $1 }
Assign
: Term 'as' '$' Ident { ($4, $1) }
Defn
: 'def' Ident '=' Exp ';' { ($2, [], $4) }
| 'def' Ident '(' ParamList ')' '=' Exp ';' { ($2, $4, $7) }
ParamList
: { [] }
| Ident { [$1] }
| Ident ';' ParamList { $1:$3 }
ExpD
: ExpD '|' ExpD { $1 >| $3 }
| ExpD '==' ExpD { liftM2 eqj $1 $3 }
| Term { $1 }
Term
: '.' { input }
| Term '.' Ident { $1 >| index $3 }
| '.' Ident { index $2 }
| String { constStr $1 }
| Term '[' Exp ']' { do {t <- $1; i <- subexp $3; lookupj t i} }
| Term '[' ']' { $1 >| childp }
| '(' Exp ')' { $2 }
| '[' Exp ']' { arrayp $2 }
| '[' ']' { arrayp (callp "empty" []) }
| Int { constInt $1 }
| '{' MkDict '}' { dictp $2 }
| Ident '(' Exp ')' { callp $1 [$3] }
| Ident { callp $1 [] }
| '$' Ident { getvar $2 }
MkDict
: { [] }
| MkDictPair { [$1] }
| MkDictPair ',' MkDict { $1:$3 }
MkDictPair
: Ident ':' ExpD { (constStr $1, $3) }
| Ident { (constStr $1, index $1) }
| String ':' ExpD { (constStr $1, $3) }
| '(' Exp ')' ':' ExpD{ ($2, $5) }
-12
View File
@@ -1,12 +0,0 @@
module PrettyJSON where
import Text.JSON
import Text.PrettyPrint
renderJSON (JSArray vals) = brackets $ fsep $ punctuate comma $ map renderJSON vals
renderJSON (JSObject jsObject) =
let object = fromJSObject jsObject
in braces $ fsep $ punctuate comma $
[hang (renderJSON (JSString $ toJSString $ k) <> colon) 2 (renderJSON v)
| (k,v) <- object]
renderJSON x = text $ encode x
Symlink
+1
View File
@@ -0,0 +1 @@
README.md
+64
View File
@@ -0,0 +1,64 @@
jq
==
jq is a command-line JSON processor.
If you want to learn to use jq, read the documentation at
[http://stedolan.github.io/jq](http://stedolan.github.io/jq). This
documentation is generated from the docs/ folder of this repository.
You can also try it online at [jqplay.org](http://jqplay.org).
If you want to hack on jq, feel free, but be warned that its internals
are not well-documented at the moment. Bring a hard hat and a
shovel. Also, read the wiki: http://github.com/stedolan/jq/wiki
Source tarball and built executable releases can be found on the
homepage and on the github release page, https://github.com/stedolan/jq/releases
If you're building directly from the latest git, you'll need flex,
bison (3.0 or newer), libtool, make, and autoconf installed. To get
regexp support you'll also need to install Oniguruma (note that jq's
tests require regexp support to pass). To build, run:
autoreconf -i # if building from git
./configure
make -j8
make check
To build without bison or flex, add `--disable-maintainer-mode` to the
./configure invocation:
./configure --disable-maintainer-mode
(Developers must not use `--disable-maintainer-mode`, not when making
changes to the jq parser and/or lexer.)
To build a statically linked version of jq, run:
make LDFLAGS=-all-static
After make finishes, you'll be able to use `./jq`. You can also
install it using:
sudo make install
If you're not using the latest git version but instead building a
released tarball (available on the website), then you won't need to
run `autoreconf` (and shouldn't), and you won't need flex or bison.
To cross-compile for OS X and Windows, see docs/Rakefile's build task
and scripts/crosscompile. You'll need a cross-compilation environment,
such as Mingw for cross-compiling for Windows.
Cross-compilation requires a clean workspace, then:
# git clean ...
autoreconf -i
./configure
make distclean
scripts/crosscompile <name-of-build> <configure-options>
Use the --host= and --target= ./configure options to select a
cross-compilation environment. See also the wiki.
Send questions to http://stackoverflow.com/questions/tagged/jq or to the #jq channel (http://irc.lc/freenode/%23jq/) on Freenode (https://webchat.freenode.net/).
+1
View File
@@ -0,0 +1 @@
*
+1684
View File
File diff suppressed because it is too large. Load diff
+10
View File
@@ -0,0 +1,10 @@
#ifndef BUILTIN_H
#define BUILTIN_H
#include "jq.h"
#include "bytecode.h"
#include "compile.h"
int builtins_bind(jq_state *, block*);
#endif
+161
View File
@@ -0,0 +1,161 @@
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include "bytecode.h"
#include "jv_alloc.h"
// flags, length
#define NONE 0, 1
#define CONSTANT OP_HAS_CONSTANT, 2
#define VARIABLE (OP_HAS_VARIABLE | OP_HAS_BINDING), 3
#define GLOBAL (OP_HAS_CONSTANT | OP_HAS_VARIABLE | OP_HAS_BINDING | OP_IS_CALL_PSEUDO), 4
#define BRANCH OP_HAS_BRANCH, 2
#define CFUNC (OP_HAS_CFUNC | OP_HAS_BINDING), 3
#define UFUNC (OP_HAS_UFUNC | OP_HAS_BINDING | OP_IS_CALL_PSEUDO), 4
#define DEFINITION (OP_IS_CALL_PSEUDO | OP_HAS_BINDING), 0
#define CLOSURE_REF_IMM (OP_IS_CALL_PSEUDO | OP_HAS_BINDING), 2
#define OP(name, imm, in, out) \
{name, #name, imm, in, out},
static const struct opcode_description opcode_descriptions[] = {
#include "opcode_list.h"
};
static const struct opcode_description invalid_opcode_description = {
-1, "#INVALID", 0, 0, 0, 0
};
const struct opcode_description* opcode_describe(opcode op) {
if ((int)op >= 0 && (int)op < NUM_OPCODES) {
return &opcode_descriptions[op];
} else {
return &invalid_opcode_description;
}
}
int bytecode_operation_length(uint16_t* codeptr) {
int length = opcode_describe(*codeptr)->length;
if (*codeptr == CALL_JQ || *codeptr == TAIL_CALL_JQ) {
length += codeptr[1] * 2;
}
return length;
}
static void dump_code(int indent, struct bytecode* bc) {
int pc = 0;
while (pc < bc->codelen) {
printf("%*s", indent, "");
dump_operation(bc, bc->code + pc);
printf("\n");
pc += bytecode_operation_length(bc->code + pc);
}
}
static void symbol_table_free(struct symbol_table* syms) {
jv_mem_free(syms->cfunctions);
jv_free(syms->cfunc_names);
jv_mem_free(syms);
}
void dump_disassembly(int indent, struct bytecode* bc) {
if (bc->nclosures > 0) {
printf("%*s[params: ", indent, "");
jv params = jv_object_get(jv_copy(bc->debuginfo), jv_string("params"));
for (int i=0; i<bc->nclosures; i++) {
if (i) printf(", ");
jv name = jv_array_get(jv_copy(params), i);
printf("%s", jv_string_value(name));
jv_free(name);
}
jv_free(params);
printf("]\n");
}
dump_code(indent, bc);
for (int i=0; i<bc->nsubfunctions; i++) {
struct bytecode* subfn = bc->subfunctions[i];
jv name = jv_object_get(jv_copy(subfn->debuginfo), jv_string("name"));
printf("%*s%s:%d:\n", indent, "", jv_string_value(name), i);
jv_free(name);
dump_disassembly(indent+2, subfn);
}
}
static struct bytecode* getlevel(struct bytecode* bc, int level) {
while (level > 0) {
bc = bc->parent;
level--;
}
return bc;
}
void dump_operation(struct bytecode* bc, uint16_t* codeptr) {
int pc = codeptr - bc->code;
printf("%04d ", pc);
const struct opcode_description* op = opcode_describe(bc->code[pc++]);
printf("%s", op->name);
if (op->length > 1) {
uint16_t imm = bc->code[pc++];
if (op->op == CALL_JQ || op->op == TAIL_CALL_JQ) {
for (int i=0; i<imm+1; i++) {
uint16_t level = bc->code[pc++];
uint16_t idx = bc->code[pc++];
jv name;
if (idx & ARG_NEWCLOSURE) {
idx &= ~ARG_NEWCLOSURE;
name = jv_object_get(jv_copy(getlevel(bc,level)->subfunctions[idx]->debuginfo),
jv_string("name"));
} else {
name = jv_array_get(jv_object_get(jv_copy(getlevel(bc,level)->debuginfo),
jv_string("params")), idx);
}
printf(" %s:%d",
jv_string_value(name),
idx);
jv_free(name);
if (level) {
printf("^%d", level);
}
}
} else if (op->op == CALL_BUILTIN) {
int func = bc->code[pc++];
jv name = jv_array_get(jv_copy(bc->globals->cfunc_names), func);
printf(" %s", jv_string_value(name));
jv_free(name);
} else if (op->flags & OP_HAS_BRANCH) {
printf(" %04d", pc + imm);
} else if (op->flags & OP_HAS_CONSTANT) {
printf(" ");
jv_dump(jv_array_get(jv_copy(bc->constants), imm), 0);
} else if (op->flags & OP_HAS_VARIABLE) {
uint16_t v = bc->code[pc++];
jv name = jv_array_get(jv_object_get(jv_copy(getlevel(bc,imm)->debuginfo), jv_string("locals")), v);
printf(" $%s:%d",
jv_string_value(name),
v);
jv_free(name);
if (imm) {
printf("^%d", imm);
}
} else {
printf(" %d", imm);
}
}
}
void bytecode_free(struct bytecode* bc) {
if (!bc)
return;
jv_mem_free(bc->code);
jv_free(bc->constants);
for (int i=0; i<bc->nsubfunctions; i++)
bytecode_free(bc->subfunctions[i]);
if (!bc->parent)
symbol_table_free(bc->globals);
jv_mem_free(bc->subfunctions);
jv_free(bc->debuginfo);
jv_mem_free(bc);
}
+92
View File
@@ -0,0 +1,92 @@
#ifndef BYTECODE_H
#define BYTECODE_H
#include <stdint.h>
#include "jv.h"
typedef enum {
#define OP(name, imm, in, out) name,
#include "opcode_list.h"
#undef OP
} opcode;
enum {
NUM_OPCODES =
#define OP(name, imm, in, out) +1
#include "opcode_list.h"
#undef OP
};
enum {
OP_HAS_CONSTANT = 2,
OP_HAS_VARIABLE = 4,
OP_HAS_BRANCH = 8,
OP_HAS_CFUNC = 32,
OP_HAS_UFUNC = 64,
OP_IS_CALL_PSEUDO = 128,
OP_HAS_BINDING = 1024,
// NOTE: Not actually part of any op -- a pseudo-op flag for special
// handling of `break`.
OP_BIND_WILDCARD = 2048,
};
struct opcode_description {
opcode op;
const char* name;
int flags;
// length in 16-bit units
int length;
int stack_in, stack_out;
};
const struct opcode_description* opcode_describe(opcode op);
#define MAX_CFUNCTION_ARGS 10
typedef void (*cfunction_ptr)();
struct cfunction {
cfunction_ptr fptr;
const char* name;
int nargs;
};
struct symbol_table {
struct cfunction* cfunctions;
int ncfunctions;
jv cfunc_names;
};
// The bytecode format matters in:
// execute.c - interpreter
// compile.c - compiler
// bytecode.c - disassembler
#define ARG_NEWCLOSURE 0x1000
struct bytecode {
uint16_t* code;
int codelen;
int nlocals;
int nclosures;
jv constants; // JSON array of constants
struct symbol_table* globals;
struct bytecode** subfunctions;
int nsubfunctions;
struct bytecode* parent;
jv debuginfo;
};
void dump_disassembly(int, struct bytecode* code);
void dump_operation(struct bytecode* bc, uint16_t* op);
int bytecode_operation_length(uint16_t* codeptr);
void bytecode_free(struct bytecode* bc);
#endif
Executable
+102
View File
@@ -0,0 +1,102 @@
#!/bin/bash
# old values
OLD_CFLAGS=$CFLAGS
OLD_LDFLAGS=$LDFLAGS
ORIG_PWD=`pwd`
# For parallelism in make
NJOBS="-j`sysctl -n hw.ncpu || echo 1`"
# Get oniguruma
rm -rf $PWD/build/ios onig-5.9.5
echo "Downloading oniguruma 5.9.5"
curl http://www.geocities.jp/kosako3/oniguruma/archive/onig-5.9.5.tar.gz | tar xz
cd onig-5.9.5
# So, we need to remake the configure scripts so that the arm64 architecture
# exists in config.sub. In order to keep autoreconf from failing, create
# NEWS and ChangeLog.
touch NEWS ChangeLog
autoreconf -fi >/dev/null 2>&1
CC=`xcrun -f clang`
cd $ORIG_PWD
autoreconf -fi
for arch in i386 x86_64 armv7 armv7s arm64; do
# Some of the architectures are a bit different...
if [[ "$arch" = "i386" || "$arch" = "x86_64" ]]
then
SYSROOT=`xcrun -f --sdk iphonesimulator --show-sdk-path`
else
SYSROOT=`xcrun -f --sdk iphoneos --show-sdk-path`
fi
if [[ "$arch" = "arm64" ]]
then
HOST="aarch64-apple-darwin"
else
HOST="$arch-apple-darwin"
fi
CFLAGS="-arch $arch -miphoneos-version-min=6.0 -isysroot $SYSROOT $OLD_CFLAGS"
LDFLAGS="-arch $arch -miphoneos-version-min=6.0 -isysroot $SYSROOT $OLD_LDFLAGS"
# Build oniguruma for this architecture
cd onig-5.9.5
CC=$CC CFLAGS=$CFLAGS LDFLAGS=$LDFLAGS ./configure --disable-shared --enable-static --host=$HOST --prefix=$ORIG_PWD/build/ios/$arch
STATUS=$?
if [ $STATUS -ne 0 ]
then
echo "Failed to configure oniguruma for architecture $arch. Check `pwd`/config.log for details."
cd $PWD
exit $STATUS
fi
make clean
make $NJBOS install
STATUS=$?
if [ $STATUS -ne 0 ]
then
echo "Failed to make oniguruma for architecture $arch."
cd $PWD
exit $STATUS
fi
# Build jq for this architecture
cd $ORIG_PWD
CC=$CC CFLAGS=$CFLAGS LDFLAGS=$LDFLAGS ./configure --disable-shared --enable-static -host=$HOST --prefix=$ORIG_PWD/build/ios/$arch --with-oniguruma=$ORIG_PWD/build/ios/$arch
STATUS=$?
if [ $STATUS -ne 0 ]
then
echo "Failed to configure jq for architecture $arch"
exit $STATUS
fi
make clean
make $NJOBS install
STATUS=$?
if [ $STATUS -ne 0 ]
then
echo "Failed to make jq for architecture $arch"
exit $STATUS
fi
done
# lipo together the different architectures into a universal 'fat' file
lipo -create -output $ORIG_PWD/build/ios/libonig.a $ORIG_PWD/build/ios/{i386,x86_64,armv7,armv7s,arm64}/lib/libonig.a
lipo -create -output $ORIG_PWD/build/ios/libjq.a $ORIG_PWD/build/ios/{i386,x86_64,armv7,armv7s,arm64}/lib/libjq.a
# copy the products into the destination directory and clean up the single-architecture files.
cp $ORIG_PWD/build/ios/i386/include/*.h $ORIG_PWD/build/ios/
rm -rf $ORIG_PWD/build/ios/{i386,x86_64,armv7,armv7s,arm64}
echo "Products are in $ORIG_PWD/build/ios"
+1210
View File
File diff suppressed because it is too large. Load diff
+101
View File
@@ -0,0 +1,101 @@
#ifndef COMPILE_H
#define COMPILE_H
#include <stdint.h>
#include "jv.h"
#include "bytecode.h"
#include "locfile.h"
struct inst;
typedef struct inst inst;
typedef struct block {
inst* first;
inst* last;
} block;
block gen_location(location, struct locfile*, block);
block gen_noop();
int block_is_noop(block b);
block gen_op_simple(opcode op);
block gen_const(jv constant);
block gen_const_global(jv constant, const char *name);
int block_is_const(block b);
int block_is_const_inf(block b);
jv_kind block_const_kind(block b);
jv block_const(block b);
block gen_op_target(opcode op, block target);
block gen_op_unbound(opcode op, const char* name);
block gen_op_bound(opcode op, block binder);
block gen_op_var_fresh(opcode op, const char* name);
block gen_module(block metadata);
jv block_module_meta(block b);
block gen_import(const char* name, block metadata, const char *as, int is_data);
block gen_function(const char* name, block formals, block body);
block gen_param_regular(const char* name);
block gen_param(const char* name);
block gen_lambda(block body);
block gen_call(const char* name, block body);
block gen_subexp(block a);
block gen_both(block a, block b);
block gen_const_object(block expr);
block gen_collect(block expr);
block gen_reduce(block source, block matcher, block init, block body);
block gen_foreach(block source, block matcher, block init, block update, block extract);
block gen_definedor(block a, block b);
block gen_condbranch(block iftrue, block iffalse);
block gen_and(block a, block b);
block gen_or(block a, block b);
block gen_var_binding(block var, const char* name, block body);
block gen_array_matcher(block left, block curr);
block gen_object_matcher(block name, block curr);
block gen_destructure(block var, block matcher, block body);
block gen_cond(block cond, block iftrue, block iffalse);
block gen_try_handler(block handler);
block gen_try(block exp, block handler);
block gen_label(const char *label, block exp);
block gen_cbinding(const struct cfunction* functions, int nfunctions, block b);
void block_append(block* b, block b2);
block block_join(block a, block b);
int block_has_only_binders_and_imports(block, int bindflags);
int block_has_only_binders(block, int bindflags);
int block_has_main(block);
int block_is_funcdef(block b);
int block_is_single(block b);
block block_bind(block binder, block body, int bindflags);
block block_bind_library(block binder, block body, int bindflags, const char* libname);
block block_bind_referenced(block binder, block body, int bindflags);
block block_drop_unreferenced(block body);
jv block_take_imports(block* body);
int block_compile(block, struct bytecode**, struct locfile*);
void block_free(block);
// Here's some horrible preprocessor gunk so that code
// sequences can be contructed as BLOCK(block1, block2, block3)
#define BLOCK_1(b1) (b1)
#define BLOCK_2(b1,b2) (block_join((b1),(b2)))
#define BLOCK_3(b1,b2,b3) (block_join(BLOCK_2(b1,b2),(b3)))
#define BLOCK_4(b1,b2,b3,b4) (block_join(BLOCK_3(b1,b2,b3),(b4)))
#define BLOCK_5(b1,b2,b3,b4,b5) (block_join(BLOCK_4(b1,b2,b3,b4),(b5)))
#define BLOCK_6(b1,b2,b3,b4,b5,b6) (block_join(BLOCK_5(b1,b2,b3,b4,b5),(b6)))
#define BLOCK_7(b1,b2,b3,b4,b5,b6,b7) (block_join(BLOCK_6(b1,b2,b3,b4,b5,b6),(b7)))
#define BLOCK_8(b1,b2,b3,b4,b5,b6,b7,b8) (block_join(BLOCK_7(b1,b2,b3,b4,b5,b6,b7),(b8)))
#define BLOCK_IDX(_1,_2,_3,_4,_5,_6,_7,_8,NAME,...) NAME
#define BLOCK(...) \
BLOCK_IDX(__VA_ARGS__, BLOCK_8, BLOCK_7, BLOCK_6, BLOCK_5, BLOCK_4, BLOCK_3, BLOCK_2, BLOCK_1)(__VA_ARGS__)
#endif
+10
View File
@@ -0,0 +1,10 @@
config.guess
config.sub
compile
depcomp
install-sh
ltmain.sh
missing
ylwrap
test-driver
ar-lib
+2
View File
@@ -0,0 +1,2 @@
libtool.m4
lt*.m4
+4
View File
@@ -0,0 +1,4 @@
dnl AC_FIND_FUNC(func, arguments)
AC_DEFUN([AC_CHECK_MATH_FUNC], [
AC_FIND_FUNC_NO_LIBS([$1], [m], [#include <math.h>], [$2])
])
+8
View File
@@ -0,0 +1,8 @@
dnl Shamelessly stolen from Heimdal
dnl
dnl Look for function in any of the specified libraries
dnl
dnl AC_FIND_FUNC_NO_LIBS(func, libraries, includes, arguments, extra libs, extra args)
AC_DEFUN([AC_FIND_FUNC_NO_LIBS], [
AC_FIND_FUNC_NO_LIBS2([$1], ["" $2], [$3], [$4], [$5], [$6])])
+62
View File
@@ -0,0 +1,62 @@
dnl Shamelessly stolen from Heimdal
dnl
dnl Look for function in any of the specified libraries
dnl
dnl AC_FIND_FUNC_NO_LIBS2(func, libraries, includes, arguments, extra libs, extra args)
AC_DEFUN([AC_FIND_FUNC_NO_LIBS2], [
AC_MSG_CHECKING([for $1])
AC_CACHE_VAL(ac_cv_funclib_$1,
[
if eval "test \"\$ac_cv_func_$1\" != yes" ; then
ac_save_LIBS="$LIBS"
for ac_lib in $2; do
case "$ac_lib" in
"") ;;
yes) ac_lib="" ;;
no) continue ;;
-l*) ;;
*) ac_lib="-l$ac_lib" ;;
esac
LIBS="$6 $ac_lib $5 $ac_save_LIBS"
AC_LINK_IFELSE([AC_LANG_PROGRAM([[$3]],[[$1($4)]])],[eval "if test -n \"$ac_lib\";then ac_cv_funclib_$1=$ac_lib; else ac_cv_funclib_$1=yes; fi";break])
done
eval "ac_cv_funclib_$1=\${ac_cv_funclib_$1-no}"
LIBS="$ac_save_LIBS"
fi
])
eval "ac_res=\$ac_cv_funclib_$1"
if false; then
AC_CHECK_FUNCS($1)
dnl AC_CHECK_LIBS($2, foo)
fi
# $1
eval "ac_tr_func=HAVE_[]upcase($1)"
eval "ac_tr_lib=HAVE_LIB[]upcase($ac_res | sed -e 's/-l//')"
eval "LIB_$1=$ac_res"
case "$ac_res" in
yes)
eval "ac_cv_func_$1=yes"
eval "LIB_$1="
AC_DEFINE_UNQUOTED($ac_tr_func)
AC_MSG_RESULT([yes])
;;
no)
eval "ac_cv_func_$1=no"
eval "LIB_$1="
AC_MSG_RESULT([no])
;;
*)
eval "ac_cv_func_$1=yes"
eval "ac_cv_lib_`echo "$ac_res" | sed 's/-l//'`=yes"
AC_DEFINE_UNQUOTED($ac_tr_func)
AC_DEFINE_UNQUOTED($ac_tr_lib)
AC_MSG_RESULT([yes, in $ac_res])
;;
esac
AC_SUBST(LIB_$1)
])
+9
View File
@@ -0,0 +1,9 @@
dnl Shamelessly stolen from Heimdal
dnl
dnl AC_FIND_FUNC(func, libraries, includes, arguments)
AC_DEFUN([AC_FIND_FUNC], [
AC_FIND_FUNC_NO_LIBS([$1], [$2], [$3], [$4])
if test -n "$LIB_$1"; then
LIBS="$LIB_$1 $LIBS"
fi
])
+3
View File
@@ -0,0 +1,3 @@
dnl Shamelessly stolen from Heimdal
AC_DEFUN([upcase],[`echo $1 | tr abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ`])dnl
+221
View File
@@ -0,0 +1,221 @@
m4_define([jq_version], m4_esyscmd_s([git describe --tags --dirty --match 'jq-*'|sed 's/^jq-//']))
AC_INIT([jq], [jq_version], [https://github.com/stedolan/jq/issues],
[jq], [http://stedolan.github.io/jq])
m4_include([m4/ax_compare_version.m4])
m4_include([m4/ax_prog_bison_version.m4])
dnl Created autoconf implementation thompson@dtosolutions, 26NOV12
AC_PREREQ([2.64])
AC_CONFIG_AUX_DIR([config])
AM_INIT_AUTOMAKE([1.11.2 parallel-tests foreign -Wall])
AM_SILENT_RULES([yes])
AM_PROG_AR
AM_MAINTAINER_MODE([enable])
AC_PROG_CC
AC_PROG_CC_STDC
AC_PROG_CPP_WERROR
AC_PROG_YACC
AC_OBJEXT
AC_EXEEXT
LT_INIT([shared static win32-dll])
AM_PROG_CC_C_O
dnl couldn't use AM_PROG_LEX as it doesn't support header files like the
dnl AC_PROG_YACC macros...
dnl check bison version
if test "$USE_MAINTAINER_MODE" = yes; then
if test "$YACC" != "bison -y"; then
AC_MSG_CHECKING([bison version])
AC_MSG_RESULT([not bison])
else
AX_PROG_BISON_VERSION([3],
[],
[AC_MSG_ERROR([You need bison version 3.0 or greater, or use --disable-maintainer-mode.])])
fi
AC_PROG_LEX
if test "x$LEX" != xflex; then
LEX="$SHELL $missing_dir/missing flex"
fi
fi
##########################################################################
# check for ONIGURUMA library
##########################################################################
AC_ARG_WITH([oniguruma],
[AS_HELP_STRING([--with-oniguruma=prefix],
[try this for a non-standard install prefix of the oniguruma library])],
[ONIGURUMAPATHSET=1],
[ONIGURUMAPATHSET=0])
if test $ONIGURUMAPATHSET = 1; then
CFLAGS="$CFLAGS -I${with_oniguruma}/include"
LDFLAGS="$LDFLAGS -L${with_oniguruma}/lib"
fi
# check for ONIGURUMA library
HAVE_ONIGURUMA=0
AC_CHECK_HEADER("oniguruma.h",
AC_CHECK_LIB([onig],[onig_version],[LIBS="$LIBS -lonig"; HAVE_ONIGURUMA=1;]))
# handle check results
if test $HAVE_ONIGURUMA != 1; then
AC_MSG_NOTICE([Oniguruma was not found.])
AC_MSG_NOTICE([ Try setting the location using '--with-oniguruma=PREFIX' ])
else
AC_DEFINE([HAVE_ONIGURUMA],1,[Define to 1 if Oniguruma is installed])
fi
dnl Check for valgrind
AC_CHECK_PROGS(valgrind_cmd, valgrind)
if test "x$valgrind_cmd" = "x" ; then
AC_MSG_WARN([valgrind is required to test jq.])
fi
AC_CHECK_FUNCS(memmem)
AC_CHECK_FUNCS(mkstemp)
dnl Running tests with Valgrind is slow. It is faster to iterate on
dnl code without Valgrind until tests pass, then enable Valgrind and
dnl fix leaks.
AC_ARG_ENABLE([valgrind],
AC_HELP_STRING([--disable-valgrind], [do not run tests under Valgrind]))
dnl Code coverage
AC_ARG_ENABLE([gcov],
AC_HELP_STRING([--enable-gcov], [enable gcov code coverage tool]))
dnl Don't attempt to build docs if there's no Ruby lying around
AC_ARG_ENABLE([docs],
AC_HELP_STRING([--disable-docs], [don't build docs]))
dnl Don't attempt to build the error injection object (if there is no LD_PRELOAD support)
AC_ARG_ENABLE([error-injection],
AC_HELP_STRING([--enable-error-injection], [build and test with error injection]))
dnl Enable building all static
AC_ARG_ENABLE([all-static],
AC_HELP_STRING([--enable-all-static], [link jq with static libraries only]))
AS_IF([test "x$enable_docs" != "xno"],[
AC_CHECK_PROGS(bundle_cmd, bundle)
AC_CACHE_CHECK([for Ruby dependencies], [jq_cv_ruby_deps],
[jq_cv_ruby_deps=yes;
AS_IF([test "x$bundle_cmd" = "x" || \
! bmsg="`cd ${srcdir}/docs; "$bundle_cmd" check 2>/dev/null`"],[
AC_MSG_WARN([$bmsg])
cat <<EOF
*****************************************************************
* Ruby dependencies for building jq documentation not found. *
* You can still build, install and hack on jq, but the manpage *
* will not be rebuilt and some of the tests won't run. *
* See docs/README.md for how to install the docs dependencies. *
*****************************************************************
EOF
jq_cv_ruby_deps=no
])])
if test "x$jq_cv_ruby_deps" != "xyes"; then
enable_docs=no
fi
])
AM_CONDITIONAL([ENABLE_VALGRIND], [test "x$enable_valgrind" != xno])
AM_CONDITIONAL([ENABLE_GCOV], [test "x$enable_gcov" = xyes])
AM_CONDITIONAL([ENABLE_DOCS], [test "x$enable_docs" != xno])
AM_CONDITIONAL([ENABLE_ERROR_INJECTION], [test "x$enable_error_injection" = xyes])
AM_CONDITIONAL([ENABLE_ALL_STATIC], [test "x$enable_all_static" = xyes])
AC_FUNC_ALLOCA
AC_FIND_FUNC([isatty], [c], [#include <unistd.h>], [0])
AC_FIND_FUNC([_isatty], [c], [#include <io.h>], [0])
AC_FIND_FUNC([strptime], [c], [#include <time.h>], [0, 0, 0])
AC_FIND_FUNC([strftime], [c], [#include <time.h>], [0, 0, 0, 0])
AC_FIND_FUNC([timegm], [c], [#include <time.h>], [0])
AC_FIND_FUNC([gmtime_r], [c], [#include <time.h>], [0, 0])
AC_FIND_FUNC([gmtime], [c], [#include <time.h>], [0])
AC_FIND_FUNC([gettimeofday], [c], [#include <time.h>], [0, 0])
AC_CHECK_MEMBER([struct tm.tm_gmtoff], [AC_DEFINE([HAVE_TM_TM_GMT_OFF],1,[Define to 1 if the system has the tm_gmt_off field in struct tm])],
[], [[#include <time.h>]])
AC_CHECK_MEMBER([struct tm.__tm_gmtoff], [AC_DEFINE([HAVE_TM___TM_GMT_OFF],1,[Define to 1 if the system has the __tm_gmt_off field in struct tm])],
[], [[#include <time.h>]])
AC_ARG_ENABLE([pthread-tls],
[AC_HELP_STRING([--enable-pthread-tls],
[Enable use of pthread thread local storage])],
[],
[enable_pthread_tls=no])
if test $enable_pthread_tls = yes; then
AC_FIND_FUNC([pthread_key_create], [pthread], [#include <pthread.h>], [NULL, NULL])
AC_FIND_FUNC([pthread_once], [pthread], [#include <pthread.h>], [NULL, NULL])
AC_FIND_FUNC([atexit], [pthread], [#include <stdlib.h>], [NULL])
fi
dnl libm math.h functions
AC_CHECK_MATH_FUNC(acos, [.5])
AC_CHECK_MATH_FUNC(acosh, [.5])
AC_CHECK_MATH_FUNC(asin, [.5])
AC_CHECK_MATH_FUNC(asinh, [.5])
AC_CHECK_MATH_FUNC(atan, [.5])
AC_CHECK_MATH_FUNC(atanh, [.5])
AC_CHECK_MATH_FUNC(cbrt, [.5])
AC_CHECK_MATH_FUNC(cos, [.5])
AC_CHECK_MATH_FUNC(cosh, [.5])
AC_CHECK_MATH_FUNC(exp2, [.5])
AC_CHECK_MATH_FUNC(exp, [.5])
AC_CHECK_MATH_FUNC(floor, [.5])
AC_CHECK_MATH_FUNC(j0, [.5])
AC_CHECK_MATH_FUNC(j1, [.5])
AC_CHECK_MATH_FUNC(log10, [.5])
AC_CHECK_MATH_FUNC(log2, [.5])
AC_CHECK_MATH_FUNC(log, [.5])
AC_CHECK_MATH_FUNC(sin, [.5])
AC_CHECK_MATH_FUNC(sinh, [.5])
AC_CHECK_MATH_FUNC(sqrt, [.5])
AC_CHECK_MATH_FUNC(tan, [.5])
AC_CHECK_MATH_FUNC(tanh, [.5])
AC_CHECK_MATH_FUNC(tgamma, [.5])
AC_CHECK_MATH_FUNC(y0, [.5])
AC_CHECK_MATH_FUNC(y1, [.5])
AC_CHECK_MATH_FUNC(pow, [2,2])
AC_CHECK_MATH_FUNC(atan2, [.5,.5])
AC_CHECK_MATH_FUNC(hypot, [.5,.5])
AC_CHECK_MATH_FUNC(remainder, [3,2])
dnl Thread local storage
have___thread=no
AC_MSG_CHECKING(for thread-local storage)
AC_LINK_IFELSE([AC_LANG_SOURCE([
static __thread int x ;
int main () { x = 123; return x; }
])], have___thread=yes)
if test $have___thread = yes; then
AC_DEFINE([HAVE___THREAD],1,[Define to 1 if the system supports __thread])
fi
AC_MSG_RESULT($have___thread)
AC_C_BIGENDIAN(
AC_DEFINE([IEEE_MC68k], 1, [machine is bigendian]),
AC_DEFINE([IEEE_8087], 1, [machine is littleendian]),
AC_MSG_ERROR(unknown endianess),
AC_MSG_ERROR(universial endianess not supported)
)
AC_SUBST([BUNDLER], ["$bundle_cmd"])
AC_CONFIG_MACRO_DIR([config/m4])
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
+6
View File
@@ -0,0 +1,6 @@
.DS_Store
.sass-cache
output/*
# Autogenerated from public/css/base.scss
public/css/base.css
+6
View File
@@ -0,0 +1,6 @@
source "https://rubygems.org"
gem "rake"
gem "bonsai"
gem "maruku"
gem "ronn"
+63
View File
@@ -0,0 +1,63 @@
GEM
remote: https://rubygems.org/
specs:
POpen4 (0.1.4)
Platform (>= 0.4.0)
open4
Platform (0.4.0)
activesupport (3.2.12)
i18n (~> 0.6)
multi_json (~> 1.0)
addressable (2.3.4)
bonsai (1.4.8)
activesupport (>= 3.0.3)
builder (>= 3.0.0)
i18n (>= 0.5.0)
launchy (>= 0.3.7)
liquid (>= 2.2.2)
maruku (>= 0.6.0)
rack
sass
sinatra (>= 1.0)
tilt (>= 1.3)
watch (>= 0.1.0)
yui-compressor
builder (3.2.0)
hpricot (0.8.6)
i18n (0.6.4)
launchy (2.3.0)
addressable (~> 2.3)
liquid (2.5.0)
maruku (0.6.1)
syntax (>= 1.0.0)
multi_json (1.7.3)
mustache (0.99.4)
open4 (1.3.0)
rack (1.5.2)
rack-protection (1.5.0)
rack
rake (10.0.3)
rdiscount (2.0.7.2)
ronn (0.7.3)
hpricot (>= 0.8.2)
mustache (>= 0.7.0)
rdiscount (>= 1.5.8)
sass (3.2.8)
sinatra (1.4.2)
rack (~> 1.5, >= 1.5.2)
rack-protection (~> 1.4)
tilt (~> 1.3, >= 1.3.4)
syntax (1.0.0)
tilt (1.4.0)
watch (0.1.0)
yui-compressor (0.9.6)
POpen4 (>= 0.1.4)
PLATFORMS
ruby
DEPENDENCIES
bonsai
maruku
rake
ronn
+25
View File
@@ -0,0 +1,25 @@
Documentation
=============
The jq website, manpages and some of the tests are generated from this
directory. The directory holds a [Bonsai](http://tinytree.info)
website, and the manual is a YAML file in `content/3.manual`.
To build the documentation (including building the jq manpage), you'll
need a working Ruby setup. The easiest way to get one is to install
RVM and Ruby 1.9.3 like so:
\curl -L https://get.rvm.io | bash -s stable --ruby=1.9.3
After that finishes installing, you'll need to make sure RVM is on
your path by doing `source $HOME/.rvm/scripts/rvm`, or just opening a
new shell. See <http://rvm.io> for more info on RVM.
Once RVM is installed, you can install all the dependencies for jq's
documentation build by running this from the `docs` directory:
bundle install
When bundle manages to install the dependencies, rerun `./configure`
in the jq root directory and then the Makefile will be able to
generate the jq manpage.
+145
View File
@@ -0,0 +1,145 @@
require 'yaml'
require 'bonsai'
require 'liquid'
require 'maruku'
require 'json'
require 'ronn'
require 'tempfile'
require 'yaml'
module ExtraFilters
def markdownify(input)
Maruku.new(input).to_html
end
def sanitize(input)
input.gsub(/[^a-zA-Z0-9_]/,"")
end
def json(input)
input.to_json
end
def unique(input)
@n = (@n || 0) + 1
input + @n.to_s
end
end
Liquid::Template.register_filter(ExtraFilters)
task :serve do
begin
Bonsai.log "Press Control+C to quit"
require 'rack'
require 'sinatra'
require 'watch'
require 'launchy'
Bonsai.root_dir = Dir.pwd
server = fork {
app = Rack::Builder.app {
map "/jq" do
use Bonsai::StaticPassThrough, :root => Bonsai.root_dir + "/output", :urls => ["/"]
end
run Bonsai::DevelopmentServer
}
Rack::Handler.default.run(app, :Port => 5000) do
Launchy.open("http://localhost:5000/jq/")
end
}
Watch.new("{content,templates,public}/**/*") { Bonsai::Exporter.process! }
rescue Interrupt
Process.kill("QUIT", server)
Process.wait(server)
exit
end
end
task :build do
Bonsai.root_dir = Dir.pwd
Bonsai::Exporter.publish!
end
$BINARIES = {
:osx32 => "--host=i686-apple-darwin10 CFLAGS='-m32 -g -O'",
:osx64 => "--host=i686-apple-darwin10 CFLAGS='-m64 -g -O'",
:win32 => "--host=i686-w64-mingw32 CFLAGS='-g -O'",
:win64 => "--host=x86_64-w64-mingw32 CFLAGS='-g -O'",
:linux32 => "--host=x86_64-linux-gnu CFLAGS='-m32 -g -O'",
:linux64 => "--host=x86_64-linux-gnu CFLAGS='-m64 -g -O'"
}
$BINARIES.each do |name, args|
file "../build/#{name}" do |t|
sh "../scripts/crosscompile -j4 #{name} #{args}"
end
task :binaries => ["../build/#{name}"]
end
task :binaries => ["output/download"] do
$BINARIES.each do |name, args|
FileUtils.cp_r "../build/#{name}", "output/download/"
end
end
def load_manual
YAML::load(File.open("content/3.manual/manual.yml"))
end
task :manpage do
Tempfile.open "manpage" do |f|
manual = load_manual
f.puts manual['manpage_intro']
f.puts manual['body']
manual['sections'].each do |section|
f.puts "## #{section['title'].upcase}\n"
f.puts section['body']
f.puts ""
(section['entries'] || []).each do |entry|
f.puts "### #{entry['title']}\n"
f.puts entry['body']
f.puts ""
(entry['examples'] || []).each do |example|
f.puts " jq '#{example['program']}'"
f.puts " #{example['input']}"
f.puts " => #{example['output'].join(", ")}"
f.puts
end
end
f.puts ""
end
f.puts manual['manpage_epilogue']
f.close
puts Ronn::Document.new(f.path).convert('roff').gsub(/<\/?code>/,"")
end
end
task :manpage_default => ["default_manpage.md"] do
puts Ronn::Document.new("default_manpage.md").convert('roff').gsub(/<\/?code>/,"")
end
task :mantests do
load_manual['sections'].each do |section|
(section['entries'] || []).each do |entry|
(entry['examples'] || []).each do |example|
puts example['program'].gsub("\n", " ")
puts example['input']
example['output'].each do |s| puts s end
puts
end
end
end
end
directory "output/download/source"
task :tarball => ["output/download/source"] do
sh "cd ..; ./configure && make dist && make distclean"
sh "cp ../jq-*.tar.gz output/download/source"
end
task :dist => [:build, :binaries, :tarball]
+322
View File
@@ -0,0 +1,322 @@
headline: Tutorial
body:
- text: |
GitHub has a JSON API, so let's play with that. This URL gets
us the last 5 commits from the jq repo:
- command: "curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5'"
result: |
[
{
"sha": "d25341478381063d1c76e81b3a52e0592a7c997f",
"commit": {
"author": {
"name": "Stephen Dolan",
"email": "mu@netsoc.tcd.ie",
"date": "2013-06-22T16:30:59Z"
},
"committer": {
"name": "Stephen Dolan",
"email": "mu@netsoc.tcd.ie",
"date": "2013-06-22T16:30:59Z"
},
"message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161",
"tree": {
"sha": "6ab697a8dfb5a96e124666bf6d6213822599fb40",
"url": "https://api.github.com/repos/stedolan/jq/git/trees/6ab697a8dfb5a96e124666bf6d6213822599fb40"
},
"url": "https://api.github.com/repos/stedolan/jq/git/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
"comment_count": 0
},
"url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
"html_url": "https://github.com/stedolan/jq/commit/d25341478381063d1c76e81b3a52e0592a7c997f",
"comments_url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f/comments",
"author": {
"login": "stedolan",
...
- text: |
Github returns a ton of info here, and it's relatively hard to
read as it is. To syntax highlight it and order the attributes we pipe
it through jq, telling jq to just spit the input back at us using
the expression `.`:
- command: "curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '.'"
result: |
[
{
"parents": [
{
"html_url": "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
"url": "https://api.github.com/repos/stedolan/jq/commits/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
"sha": "54b9c9bdb225af5d886466d72f47eafc51acb4f7"
},
{
"html_url": "https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
"url": "https://api.github.com/repos/stedolan/jq/commits/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
"sha": "8b1b503609c161fea4b003a7179b3fbb2dd4345a"
}
],
"committer": {
"type": "User",
"received_events_url": "https://api.github.com/users/stedolan/received_events",
"events_url": "https://api.github.com/users/stedolan/events{/privacy}",
"repos_url": "https://api.github.com/users/stedolan/repos",
"organizations_url": "https://api.github.com/users/stedolan/orgs",
...
- text: |
There's still far too much info for our purposes, so we can try
just grab the first commit:
- command: "curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '.[0]'"
result: |
{
"parents": [
{
"html_url": "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
"url": "https://api.github.com/repos/stedolan/jq/commits/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
"sha": "54b9c9bdb225af5d886466d72f47eafc51acb4f7"
},
{
"html_url": "https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
"url": "https://api.github.com/repos/stedolan/jq/commits/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
"sha": "8b1b503609c161fea4b003a7179b3fbb2dd4345a"
}
],
"committer": {
"type": "User",
"received_events_url": "https://api.github.com/users/stedolan/received_events",
"events_url": "https://api.github.com/users/stedolan/events{/privacy}",
"repos_url": "https://api.github.com/users/stedolan/repos",
"organizations_url": "https://api.github.com/users/stedolan/orgs",
"subscriptions_url": "https://api.github.com/users/stedolan/subscriptions",
"starred_url": "https://api.github.com/users/stedolan/starred{/owner}{/repo}",
"gists_url": "https://api.github.com/users/stedolan/gists{/gist_id}",
"login": "stedolan",
"id": 79765,
"avatar_url": "https://1.gravatar.com/avatar/31de909d8e55dd07ed782d92ece59842?d=https%3A%2F%2Fidenticons.github.com%2Ffc5b6765b1c9cfaecea48ae71df4d279.png",
"gravatar_id": "31de909d8e55dd07ed782d92ece59842",
"url": "https://api.github.com/users/stedolan",
"html_url": "https://github.com/stedolan",
"followers_url": "https://api.github.com/users/stedolan/followers",
"following_url": "https://api.github.com/users/stedolan/following{/other_user}"
},
"author": {
"type": "User",
"received_events_url": "https://api.github.com/users/stedolan/received_events",
"events_url": "https://api.github.com/users/stedolan/events{/privacy}",
"repos_url": "https://api.github.com/users/stedolan/repos",
"organizations_url": "https://api.github.com/users/stedolan/orgs",
"subscriptions_url": "https://api.github.com/users/stedolan/subscriptions",
"starred_url": "https://api.github.com/users/stedolan/starred{/owner}{/repo}",
"gists_url": "https://api.github.com/users/stedolan/gists{/gist_id}",
"login": "stedolan",
"id": 79765,
"avatar_url": "https://1.gravatar.com/avatar/31de909d8e55dd07ed782d92ece59842?d=https%3A%2F%2Fidenticons.github.com%2Ffc5b6765b1c9cfaecea48ae71df4d279.png",
"gravatar_id": "31de909d8e55dd07ed782d92ece59842",
"url": "https://api.github.com/users/stedolan",
"html_url": "https://github.com/stedolan",
"followers_url": "https://api.github.com/users/stedolan/followers",
"following_url": "https://api.github.com/users/stedolan/following{/other_user}"
},
"comments_url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f/comments",
"html_url": "https://github.com/stedolan/jq/commit/d25341478381063d1c76e81b3a52e0592a7c997f",
"url": "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
"commit": {
"comment_count": 0,
"url": "https://api.github.com/repos/stedolan/jq/git/commits/d25341478381063d1c76e81b3a52e0592a7c997f",
"tree": {
"url": "https://api.github.com/repos/stedolan/jq/git/trees/6ab697a8dfb5a96e124666bf6d6213822599fb40",
"sha": "6ab697a8dfb5a96e124666bf6d6213822599fb40"
},
"message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161",
"committer": {
"date": "2013-06-22T16:30:59Z",
"email": "mu@netsoc.tcd.ie",
"name": "Stephen Dolan"
},
"author": {
"date": "2013-06-22T16:30:59Z",
"email": "mu@netsoc.tcd.ie",
"name": "Stephen Dolan"
}
},
"sha": "d25341478381063d1c76e81b3a52e0592a7c997f"
}
- text: |
For the rest of the examples, I'll leave out the `curl` command - it's not going to change.
There's still a lot of info we don't care about there, so we'll
restrict it down to the most interesting fields.
- command: "jq '.[0] | {message: .commit.message, name: .commit.committer.name}'"
result: |
{
"name": "Stephen Dolan",
"message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161"
}
- text: |
The `|` operator in jq feeds the output of one filter
(`.[0]` which gets the first element of the array in the response)
into the input of another (`{...}` which
builds an object out of those fields). You can access nested attributes,
such as `.commit.message`:
Now let's get the rest of the commits:
- command: "jq '.[] | {message: .commit.message, name: .commit.committer.name}'"
result: |
{
"name": "Stephen Dolan",
"message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161"
}
{
"name": "Stephen Dolan",
"message": "Reject all overlong UTF8 sequences."
}
{
"name": "Stephen Dolan",
"message": "Fix various UTF8 parsing bugs.\n\nIn particular, parse bad UTF8 by replacing the broken bits with U+FFFD\nand resychronise correctly after broken sequences."
}
{
"name": "Stephen Dolan",
"message": "Fix example in manual for `floor`. See #155."
}
{
"name": "Nicolas Williams",
"message": "Document floor"
}
- text: |
`.[]` returns each element of the array returned in the response,
one at a time, which are all fed into `{message: .commit.message, name: .commit.committer.name}`.
Data in jq is represented as streams of JSON values - every jq
expression runs for each value in its input stream, and can
produce any number of values to its output stream.
Streams are serialised by just separating JSON values with
whitespace. This is a `cat`-friendly format - you can just join
two JSON streams together and get a valid JSON stream.
If you want to get the output as a single array, you can tell jq to
"collect" all of the answers by wrapping the filter in square
brackets:
- command: "jq '[.[] | {message: .commit.message, name: .commit.committer.name}]'"
result: |
[
{
"name": "Stephen Dolan",
"message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161"
},
{
"name": "Stephen Dolan",
"message": "Reject all overlong UTF8 sequences."
},
{
"name": "Stephen Dolan",
"message": "Fix various UTF8 parsing bugs.\n\nIn particular, parse bad UTF8 by replacing the broken bits with U+FFFD\nand resychronise correctly after broken sequences."
},
{
"name": "Stephen Dolan",
"message": "Fix example in manual for `floor`. See #155."
},
{
"name": "Nicolas Williams",
"message": "Document floor"
}
]
- text: |
- - -
Next, let's try getting the URLs of the parent commits out of the
API results as well. In each commit, the GitHub API includes information
about "parent" commits. There can be one or many.
"parents": [
{
"html_url": "https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
"url": "https://api.github.com/repos/stedolan/jq/commits/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
"sha": "54b9c9bdb225af5d886466d72f47eafc51acb4f7"
},
{
"html_url": "https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
"url": "https://api.github.com/repos/stedolan/jq/commits/8b1b503609c161fea4b003a7179b3fbb2dd4345a",
"sha": "8b1b503609c161fea4b003a7179b3fbb2dd4345a"
}
]
We want to pull out all of the "html_url" fields inside that array of parent
commits and make a simple list of strings to go along with the
"message" and "author" fields we already have.
- command: "jq '[.[] | {message: .commit.message, name: .commit.committer.name, parents: [.parents[].html_url]}]'"
result: |
[
{
"parents": [
"https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7",
"https://github.com/stedolan/jq/commit/8b1b503609c161fea4b003a7179b3fbb2dd4345a"
],
"name": "Stephen Dolan",
"message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161"
},
{
"parents": [
"https://github.com/stedolan/jq/commit/ff48bd6ec538b01d1057be8e93b94eef6914e9ef"
],
"name": "Stephen Dolan",
"message": "Reject all overlong UTF8 sequences."
},
{
"parents": [
"https://github.com/stedolan/jq/commit/54b9c9bdb225af5d886466d72f47eafc51acb4f7"
],
"name": "Stephen Dolan",
"message": "Fix various UTF8 parsing bugs.\n\nIn particular, parse bad UTF8 by replacing the broken bits with U+FFFD\nand resychronise correctly after broken sequences."
},
{
"parents": [
"https://github.com/stedolan/jq/commit/3dcdc582ea993afea3f5503a78a77675967ecdfa"
],
"name": "Stephen Dolan",
"message": "Fix example in manual for `floor`. See #155."
},
{
"parents": [
"https://github.com/stedolan/jq/commit/7c4171d414f647ab08bcd20c76a4d8ed68d9c602"
],
"name": "Nicolas Williams",
"message": "Document floor"
}
]
- text: |
Here we're making an object as before, but this time the `parents`
field is being set to `[.parents[].html_url]}]`, which collects
all of the parent commit URLs defined in the parents object.
- text: |
- - -
Here endeth the tutorial! There's lots more to play with, go
read [the manual](../manual/) if you're interested, and [download
jq](../download/) if you haven't already.
+113
View File
@@ -0,0 +1,113 @@
headline: Download jq
body:
- text: |
jq is written in C and has no runtime dependencies, so it should be
possible to build it for nearly any platform. Prebuilt binaries are
available for Linux, OS X and Windows.
The binaries should just run, but on OS X and Linux you may need
to make them executable first using `chmod +x jq`.
jq is licensed under the MIT license. For all of the gory
details, read the file `COPYING` in the source distribution.
### Linux
* jq 1.4 binaries for [64-bit](linux64/jq) or [32-bit](linux32/jq) systems.
* jq 1.3 binaries for [64-bit](linux64/jq-1.3/jq) or [32-bit](linux32/jq-1.3/jq) systems.
* For Arch users, a PKGBUILD for jq-1.1 is in the
[AUR](https://aur.archlinux.org/packages.php?ID=63849), as
well as a PKGBUILD for the HEAD of master
([jq-git](https://aur.archlinux.org/packages.php?ID=63850)).
You can install them using
[Yaourt](https://wiki.archlinux.org/index.php/Yaourt) or
manually by following instructions on [Arch Linux's
Wiki](https://wiki.archlinux.org/index.php/Arch_UseRepository).
* jq 1.4 is [in Debian](http://packages.debian.org/jq)
* jq 1.3.2 is in the official Fedora repository. You can install using
`sudo yum install jq`.
### OS X
* `brew install jq` using [homebrew](http://mxcl.github.com/homebrew/)
* Or, grab prebuilt jq 1.4 [64-bit binaries](osx64/jq) or [32-bit
binaries](osx32/jq)
* Or, grab prebuilt jq 1.3 [64-bit binaries](osx64/jq-1.3/jq) or [32-bit
binaries](osx32/jq-1.3/jq)
### Solaris
* `pkgutil -i jq` in [OpenCSW](http://www.opencsw.org/p/jq) for Solaris 10+, Sparc and x86
* jq 1.4 executables for Solaris 11 [64-bit](solaris11-64/jq) or [32-bit](solaris11-32/jq)
### Windows
* `chocolatey install jq` using [Chocolatey NuGet](https://chocolatey.org/)
* jq 1.4 executables for [64-bit](win64/jq.exe) or [32-bit](win32/jq.exe)
* jq 1.3 executables for [64-bit](win64/jq-1.3/jq-1.3.exe) or [32-bit](win32/jq-1.3/jq-1.3.exe)
### From source on Linux, OS X, Cygwin, and other POSIX-like operating systems
* [Source tarball for jq 1.4](source/jq-1.4.tar.gz)
You can build it using the usual `./configure && make && sudo
make install` rigmarole.
If you're interested in using the lastest development version, try:
git clone https://github.com/stedolan/jq.git
cd jq
autoreconf -i
./configure
make
sudo make install
To build it from a git clone, you'll need to install a few
packages first:
* [Flex](http://www.gnu.org/software/flex/)
* [Bison](http://www.gnu.org/software/bison/)
* [GCC](http://gcc.gnu.org)
* [Make](http://www.gnu.org/software/make)
* [Autotools](http://www.gnu.org/software/automake)
For Linux systems, these will all be in your system's package
manager, and if you do development on the machine they're most
likely already installed.
On OS X, these are all included in Apple's command line tools, which
can be installed from [Xcode](http://developer.apple.com/technologies/tools/).
However, you may find that you need a newer version of Bison than the one provided
by Apple. This can be found in [Homebrew](http://mxcl.github.com/homebrew/)
or [MacPorts](http://macports.org/).
`flex` and `bison` are used to generate the lexer and parser for
jq. For jq 1.5, neither is required if you use `./configure
--disable-maintainer-mode` instead of `./configure`. This flag says to use
the pre-generated lexer and parser that come with the code.
#### Building the documentation
jq's documentation is compiled into static HTML using
[Bonsai](http://www.tinytree.info). To view the documentation
locally, run `rake serve` (or `bundle exec rake serve`) from the
docs/ subdirectory. To build the docs just `rake build` from the
docs subdirectory. You'll need a few Ruby dependencies, which can
be installed by following the instructions in `docs/README.md`.
The man page is built by `make jq.1`, or just `make`, also from
the YAML docs, and you'll still need the Ruby dependencies to
build the manpage.
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+50
View File
@@ -0,0 +1,50 @@
headline: jq
blurb: |
jq is a lightweight and flexible command-line JSON processor.
body1: |
jq is like `sed` for JSON data - you can use it to slice and filter
and map and transform structured data with the same ease that `sed`,
`awk`, `grep` and friends let you play with text.
body2: |
jq is written in portable C, and it has zero runtime
dependencies. You can download a single binary, `scp` it to a far away
machine of the same type, and expect it to work.
body3: |
jq can mangle the data format that you have into the one that you
want with very little effort, and the program to do so is often
shorter and simpler than you'd expect.
tail: |
Go read the [tutorial](tutorial/) for more, or the [manual](manual/)
for *way* more. Ask questions over on
[stackoverflow](http://stackoverflow.com/) using the [jq
tag](http://stackoverflow.com/questions/tagged/jq), or on the
[#jq](http://irc.lc/freenode/%23jq/) channel over on
[Freenode](https://webchat.freenode.net/).
news:
- date: 2015-01-01
body: |
jq 1.5rc1 Get it on the
[releases](https://github.com/stedolan/jq/releases) page.
- date: 2014-06-08
body: |
jq 1.4 (finally) released! Get it on the [download](download/) page.
- date: 2013-05-19
body: |
jq 1.3 released.
+22
View File
@@ -0,0 +1,22 @@
jq(1) -- Command-line JSON processor
====================================
## DESCRIPTION
`jq` can transform JSON in various ways, by selecting, iterating,
reducing and otherwise mangling JSON documents.
This version of `jq` was built without a manual, so this manpage is a
stub. For full documentation of the `jq` language, see:
http://stedolan.github.io/jq
## BUGS
Presumably. Report them or discuss them at:
https://github.com/stedolan/jq/issues
## AUTHOR
Stephen Dolan `<mu@netsoc.tcd.ie>`
+28
View File
@@ -0,0 +1,28 @@
DirectoryIndex index.html
FileETag All
# Compress all static assets
<IfModule mod_deflate.c>
# compress content with type html, text, and css
AddOutputFilterByType DEFLATE text/css text/html text/javascript application/javascript application/x-javascript text/js text/plain text/xml
<IfModule mod_headers.c>
# properly handle requests coming from behind proxies
Header append Vary User-Agent
</IfModule>
</IfModule>
# Cache, aggressively
<IfModule mod_expires.c>
ExpiresActive On
ExpiresDefault "access plus 10 days"
<FilesMatch "\.(eot|ttf|otf)$">
ExpiresDefault "access plus 10 years"
</filesMatch>
</IfModule>
# Mime-types
AddType application/vnd.ms-fontobject .eot
AddType font/ttf .ttf
AddType font/otf .otf
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,9 @@
/*!
* Bootstrap Responsive v2.1.1
*
* Copyright 2012 Twitter, Inc
* Licensed under the Apache License v2.0
* http://www.apache.org/licenses/LICENSE-2.0
*
* Designed and built with all the love in the world @twitter by @mdo and @fat.
*/.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.hidden{display:none;visibility:hidden}.visible-phone{display:none!important}.visible-tablet{display:none!important}.hidden-desktop{display:none!important}.visible-desktop{display:inherit!important}@media(min-width:768px) and (max-width:979px){.hidden-desktop{display:inherit!important}.visible-desktop{display:none!important}.visible-tablet{display:inherit!important}.hidden-tablet{display:none!important}}@media(max-width:767px){.hidden-desktop{display:inherit!important}.visible-desktop{display:none!important}.visible-phone{display:inherit!important}.hidden-phone{display:none!important}}@media(min-width:1200px){.row{margin-left:-30px;*zoom:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;min-height:1px;margin-left:30px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:1170px}.span12{width:1170px}.span11{width:1070px}.span10{width:970px}.span9{width:870px}.span8{width:770px}.span7{width:670px}.span6{width:570px}.span5{width:470px}.span4{width:370px}.span3{width:270px}.span2{width:170px}.span1{width:70px}.offset12{margin-left:1230px}.offset11{margin-left:1130px}.offset10{margin-left:1030px}.offset9{margin-left:930px}.offset8{margin-left:830px}.offset7{margin-left:730px}.offset6{margin-left:630px}.offset5{margin-left:530px}.offset4{margin-left:430px}.offset3{margin-left:330px}.offset2{margin-left:230px}.offset1{margin-left:130px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.564102564102564%;*margin-left:2.5109110747408616%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.45299145299145%;*width:91.39979996362975%}.row-fluid .span10{width:82.90598290598291%;*width:82.8527914166212%}.row-fluid .span9{width:74.35897435897436%;*width:74.30578286961266%}.row-fluid .span8{width:65.81196581196582%;*width:65.75877432260411%}.row-fluid .span7{width:57.26495726495726%;*width:57.21176577559556%}.row-fluid .span6{width:48.717948717948715%;*width:48.664757228587014%}.row-fluid .span5{width:40.17094017094017%;*width:40.11774868157847%}.row-fluid .span4{width:31.623931623931625%;*width:31.570740134569924%}.row-fluid .span3{width:23.076923076923077%;*width:23.023731587561375%}.row-fluid .span2{width:14.52991452991453%;*width:14.476723040552828%}.row-fluid .span1{width:5.982905982905983%;*width:5.929714493544281%}.row-fluid .offset12{margin-left:105.12820512820512%;*margin-left:105.02182214948171%}.row-fluid .offset12:first-child{margin-left:102.56410256410257%;*margin-left:102.45771958537915%}.row-fluid .offset11{margin-left:96.58119658119658%;*margin-left:96.47481360247316%}.row-fluid .offset11:first-child{margin-left:94.01709401709402%;*margin-left:93.91071103837061%}.row-fluid .offset10{margin-left:88.03418803418803%;*margin-left:87.92780505546462%}.row-fluid .offset10:first-child{margin-left:85.47008547008548%;*margin-left:85.36370249136206%}.row-fluid .offset9{margin-left:79.48717948717949%;*margin-left:79.38079650845607%}.row-fluid .offset9:first-child{margin-left:76.92307692307693%;*margin-left:76.81669394435352%}.row-fluid .offset8{margin-left:70.94017094017094%;*margin-left:70.83378796144753%}.row-fluid .offset8:first-child{margin-left:68.37606837606839%;*margin-left:68.26968539734497%}.row-fluid .offset7{margin-left:62.393162393162385%;*margin-left:62.28677941443899%}.row-fluid .offset7:first-child{margin-left:59.82905982905982%;*margin-left:59.72267685033642%}.row-fluid .offset6{margin-left:53.84615384615384%;*margin-left:53.739770867430444%}.row-fluid .offset6:first-child{margin-left:51.28205128205128%;*margin-left:51.175668303327875%}.row-fluid .offset5{margin-left:45.299145299145295%;*margin-left:45.1927623204219%}.row-fluid .offset5:first-child{margin-left:42.73504273504273%;*margin-left:42.62865975631933%}.row-fluid .offset4{margin-left:36.75213675213675%;*margin-left:36.645753773413354%}.row-fluid .offset4:first-child{margin-left:34.18803418803419%;*margin-left:34.081651209310785%}.row-fluid .offset3{margin-left:28.205128205128204%;*margin-left:28.0987452264048%}.row-fluid .offset3:first-child{margin-left:25.641025641025642%;*margin-left:25.53464266230224%}.row-fluid .offset2{margin-left:19.65811965811966%;*margin-left:19.551736679396257%}.row-fluid .offset2:first-child{margin-left:17.094017094017094%;*maLine truncated
File diff suppressed because it is too large. Load diff
+9
View File
@@ -0,0 +1,9 @@
/*!
* Bootstrap v2.1.1
*
* Copyright 2012 Twitter, Inc
* Licensed under the Apache License v2.0
* http://www.apache.org/licenses/LICENSE-2.0
*
* Designed and built with all the love in the world @twitter by @mdo and @fat.
*/article,aside,details,figcaption,figure,footer,header,hgroup,nav,section{display:block}audio,canvas,video{display:inline-block;*display:inline;*zoom:1}audio:not([controls]){display:none}html{font-size:100%;-webkit-text-size-adjust:100%;-ms-text-size-adjust:100%}a:focus{outline:thin dotted #333;outline:5px auto -webkit-focus-ring-color;outline-offset:-2px}a:hover,a:active{outline:0}sub,sup{position:relative;font-size:75%;line-height:0;vertical-align:baseline}sup{top:-0.5em}sub{bottom:-0.25em}img{width:auto\9;height:auto;max-width:100%;vertical-align:middle;border:0;-ms-interpolation-mode:bicubic}#map_canvas img{max-width:none}button,input,select,textarea{margin:0;font-size:100%;vertical-align:middle}button,input{*overflow:visible;line-height:normal}button::-moz-focus-inner,input::-moz-focus-inner{padding:0;border:0}button,input[type="button"],input[type="reset"],input[type="submit"]{cursor:pointer;-webkit-appearance:button}input[type="search"]{-webkit-box-sizing:content-box;-moz-box-sizing:content-box;box-sizing:content-box;-webkit-appearance:textfield}input[type="search"]::-webkit-search-decoration,input[type="search"]::-webkit-search-cancel-button{-webkit-appearance:none}textarea{overflow:auto;vertical-align:top}.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}body{margin:0;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:14px;line-height:20px;color:#333;background-color:#fff}a{color:#08c;text-decoration:none}a:hover{color:#005580;text-decoration:underline}.img-rounded{-webkit-border-radius:6px;-moz-border-radius:6px;border-radius:6px}.img-polaroid{padding:4px;background-color:#fff;border:1px solid #ccc;border:1px solid rgba(0,0,0,0.2);-webkit-box-shadow:0 1px 3px rgba(0,0,0,0.1);-moz-box-shadow:0 1px 3px rgba(0,0,0,0.1);box-shadow:0 1px 3px rgba(0,0,0,0.1)}.img-circle{-webkit-border-radius:500px;-moz-border-radius:500px;border-radius:500px}.row{margin-left:-20px;*zoom:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;min-height:1px;margin-left:20px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:940px}.span12{width:940px}.span11{width:860px}.span10{width:780px}.span9{width:700px}.span8{width:620px}.span7{width:540px}.span6{width:460px}.span5{width:380px}.span4{width:300px}.span3{width:220px}.span2{width:140px}.span1{width:60px}.offset12{margin-left:980px}.offset11{margin-left:900px}.offset10{margin-left:820px}.offset9{margin-left:740px}.offset8{margin-left:660px}.offset7{margin-left:580px}.offset6{margin-left:500px}.offset5{margin-left:420px}.offset4{margin-left:340px}.offset3{margin-left:260px}.offset2{margin-left:180px}.offset1{margin-left:100px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.127659574468085%;*margin-left:2.074468085106383%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.48936170212765%;*width:91.43617021276594%}.row-fluid .span10{width:82.97872340425532%;*width:82.92553191489361%}.row-fluid .span9{width:74.46808510638297%;*width:74.41489361702126%}.row-fluid .span8{width:65.95744680851064%;*width:65.90425531914893%}.row-fluid .span7{width:57.44680851063829%;*width:57.39361702127659%}.row-fluid .span6{width:48.93617021276595%;*width:48.88297872340425%}.row-fluid .span5{width:40.42553191489362%;*width:40.37234042553192%}.row-fluid .span4{width:31.914893617021278%;*width:31.861702127659576%}.row-fluid .span3{width:23.404255319148934%;*width:23.351063829787233%}.row-fluid .span2{width:14.893617021276595%;*width:14.840425531914894%}.row-fluid .span1{width:6.382978723404255%;*width:6.329787234042553%}.row-fluid .offset12{margin-left:104.25531914893617%;*margin-left:104.14893617021275%}.row-fluid .offset12:first-child{margin-left:102.12765957446808%;*margin-left:102.02127659574467%}.row-fluid .offset11{margin-left:95.74468085106382%;*margin-left:95.6382978723404%}.row-fluid .offset11:first-child{margin-left:93.61702127659574%;*margin-left:93.51063829787232%}.row-fluid .offset10{margin-left:87.23404255319149%;*margin-left:87.12765957446807%}.row-fluid .offset10:first-child{margin-left:85.1063829787234%;*margin-left:84.99999999999999%}.row-fluid .offset9{margin-left:78.72340425531914%;*margin-left:78.61702127659572%}.row-fluid .offset9:first-child{margin-left:76.59574468085106%;*margin-left:76.48936170212764%}.row-fluiLine truncated
Binary file not shown.

After

Width:  |  Height:  |  Size: 8.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

File diff suppressed because it is too large. Load diff
+6
View File
@@ -0,0 +1,6 @@
/*!
* Bootstrap.js by @fat & @mdo
* Copyright 2012 Twitter, Inc.
* http://www.apache.org/licenses/LICENSE-2.0.txt
*/
!function(e){e(function(){"use strict";e.support.transition=function(){var e=function(){var e=document.createElement("bootstrap"),t={WebkitTransition:"webkitTransitionEnd",MozTransition:"transitionend",OTransition:"oTransitionEnd otransitionend",transition:"transitionend"},n;for(n in t)if(e.style[n]!==undefined)return t[n]}();return e&&{end:e}}()})}(window.jQuery),!function(e){"use strict";var t='[data-dismiss="alert"]',n=function(n){e(n).on("click",t,this.close)};n.prototype.close=function(t){function s(){i.trigger("closed").remove()}var n=e(this),r=n.attr("data-target"),i;r||(r=n.attr("href"),r=r&&r.replace(/.*(?=#[^\s]*$)/,"")),i=e(r),t&&t.preventDefault(),i.length||(i=n.hasClass("alert")?n:n.parent()),i.trigger(t=e.Event("close"));if(t.isDefaultPrevented())return;i.removeClass("in"),e.support.transition&&i.hasClass("fade")?i.on(e.support.transition.end,s):s()},e.fn.alert=function(t){return this.each(function(){var r=e(this),i=r.data("alert");i||r.data("alert",i=new n(this)),typeof t=="string"&&i[t].call(r)})},e.fn.alert.Constructor=n,e(function(){e("body").on("click.alert.data-api",t,n.prototype.close)})}(window.jQuery),!function(e){"use strict";var t=function(t,n){this.$element=e(t),this.options=e.extend({},e.fn.button.defaults,n)};t.prototype.setState=function(e){var t="disabled",n=this.$element,r=n.data(),i=n.is("input")?"val":"html";e+="Text",r.resetText||n.data("resetText",n[i]()),n[i](r[e]||this.options[e]),setTimeout(function(){e=="loadingText"?n.addClass(t).attr(t,t):n.removeClass(t).removeAttr(t)},0)},t.prototype.toggle=function(){var e=this.$element.closest('[data-toggle="buttons-radio"]');e&&e.find(".active").removeClass("active"),this.$element.toggleClass("active")},e.fn.button=function(n){return this.each(function(){var r=e(this),i=r.data("button"),s=typeof n=="object"&&n;i||r.data("button",i=new t(this,s)),n=="toggle"?i.toggle():n&&i.setState(n)})},e.fn.button.defaults={loadingText:"loading..."},e.fn.button.Constructor=t,e(function(){e("body").on("click.button.data-api","[data-toggle^=button]",function(t){var n=e(t.target);n.hasClass("btn")||(n=n.closest(".btn")),n.button("toggle")})})}(window.jQuery),!function(e){"use strict";var t=function(t,n){this.$element=e(t),this.options=n,this.options.slide&&this.slide(this.options.slide),this.options.pause=="hover"&&this.$element.on("mouseenter",e.proxy(this.pause,this)).on("mouseleave",e.proxy(this.cycle,this))};t.prototype={cycle:function(t){return t||(this.paused=!1),this.options.interval&&!this.paused&&(this.interval=setInterval(e.proxy(this.next,this),this.options.interval)),this},to:function(t){var n=this.$element.find(".item.active"),r=n.parent().children(),i=r.index(n),s=this;if(t>r.length-1||t<0)return;return this.sliding?this.$element.one("slid",function(){s.to(t)}):i==t?this.pause().cycle():this.slide(t>i?"next":"prev",e(r[t]))},pause:function(t){return t||(this.paused=!0),this.$element.find(".next, .prev").length&&e.support.transition.end&&(this.$element.trigger(e.support.transition.end),this.cycle()),clearInterval(this.interval),this.interval=null,this},next:function(){if(this.sliding)return;return this.slide("next")},prev:function(){if(this.sliding)return;return this.slide("prev")},slide:function(t,n){var r=this.$element.find(".item.active"),i=n||r[t](),s=this.interval,o=t=="next"?"left":"right",u=t=="next"?"first":"last",a=this,f=e.Event("slide",{relatedTarget:i[0]});this.sliding=!0,s&&this.pause(),i=i.length?i:this.$element.find(".item")[u]();if(i.hasClass("active"))return;if(e.support.transition&&this.$element.hasClass("slide")){this.$element.trigger(f);if(f.isDefaultPrevented())return;i.addClass(t),i[0].offsetWidth,r.addClass(o),i.addClass(o),this.$element.one(e.support.transition.end,function(){i.removeClass([t,o].join(" ")).addClass("active"),r.removeClass(["active",o].join(" ")),a.sliding=!1,setTimeout(function(){a.$element.trigger("slid")},0)})}else{this.$element.trigger(f);if(f.isDefaultPrevented())return;r.removeClass("active"),i.addClass("active"),this.sliding=!1,this.$element.trigger("slid")}return s&&this.cycle(),this}},e.fn.carousel=function(n){return this.each(function(){var r=e(this),i=r.data("carousel"),s=e.extend({},e.fn.carousel.defaults,typeof n=="object"&&n),o=typeof n=="string"?n:s.slide;i||r.data("carousel",i=new t(this,s)),typeof n=="number"?i.to(n):o?i[o]():s.interval&&i.cycle()})},e.fn.carousel.defaults={interval:5e3,pause:"hover"},e.fn.carousel.Constructor=t,e(function(){e("body").on("click.carousel.data-api","[data-slide]",function(t){var n=e(this),r,i=e(n.attr("data-target")||(r=n.attr("href"))&&r.replace(/.*(?=#[^\s]+$)/,"")),s=!i.data("modal")&&e.extend({},i.data(),n.data());i.carousel(s),t.preventDefault()})})}(window.jQuery),!function(e){"use strict";var t=function(t,n){this.$element=e(t),this.options=e.extend({},e.fn.collapse.defaults,n),this.options.parent&&(this.$parent=e(this.options.parent)),this.options.toggle&&this.toggle()};t.prototype={constructor:t,dimension:function(){var e=this.$element.hLine truncated
+99
View File
@@ -0,0 +1,99 @@
@charset "utf-8";
body {
padding-top: 60px;
}
footer {
background-color: #F5F5F5;
padding: 40px 0;
height: 100px;
margin-top: 40px;
color: #999999;
}
section {
padding-top: 30px;
}
.manual-example {
border-top: 1px solid #E5E5E5;
}
.manual-example td {
white-space: pre-wrap;
font-family: Monaco, Menlo, Consolas, "Courier New", monospace;
}
.manual-example td.jqprogram {
font-weight: bold;
}
.manual-example th {
text-align: right;
padding-right: 10px;
}
.tutorial-example {
position: relative;
margin-bottom: 10px;
}
.tutorial-example pre {
margin-bottom: 0px;
}
.tutorial-example a {
position: absolute;
top: 0px;
right: 0px;
padding: 15px 8px;
color: #777777;
font-weight: bold;
line-height: 10px;
font-size: 12px;
border-left: 1px solid #DDDDDD;
bottom: 0;
display: block;
}
#blurb {
padding-top: 40px;
padding-right: 50px;
}
#blurb p {
font-size: 20pt;
}
#blurb .btn-group {
margin-top: 20px;
}
#multiblurb {
line-height: 1.7;
text-align: center;
font-size: 12pt;
}
#multiblurb code {
border: 0;
font-size: 12pt;
}
h3 code {
border: 0;
font-size: 20px;
}
footer {
text-align: center;
}
#news {
font-size: 12pt;
.date {
font-style: italic;
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 8.0 KiB

+2
View File
@@ -0,0 +1,2 @@
User-agent: *
Allow: /
+18
View File
@@ -0,0 +1,18 @@
# The key value pairs found below are available within the templates.
:url: http://stedolan.github.io/jq
# This line is modified by the Makefile. To change the version number,
# edit the Autoconf version number at the top of configure.ac
jq_version: "1.4-2-g15c4a7f-dirty"
root: '/jq'
footer: |
This website is made with [Bonsai](http://www.tinytree.info) and
[Twitter Bootstrap](http://twitter.github.com/bootstrap/), themed
with [Bootswatch](http://bootswatch.com).
jq is licensed under the MIT license (code) and the
[CC-BY-3.0](http://creativecommons.org/licenses/by/3.0/) license
(docs).
+34
View File
@@ -0,0 +1,34 @@
<!DOCTYPE html>
<html lang="en">
{% include "shared/head" %}
<body id="{{slug}}">
{% include "shared/header" %}
<div class="container">
<div class="row">
<h1>{{headline}}</h1>
{% for item in body %}
{% if item.text %}
{{ item.text | replace: '$JQ_VERSION', jq_version | markdownify }}
{% endif %}
{% if item.command %}
{% capture resultID %}{{ "result" | unique}}{% endcapture %}
<div class="tutorial-example">
<div class="accordion-heading">
<pre>{{item.command}}</pre>
<a data-toggle="collapse" href="#{{resultID}}">Show result</a>
</div>
<div id="{{resultID}}" class="accordion-body collapse"><pre>{{item.result}}</pre></div>
</div>
{% endif %}
{% endfor %}
</div>
</div>
{% include "shared/footer" %}
</body>
</html>
+60
View File
@@ -0,0 +1,60 @@
<!DOCTYPE html>
<html lang="en">
{% include "shared/head" %}
<body id="{{slug}}">
{% include "shared/header" %}
<div class="container">
<div class="row">
<div class="hero-unit">
<img src="{{root}}/jq.png" style="width: 400px; height: 220px; float: left; margin-right: 60px">
<div id="blurb">
{{blurb | markdownify}}
<div class="btn-group" style="float:right">
<a class="btn btn-primary dropdown-toggle" data-toggle="dropdown" href="#">
Download jq-{{jq_version}}
<span class="caret"></span>
</a>
<ul class="dropdown-menu">
<li><a href="{{root}}/download/linux64/jq">Linux (64-bit)</a></li>
<li><a href="{{root}}/download/osx64/jq">OS X (64-bit)</a></li>
<li><a href="{{root}}/download/solaris11-64/jq">Solaris (64-bit)</a></li>
<li><a href="{{root}}/download/win64/jq.exe">Windows (64-bit)</a></li>
<li><a href="{{root}}/download/">Other platforms, older versions, and source</a></li>
</ul>
</div>
<div class="btn-group" style="float:right">
<a class="btn btn-primary" href="http://jqplay.org">
Try online at jqplay.org!
</a>
</div>
</div>
<div style="clear:both;"></div>
</div>
</div>
<div class="row" id="multiblurb">
<div class="span4">{{body1 | markdownify}}</div>
<div class="span4">{{body2 | markdownify}}</div>
<div class="span4">{{body3 | markdownify}}</div>
</div>
<div class="row" style="text-align:center; margin-top: 30px">
{{tail | markdownify}}
</div>
<div class="row">
<h2>News</h2>
<div id="news">
<ul>
{% for item in news %}
<li><span class="date">{{ item.date | date: '%m %B %Y' }}</span>{{item.body | markdownify}}</li>
{% endfor %}
</ul>
</div>
</div>
</div>
{% include "shared/footer" %}
</body>
</html>
+122
View File
@@ -0,0 +1,122 @@
<!DOCTYPE html>
<html lang="en">
{% include "shared/head" %}
<body id="{{slug}}" data-spy="scroll" data-target="#navcolumn" data-offset="100">
{% include "shared/header" %}
<div class="container">
<div class="row">
<div class="span3" id="navcolumn">
<div class="affix">
<h4>Contents</h4>
<ul class="nav nav-pills nav-stacked">
{% for section in sections %}
<li {% if forloop.first %} class="active" {% endif %}>
<a href="#{{section.title | sanitize}}">{{section.title}}</a>
</li>
{% endfor %}
</ul>
<form class="form-search">
<script type="text/javascript">
var section_map = {
{% for section in sections %}
{% for entry in section.entries %}
{{entry.title | json}} : {{entry.title | sanitize | json}},
{% endfor %}
{{section.title | json}} : {{section.title | sanitize | json}}
{% unless forloop.last %},{% endunless %}
{% endfor %}
};
var section_names = $.map(section_map, function(v,k){return k});
$(function(){
$('#searchbox').typeahead({source: section_names})
$('#searchbox').change(function() {
if ($(this).val() in section_map) {
location.hash = '#' + section_map[$(this).val()];
}
});
});
// add "Run" button to execute examples on jqplay.org
$(function() {
$.each($('table.manual-example'), function(index, value) {
$value = $(value)
var j = $value.find('tr:nth-child(2) td:first').text();
var q = $value.find('.jqprogram').text().replace(/^jq /, '').replace(/^'(.+)'$/, '$1');
var url = 'https://jqplay.org/jq?q=' + encodeURIComponent(q) +'&j=' + encodeURIComponent(j)
var $last_tr = $value.find('tr:last');
$last_tr.after('<tr><th><a href="' + url + '" class="btn btn-primary btn-small">Run</a></th><th></th></tr><tr><th></th><th></th></tr>');
});
});
</script>
<input type="text"
class="input-medium search-query"
placeholder="Search"
autocomplete="off"
id="searchbox">
</form>
</div>
</div>
<div class="span9">
<h1>{{headline}}</h1>
{{ history | markdownify }}
{{ body | markdownify }}
{% for section in sections %}
<section id="{{section.title | sanitize}}">
<h2>{{section.title}}</h2>
{{section.body | markdownify}}
{% for entry in section.entries %}
<section id="{{entry.title | sanitize}}">
<h3>
{{entry.title | markdownify}}
{% if entry.subtitle %}<small>{{entry.subtitle}}</small>{% endif %}
</h3>
{{entry.body | markdownify}}
{% if entry.examples %}
<div>
{% capture exampleID %}{{ "" | unique }}{% endcapture %}
<a data-toggle="collapse" href="#example{{exampleID}}">
<i class="icon-chevron-right"></i>
{% if entry.examples[1] %}Examples{%else%}Example{%endif%}
</a>
<div id="example{{exampleID}}" class="collapse">
{% for example in entry.examples %}
<table class="manual-example">
<tr><th></th><td class="jqprogram">jq '{{example.program | escape}}'</td></tr>
<tr><th>Input</th><td>{{example.input | escape}}</td></tr>
{% unless example.output[0] %}
<tr>
<th>Output</th>
<td><i>none</i></td>
</tr>
{% endunless %}
{% for output in example.output %}
<tr>
{% if forloop.index == 1 %}
<th>Output</th>
{% else %}
<th></th>
{% endif %}
<td>{{output | escape}}</td>
</tr>
{% endfor %}
</table>
{% endfor %}
</div>
</div>
{% endif %}
</section>
{% endfor %}
</section>
{% endfor %}
</div>
</div>
</div>
{% include "shared/footer" %}
</body>
</html>
+5
View File
@@ -0,0 +1,5 @@
<footer>
<div class="container">
{{ footer | markdownify }}
</div>
</footer>
+12
View File
@@ -0,0 +1,12 @@
<head>
<meta charset=UTF-8>
<title>{{headline}}</title>
<link rel="stylesheet" href="{{root}}/bootstrap/css/bootstrap.css" type="text/css">
<link rel="stylesheet" href="{{root}}/css/base.css" type="text/css">
<!--[if IE]>
<script src="http://html5shiv.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.8.1/jquery.min.js"></script>
<script src="{{root}}/bootstrap/js/bootstrap.js"></script>
</head>
+26
View File
@@ -0,0 +1,26 @@
<div class="navbar navbar-inverse navbar-fixed-top">
<div class="navbar-inner">
<div class="container">
<button type="button" class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="brand" href="{{root}}/">jq</a>
<div class="nav-collapse collapse">
<ul class="nav">
{% for item in navigation %}
<li {% if item.permalink == permalink %} class="active" {% endif %}>
<a href="{{root}}{{item.permalink}}">{{item.name}}</a>
</li>
{% endfor %}
<li><a href="https://github.com/stedolan/jq/issues">Issues</a></li>
<li><a href="https://github.com/stedolan/jq">Source</a></li>
<li><a href="http://jqplay.org">Try online!</a></li>
<li><a href="https://raw.githubusercontent.com/stedolan/jq/master/NEWS">News</a></li>
</ul>
</div>
</div>
</div>
</div>
+112
View File
@@ -0,0 +1,112 @@
#ifndef EXEC_STACK_H
#define EXEC_STACK_H
#include <stddef.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include "jv_alloc.h"
/*
* The stack is a directed forest of variably sized blocks. Each block has a
* "next" block which is at a higher memory address, or 0 if the block has no
* "next" block. More than one block may have no "next" block. A block may be
* the "next" block of more than one other block. Pushed blocks are added at
* the low-address end of the stack.
*
* Stack pointers are negative integers that are offsets relative to "mem_end",
* the end of the allocated region. The stack "bound" is the stack pointer of
* the last block that would be able to fit in the currently allocated region.
* The stack "limit" is the stack pointer of the last block currently in the
* stack. The stack pointer of the "next" block is stored directly below each
* block.
*
* <- mem_end = 0x100
* 0xF8 +------------+
* 0xF0 | |
* 0xE8 +------------+ <- stack_ptr1 = -0x18
* 0xE0 next = 0
* 0xD8 +------------+
* 0xD0 | |
* 0xC8 | |
* 0xC0 +------------+ <- stack_ptr2 = limit = -0x40
* 0xB8 next = -0x18
* 0xB0
* 0xA8 <- bound = -0x58
* 0xA0
*/
struct determine_alignment {
char x;
union { int i; double d; uint64_t u64; size_t sz; void* ptr; } u;
};
enum {ALIGNMENT = offsetof(struct determine_alignment, u)};
static size_t align_round_up(size_t sz) {
return ((sz + (ALIGNMENT - 1)) / ALIGNMENT) * ALIGNMENT;
}
typedef int stack_ptr;
struct stack {
char* mem_end; // one-past-the-end of allocated region
stack_ptr bound;
stack_ptr limit; // 0 - stack is empty
};
static void stack_init(struct stack* s) {
s->mem_end = 0;
s->bound = ALIGNMENT;
s->limit = 0;
}
static void stack_reset(struct stack* s) {
assert(s->limit == 0 && "stack freed while not empty");
char* mem_start = s->mem_end - ( -s->bound + ALIGNMENT);
free(mem_start);
stack_init(s);
}
static int stack_pop_will_free(struct stack* s, stack_ptr p) {
return p == s->limit;
}
static void* stack_block(struct stack* s, stack_ptr p) {
return (void*)(s->mem_end + p);
}
static stack_ptr* stack_block_next(struct stack* s, stack_ptr p) {
return &((stack_ptr*)stack_block(s, p))[-1];
}
static void stack_reallocate(struct stack* s, size_t sz) {
int old_mem_length = -(s->bound) + ALIGNMENT;
char* old_mem_start = s->mem_end - old_mem_length;
int new_mem_length = align_round_up((old_mem_length + sz + 256) * 2);
char* new_mem_start = jv_mem_realloc(old_mem_start, new_mem_length);
memmove(new_mem_start + (new_mem_length - old_mem_length),
new_mem_start, old_mem_length);
s->mem_end = new_mem_start + new_mem_length;
s->bound = -(new_mem_length - ALIGNMENT);
}
static stack_ptr stack_push_block(struct stack* s, stack_ptr p, size_t sz) {
int alloc_sz = align_round_up(sz) + ALIGNMENT;
stack_ptr r = s->limit - alloc_sz;
if (r < s->bound) {
stack_reallocate(s, alloc_sz);
}
s->limit = r;
*stack_block_next(s, r) = p;
return r;
}
static stack_ptr stack_pop_block(struct stack* s, stack_ptr p, size_t sz) {
stack_ptr r = *stack_block_next(s, p);
if (p == s->limit) {
int alloc_sz = align_round_up(sz) + ALIGNMENT;
s->limit += alloc_sz;
}
return r;
}
#endif
+1155
View File
File diff suppressed because it is too large. Load diff
+112
View File
@@ -0,0 +1,112 @@
#define _GNU_SOURCE /* for RTLD_NEXT */
#include <assert.h>
#include <dlfcn.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
static FILE *fail;
static FILE *fail_read;
static FILE *fail_write;
static FILE *fail_close;
static int error;
static FILE * (*real_fopen)(const char *, const char *);
static int (*real_fclose)(FILE *);
static int (*real_ferror)(FILE *);
static void (*real_clearerr)(FILE *);
static char * (*real_fgets)(char *, int, FILE *);
static size_t (*real_fread)(void *, size_t, size_t, FILE *);
static size_t (*real_fwrite)(const void *, size_t, size_t, FILE *);
#define GET_REAL(sym) \
do { \
if (real_ ## sym == 0) { \
real_ ## sym = dlsym(RTLD_NEXT, #sym); \
assert(real_ ## sym != 0); \
} \
} while (0)
#define dbg_write(msg) (void)write(2, msg, sizeof(msg) - 1)
#define dbg() \
do { \
dbg_write("here: "); \
dbg_write(__func__); \
dbg_write("!\n"); \
} while (0)
FILE *fopen(const char *path, const char *mode) {
GET_REAL(fopen);
fail = fail_read = fail_write = fail_close = 0;
FILE *f = real_fopen(path, mode);
error = EIO;
if (strcmp(path, "fail_read") == 0) {
fail = fail_read = f;
} else if (strncmp(path, "fail_write", sizeof("fail_write") - 1) == 0) {
// Not that jq opens files for write anyways...
fail = fail_write = f;
if (strcmp(path, "fail_write_enospc") == 0)
error = ENOSPC;
} else if (strncmp(path, "fail_close", sizeof("fail_close") - 1) == 0) {
fail = fail_close = f;
if (strcmp(path, "fail_close_enospc") == 0)
error = ENOSPC;
}
return f;
}
int fclose(FILE *f) {
GET_REAL(fclose);
int res = real_fclose(f);
if (fail_close == f) {
fail = fail_read = fail_write = fail_close = 0;
return EOF;
}
return res;
}
char * fgets(char *buf, int len, FILE *f) {
GET_REAL(fgets);
char *res = real_fgets(buf, len, f);
if (fail_read == f)
return 0;
return res;
}
size_t fread(void *buf, size_t sz, size_t nemb, FILE *f) {
GET_REAL(fread);
size_t res = real_fread(buf, sz, nemb, f);
if (fail_read == f)
return 0;
return res;
}
size_t fwrite(const void *buf, size_t sz, size_t nemb, FILE *f) {
GET_REAL(fwrite);
size_t res = real_fwrite(buf, sz, nemb, f);
if (fail_write == f)
return 0;
return res;
}
int ferror(FILE *f) {
GET_REAL(ferror);
int res = real_ferror(f);
if (fail == f) {
errno = error;
return 1;
}
return res;
}
void clearerr(FILE *f) {
GET_REAL(clearerr);
real_clearerr(f);
if (fail == f) {
fail = fail_read = fail_write = fail_close = 0;
error = 0;
}
}
+39
View File
@@ -0,0 +1,39 @@
.\" generated with Ronn/v0.7.3
.\" http://github.com/rtomayko/ronn/tree/0.7.3
.
.TH "JQ" "1" "May 2013" "" ""
.
.SH "NAME"
\fBjq\fR \- Command\-line JSON processor
.
.SH "DESCRIPTION"
\fBjq\fR can transform JSON in various ways, by selecting, iterating, reducing and otherwise mangling JSON documents\.
.
.P
This version of \fBjq\fR was built without a manual, so this manpage is a stub\. For full documentation of the \fBjq\fR language, see:
.
.IP "" 4
.
.nf
http://stedolan\.github\.com/jq
.
.fi
.
.IP "" 0
.
.SH "BUGS"
Presumably\. Report them or discuss them at:
.
.IP "" 4
.
.nf
https://github\.com/stedolan/jq/issues
.
.fi
.
.IP "" 0
.
.SH "AUTHOR"
Stephen Dolan \fB<mu@netsoc\.tcd\.ie>\fR
+3075
View File
File diff suppressed because it is too large. Load diff
+60
View File
@@ -0,0 +1,60 @@
#ifndef JQ_H
#define JQ_H
#include <stdio.h>
#include <jv.h>
enum {JQ_DEBUG_TRACE = 1};
typedef struct jq_state jq_state;
typedef void (*jq_msg_cb)(void *, jv);
jq_state *jq_init(void);
void jq_set_error_cb(jq_state *, jq_msg_cb, void *);
void jq_get_error_cb(jq_state *, jq_msg_cb *, void **);
void jq_set_nomem_handler(jq_state *, void (*)(void *), void *);
jv jq_format_error(jv msg);
void jq_report_error(jq_state *, jv);
int jq_compile(jq_state *, const char*);
int jq_compile_args(jq_state *, const char*, jv);
void jq_dump_disassembly(jq_state *, int);
void jq_start(jq_state *, jv value, int);
jv jq_next(jq_state *);
void jq_teardown(jq_state **);
typedef jv (*jq_input_cb)(jq_state *, void *);
void jq_set_input_cb(jq_state *, jq_input_cb, void *);
void jq_get_input_cb(jq_state *, jq_input_cb *, void **);
void jq_set_debug_cb(jq_state *, jq_msg_cb, void *);
void jq_get_debug_cb(jq_state *, jq_msg_cb *, void **);
void jq_set_attrs(jq_state *, jv);
jv jq_get_attrs(jq_state *);
jv jq_get_jq_origin(jq_state *);
jv jq_get_prog_origin(jq_state *);
jv jq_get_lib_dirs(jq_state *);
void jq_set_attr(jq_state *, jv, jv);
jv jq_get_attr(jq_state *, jv);
/*
* We use char * instead of jf for filenames here because filenames
* should be in the process' locale's codeset, which may not be UTF-8,
* whereas jv string values must be in UTF-8. This way the caller
* doesn't have to perform any codeset conversions.
*/
typedef struct jq_util_input_state jq_util_input_state;
typedef void (*jq_util_msg_cb)(void *, const char *);
jq_util_input_state *jq_util_input_init(jq_util_msg_cb, void *);
void jq_util_input_set_parser(jq_util_input_state *, jv_parser *, int);
void jq_util_input_free(jq_util_input_state **);
void jq_util_input_add_input(jq_util_input_state *, const char *);
int jq_util_input_errors(jq_util_input_state *);
jv jq_util_input_next_input(jq_util_input_state *);
jv jq_util_input_next_input_cb(jq_state *, void *);
jv jq_util_input_get_position(jq_state*);
jv jq_util_input_get_current_filename(jq_state*);
jv jq_util_input_get_current_line(jq_state*);
#endif /* !JQ_H */
+70
View File
@@ -0,0 +1,70 @@
# This is spec file maintained by developers of JQ, not by a OS distro.
# Your OS of choice will likely ignore this RPM spec file.
Summary: Command-line JSON processor
Name: jq
Version: %{myver}
Release: %{myrel}%{?dist}
Source0: jq-%{myver}.tar.gz
URL: http://stedolan.github.io/jq
License: BSD
AutoReqProv: no
#BuildPrereq: autoconf, libtool, automake, flex, bison, python
Group: Applications/System
# Requires:
# Disables debug packages and stripping of binaries:
%global _enable_debug_package 0
%global debug_package %{nil}
%global __os_install_post %{nil}
# Crank up the compression
%define _binary_payload w7.lzdio
%description
jq is a command-line JSON processor
%prep
%setup
%build
echo "Building in: \"$(pwd)\""
%if "%{devbuild}" == "yes"
./configure --prefix=%{_prefix} --enable-devel
%else
./configure --prefix=%{_prefix}
%endif
make
%install
echo "Installing to: \"%{buildroot}\""
make install DESTDIR=%{buildroot}
%clean
rm -rf %{buildroot}
%files
%defattr(-,root,root)
%{_bindir}/jq
%if "%{devbuild}" == "yes"
%{_libexecdir}/%{name}/jq_test
%{_libexecdir}/%{name}/testdata
%endif
%{_datadir}/doc/%{name}/AUTHORS
%{_datadir}/doc/%{name}/COPYING
%{_datadir}/doc/%{name}/README
%{_datadir}/doc/%{name}/README.md
%{_datadir}/man/man1/jq.1
%{_includedir}/jq.h
%{_includedir}/jv.h
%{_prefix}/lib/libjq.a
%{_prefix}/lib/libjq.la
%{_prefix}/lib/libjq.so
%{_prefix}/lib/libjq.so.1
%{_prefix}/lib/libjq.so.1.0.4
%changelog
%pre
%post
+9
View File
@@ -0,0 +1,9 @@
#ifndef JQ_PARSER_H
#define JQ_PARSER_H
#include "locfile.h"
#include "compile.h"
int jq_parse(struct locfile* source, block* answer);
int jq_parse_library(struct locfile* locations, block* answer);
#endif
+346
View File
@@ -0,0 +1,346 @@
#include <assert.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include "jv.h"
#include "jq.h"
static void jv_test();
static void run_jq_tests(jv, int, FILE *);
int jq_testsuite(jv libdirs, int verbose, int argc, char* argv[]) {
FILE *testdata = stdin;
jv_test();
if (argc > 0) {
testdata = fopen(argv[0], "r");
if (!testdata) {
perror("fopen");
exit(1);
}
}
run_jq_tests(libdirs, verbose, testdata);
return 0;
}
static int skipline(const char* buf) {
int p = 0;
while (buf[p] == ' ' || buf[p] == '\t') p++;
if (buf[p] == '#' || buf[p] == '\n' || buf[p] == 0) return 1;
return 0;
}
static int checkerrormsg(const char* buf) {
return strcmp(buf, "%%FAIL\n") == 0;
}
static int checkfail(const char* buf) {
return strcmp(buf, "%%FAIL\n") == 0 || strcmp(buf, "%%FAIL IGNORE MSG\n") == 0;
}
struct err_data {
char buf[4096];
};
static void test_err_cb(void *data, jv e) {
struct err_data *err_data = data;
if (jv_get_kind(e) != JV_KIND_STRING)
e = jv_dump_string(e, JV_PRINT_INVALID);
if (!strncmp(jv_string_value(e), "jq: error", sizeof("jq: error") - 1))
snprintf(err_data->buf, sizeof(err_data->buf), "%s", jv_string_value(e));
if (strchr(err_data->buf, '\n'))
*(strchr(err_data->buf, '\n')) = '\0';
jv_free(e);
}
static void run_jq_tests(jv lib_dirs, int verbose, FILE *testdata) {
char prog[4096];
char buf[4096];
struct err_data err_msg;
int tests = 0, passed = 0, invalid = 0;
unsigned int lineno = 0;
int must_fail = 0;
int check_msg = 0;
jq_state *jq = NULL;
jq = jq_init();
assert(jq);
if (jv_get_kind(lib_dirs) == JV_KIND_NULL)
lib_dirs = jv_array();
jq_set_attr(jq, jv_string("JQ_LIBRARY_PATH"), lib_dirs);
while (1) {
if (!fgets(prog, sizeof(prog), testdata)) break;
lineno++;
if (skipline(prog)) continue;
if (checkfail(prog)) {
must_fail = 1;
check_msg = checkerrormsg(prog);
jq_set_error_cb(jq, test_err_cb, &err_msg);
continue;
}
if (prog[strlen(prog)-1] == '\n') prog[strlen(prog)-1] = 0;
printf("Testing '%s' at line number %u\n", prog, lineno);
int pass = 1;
tests++;
int compiled = jq_compile(jq, prog);
if (must_fail) {
jq_set_error_cb(jq, NULL, NULL);
if (!fgets(buf, sizeof(buf), testdata)) { invalid++; break; }
lineno++;
if (buf[strlen(buf)-1] == '\n') buf[strlen(buf)-1] = 0;
if (compiled) {
printf("*** Test program compiled that should not have at line %u: %s\n", lineno, prog);
must_fail = 0;
check_msg = 0;
invalid++;
continue;
}
if (check_msg && strcmp(buf, err_msg.buf) != 0) {
printf("*** Erroneous test program failed with wrong message (%s) at line %u: %s\n", err_msg.buf, lineno, prog);
invalid++;
} else {
passed++;
}
must_fail = 0;
check_msg = 0;
continue;
}
if (!compiled) {
printf("*** Test program failed to compile at line %u: %s\n", lineno, prog);
invalid++;
// skip past test data
while (fgets(buf, sizeof(buf), testdata)) {
lineno++;
if (buf[0] == '\n' || (buf[0] == '\r' && buf[1] == '\n'))
break;
}
continue;
}
if (verbose) {
printf("Disassembly:\n");
jq_dump_disassembly(jq, 2);
printf("\n");
}
if (!fgets(buf, sizeof(buf), testdata)) { invalid++; break; }
lineno++;
jv input = jv_parse(buf);
if (!jv_is_valid(input)) {
printf("*** Input is invalid on line %u: %s\n", lineno, buf);
invalid++;
continue;
}
jq_start(jq, input, verbose ? JQ_DEBUG_TRACE : 0);
while (fgets(buf, sizeof(buf), testdata)) {
lineno++;
if (skipline(buf)) break;
jv expected = jv_parse(buf);
if (!jv_is_valid(expected)) {
printf("*** Expected result is invalid on line %u: %s\n", lineno, buf);
invalid++;
continue;
}
jv actual = jq_next(jq);
if (!jv_is_valid(actual)) {
jv_free(actual);
printf("*** Insufficient results for test at line number %u: %s\n", lineno, prog);
pass = 0;
break;
} else if (!jv_equal(jv_copy(expected), jv_copy(actual))) {
printf("*** Expected ");
jv_dump(jv_copy(expected), 0);
printf(", but got ");
jv_dump(jv_copy(actual), 0);
printf(" for test at line number %u: %s\n", lineno, prog);
pass = 0;
}
jv as_string = jv_dump_string(jv_copy(expected), rand() & ~(JV_PRINT_COLOUR|JV_PRINT_REFCOUNT));
jv reparsed = jv_parse_sized(jv_string_value(as_string), jv_string_length_bytes(jv_copy(as_string)));
assert(jv_equal(jv_copy(expected), jv_copy(reparsed)));
jv_free(as_string);
jv_free(reparsed);
jv_free(expected);
jv_free(actual);
}
if (pass) {
jv extra = jq_next(jq);
if (jv_is_valid(extra)) {
printf("*** Superfluous result: ");
jv_dump(extra, 0);
printf(" for test at line number %u, %s\n", lineno, prog);
pass = 0;
} else {
jv_free(extra);
}
}
passed+=pass;
}
jq_teardown(&jq);
printf("%d of %d tests passed (%d malformed)\n", passed,tests,invalid);
if (passed != tests) exit(1);
}
static void jv_test() {
/// JSON parser regression tests
{
jv v = jv_parse("{\"a':\"12\"}");
assert(jv_get_kind(v) == JV_KIND_INVALID);
v = jv_invalid_get_msg(v);
assert(strcmp(jv_string_value(v), "Expected separator between values at line 1, column 9 (while parsing '{\"a':\"12\"}')") == 0);
jv_free(v);
}
/// Arrays and numbers
{
jv a = jv_array();
assert(jv_get_kind(a) == JV_KIND_ARRAY);
assert(jv_array_length(jv_copy(a)) == 0);
assert(jv_array_length(jv_copy(a)) == 0);
a = jv_array_append(a, jv_number(42));
assert(jv_array_length(jv_copy(a)) == 1);
assert(jv_number_value(jv_array_get(jv_copy(a), 0)) == 42);
jv a2 = jv_array_append(jv_array(), jv_number(42));
assert(jv_equal(jv_copy(a), jv_copy(a)));
assert(jv_equal(jv_copy(a2), jv_copy(a2)));
assert(jv_equal(jv_copy(a), jv_copy(a2)));
assert(jv_equal(jv_copy(a2), jv_copy(a)));
jv_free(a2);
a2 = jv_array_append(jv_array(), jv_number(19));
assert(!jv_equal(jv_copy(a), jv_copy(a2)));
assert(!jv_equal(jv_copy(a2), jv_copy(a)));
jv_free(a2);
assert(jv_get_refcnt(a) == 1);
a = jv_array_append(a, jv_copy(a));
assert(jv_get_refcnt(a) == 1);
assert(jv_array_length(jv_copy(a)) == 2);
assert(jv_number_value(jv_array_get(jv_copy(a), 0)) == 42);
for (int i=0; i<10; i++) {
jv subarray = jv_array_get(jv_copy(a), 1);
assert(jv_get_kind(subarray) == JV_KIND_ARRAY);
assert(jv_array_length(jv_copy(subarray)) == 1);
assert(jv_number_value(jv_array_get(jv_copy(subarray), 0)) == 42);
jv_free(subarray);
}
jv subarray = jv_array_get(jv_copy(a), 1);
assert(jv_get_kind(subarray) == JV_KIND_ARRAY);
assert(jv_array_length(jv_copy(subarray)) == 1);
assert(jv_number_value(jv_array_get(jv_copy(subarray), 0)) == 42);
jv sub2 = jv_copy(subarray);
sub2 = jv_array_append(sub2, jv_number(19));
assert(jv_get_kind(sub2) == JV_KIND_ARRAY);
assert(jv_array_length(jv_copy(sub2)) == 2);
assert(jv_number_value(jv_array_get(jv_copy(sub2), 0)) == 42);
assert(jv_number_value(jv_array_get(jv_copy(sub2), 1)) == 19);
assert(jv_get_kind(subarray) == JV_KIND_ARRAY);
assert(jv_array_length(jv_copy(subarray)) == 1);
assert(jv_number_value(jv_array_get(jv_copy(subarray), 0)) == 42);
jv_free(subarray);
void* before = sub2.u.ptr;
sub2 = jv_array_append(sub2, jv_number(200));
void* after = sub2.u.ptr;
assert(before == after);
jv_free(sub2);
jv a3 = jv_array_append(jv_copy(a), jv_number(19));
assert(jv_array_length(jv_copy(a3)) == 3);
assert(jv_number_value(jv_array_get(jv_copy(a3), 0)) == 42);
assert(jv_array_length(jv_array_get(jv_copy(a3), 1)) == 1);
assert(jv_number_value(jv_array_get(jv_copy(a3), 2)) == 19);
jv_free(a3);
jv a4 = jv_array();
a4 = jv_array_append(a4, jv_number(1));
a4 = jv_array_append(a4, jv_number(2));
jv a5 = jv_copy(a4);
a4 = jv_array_append(a4, jv_number(3));
a4 = jv_array_slice(a4, 0, 1);
assert(jv_array_length(jv_copy(a4)) == 1);
a4 = jv_array_append(a4, jv_number(4));
assert(jv_array_length(jv_copy(a4)) == 2);
assert(jv_array_length(jv_copy(a5)) == 2);
jv_free(a4);
jv_free(a5);
assert(jv_array_length(jv_copy(a)) == 2);
assert(jv_number_value(jv_array_get(jv_copy(a), 0)) == 42);
assert(jv_array_length(jv_array_get(jv_copy(a), 1)) == 1);
//jv_dump(jv_copy(a), 0); printf("\n");
jv_free(a);
}
/// Strings
{
assert(jv_equal(jv_string("foo"), jv_string_sized("foo", 3)));
char nasty[] = "foo\0";
jv shortstr = jv_string(nasty), longstr = jv_string_sized(nasty, sizeof(nasty));
assert(jv_string_length_bytes(jv_copy(shortstr)) == (int)strlen(nasty));
assert(jv_string_length_bytes(jv_copy(longstr)) == (int)sizeof(nasty));
jv_free(shortstr);
jv_free(longstr);
char a1s[] = "hello", a2s[] = "hello", bs[] = "goodbye";
jv a1 = jv_string(a1s), a2 = jv_string(a2s), b = jv_string(bs);
assert(jv_equal(jv_copy(a1), jv_copy(a2)));
assert(jv_equal(jv_copy(a2), jv_copy(a1)));
assert(!jv_equal(jv_copy(a1), jv_copy(b)));
assert(jv_string_hash(jv_copy(a1)) == jv_string_hash(jv_copy(a1)));
assert(jv_string_hash(jv_copy(a1)) == jv_string_hash(jv_copy(a2)));
assert(jv_string_hash(jv_copy(b)) != jv_string_hash(jv_copy(a1)));
jv_free(a1);
jv_free(a2);
jv_free(b);
assert(jv_equal(jv_string("hello42!"), jv_string_fmt("hello%d%s", 42, "!")));
char big[20000];
for (int i=0; i<(int)sizeof(big); i++) big[i] = 'a';
big[sizeof(big)-1] = 0;
jv str = jv_string_fmt("%s", big);
assert(jv_string_length_bytes(jv_copy(str)) == sizeof(big) - 1);
assert(!strcmp(big, jv_string_value(str)));
jv_free(str);
}
/// Objects
{
jv o1 = jv_object();
o1 = jv_object_set(o1, jv_string("foo"), jv_number(42));
o1 = jv_object_set(o1, jv_string("bar"), jv_number(24));
assert(jv_number_value(jv_object_get(jv_copy(o1), jv_string("foo"))) == 42);
assert(jv_number_value(jv_object_get(jv_copy(o1), jv_string("bar"))) == 24);
jv o2 = jv_object_set(jv_copy(o1), jv_string("foo"), jv_number(420));
o2 = jv_object_set(o2, jv_string("bar"), jv_number(240));
assert(jv_number_value(jv_object_get(jv_copy(o1), jv_string("foo"))) == 42);
assert(jv_number_value(jv_object_get(jv_copy(o1), jv_string("bar"))) == 24);
assert(jv_number_value(jv_object_get(jv_copy(o2), jv_string("foo"))) == 420);
jv_free(o1);
assert(jv_number_value(jv_object_get(jv_copy(o2), jv_string("bar"))) == 240);
//jv_dump(jv_copy(o2), 0); printf("\n");
jv_free(o2);
}
}
+1333
View File
File diff suppressed because it is too large. Load diff
+240
View File
@@ -0,0 +1,240 @@
#ifndef JV_H
#define JV_H
#include <stdarg.h>
#include <stdint.h>
#include <stdio.h>
typedef enum {
JV_KIND_INVALID,
JV_KIND_NULL,
JV_KIND_FALSE,
JV_KIND_TRUE,
JV_KIND_NUMBER,
JV_KIND_STRING,
JV_KIND_ARRAY,
JV_KIND_OBJECT
} jv_kind;
struct jv_refcnt;
/* All of the fields of this struct are private.
Really. Do not play with them. */
typedef struct {
unsigned char kind_flags;
unsigned char pad_;
unsigned short offset; /* array offsets */
int size;
union {
struct jv_refcnt* ptr;
double number;
} u;
} jv;
/*
* All jv_* functions consume (decref) input and produce (incref) output
* Except jv_copy
*/
jv_kind jv_get_kind(jv);
const char* jv_kind_name(jv_kind);
static int jv_is_valid(jv x) { return jv_get_kind(x) != JV_KIND_INVALID; }
jv jv_copy(jv);
void jv_free(jv);
int jv_get_refcnt(jv);
int jv_equal(jv, jv);
int jv_identical(jv, jv);
int jv_contains(jv, jv);
jv jv_invalid(void);
jv jv_invalid_with_msg(jv);
jv jv_invalid_get_msg(jv);
int jv_invalid_has_msg(jv);
jv jv_null(void);
jv jv_true(void);
jv jv_false(void);
jv jv_bool(int);
jv jv_number(double);
double jv_number_value(jv);
int jv_is_integer(jv);
jv jv_array(void);
jv jv_array_sized(int);
int jv_array_length(jv);
jv jv_array_get(jv, int);
jv jv_array_set(jv, int, jv);
jv jv_array_append(jv, jv);
jv jv_array_concat(jv, jv);
jv jv_array_slice(jv, int, int);
jv jv_array_indexes(jv, jv);
#define jv_array_foreach(a, i, x) \
for (int jv_len__ = jv_array_length(jv_copy(a)), i=0, jv_j__ = 1; \
jv_j__; jv_j__ = 0) \
for (jv x; \
i < jv_len__ ? \
(x = jv_array_get(jv_copy(a), i), 1) : 0; \
i++)
#define JV_ARRAY_1(e) (jv_array_append(jv_array(),e))
#define JV_ARRAY_2(e1,e2) (jv_array_append(JV_ARRAY_1(e1),e2))
#define JV_ARRAY_3(e1,e2,e3) (jv_array_append(JV_ARRAY_2(e1,e2),e3))
#define JV_ARRAY_4(e1,e2,e3,e4) (jv_array_append(JV_ARRAY_3(e1,e2,e3),e4))
#define JV_ARRAY_5(e1,e2,e3,e4,e5) (jv_array_append(JV_ARRAY_4(e1,e2,e3,e4),e5))
#define JV_ARRAY_6(e1,e2,e3,e4,e5,e6) (jv_array_append(JV_ARRAY_5(e1,e2,e3,e4,e5),e6))
#define JV_ARRAY_7(e1,e2,e3,e4,e5,e6,e7) (jv_array_append(JV_ARRAY_6(e1,e2,e3,e4,e5,e6),e7))
#define JV_ARRAY_8(e1,e2,e3,e4,e5,e6,e7,e8) (jv_array_append(JV_ARRAY_7(e1,e2,e3,e4,e5,e6,e7),e8))
#define JV_ARRAY_9(e1,e2,e3,e4,e5,e6,e7,e8,e9) (jv_array_append(JV_ARRAY_8(e1,e2,e3,e4,e5,e6,e7,e8),e9))
#define JV_ARRAY_IDX(_1,_2,_3,_4,_5,_6,_7,_8,_9,NAME,...) NAME
#define JV_ARRAY(...) \
JV_ARRAY_IDX(__VA_ARGS__, JV_ARRAY_9, JV_ARRAY_8, JV_ARRAY_7, JV_ARRAY_6, JV_ARRAY_5, JV_ARRAY_4, JV_ARRAY_3, JV_ARRAY_2, JV_ARRAY_1)(__VA_ARGS__)
#ifdef __GNUC__
#define JV_PRINTF_LIKE(fmt_arg_num, args_num) \
__attribute__ ((__format__( __printf__, fmt_arg_num, args_num)))
#define JV_VPRINTF_LIKE(fmt_arg_num) \
__attribute__ ((__format__( __printf__, fmt_arg_num, 0)))
#endif
jv jv_string(const char*);
jv jv_string_sized(const char*, int);
jv jv_string_empty(int len);
int jv_string_length_bytes(jv);
int jv_string_length_codepoints(jv);
unsigned long jv_string_hash(jv);
const char* jv_string_value(jv);
jv jv_string_indexes(jv j, jv k);
jv jv_string_slice(jv j, int start, int end);
jv jv_string_concat(jv, jv);
jv jv_string_vfmt(const char*, va_list) JV_VPRINTF_LIKE(1);
jv jv_string_fmt(const char*, ...) JV_PRINTF_LIKE(1, 2);
jv jv_string_append_codepoint(jv a, uint32_t c);
jv jv_string_append_buf(jv a, const char* buf, int len);
jv jv_string_append_str(jv a, const char* str);
jv jv_string_split(jv j, jv sep);
jv jv_string_explode(jv j);
jv jv_string_implode(jv j);
jv jv_object(void);
jv jv_object_get(jv object, jv key);
jv jv_object_set(jv object, jv key, jv value);
jv jv_object_delete(jv object, jv key);
int jv_object_length(jv object);
jv jv_object_merge(jv, jv);
jv jv_object_merge_recursive(jv, jv);
int jv_object_iter(jv);
int jv_object_iter_next(jv, int);
int jv_object_iter_valid(jv, int);
jv jv_object_iter_key(jv, int);
jv jv_object_iter_value(jv, int);
#define jv_object_foreach(t, k, v) \
for (int jv_i__ = jv_object_iter(t), jv_j__ = 1; jv_j__; jv_j__ = 0) \
for (jv k, v; \
jv_object_iter_valid((t), jv_i__) ? \
(k = jv_object_iter_key(t, jv_i__), \
v = jv_object_iter_value(t, jv_i__), \
1) \
: 0; \
jv_i__ = jv_object_iter_next(t, jv_i__)) \
#define JV_OBJECT_1(k) (jv_object_set(jv_object(),(k),jv_null()))
#define JV_OBJECT_2(k1,v1) (jv_object_set(jv_object(),(k1),(v1)))
#define JV_OBJECT_3(k1,v1,k2) (jv_object_set(JV_OBJECT_2(k1,v1),k2,jv_null()))
#define JV_OBJECT_4(k1,v1,k2,v2) (jv_object_set(JV_OBJECT_2(k1,v1),k2,v2))
#define JV_OBJECT_5(k1,v1,k2,v2,k3) (jv_object_set(JV_OBJECT_4(k1,v1,k2,v2),k3,jv_null))
#define JV_OBJECT_6(k1,v1,k2,v2,k3,v3) (jv_object_set(JV_OBJECT_4(k1,v1,k2,v2),k3,v3))
#define JV_OBJECT_7(k1,v1,k2,v2,k3,v3,k4) (jv_object_set(JV_OBJECT_6(k1,v1,k2,v2,k3,v3),k4,jv_null()))
#define JV_OBJECT_8(k1,v1,k2,v2,k3,v3,k4,v4) (jv_object_set(JV_OBJECT_6(k1,v1,k2,v2,k3,v3),k4,v4))
#define JV_OBJECT_IDX(_1,_2,_3,_4,_5,_6,_7,_8,NAME,...) NAME
#define JV_OBJECT(...) \
JV_OBJECT_IDX(__VA_ARGS__, JV_OBJECT_8, JV_OBJECT_7, JV_OBJECT_6, JV_OBJECT_5, JV_OBJECT_4, JV_OBJECT_3, JV_OBJECT_2, JV_OBJECT_1)(__VA_ARGS__)
int jv_get_refcnt(jv);
enum jv_print_flags {
JV_PRINT_PRETTY = 1,
JV_PRINT_ASCII = 2,
JV_PRINT_COLOUR = 4,
JV_PRINT_SORTED = 8,
JV_PRINT_INVALID = 16,
JV_PRINT_REFCOUNT = 32,
JV_PRINT_TAB = 64,
JV_PRINT_ISATTY = 128,
JV_PRINT_SPACE0 = 256,
JV_PRINT_SPACE1 = 512,
JV_PRINT_SPACE2 = 1024,
};
#define JV_PRINT_INDENT_FLAGS(n) \
((n) < 0 || (n) > 7 ? JV_PRINT_TAB | JV_PRINT_PRETTY : (n) == 0 ? 0 : (n) << 8 | JV_PRINT_PRETTY)
void jv_dumpf(jv, FILE *f, int flags);
void jv_dump(jv, int flags);
void jv_show(jv, int flags);
jv jv_dump_string(jv, int flags);
char *jv_dump_string_trunc(jv x, char *outbuf, size_t bufsize);
enum {
JV_PARSE_SEQ = 1,
JV_PARSE_STREAMING = 2,
JV_PARSE_STREAM_ERRORS = 4,
};
jv jv_parse(const char* string);
jv jv_parse_sized(const char* string, int length);
typedef void (*jv_nomem_handler_f)(void *);
void jv_nomem_handler(jv_nomem_handler_f, void *);
jv jv_load_file(const char *, int);
typedef struct jv_parser jv_parser;
jv_parser* jv_parser_new(int);
void jv_parser_set_buf(jv_parser*, const char*, int, int);
int jv_parser_remaining(jv_parser*);
jv jv_parser_next(jv_parser*);
void jv_parser_free(jv_parser*);
jv jv_get(jv, jv);
jv jv_set(jv, jv, jv);
jv jv_has(jv, jv);
jv jv_setpath(jv, jv, jv);
jv jv_getpath(jv, jv);
jv jv_delpaths(jv, jv);
jv jv_keys(jv /*object or array*/);
jv jv_keys_unsorted(jv /*object or array*/);
int jv_cmp(jv, jv);
jv jv_group(jv, jv);
jv jv_sort(jv, jv);
#endif
/*
true/false/null:
check kind
number:
introduce/eliminate jv
to integer
array:
copy
free
slice
index
update
updateslice?
*/
+179
View File
@@ -0,0 +1,179 @@
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "jv_alloc.h"
struct nomem_handler {
jv_nomem_handler_f handler;
void *data;
};
#if !defined(HAVE_PTHREAD_KEY_CREATE) || \
!defined(HAVE_PTHREAD_ONCE) || \
!defined(HAVE_ATEXIT)
/* Try thread-local storage? */
#ifdef _MSC_VER
/* Visual C++: yes */
static __declspec(thread) struct nomem_handler nomem_handler;
#define USE_TLS
#else
#ifdef HAVE___THREAD
/* GCC and friends: yes */
static __thread struct nomem_handler nomem_handler;
#define USE_TLS
#endif /* HAVE___THREAD */
#endif /* _MSC_VER */
#endif /* !HAVE_PTHREAD_KEY_CREATE */
#ifdef USE_TLS
void jv_nomem_handler(jv_nomem_handler_f handler, void *data) {
nomem_handler.handler = handler;
}
static void memory_exhausted() {
if (nomem_handler.handler)
nomem_handler.handler(nomem_handler.data); // Maybe handler() will longjmp() to safety
// Or not
fprintf(stderr, "error: cannot allocate memory\n");
abort();
}
#else /* USE_TLS */
#ifdef HAVE_PTHREAD_KEY_CREATE
#include <pthread.h>
pthread_key_t nomem_handler_key;
pthread_once_t mem_once = PTHREAD_ONCE_INIT;
static void tsd_fini(void) {
struct nomem_handler *nomem_handler;
nomem_handler = pthread_getspecific(nomem_handler_key);
if (nomem_handler) {
(void) pthread_setspecific(nomem_handler_key, NULL);
free(nomem_handler);
}
}
static void tsd_init(void) {
if (pthread_key_create(&nomem_handler_key, NULL) != 0) {
fprintf(stderr, "error: cannot create thread specific key");
abort();
}
if (atexit(tsd_fini) != 0) {
fprintf(stderr, "error: cannot set an exit handler");
abort();
}
struct nomem_handler *nomem_handler = calloc(1, sizeof(struct nomem_handler));
if (pthread_setspecific(nomem_handler_key, nomem_handler) != 0) {
fprintf(stderr, "error: cannot set thread specific data");
abort();
}
}
void jv_nomem_handler(jv_nomem_handler_f handler, void *data) {
pthread_once(&mem_once, tsd_init); // cannot fail
struct nomem_handler *nomem_handler;
nomem_handler = pthread_getspecific(nomem_handler_key);
if (nomem_handler == NULL) {
handler(data);
fprintf(stderr, "error: cannot allocate memory\n");
abort();
}
nomem_handler->handler = handler;
nomem_handler->data = data;
}
static void memory_exhausted() {
struct nomem_handler *nomem_handler;
pthread_once(&mem_once, tsd_init);
nomem_handler = pthread_getspecific(nomem_handler_key);
if (nomem_handler)
nomem_handler->handler(nomem_handler->data); // Maybe handler() will longjmp() to safety
// Or not
fprintf(stderr, "error: cannot allocate memory\n");
abort();
}
#else
/* No thread-local storage of any kind that we know how to handle */
static struct nomem_handler nomem_handler;
void jv_nomem_handler(jv_nomem_handler_f handler, void *data) {
nomem_handler.handler = handler;
nomem_handler.data = data;
}
static void memory_exhausted() {
fprintf(stderr, "error: cannot allocate memory\n");
abort();
}
#endif /* HAVE_PTHREAD_KEY_CREATE */
#endif /* USE_TLS */
void* jv_mem_alloc(size_t sz) {
void* p = malloc(sz);
if (!p) {
memory_exhausted();
}
return p;
}
void* jv_mem_alloc_unguarded(size_t sz) {
return malloc(sz);
}
void* jv_mem_calloc(size_t nemb, size_t sz) {
void* p = calloc(nemb, sz);
if (!p) {
memory_exhausted();
}
return p;
}
void* jv_mem_calloc_unguarded(size_t nemb, size_t sz) {
return calloc(nemb, sz);
}
char* jv_mem_strdup(const char *s) {
char *p = strdup(s);
if (!p) {
memory_exhausted();
}
return p;
}
char* jv_mem_strdup_unguarded(const char *s) {
return strdup(s);
}
void jv_mem_free(void* p) {
free(p);
}
void* jv_mem_realloc(void* p, size_t sz) {
p = realloc(p, sz);
if (!p) {
memory_exhausted();
}
return p;
}
#ifndef NDEBUG
volatile char jv_mem_uninitialised;
__attribute__((constructor)) void jv_mem_uninit_setup(){
// ignore warning that this reads uninitialized memory - that's the point!
#ifndef __clang_analyzer__
char* p = malloc(1);
jv_mem_uninitialised = *p;
free(p);
#endif
}
#endif
+27
View File
@@ -0,0 +1,27 @@
#ifndef JV_ALLOC_H
#define JV_ALLOC_H
#include <stddef.h>
#include "jv.h"
#ifndef NDEBUG
extern volatile char jv_mem_uninitialised;
#endif
static void jv_mem_invalidate(void* mem, size_t n) {
#ifndef NDEBUG
char* m = mem;
while (n--) *m++ ^= jv_mem_uninitialised ^ jv_mem_uninitialised;
#endif
}
void* jv_mem_alloc(size_t);
void* jv_mem_alloc_unguarded(size_t);
void* jv_mem_calloc(size_t, size_t);
void* jv_mem_calloc_unguarded(size_t, size_t);
char* jv_mem_strdup(const char *);
char* jv_mem_strdup_unguarded(const char *);
void jv_mem_free(void*);
__attribute__((warn_unused_result)) void* jv_mem_realloc(void*, size_t);
#endif
+617
View File
@@ -0,0 +1,617 @@
#include <string.h>
#include <stdlib.h>
#include <assert.h>
#include "jv_alloc.h"
static int parse_slice(jv j, jv slice, int* pstart, int* pend) {
// Array slices
jv start_jv = jv_object_get(jv_copy(slice), jv_string("start"));
jv end_jv = jv_object_get(slice, jv_string("end"));
if (jv_get_kind(start_jv) == JV_KIND_NULL) {
jv_free(start_jv);
start_jv = jv_number(0);
}
int len;
if (jv_get_kind(j) == JV_KIND_ARRAY)
len = jv_array_length(jv_copy(j));
else
len = jv_string_length_codepoints(jv_copy(j));
if (jv_get_kind(end_jv) == JV_KIND_NULL) {
jv_free(end_jv);
end_jv = jv_number(len);
}
if (jv_get_kind(start_jv) != JV_KIND_NUMBER ||
jv_get_kind(end_jv) != JV_KIND_NUMBER) {
jv_free(start_jv);
jv_free(end_jv);
return 0;
} else {
double dstart = jv_number_value(start_jv);
double dend = jv_number_value(end_jv);
if (dstart < 0) dstart += len;
if (dend < 0) dend += len;
if (dstart < 0) dstart = 0;
if (dstart > len) dstart = len;
int start = (int)dstart;
int end = (dend > len) ? len : (int)dend;
// Ends are exclusive but e.g. 1 < 1.5 so :1.5 should be :2 not :1
if(end < dend) end += 1;
if (end > len) end = len;
if (end < start) end = start;
assert(0 <= start && start <= end && end <= len);
*pstart = start;
*pend = end;
return 1;
}
}
jv jv_get(jv t, jv k) {
jv v;
if (jv_get_kind(t) == JV_KIND_OBJECT && jv_get_kind(k) == JV_KIND_STRING) {
v = jv_object_get(t, k);
if (!jv_is_valid(v)) {
jv_free(v);
v = jv_null();
}
} else if (jv_get_kind(t) == JV_KIND_ARRAY && jv_get_kind(k) == JV_KIND_NUMBER) {
if(jv_is_integer(k)){
int idx = (int)jv_number_value(k);
if (idx < 0)
idx += jv_array_length(jv_copy(t));
v = jv_array_get(t, idx);
if (!jv_is_valid(v)) {
jv_free(v);
v = jv_null();
}
} else {
jv_free(t);
jv_free(k);
v = jv_null();
}
} else if (jv_get_kind(t) == JV_KIND_ARRAY && jv_get_kind(k) == JV_KIND_OBJECT) {
int start, end;
if (parse_slice(t, k, &start, &end)) {
v = jv_array_slice(t, start, end);
} else {
v = jv_invalid_with_msg(jv_string_fmt("Start and end indices of an array slice must be numbers"));
jv_free(t);
}
} else if (jv_get_kind(t) == JV_KIND_STRING && jv_get_kind(k) == JV_KIND_OBJECT) {
int start, end;
if (parse_slice(t, k, &start, &end)) {
v = jv_string_slice(t, start, end);
} else {
v = jv_invalid_with_msg(jv_string_fmt("Start and end indices of an string slice must be numbers"));
jv_free(t);
}
} else if (jv_get_kind(t) == JV_KIND_ARRAY && jv_get_kind(k) == JV_KIND_ARRAY) {
v = jv_array_indexes(t, k);
} else if (jv_get_kind(t) == JV_KIND_NULL &&
(jv_get_kind(k) == JV_KIND_STRING ||
jv_get_kind(k) == JV_KIND_NUMBER ||
jv_get_kind(k) == JV_KIND_OBJECT)) {
jv_free(t);
jv_free(k);
v = jv_null();
} else {
/*
* If k is a short string it's probably from a jq .foo expression or
* similar, in which case putting it in the invalid msg may help the
* user. The length 30 is arbitrary.
*/
if (jv_get_kind(k) == JV_KIND_STRING && jv_string_length_bytes(jv_copy(k)) < 30) {
v = jv_invalid_with_msg(jv_string_fmt("Cannot index %s with string \"%s\"",
jv_kind_name(jv_get_kind(t)),
jv_string_value(k)));
} else {
v = jv_invalid_with_msg(jv_string_fmt("Cannot index %s with %s",
jv_kind_name(jv_get_kind(t)),
jv_kind_name(jv_get_kind(k))));
}
jv_free(t);
jv_free(k);
}
return v;
}
jv jv_set(jv t, jv k, jv v) {
if (!jv_is_valid(v)) {
jv_free(t);
jv_free(k);
return v;
}
int isnull = jv_get_kind(t) == JV_KIND_NULL;
if (jv_get_kind(k) == JV_KIND_STRING &&
(jv_get_kind(t) == JV_KIND_OBJECT || isnull)) {
if (isnull) t = jv_object();
t = jv_object_set(t, k, v);
} else if (jv_get_kind(k) == JV_KIND_NUMBER &&
(jv_get_kind(t) == JV_KIND_ARRAY || isnull)) {
if (isnull) t = jv_array();
t = jv_array_set(t, (int)jv_number_value(k), v);
} else if (jv_get_kind(k) == JV_KIND_OBJECT &&
(jv_get_kind(t) == JV_KIND_ARRAY || isnull)) {
if (isnull) t = jv_array();
int start, end;
if (parse_slice(t, k, &start, &end)) {
if (jv_get_kind(v) == JV_KIND_ARRAY) {
int array_len = jv_array_length(jv_copy(t));
assert(0 <= start && start <= end && end <= array_len);
int slice_len = end - start;
int insert_len = jv_array_length(jv_copy(v));
if (slice_len < insert_len) {
// array is growing
int shift = insert_len - slice_len;
for (int i = array_len - 1; i >= end; i--) {
t = jv_array_set(t, i + shift, jv_array_get(jv_copy(t), i));
}
} else if (slice_len > insert_len) {
// array is shrinking
int shift = slice_len - insert_len;
for (int i = end; i < array_len; i++) {
t = jv_array_set(t, i - shift, jv_array_get(jv_copy(t), i));
}
t = jv_array_slice(t, 0, array_len - shift);
}
for (int i=0; i < insert_len; i++) {
t = jv_array_set(t, start + i, jv_array_get(jv_copy(v), i));
}
jv_free(v);
} else {
jv_free(t);
jv_free(v);
t = jv_invalid_with_msg(jv_string_fmt("A slice of an array can only be assigned another array"));
}
} else {
jv_free(t);
jv_free(k);
jv_free(v);
t = jv_invalid_with_msg(jv_string_fmt("Start and end indices of an array slice must be numbers"));
}
} else {
jv err = jv_invalid_with_msg(jv_string_fmt("Cannot update field at %s index of %s",
jv_kind_name(jv_get_kind(k)),
jv_kind_name(jv_get_kind(t))));
jv_free(t);
jv_free(k);
jv_free(v);
t = err;
}
return t;
}
jv jv_has(jv t, jv k) {
assert(jv_is_valid(t));
assert(jv_is_valid(k));
jv ret;
if (jv_get_kind(t) == JV_KIND_NULL) {
jv_free(t);
jv_free(k);
ret = jv_false();
} else if (jv_get_kind(t) == JV_KIND_OBJECT &&
jv_get_kind(k) == JV_KIND_STRING) {
jv elem = jv_object_get(t, k);
ret = jv_bool(jv_is_valid(elem));
jv_free(elem);
} else if (jv_get_kind(t) == JV_KIND_ARRAY &&
jv_get_kind(k) == JV_KIND_NUMBER) {
jv elem = jv_array_get(t, (int)jv_number_value(k));
ret = jv_bool(jv_is_valid(elem));
jv_free(elem);
} else {
ret = jv_invalid_with_msg(jv_string_fmt("Cannot check whether %s has a %s key",
jv_kind_name(jv_get_kind(t)),
jv_kind_name(jv_get_kind(k))));
jv_free(t);
jv_free(k);
}
return ret;
}
// assumes keys is a sorted array
jv jv_dels(jv t, jv keys) {
assert(jv_get_kind(keys) == JV_KIND_ARRAY);
assert(jv_is_valid(t));
if (jv_get_kind(t) == JV_KIND_NULL || jv_array_length(jv_copy(keys)) == 0) {
// no change
} else if (jv_get_kind(t) == JV_KIND_ARRAY) {
// extract slices, they must be handled differently
jv orig_keys = keys;
keys = jv_array();
jv new_array = jv_array();
jv starts = jv_array(), ends = jv_array();
jv_array_foreach(orig_keys, i, key) {
if (jv_get_kind(key) == JV_KIND_NUMBER) {
keys = jv_array_append(keys, key);
} else if (jv_get_kind(key) == JV_KIND_OBJECT) {
int start, end;
if (parse_slice(t, key, &start, &end)) {
starts = jv_array_append(starts, jv_number(start));
ends = jv_array_append(ends, jv_number(end));
} else {
jv_free(new_array);
jv_free(key);
new_array = jv_invalid_with_msg(jv_string_fmt("Start and end indices of an array slice must be numbers"));
goto arr_out;
}
} else {
jv_free(new_array);
new_array = jv_invalid_with_msg(jv_string_fmt("Cannot delete %s element of array",
jv_kind_name(jv_get_kind(key))));
jv_free(key);
goto arr_out;
}
}
int kidx = 0;
jv_array_foreach(t, i, elem) {
int del = 0;
while (kidx < jv_array_length(jv_copy(keys))) {
int delidx = (int)jv_number_value(jv_array_get(jv_copy(keys), kidx));
if (i == delidx) {
del = 1;
}
if (i < delidx) {
break;
}
kidx++;
}
for (int sidx=0; !del && sidx<jv_array_length(jv_copy(starts)); sidx++) {
if ((int)jv_number_value(jv_array_get(jv_copy(starts), sidx)) <= i &&
i < (int)jv_number_value(jv_array_get(jv_copy(ends), sidx))) {
del = 1;
}
}
if (!del)
new_array = jv_array_append(new_array, elem);
else
jv_free(elem);
}
arr_out:
jv_free(starts);
jv_free(ends);
jv_free(orig_keys);
jv_free(t);
t = new_array;
} else if (jv_get_kind(t) == JV_KIND_OBJECT) {
jv_array_foreach(keys, i, k) {
if (jv_get_kind(k) != JV_KIND_STRING) {
jv_free(t);
t = jv_invalid_with_msg(jv_string_fmt("Cannot delete %s field of object",
jv_kind_name(jv_get_kind(k))));
jv_free(k);
break;
}
t = jv_object_delete(t, k);
}
} else {
jv err = jv_invalid_with_msg(jv_string_fmt("Cannot delete fields from %s",
jv_kind_name(jv_get_kind(t))));
jv_free(t);
t = err;
}
jv_free(keys);
return t;
}
jv jv_setpath(jv root, jv path, jv value) {
if (jv_get_kind(path) != JV_KIND_ARRAY) {
jv_free(value);
jv_free(root);
jv_free(path);
return jv_invalid_with_msg(jv_string("Path must be specified as an array"));
}
if (!jv_is_valid(root)){
jv_free(value);
jv_free(path);
return root;
}
if (jv_array_length(jv_copy(path)) == 0) {
jv_free(path);
jv_free(root);
return value;
}
jv pathcurr = jv_array_get(jv_copy(path), 0);
jv pathrest = jv_array_slice(path, 1, jv_array_length(jv_copy(path)));
return jv_set(root, pathcurr,
jv_setpath(jv_get(jv_copy(root), jv_copy(pathcurr)), pathrest, value));
}
jv jv_getpath(jv root, jv path) {
if (jv_get_kind(path) != JV_KIND_ARRAY) {
jv_free(root);
jv_free(path);
return jv_invalid_with_msg(jv_string("Path must be specified as an array"));
}
if (!jv_is_valid(root)) {
jv_free(path);
return root;
}
if (jv_array_length(jv_copy(path)) == 0) {
jv_free(path);
return root;
}
jv pathcurr = jv_array_get(jv_copy(path), 0);
jv pathrest = jv_array_slice(path, 1, jv_array_length(jv_copy(path)));
return jv_getpath(jv_get(root, pathcurr), pathrest);
}
// assumes paths is a sorted array of arrays
static jv delpaths_sorted(jv object, jv paths, int start) {
jv delkeys = jv_array();
for (int i=0; i<jv_array_length(jv_copy(paths));) {
int j = i;
assert(jv_array_length(jv_array_get(jv_copy(paths), i)) > start);
int delkey = jv_array_length(jv_array_get(jv_copy(paths), i)) == start + 1;
jv key = jv_array_get(jv_array_get(jv_copy(paths), i), start);
while (j < jv_array_length(jv_copy(paths)) &&
jv_equal(jv_copy(key), jv_array_get(jv_array_get(jv_copy(paths), j), start)))
j++;
// if i <= entry < j, then entry starts with key
if (delkey) {
// deleting this entire key, we don't care about any more specific deletions
delkeys = jv_array_append(delkeys, key);
} else {
// deleting certain sub-parts of this key
jv subobject = jv_get(jv_copy(object), jv_copy(key));
if (!jv_is_valid(subobject)) {
jv_free(key);
jv_free(object);
object = subobject;
break;
} else if (jv_get_kind(subobject) == JV_KIND_NULL) {
jv_free(key);
jv_free(subobject);
} else {
jv newsubobject = delpaths_sorted(subobject, jv_array_slice(jv_copy(paths), i, j), start+1);
if (!jv_is_valid(newsubobject)) {
jv_free(key);
jv_free(object);
object = newsubobject;
break;
}
object = jv_set(object, key, newsubobject);
}
if (!jv_is_valid(object)) break;
}
i = j;
}
jv_free(paths);
if (jv_is_valid(object))
object = jv_dels(object, delkeys);
else
jv_free(delkeys);
return object;
}
jv jv_delpaths(jv object, jv paths) {
paths = jv_sort(paths, jv_copy(paths));
jv_array_foreach(paths, i, elem) {
if (jv_get_kind(elem) != JV_KIND_ARRAY) {
jv_free(object);
jv_free(paths);
jv err = jv_invalid_with_msg(jv_string_fmt("Path must be specified as array, not %s",
jv_kind_name(jv_get_kind(elem))));
jv_free(elem);
return err;
}
jv_free(elem);
}
if (jv_array_length(jv_copy(paths)) == 0) {
// nothing is being deleted
jv_free(paths);
return object;
}
if (jv_array_length(jv_array_get(jv_copy(paths), 0)) == 0) {
// everything is being deleted
jv_free(paths);
jv_free(object);
return jv_null();
}
return delpaths_sorted(object, paths, 0);
}
static int string_cmp(const void* pa, const void* pb){
const jv* a = pa;
const jv* b = pb;
int lena = jv_string_length_bytes(jv_copy(*a));
int lenb = jv_string_length_bytes(jv_copy(*b));
int minlen = lena < lenb ? lena : lenb;
int r = memcmp(jv_string_value(*a), jv_string_value(*b), minlen);
if (r == 0) r = lena - lenb;
return r;
}
jv jv_keys_unsorted(jv x) {
if (jv_get_kind(x) != JV_KIND_OBJECT)
return jv_keys(x);
jv answer = jv_array_sized(jv_object_length(jv_copy(x)));
jv_object_foreach(x, key, value) {
answer = jv_array_append(answer, key);
jv_free(value);
}
jv_free(x);
return answer;
}
jv jv_keys(jv x) {
if (jv_get_kind(x) == JV_KIND_OBJECT) {
int nkeys = jv_object_length(jv_copy(x));
jv* keys = jv_mem_alloc(sizeof(jv) * nkeys);
int kidx = 0;
jv_object_foreach(x, key, value) {
keys[kidx++] = key;
jv_free(value);
}
qsort(keys, nkeys, sizeof(jv), string_cmp);
jv answer = jv_array_sized(nkeys);
for (int i = 0; i<nkeys; i++) {
answer = jv_array_append(answer, keys[i]);
}
jv_mem_free(keys);
jv_free(x);
return answer;
} else if (jv_get_kind(x) == JV_KIND_ARRAY) {
int n = jv_array_length(x);
jv answer = jv_array();
for (int i=0; i<n; i++){
answer = jv_array_set(answer, i, jv_number(i));
}
return answer;
} else {
assert(0 && "jv_keys passed something neither object nor array");
return jv_invalid();
}
}
int jv_cmp(jv a, jv b) {
if (jv_get_kind(a) != jv_get_kind(b)) {
int r = (int)jv_get_kind(a) - (int)jv_get_kind(b);
jv_free(a);
jv_free(b);
return r;
}
int r = 0;
switch (jv_get_kind(a)) {
default:
assert(0 && "invalid kind passed to jv_cmp");
case JV_KIND_NULL:
case JV_KIND_FALSE:
case JV_KIND_TRUE:
// there's only one of each of these values
r = 0;
break;
case JV_KIND_NUMBER: {
double da = jv_number_value(a), db = jv_number_value(b);
// handle NaN as though it were null
if (da != da) r = jv_cmp(jv_null(), jv_number(db));
else if (db != db) r = jv_cmp(jv_number(da), jv_null());
else if (da < db) r = -1;
else if (da == db) r = 0;
else r = 1;
break;
}
case JV_KIND_STRING: {
r = string_cmp(&a, &b);
break;
}
case JV_KIND_ARRAY: {
// Lexical ordering of arrays
int i = 0;
while (r == 0) {
int a_done = i >= jv_array_length(jv_copy(a));
int b_done = i >= jv_array_length(jv_copy(b));
if (a_done || b_done) {
r = b_done - a_done; //suddenly, logic
break;
}
jv xa = jv_array_get(jv_copy(a), i);
jv xb = jv_array_get(jv_copy(b), i);
r = jv_cmp(xa, xb);
i++;
}
break;
}
case JV_KIND_OBJECT: {
jv keys_a = jv_keys(jv_copy(a));
jv keys_b = jv_keys(jv_copy(b));
r = jv_cmp(jv_copy(keys_a), keys_b);
if (r == 0) {
jv_array_foreach(keys_a, i, key) {
jv xa = jv_object_get(jv_copy(a), jv_copy(key));
jv xb = jv_object_get(jv_copy(b), key);
r = jv_cmp(xa, xb);
if (r) break;
}
}
jv_free(keys_a);
break;
}
}
jv_free(a);
jv_free(b);
return r;
}
struct sort_entry {
jv object;
jv key;
};
static int sort_cmp(const void* pa, const void* pb) {
const struct sort_entry* a = pa;
const struct sort_entry* b = pb;
int r = jv_cmp(jv_copy(a->key), jv_copy(b->key));
// comparing by address if r == 0 makes the sort stable
return r ? r : (int)(a - b);
}
static struct sort_entry* sort_items(jv objects, jv keys) {
assert(jv_get_kind(objects) == JV_KIND_ARRAY);
assert(jv_get_kind(keys) == JV_KIND_ARRAY);
assert(jv_array_length(jv_copy(objects)) == jv_array_length(jv_copy(keys)));
int n = jv_array_length(jv_copy(objects));
struct sort_entry* entries = jv_mem_alloc(sizeof(struct sort_entry) * n);
for (int i=0; i<n; i++) {
entries[i].object = jv_array_get(jv_copy(objects), i);
entries[i].key = jv_array_get(jv_copy(keys), i);
}
jv_free(objects);
jv_free(keys);
qsort(entries, n, sizeof(struct sort_entry), sort_cmp);
return entries;
}
jv jv_sort(jv objects, jv keys) {
assert(jv_get_kind(objects) == JV_KIND_ARRAY);
assert(jv_get_kind(keys) == JV_KIND_ARRAY);
assert(jv_array_length(jv_copy(objects)) == jv_array_length(jv_copy(keys)));
int n = jv_array_length(jv_copy(objects));
struct sort_entry* entries = sort_items(objects, keys);
jv ret = jv_array();
for (int i=0; i<n; i++) {
jv_free(entries[i].key);
ret = jv_array_set(ret, i, entries[i].object);
}
jv_mem_free(entries);
return ret;
}
jv jv_group(jv objects, jv keys) {
assert(jv_get_kind(objects) == JV_KIND_ARRAY);
assert(jv_get_kind(keys) == JV_KIND_ARRAY);
assert(jv_array_length(jv_copy(objects)) == jv_array_length(jv_copy(keys)));
int n = jv_array_length(jv_copy(objects));
struct sort_entry* entries = sort_items(objects, keys);
jv ret = jv_array();
if (n > 0) {
jv curr_key = entries[0].key;
jv group = jv_array_append(jv_array(), entries[0].object);
for (int i = 1; i < n; i++) {
if (jv_equal(jv_copy(curr_key), jv_copy(entries[i].key))) {
jv_free(entries[i].key);
} else {
jv_free(curr_key);
curr_key = entries[i].key;
ret = jv_array_append(ret, group);
group = jv_array();
}
group = jv_array_append(group, entries[i].object);
}
jv_free(curr_key);
ret = jv_array_append(ret, group);
}
jv_mem_free(entries);
return ret;
}
+4275
View File
File diff suppressed because it is too large. Load diff
+22
View File
@@ -0,0 +1,22 @@
#ifndef JV_DTOA_H
#define JV_DTOA_H
#define Kmax 7
struct Bigint;
struct dtoa_context {
struct Bigint *freelist[Kmax+1];
struct Bigint *p5s;
};
void jvp_dtoa_context_init(struct dtoa_context* ctx);
void jvp_dtoa_context_free(struct dtoa_context* ctx);
double jvp_strtod(struct dtoa_context* C, const char* s, char** se);
char* jvp_dtoa(struct dtoa_context* C, double dd, int mode, int ndigits, int *decpt, int *sign, char **rve);
void jvp_freedtoa(struct dtoa_context* C, char *s);
#define JVP_DTOA_FMT_MAX_LEN 64
char* jvp_dtoa_fmt(struct dtoa_context* C, register char *b, double x);
#endif
+49
View File
@@ -0,0 +1,49 @@
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "jv.h"
jv jv_load_file(const char* filename, int raw) {
FILE* file = fopen(filename, "r");
struct jv_parser* parser;
jv data;
if (!file) {
return jv_invalid_with_msg(jv_string_fmt("Could not open %s: %s",
filename,
strerror(errno)));
}
if (raw) {
data = jv_string("");
} else {
data = jv_array();
parser = jv_parser_new(0);
}
while (!feof(file) && !ferror(file)) {
char buf[4096];
size_t n = fread(buf, 1, sizeof(buf), file);
if (raw) {
data = jv_string_concat(data, jv_string_sized(buf, (int)n));
} else {
jv_parser_set_buf(parser, buf, n, !feof(file));
jv value;
while (jv_is_valid((value = jv_parser_next(parser))))
data = jv_array_append(data, value);
if (jv_invalid_has_msg(jv_copy(value))) {
jv_free(data);
data = value;
break;
}
}
}
if (!raw)
jv_parser_free(parser);
int badread = ferror(file);
if (fclose(file) != 0 || badread) {
jv_free(data);
return jv_invalid_with_msg(jv_string_fmt("Error reading from %s",
filename));
}
return data;
}
+852
View File
@@ -0,0 +1,852 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include "jv.h"
#include "jv_dtoa.h"
#include "jv_unicode.h"
#include "jv_alloc.h"
#include "jv_dtoa.h"
typedef const char* presult;
#define TRY(x) do {presult msg__ = (x); if (msg__) return msg__; } while(0)
#ifdef __GNUC__
#define pfunc __attribute__((warn_unused_result)) presult
#else
#define pfunc presult
#endif
enum last_seen {
JV_LAST_NONE = 0,
JV_LAST_OPEN_ARRAY = '[',
JV_LAST_OPEN_OBJECT = '{',
JV_LAST_COLON = ':',
JV_LAST_COMMA = ',',
JV_LAST_VALUE = 'V',
};
struct jv_parser {
const char* curr_buf;
int curr_buf_length;
int curr_buf_pos;
int curr_buf_is_partial;
int eof;
unsigned bom_strip_position;
int flags;
jv* stack; // parser
int stackpos; // parser
int stacklen; // both (optimization; it's really pathlen for streaming)
jv path; // streamer
enum last_seen last_seen; // streamer
jv output; // streamer
jv next; // both
char* tokenbuf;
int tokenpos;
int tokenlen;
int line, column;
struct dtoa_context dtoa;
enum {
JV_PARSER_NORMAL,
JV_PARSER_STRING,
JV_PARSER_STRING_ESCAPE,
JV_PARSER_WAITING_FOR_RS // parse error, waiting for RS
} st;
unsigned int last_ch_was_ws:1;
};
static void parser_init(struct jv_parser* p, int flags) {
p->flags = flags;
if ((p->flags & JV_PARSE_STREAMING)) {
p->path = jv_array();
} else {
p->path = jv_invalid();
p->flags &= ~(JV_PARSE_STREAM_ERRORS);
}
p->stack = 0;
p->stacklen = p->stackpos = 0;
p->last_seen = JV_LAST_NONE;
p->output = jv_invalid();
p->next = jv_invalid();
p->tokenbuf = 0;
p->tokenlen = p->tokenpos = 0;
if ((p->flags & JV_PARSE_SEQ))
p->st = JV_PARSER_WAITING_FOR_RS;
else
p->st = JV_PARSER_NORMAL;
p->eof = 0;
p->curr_buf = 0;
p->curr_buf_length = p->curr_buf_pos = p->curr_buf_is_partial = 0;
p->bom_strip_position = 0;
p->last_ch_was_ws = 0;
p->line = 1;
p->column = 0;
jvp_dtoa_context_init(&p->dtoa);
}
static void parser_reset(struct jv_parser* p) {
if ((p->flags & JV_PARSE_STREAMING)) {
jv_free(p->path);
p->path = jv_array();
p->stacklen = 0;
}
p->last_seen = JV_LAST_NONE;
jv_free(p->output);
p->output = jv_invalid();
jv_free(p->next);
p->next = jv_invalid();
for (int i=0; i<p->stackpos; i++)
jv_free(p->stack[i]);
p->stackpos = 0;
p->tokenpos = 0;
p->st = JV_PARSER_NORMAL;
}
static void parser_free(struct jv_parser* p) {
parser_reset(p);
jv_free(p->path);
jv_free(p->output);
jv_mem_free(p->stack);
jv_mem_free(p->tokenbuf);
jvp_dtoa_context_free(&p->dtoa);
}
static pfunc value(struct jv_parser* p, jv val) {
if ((p->flags & JV_PARSE_STREAMING)) {
if (jv_is_valid(p->next) || p->last_seen == JV_LAST_VALUE)
return "Expected separator between values";
if (p->stacklen > 0)
p->last_seen = JV_LAST_VALUE;
else
p->last_seen = JV_LAST_NONE;
} else {
if (jv_is_valid(p->next)) return "Expected separator between values";
}
jv_free(p->next);
p->next = val;
return 0;
}
static void push(struct jv_parser* p, jv v) {
assert(p->stackpos <= p->stacklen);
if (p->stackpos == p->stacklen) {
p->stacklen = p->stacklen * 2 + 10;
p->stack = jv_mem_realloc(p->stack, p->stacklen * sizeof(jv));
}
assert(p->stackpos < p->stacklen);
p->stack[p->stackpos++] = v;
}
static pfunc parse_token(struct jv_parser* p, char ch) {
switch (ch) {
case '[':
if (jv_is_valid(p->next)) return "Expected separator between values";
push(p, jv_array());
break;
case '{':
if (jv_is_valid(p->next)) return "Expected separator between values";
push(p, jv_object());
break;
case ':':
if (!jv_is_valid(p->next))
return "Expected string key before ':'";
if (p->stackpos == 0 || jv_get_kind(p->stack[p->stackpos-1]) != JV_KIND_OBJECT)
return "':' not as part of an object";
if (jv_get_kind(p->next) != JV_KIND_STRING)
return "Object keys must be strings";
push(p, p->next);
p->next = jv_invalid();
break;
case ',':
if (!jv_is_valid(p->next))
return "Expected value before ','";
if (p->stackpos == 0)
return "',' not as part of an object or array";
if (jv_get_kind(p->stack[p->stackpos-1]) == JV_KIND_ARRAY) {
p->stack[p->stackpos-1] = jv_array_append(p->stack[p->stackpos-1], p->next);
p->next = jv_invalid();
} else if (jv_get_kind(p->stack[p->stackpos-1]) == JV_KIND_STRING) {
assert(p->stackpos > 1 && jv_get_kind(p->stack[p->stackpos-2]) == JV_KIND_OBJECT);
p->stack[p->stackpos-2] = jv_object_set(p->stack[p->stackpos-2],
p->stack[p->stackpos-1], p->next);
p->stackpos--;
p->next = jv_invalid();
} else {
// this case hits on input like {"a", "b"}
return "Objects must consist of key:value pairs";
}
break;
case ']':
if (p->stackpos == 0 || jv_get_kind(p->stack[p->stackpos-1]) != JV_KIND_ARRAY)
return "Unmatched ']'";
if (jv_is_valid(p->next)) {
p->stack[p->stackpos-1] = jv_array_append(p->stack[p->stackpos-1], p->next);
p->next = jv_invalid();
} else {
if (jv_array_length(jv_copy(p->stack[p->stackpos-1])) != 0) {
// this case hits on input like [1,2,3,]
return "Expected another array element";
}
}
jv_free(p->next);
p->next = p->stack[--p->stackpos];
break;
case '}':
if (p->stackpos == 0)
return "Unmatched '}'";
if (jv_is_valid(p->next)) {
if (jv_get_kind(p->stack[p->stackpos-1]) != JV_KIND_STRING)
return "Objects must consist of key:value pairs";
assert(p->stackpos > 1 && jv_get_kind(p->stack[p->stackpos-2]) == JV_KIND_OBJECT);
p->stack[p->stackpos-2] = jv_object_set(p->stack[p->stackpos-2],
p->stack[p->stackpos-1], p->next);
p->stackpos--;
p->next = jv_invalid();
} else {
if (jv_get_kind(p->stack[p->stackpos-1]) != JV_KIND_OBJECT)
return "Unmatched '}'";
if (jv_object_length(jv_copy(p->stack[p->stackpos-1])) != 0)
return "Expected another key-value pair";
}
jv_free(p->next);
p->next = p->stack[--p->stackpos];
break;
}
return 0;
}
static pfunc stream_token(struct jv_parser* p, char ch) {
jv_kind k;
jv last;
switch (ch) {
case '[':
if (jv_is_valid(p->next))
return "Expected a separator between values";
p->path = jv_array_append(p->path, jv_number(0)); // push
p->last_seen = JV_LAST_OPEN_ARRAY;
p->stacklen++;
break;
case '{':
if (p->last_seen == JV_LAST_VALUE)
return "Expected a separator between values";
// Push object key: null, since we don't know it yet
p->path = jv_array_append(p->path, jv_null()); // push
p->last_seen = JV_LAST_OPEN_OBJECT;
p->stacklen++;
break;
case ':':
if (p->stacklen == 0 || jv_get_kind(jv_array_get(jv_copy(p->path), p->stacklen - 1)) == JV_KIND_NUMBER)
return "':' not as part of an object";
if (!jv_is_valid(p->next) || p->last_seen == JV_LAST_NONE)
return "Expected string key before ':'";
if (jv_get_kind(p->next) != JV_KIND_STRING)
return "Object keys must be strings";
if (p->last_seen != JV_LAST_VALUE)
return "':' should follow a key";
p->last_seen = JV_LAST_COLON;
p->path = jv_array_set(p->path, p->stacklen - 1, p->next);
p->next = jv_invalid();
break;
case ',':
if (p->last_seen != JV_LAST_VALUE)
return "Expected value before ','";
if (p->stacklen == 0)
return "',' not as part of an object or array";
last = jv_array_get(jv_copy(p->path), p->stacklen - 1);
k = jv_get_kind(last);
if (k == JV_KIND_NUMBER) {
int idx = jv_number_value(last);
if (jv_is_valid(p->next)) {
p->output = JV_ARRAY(jv_copy(p->path), p->next);
p->next = jv_invalid();
}
p->path = jv_array_set(p->path, p->stacklen - 1, jv_number(idx + 1));
p->last_seen = JV_LAST_COMMA;
} else if (k == JV_KIND_STRING) {
if (jv_is_valid(p->next)) {
p->output = JV_ARRAY(jv_copy(p->path), p->next);
p->next = jv_invalid();
}
p->path = jv_array_set(p->path, p->stacklen - 1, jv_true()); // ready for another name:value pair
p->last_seen = JV_LAST_COMMA;
} else {
assert(k == JV_KIND_NULL);
// this case hits on input like {,}
// make sure to handle input like {"a", "b"} and {"a":, ...}
jv_free(last);
return "Objects must consist of key:value pairs";
}
jv_free(last);
break;
case ']':
if (p->stacklen == 0)
return "Unmatched ']' at the top-level";
if (p->last_seen == JV_LAST_COMMA)
return "Expected another array element";
if (p->last_seen == JV_LAST_OPEN_ARRAY)
assert(!jv_is_valid(p->next));
last = jv_array_get(jv_copy(p->path), p->stacklen - 1);
k = jv_get_kind(last);
jv_free(last);
if (k != JV_KIND_NUMBER)
return "Unmatched ']' in the middle of an object";
if (jv_is_valid(p->next)) {
p->output = JV_ARRAY(jv_copy(p->path), p->next, jv_true());
p->next = jv_invalid();
} else if (p->last_seen != JV_LAST_OPEN_ARRAY) {
p->output = JV_ARRAY(jv_copy(p->path));
}
p->path = jv_array_slice(p->path, 0, --(p->stacklen)); // pop
//assert(!jv_is_valid(p->next));
jv_free(p->next);
p->next = jv_invalid();
if (p->last_seen == JV_LAST_OPEN_ARRAY)
p->output = JV_ARRAY(jv_copy(p->path), jv_array()); // Empty arrays are leaves
if (p->stacklen == 0)
p->last_seen = JV_LAST_NONE;
else
p->last_seen = JV_LAST_VALUE;
break;
case '}':
if (p->stacklen == 0)
return "Unmatched '}' at the top-level";
if (p->last_seen == JV_LAST_COMMA)
return "Expected another key:value pair";
if (p->last_seen == JV_LAST_OPEN_OBJECT)
assert(!jv_is_valid(p->next));
last = jv_array_get(jv_copy(p->path), p->stacklen - 1);
k = jv_get_kind(last);
jv_free(last);
if (k == JV_KIND_NUMBER)
return "Unmatched '}' in the middle of an array";
if (jv_is_valid(p->next)) {
if (k != JV_KIND_STRING)
return "Objects must consist of key:value pairs";
p->output = JV_ARRAY(jv_copy(p->path), p->next, jv_true());
p->next = jv_invalid();
} else {
// Perhaps {"a":[]}
if (p->last_seen == JV_LAST_COLON)
// Looks like {"a":}
return "Missing value in key:value pair";
if (p->last_seen == JV_LAST_COMMA)
// Looks like {"a":0,}
return "Expected another key-value pair";
if (p->last_seen == JV_LAST_OPEN_ARRAY)
return "Unmatched '}' in the middle of an array";
if (p->last_seen != JV_LAST_VALUE && p->last_seen != JV_LAST_OPEN_OBJECT)
return "Unmatched '}'";
if (p->last_seen != JV_LAST_OPEN_OBJECT)
p->output = JV_ARRAY(jv_copy(p->path));
}
p->path = jv_array_slice(p->path, 0, --(p->stacklen)); // pop
jv_free(p->next);
p->next = jv_invalid();
if (p->last_seen == JV_LAST_OPEN_OBJECT)
p->output = JV_ARRAY(jv_copy(p->path), jv_object()); // Empty arrays are leaves
if (p->stacklen == 0)
p->last_seen = JV_LAST_NONE;
else
p->last_seen = JV_LAST_VALUE;
break;
}
return 0;
}
static void tokenadd(struct jv_parser* p, char c) {
assert(p->tokenpos <= p->tokenlen);
if (p->tokenpos == p->tokenlen) {
p->tokenlen = p->tokenlen*2 + 256;
p->tokenbuf = jv_mem_realloc(p->tokenbuf, p->tokenlen);
}
assert(p->tokenpos < p->tokenlen);
p->tokenbuf[p->tokenpos++] = c;
}
static int unhex4(char* hex) {
int r = 0;
for (int i=0; i<4; i++) {
char c = *hex++;
int n;
if ('0' <= c && c <= '9') n = c - '0';
else if ('a' <= c && c <= 'f') n = c - 'a' + 10;
else if ('A' <= c && c <= 'F') n = c - 'A' + 10;
else return -1;
r <<= 4;
r |= n;
}
return r;
}
static pfunc found_string(struct jv_parser* p) {
char* in = p->tokenbuf;
char* out = p->tokenbuf;
char* end = p->tokenbuf + p->tokenpos;
while (in < end) {
char c = *in++;
if (c == '\\') {
if (in >= end)
return "Expected escape character at end of string";
c = *in++;
switch (c) {
case '\\':
case '"':
case '/': *out++ = c; break;
case 'b': *out++ = '\b'; break;
case 'f': *out++ = '\f'; break;
case 't': *out++ = '\t'; break;
case 'n': *out++ = '\n'; break;
case 'r': *out++ = '\r'; break;
case 'u':
/* ahh, the complicated case */
if (in + 4 > end)
return "Invalid \\uXXXX escape";
int hexvalue = unhex4(in);
if (hexvalue < 0)
return "Invalid characters in \\uXXXX escape";
unsigned long codepoint = (unsigned long)hexvalue;
in += 4;
if (0xD800 <= codepoint && codepoint <= 0xDBFF) {
/* who thought UTF-16 surrogate pairs were a good idea? */
if (in + 6 > end || in[0] != '\\' || in[1] != 'u')
return "Invalid \\uXXXX\\uXXXX surrogate pair escape";
unsigned long surrogate = unhex4(in+2);
if (!(0xDC00 <= surrogate && surrogate <= 0xDFFF))
return "Invalid \\uXXXX\\uXXXX surrogate pair escape";
in += 6;
codepoint = 0x10000 + (((codepoint - 0xD800) << 10)
|(surrogate - 0xDC00));
}
if (codepoint > 0x10FFFF)
codepoint = 0xFFFD; // U+FFFD REPLACEMENT CHARACTER
out += jvp_utf8_encode(codepoint, out);
break;
default:
return "Invalid escape";
}
} else {
if (c > 0 && c < 0x001f)
return "Invalid string: control characters from U+0000 through U+001F must be escaped";
*out++ = c;
}
}
TRY(value(p, jv_string_sized(p->tokenbuf, out - p->tokenbuf)));
p->tokenpos = 0;
return 0;
}
static pfunc check_literal(struct jv_parser* p) {
if (p->tokenpos == 0) return 0;
const char* pattern = 0;
int plen;
jv v;
switch (p->tokenbuf[0]) {
case 't': pattern = "true"; plen = 4; v = jv_true(); break;
case 'f': pattern = "false"; plen = 5; v = jv_false(); break;
case 'n': pattern = "null"; plen = 4; v = jv_null(); break;
}
if (pattern) {
if (p->tokenpos != plen) return "Invalid literal";
for (int i=0; i<plen; i++)
if (p->tokenbuf[i] != pattern[i])
return "Invalid literal";
TRY(value(p, v));
} else {
// FIXME: better parser
p->tokenbuf[p->tokenpos] = 0; // FIXME: invalid
char* end = 0;
double d = jvp_strtod(&p->dtoa, p->tokenbuf, &end);
if (end == 0 || *end != 0)
return "Invalid numeric literal";
TRY(value(p, jv_number(d)));
}
p->tokenpos = 0;
return 0;
}
typedef enum {
LITERAL,
WHITESPACE,
STRUCTURE,
QUOTE,
INVALID
} chclass;
static chclass classify(char c) {
switch (c) {
case ' ':
case '\t':
case '\r':
case '\n':
return WHITESPACE;
case '"':
return QUOTE;
case '[':
case ',':
case ']':
case '{':
case ':':
case '}':
return STRUCTURE;
default:
return LITERAL;
}
}
static const presult OK = "output produced";
static int parse_check_done(struct jv_parser* p, jv* out) {
if (p->stackpos == 0 && jv_is_valid(p->next)) {
*out = p->next;
p->next = jv_invalid();
return 1;
} else {
return 0;
}
}
static int stream_check_done(struct jv_parser* p, jv* out) {
if (p->stacklen == 0 && jv_is_valid(p->next)) {
*out = JV_ARRAY(jv_copy(p->path),p->next);
p->next = jv_invalid();
return 1;
} else if (jv_is_valid(p->output)) {
if (jv_array_length(jv_copy(p->output)) > 2) {
// At end of an array or object, necessitating one more output by
// which to indicate this
*out = jv_array_slice(jv_copy(p->output), 0, 2);
p->output = jv_array_slice(p->output, 0, 1); // arrange one more output
} else {
// No further processing needed
*out = p->output;
p->output = jv_invalid();
}
return 1;
} else {
return 0;
}
}
static int parse_check_truncation(struct jv_parser* p) {
return ((p->flags & JV_PARSE_SEQ) && !p->last_ch_was_ws && (p->stackpos > 0 || p->tokenpos > 0 || jv_get_kind(p->next) == JV_KIND_NUMBER));
}
static int stream_check_truncation(struct jv_parser* p) {
jv_kind k = jv_get_kind(p->next);
return (p->stacklen > 0 || k == JV_KIND_NUMBER || k == JV_KIND_TRUE || k == JV_KIND_FALSE || k == JV_KIND_NULL);
}
static int parse_is_top_num(struct jv_parser* p) {
return (p->stackpos == 0 && jv_get_kind(p->next) == JV_KIND_NUMBER);
}
static int stream_is_top_num(struct jv_parser* p) {
return (p->stacklen == 0 && jv_get_kind(p->next) == JV_KIND_NUMBER);
}
#define check_done(p, o) \
(((p)->flags & JV_PARSE_STREAMING) ? stream_check_done((p), (o)) : parse_check_done((p), (o)))
#define token(p, ch) \
(((p)->flags & JV_PARSE_STREAMING) ? stream_token((p), (ch)) : parse_token((p), (ch)))
#define check_truncation(p) \
(((p)->flags & JV_PARSE_STREAMING) ? stream_check_truncation((p)) : parse_check_truncation((p)))
#define is_top_num(p) \
(((p)->flags & JV_PARSE_STREAMING) ? stream_is_top_num((p)) : parse_is_top_num((p)))
static pfunc scan(struct jv_parser* p, char ch, jv* out) {
p->column++;
if (ch == '\n') {
p->line++;
p->column = 0;
}
if (ch == '\036' /* ASCII RS; see draft-ietf-json-sequence-07 */) {
if (check_truncation(p)) {
if (check_literal(p) == 0 && is_top_num(p))
return "Potentially truncated top-level numeric value";
return "Truncated value";
}
TRY(check_literal(p));
if (p->st == JV_PARSER_NORMAL && check_done(p, out))
return OK;
// shouldn't happen?
assert(!jv_is_valid(*out));
parser_reset(p);
jv_free(*out);
*out = jv_invalid();
return OK;
}
presult answer = 0;
p->last_ch_was_ws = 0;
if (p->st == JV_PARSER_NORMAL) {
chclass cls = classify(ch);
if (cls == WHITESPACE)
p->last_ch_was_ws = 1;
if (cls != LITERAL) {
TRY(check_literal(p));
if (check_done(p, out)) answer = OK;
}
switch (cls) {
case LITERAL:
tokenadd(p, ch);
break;
case WHITESPACE:
break;
case QUOTE:
p->st = JV_PARSER_STRING;
break;
case STRUCTURE:
TRY(token(p, ch));
break;
case INVALID:
return "Invalid character";
}
if (check_done(p, out)) answer = OK;
} else {
if (ch == '"' && p->st == JV_PARSER_STRING) {
TRY(found_string(p));
p->st = JV_PARSER_NORMAL;
if (check_done(p, out)) answer = OK;
} else {
tokenadd(p, ch);
if (ch == '\\' && p->st == JV_PARSER_STRING) {
p->st = JV_PARSER_STRING_ESCAPE;
} else {
p->st = JV_PARSER_STRING;
}
}
}
return answer;
}
struct jv_parser* jv_parser_new(int flags) {
struct jv_parser* p = jv_mem_alloc(sizeof(struct jv_parser));
parser_init(p, flags);
p->flags = flags;
return p;
}
void jv_parser_free(struct jv_parser* p) {
parser_free(p);
jv_mem_free(p);
}
static const unsigned char UTF8_BOM[] = {0xEF,0xBB,0xBF};
int jv_parser_remaining(struct jv_parser* p) {
if (p->curr_buf == 0)
return 0;
return (p->curr_buf_length - p->curr_buf_pos);
}
void jv_parser_set_buf(struct jv_parser* p, const char* buf, int length, int is_partial) {
assert((p->curr_buf == 0 || p->curr_buf_pos == p->curr_buf_length)
&& "previous buffer not exhausted");
while (length > 0 && p->bom_strip_position < sizeof(UTF8_BOM)) {
if ((unsigned char)*buf == UTF8_BOM[p->bom_strip_position]) {
// matched a BOM character
buf++;
length--;
p->bom_strip_position++;
} else {
if (p->bom_strip_position == 0) {
// no BOM in this document
p->bom_strip_position = sizeof(UTF8_BOM);
} else {
// malformed BOM (prefix present, rest missing)
p->bom_strip_position = 0xff;
}
}
}
p->curr_buf = buf;
p->curr_buf_length = length;
p->curr_buf_pos = 0;
p->curr_buf_is_partial = is_partial;
}
static jv make_error(struct jv_parser*, const char *, ...) JV_PRINTF_LIKE(2, 3);
static jv make_error(struct jv_parser* p, const char *fmt, ...) {
va_list ap;
va_start(ap, fmt);
jv e = jv_string_vfmt(fmt, ap);
va_end(ap);
if ((p->flags & JV_PARSE_STREAM_ERRORS))
return JV_ARRAY(e, jv_copy(p->path));
return jv_invalid_with_msg(e);
}
jv jv_parser_next(struct jv_parser* p) {
if (p->eof)
return jv_invalid();
if (!p->curr_buf)
return jv_invalid(); // Need a buffer
if (p->bom_strip_position == 0xff) {
if (!(p->flags & JV_PARSE_SEQ))
return jv_invalid_with_msg(jv_string("Malformed BOM"));
p->st =JV_PARSER_WAITING_FOR_RS;
parser_reset(p);
}
jv value = jv_invalid();
if ((p->flags & JV_PARSE_STREAMING) && stream_check_done(p, &value))
return value;
char ch;
presult msg = 0;
while (!msg && p->curr_buf_pos < p->curr_buf_length) {
ch = p->curr_buf[p->curr_buf_pos++];
if (p->st == JV_PARSER_WAITING_FOR_RS) {
if (ch == '\n') {
p->line++;
p->column = 0;
} else {
p->column++;
}
if (ch == '\036')
p->st = JV_PARSER_NORMAL;
continue; // need to resync, wait for RS
}
msg = scan(p, ch, &value);
}
if (msg == OK) {
return value;
} else if (msg) {
jv_free(value);
if (ch != '\036' && (p->flags & JV_PARSE_SEQ)) {
// Skip to the next RS
p->st = JV_PARSER_WAITING_FOR_RS;
value = make_error(p, "%s at line %d, column %d (need RS to resync)", msg, p->line, p->column);
parser_reset(p);
return value;
}
value = make_error(p, "%s at line %d, column %d", msg, p->line, p->column);
parser_reset(p);
if (!(p->flags & JV_PARSE_SEQ)) {
// We're not parsing a JSON text sequence; throw this buffer away.
// XXX We should fail permanently here.
p->curr_buf = 0;
p->curr_buf_pos = 0;
} // Else ch must be RS; don't clear buf so we can start parsing again after this ch
return value;
} else if (p->curr_buf_is_partial) {
assert(p->curr_buf_pos == p->curr_buf_length);
// need another buffer
return jv_invalid();
} else {
// at EOF
p->eof = 1;
assert(p->curr_buf_pos == p->curr_buf_length);
jv_free(value);
if (p->st == JV_PARSER_WAITING_FOR_RS)
return make_error(p, "Unfinished abandoned text at EOF at line %d, column %d", p->line, p->column);
if (p->st != JV_PARSER_NORMAL) {
value = make_error(p, "Unfinished string at EOF at line %d, column %d", p->line, p->column);
parser_reset(p);
p->st = JV_PARSER_WAITING_FOR_RS;
return value;
}
if ((msg = check_literal(p))) {
value = make_error(p, "%s at EOF at line %d, column %d", msg, p->line, p->column);
parser_reset(p);
p->st = JV_PARSER_WAITING_FOR_RS;
return value;
}
if (((p->flags & JV_PARSE_STREAMING) && p->stacklen != 0) ||
(!(p->flags & JV_PARSE_STREAMING) && p->stackpos != 0)) {
value = make_error(p, "Unfinished JSON term at EOF at line %d, column %d", p->line, p->column);
parser_reset(p);
p->st = JV_PARSER_WAITING_FOR_RS;
return value;
}
// p->next is either invalid (nothing here, but no syntax error)
// or valid (this is the value). either way it's the thing to return
if ((p->flags & JV_PARSE_STREAMING) && jv_is_valid(p->next)) {
value = JV_ARRAY(jv_copy(p->path), p->next); // except in streaming mode we've got to make it [path,value]
} else {
value = p->next;
}
p->next = jv_invalid();
if ((p->flags & JV_PARSE_SEQ) && !p->last_ch_was_ws && jv_get_kind(value) == JV_KIND_NUMBER) {
jv_free(value);
return make_error(p, "Potentially truncated top-level numeric value at EOF at line %d, column %d", p->line, p->column);
}
return value;
}
}
jv jv_parse_sized(const char* string, int length) {
struct jv_parser parser;
parser_init(&parser, 0);
jv_parser_set_buf(&parser, string, length, 0);
jv value = jv_parser_next(&parser);
if (jv_is_valid(value)) {
jv next = jv_parser_next(&parser);
if (jv_is_valid(next)) {
// multiple JSON values, we only wanted one
jv_free(value);
jv_free(next);
value = jv_invalid_with_msg(jv_string("Unexpected extra JSON values"));
} else if (jv_invalid_has_msg(jv_copy(next))) {
// parser error after the first JSON value
jv_free(value);
value = next;
} else {
// a single valid JSON value
jv_free(next);
}
} else if (jv_invalid_has_msg(jv_copy(value))) {
// parse error, we'll return it
} else {
// no value at all
jv_free(value);
value = jv_invalid_with_msg(jv_string("Expected JSON value"));
}
parser_free(&parser);
if (!jv_is_valid(value) && jv_invalid_has_msg(jv_copy(value))) {
jv msg = jv_invalid_get_msg(value);
value = jv_invalid_with_msg(jv_string_fmt("%s (while parsing '%s')",
jv_string_value(msg),
string));
jv_free(msg);
}
return value;
}
jv jv_parse(const char* string) {
return jv_parse_sized(string, strlen(string));
}
+348
View File
@@ -0,0 +1,348 @@
#include <assert.h>
#include <stdio.h>
#include <float.h>
#include <string.h>
#ifdef WIN32
#include <windows.h>
#include <io.h>
#include <fileapi.h>
#endif
#include "jv.h"
#include "jv_dtoa.h"
#include "jv_unicode.h"
#define ESC "\033"
#define COL(c) (ESC "[" c "m")
#define COLRESET (ESC "[0m")
// Colour table. See http://en.wikipedia.org/wiki/ANSI_escape_code#Colors
// for how to choose these.
static const jv_kind colour_kinds[] =
{JV_KIND_NULL, JV_KIND_FALSE, JV_KIND_TRUE, JV_KIND_NUMBER,
JV_KIND_STRING, JV_KIND_ARRAY, JV_KIND_OBJECT};
static const char* const colours[] =
{COL("1;30"), COL("0;39"), COL("0;39"), COL("0;39"),
COL("0;32"), COL("1;39"), COL("1;39")};
#define FIELD_COLOUR COL("34;1")
static void put_buf(const char *s, int len, FILE *fout, jv *strout, int is_tty) {
if (strout) {
*strout = jv_string_append_buf(*strout, s, len);
} else {
#ifdef WIN32
/* See util.h */
if (is_tty)
WriteFile((HANDLE)_get_osfhandle(fileno(fout)), s, len, NULL, NULL);
else
fwrite(s, 1, len, fout);
#else
fwrite(s, 1, len, fout);
#endif
}
}
static void put_char(char c, FILE* fout, jv* strout, int T) {
put_buf(&c, 1, fout, strout, T);
}
static void put_str(const char* s, FILE* fout, jv* strout, int T) {
put_buf(s, strlen(s), fout, strout, T);
}
static void put_indent(int n, int flags, FILE* fout, jv* strout, int T) {
if (flags & JV_PRINT_TAB) {
while (n--)
put_char('\t', fout, strout, T);
} else {
n *= ((flags & (JV_PRINT_SPACE0 | JV_PRINT_SPACE1 | JV_PRINT_SPACE2)) >> 8);
while (n--)
put_char(' ', fout, strout, T);
}
}
static void jvp_dump_string(jv str, int ascii_only, FILE* F, jv* S, int T) {
assert(jv_get_kind(str) == JV_KIND_STRING);
const char* i = jv_string_value(str);
const char* end = i + jv_string_length_bytes(jv_copy(str));
const char* cstart;
int c = 0;
char buf[32];
put_char('"', F, S, T);
while ((i = jvp_utf8_next((cstart = i), end, &c))) {
assert(c != -1);
int unicode_escape = 0;
if (0x20 <= c && c <= 0x7E) {
// printable ASCII
if (c == '"' || c == '\\') {
put_char('\\', F, S, T);
}
put_char(c, F, S, T);
} else if (c < 0x20 || c == 0x7F) {
// ASCII control character
switch (c) {
case '\b':
put_char('\\', F, S, T);
put_char('b', F, S, T);
break;
case '\t':
put_char('\\', F, S, T);
put_char('t', F, S, T);
break;
case '\r':
put_char('\\', F, S, T);
put_char('r', F, S, T);
break;
case '\n':
put_char('\\', F, S, T);
put_char('n', F, S, T);
break;
case '\f':
put_char('\\', F, S, T);
put_char('f', F, S, T);
break;
default:
unicode_escape = 1;
break;
}
} else {
if (ascii_only) {
unicode_escape = 1;
} else {
put_buf(cstart, i - cstart, F, S, T);
}
}
if (unicode_escape) {
if (c <= 0xffff) {
sprintf(buf, "\\u%04x", c);
} else {
c -= 0x10000;
sprintf(buf, "\\u%04x\\u%04x",
0xD800 | ((c & 0xffc00) >> 10),
0xDC00 | (c & 0x003ff));
}
put_str(buf, F, S, T);
}
}
assert(c != -1);
put_char('"', F, S, T);
}
static void put_refcnt(struct dtoa_context* C, int refcnt, FILE *F, jv* S, int T){
char buf[JVP_DTOA_FMT_MAX_LEN];
put_char(' ', F, S, T);
put_char('(', F, S, T);
put_str(jvp_dtoa_fmt(C, buf, refcnt), F, S, T);
put_char(')', F, S, T);
}
static void jv_dump_term(struct dtoa_context* C, jv x, int flags, int indent, FILE* F, jv* S) {
char buf[JVP_DTOA_FMT_MAX_LEN];
const char* colour = 0;
double refcnt = (flags & JV_PRINT_REFCOUNT) ? jv_get_refcnt(x) - 1 : -1;
if (flags & JV_PRINT_COLOUR) {
for (unsigned i=0; i<sizeof(colour_kinds)/sizeof(colour_kinds[0]); i++) {
if (jv_get_kind(x) == colour_kinds[i]) {
colour = colours[i];
put_str(colour, F, S, flags & JV_PRINT_ISATTY);
break;
}
}
}
switch (jv_get_kind(x)) {
default:
case JV_KIND_INVALID:
if (flags & JV_PRINT_INVALID) {
jv msg = jv_invalid_get_msg(jv_copy(x));
if (jv_get_kind(msg) == JV_KIND_STRING) {
put_str("<invalid:", F, S, flags & JV_PRINT_ISATTY);
jvp_dump_string(msg, flags | JV_PRINT_ASCII, F, S, flags & JV_PRINT_ISATTY);
put_str(">", F, S, flags & JV_PRINT_ISATTY);
} else {
put_str("<invalid>", F, S, flags & JV_PRINT_ISATTY);
}
} else {
assert(0 && "Invalid value");
}
break;
case JV_KIND_NULL:
put_str("null", F, S, flags & JV_PRINT_ISATTY);
break;
case JV_KIND_FALSE:
put_str("false", F, S, flags & JV_PRINT_ISATTY);
break;
case JV_KIND_TRUE:
put_str("true", F, S, flags & JV_PRINT_ISATTY);
break;
case JV_KIND_NUMBER: {
double d = jv_number_value(x);
if (d != d) {
// JSON doesn't have NaN, so we'll render it as "null"
put_str("null", F, S, flags & JV_PRINT_ISATTY);
} else {
// Normalise infinities to something we can print in valid JSON
if (d > DBL_MAX) d = DBL_MAX;
if (d < -DBL_MAX) d = -DBL_MAX;
put_str(jvp_dtoa_fmt(C, buf, d), F, S, flags & JV_PRINT_ISATTY);
}
break;
}
case JV_KIND_STRING:
jvp_dump_string(x, flags & JV_PRINT_ASCII, F, S, flags & JV_PRINT_ISATTY);
if (flags & JV_PRINT_REFCOUNT)
put_refcnt(C, refcnt, F, S, flags & JV_PRINT_ISATTY);
break;
case JV_KIND_ARRAY: {
if (jv_array_length(jv_copy(x)) == 0) {
put_str("[]", F, S, flags & JV_PRINT_ISATTY);
break;
}
put_str("[", F, S, flags & JV_PRINT_ISATTY);
if (flags & JV_PRINT_PRETTY) {
put_char('\n', F, S, flags & JV_PRINT_ISATTY);
put_indent(indent + 1, flags, F, S, flags & JV_PRINT_ISATTY);
}
jv_array_foreach(x, i, elem) {
if (i!=0) {
if (flags & JV_PRINT_PRETTY) {
put_str(",\n", F, S, flags & JV_PRINT_ISATTY);
put_indent(indent + 1, flags, F, S, flags & JV_PRINT_ISATTY);
} else {
put_str(",", F, S, flags & JV_PRINT_ISATTY);
}
}
jv_dump_term(C, elem, flags, indent + 1, F, S);
if (colour) put_str(colour, F, S, flags & JV_PRINT_ISATTY);
}
if (flags & JV_PRINT_PRETTY) {
put_char('\n', F, S, flags & JV_PRINT_ISATTY);
put_indent(indent, flags, F, S, flags & JV_PRINT_ISATTY);
}
if (colour) put_str(colour, F, S, flags & JV_PRINT_ISATTY);
put_char(']', F, S, flags & JV_PRINT_ISATTY);
if (flags & JV_PRINT_REFCOUNT)
put_refcnt(C, refcnt, F, S, flags & JV_PRINT_ISATTY);
break;
}
case JV_KIND_OBJECT: {
if (jv_object_length(jv_copy(x)) == 0) {
put_str("{}", F, S, flags & JV_PRINT_ISATTY);
break;
}
put_char('{', F, S, flags & JV_PRINT_ISATTY);
if (flags & JV_PRINT_PRETTY) {
put_char('\n', F, S, flags & JV_PRINT_ISATTY);
put_indent(indent + 1, flags, F, S, flags & JV_PRINT_ISATTY);
}
int first = 1;
int i = 0;
jv keyset = jv_null();
while (1) {
jv key, value;
if (flags & JV_PRINT_SORTED) {
if (first) {
keyset = jv_keys(jv_copy(x));
i = 0;
} else {
i++;
}
if (i >= jv_array_length(jv_copy(keyset))) {
jv_free(keyset);
break;
}
key = jv_array_get(jv_copy(keyset), i);
value = jv_object_get(jv_copy(x), jv_copy(key));
} else {
if (first) {
i = jv_object_iter(x);
} else {
i = jv_object_iter_next(x, i);
}
if (!jv_object_iter_valid(x, i)) break;
key = jv_object_iter_key(x, i);
value = jv_object_iter_value(x, i);
}
if (!first) {
if (flags & JV_PRINT_PRETTY){
put_str(",\n", F, S, flags & JV_PRINT_ISATTY);
put_indent(indent + 1, flags, F, S, flags & JV_PRINT_ISATTY);
} else {
put_str(",", F, S, flags & JV_PRINT_ISATTY);
}
}
if (colour) put_str(COLRESET, F, S, flags & JV_PRINT_ISATTY);
first = 0;
if (colour) put_str(FIELD_COLOUR, F, S, flags & JV_PRINT_ISATTY);
jvp_dump_string(key, flags & JV_PRINT_ASCII, F, S, flags & JV_PRINT_ISATTY);
jv_free(key);
if (colour) put_str(COLRESET, F, S, flags & JV_PRINT_ISATTY);
if (colour) put_str(colour, F, S, flags & JV_PRINT_ISATTY);
put_str((flags & JV_PRINT_PRETTY) ? ": " : ":", F, S, flags & JV_PRINT_ISATTY);
if (colour) put_str(COLRESET, F, S, flags & JV_PRINT_ISATTY);
jv_dump_term(C, value, flags, indent + 1, F, S);
if (colour) put_str(colour, F, S, flags & JV_PRINT_ISATTY);
}
if (flags & JV_PRINT_PRETTY) {
put_char('\n', F, S, flags & JV_PRINT_ISATTY);
put_indent(indent, flags, F, S, flags & JV_PRINT_ISATTY);
}
if (colour) put_str(colour, F, S, flags & JV_PRINT_ISATTY);
put_char('}', F, S, flags & JV_PRINT_ISATTY);
if (flags & JV_PRINT_REFCOUNT)
put_refcnt(C, refcnt, F, S, flags & JV_PRINT_ISATTY);
}
}
jv_free(x);
if (colour) {
put_str(COLRESET, F, S, flags & JV_PRINT_ISATTY);
}
}
void jv_dumpf(jv x, FILE *f, int flags) {
struct dtoa_context C;
jvp_dtoa_context_init(&C);
jv_dump_term(&C, x, flags, 0, f, 0);
jvp_dtoa_context_free(&C);
}
void jv_dump(jv x, int flags) {
jv_dumpf(x, stdout, flags);
}
/* This one is nice for use in debuggers */
void jv_show(jv x, int flags) {
if (flags == -1)
flags = JV_PRINT_PRETTY | JV_PRINT_COLOUR | JV_PRINT_INDENT_FLAGS(2);
jv_dumpf(jv_copy(x), stderr, flags | JV_PRINT_INVALID);
fflush(stderr);
}
jv jv_dump_string(jv x, int flags) {
struct dtoa_context C;
jvp_dtoa_context_init(&C);
jv s = jv_string("");
jv_dump_term(&C, x, flags, 0, 0, &s);
jvp_dtoa_context_free(&C);
return s;
}
char *jv_dump_string_trunc(jv x, char *outbuf, size_t bufsize) {
x = jv_dump_string(x,0);
const char* p = jv_string_value(x);
const size_t len = strlen(p);
strncpy(outbuf, p, bufsize);
outbuf[bufsize - 1] = 0;
if (len > bufsize - 1 && bufsize >= 4) {
// Indicate truncation with '...'
outbuf[bufsize - 2]='.';
outbuf[bufsize - 3]='.';
outbuf[bufsize - 4]='.';
}
jv_free(x);
return outbuf;
}
+96
View File
@@ -0,0 +1,96 @@
#include <stdio.h>
#include <assert.h>
#include "jv_unicode.h"
#include "jv_utf8_tables.h"
const char* jvp_utf8_next(const char* in, const char* end, int* codepoint_ret) {
assert(in <= end);
if (in == end) {
return 0;
}
int codepoint = -1;
unsigned char first = (unsigned char)in[0];
int length = utf8_coding_length[first];
if ((first & 0x80) == 0) {
/* Fast-path for ASCII */
codepoint = first;
length = 1;
} else if (length == 0 || length == UTF8_CONTINUATION_BYTE) {
/* Bad single byte - either an invalid byte or an out-of-place continuation byte */
length = 1;
} else if (in + length > end) {
/* String ends before UTF8 sequence ends */
length = end - in;
} else {
codepoint = ((unsigned)in[0]) & utf8_coding_bits[first];
for (int i=1; i<length; i++) {
unsigned ch = (unsigned char)in[i];
if (utf8_coding_length[ch] != UTF8_CONTINUATION_BYTE){
/* Invalid UTF8 sequence - not followed by the right number of continuation bytes */
codepoint = -1;
length = i;
break;
}
codepoint = (codepoint << 6) | (ch & 0x3f);
}
if (codepoint < utf8_first_codepoint[length]) {
/* Overlong UTF8 sequence */
codepoint = -1;
}
if (0xD800 <= codepoint && codepoint <= 0xDFFF) {
/* Surrogate codepoints can't be encoded in UTF8 */
codepoint = -1;
}
if (codepoint > 0x10FFFF) {
/* Outside Unicode range */
codepoint = -1;
}
}
assert(length > 0);
*codepoint_ret = codepoint;
return in + length;
}
int jvp_utf8_is_valid(const char* in, const char* end) {
int codepoint;
while ((in = jvp_utf8_next(in, end, &codepoint))) {
if (codepoint == -1) return 0;
}
return 1;
}
int jvp_utf8_decode_length(char startchar) {
if ((startchar & 0x80) == 0) return 1;
else if ((startchar & 0xC0) == 0xC0) return 2;
else if ((startchar & 0xE0) == 0xE0) return 3;
else return 4;
}
int jvp_utf8_encode_length(int codepoint) {
if (codepoint <= 0x7F) return 1;
else if (codepoint <= 0x7FF) return 2;
else if (codepoint <= 0xFFFF) return 3;
else return 4;
}
int jvp_utf8_encode(int codepoint, char* out) {
assert(codepoint >= 0 && codepoint <= 0x10FFFF);
char* start = out;
if (codepoint <= 0x7F) {
*out++ = codepoint;
} else if (codepoint <= 0x7FF) {
*out++ = 0xC0 + ((codepoint & 0x7C0) >> 6);
*out++ = 0x80 + ((codepoint & 0x03F));
} else if(codepoint <= 0xFFFF) {
*out++ = 0xE0 + ((codepoint & 0xF000) >> 12);
*out++ = 0x80 + ((codepoint & 0x0FC0) >> 6);
*out++ = 0x80 + ((codepoint & 0x003F));
} else {
*out++ = 0xF0 + ((codepoint & 0x1C0000) >> 18);
*out++ = 0x80 + ((codepoint & 0x03F000) >> 12);
*out++ = 0x80 + ((codepoint & 0x000FC0) >> 6);
*out++ = 0x80 + ((codepoint & 0x00003F));
}
assert(out - start == jvp_utf8_encode_length(codepoint));
return out - start;
}
+11
View File
@@ -0,0 +1,11 @@
#ifndef JV_UNICODE_H
#define JV_UNICODE_H
const char* jvp_utf8_next(const char* in, const char* end, int* codepoint);
int jvp_utf8_is_valid(const char* in, const char* end);
int jvp_utf8_decode_length(char startchar);
int jvp_utf8_encode_length(int codepoint);
int jvp_utf8_encode(int codepoint, char* out);
#endif
+37
View File
@@ -0,0 +1,37 @@
#define UTF8_CONTINUATION_BYTE ((unsigned char)255)
static const unsigned char utf8_coding_length[] =
{0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01,
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
0x00, 0x00, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02,
0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02, 0x02,
0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03, 0x03,
0x04, 0x04, 0x04, 0x04, 0x04, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
static const unsigned char utf8_coding_bits[] =
{0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f,
0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f,
0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f,
0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f,
0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f,
0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f,
0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f,
0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f, 0x7f,
0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f,
0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f,
0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f,
0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f, 0x3f,
0x00, 0x00, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f,
0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f, 0x1f,
0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f, 0x0f,
0x07, 0x07, 0x07, 0x07, 0x07, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
static const int utf8_first_codepoint[] =
{0x00, 0x00, 0x80, 0x800, 0x10000};
+2442
View File
File diff suppressed because it is too large. Load diff
+362
View File
@@ -0,0 +1,362 @@
#ifndef jq_yyHEADER_H
#define jq_yyHEADER_H 1
#define jq_yyIN_HEADER 1
#line 6 "lexer.h"
#line 8 "lexer.h"
#define YY_INT_ALIGNED short int
/* A lexical scanner generated by flex */
#define FLEX_SCANNER
#define YY_FLEX_MAJOR_VERSION 2
#define YY_FLEX_MINOR_VERSION 5
#define YY_FLEX_SUBMINOR_VERSION 35
#if YY_FLEX_SUBMINOR_VERSION > 0
#define FLEX_BETA
#endif
/* First, we deal with platform-specific or compiler-specific issues. */
/* begin standard C headers. */
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <stdlib.h>
/* end standard C headers. */
/* flex integer type definitions */
#ifndef FLEXINT_H
#define FLEXINT_H
/* C99 systems have <inttypes.h>. Non-C99 systems may or may not. */
#if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
/* C99 says to define __STDC_LIMIT_MACROS before including stdint.h,
* if you want the limit (max/min) macros for int types.
*/
#ifndef __STDC_LIMIT_MACROS
#define __STDC_LIMIT_MACROS 1
#endif
#include <inttypes.h>
typedef int8_t flex_int8_t;
typedef uint8_t flex_uint8_t;
typedef int16_t flex_int16_t;
typedef uint16_t flex_uint16_t;
typedef int32_t flex_int32_t;
typedef uint32_t flex_uint32_t;
#else
typedef signed char flex_int8_t;
typedef short int flex_int16_t;
typedef int flex_int32_t;
typedef unsigned char flex_uint8_t;
typedef unsigned short int flex_uint16_t;
typedef unsigned int flex_uint32_t;
/* Limits of integral types. */
#ifndef INT8_MIN
#define INT8_MIN (-128)
#endif
#ifndef INT16_MIN
#define INT16_MIN (-32767-1)
#endif
#ifndef INT32_MIN
#define INT32_MIN (-2147483647-1)
#endif
#ifndef INT8_MAX
#define INT8_MAX (127)
#endif
#ifndef INT16_MAX
#define INT16_MAX (32767)
#endif
#ifndef INT32_MAX
#define INT32_MAX (2147483647)
#endif
#ifndef UINT8_MAX
#define UINT8_MAX (255U)
#endif
#ifndef UINT16_MAX
#define UINT16_MAX (65535U)
#endif
#ifndef UINT32_MAX
#define UINT32_MAX (4294967295U)
#endif
#endif /* ! C99 */
#endif /* ! FLEXINT_H */
#ifdef __cplusplus
/* The "const" storage-class-modifier is valid. */
#define YY_USE_CONST
#else /* ! __cplusplus */
/* C99 requires __STDC__ to be defined as 1. */
#if defined (__STDC__)
#define YY_USE_CONST
#endif /* defined (__STDC__) */
#endif /* ! __cplusplus */
#ifdef YY_USE_CONST
#define yyconst const
#else
#define yyconst
#endif
/* An opaque pointer. */
#ifndef YY_TYPEDEF_YY_SCANNER_T
#define YY_TYPEDEF_YY_SCANNER_T
typedef void* yyscan_t;
#endif
/* For convenience, these vars (plus the bison vars far below)
are macros in the reentrant scanner. */
#define yyin yyg->yyin_r
#define yyout yyg->yyout_r
#define yyextra yyg->yyextra_r
#define yyleng yyg->yyleng_r
#define yytext yyg->yytext_r
#define yylineno (YY_CURRENT_BUFFER_LVALUE->yy_bs_lineno)
#define yycolumn (YY_CURRENT_BUFFER_LVALUE->yy_bs_column)
#define yy_flex_debug yyg->yy_flex_debug_r
/* Size of default input buffer. */
#ifndef YY_BUF_SIZE
#ifdef __ia64__
/* On IA-64, the buffer size is 16k, not 8k.
* Moreover, YY_BUF_SIZE is 2*YY_READ_BUF_SIZE in the general case.
* Ditto for the __ia64__ case accordingly.
*/
#define YY_BUF_SIZE 32768
#else
#define YY_BUF_SIZE 16384
#endif /* __ia64__ */
#endif
#ifndef YY_TYPEDEF_YY_BUFFER_STATE
#define YY_TYPEDEF_YY_BUFFER_STATE
typedef struct yy_buffer_state *YY_BUFFER_STATE;
#endif
#ifndef YY_TYPEDEF_YY_SIZE_T
#define YY_TYPEDEF_YY_SIZE_T
typedef size_t yy_size_t;
#endif
#ifndef YY_STRUCT_YY_BUFFER_STATE
#define YY_STRUCT_YY_BUFFER_STATE
struct yy_buffer_state
{
FILE *yy_input_file;
char *yy_ch_buf; /* input buffer */
char *yy_buf_pos; /* current position in input buffer */
/* Size of input buffer in bytes, not including room for EOB
* characters.
*/
yy_size_t yy_buf_size;
/* Number of characters read into yy_ch_buf, not including EOB
* characters.
*/
int yy_n_chars;
/* Whether we "own" the buffer - i.e., we know we created it,
* and can realloc() it to grow it, and should free() it to
* delete it.
*/
int yy_is_our_buffer;
/* Whether this is an "interactive" input source; if so, and
* if we're using stdio for input, then we want to use getc()
* instead of fread(), to make sure we stop fetching input after
* each newline.
*/
int yy_is_interactive;
/* Whether we're considered to be at the beginning of a line.
* If so, '^' rules will be active on the next match, otherwise
* not.
*/
int yy_at_bol;
int yy_bs_lineno; /**< The line count. */
int yy_bs_column; /**< The column count. */
/* Whether to try to fill the input buffer when we reach the
* end of it.
*/
int yy_fill_buffer;
int yy_buffer_status;
};
#endif /* !YY_STRUCT_YY_BUFFER_STATE */
void jq_yyrestart (FILE *input_file ,yyscan_t yyscanner );
void jq_yy_switch_to_buffer (YY_BUFFER_STATE new_buffer ,yyscan_t yyscanner );
YY_BUFFER_STATE jq_yy_create_buffer (FILE *file,int size ,yyscan_t yyscanner );
void jq_yy_delete_buffer (YY_BUFFER_STATE b ,yyscan_t yyscanner );
void jq_yy_flush_buffer (YY_BUFFER_STATE b ,yyscan_t yyscanner );
void jq_yypush_buffer_state (YY_BUFFER_STATE new_buffer ,yyscan_t yyscanner );
void jq_yypop_buffer_state (yyscan_t yyscanner );
YY_BUFFER_STATE jq_yy_scan_buffer (char *base,yy_size_t size ,yyscan_t yyscanner );
YY_BUFFER_STATE jq_yy_scan_string (yyconst char *yy_str ,yyscan_t yyscanner );
YY_BUFFER_STATE jq_yy_scan_bytes (yyconst char *bytes,int len ,yyscan_t yyscanner );
void *jq_yyalloc (yy_size_t ,yyscan_t yyscanner );
void *jq_yyrealloc (void *,yy_size_t ,yyscan_t yyscanner );
void jq_yyfree (void * ,yyscan_t yyscanner );
/* Begin user sect3 */
#define jq_yywrap(n) 1
#define YY_SKIP_YYWRAP
#define yytext_ptr yytext_r
#ifdef YY_HEADER_EXPORT_START_CONDITIONS
#define INITIAL 0
#define IN_PAREN 1
#define IN_BRACKET 2
#define IN_BRACE 3
#define IN_QQINTERP 4
#define IN_QQSTRING 5
#endif
#ifndef YY_NO_UNISTD_H
/* Special case for "unistd.h", since it is non-ANSI. We include it way
* down here because we want the user's section 1 to have been scanned first.
* The user has a chance to override it with an option.
*/
#include <unistd.h>
#endif
#define YY_EXTRA_TYPE int
int jq_yylex_init (yyscan_t* scanner);
int jq_yylex_init_extra (YY_EXTRA_TYPE user_defined,yyscan_t* scanner);
/* Accessor methods to globals.
These are made visible to non-reentrant scanners for convenience. */
int jq_yylex_destroy (yyscan_t yyscanner );
int jq_yyget_debug (yyscan_t yyscanner );
void jq_yyset_debug (int debug_flag ,yyscan_t yyscanner );
YY_EXTRA_TYPE jq_yyget_extra (yyscan_t yyscanner );
void jq_yyset_extra (YY_EXTRA_TYPE user_defined ,yyscan_t yyscanner );
FILE *jq_yyget_in (yyscan_t yyscanner );
void jq_yyset_in (FILE * in_str ,yyscan_t yyscanner );
FILE *jq_yyget_out (yyscan_t yyscanner );
void jq_yyset_out (FILE * out_str ,yyscan_t yyscanner );
int jq_yyget_leng (yyscan_t yyscanner );
char *jq_yyget_text (yyscan_t yyscanner );
int jq_yyget_lineno (yyscan_t yyscanner );
void jq_yyset_lineno (int line_number ,yyscan_t yyscanner );
YYSTYPE * jq_yyget_lval (yyscan_t yyscanner );
void jq_yyset_lval (YYSTYPE * yylval_param ,yyscan_t yyscanner );
YYLTYPE *jq_yyget_lloc (yyscan_t yyscanner );
void jq_yyset_lloc (YYLTYPE * yylloc_param ,yyscan_t yyscanner );
/* Macros after this point can all be overridden by user definitions in
* section 1.
*/
#ifndef YY_SKIP_YYWRAP
#ifdef __cplusplus
extern "C" int jq_yywrap (yyscan_t yyscanner );
#else
extern int jq_yywrap (yyscan_t yyscanner );
#endif
#endif
#ifndef yytext_ptr
static void yy_flex_strncpy (char *,yyconst char *,int ,yyscan_t yyscanner);
#endif
#ifdef YY_NEED_STRLEN
static int yy_flex_strlen (yyconst char * ,yyscan_t yyscanner);
#endif
#ifndef YY_NO_INPUT
#endif
/* Amount of stuff to slurp up with each read. */
#ifndef YY_READ_BUF_SIZE
#ifdef __ia64__
/* On IA-64, the buffer size is 16k, not 8k */
#define YY_READ_BUF_SIZE 16384
#else
#define YY_READ_BUF_SIZE 8192
#endif /* __ia64__ */
#endif
/* Number of entries by which start-condition stack grows. */
#ifndef YY_START_STACK_INCR
#define YY_START_STACK_INCR 25
#endif
/* Default declaration of generated scanner - a define so the user can
* easily add parameters.
*/
#ifndef YY_DECL
#define YY_DECL_IS_OURS 1
extern int jq_yylex \
(YYSTYPE * yylval_param,YYLTYPE * yylloc_param ,yyscan_t yyscanner);
#define YY_DECL int jq_yylex \
(YYSTYPE * yylval_param, YYLTYPE * yylloc_param , yyscan_t yyscanner)
#endif /* !YY_DECL */
/* yy_get_previous_state - get the state just before the EOB char was reached */
#undef YY_NEW_FILE
#undef YY_FLUSH_BUFFER
#undef yy_set_bol
#undef yy_new_buffer
#undef yy_set_interactive
#undef YY_DO_BEFORE_ACTION
#ifdef YY_DECL_IS_OURS
#undef YY_DECL_IS_OURS
#undef YY_DECL
#endif
#line 129 "lexer.l"
#line 361 "lexer.h"
#undef jq_yyIN_HEADER
#endif /* jq_yyHEADER_H */
+184
View File
@@ -0,0 +1,184 @@
%{
#include <assert.h>
#include "jv_alloc.h"
#include "compile.h"
struct lexer_param;
#include "parser.h" /* Generated by bison. */
#define YY_USER_ACTION \
do { \
yylloc->start = yyget_extra(yyscanner); \
yylloc->end = yylloc->start + yyleng; \
yyset_extra(yylloc->end, yyscanner); \
} while (0);
%}
%s IN_PAREN
%s IN_BRACKET
%s IN_BRACE
%s IN_QQINTERP
%x IN_QQSTRING
%{
static int enter(int opening, int state, yyscan_t yyscanner);
static int try_exit(int closing, int state, yyscan_t yyscanner);
%}
%option noyywrap nounput noinput nodefault
%option noyyalloc noyyrealloc noyyfree
%option reentrant
%option extra-type="int"
%option bison-bridge bison-locations
%option prefix="jq_yy"
%option stack
%%
"#"[^\r\n]* { /* comments */ }
"!=" { return NEQ; }
"==" { return EQ; }
"as" { return AS; }
"import" { return IMPORT; }
"include" { return INCLUDE; }
"module" { return MODULE; }
"def" { return DEF; }
"if" { return IF; }
"then" { return THEN; }
"else" { return ELSE; }
"elif" { return ELSE_IF; }
"and" { return AND; }
"or" { return OR; }
"end" { return END; }
"reduce" { return REDUCE; }
"foreach" { return FOREACH; }
"//" { return DEFINEDOR; }
"try" { return TRY; }
"catch" { return CATCH; }
"label" { return LABEL; }
"break" { return BREAK; }
"__loc__" { return LOC; }
"|=" { return SETPIPE; }
"+=" { return SETPLUS; }
"-=" { return SETMINUS; }
"*=" { return SETMULT; }
"/=" { return SETDIV; }
"%=" { return SETMOD; }
"//=" { return SETDEFINEDOR; }
"<=" { return LESSEQ; }
">=" { return GREATEREQ; }
".." { return REC; }
"."|"?"|"="|";"|","|":"|"|"|"+"|"-"|"*"|"/"|"%"|"\$"|"<"|">" { return yytext[0];}
"["|"{"|"(" {
return enter(yytext[0], YY_START, yyscanner);
}
"]"|"}"|")" {
return try_exit(yytext[0], YY_START, yyscanner);
}
"@"[a-zA-Z0-9_]+ {
yylval->literal = jv_string_sized(yytext + 1, yyleng - 1); return FORMAT;
}
[0-9.]+([eE][+-]?[0-9]+)? {
yylval->literal = jv_parse_sized(yytext, yyleng); return LITERAL;
}
"\"" {
yy_push_state(IN_QQSTRING, yyscanner);
return QQSTRING_START;
}
<IN_QQSTRING>{
"\\(" {
return enter(QQSTRING_INTERP_START, YY_START, yyscanner);
}
"\"" {
yy_pop_state(yyscanner);
return QQSTRING_END;
}
(\\[^u(]|\\u[a-zA-Z0-9]{0,4})+ {
/* pass escapes to the json parser */
jv escapes = jv_string_fmt("\"%.*s\"", (int)yyleng, yytext);
yylval->literal = jv_parse_sized(jv_string_value(escapes), jv_string_length_bytes(jv_copy(escapes)));
jv_free(escapes);
return QQSTRING_TEXT;
}
[^\\\"]+ {
yylval->literal = jv_string_sized(yytext, yyleng);
return QQSTRING_TEXT;
}
. {
return INVALID_CHARACTER;
}
}
([a-zA-Z_][a-zA-Z_0-9]*::)*[a-zA-Z_][a-zA-Z_0-9]* { yylval->literal = jv_string(yytext); return IDENT;}
\.[a-zA-Z_][a-zA-Z_0-9]* { yylval->literal = jv_string(yytext+1); return FIELD;}
[ \n\t]+ {}
. { return INVALID_CHARACTER; }
%%
/* perhaps these should be calls... */
/*
"true" { return TRUE; }
"false" { return FALSE; }
"null" { return NULL; }
*/
static int try_exit(int c, int state, yyscan_t yyscanner) {
char match = 0;
int ret;
switch (state) {
case IN_PAREN: match = ret = ')'; break;
case IN_BRACKET: match = ret = ']'; break;
case IN_BRACE: match = ret = '}'; break;
case IN_QQINTERP:
match = ')';
ret = QQSTRING_INTERP_END;
break;
default:
// may not be the best error to give
return INVALID_CHARACTER;
}
assert(match);
if (match == c) {
yy_pop_state(yyscanner);
return ret;
} else {
// FIXME: should we pop? Give a better error at least
return INVALID_CHARACTER;
}
}
static int enter(int c, int currstate, yyscan_t yyscanner) {
int state = 0;
switch (c) {
case '(': state = IN_PAREN; break;
case '[': state = IN_BRACKET; break;
case '{': state = IN_BRACE; break;
case QQSTRING_INTERP_START: state = IN_QQINTERP; break;
}
assert(state);
yy_push_state(state, yyscanner);
return c;
}
void* yyalloc(size_t sz, void* extra) {
return jv_mem_alloc(sz);
}
void* yyrealloc(void* p, size_t sz, void* extra) {
return jv_mem_realloc(p, sz);
}
void yyfree(void* p, void* extra) {
jv_mem_free(p);
}
+160
View File
@@ -0,0 +1,160 @@
#ifdef HAVE_ACOS
LIBM_DD(acos)
#else
LIBM_DD_NO(acos)
#endif
#ifdef HAVE_ACOSH
LIBM_DD(acosh)
#else
LIBM_DD_NO(acosh)
#endif
#ifdef HAVE_ASIN
LIBM_DD(asin)
#else
LIBM_DD_NO(asin)
#endif
#ifdef HAVE_ASINH
LIBM_DD(asinh)
#else
LIBM_DD_NO(asinh)
#endif
#ifdef HAVE_ATAN
LIBM_DD(atan)
#else
LIBM_DD_NO(atan)
#endif
#ifdef HAVE_ATANH
LIBM_DD(atanh)
#else
LIBM_DD_NO(atanh)
#endif
#ifdef HAVE_CBRT
LIBM_DD(cbrt)
#else
LIBM_DD_NO(cbrt)
#endif
#ifdef HAVE_COS
LIBM_DD(cos)
#else
LIBM_DD_NO(cos)
#endif
#ifdef HAVE_COSH
LIBM_DD(cosh)
#else
LIBM_DD_NO(cosh)
#endif
#ifdef HAVE_EXP2
LIBM_DD(exp2)
#else
LIBM_DD_NO(exp2)
#endif
#ifdef HAVE_EXP
LIBM_DD(exp)
#else
LIBM_DD_NO(exp)
#endif
#ifdef HAVE_FLOOR
LIBM_DD(floor)
#else
LIBM_DD_NO(floor)
#endif
#ifdef HAVE_J0
LIBM_DD(j0)
#else
LIBM_DD_NO(j0)
#endif
#ifdef HAVE_J1
LIBM_DD(j1)
#else
LIBM_DD_NO(j1)
#endif
#ifdef HAVE_LOG10
LIBM_DD(log10)
#else
LIBM_DD_NO(log10)
#endif
#ifdef HAVE_LOG2
LIBM_DD(log2)
#else
LIBM_DD_NO(log2)
#endif
#ifdef HAVE_LOG
LIBM_DD(log)
#else
LIBM_DD_NO(log)
#endif
#ifdef HAVE_SIN
LIBM_DD(sin)
#else
LIBM_DD_NO(sin)
#endif
#ifdef HAVE_SINH
LIBM_DD(sinh)
#else
LIBM_DD_NO(sinh)
#endif
#ifdef HAVE_SQRT
LIBM_DD(sqrt)
#else
LIBM_DD_NO(sqrt)
#endif
#ifdef HAVE_TAN
LIBM_DD(tan)
#else
LIBM_DD_NO(tan)
#endif
#ifdef HAVE_TANH
LIBM_DD(tanh)
#else
LIBM_DD_NO(tanh)
#endif
#ifdef HAVE_TGAMMA
LIBM_DD(tgamma)
#else
LIBM_DD_NO(tgamma)
#endif
#ifdef HAVE_Y0
LIBM_DD(y0)
#else
LIBM_DD_NO(y0)
#endif
#ifdef HAVE_Y1
LIBM_DD(y1)
#else
LIBM_DD_NO(y1)
#endif
#ifdef HAVE_POW
LIBM_DDD(pow)
#else
LIBM_DDD_NO(pow)
#endif
#ifdef HAVE_JN
LIBM_DID(jn)
#endif
#ifdef HAVE_YN
LIBM_DID(yn)
#endif
#ifdef HAVE_ATAN2
LIBM_DDD(atan2)
#else
LIBM_DDD_NO(atan2)
#endif
#ifdef HAVE_HYPOT
LIBM_DDD(hypot)
#else
LIBM_DDD_NO(hypot)
#endif
#ifdef HAVE_REMAINDER
LIBM_DDD(remainder)
#else
LIBM_DDD_NO(remainder)
#endif
#if 0
#ifdef HAVE_SCALBN
LIBM_DDI(scalbn)
#endif
#ifdef HAVE_LGAMMA_R
LIBM_DDIP(lgamma_r)
#endif
#endif
+358
View File
@@ -0,0 +1,358 @@
#include <assert.h>
#include <errno.h>
#include <limits.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <sys/stat.h>
#include <libgen.h>
#include "jq_parser.h"
#include "locfile.h"
#include "jv.h"
#include "jq.h"
#include "parser.h"
#include "util.h"
#include "compile.h"
struct lib_loading_state {
char **names;
block *defs;
uint64_t ct;
};
static int load_library(jq_state *jq, jv lib_path, int is_data, int raw,
const char *as, block *out_block,
struct lib_loading_state *lib_state);
// Given a lib_path to search first, creates a chain of search paths
// in the following order:
// 1. lib_path
// 2. -L paths passed in on the command line (from jq_state*) or builtin list
static jv build_lib_search_chain(jq_state *jq, jv search_path, jv jq_origin, jv lib_origin) {
assert(jv_get_kind(search_path) == JV_KIND_ARRAY);
jv expanded = jv_array();
jv expanded_elt;
jv_array_foreach(search_path, i, path) {
if (strcmp(".",jv_string_value(path)) == 0) {
expanded_elt = jv_copy(path);
} else if (jv_get_kind(lib_origin) == JV_KIND_STRING &&
strncmp("./", jv_string_value(path),sizeof("./")-1) == 0) {
expanded_elt = jv_string_fmt("%s/%s",
jv_string_value(lib_origin),
jv_string_value(path) + sizeof ("./") - 1);
} else if (strncmp("$ORIGIN/",jv_string_value(path),sizeof("$ORIGIN/")-1) == 0) {
expanded_elt = jv_string_fmt("%s/%s",
jv_string_value(jq_origin),
jv_string_value(path) + sizeof ("$ORIGIN/") - 1);
} else {
expanded_elt = expand_path(path);
if (!jv_is_valid(expanded_elt)) {
jv_free(search_path);
jv_free(expanded);
jv_free(path);
return expanded_elt;
}
path = jv_invalid();
}
expanded = jv_array_append(expanded, expanded_elt);
jv_free(path);
}
jv_free(jq_origin);
jv_free(lib_origin);
jv_free(search_path);
return expanded;
}
// Doesn't actually check that name not be an absolute path, and we
// don't have to: we always append relative paths to others (with a '/'
// in between).
static jv validate_relpath(jv name) {
const char *s = jv_string_value(name);
if (strchr(s, '\\')) {
jv res = jv_invalid_with_msg(jv_string_fmt("Modules must be named by relative paths using '/', not '\\' (%s)", s));
jv_free(name);
return res;
}
jv components = jv_string_split(jv_copy(name), jv_string("/"));
jv rp = jv_array_get(jv_copy(components), 0);
components = jv_array_slice(components, 1, jv_array_length(jv_copy(components)));
jv_array_foreach(components, i, x) {
if (!strcmp(jv_string_value(x), "..")) {
jv_free(x);
jv_free(rp);
jv_free(components);
jv res = jv_invalid_with_msg(jv_string_fmt("Relative paths to modules may not traverse to parent directories (%s)", s));
jv_free(name);
return res;
}
if (i > 0 && jv_equal(jv_copy(x), jv_array_get(jv_copy(components), i - 1))) {
jv_free(x);
jv_free(rp);
jv_free(components);
jv res = jv_invalid_with_msg(jv_string_fmt("module names must not have equal consecutive components: %s",
jv_string_value(name)));
jv_free(name);
return res;
}
rp = jv_string_concat(rp, jv_string_concat(jv_string("/"), x));
}
jv_free(components);
jv_free(name);
return rp;
}
// Assumes name has been validated
static jv jv_basename(jv name) {
const char *s = jv_string_value(name);
const char *p = strrchr(s, '/');
if (!p)
return name;
jv res = jv_string_fmt("%s", p);
jv_free(name);
return res;
}
// Asummes validated relative path to module
static jv find_lib(jq_state *jq, jv rel_path, jv search, const char *suffix, jv jq_origin, jv lib_origin) {
if (jv_get_kind(search) != JV_KIND_ARRAY)
return jv_invalid_with_msg(jv_string_fmt("Module search path must be an array"));
if (jv_get_kind(rel_path) != JV_KIND_STRING)
return jv_invalid_with_msg(jv_string_fmt("Module path must be a string"));
struct stat st;
int ret;
// Ideally we should cache this somewhere
search = build_lib_search_chain(jq, search, jq_origin, lib_origin);
jv bname = jv_basename(jv_copy(rel_path));
jv_array_foreach(search, i, spath) {
if (jv_get_kind(spath) == JV_KIND_NULL) {
jv_free(spath);
break;
}
if (jv_get_kind(spath) != JV_KIND_STRING ||
strcmp(jv_string_value(spath), "") == 0) {
jv_free(spath);
continue; /* XXX report non-strings in search path?? */
}
// Try ${search_dir}/${rel_path}.jq
jv testpath = jq_realpath(jv_string_fmt("%s/%s%s",
jv_string_value(spath),
jv_string_value(rel_path),
suffix));
ret = stat(jv_string_value(testpath),&st);
if (ret == -1 && errno == ENOENT) {
jv_free(testpath);
// Try ${search_dir}/$(dirname ${rel_path})/jq/main.jq
testpath = jq_realpath(jv_string_fmt("%s/%s/%s%s",
jv_string_value(spath),
jv_string_value(rel_path),
"jq/main",
suffix));
ret = stat(jv_string_value(testpath),&st);
}
if (ret == -1 && errno == ENOENT) {
jv_free(testpath);
// Try ${search_dir}/${rel_path}/$(basename ${rel_path}).jq
testpath = jq_realpath(jv_string_fmt("%s/%s/%s%s",
jv_string_value(spath),
jv_string_value(rel_path),
jv_string_value(bname),
suffix));
ret = stat(jv_string_value(testpath),&st);
}
if (ret == 0) {
jv_free(rel_path);
jv_free(search);
jv_free(bname);
jv_free(spath);
return testpath;
}
jv_free(testpath);
jv_free(spath);
}
jv output = jv_invalid_with_msg(jv_string_fmt("module not found: %s", jv_string_value(rel_path)));
jv_free(rel_path);
jv_free(search);
jv_free(bname);
return output;
}
static jv default_search(jq_state *jq, jv value) {
if (!jv_is_valid(value)) {
// dependent didn't say; prepend . to system search path listj
jv_free(value);
return jv_array_concat(JV_ARRAY(jv_string(".")), jq_get_lib_dirs(jq));
}
if (jv_get_kind(value) != JV_KIND_ARRAY)
return JV_ARRAY(value);
return value;
}
// XXX Split this into a util that takes a callback, and then...
static int process_dependencies(jq_state *jq, jv jq_origin, jv lib_origin, block *src_block, struct lib_loading_state *lib_state) {
jv deps = block_take_imports(src_block);
block bk = *src_block;
int nerrors = 0;
const char *as_str = NULL;
jv_array_foreach(deps, i, dep) {
int is_data = jv_get_kind(jv_object_get(jv_copy(dep), jv_string("is_data"))) == JV_KIND_TRUE;
int raw = 0;
jv v = jv_object_get(jv_copy(dep), jv_string("raw"));
if (jv_get_kind(v) == JV_KIND_TRUE)
raw = 1;
jv_free(v);
jv relpath = validate_relpath(jv_object_get(jv_copy(dep), jv_string("relpath")));
jv as = jv_object_get(jv_copy(dep), jv_string("as"));
assert(!jv_is_valid(as) || jv_get_kind(as) == JV_KIND_STRING);
if (jv_get_kind(as) == JV_KIND_STRING)
as_str = jv_string_value(as);
jv search = default_search(jq, jv_object_get(dep, jv_string("search")));
// dep is now freed; do not reuse
// find_lib does a lot of work that could be cached...
jv resolved = find_lib(jq, relpath, search, is_data ? ".json" : ".jq", jv_copy(jq_origin), jv_copy(lib_origin));
// XXX ...move the rest of this into a callback.
if (!jv_is_valid(resolved)) {
jv emsg = jv_invalid_get_msg(resolved);
jq_report_error(jq, jv_string_fmt("jq: error: %s\n",jv_string_value(emsg)));
jv_free(emsg);
jv_free(as);
jv_free(deps);
jv_free(jq_origin);
jv_free(lib_origin);
return 1;
}
uint64_t state_idx = 0;
for (; state_idx < lib_state->ct; ++state_idx) {
if (strcmp(lib_state->names[state_idx],jv_string_value(resolved)) == 0)
break;
}
if (state_idx < lib_state->ct) { // Found
jv_free(resolved);
// Bind the library to the program
bk = block_bind_library(lib_state->defs[state_idx], bk, OP_IS_CALL_PSEUDO, as_str);
} else { // Not found. Add it to the table before binding.
block dep_def_block = gen_noop();
nerrors += load_library(jq, resolved, is_data, raw, as_str, &dep_def_block, lib_state);
// resolved has been freed
if (nerrors == 0) {
// Bind the library to the program
bk = block_bind_library(dep_def_block, bk, OP_IS_CALL_PSEUDO, as_str);
}
}
jv_free(as);
}
jv_free(lib_origin);
jv_free(jq_origin);
jv_free(deps);
return nerrors;
}
// Loads the library at lib_path into lib_state, putting the library's defs
// into *out_block
static int load_library(jq_state *jq, jv lib_path, int is_data, int raw, const char *as, block *out_block, struct lib_loading_state *lib_state) {
int nerrors = 0;
struct locfile* src = NULL;
block program;
jv data;
if (is_data && !raw)
data = jv_load_file(jv_string_value(lib_path), 0);
else
data = jv_load_file(jv_string_value(lib_path), 1);
int state_idx;
if (!jv_is_valid(data)) {
if (jv_invalid_has_msg(jv_copy(data)))
data = jv_invalid_get_msg(data);
else
data = jv_string("unknown error");
jq_report_error(jq, jv_string_fmt("jq: error loading data file %s: %s\n", jv_string_value(lib_path), jv_string_value(data)));
nerrors++;
goto out;
} else if (is_data) {
// import "foo" as $bar;
program = gen_const_global(jv_copy(data), as);
} else {
// import "foo" as bar;
src = locfile_init(jq, jv_string_value(lib_path), jv_string_value(data), jv_string_length_bytes(jv_copy(data)));
nerrors += jq_parse_library(src, &program);
if (nerrors == 0) {
char *lib_origin = strdup(jv_string_value(lib_path));
nerrors += process_dependencies(jq, jq_get_jq_origin(jq),
jv_string(dirname(lib_origin)),
&program, lib_state);
free(lib_origin);
}
}
state_idx = lib_state->ct++;
lib_state->names = realloc(lib_state->names, lib_state->ct * sizeof(const char *));
lib_state->defs = realloc(lib_state->defs, lib_state->ct * sizeof(block));
lib_state->names[state_idx] = strdup(jv_string_value(lib_path));
lib_state->defs[state_idx] = program;
*out_block = program;
if (src)
locfile_free(src);
out:
jv_free(lib_path);
jv_free(data);
return nerrors;
}
// FIXME It'd be nice to have an option to search the same search path
// as we do in process_dependencies.
jv load_module_meta(jq_state *jq, jv mod_relpath) {
// We can't know the caller's origin; we could though, if it was passed in
jv lib_path = find_lib(jq, validate_relpath(mod_relpath), jq_get_lib_dirs(jq), ".jq", jq_get_jq_origin(jq), jv_null());
if (!jv_is_valid(lib_path))
return lib_path;
jv meta = jv_null();
jv data = jv_load_file(jv_string_value(lib_path), 1);
if (jv_is_valid(data)) {
block program;
struct locfile* src = locfile_init(jq, jv_string_value(lib_path), jv_string_value(data), jv_string_length_bytes(jv_copy(data)));
int nerrors = jq_parse_library(src, &program);
if (nerrors == 0) {
meta = block_module_meta(program);
if (jv_get_kind(meta) == JV_KIND_NULL)
meta = jv_object();
meta = jv_object_set(meta, jv_string("deps"), block_take_imports(&program));
}
locfile_free(src);
block_free(program);
}
jv_free(lib_path);
jv_free(data);
return meta;
}
int load_program(jq_state *jq, struct locfile* src, block *out_block) {
int nerrors = 0;
block program;
struct lib_loading_state lib_state = {0,0,0};
nerrors = jq_parse(src, &program);
if (nerrors)
return nerrors;
nerrors = process_dependencies(jq, jq_get_jq_origin(jq), jq_get_prog_origin(jq), &program, &lib_state);
block libs = gen_noop();
for (uint64_t i = 0; i < lib_state.ct; ++i) {
free(lib_state.names[i]);
if (nerrors == 0 && !block_is_const(lib_state.defs[i]))
libs = block_join(libs, lib_state.defs[i]);
else
block_free(lib_state.defs[i]);
}
free(lib_state.names);
free(lib_state.defs);
if (nerrors)
block_free(program);
else
*out_block = block_drop_unreferenced(block_join(libs, program));
return nerrors;
}
+7
View File
@@ -0,0 +1,7 @@
#ifndef LINKER_H
#define LINKER_H
int load_program(jq_state *jq, struct locfile* src, block *out_block);
jv load_module_meta(jq_state *jq, jv modname);
#endif
+91
View File
@@ -0,0 +1,91 @@
#include <assert.h>
#include <errno.h>
#include <stdarg.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include "jq.h"
#include "jv_alloc.h"
#include "locfile.h"
struct locfile* locfile_init(jq_state *jq, const char *fname, const char* data, int length) {
struct locfile* l = jv_mem_alloc(sizeof(struct locfile));
l->jq = jq;
l->fname = jv_string(fname);
l->data = jv_mem_alloc(length);
memcpy((char*)l->data,data,length);
l->length = length;
l->nlines = 1;
l->refct = 1;
for (int i=0; i<length; i++) {
if (data[i] == '\n') l->nlines++;
}
l->linemap = jv_mem_alloc(sizeof(int) * (l->nlines + 1));
l->linemap[0] = 0;
int line = 1;
for (int i=0; i<length; i++) {
if (data[i] == '\n') {
l->linemap[line] = i+1; // at start of line, not of \n
line++;
}
}
l->linemap[l->nlines] = length+1; // virtual last \n
return l;
}
struct locfile* locfile_retain(struct locfile* l) {
l->refct++;
return l;
}
void locfile_free(struct locfile* l) {
if (--(l->refct) == 0) {
jv_free(l->fname);
jv_mem_free(l->linemap);
jv_mem_free((char*)l->data);
jv_mem_free(l);
}
}
int locfile_get_line(struct locfile* l, int pos) {
assert(pos < l->length);
int line = 1;
while (l->linemap[line] <= pos) line++; // == if pos at start (before, never ==, because pos never on \n)
assert(line-1 < l->nlines);
return line-1;
}
static int locfile_line_length(struct locfile* l, int line) {
assert(line < l->nlines);
return l->linemap[line+1] - l->linemap[line] -1; // -1 to omit \n
}
void locfile_locate(struct locfile* l, location loc, const char* fmt, ...) {
va_list fmtargs;
va_start(fmtargs, fmt);
int startline;
int offset;
if (loc.start != -1) {
startline = locfile_get_line(l, loc.start);
offset = l->linemap[startline];
}
jv m1 = jv_string_vfmt(fmt, fmtargs);
if (!jv_is_valid(m1)) {
jq_report_error(l->jq, m1);
return;
}
if (loc.start == -1) {
jq_report_error(l->jq, jv_string_fmt("jq: error: %s\n<unknown location>", jv_string_value(m1)));
jv_free(m1);
return;
}
jv m2 = jv_string_fmt("%s at %s, line %d:\n%.*s%*s", jv_string_value(m1),
jv_string_value(l->fname), startline + 1,
locfile_line_length(l, startline), l->data + offset,
loc.start - offset, "");
jv_free(m1);
jq_report_error(l->jq, m2);
return;
}
+29
View File
@@ -0,0 +1,29 @@
#ifndef LOCFILE_H
#define LOCFILE_H
#include "jq.h"
typedef struct {
int start, end;
} location;
static const location UNKNOWN_LOCATION = {-1, -1};
struct locfile {
jv fname;
const char* data;
int length;
int* linemap;
int nlines;
char *error;
jq_state *jq;
int refct;
};
struct locfile* locfile_init(jq_state *, const char *, const char *, int);
struct locfile* locfile_retain(struct locfile *);
int locfile_get_line(struct locfile *, int);
void locfile_free(struct locfile *);
void locfile_locate(struct locfile *, location, const char *, ...);
#endif
+177
View File
@@ -0,0 +1,177 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_compare_version.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_COMPARE_VERSION(VERSION_A, OP, VERSION_B, [ACTION-IF-TRUE], [ACTION-IF-FALSE])
#
# DESCRIPTION
#
# This macro compares two version strings. Due to the various number of
# minor-version numbers that can exist, and the fact that string
# comparisons are not compatible with numeric comparisons, this is not
# necessarily trivial to do in a autoconf script. This macro makes doing
# these comparisons easy.
#
# The six basic comparisons are available, as well as checking equality
# limited to a certain number of minor-version levels.
#
# The operator OP determines what type of comparison to do, and can be one
# of:
#
# eq - equal (test A == B)
# ne - not equal (test A != B)
# le - less than or equal (test A <= B)
# ge - greater than or equal (test A >= B)
# lt - less than (test A < B)
# gt - greater than (test A > B)
#
# Additionally, the eq and ne operator can have a number after it to limit
# the test to that number of minor versions.
#
# eq0 - equal up to the length of the shorter version
# ne0 - not equal up to the length of the shorter version
# eqN - equal up to N sub-version levels
# neN - not equal up to N sub-version levels
#
# When the condition is true, shell commands ACTION-IF-TRUE are run,
# otherwise shell commands ACTION-IF-FALSE are run. The environment
# variable 'ax_compare_version' is always set to either 'true' or 'false'
# as well.
#
# Examples:
#
# AX_COMPARE_VERSION([3.15.7],[lt],[3.15.8])
# AX_COMPARE_VERSION([3.15],[lt],[3.15.8])
#
# would both be true.
#
# AX_COMPARE_VERSION([3.15.7],[eq],[3.15.8])
# AX_COMPARE_VERSION([3.15],[gt],[3.15.8])
#
# would both be false.
#
# AX_COMPARE_VERSION([3.15.7],[eq2],[3.15.8])
#
# would be true because it is only comparing two minor versions.
#
# AX_COMPARE_VERSION([3.15.7],[eq0],[3.15])
#
# would be true because it is only comparing the lesser number of minor
# versions of the two values.
#
# Note: The characters that separate the version numbers do not matter. An
# empty string is the same as version 0. OP is evaluated by autoconf, not
# configure, so must be a string, not a variable.
#
# The author would like to acknowledge Guido Draheim whose advice about
# the m4_case and m4_ifvaln functions make this macro only include the
# portions necessary to perform the specific comparison specified by the
# OP argument in the final configure script.
#
# LICENSE
#
# Copyright (c) 2008 Tim Toolan <toolan@ele.uri.edu>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 11
dnl #########################################################################
AC_DEFUN([AX_COMPARE_VERSION], [
AC_REQUIRE([AC_PROG_AWK])
# Used to indicate true or false condition
ax_compare_version=false
# Convert the two version strings to be compared into a format that
# allows a simple string comparison. The end result is that a version
# string of the form 1.12.5-r617 will be converted to the form
# 0001001200050617. In other words, each number is zero padded to four
# digits, and non digits are removed.
AS_VAR_PUSHDEF([A],[ax_compare_version_A])
A=`echo "$1" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
-e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
-e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
-e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
-e 's/[[^0-9]]//g'`
AS_VAR_PUSHDEF([B],[ax_compare_version_B])
B=`echo "$3" | sed -e 's/\([[0-9]]*\)/Z\1Z/g' \
-e 's/Z\([[0-9]]\)Z/Z0\1Z/g' \
-e 's/Z\([[0-9]][[0-9]]\)Z/Z0\1Z/g' \
-e 's/Z\([[0-9]][[0-9]][[0-9]]\)Z/Z0\1Z/g' \
-e 's/[[^0-9]]//g'`
dnl # In the case of le, ge, lt, and gt, the strings are sorted as necessary
dnl # then the first line is used to determine if the condition is true.
dnl # The sed right after the echo is to remove any indented white space.
m4_case(m4_tolower($2),
[lt],[
ax_compare_version=`echo "x$A
x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/false/;s/x${B}/true/;1q"`
],
[gt],[
ax_compare_version=`echo "x$A
x$B" | sed 's/^ *//' | sort | sed "s/x${A}/false/;s/x${B}/true/;1q"`
],
[le],[
ax_compare_version=`echo "x$A
x$B" | sed 's/^ *//' | sort | sed "s/x${A}/true/;s/x${B}/false/;1q"`
],
[ge],[
ax_compare_version=`echo "x$A
x$B" | sed 's/^ *//' | sort -r | sed "s/x${A}/true/;s/x${B}/false/;1q"`
],[
dnl Split the operator from the subversion count if present.
m4_bmatch(m4_substr($2,2),
[0],[
# A count of zero means use the length of the shorter version.
# Determine the number of characters in A and B.
ax_compare_version_len_A=`echo "$A" | $AWK '{print(length)}'`
ax_compare_version_len_B=`echo "$B" | $AWK '{print(length)}'`
# Set A to no more than B's length and B to no more than A's length.
A=`echo "$A" | sed "s/\(.\{$ax_compare_version_len_B\}\).*/\1/"`
B=`echo "$B" | sed "s/\(.\{$ax_compare_version_len_A\}\).*/\1/"`
],
[[0-9]+],[
# A count greater than zero means use only that many subversions
A=`echo "$A" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
B=`echo "$B" | sed "s/\(\([[0-9]]\{4\}\)\{m4_substr($2,2)\}\).*/\1/"`
],
[.+],[
AC_WARNING(
[illegal OP numeric parameter: $2])
],[])
# Pad zeros at end of numbers to make same length.
ax_compare_version_tmp_A="$A`echo $B | sed 's/./0/g'`"
B="$B`echo $A | sed 's/./0/g'`"
A="$ax_compare_version_tmp_A"
# Check for equality or inequality as necessary.
m4_case(m4_tolower(m4_substr($2,0,2)),
[eq],[
test "x$A" = "x$B" && ax_compare_version=true
],
[ne],[
test "x$A" != "x$B" && ax_compare_version=true
],[
AC_WARNING([illegal OP parameter: $2])
])
])
AS_VAR_POPDEF([A])dnl
AS_VAR_POPDEF([B])dnl
dnl # Execute ACTION-IF-TRUE / ACTION-IF-FALSE.
if test "$ax_compare_version" = "true" ; then
m4_ifvaln([$4],[$4],[:])dnl
m4_ifvaln([$5],[else $5])dnl
fi
]) dnl AX_COMPARE_VERSION
+68
View File
@@ -0,0 +1,68 @@
# ===========================================================================
# Modified from
# http://www.gnu.org/software/autoconf-archive/ax_prog_perl_version.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_PROG_BISON_VERSION([VERSION],[ACTION-IF-TRUE],[ACTION-IF-FALSE])
#
# DESCRIPTION
#
# Makes sure that bison supports the version indicated. If true the shell
# commands in ACTION-IF-TRUE are executed. If not the shell commands in
# ACTION-IF-FALSE are run. Note if $PERL is not set the macro will fail.
#
# Example:
#
# AC_PROG_YACC
# AX_PROG_BISON_VERSION([3.0.0],[ ... ],[ ... ])
#
# This will check to make sure that the bison you have supports at least
# version 3.0.0.
#
# NOTE: This macro uses the $YACC variable to perform the check.
# AX_WITH_YACC can be used to set that variable prior to running this
# macro. The $BISON_VERSION variable will be set with the detected
# version.
#
# LICENSE
#
# Copyright (c) 2009 Francesco Salvestrini <salvestrini@users.sourceforge.net>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 12
AC_DEFUN([AX_PROG_BISON_VERSION],[
AC_REQUIRE([AC_PROG_SED])
AC_REQUIRE([AC_PROG_GREP])
AS_IF([test -n "$YACC"],[
ax_bison_version="$1"
AC_MSG_CHECKING([for bison version])
changequote(<<,>>)
bison_version=`$YACC --version 2>&1 \
| $GREP bison \
| $SED -e 's/^[^0-9]*//' -e 's/[^0-9]*$//'`
changequote([,])
AC_MSG_RESULT($bison_version)
AC_SUBST([BISON_VERSION],[$bison_version])
AX_COMPARE_VERSION([$ax_bison_version],[le],[$bison_version],[
:
$2
],[
:
$3
])
],[
AC_MSG_WARN([could not find bison])
$3
])
])
+566
View File
@@ -0,0 +1,566 @@
#include <assert.h>
#include <ctype.h>
#include <errno.h>
#include <libgen.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#ifdef WIN32
#include <windows.h>
#include <io.h>
#include <fcntl.h>
#include <processenv.h>
#include <shellapi.h>
#include <wchar.h>
#include <wtypes.h>
#endif
#include "compile.h"
#include "jv.h"
#include "jq.h"
#include "jv_alloc.h"
#include "util.h"
#include "version.h"
int jq_testsuite(jv lib_dirs, int verbose, int argc, char* argv[]);
static const char* progname;
/*
* For a longer help message we could use a better option parsing
* strategy, one that lets stack options.
*/
static void usage(int code) {
FILE *f = stderr;
if (code == 0)
f = stdout;
int ret = fprintf(f,
"jq - commandline JSON processor [version %s]\n"
"Usage: %s [options] <jq filter> [file...]\n\n"
"\tjq is a tool for processing JSON inputs, applying the\n"
"\tgiven filter to its JSON text inputs and producing the\n"
"\tfilter's results as JSON on standard output.\n"
"\tThe simplest filter is ., which is the identity filter,\n"
"\tcopying jq's input to its output unmodified (except for\n"
"\tformatting).\n"
"\tFor more advanced filters see the jq(1) manpage (\"man jq\")\n"
"\tand/or http://stedolan.github.io/jq\n\n"
"\tSome of the options include:\n"
"\t -c\t\tcompact instead of pretty-printed output;\n"
"\t -n\t\tuse `null` as the single input value;\n"
"\t -e\t\tset the exit status code based on the output;\n"
"\t -s\t\tread (slurp) all inputs into an array; apply filter to it;\n"
"\t -r\t\toutput raw strings, not JSON texts;\n"
"\t -R\t\tread raw strings, not JSON texts;\n"
"\t -C\t\tcolorize JSON;\n"
"\t -M\t\tmonochrome (don't colorize JSON);\n"
"\t -S\t\tsort keys of objects on output;\n"
"\t --tab\tuse tabs for indentation;\n"
"\t --arg a v\tset variable $a to value <v>;\n"
"\t --argjson a v\tset variable $a to JSON value <v>;\n"
"\t --slurpfile a f\tset variable $a to an array of JSON texts read from <f>;\n"
"\tSee the manpage for more options.\n", JQ_VERSION, progname);
exit((ret < 0 && code == 0) ? 2 : code);
}
static void die() {
fprintf(stderr, "Use %s --help for help with command-line options,\n", progname);
fprintf(stderr, "or see the jq manpage, or online docs at http://stedolan.github.io/jq\n");
exit(2);
}
static int isoptish(const char* text) {
return text[0] == '-' && (text[1] == '-' || isalpha(text[1]));
}
static int isoption(const char* text, char shortopt, const char* longopt, size_t *short_opts) {
if (text[0] != '-' || text[1] == '-')
*short_opts = 0;
if (text[0] != '-') return 0;
// check long option
if (text[1] == '-' && !strcmp(text+2, longopt)) return 1;
else if (text[1] == '-') return 0;
// must be short option; check it and...
if (!shortopt) return 0;
if (strchr(text, shortopt) != NULL) {
(*short_opts)++; // ...count it (for option stacking)
return 1;
}
return 0;
}
enum {
SLURP = 1,
RAW_INPUT = 2,
PROVIDE_NULL = 4,
RAW_OUTPUT = 8,
ASCII_OUTPUT = 32,
COLOUR_OUTPUT = 64,
NO_COLOUR_OUTPUT = 128,
SORTED_OUTPUT = 256,
FROM_FILE = 512,
RAW_NO_LF = 1024,
UNBUFFERED_OUTPUT = 2048,
EXIT_STATUS = 4096,
SEQ = 8192,
RUN_TESTS = 16384,
/* debugging only */
DUMP_DISASM = 32768,
};
static int options = 0;
static int process(jq_state *jq, jv value, int flags, int dumpopts) {
int ret = 14; // No valid results && -e -> exit(4)
jq_start(jq, value, flags);
jv result;
while (jv_is_valid(result = jq_next(jq))) {
if ((options & RAW_OUTPUT) && jv_get_kind(result) == JV_KIND_STRING) {
fwrite(jv_string_value(result), 1, jv_string_length_bytes(jv_copy(result)), stdout);
ret = 0;
jv_free(result);
} else {
if (jv_get_kind(result) == JV_KIND_FALSE || jv_get_kind(result) == JV_KIND_NULL)
ret = 11;
else
ret = 0;
if (options & SEQ)
priv_fwrite("\036", 1, stdout, dumpopts & JV_PRINT_ISATTY);
jv_dump(result, dumpopts);
}
if (!(options & RAW_NO_LF))
priv_fwrite("\n", 1, stdout, dumpopts & JV_PRINT_ISATTY);
if (options & UNBUFFERED_OUTPUT)
fflush(stdout);
}
if (jv_invalid_has_msg(jv_copy(result))) {
// Uncaught jq exception
jv msg = jv_invalid_get_msg(jv_copy(result));
jv input_pos = jq_util_input_get_position(jq);
if (jv_get_kind(msg) == JV_KIND_STRING) {
fprintf(stderr, "jq: error (at %s): %s\n",
jv_string_value(input_pos), jv_string_value(msg));
} else {
msg = jv_dump_string(msg, 0);
fprintf(stderr, "jq: error (at %s) (not a string): %s\n",
jv_string_value(input_pos), jv_string_value(msg));
}
ret = 5;
jv_free(input_pos);
jv_free(msg);
}
jv_free(result);
return ret;
}
static void debug_cb(void *data, jv input) {
int dumpopts = *(int *)data;
jv_dumpf(JV_ARRAY(jv_string("DEBUG:"), input), stderr, dumpopts & ~(JV_PRINT_PRETTY));
fprintf(stderr, "\n");
}
int main(int argc, char* argv[]) {
jq_state *jq = NULL;
int ret = 0;
int compiled = 0;
int parser_flags = 0;
int nfiles = 0;
int badwrite;
jv program_arguments = jv_array();
#ifdef WIN32
SetConsoleOutputCP(CP_UTF8);
fflush(stdout);
fflush(stderr);
_setmode(fileno(stdout), _O_TEXT | _O_U8TEXT);
_setmode(fileno(stderr), _O_TEXT | _O_U8TEXT);
int wargc;
wchar_t **wargv = CommandLineToArgvW(GetCommandLineW(), &wargc);
assert(wargc == argc);
size_t arg_sz;
for (int i = 0; i < argc; i++) {
argv[i] = alloca((arg_sz = WideCharToMultiByte(CP_UTF8,
0,
wargv[i],
-1, 0, 0, 0, 0)));
WideCharToMultiByte(CP_UTF8, 0, wargv[i], -1, argv[i], arg_sz, 0, 0);
}
#endif
if (argc) progname = argv[0];
jq = jq_init();
if (jq == NULL) {
perror("malloc");
ret = 2;
goto out;
}
int dumpopts = JV_PRINT_INDENT_FLAGS(2);
const char* program = 0;
jq_util_input_state *input_state = jq_util_input_init(NULL, NULL); // XXX add err_cb
int further_args_are_files = 0;
int jq_flags = 0;
size_t short_opts = 0;
jv lib_search_paths = jv_null();
for (int i=1; i<argc; i++, short_opts = 0) {
if (further_args_are_files) {
jq_util_input_add_input(input_state, argv[i]);
nfiles++;
} else if (!strcmp(argv[i], "--")) {
if (!program) usage(2);
further_args_are_files = 1;
} else if (!isoptish(argv[i])) {
if (program) {
jq_util_input_add_input(input_state, argv[i]);
nfiles++;
} else {
program = argv[i];
}
} else {
if (argv[i][1] == 'L') {
if (jv_get_kind(lib_search_paths) == JV_KIND_NULL)
lib_search_paths = jv_array();
if (argv[i][2] != 0) { // -Lname (faster check than strlen)
lib_search_paths = jv_array_append(lib_search_paths, jv_string(argv[i]+2));
} else if (i >= argc - 1) {
fprintf(stderr, "-L takes a parameter: (e.g. -L /search/path or -L/search/path)\n");
die();
} else {
lib_search_paths = jv_array_append(lib_search_paths, jv_string(argv[i+1]));
i++;
}
continue;
}
if (isoption(argv[i], 's', "slurp", &short_opts)) {
options |= SLURP;
if (!short_opts) continue;
}
if (isoption(argv[i], 'r', "raw-output", &short_opts)) {
options |= RAW_OUTPUT;
if (!short_opts) continue;
}
if (isoption(argv[i], 'c', "compact-output", &short_opts)) {
dumpopts &= ~(JV_PRINT_TAB | JV_PRINT_INDENT_FLAGS(7));
if (!short_opts) continue;
}
if (isoption(argv[i], 'C', "color-output", &short_opts)) {
options |= COLOUR_OUTPUT;
if (!short_opts) continue;
}
if (isoption(argv[i], 'M', "monochrome-output", &short_opts)) {
options |= NO_COLOUR_OUTPUT;
if (!short_opts) continue;
}
if (isoption(argv[i], 'a', "ascii-output", &short_opts)) {
options |= ASCII_OUTPUT;
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "unbuffered", &short_opts)) {
options |= UNBUFFERED_OUTPUT;
if (!short_opts) continue;
}
if (isoption(argv[i], 'S', "sort-keys", &short_opts)) {
options |= SORTED_OUTPUT;
if (!short_opts) continue;
}
if (isoption(argv[i], 'R', "raw-input", &short_opts)) {
options |= RAW_INPUT;
if (!short_opts) continue;
}
if (isoption(argv[i], 'n', "null-input", &short_opts)) {
options |= PROVIDE_NULL;
if (!short_opts) continue;
}
if (isoption(argv[i], 'f', "from-file", &short_opts)) {
options |= FROM_FILE;
if (!short_opts) continue;
}
if (isoption(argv[i], 'j', "join-output", &short_opts)) {
options |= RAW_OUTPUT | RAW_NO_LF;
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "tab", &short_opts)) {
dumpopts &= ~JV_PRINT_INDENT_FLAGS(7);
dumpopts |= JV_PRINT_TAB | JV_PRINT_PRETTY;
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "indent", &short_opts)) {
if (i >= argc - 1) {
fprintf(stderr, "%s: --indent takes one parameter\n", progname);
die();
}
dumpopts &= ~(JV_PRINT_TAB | JV_PRINT_INDENT_FLAGS(7));
int indent = atoi(argv[i+1]);
if (indent < -1 || indent > 7) {
fprintf(stderr, "%s: --indent takes a number between -1 and 7\n", progname);
die();
}
dumpopts |= JV_PRINT_INDENT_FLAGS(indent);
i++;
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "seq", &short_opts)) {
options |= SEQ;
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "stream", &short_opts)) {
parser_flags |= JV_PARSE_STREAMING;
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "stream-errors", &short_opts)) {
parser_flags |= JV_PARSE_STREAM_ERRORS;
if (!short_opts) continue;
}
if (isoption(argv[i], 'e', "exit-status", &short_opts)) {
options |= EXIT_STATUS;
if (!short_opts) continue;
}
// FIXME: For --arg* we should check that the varname is acceptable
if (isoption(argv[i], 0, "arg", &short_opts)) {
if (i >= argc - 2) {
fprintf(stderr, "%s: --arg takes two parameters (e.g. --arg varname value)\n", progname);
die();
}
jv arg = jv_object();
arg = jv_object_set(arg, jv_string("name"), jv_string(argv[i+1]));
arg = jv_object_set(arg, jv_string("value"), jv_string(argv[i+2]));
program_arguments = jv_array_append(program_arguments, arg);
i += 2; // skip the next two arguments
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "argjson", &short_opts)) {
if (i >= argc - 2) {
fprintf(stderr, "%s: --argjson takes two parameters (e.g. --argjson varname text)\n", progname);
die();
}
jv v = jv_parse(argv[i+2]);
if (!jv_is_valid(v)) {
fprintf(stderr, "%s: invalid JSON text passed to --argjson\n", progname);
die();
}
jv arg = jv_object();
arg = jv_object_set(arg, jv_string("name"), jv_string(argv[i+1]));
arg = jv_object_set(arg, jv_string("value"), v);
program_arguments = jv_array_append(program_arguments, arg);
i += 2; // skip the next two arguments
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "argfile", &short_opts) ||
isoption(argv[i], 0, "slurpfile", &short_opts)) {
const char *which;
if (isoption(argv[i], 0, "argfile", &short_opts))
which = "argfile";
else
which = "slurpfile";
if (i >= argc - 2) {
fprintf(stderr, "%s: --%s takes two parameters (e.g. --%s varname filename)\n", progname, which, which);
die();
}
jv arg = jv_object();
arg = jv_object_set(arg, jv_string("name"), jv_string(argv[i+1]));
jv data = jv_load_file(argv[i+2], 0);
if (!jv_is_valid(data)) {
data = jv_invalid_get_msg(data);
fprintf(stderr, "%s: Bad JSON in --%s %s %s: %s\n", progname, which,
argv[i+1], argv[i+2], jv_string_value(data));
jv_free(data);
jv_free(arg);
ret = 2;
goto out;
}
if (isoption(argv[i], 0, "argfile", &short_opts) &&
jv_get_kind(data) == JV_KIND_ARRAY && jv_array_length(jv_copy(data)) == 1)
data = jv_array_get(data, 0);
arg = jv_object_set(arg, jv_string("value"), data);
program_arguments = jv_array_append(program_arguments, arg);
i += 2; // skip the next two arguments
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "debug-dump-disasm", &short_opts)) {
options |= DUMP_DISASM;
if (!short_opts) continue;
}
if (isoption(argv[i], 0, "debug-trace", &short_opts)) {
jq_flags |= JQ_DEBUG_TRACE;
if (!short_opts) continue;
}
if (isoption(argv[i], 'h', "help", &short_opts)) {
usage(0);
if (!short_opts) continue;
}
if (isoption(argv[i], 'V', "version", &short_opts)) {
printf("jq-%s\n", JQ_VERSION);
ret = 0;
goto out;
}
if (isoption(argv[i], 0, "run-tests", &short_opts)) {
i++;
// XXX Pass program_arguments, even a whole jq_state *, through;
// could be useful for testing
ret = jq_testsuite(lib_search_paths,
(options & DUMP_DISASM) || (jq_flags & JQ_DEBUG_TRACE),
argc - i, argv + i);
goto out;
}
// check for unknown options... if this argument was a short option
if (strlen(argv[i]) != short_opts + 1) {
fprintf(stderr, "%s: Unknown option %s\n", progname, argv[i]);
die();
}
}
}
if (isatty(fileno(stdout))) {
dumpopts |= JV_PRINT_ISATTY;
#ifndef WIN32
/* Disable colour by default on Windows builds as Windows
terminals tend not to display it correctly */
dumpopts |= JV_PRINT_COLOUR;
#endif
}
if (options & SORTED_OUTPUT) dumpopts |= JV_PRINT_SORTED;
if (options & ASCII_OUTPUT) dumpopts |= JV_PRINT_ASCII;
if (options & COLOUR_OUTPUT) dumpopts |= JV_PRINT_COLOUR;
if (options & NO_COLOUR_OUTPUT) dumpopts &= ~JV_PRINT_COLOUR;
if (jv_get_kind(lib_search_paths) == JV_KIND_NULL) {
// Default search path list
lib_search_paths = JV_ARRAY(jv_string("~/.jq"),
jv_string("$ORIGIN/../lib/jq"),
jv_string("$ORIGIN/lib"));
}
jq_set_attr(jq, jv_string("JQ_LIBRARY_PATH"), lib_search_paths);
char *origin = strdup(argv[0]);
if (origin == NULL) {
fprintf(stderr, "Error: out of memory\n");
exit(1);
}
jq_set_attr(jq, jv_string("JQ_ORIGIN"), jv_string(dirname(origin)));
free(origin);
if (strchr(JQ_VERSION, '-') == NULL)
jq_set_attr(jq, jv_string("VERSION_DIR"), jv_string(JQ_VERSION));
else
jq_set_attr(jq, jv_string("VERSION_DIR"), jv_string_fmt("%.*s-master", (int)(strchr(JQ_VERSION, '-') - JQ_VERSION), JQ_VERSION));
#if (!defined(WIN32) && defined(HAVE_ISATTY)) || defined(HAVE__ISATTY)
#if defined(HAVE__ISATTY) && defined(isatty)
#undef isatty
#define isatty _isatty
#endif
if (!program && isatty(STDOUT_FILENO) && !isatty(STDIN_FILENO))
program = ".";
#endif
if (!program) usage(2);
if (options & FROM_FILE) {
char *program_origin = strdup(program);
if (program_origin == NULL) {
perror("malloc");
exit(2);
}
jv data = jv_load_file(program, 1);
if (!jv_is_valid(data)) {
data = jv_invalid_get_msg(data);
fprintf(stderr, "%s: %s\n", progname, jv_string_value(data));
jv_free(data);
ret = 2;
goto out;
}
jq_set_attr(jq, jv_string("PROGRAM_ORIGIN"), jq_realpath(jv_string(dirname(program_origin))));
compiled = jq_compile_args(jq, jv_string_value(data), jv_copy(program_arguments));
free(program_origin);
jv_free(data);
} else {
jq_set_attr(jq, jv_string("PROGRAM_ORIGIN"), jq_realpath(jv_string("."))); // XXX is this good?
compiled = jq_compile_args(jq, program, jv_copy(program_arguments));
}
if (!compiled){
ret = 3;
goto out;
}
if (options & DUMP_DISASM) {
jq_dump_disassembly(jq, 0);
printf("\n");
}
if ((options & SEQ))
parser_flags |= JV_PARSE_SEQ;
if ((options & RAW_INPUT))
jq_util_input_set_parser(input_state, NULL, (options & SLURP) ? 1 : 0);
else
jq_util_input_set_parser(input_state, jv_parser_new(parser_flags), (options & SLURP) ? 1 : 0);
// Let jq program read from inputs
jq_set_input_cb(jq, jq_util_input_next_input_cb, input_state);
// Let jq program call `debug` builtin and have that go somewhere
jq_set_debug_cb(jq, debug_cb, &dumpopts);
if (nfiles == 0)
jq_util_input_add_input(input_state, "-");
if (options & PROVIDE_NULL) {
ret = process(jq, jv_null(), jq_flags, dumpopts);
} else {
jv value;
while (jq_util_input_errors(input_state) == 0 &&
(jv_is_valid((value = jq_util_input_next_input(input_state))) || jv_invalid_has_msg(jv_copy(value)))) {
if (jv_is_valid(value)) {
ret = process(jq, value, jq_flags, dumpopts);
continue;
}
// Parse error
jv msg = jv_invalid_get_msg(value);
if (!(options & SEQ)) {
// --seq -> errors are not fatal
ret = 4;
fprintf(stderr, "parse error: %s\n", jv_string_value(msg));
jv_free(msg);
break;
}
fprintf(stderr, "ignoring parse error: %s\n", jv_string_value(msg));
jv_free(msg);
}
}
if (jq_util_input_errors(input_state) != 0)
ret = 2;
out:
badwrite = ferror(stdout);
if (fclose(stdout)!=0 || badwrite) {
fprintf(stderr,"Error: writing output failed: %s\n", strerror(errno));
ret = 2;
}
jv_free(program_arguments);
jq_util_input_free(&input_state);
jq_teardown(&jq);
if (ret >= 10 && (options & EXIT_STATUS))
return ret - 10;
if (ret >= 10)
return 0;
return ret;
}
+44
View File
@@ -0,0 +1,44 @@
OP(LOADK, CONSTANT, 1, 1)
OP(DUP, NONE, 1, 2)
OP(DUPN, NONE, 1, 2)
OP(DUP2, NONE, 2, 3)
OP(POP, NONE, 1, 0)
OP(LOADV, VARIABLE, 1, 1)
OP(LOADVN, VARIABLE, 1, 1)
OP(STOREV, VARIABLE, 1, 0)
OP(STORE_GLOBAL, GLOBAL, 0, 0)
OP(INDEX, NONE, 2, 1)
OP(INDEX_OPT, NONE, 2, 1)
OP(EACH, NONE, 1, 1)
OP(EACH_OPT, NONE, 1, 1)
OP(FORK, BRANCH, 0, 0)
OP(FORK_OPT, BRANCH, 0, 0)
OP(JUMP, BRANCH, 0, 0)
OP(JUMP_F,BRANCH, 1, 0)
OP(BACKTRACK, NONE, 0, 0)
OP(APPEND, VARIABLE,1, 0)
OP(INSERT, NONE, 4, 2)
OP(RANGE, VARIABLE, 1, 1)
OP(SUBEXP_BEGIN, NONE, 1, 2)
OP(SUBEXP_END, NONE, 2, 2)
OP(PATH_BEGIN, NONE, 1, 2)
OP(PATH_END, NONE, 2, 1)
OP(CALL_BUILTIN, CFUNC, -1, 1)
OP(CALL_JQ, UFUNC, 1, 1)
OP(RET, NONE, 1, 1)
OP(TAIL_CALL_JQ, UFUNC, 1, 1)
OP(CLOSURE_PARAM, DEFINITION, 0, 0)
OP(CLOSURE_REF, CLOSURE_REF_IMM, 0, 0)
OP(CLOSURE_CREATE, DEFINITION, 0, 0)
OP(CLOSURE_CREATE_C, DEFINITION, 0, 0)
OP(TOP, NONE, 0, 0)
OP(CLOSURE_PARAM_REGULAR, DEFINITION, 0, 0)
OP(DEPS, CONSTANT, 0, 0)
OP(MODULEMETA, CONSTANT, 0, 0)
OP(GENLABEL, NONE, 0, 1)
+3914
View File
File diff suppressed because it is too large. Load diff
+193
View File
@@ -0,0 +1,193 @@
/* A Bison parser, made by GNU Bison 3.0.2. */
/* Bison interface for Yacc-like parsers in C
Copyright (C) 1984, 1989-1990, 2000-2013 Free Software Foundation, Inc.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>. */
/* As a special exception, you may create a larger work that contains
part or all of the Bison parser skeleton and distribute that work
under terms of your choice, so long as that work isn't itself a
parser generator using the skeleton or a modified version thereof
as a parser skeleton. Alternatively, if you modify or redistribute
the parser skeleton itself, you may (at your option) remove this
special exception, which will cause the skeleton and the resulting
Bison output files to be licensed under the GNU General Public
License without this special exception.
This special exception was added by the Free Software Foundation in
version 2.2 of Bison. */
#ifndef YY_YY_PARSER_H_INCLUDED
# define YY_YY_PARSER_H_INCLUDED
/* Debug traces. */
#ifndef YYDEBUG
# define YYDEBUG 0
#endif
#if YYDEBUG
extern int yydebug;
#endif
/* "%code requires" blocks. */
#line 11 "parser.y" /* yacc.c:1909 */
#include "locfile.h"
struct lexer_param;
#define YYLTYPE location
#define YYLLOC_DEFAULT(Loc, Rhs, N) \
do { \
if (N) { \
(Loc).start = YYRHSLOC(Rhs, 1).start; \
(Loc).end = YYRHSLOC(Rhs, N).end; \
} else { \
(Loc).start = YYRHSLOC(Rhs, 0).end; \
(Loc).end = YYRHSLOC(Rhs, 0).end; \
} \
} while (0)
#line 61 "parser.h" /* yacc.c:1909 */
/* Token type. */
#ifndef YYTOKENTYPE
# define YYTOKENTYPE
enum yytokentype
{
INVALID_CHARACTER = 258,
IDENT = 259,
FIELD = 260,
LITERAL = 261,
FORMAT = 262,
REC = 263,
SETMOD = 264,
EQ = 265,
NEQ = 266,
DEFINEDOR = 267,
AS = 268,
DEF = 269,
MODULE = 270,
IMPORT = 271,
INCLUDE = 272,
IF = 273,
THEN = 274,
ELSE = 275,
ELSE_IF = 276,
REDUCE = 277,
FOREACH = 278,
END = 279,
AND = 280,
OR = 281,
TRY = 282,
CATCH = 283,
LABEL = 284,
BREAK = 285,
LOC = 286,
SETPIPE = 287,
SETPLUS = 288,
SETMINUS = 289,
SETMULT = 290,
SETDIV = 291,
SETDEFINEDOR = 292,
LESSEQ = 293,
GREATEREQ = 294,
QQSTRING_START = 295,
QQSTRING_TEXT = 296,
QQSTRING_INTERP_START = 297,
QQSTRING_INTERP_END = 298,
QQSTRING_END = 299,
FUNCDEF = 300,
NONOPT = 301
};
#endif
/* Tokens. */
#define INVALID_CHARACTER 258
#define IDENT 259
#define FIELD 260
#define LITERAL 261
#define FORMAT 262
#define REC 263
#define SETMOD 264
#define EQ 265
#define NEQ 266
#define DEFINEDOR 267
#define AS 268
#define DEF 269
#define MODULE 270
#define IMPORT 271
#define INCLUDE 272
#define IF 273
#define THEN 274
#define ELSE 275
#define ELSE_IF 276
#define REDUCE 277
#define FOREACH 278
#define END 279
#define AND 280
#define OR 281
#define TRY 282
#define CATCH 283
#define LABEL 284
#define BREAK 285
#define LOC 286
#define SETPIPE 287
#define SETPLUS 288
#define SETMINUS 289
#define SETMULT 290
#define SETDIV 291
#define SETDEFINEDOR 292
#define LESSEQ 293
#define GREATEREQ 294
#define QQSTRING_START 295
#define QQSTRING_TEXT 296
#define QQSTRING_INTERP_START 297
#define QQSTRING_INTERP_END 298
#define QQSTRING_END 299
#define FUNCDEF 300
#define NONOPT 301
/* Value type. */
#if ! defined YYSTYPE && ! defined YYSTYPE_IS_DECLARED
typedef union YYSTYPE YYSTYPE;
union YYSTYPE
{
#line 31 "parser.y" /* yacc.c:1909 */
jv literal;
block blk;
#line 170 "parser.h" /* yacc.c:1909 */
};
# define YYSTYPE_IS_TRIVIAL 1
# define YYSTYPE_IS_DECLARED 1
#endif
/* Location type. */
#if ! defined YYLTYPE && ! defined YYLTYPE_IS_DECLARED
typedef struct YYLTYPE YYLTYPE;
struct YYLTYPE
{
int first_line;
int first_column;
int last_line;
int last_column;
};
# define YYLTYPE_IS_DECLARED 1
# define YYLTYPE_IS_TRIVIAL 1
#endif
int yyparse (block* answer, int* errors, struct locfile* locations, struct lexer_param* lexer_param_ptr);
#endif /* !YY_YY_PARSER_H_INCLUDED */
Loaded 100 of 130 files, more files were not shown because too many files have changed in this diff. Show more