Fix a parser bug where tokens are misidentified as commas. (#1502)

* Fix a parser bug where tokens are misidentified as commas.

In the old and new readers, when parsing an object, a comment
followed by any non-`}` token is treated as a comma.

The new unit test required changing the runjsontests.py
flag regime so that failure tests could be run with default settings.

* Honor allowComments==false mode.

Much of the comment handling in the parsers is bespoke, and does not
honor this flag.  By unfiying it under a common API, the parser is
simplified and strict mode is now more correctly strict.

Note that allowComments mode does not allow for comments in
arbitrary locations; they are allowed only in certain positions.
Rectifying this is a bigger effort, since collectComments mode requires
storing the comments somewhere, and it's not immediately clear
where in the DOM all such comments should live.

---------

Co-authored-by: Jordan Bayles <bayles.jordan@gmail.com>
This commit is contained in:
vslashg
2024-09-09 20:30:16 -04:00
committed by GitHub
parent c3a986600f
commit 0a9b9d9c6e
8 changed files with 49 additions and 55 deletions

View File

@@ -0,0 +1,4 @@
{
"a": "aaa",
"b": "bbb" // comments not allowed in strict mode
}

View File

@@ -0,0 +1,4 @@
{
"a": "aaa", // comments not allowed in strict mode
"b": "bbb"
}

View File

@@ -0,0 +1,3 @@
{
"array" : [1, 2, 3 /* comments not allowed in strict mode */]
}

View File

@@ -0,0 +1 @@
{"one": 1 /* } */ { "two" : 2 }

View File

@@ -97,14 +97,17 @@ def runAllTests(jsontest_executable_path, input_dir = None,
valgrind_path = use_valgrind and VALGRIND_CMD or ''
for input_path in tests + test_jsonchecker:
expect_failure = os.path.basename(input_path).startswith('fail')
is_json_checker_test = (input_path in test_jsonchecker) or expect_failure
is_json_checker_test = input_path in test_jsonchecker
is_parse_only = is_json_checker_test or expect_failure
is_strict_test = ('_strict_' in os.path.basename(input_path)) or is_json_checker_test
print('TESTING:', input_path, end=' ')
options = is_json_checker_test and '--json-checker' or ''
options = is_parse_only and '--parse-only' or ''
options += is_strict_test and ' --strict' or ''
options += ' --json-writer %s'%writerClass
cmd = '%s%s %s "%s"' % ( valgrind_path, jsontest_executable_path, options,
input_path)
status, process_output = getStatusOutput(cmd)
if is_json_checker_test:
if is_parse_only:
if expect_failure:
if not status:
print('FAILED')