Fix a parser bug where tokens are misidentified as commas. (#1502)

* Fix a parser bug where tokens are misidentified as commas.

In the old and new readers, when parsing an object, a comment
followed by any non-`}` token is treated as a comma.

The new unit test required changing the runjsontests.py
flag regime so that failure tests could be run with default settings.

* Honor allowComments==false mode.

Much of the comment handling in the parsers is bespoke, and does not
honor this flag.  By unfiying it under a common API, the parser is
simplified and strict mode is now more correctly strict.

Note that allowComments mode does not allow for comments in
arbitrary locations; they are allowed only in certain positions.
Rectifying this is a bigger effort, since collectComments mode requires
storing the comments somewhere, and it's not immediately clear
where in the DOM all such comments should live.

---------

Co-authored-by: Jordan Bayles <bayles.jordan@gmail.com>
This commit is contained in:
vslashg
2024-09-09 20:30:16 -04:00
committed by GitHub
parent c3a986600f
commit 0a9b9d9c6e
8 changed files with 49 additions and 55 deletions

View File

@@ -190,6 +190,7 @@ private:
using Errors = std::deque<ErrorInfo>;
bool readToken(Token& token);
bool readTokenSkippingComments(Token& token);
void skipSpaces();
bool match(const Char* pattern, int patternLength);
bool readComment();
@@ -221,7 +222,6 @@ private:
int& column) const;
String getLocationLineAndColumn(Location location) const;
void addComment(Location begin, Location end, CommentPlacement placement);
void skipCommentTokens(Token& token);
static bool containsNewLine(Location begin, Location end);
static String normalizeEOL(Location begin, Location end);