The Grit parser has taken a big step forward this week. I have been using it for quite a while, but only on little grammars, but now I am in the deep end with the asciimath grammar. I got myself into a big mess, and I could not see what was going wrong. That forced me to sort out some good debug tools.
I needed to see all the gory details since I was not sure if the problem was my grammar rules, or perhaps a bug in Grit itself. The problem with a low level trace is the huge volume of information, it was just too much to wade through.
I tried a number of other tools that seem to be useful. They can print out the result of a rule match and track the current position. But for my big mess I really wanted to know exactly what was going on.
So I came up with a low level trace crunched into a compact format that allowed me to see all the backtracking and rule matching progress. I was then able to quickly sort out my mess, even though it was a huge number of rules with tons of backtracking.
I thought I might need selective rule trace, but the trace is compact enough that you should be able to simply turn it on for the whole grammar, and let it rip.
Below is an example of what a trace looks like, it is a small sample using the asciimath grammar. The details don't matter, but everything the parser did can be seen. On the left is the progress through the input with each match, and on the right is a trace of all the rules that were called to reach the match. The ! marks are failure back-track points, and they are edited out before the next match report.
I had to use a narrow format for this blog, but on a bigger screen this trace would only need one line per match.
0..0 ws: math E ws
0..2 constant: pi math E S special underover uoop
! ! derivative ! matrix l ! ! !
quoted ! bracketed l ! ! text !
unary uniop ! ! binary bop ! ! constant
0..2 S: pi math E S
2..3 ws: math E fraction '//' ! '/' ! !
subsup '_' ! '^' ! ! ws
3..4 constant: r math E S special underover uoop
! ! derivative ! matrix l ! ! !
quoted ! bracketed l ! ! text !
unary uniop ! ! binary bop ! ! constant
3..4 S: r math E S
4..5 '^': ^ math E fraction '//' ! '/' ! !
subsup '_' ! '^'
5..6 constant: 2 math E subsup S special
underover uoop ! ! derivative !
matrix l ! ! ! quoted !
bracketed l ! ! text ! unary
uniop ! ! binary bop ! ! constant
5..6 S: 2 math E subsup S
4..6 subsup: ^2 math E subsup
6..6 ws: math E ws
0..6 E: pi r^2 math E S special underover uoop
! ! derivative ! matrix l ! ! !
quoted ! bracketed l ! ! text !
unary uniop ! ! binary bop ! !
constant ! !
6..6 ws: math E ws
0..6 math: pi r^2 math E S special underover uoop
! ! derivative ! matrix l ! ! !
quoted ! bracketed l ! ! text !
unary uniop ! ! binary bop ! !
constant ! ! ! other !
Of course you only need the trace when you get stuck and need the nitty gritty details. Most of the time you can just use the parse tree.
The parse tree result for this previous trace is shown next. The list structure details are preserved here to help you write the code needed to translate these rule results. The * marks a list, and the multiple matched strings are the result of a regular-expression rule with multiple groups:
math
*
E
*
ws ""
S
constant "pi" "" "pi" "" "" "" "" "" "" "" ""
*
ws " "
S
constant "r" "" "" "" "" "" "" "" "" "" "r"
*
subsup
'^' "^"
S
constant "2" "2" "" "" "" "" "" "" "" "" ""