Here are some changes in the java grammar that I would like you to take a look at:

SourceFile

The ASTs in xml format start with the SourceFile tag. Even though it's not really part of the language's grammar, it would be useful to have it to match the xml's structure.

Mandatory/Optional

Some tags were wrongly marked as mandatory based on the jhotdraw ASTs. I updated the attributes to match (I was working on it when the mandatory attribute was still there but I switched to using optionals). Here's a non-exhaustive list of fixes and their reason (if some are missing and you're not convinced, I'll provide an example on request).

• TryStatement
  • catchClauses not mandatory - PrintFileAction ID="234"
• ArrayCreation
  • dimensions not mandatory - ToggleViewPropertyAction ID="856"
• CompilationUnit
  • imports not mandatory - AbstractHarmonicRule ID="1"
• Initializer
  • modifiers not mandatory - JColorWheel ID="3492"
• InfixExpression
  • missing this, makes leftoperand not mandatory - AbstractDrawing ID="301"
  • missing this, makes rightoperand not mandatory - AbstractApplication ID="4820"
• ConditionalExpression
  • missing this, makes thenExpression not mandatory - AbstractFigure ID="2005"
• FieldDeclaration
  • modifiers not mandatory - GeneralPathTip ID="401
• MethodDeclaration
  • modifiers not mandatory - DefaultDrawingViewTransferHandler ID="80"
• ForStatement
  • updaters not mandatory - MDIApplication.java ID="712"
  • body not mandatory - TextAreaFigure ID="4542"
  • initializers not mandatory (because no EmptyExpression in AST) - BezierPath ID="6858"
• TypeDeclaration
  • bodyDeclarations not mandatory - SelectAreaTracker ID="6
  • modifiers not mandatory - Polygon2D ID="15"
• EnhancedForStatement
  • expression not mandatory (because 'this' missing in AST) - BezierPath ID="8416"
  • body not mandatory - similar to ForStatement
• SuperConstructorInvocation
  • arguments not mandatory - PaletteTabbedPanelUI ID="290"
• WhileStatement
  • body not mandatory - StreamPosTokenizer ID="1396"`
• ...

IExtendedModifier

I added <Annotation/> as a possible IExtendedModifier as explained here

Abstract tags

Some tags such as Operator appear in the ASTs as Strings (e.g. <operator>+</operator>). To respect the grammar, it should be <operator><PLUS_ASSIGN/></operator>. To solve this issue temporarily I added a image attribute to be able to map those strings to proper tags but, ideally, it should be generated as actual xml tags (unless there's a good reason why it isn't).

Other

Also, some other related remarks that are not part of the commit but I'd like your input on it:

NullLiteral

The NullLiteral tag should have one child token with a simplevalue of String. This is not the case in the ASTs were you find things like: <NullLiteral>null</NullLiteral> which doesn't respect the grammar. This should be changed in the ASTs generator (unless you already changed it in the meantime).

ThisExpression

I noticed the ThisExpression tag is missing in the ASTs. This has an impact on the grammar (optional/mandatory). Some examples are included in the list above. Is this normal? I know for example that EmptyStatement tags are excluded on purpose but not sure why ThisExpression would also be.