Table of Contents

Overview

IntelliLang is a combination of three basic kinds of functionality that are meant to support the developer in dealing with certain tasks that relate to (custom) languages in IntelliJ IDEA:

• "Language Injection": Editing code that is embedded into String literals and XML fragments where it would be great to get the same kind of IDE support like if the code was being edited "natively"
• "Pattern Validation": Provides assistance in making sure that Strings being passed to and from methods match a particular regular expression
• "Regular Expression Support": A custom language implementation for regular expressions

Following an old tradition, IntelliLang and Regular Expression Support have been inspired by an open IntelliJ IDEA feature request: IDEADEV-2599

Language Injection

This makes use of the new possibilities IntelliJ IDEA 6.0 to treat String literals, XML text and attributes as fragments of an arbitrary language (called "Language Injection"). The plugin makes this newly introduced API readily available to everybody for their daily use through two very simple means: Either by using some Java annotations to mark String fields, local variables, method parameters and methods returning Strings as containing a certain language, or by just using a simple UI configuration. There is a set of annotations provided by the plugin, but the actual annotations are freely configurable to avoid any unwanted dependencies.

This enables the developer to get the benefit of a wide range of edit-time features, such as syntax error highlighting, completion, inspections and a lot more while editing fragments of e.g. JavaScript inside regular Java code or in XML files of custom schemas that IntelliJ IDEA usually doesn't know about.

Pattern Validation

Additionally, the plugin allows to annotate Java elements of type String to have them checked for compliance with certain Regular Expressions. This can be useful for very simple "languages" where the developer needs to make sure that an expression conforms to a certain syntax, e.g. that a String is a legal Java identifier or a valid printf-like pattern used by java.util.Formatter.

This can both be validated on-the-fly while editing the code as well as during runtime (method parameters and return value only, like the @NotNull instrumentation of IDEA core) by instrumenting the compiled classes with assertions that match the value against the supplied pattern.

Regular Expression Support

This part of the plugin implements language-support for java.util.regex.Pattern and has been mainly created to support the IntelliLang plugin by adding support for the micro-language that is probably one of the most often used one inside Strings. It features complete support for the syntax of the JDK's regular expression implementation and adds some further features, such as

• Completion and validation for character property names (e.g. \p{javaJavaIdentifierStart}) which nobody can usually remember anyway
• Validation and navigation for the use of back-references (e.g. \1), e.g ctrl-b navigates to the capturing group the backref refers to.
• Intention Actions to simplify usages of repeated character occurrences, e.g. a{0,1} is offered to be converted to a?
• "Surround With" capturing/non-capturing group
• and more

General Usage

The plugin's usage is simple and straightforward, yet very flexible. Either add the provided set of annotations to your project and start using them, configure the plugin to use a custom set of annotations or simply use the UI configuration to make IDEA learn that e.g. the String argument of Pattern.compile() should be treated as a regular expression.

Using the Annotations

IntelliLang makes use of three base-annotations: @Language, @Pattern and @Subst

• @org.intellij.lang.annotations.Language is responsible for the Language injection feature
• @org.intellij.lang.annotations.Pattern is used to validate Strings against a certain regular expression pattern
• @org.intellij.lang.annotations.Subst is used to substitute non-compile time constant expressions with a fixed value. This allows to validate patterns and build the prefix/suffix of an injected language (see below) even for non-constant expressions that are known to contain certain kinds of values during runtime.

The annotations supplied with IntelliLang are located in the file annotations.jar which can be found in %IDEA-CONFIG%/plugins/IntelliLang/lib.

@Language

The @Language annotation can be used to annotate String fields, local variables, method parameters and methods returning Strings. This will cause String-literals that are assigned to a field/variable, passed as a parameter or are used as a method's return value to be interpreted as the specified language.

Additionally, there's the Language Mismatch inspection which checks for clashes between the "expected language" and the "actual language" when a field/variable is assigned or a value returned from a method.

The plugin supports "direct" and "indirect" annotations, i.e. you can either directly use the annotation like this:

or annotate another annotation class like this:

which can then simply be used to annotate elements as @XPath.

It's very easy to obtain the language-id that must be supplied as the annotation's value attribute: IntelliLang provides the list of available language via the regular code-completion action. Just select the appropriate language from the ctrl-space popup:

@Pattern

The @Pattern annotation is responsible for marking Strings that have to comply with a certain regular expression and can be used in just the same way as the @Language annotation. That means, it's possible to create derived annotations, such as an @Number annotation that requires a String to consist of one or more digits:

In fact, the predefined annotations already contain two of such derived annotations: The first one, @PrintFormat, matches the printf-like pattern used by java.util.Formatter and another one, @Identifier, describes a valid Java identifier. These are ready to use without having to specify any additional pattern.

@Subst

The @Subst annotation is used to substitute references that are not compile-time constant which enables the plugin to do Pattern Validation based on the assumption that the substituted value is compatible with the values that are expected during runtime. The plugin will complain if the value does not match the expected pattern.

It also helps to build a valid context of prefix/suffix (see next section) for the Language Injection feature. Consider this example:

Without substituting the value of the variable font with the value "Tahoma" (actually it could just be a single character here), the injected fragment would be syntactically incorrect, causing the error "a term expected" to be displayed after the "font:" instruction.

Supplying Context: Prefix and Suffix

When annotating an element, it's possible to supply a prefix and a suffix that should be prepended/appended when the language fragment is being parsed. This can be used to supply context information, i.e. if the prefix for a JavaScript injection is "var someContextVariable;", IDEA will know that the variable "someContextVariable" is declared and will not warn about it being undeclared when it's used.

Apart from the manual possibility to supply a prefix and suffix, IntelliLang dynamically determines those values from the context a String literal is being used in:

In this example, the JavaScript language will be injected into each of the three String literals, and each one's prefix and suffix will be calculated from the preceding and following expressions so that the resulting text that's being parsed is valid JavaScript syntax:

• no "missing '}'" error will be displayed: The closing brace is part of the first literal's suffix.
• the variable x that is used in doSomethingElse(a); will be declared: Its declaration is part of the second literal's prefix
• the function doSomethingElse() will be known as well: It's defined in the statically supplied prefix

Configuration

The configuration dialog provides the following options:

• Language injection for XML text
• Language injection for XML attributes
• Language injection for method parameters
• Advanced settings that allows to change the names of the recognized base-annotations and the way how the pattern-validation is performed during runtime.

Screenshots

The screenshots below show come of the configuration options that can be tweaked and will be described in the following sections.

Language Injection

This tab allows to configure the language injection feature for XML text, attributes and method parameters. Use the buttons in the toolbar or the context menu actions to add, remove, copy or import new entries. To add a new entry, the appropriate group has to be selected first.

Reordering the entries is possible with the Move Up and Move Down buttons. This can be important to define the precedence of different XML-injection entries. If an entry matches, no more injections are applied unless the injection specifies a value pattern.

XML Text

After adding a new XML text injection, select the ID of the Language to inject and optionally specify the prefix/suffix that make up the injection's context.

In the "XML Tag" panel, specify the local name (i.e. the name without any namespace prefix) and the namespace URI of the XML tag surrounding the text that should be treated as the selected language. The "name" field should not be empty, however the "namespace" field is optional.

The "Local Name" field takes a regular expression which makes it is possible to specify e.g. multiple tag names (name1|name2), case-insensitive names (e.g. (?i)tagname matches "tagname" as well as "TagName"), etc. Be sure not to enter any whitespace characters as they would be significant for the match.

XML Attributes

This is similar to the XML text configuration. However, it is possible to leave the "name" of the "XML Tag" empty which means that the configuration will apply to any attribute that matches the configured name, regardless of its containing XML tag.

The attribute's name takes the local name of the attribute and is also specified as a regular expression. This can be e.g. used to match e.g. HTML event handler attributes by specifying the name "on.*". The attribute name may also be empty (unless the tag name is empty as well) which means that the configuration applies to all attributes of the containing tag.

Advanced XML Options

The "Advanced" panel for XML Text and Attributes allows an even more fine-grained control over the injection process.

Value Pattern

This field takes a regular expression that determines what part of the XML text's or attribute's value the language should be injected into. This can be used to inject the language only into values that match a certain pattern or to inject it into multiple parts that match the pattern. This is done by using the first capturing group of the pattern as the target for the injection.

Examples:

[$#]\{(.*?)\}

This matches the pattern used by the JSP/JSF Expression Language. Injecting Expression Language (EL) doesn't work currently, but will be available with IntelliJ IDEA 6.0.2.

^javascript:(.*)

This matches the "javascript"-protocol that can be used in hyperlink-hrefs to execute JS code.

XPath Condition

This field takes an XPath expression that can be used to address the injection-target more precisely than just by supplying its name and namespace URI. The context the expression is evaluated in is the surrounding XML tag for XML Text injection and the attribute itself for XML Attribute injection.

Example:

lower-case(@type)='text/javascript'

This limits the injection to tags whose type attribute contains the value "text/javascript".

Method Parameters

This is a possibility to make use of IntelliLang's features, if, for any reason, the injection annotations cannot be used. This mainly applies to configuring 3rd party/library methods as well as projects that still have to use Java 1.4.

The language selection is identical to the one described above. To select one or more parameters of a certain method, first choose its containing class either by typing in the name (the textfield supports completion) or by using the class chooser available through the [...] button. Next, select a method using the [...] button inside the "Method-Name" panel (it's not possible to edit the method name manually). Once a method has been chosen, the table in the "Parameters" panel is populated with the parameters of the selected method. Select the checkbox in the first column to have the selected language injected into arguments passed for this parameter. Note that only parameters of type String can be selected.

Importing Configuration Entries

To import the injection configuration from another IDEA installation use the "Import" button from the toolbar and select the file "IntelliLang.xml" from %IDEA-CONFIG-HOME%/options or from a JAR file that contains some exported (via File | Export Settings) IDEA settings. After selecting the file, a new dialog displays the entries contained in that file. Use the Delete-button to remove all entries you don't want to import. Note that this will not change the selected configuration file.

This selective import-feature makes it easy to share certain configurations in a team without losing any local entries like when importing settings via the core File | Import Settings action.

Advanced Settings

The advanced configuration tab allows to specify different names for the base-annotations that should be used. This helps to avoid any dependencies on foreign code where this is not desired or possible. The custom annotations should just provide the same properties as the original ones, i.e. value for all of them and an optional (default = "") prefix and suffix for the @Language replacement.

Here it's also possible to configure the kind of runtime checks the plugin should generate for the @Pattern validation:

• No instrumentation: No checks will be inserted and doesn't touch any compiled class files
• Instrumentation using assert: Pattern-validation is controlled with the -ea JVM switch and throws AssertionError. This is the recommended way due to the potentially negative impact on the performance for methods that are invoked very often.
• Instrumentation using IllegalArgument- (for method parameters) and IllegalStateExceptions (for return values). This is the same the @NotNull instrumentation of IntelliJ IDEA does.

Contributed Configurations

Additional configurations can be downloaded and imported from here. This community effect can help to minimize the configuration efforts and make IntelliLang even easier to use.

Code Inspections

IntelliLang contains a set of inspections that validate the correct use of the supplied annotations (or custom configured ones). These inspections can be configured through the regular Settings | Errors configuration dialog.

Language Injection

The inspections in this category apply to the language injection feature related to the @Language annotation

Unknown Language ID

This inspection provides validation for the use of non-existing Language-IDs. It flags usages of incorrect values of the @Language's value attribute, such as @Language("NonExistingID")

Language Mismatch

Validation for using references to elements that are annotated as containing different languages or are not annotated at all. The inspection offers a QuickFix to annotate such elements with the right annotation for the expected language.

Injection not applicable

This inspection checks whether an @Language or any derived annotation is used for anything other than elements of type String or String[].

Pattern Validation

This category contains inspections about validating the use of the @Pattern or its derived annotations.

Validate Annotated Patterns

This inspection validates that expressions (String literals, as well as other compile-time constant or substituted expressions) match the pattern required by the @Pattern annotation. The inspection has an option to ignore non-constant expressions that contain non-substituted references and offers a QuickFix to add a substitution where applicable.

Pattern Annotation not applicable

Checks whether a pattern-validation annotation (@Pattern or derived ones) is valid to be applied to the annotated element. Only elements of type String may be annotated.

Non-annotated Method overrides @Pattern Method

This inspection checks whether a method without any @Pattern or derived annotation overrides an annotated method from its base classes. This is not necessary for the error-highlighting inside the editor, however the runtime-check instrumentation doesn't pick up annotations from base-class methods.

A QuickFix is provided to add an annotation that matches the one from the base-class method. This ensures that the runtime-check instrumentation works correctly.

Quick Edit Language

This is an experimental feature that can be extremely helpful, especially for editing Regular Expression patterns inside String literals due to the "double escaping" requirement. For example, in a regular expression, a literal backslash character has to be written as a double backslash and each of them has to be escaped with another backslash when written inside a String literal.

The Quick Edit function that appears as an Intention Action for any injected language fragment displays a popup dialog that allows to edit the string's value without the double escaping requirement. The dialog also shows the particular prefix/suffix of the fragment in a non-editable area.

The popup can be dismissed by pressing Escape or clicking somewhere outside the popup. Any changes made in the popup are committed by pressing Ctrl-Enter.

Please note that the formatting of the non-editable prefix/suffix may differ from the actual value due to the problems described here. However, even though the formatting/alignment of the editable text may differ from the expected text, making changes to the text still works as expected.

Usage Examples

Among the rather simple use-cases, like having detailed syntax checks for all kinds of "micro-languages" that are used e.g. in Pattern.compile(), XPath.compile() and so on, here are some less obvious, yet very useful examples how IntelliLang can leverage IDEA's support for a much better coding assistance.

The new scripting support in the upcoming Java 1.6 is another case where it will be important to get as much as possible edit-time assistance when script-code is constructed from Java code.

Extended JavaScript Support

When dealing with JavaScript that's not directly embedded inside an HTML page, IDEA usually just treats it as plain text. Consider the following example that creates an HTML page from an XSLT script. Without the JavaScript language being injected into the "script" tag with the XHTML namespace as shown in the screenshot below, this would be treated as plain text, with no further code assistance.

Support for JSP Custom Tags

With IntelliLang it's also possible to make the content and attributes of custom JSP tags being treated as another language. This can be useful e.g. for server-side scripting using JavaScript or any other Language implementation available for IntelliJ IDEA.

One thing that's important to know is that the taglib's URI which supplies a custom tag should be used as the namespace URI of the XML tag to inject a language into. The namespace-textfields contain a list of all known taglib URIs in the project.

Pattern Validation

Here's an obvious example right from IntelliJ IDEA's OpenAPI:

The contract of the method getID() is that it should only return strings that match the pattern "[a-zA-Z_0-9]+". The short note in the JavaDoc can be easily overlooked though because the contract isn't specified in an automatically verifiable way.

However, if this method were annotated as @Pattern("[a-zA-Z_0-9]+"), any attempt to return a string that doesn't match that pattern would be flagged in the editor:

Pattern Completion

If a regular expression pattern represents an enumeration of different literal values, the plugin offers completion for those values:

Regular Expression Editing

Here are some examples of the enhanced coding support for regular expression patterns:

Backref Validation

Surround With

Character Category Validation

Character Category Completion

Availability

The plugin is available for IntelliJ IDEA 6.0 via the built-in plugin manager and at the IntelliJ Plugin Repository.

Known Issues

Unfortunately, there's a number of issues that can cause problems, have a negative impact on the plugin's stability or just reduce the possible usefulness of the plugin:

• IDEADEV-11227: No refactoring and navigation for language fragments injected into JSP custom tags
• IDEADEV-11212: Formatting of Injected Languages
• IDEADEV-11226: Completion in injection language fragment causes NPE
• IDEADEV-11232: Injecting a Language into partial content of XML CDATA section fails
• Comment on IDEADEV-8302: Why glued tokens are bad
• IDEA-9573: Injecting CSS language needs a special prefix and suffix to offer proper completion and error highlighting. It's required to configure the prefix "dummy_selector {" and suffix "}" to make this work for "raw" CSS e.g. in HTML "style"-Attributes.
• If an injected fragment has a suffix (statically or dynamically) configured, it can happen that the highlighting of syntax errors is not visible in the editor. This is due to the way the plugin tries to avoid the "glued token" problem by adding a space character to the start of the suffix which can cause the syntax error to be disappear in the non-visible part of the fragment. The error stripe however will show that there is an error which can be further examined by using the Quick Edit Language feature.
• The Injection API does not allow to inject a language into Javadoc comments. This would have been great to support EJB-QL in XDoclet comments.

You can vote for and watch the open issues to get notified about their progress.

History

Version 0.8 (2006-10-03)

• Initial version

Version 0.9 (2006-10-09)

• Completion for values of simple enumeration-like patterns
• Ability to specify pattern for language injection in XML text/attribute values
• Ability to use boolean XPath expression to select XML tag/attribute for injection (requires the XPathView + XSLT-Support plugin to be installed)
• Ability to import/export plugin settings

Version 1.0 (2006-10-18)

• Language Injection is supported for String-arrays
• Added language-icon to UI configuration and language-id completion
• New inspection: Non-annotated Method overrides @Pattern Method
• Added ability to reorder configured injection entries
• Minor bugs and cosmetic issues fixed

Version 1.1 (2006-10-26)

• XML tag names may be specified as regular expression
• Small UI usability improvements (added missing labels, mnemonics, shift-tab navigation)
• Bugfixes, refactoring, cosmetic changes

Version 1.2 (2006-10-31)

• Fixed incorrect possibility to inject language into Attribute Values with embedded content, e.g. JavaScript in (X)HTML
• Cleaned up some UI code
• Integrated with IDEA's help-system

Version 1.2.1 (2006-11-12)

• Fixed severe performance problem in Regular Expression Parser

Version 1.2.2 (2006-12-13)

• Fixed an ugly bug in runtime pattern-validation instrumentation

Version 1.2.3 (2007-01-23)

• Fixed incorrect pattern-instrumentation for constructor parameters of non-static inner classes

Version 2.0 (2007-01-23)

• First Selena-compatible release (no major changes in functionality)

Version 2.0.2 (2007-02-12)

• Updated for Selena-API changes (no major changes in functionality)

Version 2.0.3 (2007-05-10)

• Updated for Selena-API changes (no major changes in functionality)

Version 2.1 (2007-05-25)

• API update
• Added "Inject Language" intention to temporarily inject a language into an arbitrary place
• "Quick Edit Language" editor strips leading and trailing whitespace.
• Configuration dialog remembers its layout again
• Fixed a potential performance problem in config dialog

Version 2.1.1 (2007-06-08)

• Updated "until-build" number

Version 2.1.3 (2007-08-15)

• Don't flag \] and \} as redundant escapes
• "Remove Redundant Escape" QuickFix screwed up code
• Adapted to ASM v3 now used in IDEA

Version 2.1.4 (2007-08-31)

• API update. "Quick Edit Language" had to be disabled for now

Version 2.1.5 (2007-09-19)

• API update.

Feedback

I'm curious to hear if people find the plugin and the idea behind it useful and will be happy about any comments, critics and suggestions. Just leave a comment here, post in the plugin newsgroup/forum or just send me an email.