Custom Language Aspects
MPS 3.3 aims at allowing language designers to implement and use their own language aspects, such as custom editors or type-system.
Several new features make it already possible to define new language aspects:
- runtime class instantiation in the language descriptor (generated Language.class)
- customizable icons for the aspect models
- adjustment of the "new root" menu
- integration of a few minor features
It still has to be implemented:Planned:
- a better DSL for describing the language aspects
- support in the smodel language in order to work with language aspects, languages and modules programatically
Typesystem: Overriding default type node
Allows to substitute another node to represent the type instead of the node originally used as a type.
- Substitute Type Rule applied to original type nodes, returns alternative type representation;
- the same rule applied to an attribute of the original type node enables aspect-oriented way of altering the default type representation.
More in the Typesystem documentation.
No Used Languages in modules
The need for importing languages twice - both to model, where the language is used, and to the model's module, is gone. Modules now derive used languages from the models that the module owns. It is no longer possible to add/remove languages in the 'Used Languages' tab in module properties.
Attribute support in TextGen and Generator
A prototype of the feature comes as part of EAP2. It can be considered to be in a 'Proof of concept+' state. Node Attributes are handled both in TextGen and Generator. In TextGen, an attribute on a node changes the control flow, and a TextGen component of an attribute takes precedence over that of the attributed node, pretty much like it's done in the
editor aspect. There's a new 'attributed node' append part, available in the textgen components of NodeAttribute, to delegate to the component of the attributed node. Unlike TextGen, Generator merely preserves particular node attributes during the transformation process - attributes do not affect the control flow of the process.
For migration purposes, attribute processing could be switched off in the Generator settings (the TextGen section, 'Enable node attributes') in case there's code that processes node attributes (particularly in TextGen) manually. Note, however, that MPS itself now depends on attributes, as the JavaDoc tags are implemented as NoteAttributes and thus the generation of Java code relies on these attributes to get comments generated (you might face missing javadoc attributes if you turn this option off). Attribute processing in TextGen is deemed complete.
For node attributes to survive the generation process, it's vital for them to implement a marker interface
PersistGeneration. We aim to lift the restriction later, when 'Phase 2' comes to life, with sophisticated handling of attribute meta-information. Processing of node attributes during generation is guarded by aforementioned setting as well. The present state seems sufficient to give the feature a try and to confirm that the desires behind the feature and its implementation are on the same page.
There's sample project you might find handy,
'jetbrains.mps.samples.attribute', to show few simple scenarios.
In addition to node attribute handling, TextGen component has been refactored significantly. It's now part of the general language aspect loading mechanism (reflective access is supported for compatibility with TextGen aspects generated in MPS 3.2, it's highly recommended to re-build your
textgen aspect model, nevertheless). Besides changes in node attribute handling (see above for details, and also https://youtrack.jetbrains.com/issue/MPS-21033), there's a change in the convention how files emerge from a model. Unless your language extends concepts of another language, and relies on the TextGen of the extended language, there's nothing to worry about. MPS used to create a file for each root concept, including sub-concepts, of a language. Now, only exact concept matches are considered, and if an extending concept desires to re-use textgen component of an ancestor, it shall declare its own, empty TextGen component, stating essentials as the file name, encoding and extension, and leaving the body of the component empty. For details, see https://youtrack.jetbrains.com/issue/MPS-22220. It is a part of an ongoing process to separate the text generation control from the file management.
Generic support for commenting out nodes
MPS now provides a universal way to comment out nodes in models. In previous versions this functionality had to be implemented in all languages separately, either through node attributes or dedicated "comment" nodes. In 3.3 the information about a node being commented out is stored in the model in a generic way. The smodel language ignores commented out nodes by default so that your queries do not have to filter out commented out nodes explicitly. Additionally, actions have been created to comment/uncomment out a node.
Using annotations to override inference rules for expressions/literals
Type Substitution Annotations have been introduced in order to reduce the boilerplate required in order to implement type substitutions. Types are represented in code as nodes, now with optional substitute annotations. The annotations are implemented as node attributes. The annotations help transform the type when introduced to the type-checking engine at runtime, much like the editor uses attributes to provide alternative cell editors. There is now also support for overriding the type inference rules for literals or expressions that have specific annotations. The inference rule activated by an annotation has greater priority than the default one defined for the node itself.
Intention and script aspects plug in as generated language aspects now
This is a change that you are unlikely to notice, still it is worth a good mention. These aspects used to have been pulled in into MPS through java reflection, while now they are registered into the runtime with the same general mechanism that most of the other language aspects use, i.e. part of a generated language runtime class. Though we keep backward compatibility with code generated in MPS 3.2, it's recommended to rebuild
script aspects of your languages before attempting to use the functionality they provide.
The newly added jetbrains.mps.baselanguage.lightweightdsl language enables internal DSLs to be embedded inside BaseLanguage classes. Internal DSLs in general are easier and faster to develop than full-blown external DSLs, they typically reuse the syntax of the host language and tightly integrate with the surrounding non-DSL code. Similarly, lightweight DSLs in MPS can be created by defining a single node and then weaving the node into a BaseLanguage ClassConcept or its subconcepts.
MPS itself leverages this mechanism in several places:
- MigrationScript concept, which is a mere BaseLanguage class, is enhanced by the Migration DSLDescriptor that adds a few extra properties, members and custom members
- Custom language aspects can be described with CustomLanguageDescriptors, which uses the LanguageAspect DSLDescriptor to enforce proper aspect definition
- The extension mechanism in jetbrains.mps.lang.extension uses the Extension DSLDescriptor to enhance the Extension concept
You can have your lightweight DSL weaved into a plain BaseLanguage class or into your own concept extending ClassConcept. Check out the Lightweight DSL documentation.
New editor cell - Next applicable editor
A new editor cell has been introduced that simplifies the task of language designers when they need to customize the editors of existing languages.
The extending language may wrap the editors of existing concepts with additional cells and then have the next applicable editor cell reapply the logic for finding the applicable editor. The editor will be inserted in the place of the next applicable editor cell.
Generic way to customize appearance of commented out code
In EAP2 MPS added a generic support for commenting out nodes. The feature is fully described at the Commenting out nodes documentation page. In EAP3 the editor definition language contains a new next applicable editor cell, which can, among other things, greatly simplify customizing the appearance of nodes that are commented out. You no longer have to clone the default editors for all concepts and bind these clones to the comment editor hint. Instead you define a single editor for a common (abstract) super-concept, bind it to the comment editor hint and then use the next applicable editor cell to remove the comment hint from the context and to find and insert the original editor.
Custom language aspects
Using “console” commands in migrations and other code
Since EAP3, it is possible to use the Console query expressions anywhere, which is especially convenient for writing migrations. You need to add the jetbrains.mps.lang.smodel.query language to your model, create a with-statement and use queries like #instances, #models inside of it.
A good use case, for example, is in the lightweightdsl’s migration model to ease retrieval of applicable nodes:
For comparison, you can see both styles of writing the migration - the old and the new one.
BaseLanguage compile-time expression value assistance
BaseLanguage has been enhanced in order to understand compile-time values of expressions. Intentions now offer the user options to simplify expressions under caret. Additionally, boolean expressions used as conditions that evaluate to either true or false are reported to the user. Check out a short video showing BaseLanguage expression compile-time simplification.
Collapse by default
It is possible to specify the default state for foldable collections of editor cells. The collapse by default property value can be true/false/or a custom query.
Paste from History
You can now use Paste from History action in the projectional editor (Edit -> Paste from History, or Ctrl/Cmd+Shift+V). It lets you paste an entry from the list of recent Clipboard entries.
Generating images of MPS nodes
Using the jetbrains.mps.lang.editor.imageGen language it is possible to create images of MPS nodes as a part of model build process. Images can be generated from any custom DSL extension if jetbrains.mps.lang.editor.imageGen is used as generation target.
Show reflective editor
Right click on any node (including diff) and choose "Show Reflective Editor". Reflective editor will be shown.
To show regular editor right click on the same node and choose "Show Regular Editor". Or just press F5.
Compare Two Nodes
You can select two nodes in the project tree and compare their structure with "Compare Two Nodes" action:
Nodes can be in the same or in different models. If you have "show node structure" on you can select not only root nodes.
The action compares only structure of the nodes. Node ids are ignored. In the dialog you can see difference and if necessary apply some of the changes to the left or right node:
Generator: support anchors in weavings rules
Weaving rules used to append weaved nodes to the end of context node children list. Now, with an extra query, one can specify an anchor node to follow a newly inserted node (thus, null anchor value indicates tail of the children list, compatible with the old behavior).
Generator: preview for another cross-model generation approach
With $EXPORT$ labels cross-model generation turned out to be quite tiresome and full of boilerplate activities, and we turned back to the idea of transient models being preserved during generation. We don't save all of them, though, there's notion of checkpoint step in the transformation sequence, where actual transient model is captured for later use. Regular label mappings ($LABEL$ and alike) now consider these checkpoint models for possible output, thus facilitating generation of two models with cross-references independently. Support is primarily in Generator internals to publish, keep, and access these checkpoint models. Simple UI has been added to showcase potential use of the new generation alternative. There's new module facet, named 'Custom Generation', which allows you to pick a model with generation plan (instances of jetbrains.mps.lang.generator.plan language). With the plan associated, any model within the module is generated according to the plan specified, and thus can share checkpoint information.
Checkpoint models are not persisted and are intended for use throughout MPS session, with manual control to drop them as appropriate.
Note, this is a prototype of a new feature and will change as we move forward.
TextGen: provisional mechanism to control layout of output files
Text generation is not always possible in a sequence that corresponds to lines in a physical file. E.g. for a Java source, one could distinguish 2 distinct areas, e.g. imports and class body, where imports is populated along with the body. Passionate language designer might want to break up the file further, e.g. up to file comment, package statement, imports, and class body that consists of fields and methods, and populate each one independently while traversing a ClassConcept. That's what we call a layout of an output file, and that's what we give control over now. MPS veterans might be aware of two buffers (TOP and BOTTOM) that used to be available in TextGen for years. These were predefined, hard-coded values. Now it's up to language designer to designate areas of an output file and their order.
Note, distinct areas come handy especially when generating text from attributes, as they change order of execution. With them, it's even more tricky to make sure flow of text gen corresponds to physical text lines, and designated areas make generation a lot more comfortable.
Layout of the file could be specified for a top text gen, the one that produces files.
Again, support for this mechanism is preliminary and is quite rudimentary now. We utilize it in our BaseLanguage implementation, so this notice is likely to explain you what's going on rather than encourage you to put this into production.