Child pages
  • Fast Track to MPS
Skip to end of metadata
Go to start of metadata

Welcome! This tutorial was designed specifically for developers, who are completely new to MPS and prefer a guided tour through the MPS landscape. You will walk the beaten path one step at a time, following clear marks that show you where to go next. The information is structured so that you progress from simpler concepts to the more involved ones and at the end of the journey you'll understand MPS and will be able to use it effectively on your projects.

Icon

Hi, my name is Václav and I've created this tutorial for you. The tutorial is a living creature that I constantly maintain and evolve to make it better serve its purpose - to help you learn how to design languages with MPS.

You can help me on that mission - if you spot a mistake, hit a road-block or become confused - let me know. I'll be very grateful, if you leave a comment or file a YouTrack issue for me. Thank you.

 

We admit that there are easier tasks in the world than learning MPS. Language design is a complex domain and projectional editing needs some getting used to. It helps a lot if you have prior experience with DSLs or language workbenches, but it is not necessary for understanding this tutorial. Just expect more bumps on the road. There certainly will be a lot of them - partly also because MPS has its own problems and bugs. So you are in for a tough experience that may take quite a big amount of your time. 

Bear in mind that many have walked through here before and succeeded. If you persist you'll be rewarded - you'll become a language designer who can prepare custom-tailored DSLs, integrate them into projects and develop high-value software with them.

So, take on good shoes, lace them tight and start going ...

Ten steps to mastery

Step 1 - Install MPS

Plain reading about MPS will never teach you as much as trying it for real yourself. You might have done this already, but if not, grab MPS from the download site and install it (You may ignore the IntelliJ IDEA MPS plugin for now).

With MPS installed you will be able to continue on the Fast Track.

Icon

While downloading you may check out a two-minute overview video about MPS - Why JetBrains MPS

Step 2 - MPS from the programmer's perspective

Raw time estimate - 45 minutes

Before you continue, you should probably get to know the most distinct feature of MPS - the projectional editor. Check out a video that explains all the differences between traditional textual editors and the MPS projectional editor and lists the numerous benefits of this approach.

Since we programmers like trying things, how about taking the MPS editor for a simple test ride first? You may get the first impressions instantly, if you start MPS and open the Robot Kaja sample project. This project defines an artificial language to command a virtual robot. The robot can move within a rectangular area, drop and collect marks and check the surrounding cells for walls and marks. You write Scripts using the built-in commands as well as new commands that you create on the fly. When you run the script a simulator written in Java is started to visualize the execution of your program.

To open the project, start your fresh MPS, click on the Open Sample Project button and pick robot_Kaja. The sample projects have been installed into your HOME/MPSSamples folder.

                                                                                                                              

We also shot an instructional screen-cast to help you start. Check it out. 

Icon

Do not worry to break things while playing with the MPS samples. You can always delete the HOME/MPSSamples folder and MPS will re-create it the next time you click the Open Sample Project button. Experiment, experiment, experiment!

Step 3 - MPS from the language designer's perspective

Raw time estimate - 40 minutes

Now you must be curious about how the robot sample language has been created. Please, open again the Robot Kaja sample project and play our explanatory screen-cast showing you round the language definition fundamentals. As you go through the video, pause frequently and look around the sample project yourself. This way you will understand a typical language structure and see the relationships between its individual parts.

At this point you should understand that language definition consists of several aspects. Some of these are mandatory, some are optional:

  • Structure - Defines the kinds of nodes (called Concepts) that may be used in user models. Each node in the program (model) refers to its concept. Concepts specify, which propertieschildren and references nodes may have. Concepts can extend other Concepts and implement ConceptInterfaces.
  • Constraints - Restricts the relationships between nodes as well as the allowed values for properties beyond the rules defined in Structure. Constraints typicaly define:
    • the target scope for references (a collection of allowed nodes a reference can point to) 
    • situations, in which a node can be a child/parent/ancestor of another node 
    • allowed values for properties 
    • property accessor methods (getters and setters)
  • Behavior - Just like classes in OOP hold methods, Concepts may define methods and static methods that can be invoked on nodes in a polymorphic way. Nodes thus carry behaviour alongside their properties and relationships.
  • Editor - Instead of defining a parser that would translate code from an editable form (i.e. text) into the tree-like structure that a computer could manipulate, MPS offers the concept of projectional editor, which let's the user edit the AST directly. The Editor aspect enables language designers to create a UI for editing their concept concepts.
  • Actions - Since the MPS editor manipulates the underlying AST directly, some editing actions, such as typing certain characters or strings, translate into non-trivial AST changes. For example, typing the plus sign next to a number should insert a PlusExpression, hook its left child to the original number and set the cursor to the empty cell for the right child. 
    The Actions aspect holds definitions of such customised AST transformations.
  • Intentions - All modern IDEs assist developers with instant code manipulating action available under a handy key-shortcut (Alt + Enter in MPS). Language authors can define such little code transformations for their languages in the Intentions aspect.
  • Typesystem - Language that need to type-check their code need to provide type-system rules. The MPS type-system engine will evaluate the rules on-the-fly, calculate types for nodes and report 
    errors, wherever the calculated type differs from the expectations.
    So called checking rules may additionally be defined to verify non-typesystem assertions about the model.
  • Dataflow - The ability to understand the flow of values and the flow of control through language constructs helps languages report issues such as unreachable code or potential null-pointer error. Language designer can leverage the Dataflow aspect to define the flow for each concept, which MPS will then use to calculate the dataflow for the whole program.
  • Generator - Models written in one or more languages get ultimately translated into runnable code in some target general-purpose language and platform, such as Java. Along the way models get gradually transformed so that repeatedly concepts get replaced with concepts from a lower level of abstraction until the bottom-line level is reached. The rules for translating concepts and their proper ordering is defined in the Generator aspect.
  • TextGen - During code generation after the Generator has reached the bottom-line AST representation, the TextGen phase kicks in and translates all nodes in the model into their textual representation and saves the resulting textual source files on disk.
Icon

Before you proceed, please bookmark the MPS Glossary page and come back to it frequently, until you get a good command of the vocabulary that the MPS community uses.

Step 4 - Creating your first language 

Raw time estimate - 3 hours

The next screen-cast to watch, MPS basics - creating your first language, will show you the typical workflow when creating a new language - building the structure, defining the editor and implementing the generator. You will see the creation process of the Constants language, which is also included as a sample project with MPS distributions.

Now you're recommended to watch a series of short high-level videos that will give you a brief introduction into each of the aspects of a language definition:

  1. Introduction to JetBrains MPS, part 1: Projects - This episode covers the MPS project setup and organisation of modules and models including their dependencies.
  2. Introduction to JetBrains MPS, part 2: Structure - This episode provides a brief theoretical background into models, meta-models and abstract syntax trees and then applies the knowledge to the MPS structure aspect.
  3. Introduction to JetBrains MPS, part 3: Constraints - This episode details the constraints aspect of language definition in MPS, mainly how to restrict properties, links and how to define scopes.
  4. Introduction to JetBrains MPS, part 4: Behavior - This episodes adds a few useful tips on adding functionality to concepts and nodes.
  5. Introduction to JetBrains MPS, part 5: Editor - Make'em see your code - defining editors that project the AST on the screen
  6. Introduction to JetBrains MPS, part 6: Actions - Polish the editors - smoothing the editing experience by defining transformations and substitutions
  7. Introduction to JetBrains MPS, part 7: Intentions - Assist the developer with context-sensitive hints and refactorings
  8. Introduction to JetBrains MPS, part 8: Generator - Converting models - defining model-to-model transformations
  9. Introduction to JetBrains MPS, part 9: Text-Gen - Here's what I've written - converting models to text
  10. Introduction to JetBrains MPS, part 10: Dataflow - Go with the flow - defining dataflow definitions so that MPS could automatically detect issues in code structure
  11. Introduction to JetBrains MPS, part 11: Type-system - They are just my type - defining types and type-system rules to validate expressions early-on

You may also watch these as a single YouTube playlist.

Step 5 - Why MPS

Raw time estimate - 30 minutes

Now that you understand the internal workings of MPS you may like to slow down a bit and read a few documents describing how people use MPS. We've collected a few case studies that may inspire you to your own ideas on building something great around MPS. 

Step 6 - Projectional editing tips and tricks

Raw time estimate - 30 minutes

When editing code in MPS you will run into situations when your coding habits interfere with the projectional nature of the MPS editor. I'm actually pretty confident that you have already discovered such surprises during your experiments. Code in MPS is not text, but instead it is a projection of the abstract syntax (AST). This gives several benefits to language designers and, at the same time, makes the editing experience slightly different and more-restrictive than in the commonplace text editors. This may or may not be considered to be an advantage, nevertheless, there is a learning curve to projectional editing. In general, in projectional code you more rely on the assistance of the editor. Code-completion, block selection and intention actions should become your close friends in order to become an efficient projectional coder. There's a good and a bad side to it:

  • good - in less than a week of programming in the MPS editor people typically get back to their full speed of coding they experienced before in text-based IDEs
  • bad - projectional editing is highly addictive and you may find text-based editors less compelling and less helpful than you thought they'd been before
Icon

We summarized the core essential tips on how to leverage the projectional editor. Please, check out the Commanding the editor page and keep it ready when making your first steps in MPS. Although you do not have to memorise all the keyboard shortcuts immediately, they will certainly make your coding more efficient once you start developing your own projects.

Step 7 - Shapes tutorial

Raw estimate - 2 hours

Here's an opportunity for you to quickly get your own language. Check out the MPS entry-level Shapes Tutorial and build a language that enables non-programmers to build scenes of graphical shapes.


Icon

We've collected links to useful documentation sources at the bottom of this page. Feel free to consult them whenever you need some additional information.

Step 8 - Calculator tutorial

Raw time estimate - 8 hours

Now you must be curious to find out more details about language design in MPS, right? Having given you all the fundamental information and some level of experience, the introductory tutorials should have left you in a good position to dive deeper and build a sample language all the way from the ground up. The Calculator tutorial will guide you through the process of language creation. You start with an empty project and end up with a language to build Java Swing-based visual applications. Try it out and have fun.

Icon

Beware, you are entering a "Suck Zone" here. We're happy to see you here, but you need to be warned that the road gets quite steep and more bumpy as you progress through the Calculator tutorial and the following exercises. No wonder, you'll be doing real language design now and you'll be left intentionally on your own to implement parts of the languages. If things do not work out for you, relax, step back and try again. Sometimes the problem may be as small as a semicolon missing from selection or a method placed in a wrapper class instead of an anonymous inner class. Remember, you can always ask for assistance on the forum and we'll help you get back on track.

The Calculator tutorial in JavaScript

As an illustration that MPS is not limited to generating Java, we've prepared a similar Calculator sample project - CalculatorJS, which generates an HTML page and a JavaScript program to hold the computation. You may check it out on GitHub.

Step 9 - Other sample projects

MPS comes with several sample projects that you may play with to get a better grasp of language development. I recommend you tried them out, perhaps in the order as they are listed below. Find out what they do and how they are implemented. They may teach you important lessons. Don't worry if you sometimes cannot figure out how a particular feature has been implemented. Most likely you will find an answer in one of the following exercises and then you can revisit the sample and the thing will just click instantly.

Here's a list of the samples that we've prepared for you:

Language extensions:

  • SampleJavaExtensions (updated) - a collection of small handy Java enhancements, such as parallel for loop, unless statement, Money literal, decision tables and Constants definition. The sample will teach you how to extend existing general purpose languages with custom constructs. You may also like to watch a short screen-cast introducing the individual extensions.
  • Complex - defines a Java extension to work with complex numbers
  • XML Literals - allows XML literals to be used directly in Java code as first-class language elements. The language builds on top of jetbrains.mp.xml and generates into into org.jdom.* API. The user-inserted XML code may refer back to the surrounding Java context to easily parametrize.
  • math (updated) - provides fancy-looking math language extension to Java, which allows you to enter complex numbers, matrixes, intervals, summation, product and other common mathematical operations in a natural way.

Standalone languages:

  • LanguagePatterns (updated) - a set of sample languages illustrating implementations of common language patterns in MPS. It is useful for beginners to see in isolation concrete implementations of typical language constructs. This sample project is gradually being updated with new patterns.
  • MultipleProjections - an example of a simple state machine language that focuses on the mechanism of multiple projections per concept. The programmer, who uses the language, can on-the-fly switch between several pre-defined notations as she progressed with her state-machine definition. All the available notations are defined in the language or its extensions.
  • MultiTarget - illustrates how to have a single language generated into multiple different target languages.
  • Custom Aspects - shows the ways to define custom aspects for language definition. Check out the Custom language aspect cookbook for details.
  • Component Dependencies - a minimalistic example of a language for expressing dependencies between components of a system, which provides three alternative editor notations:
    1. Textual
    2. Tabular
    3. Diagrams
  • Customized Debugger (new) - an example of hooking a languages with the Java debugger, even when the language neither extends nor generates into BaseLanguage.
  • Mind maps - an intuitive example of defining a diagramming notation, including a set of customized graphical symbols
  • State machine (Secret Compartment language/DSL book sample) - an example of a state machine language, including a state machine test language. An overview screen-cast is available to help you get started.
  • The Simplest Language - a minimalistic example generating a Hello world application printing out a customized message. Shows the very basics of language generation.
  • Migrations - a simple example of migration scripts to migrate two interconnected languages using both the smodel API and TransformStatement.
  • Lambda Calculus - a language simulating lambda calculus, with projectional editor and generation into Java
  • Agreement - an example of a business-oriented DSL
  • Expressions - a simplified expression language to allow you to play with the type system and the Type-system Trace tool. The language gives you a bare-bone expressions language with minimal editor support, but with well structured type-system rules, which you can help you learn the core of how MPS calculates types. Read more in the Typesystem Debugging documentation.
  • Fixed Length Reader - illustrates how to create a language for describing binary data formats. Comes with a stub for a hypothetical runtime library that would extract elements from binary data using the supplied data format description (called configuration).
  • HighLevel Languages (new) - an example of defining a custom set of language definition languages. Illustrates how to define your own way to codify structure and editor for a language.
  • Sample Language - A very intuitive and minimalistic example showing how to implement a common pattern of a root concept containing definitions as well as references to them. In this sample, we simulate Java classes, method definitions and method calls. However, such a pattern is quite frequent in many languages and it is worthwhile to spend some time investigating. You may also check out a brief screen-cast showing you round this sample.
  • Formula Language - a sample definition of an expression language, including editor actions and a generator
  • NotesOrganizer (new) - a sample DSL plugin for IntelliJ IDEA including a build script, that simulates a task/todo list. Tasks can have different priorities, states and categories, which by themselves can also be customized. The task lists can be filtered using several kinds of Swing UI components and the user can choose among several alternative visualizations of the task lists. Shows as an example of filtering node collections, alternative editors and direct incorporation of Swing components into form-like notations.
  • Attributes - a sample illustrating the transparent handling of attributes in text-gen and the generator. The two CommentAttribute node attribute concepts are passed through the generator unchanged so that their text-gen can convert them to text.
  • Sample Facet (updated) - illustrates a simple make facet that intercepts the make process.
  • Lightweight DSL - examples of usages of the jetbrains.mps.baselanguage.lightweightDSL language to build internal "lightweight" DSLs embedded into BaseLanguage classes.
  • Heating controller  - a simple language providing two notations to specify heating plans for a hypothetical home heating controller. The default notation allows for textual representation, while a language extension provides an alternative tabular notation.
  • Progress Indicator - a set of actions that illustrate the proper way to handle progress bars in actions, how to make actions cancellable and enable them to run in the background.
  • xmlPersistence - a non-trivial example of implementing custom persistence using the new Open API.The plugin contains three solutions (a build script, a persistence implementation and a plugin descriptor). The custom persistence implementation will load/save XML documents. Please check out our Custom Persistence Cookbook for details.

Samples to tutorials:

  • Shapes (updated) - the language created as part of the introductory Shapes Tutorial.
  • Robot Kaja (updated) - the language for commanding a virtual robot used in many of the introductory screencasts. Now includes a sample Context Assistant implementation.
  • Calculator - the language created as part of the Calculator Tutorial.
  • Generator demo - the set of languages developed as part of the Generator Tutorial.

Step 10 - Generator tutorial

Raw time estimate - 12 hours

The Calculator tutorial should have left you with good overall knowledge of language design. The Generator tutorial will now teach you more subtle options that the MPS generator offers. You will learn about the different types of macros, generator scripts, utility models and more. 

Reviewing the progress

Congratulations! By completing the above tutorials and exercises you've made a serious dent into the language design world. Your knowledge and command of MPS could now be qualified somewhere above the Advanced beginner stage of the Dreyfus model of skill acquisition. This is definitely a good time to celebrate and perhaps also to let us know about you success.

Moving up the ladder

The next stage in the Dreyfus model of skill acquisition is Competent. Once you reach that stage you'll be able to work on your own without concrete guidelines and supervision. But for this to happen, you'll need to spend more time solving various problems with MPS. If you already have an idea of a language to implement, go ahead and do it. If not, try to come up with a useful little language. You could as well try to re-implement one or more of the MPS samples and then see how closely you got. Or just pick a sample as the starting point and move it forward - add language features, implement a generator, add type-system rules, polish the editing experience - whatever feels like the right challenge for you.

Additionally, you should try these in order to become Competent in MPS and language design:

  1. Study the Common language patterns
  2. Study the cookbooks to deepen your knowledge of the various aspects of language design
  3. Get to know quotations and anti-quotations
  4. Familiarise yourself with the languages bundled in MPS, such as closures, collections, concept functionsregexp, tuples, smodel, build language and other
  5. Learn how to build MPS projects from the command line, how to package languages into IntelliJ IDEA as well as MPS plugins, how to use the plugins and how to build standalone DSL IDEs
  6. Get a feel for the user guide so you can quickly find the information that you need
  7. Learn some of the keyboard shortcuts by heart
  8. Become active in the MPS community and try to answer the forum questions of people that walk behind you on the same path to MPS mastery

After becoming Competent you should definitely continue gaining experience. The more you work with MPS and the more divert your projects are, the faster you'll be moving up on the Dreyfus model of skill acquisition scale. If you set yourself a goal to become Proficient and ultimately an Expert in MPS, you might consider trying:

  1. Studying and practising advanced topics, such as:
    1. Find usages
    2. MPS Make Framework
    3. Debugger enhancements and customisation
    4. Custom persistence
    5. Dependency analyzer and Module dependency tool
    6. Custom dataflow analyzers
    7. Extensions 
  2. Helping MPS expand into there areas by creating a base language for a different target language/platform - Python, Ruby, Dart, Haskell, Go, etc
  3. Speaking about MPS at events
  4. Helping us evolve the documentation
  5. Contributing code to the MPS project

Although there's not much detailed documentation to guide you at this experience level and you'll have to be self-driven most of the time, we'll always be happy to discuss, advise and cooperate with you.

Documentation sources

Here's a list of documentation sources that you may need on your path to MPS mastery

Books

Fabien Campagne wrote The MPS Language Workbench: Volume I and Volume II books, which you can purchase both electronically and as paper-back. Get them here.

Academic papers

The publications page collects all relevant papers about MPS and the related field.

Cookbooks

The get slightly simplified practical information as well as some tips and tricks on individual aspects of language design, you may try some of the cookbooks.

User guides

Full-blown user guides will provide exhaustive information on their respective subject.

Screencasts

Over time the MPS team has created a large collection of screen-casts covering many aspects of MPS use and language design. You may check them out all nicely sorted at the MPS screen-cast page or directly at the MPS channel of JetBrains TV.

Guided Trainings

The MPS team offers hands-on trainings. Check out the details:

General documentation

The MPS documentation page will give you all the remaining pointers to case studies, tutorials, FAQs and other pieces of MPS documentation.

Glossary

To set common terminology and avoid confusion, we wrote down the most frequent terms for you: 

MPS glossary

Abstract Syntax Tree (AST)

a logical representation of code in memory (and disk) in the shape of a tree forest that describes hierarchies of nodes. These nodes have a notion of a parent-child relationship. Additionally, two nodes can be mutually connected with explicit references that go across the hierarchy structure.

BaseLanguage

a projectional clone of Java 6 (with optional extensions for Java 7 and 8). It follows the Java specification and is 1:1 compatible with Java 6. Additionally, MPS provides several handy extensions to BaseLanguage, such as dates, collections, closures and many others.

Code generation

the process of transfromation code from one model (AST) into another model. For example, code describing a set of business rules can be transformed into plain Java so that it can be compiled with javac and run as part of an enterprise application.
Code generation in MPS has two phases - first a series of model-to-model transformations gradually reduce the concepts used in the AST of the program until a bottom-line set of base concepts is reached. Then a text-generating phase translates the AST into textual files.

DevKit

A package of related languages that have been grouped for user convenience.

Domain Specific Language (DSL)

a language dedicated to a particular problem domain, typically created with the aim of simplicity and greater expressivity compared to a general purpose language.

Language plugin

a packaged library ( a zip file) containing all the required elements in order to use a language either inside either IntelliJ IDEA or MPS.

Projectional editor

an editor that allows the user to edit the AST representation of code directly, while mimicing the behavior of a text editor to some extent. The user sees text on teh screen and edits it, however, in reality the text is only an illusion (projection) of an AST.

Module

The top-level organization element of an MPS project that typically groups several models together. It can have three basic types: Solution, Language and DevKit and may depend on other modules and models.

Model

A lower-level organizational element grouping individual concepts. It may depend on other models.

Runtime solution

A solution that is required by a language, sometimes also called a library. Runtime solutions may contain normal models as well as stubs for Java sources, classes or jar files external to MPS.

Structure

A language aspect defining all types (concepts) of AST nodes that can be used in the language together with their relationships.

Concept

A definition that describes the abstract structure of a syntax element. E.g. the IfStatement concepts says that an if holds a boolean Expression and up-to two StatementLists.

Constraints

A language aspect holding additional restrictions on concepts, their properties and relationships.

Behavior

Allows the language designer to define behavior of the language concepts.

Editor

Holds vizualization definitions of individual language concepts. Since the way concepts are viewed and edited on the screen can be customized, the editors specify how the user will interact with the language.

Scope

The set of elements that are visible and applicable to a particular position within a program. Typically only a sub-set of all elements of a particular kind can be used at any given program location.

Typesystem

A set of rules that validate and infer types of concepts in a program. 

Actions

User-invoked commands that may perform changes to the code. Actions can be attached to keyboard shortcuts or menu items.

Intention actions

Context-sensitive actions offered to the language user through a small pop-up window triggered by the Alt + Enter key shortcut. These actions typically perform a relatively local refactoring to the code under carret or a selected block of code.

Surround With intention actions

Intentions applicable to a selected block of code that wrap the block by another concept. E.g. Surround with Try-Catch.

Refactoring

A potentially substantial automated change in code structure triggered by a user action.

  • No labels
Write a comment…