The new Mathematica IntelliJ Plugin 3.0

It’s been a while, since the last, official update of the Mathematica IntelliJ Plugin (MIP) in July last year. Although it might seem that not much happened in the meantime, this is not the case. Last year, I had to make a decision which either meant tremendous pain when implementing future features or tremendous pain by reworking the very core of the MIP. I chose the latter because it seemed the better alternative in the long run and new features would be easier to implement with a fixed core system.

Before we start jumping into the details, I want to thank Szabolcs for his constant availability and for being so many people in one person: chief tester, critical judge, and feature requestor. During the development of MIP 3.0, we mainly used his IGraphM and LTemplate packages to test and to design new features and
-oh boy- we had some performance battles to fight. The main package of IGraphM is a monster with over 5k lines of code, and every implementation decision in the MIP 3.0 was a careful balance between semantic correctness, practical usability, and performance.

The last months showed that we were on the right track and that it is time for a public beta phase, where users can try the new version. If you want to participate in the beta, please use the latest IntelliJ IDEA 2018 and configure your plugin channel accordingly by following this tutorial .

Please remember that this is an early version of the MIP 3.0 that may be unreliable, may not work as intended, and will certainly contain errors. If you come across a bug, I would highly appreciate if you open an issue on the bug-tracker. If you want to discuss how things work and why some things don’t work, feel free to join our Gitter Chat.

New Features

In the following, you find the most significant changes and feature additions that are available in the new version. I won’t recap existing features like smart completion, structure view, code inspections, code folding, etc. that already existed. Only features that are new or underwent a drastic change are mentioned and even there, I concentrated on the ones that are visible to the user.

If you are new to using IDEA, I strongly recommend you get accustomed to its powerful editing tools that will work out of the box when writing Mathematica code. In addition, I have highlighted some MIP-specific features on the official documentation wiki.

Project-wide completion, navigation, and refactoring

The primary goal of the new version is to provide completion, navigation, and resolving of functions and symbols across the boundaries of the file you are currently working on. Therefore, the MIP is now able to handle projects that contain several package files or use code from other projects. Features like auto-completion, highlighting, navigation, or refactoring work now project-wide.

Above image shows a small function from the IGraphM package, where you can find all different types of symbols.

• Symbols in green are the local pattern or Block variables
• Symbols in blue are built-in Mathematica functions
• Symbols in white are definitions from the current package file
• Underlined symbols are external functions that sit in libraries or other package files of your project

Therefore, while the underlined Make function is defined in the external LTemplate project, the underlined applyGraphOpt is a function that is defined in a different file of the same project. Even for these external symbols, the new MIP 3.0 lets you quickly navigate to definitions, look up usage-messages, find all places where they are used or globally rename them.

New project, module, and library system

The project settings have undergone a complete rewrite. Now, your settings of what is a source directory inside your module are taken into account for resolving function definition and cross-file navigation.

The most significant improvement, however, is the possibility to attach “Mathematica libraries” to your project. These libraries are nothing more than directories with package files that can be regarded as external sources. If you are writing a package that depends on another package which is in a different project, you can attach its sources as a library, and you get auto-completion and navigation for the exported symbols of your library code.

In the image, you can see that the IGraphM module depends on the external LTemplate library. With this setting, you can use and auto-complete functions from LTemplate inside your current project.

Additionally, it is now possible to use a different Mathematica language version for IDEA’s modules. This means, that the annotator that checks if all the functions you use in your package are available reads the language version from your project module and you can develop different projects for different Mathematica versions. Until recently, you needed to set a global language value that is used for all your projects.

Show usage documentation of your functions

While showing the usage of built-in functions and operators worked for a very long time, you can now even show the usage of your functions if you assigned one. To do this, press Ctrl + Q when your cursor is over a function.

This even works for functions that you attached as external library

Go to declarations and related symbols

Navigating to the declaration (Ctrl + B) shows now all places where a symbol gets a value; be it a usage message, options, attributes, or several different patterns.

Its close companion is “Go to related Symbol” which will not only show the declarations but all usages of a symbol. With this, you can quickly navigate to all places in a file, where you have used a function or symbol.

Find symbols

Sometimes, you want to type in the name of a symbol and navigate to its declarations which you can do by using Ctrl + Alt + Shift + N. In the appearing box, you can start typing and get a list of all matching symbols. This feature works currently only on symbols that have a usage message.

Support for Wolfram Language version 11.3

MIP 3.0 supports version 11.3 of Mathematica and contains all built-in functions for auto-completion and usage messages for a quick documentation look-up. However, there is more to it. When everything is finalized, the new MIP will contain most of the symbols that Mathematica provides outside the System` context. This makes it easy to spelunk and find hidden functionality that you might not be aware of.

To see all these suggestions, you have to invoke auto-complete (Ctrl+Space) manually one or two times, depending on if the completion window is already open or not. As an example, see below how you can discover JSON-related functions:

Support of extended With

Some time ago, Wolfram introduced the chained With syntax that is essentially a very convenient form of several nested With expressions. It works like this and you should pay attention that you can use var1 in the second definition list:

With[{var1 = 3}, {var2 = var1 + 1},
 var2 + var1
 ]

Basically, you can have as many chained definition lists as you want, but as of Mathematica 11.3, highlighting is still not supported by the official notebook front end and it will mark this correct code as error:

The MIP 3.0 fully supports this with highlighting, refactoring, and auto-completion of variable names.

Spellcheck that includes all Mathematica functions

Spellcheck works consistently for code, comments, and strings. One nifty detail is that the default MIP dictionary comes with all built-in Mathematica names included which ensures that no Mathematica function is marked as misspelled.

Basic auto-completion for comments

Inside comments, you usually don’t need auto-completion, but you still can invoke it manually by pressing Ctrl + Space. Inside an empty comment (**), you get a list of special comment tags like ::Section:: or :Author: that are typical for Mathematica. While writing a comment, you can easily insert function names when invoking completion.

A Final Warning

In Mathematica there is no real concept of local variables. However, to make the MIP usable, we will often assume that a variable is meant to be local although, in reality, it is not. Take this short snippet for instance

x = a;
Integrate[Sqrt[x], x]

The Mathematica frontend suggests through its coloring that the x in Integrate is local but this is not the case. When you evaluate this, you will see that you get a weird antiderivative that contains a. The reason is that Integrate is one of many functions that seems to localize its variables when in truth, it doesn’t because it uses the global x (which has the value a).

The MIP can and will never be fully aware of these things. I could quickly show that in order to do this, you need to completely evaluate all your package code and you would need a Mathematica kernel inside the MIP to do this.

The same argument can be used to highlight that the MIP might not be able to find all your function or variable definitions. While it is possible to analyze the syntax of your code and make the MIP understand that you define a function with the name func here

func[x_] := x^2

there are many, not uncommon expressions where it is not possible anymore. Look at this example:

SetDelayed @@ {func[x_], x^2}

For this example, you need to be able to evaluate the snippet to understand that it also defines func. That means, there is a vast difference between verifying correct syntax and understanding semantics. The first one can be achieved with a parser (which the MIP has), the latter one, however, needs in the worst case a full implementation of the Mathematica kernel and too much run-time to be part of the MIP.

The good news is that even in advanced package code, such unusual constructs are rare. However, you should always be aware of these limitations and use the powerful features of the MIP with care. Especially, when you rely on renaming of variables or other refactoring transformations, where it is important to change the correct symbols.