(untagged)

Extending Visual Studio to Provide a Colorful Language Editor

Eric Lynch

0.00/5 (No votes)

20 May 2018

In this article, we explore the implementation of a Visual Studio editor that allows editing of a fictitious Colorful language. The editor minimally implements both syntax classification / coloring and IntelliSense completion.

Background

Recently, I've become interested in writing Visual Studio extensions. The Visual Studio extensibility framework provides a somewhat dizzying number of mechanisms to accomplish this goal. This article merely describes the one of those mechanisms.

Specifically, it discusses implementing syntax coloring and IntelliSense completion. Syntax coloring is implemented via the IClassifierProvider / IClassifier interfaces. IntelliSense completion is implemented via the ICompletionSource / ICompletionSourceProvider interfaces.

While I do think the concepts presented in this article have value, this is not the only way to implement these features. To be clear, I am not endorsing these particular interfaces. I am merely sharing what I have learned. In fact, based on the following link, there are probably easier ways of accomplishing this same goal.

Language Extension Guidelines
https://code.visualstudio.com/docs/extensionAPI/language-support

I may, or may not, examine these guidelines in a future article. This will depend both on my own continuing interest and the level of reader interest.

Requirements

The solution included with this article assumes you are using Visual Studio 2017. While it's probably not very difficult to port the solution to earlier versions, it would require some work.

In my case, I use Visual Studio 2017 Professional. I believe it is also possible to create a VSIX Project with Visual Studio 2017 Community. However, I have not tested that configuration.

The solution also assumes that you have installed the "Visual Studio extension development" feature.

Introduction

This article describes the implementation of an editor for the fictitious "Colorful" language. The editor is implemented as an extension to Visual Studio's Integrated Development Environment (IDE).

To run or test the code, simply open up the solution and run it like you would any other application. A second instance of Visual Studio will fire up. An indication that it is an "Experimental Instance" will appear in the title bar as shown below.

In the experimental instance of Visual Studio, opening a file with the extension .colorful will execute the code in this solution. The solution includes the file Sample.colorful to help.

To install it for real, in a non-experimental instance, simply double click the VSIX file created by this solution. Once it's installed, you can navigate to "Tools->Extensions and Updates" to confirm the installation (and later uninstall it). It should appear something like the following:

If you are unfamiliar with creating Visual Studio extensions (VSIX Project), a step-by-step guide is presented later in this article. Before we can describe this implementation, it is necessary to first introduce a few basic concepts.

VSIX

The unit of delivery for these extensions is a VSIX file. The VSIX file is a ZIP file that implements Microsoft's Open Packaging Conventions. By simply renaming the file to use the .zip file extension, it's very easy to explore the contents of the file.

Unsurprisingly, VSIX files are created and built in Visual Studio via the VSIX Project. This project template is found under Extensibility.

When VSIX files are deployed, they are copied into a hidden directory under:

%LocalAppData%\Microsoft\VisualStudio\<version>\Extensions

For Example:

C:\Users\JohnDoe\AppData\Local\Microsoft\VisualStudio\15.0_da79fc12\Extensions

Managed Extensibility Framework (MEF)

The Managed Extensibility Framework (MEF) provides a mechanism to advertise the availability of components (called parts) to Visual Studio (or other applications). It also provides a mechanism for those applications to share information with those components (parts).

To advertise the existence of a part, we simply decorate the associated class with an Export attribute (System.ComponentModel.Composition.ExportAttribute), as follows:

[Export(typeof(IClassifierProvider))]
internal sealed class ColorfulClassifierProvider : IClassifierProvider
{
  [Import]
  private IClassificationTypeRegistryService classificationRegistry = null;
}

In the above example, you'll also notice an Import attribute (System.ComponentModel.Composition.ImportAttribute). This attribute indicates that we are requesting that MEF initialize this variable with information provided by the host application (in this case Visual Studio).

Syntax Classification

By implementing the IClassifier / IClassifierProvider interfaces (in the Microsoft.VisualStudio.Text.Classification namespace), we can "classify" spans of text within a file. For example, we might classify a span of text as a "keyword". In Visual Studio, by default, keywords are displayed using the blue color.

In Visual Studio 2017, available classifications, and their associated colors, can be found under "Tools->Options->Fonts and Colors", as follows:

IntelliSense Completion

IntelliSense completion is the nifty feature in Visual Studio that presents a pop-up list of suggestions to complete words that you have partially typed.

By implementing the ICompletionSource / ICompletionSourceProvider interfaces (in the Microsoft.VisualStudio.Language.Intellisense namespace), we can provide suggested completions for a partial word.

Implementing Syntax Classification

Syntax classification is implemented via the IClassifierProvider / IClassifier interfaces (in the Microsoft.VisualStudio.Text.Classification namespace). In our code, this involves the following five classes:

Class Name	Description
`Colorful`	This class exports (via MEF) a content type definition (for the "Colorful" language) and the mapping of a file extension (`.colorful`) to that content type definition.
`ColorfulClassifier`	This class implements the `IClassifier` interface to return classifications for each part (span) within a line of text.
`ColorfulClassifierProvider`	This class implements the `IClassifierProvider` interface and acts as a factory to manufacture instances of the `ColorfulClassifier` class.
`ColorfulKeywords`	This class provides a dictionary of keywords (color names) for the "Colorful" language.
`ColorfulTokenizer`	This class separates the parts (tokens) within a line of text. It classifies color names as keywords, text starting with "//" as comments, punctuation (e.g. comma) as operators, decimal integers as string literals, and everything else as "other".

Implementing IntelliSense Completion

IntelliSense completion is implemented via the ICompletionSource / ICompletionSourceProvider interfaces (in the Microsoft.VisualStudio.Language.Intellisense namespace). In our code, this involves the following six classes:

Class Name	Description
`Colorful`	This class exports (via MEF) a content type definition (for the "Colorful" language) and the mapping of a file extension (`.colorful`) to that content type definition.
`ColorfulCompletionSource`	This class implements the `ICompletionSource` interface to provide a filtered set of matching color names (from `ColorfulKeywords`) for the partial word located at the current position of the editor's cursor (caret).
`ColorfulCompletionSourceProvider`	This class implements the `ICompletionSourceProvider` interface and acts as a factory to manufacture instances of the `ColorfulCompletionSource` class.
`ColorfulKeywords`	This class provides a dictionary of keywords (color names) for the "Colorful" language.
`ColorfulOleCommandTarget`	This class implements the `IOleCommandTarget` interface (in the `Microsoft.VisualStudio.OLE.Interop` namespace) to initiate, commit, and dismiss IntelliSense completion sessions.
`ColorfulTextViewCreationListener`	This class implements `IVsTextViewCreationListener` (in the `Microsoft.VisualStudio.Editor` namespace) to listen for the creation of text views associated with the `Colorful` content type. For those text views, it also acts as a factory to manufacture instances of the `ColorfulOleCommandTarget` class.

Colorful Life Cycle

The diagram below describes the life cycle of the different class instances from this solution.

While most of the steps were anticipated, I must admit there were a couple of surprises. I was surprised that the instantiations of ColorfulTextViewCreationListener (step 6) and ColorfulOleCommandTarget (step 7) did not come first. I was more surprised by the second instantiation of ColorfulClassifierProvider (step 5).

Understanding the Code

In this portion of the article, we'll examine each of the classes in more detail.

Colorful

This class is the easiest to explain. It has no moving parts. Instead, it serves mostly to advertise the existence of this editor to Visual Studio (via MEF). It also provides a couple of global constants. One of these constants, the content type name (ContentType) is used in other modules to associate all of the parts responsible for handling the Colorful content type.

[Export]
[Name(ContentType)]
[BaseDefinition("code")]
internal static ContentTypeDefinition ContentTypeDefinition = null;

These lines merely advertise the existence of a new content type (Colorful) to Visual Studio (via MEF). The new content type (Colorful) is based on one of the built-in content types (code) exposed by Visual Studio.

[Export]
[Name(ContentType + nameof(FileExtensionToContentTypeDefinition))]
[ContentType(ContentType)]
[FileExtension(FileExtension)]
internal static FileExtensionToContentTypeDefinition FileExtensionToContentTypeDefinition = null;

These lines merely advertise a relationship between a file extension (.colorful) and a content type (Colorful) to Visual Studio (via MEF).

ColorfulClassifier

This class, instantiated by ColorfulClassifierProvider, implements the IClassifier interface. It is responsible for the syntax classification of all of the spans of text within a document of content type Colorful.

The IClassifier interface requires only two members:

Member Name	Description
`ClassificationChanged`	This event occurs after the syntax classification of a span of text changes unexpectedly. The code provided with the solution does not raise this event.
`GetClassificationSpans(SnapshotSpan)`	This method gets syntax classifications for each of the spans of text that intersect the specified text span. The "snapshot" portion of the type name refers to the fact that the provided span is a snapshot of text (as it existed at a particular moment in time).

internal ColorfulClassifier(ITextBuffer buffer,
  IStandardClassificationService classifications,
  IClassificationTypeRegistryService classificationRegistry)

The constructor accepts three arguments. They are as follows:

Parameter Name	Description
`buffer`	This is the text buffer that contains the entire `Colorful` document.
`classifications`	This is a service that exposes standard syntax classifications (e.g. keyword, whitespace, string literal, etc.) that are common to many languages.
`classificationRegistry`	This is a service that exposes all known syntax classifications. It can be used (although not by this code) to obtain access to language-specific syntax classifications.

Let's examine the code within the GetClassifications method.

var list = new List<ClassificationSpan>();

Here, we create a list of classification spans that we will later populate. A classification span is a combination of a syntax classification and a span of text.

ITextSnapshot snapshot = span.Snapshot;
string text = span.GetText();
int length = span.Length;
int index = 0;

Here, we simply get some information about the "snapshot" of the span of text we will process. Generally, the text spans provided by the Visual Studio code editor contain all of the characters for an entire individual line of code.

while(index < length)
{
  int start = index;
  index = tokenizer.AdvanceWord(text, start, out IClassificationType type);

  list.Add(new ClassificationSpan(new SnapshotSpan(snapshot,
    new Span(span.Start + start, index - start)), type));
}

Here, we process the input text. Using the ColorfulTokenizer.AdvanceWord method, we break the line of text into words (or tokens). Each word is a set of contiguous white space characters, a set of contiguous non-white space characters, individual punctuation characters (e.g. comma), or an entire comment (beginnng with the text "//").

The AdvanceWord method returns two pieces of information about each word: the zero-based index at which it ends (index) and its syntax classification type (type).

Using this information, in combination with the zero-based index at which the word starts (start) we create a new classification span for the word. We first create a SnapshotSpan which represents the text span for the word at a specific moment in time. Using this, in combination with the syntax classification type, we create a classification span. Finally, we add that classification span to the list we created earlier.

return list;

After we've built a list of all of the classification spans, we return it to the caller (Visual Studio).

Because our syntax is artificial, and intentionally simple, we can cheat on our implementation.

A true implementation would most likely perform much of the classification logic in a background thread. In that background thread, it would likely create an Abstract Syntax Tree (AST) for the entire document. It would likely monitor the text buffer (provided in the constructor) for changes, updating the AST as those changes occurred.

The GetClassificationSpans method would then simply find the elements of the AST that intersect the input text. For each of these elements, it would create a classification span.

ColorfulClassifierProvider

This class simply implements the IClassifierProvider interface to provide a factory that manufactures instances of ColorfulClassifier. It is instantiated by Visual Studio (via MEF).

[Export(typeof(IClassifierProvider))]
[Name(nameof(ColorfulClassifierProvider))]
[ContentType(Colorful.ContentType)]
internal sealed class ColorfulClassifierProvider : IClassifierProvider

Here, we simply advertise the existence of an implementation of IClassifierProvider to Visual Studio (via MEF) for the Colorful content type. The name of the part is optional, and largely arbitrary, but should be somewhat unique.

[Import]
private IClassificationTypeRegistryService classificationRegistry = null;

Here, we import a service, from Visual Studio (via MEF), that provides us access to all of the known syntax classification types. We later share this service with the ColorfulClassifier instance we create.

[Import]
private IStandardClassificationService classifications = null;

Here, we import a service, from Visual Studio (via MEF), that provides us access to the standard syntax classification types (e.g. keyword) that are common to many languages. We later share this service with the ColorfulClassifier instance we create.

public IClassifier GetClassifier(ITextBuffer buffer) =>
  buffer.Properties.GetOrCreateSingletonProperty(() =>
    new ColorfulClassifier(buffer, classifications, classificationRegistry));

The IClassifierProvider interface has only one required member: the GetClassifier method.

In our implementation, we create an instance of ColorfulClassifier associated with the text buffer provided as an argument by Visual Studio. We use the GetOrCreateSingletonProperty method to insure that we only create one instance per text buffer. Subsequent invocations, for the same text buffer, simply return the previously created instance.

ColorfulCompletionSource

This class, instantiated by ColorfulCompletionSourceProvider, implements the ICompletionSource interface. This interface only requires two methods: AugmentCompletionSession, which provides a list of words that match a word partially typed by a user, and Dispose, which can be used to dispose of unmanaged resources.

internal ColorfulCompletionSource(ITextBuffer buffer,
  ITextStructureNavigatorSelectorService navigatorService)

The constructor accepts two arguments: the text buffer containing the word the user is typing and a service (from Visual Studio) to obtain a navigator for that text buffer.

public void AugmentCompletionSession(ICompletionSession session,
  IList<CompletionSet> completionSets)

The AugmentCompletionSession method accepts two parameters: an IntelliSense completion session (ICompletionSession) and a list of completion sets (CompletionSet). A completion set is basically a list of completed words coupled with a text span containing the partial word. The latter is a "tracking" text span. A "tracking" text span allows IntelliSense to sense changes to the partial word and adjust the selections that are displayed as the user continues to type.

ITrackingSpan wordToComplete = GetTrackingSpanForWordToComplete(session);

When the user begins to type a partial word, this triggers an IntelliSense completion session. Above, we use our own GetTrackingSpanForWordToComplete method to get a "tracking" text span for that partial word.

CompletionSet completionSet = new CompletionSet(Moniker, DisplayName,
  wordToComplete, completions, null);

Above, we create a completion set that pairs the "tracking" text span, for the partial word, with a list of all possible complete words.

completionSets.Add(completionSet);

Above, we add this completion set to list of available completion sets; thus, augmenting the completion session. IntelliSense supports the concept of multiple tabs, each with a different completion set. This is why there is a list of completion sets, instead of only one.

Let's take a quick look at GetTrackingSpanForWordToComplete, so we understand how we got the partial word.

SnapshotPoint currentPoint = session.TextView.Caret.Position.BufferPosition - 1;

Here, we find the position of the cursor within the text view. This will help us determine what partial word the user is typing.

TextExtent extent = Navigator.GetExtentOfWord(currentPoint);

Next, we get the word that the user is typing. In this case, a "word" is defined as a contiguous set of non-whitespace characters surrounded by whitespace or punctuation.

return currentPoint.Snapshot.CreateTrackingSpan(extent.Span,
  SpanTrackingMode.EdgeInclusive);

Finally, we create a "tracking" text span for the word, which adjusts itself as the user continues typing.

ColorfulCompletionSourceProvider

This class simply implements IClassifierProvider to provide a factory that manufactures instances of ColorfulCompletionSource. It is instantiated by Visual Studio (via MEF).

[Export(typeof(ICompletionSourceProvider))]
[Name(nameof(ColorfulCompletionSourceProvider))]
[ContentType(Colorful.ContentType)]
internal class ColorfulCompletionSourceProvider : ICompletionSourceProvider

Here, we simply advertise the existence of an implementation of ICompletionSourceProvider to Visual Studio (via MEF) for the Colorful content type. The name of the part is optional, and largely arbitrary, but should be somewhat unique.

[Import]
private ITextStructureNavigatorSelectorService navigatorService = null;

Here, we import a service, from Visual Studio (via MEF), that provides us access to navigators (also from Visual Studio) for a text buffer. A "navigator" allows us to perform simple operations like getting a word from the text.

public ICompletionSource TryCreateCompletionSource(ITextBuffer textBuffer) =>
  new ColorfulCompletionSource(textBuffer, navigatorService);

The ICompletionSourceProvider interface has only one required member: the TryCreateCompletionSource method. In our implementation, we create an instance of ColorfulCompletionSource associated with the text buffer provided as an argument by Visual Studio.

ColorfulKeywords

This class is simply a singleton dictionary of keywords (color names). It exposes only two members: the All property, which provides a list of all keywords (in alphabetic order) and the Contains method, which allows us to test if a word is a keyword. The Contains method is case-insensitive.

ColorfulTokenizer

This class contains logic to parse a simple line of text into separate words (or tokens) and provide a syntax classification for each word. Since the implementation details are unimportant, we'll simply describe what it does instead of how it does it.

The primary method in this class is the AdvanceWord method. It accepts three arguments: the line of text to process, the current zero-based index within that text, and an output parameter where it returns the syntax classification for the most recently processed word. In addition to returning a syntax classification (via an out parameter), it also returns the zero-based index of the next word.

By repeatedly calling this method until the returned index reaches the length of the text, it is possible to find and classify all of the tokens (words) in that text.

For the purposes of this method, a "word" is: a contiguous sequence of whitespace characters, a contiguous sequence of non-whitespace characters, a single punctuation character, or an entire comment (which is a sequence of characters beginning with "//").

The syntax classification is one of the following members of IStandardClassificationService (in the Microsoft.VisualStudio.Language.StandardClassification namespace), each of which is an instance of IClassificationType (in the Microsoft.VisualStudio.Text.Classification.IClassificationType namespace):

Member Name	Description
`Keyword`	A contiguous sequence of non-whitespace characters containing a keyword / color name (e.g. Blue).
`Operator`	A single punctuation character (e.g. comma).
`Other`	A contiguous sequence of non-whitespace characters that is not classified as an `Operator`, `Keyword`, or `StringLiteral`.
`StringLiteral`	A contiguous sequence of non-whitespace characters consisting solely of the characters "0" through "9". We chose the `StringLiteral` classification instead of `NumberLiteral`, because `NumberLiteral` is (by default) the same color as `Other`.
`WhiteSpace`	A contiguous sequence of whitespace characters.

ColorfulOleCommandTarget

This class, instantiated by ColorfulTextViewCreationListener, implements a bunch of (in my opinion) hacky behavior necessary to initiate and dismiss IntelliSense completion sessions in reaction to user interface behavior (e.g. the user types a character).

Basically, we need to fire up the "way back" machine and interact with OLE (Object Linking and Embedding) so that we can monitor a text view.

internal ColorfulOleCommandTarget(IVsTextView vsTextView, ITextView textView,
  ICompletionBroker completionBroker, SVsServiceProvider serviceProvider)

The constructor accepts four arguments. They are as follows:

Parameter Name	Description
`vsTextView`	The Visual Studio text view monitored by this class.
`textView`	The WPF text view monitored by this class. The WPF text view is obtained from the Visual Studio text view.
`completionBroker`	The central completion broker responsible for IntelliSense completion.
`serviceProvider`	The central Visual Studio service provider.

vsTextView.AddCommandFilter(this, out nextCommandTarget);

Here, we add ourselves into the linked list of OLE command filters for the Visual Studio text view we are monitoring.

The IOleCommandTarget requires two methods: the Exec method, which executes OLE commands, and the QueryStatus method, which queries the status of an OLE command.

QueryStatus

This method queries the status of an OLE command.

public int QueryStatus(ref Guid pguidCmdGroup, uint cCmds,
  OLECMD[] prgCmds, IntPtr pCmdText) =>
  nextCommandTarget.QueryStatus(ref pguidCmdGroup, cCmds, prgCmds, pCmdText);

We have little to offer here, so we simply allow the next OLE command target in the chain to respond with a status.

Exec

if (VsShellUtilities.IsInAutomationFunction(ServiceProvider))
  return nextCommandTarget.Exec(ref pguidCmdGroup, nCmdID, nCmdexecopt, pvaIn, pvaOut);

If this is an automation function (e.g. test automation), we simply stay out of the way and allow the next OLE command target in the chain handle the command.

char? typedChar = GetTypedChar(pguidCmdGroup, nCmdID, pvaIn);

The command can come either in the form of a command code or a typed character. Above, we get the typed character (if any).

if (HandleCommit(nCmdID, typedChar))
  return VSConstants.S_OK;

Above, we check to see if a user has either committed a selection from the IntelliSense pop-up (e.g. by pressing the RETURN key) or dismissed the pop-up (e.g. by pressing the ESCAPE key). If either of these are the case, we simply return the S_OK code, to indicate that we handled the command.

int result = nextCommandTarget.Exec(ref pguidCmdGroup, nCmdID, nCmdexecopt,
  pvaIn, pvaOut);

We allow the next command target in the chain to handle the command. This is necessary, because we want any typed characters to end up in the text view, before we proceed any further.

return ErrorHandler.Succeeded(result) ?
  HandleCompletion(nCmdID, typedChar, result) : result;

If the next command target successfully handled the command, we check whether we need to initiate an IntelliSense completion session. If we do initiate one, we return S_OK; otherwise, we return the result received from the next command target in the chain.

While we won't examine every low-level method in this class, since some of the implementations are pretty obvious, we will examine those that may be a bit more difficult to understand. These include: HandleCommit, HandleCompletion, and TriggerCompletion.

HandleCommit

This method checks to see if a user has either committed a selection from the IntelliSense pop-up (e.g. by pressing the RETURN key) or dismissed the pop-up (e.g. by pressing the ESCAPE key).

if (!HasCompletionSession() || (!IsCommitCommand(commandId) && !IsCommitChar(typedChar)))
  return false;

If we don't have an IntelliSense completion session, we simply return. If the user did not initiate a commitment by command (RETURN or TAB) or by typed character (white space or punctuation), we simply return.

if (!completionSession.SelectedCompletionSet.SelectionStatus.IsSelected)
{
  completionSession.Dismiss();
  return false;
}

If there is no selection, we simply dismiss IntelliSense completion session.

completionSession.Commit();
return true;

Otherwise, we commit the selection made by the user during the IntelliSense completion session. Committing the session causes the remaining characters of the partial word to be added to the text view.

HandleCompletion

This method reacts to commands (e.g. typed characters) from the user interface. If necessary, it initiates an IntelliSense completion session. Also, if necessary, it initiates further filtering of previous selections displayed by that IntelliSense completion session.

if (typedChar.HasValue && Char.IsLetterOrDigit(typedChar.Value))

This checks if the user typed a letter or a digit. We initiate an IntelliSense completion session when this occurs, if it has not already previously been initiated.

if (!TriggerCompletion())
  return result;

If the user typed a letter or digit, we attempt to initiate an IntelliSense completion session. If we cannot initiate an IntelliSense completion session (unlikely), we simply return the result received by the next OLE command in the chain.

if ((commandId != (uint)VSConstants.VSStd2KCmdID.BACKSPACE &&
  commandId != (uint)VSConstants.VSStd2KCmdID.DELETE) ||
  !HasCompletionSession())
  return result;

If the user did not type a letter or digit, and we have a completion session, we still may have work to do. If the user deleted or backspaced, we need to adjust the selections that are displayed by that IntelliSense completion session. Otherwise, we simply return the result received by the next OLE command target in the chain.

completionSession.Filter();

We adjust the selections that are displayed by the IntelliSense completion session.

return VSConstants.S_OK;

We indicate that we handled the command.

TriggerCompletion

This method creates the IntelliSense completion session, which results in the display of the associated pop-up list.

if (HasCompletionSession())
  return true;

If we have already created the IntelliSense completion session, simply return.

SnapshotPoint? caretPoint = TextView.Caret.Position.Point.GetPoint(
  IsNotProjection, PositionAffinity.Predecessor);
if (!caretPoint.HasValue)
  return false;

Basically, this simply gets the position of the caret (cursor). If no position is available (unlikely), we simply return.

ITrackingPoint trackingPoint = caretPoint.Value.Snapshot.CreateTrackingPoint(
  caretPoint.Value.Position, PointTrackingMode.Positive);

We create a "tracking" position (point) for the cursor. If something causes the position to move, the "tracking" point tracks that change.

completionSession = CompletionBroker.CreateCompletionSession(
  TextView, trackingPoint, true);

We ask the central completion broker responsible for IntelliSense completion to create a completion session for us.

completionSession.Dismissed += OnSessionDismissed;

We subscribe to the Dismissed event. This way, if the IntelliSense completion session ends without our knowledge, we are still aware that it happened.

completionSession.Start();
return true;

We start the IntelliSense completion session and return.

ColorfulTextViewCreationListener

This class simply implements IVsTextViewCreationListener to provide a factory that manufactures instances of ColorfulOleCommandTarget. It is instantiated by Visual Studio (via MEF). It acts as a factory when an editable text view containing Colorful content is created.

[Export(typeof(IVsTextViewCreationListener))]
[Name(nameof(ColorfulTextViewCreationListener))]
[ContentType(Colorful.ContentType)]
[TextViewRole(PredefinedTextViewRoles.Editable)]
public class ColorfulTextViewCreationListener : IVsTextViewCreationListener

Here, we simply advertise the existence of an implementation of IVsTextViewCreationListener to Visual Studio (via MEF) for the Colorful content type. We indicate that we are interested in any text views that can be edited. The name of the part is optional, and largely arbitrary, but should be somewhat unique.

[Import]
private IVsEditorAdaptersFactoryService adapterService = null;

Here, we import a service, from Visual Studio (via MEF), that provides us access to adapters (also from Visual Studio). The adapter allows us to obtain a WPF text view from a Visual Studio text view.

[Import]
private ICompletionBroker completionBroker = null;

Here, we import, from Visual Studio (via MEF), the central broker responsible for IntelliSense completion.

[Import]
private SVsServiceProvider serviceProvider = null;

Here, we import, from Visual Studio (via MEF), the central service provider for Visual Studio.

public void VsTextViewCreated(IVsTextView vsTextView)

The IVsTextViewCreationListener interface has only one required member: the VsTextViewCreated method.

ITextView textView = adapterService.GetWpfTextView(vsTextView);

First, we get a WPF text view from the Visual Studio text view. Most of the logic in ColorfulOleCommandTarget requires a WPF text view.

textView.Properties.GetOrCreateSingletonProperty(() =>
  new ColorfulOleCommandTarget(vsTextView, textView,
    completionBroker, serviceProvider));

Above, we create an instance of ColorfulOleCommandTarget for the specified text view and share the central completion broker for IntelliSense and the central service provider for Visual Studio. We use GetOrCreateSingletonProperty to insure that we only create one instance per text view. Subsequent invocations, for the same text buffer, simply return the previously created instance.

VSIX Projects Step By Step

The creation of VSIX projects, for this sort of thing, is a bit cumbersome. I'll do my best to cover it here, step by step.

To create a VSIX project, navigate to "File->New Project". Select "Visual C#->Extensibility". You should see the "VSIX Project" template as follows:

If you do not see it, click "Open Visual Studio Installer". If you scroll down a bit, you should find "Other Toolsets" and "Visual Studio extension development" in this category as follows:

Once you've created your solution, you'll notice that it contains very little and has few references. This is unfortunate.

To fix this issue, you'll need to first add an item to your project. Specifically, for this example, you'll need to add an "Editor Classifier" as follows:

After adding it, you'll see some new files and (more importantly) you'll see a bunch of new references.

While the template source files are helpful to provide a jumping off point, they are not useful for much else. I end up deleting a lot of them.

One reference that I really wish was include is Microsoft.VisualStudio.Language.StandardClassification. This is necessary to reference any of the standard classifications. Since virtually every true Editor Classifier should reference these, its a mystery to me why this reference is omitted. You'll need to add it yourself.

There is not much support available for solutions that implement IntelliSense completion. You are largely on your own. This is a shame, since a whole bunch of references need to be added manually.

Additional references that are required for effective syntax coloring include:

Microsoft.VisualStudio.Language.StandardClassification

Additional references that are required for effective IntelliSense completion include:

Microsoft.VisualStudio.Editor
Microsoft.VisualStudio.Language.Intellisense
Microsoft.VisualStudio.OLE.Interop
Microsoft.VisualStudio.Shell.15.0
Microsoft.VisualStudio.Shell.Framework
Microsoft.VisualStudio.TextManager.Interop

Additional Reading

While it's a bit difficult to find, there are actually some decent explanations of these topics spread throughout the Microsoft documentation. For further reading, check out the following:

Start to Develop Visual Studio Extensions
https://docs.microsoft.com/en-us/visualstudio/extensibility/starting-to-develop-visual-studio-extensions

Inside the Editor
https://docs.microsoft.com/en-us/visualstudio/extensibility/inside-the-editor