Is there a way to implement custom language features in C#?

Microsoft proposes Rolsyn API as an implementation of C# compiler with public API. It contains individual APIs for each of compiler pipeline stages: syntax analysis, symbol creation, binding, MSIL emission. You can provide your own implementation of syntax parser or extend existing one in order to get C# compiler w/ any features you would like.

Roslyn CTP

Let's extend C# language using Roslyn! In my example I'm replacing do-until statement w/ corresponding do-while:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Roslyn.Compilers.CSharp;

namespace RoslynTest
{

    class Program
    {
        static void Main(string[] args)
        {

            var code = @"

            using System;

            class Program {
                public void My() {
                    var i = 5;
                    do {
                        Console.WriteLine(""hello world"");
                        i++;
                    }
                    until (i > 10);
                }
            }
            ";



            //Parsing input code into a SynaxTree object.
            var syntaxTree = SyntaxTree.ParseCompilationUnit(code);

            var syntaxRoot = syntaxTree.GetRoot();

            //Here we will keep all nodes to replace
            var replaceDictionary = new Dictionary<DoStatementSyntax, DoStatementSyntax>();

            //Looking for do-until statements in all descendant nodes
            foreach (var doStatement in syntaxRoot.DescendantNodes().OfType<DoStatementSyntax>())
            {
                //Until token is treated as an identifier by C# compiler. It doesn't know that in our case it is a keyword.
                var untilNode = doStatement.Condition.ChildNodes().OfType<IdentifierNameSyntax>().FirstOrDefault((_node =>
                {
                    return _node.Identifier.ValueText == "until";
                }));

                //Condition is treated as an argument list
                var conditionNode = doStatement.Condition.ChildNodes().OfType<ArgumentListSyntax>().FirstOrDefault();

                if (untilNode != null && conditionNode != null)
                {

                    //Let's replace identifier w/ correct while keyword and condition

                    var whileNode = Syntax.ParseToken("while");

                    var condition = Syntax.ParseExpression("(!" + conditionNode.GetFullText() + ")");

                    var newDoStatement = doStatement.WithWhileKeyword(whileNode).WithCondition(condition);

                    //Accumulating all replacements
                    replaceDictionary.Add(doStatement, newDoStatement);

                }

            }

            syntaxRoot = syntaxRoot.ReplaceNodes(replaceDictionary.Keys, (node1, node2) => replaceDictionary[node1]);

            //Output preprocessed code
            Console.WriteLine(syntaxRoot.GetFullText());

        }
    }
}
///////////
//OUTPUT://
///////////
//            using System;

//            class Program {
//                public void My() {
//                    var i = 5;
//                    do {
//                        Console.WriteLine("hello world");
//                        i++;
//                    }
//while(!(i > 10));
//                }
//            }

Now we can compile updated syntax tree using Roslyn API or save syntaxRoot.GetFullText() to text file and pass it to csc.exe.

You can check www.metaprogramming.ninja (I am the developer), it provides an easy way to accomplish language extensions (I provide examples for constructors, properties, even js-style functions) as well as full-blown grammar based DSLs.

The project is open source as well. You can find documentations, examples, etc at github.

Hope it helps.

The big missing piece is hooking into the pipeline, otherwise you're not much further along than what .Emit provided. Don't misunderstand, Roslyn brings alot of great things, but for those of us who want to implement preprocessors and meta programming, it seems for now that was not on the plate. You can implement "code suggestions" or what they call "issues"/"actions" as an extension, but this is basically a one off transformation of code that acts as a suggested inline replacement and is not the way you would implement a new language feature. This is something you could always do with extensions, but Roslyn makes the code analysis/transformation tremendously easier:

From what I've read of comments from Roslyn developers on the codeplex forums, providing hooks into the pipeline has not been an initial goal. All of the new C# language features they've provided in C# 6 preview involved modifying Roslyn itself. So you'd essentially need to fork Roslyn. They have documentation on how to build Roslyn and test it with Visual Studio. This would be a heavy handed way to fork Roslyn and have Visual Studio use it. I say heavy-handed because now anyone who wants to use your new language features must replace the default compiler with yours. You could see where this would begin to get messy.

Building Roslyn and replacing Visual Studio 2015 Preview's compiler with your own build

Another approach would be to build a compiler that acts as a proxy to Roslyn. There are standard APIs for building compilers that VS can leverage. It's not a trivial task though. You'd read in the code files, call upon the Roslyn APIs to transform the syntax trees and emit the results.

The other challenge with the proxy approach is going to be getting intellisense to play nicely with any new language features you implement. You'd probably have to have your "new" variant of C#, use a different file extension, and implement all the APIs that Visual Studio requires for intellisense to work.

Lastly, consider the C# ecosystem, and what an extensible compiler would mean. Let's say Roslyn did support these hooks, and it was as easy as providing a Nuget package or a VS extension to support a new language feature. All of your C# leveraging the new Do-Until feature is essentially invalid C#, and will not compile without the use of your custom extension. If you go far enough down this road with enough people implementing new features, very quickly you will find incompatible language features. Maybe someone implements a preprocessor macro syntax, but it can't be used along side someone else's new syntax because they happened to use similar syntax to delineate the beginning of the macro. If you leverage alot of open source projects and find yourself digging into their code, you would encounter alot of strange syntax that would require you side track and research the particular language extensions that project is leveraging. It could be madness. I don't mean to sound like a naysayer, as I have alot of ideas for language features and am very interested in this, but one should consider the implications of this, and how maintainable it would be. Imagine if you got hired to work somewhere and they had implemented all kinds of new syntax that you had to learn, and without those features having been vetted the same way C#'s features have, you can bet some of them would be not well designed/implemented.