In the world of modern software development, a language that is not supported in any of the major Integrated Development Environments is not going to gain very much traction. For better or worse, the IDE has become a widespread and permanent fixture. Programmers are free to use any editor or environment they want in their own time, but those who write software for a living and are not self-employed are going to have to work in the environment(s) allowed by their employer. In Windows shops, that often means Visual Studio.

For the first several years of D’s existence, not only was there no support for it in Visual Studio, the toolchain on Windows produced binaries only in the OMF format, making them incompatible with the Microsoft tools out of the box. Today, that has changed, and both DMD and LDC can output COFF files that are compatible with the MS linker. But even before DMD got support for COFF, one community member initiated Visual D, a project to bring D to the Visual Studio environment.

Rainer Schuetze first discovered D shortly after work on D2 began.

It must have been an article in the German c’t magazine (maybe this one) that brought my attention to the D language. The meta programming features of D2 caught my eye, namely templates and compile-time function evaluation. It was refreshing to see these elements being neatly integrated into the language, not bolted on top of it with a different language as in C++.

He decided to give D a try. The road that led him to Visual D originated not from a longing for an IDE, but from his attempts at debugging DMD’s output.

The options D had to offer on Windows were rather disappointing: the version of Microsoft’s Windbg distributed with DMD was barely usable as it was already 12 years old back then in 2008. Newer versions didn’t work at all with the executables generated by DMD. The debugger being one of the highlights of Visual Studio, I tried it, too, but anything newer than VS.NET 2003 failed miserably, with the latter being almost acceptable. It didn’t stop execution on breakpoints. So it seemed that there might be just a small tweak to make debugging work considerably better with Visual Studio.

It was this desire to debug D programs that led Rainer to start his first open source project, cv2pdb. Rainer’s tool can convert CodeView format output by DMD to the PDB format used by the Visual Studio debugger. Once that was working, it was no great leap to want better integration with the Visual Studio environment, such as build support and code completion.

Rainer found one abandoned project that had been started toward that end already, but he couln’t get it functional. He also experimented with modifying an existing language service written in C#, but had trouble getting it to work with a parser written in D. In the end, he decided that spending his time with D “should not mean writing code in other languages.” So, in 2009, Visual D was born. He ran into some difficulties almost immediately, as anyone using D in those days was bound to.

At the time, you would hit a compiler bug every few hundred lines of code, but these were not the biggest obstacles on the way to getting this extension to work. Visual Studio is a Win32 application and loads its extensions dynamically as DLLs. D very much relies on Thread Local Storage (TLS) as it is the default for global variables. Under Windows this uses “implicit TLS” built into the binary. Unfortunately, Windows Vista was the first version to support this for dynamically loaded DLLs, but Windows XP was still the most widely used version. It only supports TLS for the DLLs that are loaded implicitly with the application.

Eventually, after a lot of debugging, he managed to work around his Windows XP problems by tricking the system into believing a manually loaded DLL had been implicitly loaded with the application. The result of these efforts can be seen in the DRuntime modules core.sys.windows.dll and core.sys.windows.threadaux. This implementation comes with the drawback that DLLs using it cannot be unloaded. An improved version by Denis Shelomovskii works around this. Given the decline of XP usage, the need for this will eventually fade away.

TLS wasn’t his only problem.

Another issue turned up with the interfaces that create the API between Visual Studio and its extensions: these are all COM interfaces. Fortunately, D supports COM out of the box, virtual function calls work nicely. Unfortunately, the implementation in the library is currently unusable. The allocation of a COM object (every class derived from IUnknown) is done using C’s malloc, assuming the usual approach to free this memory in the Release() function when the reference count goes down to zero. Unfortunately, this is not possible from within a member function of a D class as the invariant is still called before the function returns. That’s why the runtime implementation just leaks the memory.

As a solution, he initially implemented his own ComObject class and overloaded new. Later, when the overloading of new was deprecated in the language, he switched to using a factory function for COM objects. He wasn’t finished yet.

The Visual Studio SDK provides COM interface definitions for C++ that I started to convert manually on a per case basis whenever I needed a declaration in D. This proved rather tedious as dependencies grew, so I considered converting the interface definitions automatically. These are given by Interface Definition Language (IDL) files or C++ header files.

So he created a conversion tool.

It uses a tokenizer followed by some custom conversion functions (mostly dealing with C pre-processor code) and a long list of replacement rules using the function now accessible within Visual D as Search/Replace Tokens. Each token comes with the comment preceding it, so the D files very much resemble the original files and still contain documentation, which is very helpful for the VS SDK. The conversion now works for all files of the VS SDK versions 2008 to 2015 and a selection of required files from the Windows SDKs 6.0 to 10 (about 90 header files converted, some more stubbed to be mostly empty).

Visual D has come a long way since Rainer first started working on it, but the journey is by no means complete. The next release will integrate the expression evaluator of the Mago debugger with the Visual Studio debug engine, thanks to Microsoft’s publication of information about the debug engine’s extensibility. Additionally, Visual D users will gain the ability to integrate with C and C++ projects.

Just drop your D files into a C/C++ project and they compile with the rest of your application without further ado. Enjoy settings integration instead of the ancient looking project dialogs of visualdproj files. More importantly, all the additional tools like the Manifest Tool, MIDL Compiler and even Build Extensions (e.g. Assembler) are easily available to D projects, too.

These new features are available in a preview version right now. Rainer has been short on time, so the features haven’t improved much since the preview release, but once he finds the time you can be sure to see these new features in the next release.