{"id":1148,"date":"2017-10-25T15:54:36","date_gmt":"2017-10-25T15:54:36","guid":{"rendered":"http:\/\/dlang.org\/blog\/?p=1148"},"modified":"2021-10-08T11:07:33","modified_gmt":"2021-10-08T11:07:33","slug":"dmd-windows-and-c","status":"publish","type":"post","link":"https:\/\/dlang.org\/blog\/2017\/10\/25\/dmd-windows-and-c\/","title":{"rendered":"DMD, Windows, and C"},"content":{"rendered":"<p><img loading=\"lazy\" class=\"alignleft size-full wp-image-181\" src=\"http:\/\/dlang.org\/blog\/wp-content\/uploads\/2016\/08\/d6.png\" alt=\"\" width=\"200\" height=\"200\" \/>The ability to interface with C was baked into D from the beginning. Most of the time, it\u2019s something that requires little thought \u2013 as long as the declarations on the D side match what exists on the C side, things will usually just work. However, there are a few corner-case gotchas that arise from the simple fact that D, though compatible, is not C.<\/p>\n<p>An upcoming series of posts here on the D blog will delve into some of these dark corners and shine a light on the traps lying in wait. In these posts, readers will be asked to follow along by compiling and executing the examples themselves so they may more thoroughly understand the issues discussed. This means that, in addition to a D compiler, readers will need access to a C compiler.<\/p>\n<p>That raises a potential snafu. On the systems that the DMD frontend groups under the <a href=\"https:\/\/dlang.org\/spec\/version.html#predefined-versions\"><code>version(Posix)<\/code><\/a> umbrella, it\u2019s a reliable assumption that a C compiler is easily available (if a D compiler is installed and functioning properly, the C compiler will already be installed). The concept of a <em>system compiler<\/em> is a long established tradition on those systems. On Windows\u2026 not so much.<\/p>\n<p>So before diving into a series about C and D, a bit of a primer is called for. That\u2019s where this post comes in. The primary goal is to help ensure a C environment is installed and working on Windows. It\u2019s also useful to understand why things are different on that platform than on the others. Before we get to the why, we&#8217;ll dig into the how.<\/p>\n<p>First, assume we have the following two source files in the same directory.<\/p>\n<p><strong>cfoo.c<\/strong><\/p>\n<pre class=\"prettyprint lang-c_cpp\" data-start-line=\"1\" data-visibility=\"visible\" data-highlight=\"\" data-caption=\"\">#include &lt;stdio.h&gt;\r\n\r\nvoid say_hello(void) \r\n{\r\n    puts(\"Hello!\");\r\n}<\/pre>\n<p><strong>dfoo.d<\/strong><\/p>\n<pre class=\"prettyprint lang-d\" data-start-line=\"1\" data-visibility=\"visible\" data-highlight=\"\" data-caption=\"\">extern(C) void say_hello();\r\n\r\nvoid main() \r\n{\r\n    say_hello();\r\n}<\/pre>\n<p>Now let\u2019s see how to get the two working together.<\/p>\n<h3 id=\"dmdandc\">DMD and C<\/h3>\n<p>The <a href=\"https:\/\/dlang.org\/download.html\">DMD packages for Windows<\/a> ship with everything the compiler needs: a linker and other tools, plus a handful of critical system libraries. So on the one hand, Windows is the only platform where DMD has no external dependencies out of the box. On the other hand, it\u2019s the only platform where a working DMD installation does not imply a C compiler is also installed. And these days, the out-of-the-box experience often isn&#8217;t the one you want.<\/p>\n<p>On all the other platforms, the C compiler option is the system compiler, which in practice means GCC or Clang. The system linker to which DMD sends its generated object files might be <a href=\"ftp:\/\/ftp.gnu.org\/old-gnu\/Manuals\/ld-2.9.1\/html_mono\/ld.html\"><code>ld<\/code><\/a>, <a href=\"https:\/\/lld.llvm.org\/\"><code>lld<\/code><\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/Gold_(linker)\"><code>ld.gold<\/code><\/a>, or any <code>ld<\/code>-compatible linker. On Windows, there are currently two compiler choices, and neither can be assumed to be installed by default: the Digital Mars C and C++ compiler, <code>dmc<\/code>, or the Microsoft compiler, <code>cl<\/code>. We\u2019ll look at each in turn.<\/p>\n<h4 id=\"dmdanddmc\">DMD and DMC<\/h4>\n<p>The linker (<a href=\"http:\/\/www.digitalmars.com\/ctg\/optlink.html\">Optlink<\/a>) and other tools that ship with DMD are also part of the DMC distribution. DMD uses these tools by default (or when the <code>-m32<\/code> switch is passed on the command line). To link any C objects or static libraries, they should be in the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Relocatable_Object_Module_Format\">OMF format<\/a>. Compilers that can generate OMF are a rare breed these days, and while something like <a href=\"https:\/\/en.wikipedia.org\/wiki\/Watcom_C\/C%2B%2B\">Open Watcom<\/a> may work, using DMC will guarantee 100% compatibility.<\/p>\n<p>The DMC package (version 8.57 as I write) can be <a href=\"http:\/\/digitalmars.com\/download\/freecompiler.html\">downloaded from digitalmars.com<\/a>. It\u2019s a 3 MB zip file that can be unzipped anywhere. Personally, since I only ever use it in conjunction with DMD, I keep it in <code>C:\\D<\/code> so that the <code>dm<\/code> directory is a sibling of the <code>dmd2<\/code> directory. Once it\u2019s unzipped, it can be added to the path if desired. Be aware that some of the tools DMD and DMC ship with may conflict with tools in other packages if they are on the global path.<\/p>\n<p>For example, both come with Digital Mars <code>make<\/code> and Optlink, which is named <code>link.exe<\/code>. The former might conflict with Cygwin, or a MinGW distribution that\u2019s independent of MSYS2 (if <code>mingw32-make<\/code> has been renamed), and the latter with Microsoft\u2019s linker (which generally shouldn\u2019t be on the global path anyway). Some may prefer just to keep it all off the global path. In that case, it\u2019s simple to configure a command prompt shortcut that sets the PATH when it launches. For example, create a batch file, that looks like this:<\/p>\n<pre class=\"prettyprint lang-batchfile\" data-start-line=\"1\" data-visibility=\"visible\" data-highlight=\"\" data-caption=\"\">echo Welcome to your Digital Mars environment.\r\n@set PATH=C:\\D\\dmd2\\windows\\bin;C:\\D\\dm\\bin;%PATH%<\/pre>\n<p>Save it as <code>C:\\D\\dmenv.bat<\/code>. Right click an empty spot on the desktop and, from the popup menu, select <code>New-&gt;Shortcut<\/code>. In the location field, enter the following:<\/p>\n<pre class=\"prettyprint lang-text\" data-start-line=\"1\" data-visibility=\"visible\" data-highlight=\"\" data-caption=\"\">C:\\System\\Win32\\cmd.exe \/k C:\\d\\dmenv.bat<\/pre>\n<p>Now you have a shortcut that, when double clicked, will launch a command prompt that has both <code>dmd<\/code> and <code>dmc<\/code> on the path.<\/p>\n<p>Once installed, documentation on the command-line switches for the tools is available at the Digital Mars site. The most relevant are the docs for <a href=\"http:\/\/www.digitalmars.com\/ctg\/sc.html\">DMC<\/a>, <a href=\"http:\/\/www.digitalmars.com\/ctg\/optlink.html\">Optlink<\/a>, and <a href=\"http:\/\/www.digitalmars.com\/ctg\/lib.html\">Librarian<\/a> (<code>lib.exe<\/code>). The latter two will come in handy even when doing pure D development with vanilla DMD, as those are the tools needed to when manually manipulating its object file output.<\/p>\n<p>That\u2019s all there is to it. As long as both <code>dmc.exe<\/code> and <code>dmd.exe<\/code> are on the path in any given command prompt, both compilers will find the tools they need via the default settings in their configuration files. For knocking together quick tests with both C and D on Windows, it\u2019s a quick thing to launch a command prompt, compile &amp; link, and execute:<\/p>\n<pre class=\"prettyprint lang-text\" data-start-line=\"1\" data-visibility=\"visible\" data-highlight=\"\" data-caption=\"\">dmc -c cfoo.c\r\ndmd dfoo.d cfoo.obj\r\ndfoo<\/pre>\n<p>Easy peasy. Now let\u2019s look at the other option.<\/p>\n<h4 id=\"dmdandmicrosoftscl\">DMD and Microsoft\u2019s CL<\/h4>\n<p>Getting DMD to work with the Microsoft toolchain requires installing the Microsoft build tools and the Windows SDK. The easiest way to get everything is to use one of the Community editions of Visual Studio. The installer will download and install all the tools and the SDK. The latest is always available from <a href=\"https:\/\/www.visualstudio.com\/\">https:\/\/www\/visualstudio.com<\/a>. With VS 2017, the installer has been overhauled such that it\u2019s possible to minimize the size of the install more than was possible with past editions. An alternative is to install the <a href=\"http:\/\/landinghub.visualstudio.com\/visual-cpp-build-tools\">Microsoft Build Tools<\/a>\u00a0and the <a href=\"https:\/\/developer.microsoft.com\/en-us\/windows\/downloads\/windows-10-sdk\">Windows SDK<\/a> separately. However, this is still a large install that isn\u2019t much of a win in light of the new VS 2017 installer options (for those on Windows 8.1 or 10).<\/p>\n<p>Once the tooling is installed, DMD\u2019s configuration file needs to be modified to point its environment variables to the proper locations. Rather than repeat all of that here, I\u2019ll direct you to the <a href=\"https:\/\/wiki.dlang.org\/Installing_DMD\">DMD installation page<\/a> at the D Wiki. One of the reasons to prefer the DMD installer over the zip archive is that it will detect any installation of Visual Studio or the Microsoft Build Tools and automatically modify the configuration as needed. This is more convenient than needing to remember to update the configuration every time a new version of DMD is installed. It also offers to install VS 2013 Community if the tooling isn\u2019t found, can install <a href=\"http:\/\/rainers.github.io\/visuald\/visuald\/StartPage.html\">Visual D<\/a> (the D plugin for Visual Studio), and will add DMD to the system path if you want it to.<\/p>\n<p>It\u2019s a bit of an annoyance to launch Visual Studio for simple tests between C and D. Since it\u2019s not recommended to put the MS tools on the system path, each VS and Microsoft Build Tools installation ships with a number of batch files that will set the path for you (like the one we created for DMC above). The installer sets up shortcuts in the Windows Start menu. There are several different options to choose from. To launch a 64-bit environment with VS 2017 (or the 2017 build tools), find <code>Visual Studio 2017<\/code> in the Start menu and select <code>x64 Native Tools Command Prompt for VS 2017<\/code>. For VS 2015 (or the 2015 build tools), go to <code>Visual Studio 2015<\/code> and click on <code>VS 2015 x64 Native Build Tools Command Prompt<\/code>. Similar options exist for 32-bit (where <code>x86<\/code> replaces <code>x64<\/code>) and cross compiling.<\/p>\n<p>From the VS-enabled 64-bit environment, we can run the following commands to compile our two files.<\/p>\n<pre class=\"prettyprint lang-text\" data-start-line=\"1\" data-visibility=\"visible\" data-highlight=\"\" data-caption=\"\">cl \/c cfoo.c\r\ndmd -m64 dfoo.d cfoo.obj\r\ndfoo<\/pre>\n<p>In a 32-bit VS environment, replace <code>-m64<\/code> with <code>-m32mscoff<\/code>.<\/p>\n<h3 id=\"theconsequencesofhistory\">The consequences of history<\/h3>\n<p>When a new programming language is born these days, it\u2019s not uncommon for its tooling to be built on top of an existing toolchain rather than completely from scratch. Whether we\u2019re talking about languages like Kotlin built on the JRE, or those like Rust using LLVM, reusing existing tools saves time and allows the developers to focus their precious man-hours on the language itself and any language-specific tooling they require.<\/p>\n<p>When Walter Bright first started putting D together in 1999, that trend had not yet come around. However, he already had an existing toolchain in the form of the Digital Mars C and C++ compiler tools. So it was a no-brainer to make use of his existing tools and compiler backend and just focus on making a new frontend for DMD. There were four major side-effects of this decision, all of which had varying consequences in D&#8217;s future development.<\/p>\n<p>First, the DMC tools were Windows-only, so the early versions of DMD would be as well. Second, the linker, Optlink, only supports the OMF format. That meant that DMD\u2019s output would be incompatible with the more common <a href=\"https:\/\/en.wikipedia.org\/wiki\/COFF\">COFF<\/a> output of most modern C and C++ compilers on Windows. Third, the DMC tools do not support 64-bit, so DMD would be restricted to 32-bit output. Finally, Symantec had the legal rights to the existing backend, which meant their license would apply to DMD. While the frontend was open source, the backend license required one to get permission from Walter to distribute DMD (on a side note, this prevented DMD from being included in official Linux package repositories once Linux support was added, but <a href=\"http:\/\/forum.dlang.org\/thread\/oc8acc$1ei9$1@digitalmars.com\">Symantec granted permission<\/a> to relicense the backend earlier this year and it is now freely distributable under the Boost license).<\/p>\n<p><a href=\"http:\/\/www.digitalmars.com\/d\/1.0\/changelog1.html#new000\">DMD 0.00<\/a> was released in December of 2001. The <a href=\"http:\/\/www.digitalmars.com\/d\/1.0\/changelog1.html#new063\">0.63 release<\/a> brought Linux support in May of 2003. Walter could have based the Linux version on the GCC backend, but as a business owner, and through a caution born from past experience, he was concerned about any legal issues that could arise from his working with GPL code on one platform and maintaining a proprietary backend on another. Instead, he modified the DMD backend to generate ELF objects and hand them off to the GCC tools. This decision to enhance the backend became the approach for all new formats going forward. He did the same when adding support for Mac OS X: he modified the backend to work with the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Mach-O\">Mach-O object format<\/a>.<\/p>\n<p>Along with the new formats, the compiler gained the ability to generate 64-bit binaries everywhere except Windows. In order to interface with C on Windows, it was usually necessary to convert COFF object files and static libraries to OMF, to use a tool like <a href=\"http:\/\/www.digitalmars.com\/ctg\/coffimplib.html\">coffimplib<\/a> to generate DLL import libraries in the OMF format, or to create dynamic bindings and load DLLs manually via <code>LoadLibrary<\/code> and <code>GetProcAddress<\/code>. Then Remedy Games decided to use D.<\/p>\n<p><a href=\"https:\/\/www.remedygames.com\/games\/quantumbreak\/\">Quantum Break<\/a> was the first AAA game title <a href=\"https:\/\/dconf.org\/2016\/talks\/watson.html\">to ship with D<\/a> as part of its development process. Remedy used it for their gameplay code, creating their own <a href=\"https:\/\/github.com\/Remedy-Entertainment\/binderoo\">open source tool<\/a> to <a href=\"https:\/\/dconf.org\/2017\/talks\/watson.html\">bind with<\/a> their C++ game engine. Before they could get that far, however, they needed 64-bit support in DMD on Windows. That was the motivator to get it implemented. It took a while (apparently, there are some undocumented quirks in <a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/ms680547(v=vs.85).aspx\">Microsoft&#8217;s variant of COFF<\/a>, a.k.a PECOFF, a.k.a. MS-COFF), but Walter eventually got it done, and support for 32-bit COFF along with it. Again, as he had on other platforms, he modified the backend to generate object files in the new format.<\/p>\n<p>This is why it\u2019s necessary to have the Microsoft toolchain installed in order to produce 64-bit binaries with DMD on Windows. Microsoft\u2019s <code>cl<\/code> is as close to a <em>system compiler<\/em> as one is going to get on Windows. There is, however, an option that has not yet been fully explored. It&#8217;s <a href=\"http:\/\/mingw-w64.org\/doku.php\">a toolchain<\/a> that can be freely distributed, packaged with <a href=\"https:\/\/nuwen.net\/mingw.html\">a reasonable download size<\/a>, supports 32-bit and 64-bit output, and is mostly compatible with PECOFF. There is a possibility that it may be investigated as an option for future DMD releases to be built upon.<\/p>\n<h3 id=\"goingfromhere\">Going from here<\/h3>\n<p>Now that this primer is out of the way, the short series on C is just about ready to go. It will kick off with a brief summary of existing material, showing how easy it is to get D and C to work together in the general case. That will be followed up by two posts on arrays and strings. This is where most of the gotchas come into play, and anyone using D and C in the same program should understand what they are and how to avoid them.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Before diving into a series about C and D, a bit of a primer is called for. That\u2019s where this post comes in. The primary goal is to help ensure a C environment is installed and working on Windows. It\u2019s also useful to understand why things are different on that platform than on the others. Before we get to the why, we&#8217;ll dig into the how.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[26,12],"tags":[],"_links":{"self":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1148"}],"collection":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/comments?post=1148"}],"version-history":[{"count":58,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1148\/revisions"}],"predecessor-version":[{"id":1372,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1148\/revisions\/1372"}],"wp:attachment":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/media?parent=1148"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/categories?post=1148"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/tags?post=1148"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}