{"id":1026,"date":"2017-08-01T14:29:27","date_gmt":"2017-08-01T14:29:27","guid":{"rendered":"http:\/\/dlang.org\/blog\/?p=1026"},"modified":"2021-10-08T11:08:31","modified_gmt":"2021-10-08T11:08:31","slug":"a-dub-case-study-compiling-dmd-as-a-library","status":"publish","type":"post","link":"https:\/\/dlang.org\/blog\/2017\/08\/01\/a-dub-case-study-compiling-dmd-as-a-library\/","title":{"rendered":"A DUB Case Study: Compiling DMD as a Library"},"content":{"rendered":"<p><em>In his day job, Jacob Carlborg\u00a0is a Ruby backend developer for\u00a0<a href=\"http:\/\/www.derivco.se\">Derivco\u00a0Sweden<\/a>, but he&#8217;s been using D on his own time since 2006. He is the maintainer of numerous open source projects, including <a href=\"https:\/\/github.com\/jacob-carlborg\/dstep\">DStep<\/a>, a utility that generates D bindings from\u00a0C and Objective-C headers, <a href=\"https:\/\/github.com\/d-widget-toolkit\">DWT<\/a>, a port of the Java GUI library <a href=\"https:\/\/www.eclipse.org\/swt\/\">SWT<\/a>, and\u00a0<\/em><em><a href=\"https:\/\/github.com\/jacob-carlborg\/dvm\">DVM<\/a><\/em><em>, the topic of another post <a href=\"http:\/\/dlang.org\/blog\/2016\/08\/17\/inside-d-version-manager\/\">on this blog<\/a>. He implemented native Thread Local Storage support for DMD on OS X and contributed, along with Michel Fortin, to <a href=\"https:\/\/dlang.org\/spec\/objc_interface.html\">the integration of Objective-C<\/a>\u00a0in D.<\/em><\/p>\n<hr \/>\n<p><a href=\"https:\/\/code.dlang.org\/getting_started\"><img loading=\"lazy\" class=\"alignright size-full wp-image-181\" src=\"http:\/\/dlang.org\/blog\/wp-content\/uploads\/2016\/08\/d6.png\" alt=\"\" width=\"200\" height=\"200\" \/>DUB<\/a> is the official build tool and package manager for the D programming language. Originally written and currently maintained by S\u00f6nke Ludwig as part of the <a href=\"http:\/\/vibed.org\/\">vibe.d<\/a> web framework, its acceptance as an official part of the D toolchain means it is now shipping with the most recent DMD and LDC compilers.<\/p>\n<h3>A Quick Introduction to DUB<\/h3>\n<p>If you have have the latest DMD or LDC installed, you already have DUB installed as well. If not, or if you want to check for a more recent version, you can get the very latest release, beta or release candidate from the <a href=\"https:\/\/code.dlang.org\/download\">DUB download page<\/a>.<\/p>\n<p>You can create a new DUB project by executing the <code>dub init<\/code> command. This will start an interactive setup that guides you through project creation.<\/p>\n<ol>\n<li>First decide the format of the package recipe. Two formats are supported:\u00a0<a href=\"https:\/\/code.dlang.org\/package-format?lang=json\">JSON<\/a> and <a href=\"https:\/\/code.dlang.org\/package-format?lang=sdl\">SDLang<\/a>. Here we picked SDLang.<\/li>\n<li>Then specify the name of the project. Press enter to use the default name, which is displayed in brackets and is inferred from the directory<\/li>\n<li>Do the same for the description, author, license, copyright, and dependencies to select the default values<\/li>\n<\/ol>\n<pre class=\"prettyprint lang-plain_text\">$ dub init foo\r\nPackage recipe format (sdl\/json) [json]: sdl\r\nName [foo]:\r\nDescription [A minimal D application.]:\r\nAuthor name [Jacob Carlborg]:\r\nLicense [proprietary]:\r\nCopyright string [Copyright \u00a9 2017, Jacob Carlborg]:\r\nAdd dependency (leave empty to skip) []:\r\nSuccessfully created an empty project in '\/Users\/jacob\/tmp\/foo'.\r\nPackage successfully created in foo\r\n<\/pre>\n<p>After the setup has completed, the following files and directories will have been created:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ tree foo\r\nfoo\r\n\u251c\u2500\u2500 dub.sdl\r\n\u2514\u2500\u2500 source\r\n    \u2514\u2500\u2500 app.d\r\n\r\n1 directory, 2 files<\/pre>\n<ul>\n<li><code>dub.sdl<\/code> is the package recipe file, which provides instructions telling DUB how to build the package<\/li>\n<li><code>source<\/code> is the default path where DUB looks for D source files<\/li>\n<li><code>app.d<\/code> contains the <code>main<\/code> function and is an example Hello World generated by DUB with the following content:<\/li>\n<\/ul>\n<pre class=\"prettyprint lang-d\">import std.stdio;\r\n\r\nvoid main()\r\n{\r\n\twriteln(\"Edit source\/app.d to start your project.\");\r\n}<\/pre>\n<p>The content of the <code>dub.sdl<\/code> file is the following:<\/p>\n<pre class=\"prettyprint lang-plain_text\">name \"foo\"\r\ndescription \"A minimal D application.\"\r\nauthors \"Jacob Carlborg\"\r\ncopyright \"Copyright \u00a9 2017, Jacob Carlborg\"\r\nlicense \"proprietary\"<\/pre>\n<p>All of which was taken from what we specified during project creation. By default, DUB looks for D source files in either <code>source<\/code> or <code>src<\/code> directories and compiles all files it finds there and in any subdirectories.<\/p>\n<p>To build and run the application, navigate to the project&#8217;s root directory, <code>foo<\/code> in this case, and invoke <code>dub<\/code>:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dub\r\nPerforming \"debug\" build using dmd for x86_64.\r\nfoo ~master: building configuration \"application\"...\r\nLinking...\r\nRunning .\/foo\r\nEdit source\/app.d to start your project.<\/pre>\n<p>To build without running, invoke <code>dub build<\/code>:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dub build\r\nPerforming \"debug\" build using dmd for x86_64.\r\nfoo ~master: building configuration \"application\"...\r\nLinking...<\/pre>\n<h3>Case Study: DMD as a Library<\/h3>\n<p>Recently there has been some progress in making the D compiler (DMD) available as a library. Razvan Nitu has been working on it as part of his D Foundation scholarship at the University Politechnica of Bucharest. He gave <a href=\"http:\/\/dconf.org\/2017\/talks\/nitu.html\">a presentation at DConf 2017<\/a> (a\u00a0<a href=\"https:\/\/youtu.be\/tK072jcoWv4?list=PL3jwVPmk_PRxo23yyoc0Ip_cP3-rCm7eB\">video<\/a>\u00a0of the talk is available, as well as <a href=\"https:\/\/github.com\/dlang\/dmd\/tree\/master\/src\/examples\">examples<\/a> in the DMD repository). So I had the idea that as part of the <a href=\"http:\/\/dconf.org\/2017\/index.html\">DConf 2017<\/a> hackathon I could create a simple DUB package for DMD to make only the lexer and the parser available as a library, something his work has made possible.<\/p>\n<p>Currently DMD is built using <code>make<\/code>. There are three Makefiles, one for Posix, one for 32-bit Windows and one for 64-bit Windows \u00a0(which is only a wrapper of the 32-bit one). I don&#8217;t intend to try to completely replicate the Makefiles as a DUB package (they contain some additional tasks besides building the compiler), but instead will start out fresh and only include what&#8217;s necessary to build the lexer and parser.<\/p>\n<p>DMD already has all the source code in the <code>src<\/code> directory, which is one of the directories DUB searches by default. If we would leave it as is, DUB would include the entirety of DMD, including the backend and other parts we don&#8217;t want to include at this point.<\/p>\n<p>The first step is to create the DUB package recipe file. We start simple with only the metadata (here using the SDLang format):<\/p>\n<pre class=\"prettyprint lang-plain_text\">name \"dmd\"\r\ndescription \"The DMD compiler\"\r\nauthors \"Walter Bright\"\r\ncopyright \"Copyright \u00a9 1999-2017, Digital Mars\"\r\nlicense \"BSL-1.0\"<\/pre>\n<p>When we have this we need to figure out which files to include in the package. We can do this by invoking DMD with the <code>-deps flag<\/code> to generate the imports of a module. A good start is the lexer, which is located in <a href=\"https:\/\/github.com\/dlang\/dmd\/blob\/master\/src\/ddmd\/lexer.d\"><code>src\/ddmd\/lexer.d<\/code><\/a>. We run the following command to output the imports that <code>lexer.d<\/code> is using:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dmd -deps=deps.txt -o- -Isrc src\/ddmd\/lexer.d<\/pre>\n<p>This will write a file named <code>deps.txt<\/code> containing all the imports used by <code>lexer.d<\/code>. The <code>-o-<\/code> flag is used to tell the compiler not to generate any code. The <code>-I<\/code> flag is used to add an import path where the compiler will look for additional modules to import (but not compile). An example of the output looks like this (the long path names have been reduced to save space):<\/p>\n<pre class=\"prettyprint lang-plain_text\">core.attribute (druntime\/import\/core\/attribute.d) : private : object (druntime\/import\/object.d)\r\nobject (druntime\/import\/object.d) : public : core.attribute (druntime\/import\/core\/attribute.d):selector\r\nddmd.lexer (ddmd\/lexer.d) : private : object (druntime\/import\/object.d)\r\ncore.stdc.ctype (druntime\/import\/core\/stdc\/ctype.d) : private : object (druntime\/import\/object.d)\r\nddmd.root.array (ddmd\/root\/array.d) : private : object (druntime\/import\/object.d)\r\nddmd.root.array (ddmd\/root\/array.d) : private : core.stdc.string (druntime\/import\/core\/stdc\/string.d)<\/pre>\n<p>The most interesting part of this output, in this case, is the first column, which consists of a long list of module names. What we are interested in here is a unique list of modules that are located in the <code>ddmd<\/code> package. All modules in the <code>core<\/code> package are part of the D runtime and are already precompiled as a library and automatically linked when compiling a D executable, so these modules don&#8217;t need to be compiled. The modules from the <code>ddmd<\/code> package can be extracted with some search-and-replace in your favorite text editor or using some standard Unix command lines tools:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ cat deps.txt | cut -d ' ' -f 1 | grep ddmd | sort | uniq\r\nddmd.console\r\nddmd.entity\r\nddmd.errors\r\nddmd.globals\r\nddmd.id\r\nddmd.identifier\r\nddmd.lexer\r\nddmd.root.array\r\nddmd.root.ctfloat\r\nddmd.root.file\r\nddmd.root.filename\r\nddmd.root.hash\r\nddmd.root.outbuffer\r\nddmd.root.port\r\nddmd.root.rmem\r\nddmd.root.rootobject\r\nddmd.root.stringtable\r\nddmd.tokens\r\nddmd.utf<\/pre>\n<p>Here we can see that a set of modules is located in the nested package <code>ddmd.root<\/code>. This package contains common functionality used throughout the DMD source code. Since it doesn&#8217;t have any dependencies on any code outside the package it&#8217;s a good fit to place in a DUB <code>subpackage<\/code>. This can be done using the <code>subPackage<\/code> directive, as follows:<\/p>\n<pre class=\"prettyprint lang-plain_text\">subPackage {\r\n  name \"root\"\r\n  targetType \"library\"\r\n  sourcePaths \"src\/ddmd\/root\"\r\n}<\/pre>\n<p>We specify the name of the subpackage, <code>root<\/code>. The <code>targetType<\/code> directive is used to tell DUB whether it should build an executable or a library (though it&#8217;s optional &#8212; DUB will build an executable if it finds an <code>app.d<\/code> in the root of the source directory and a library if it doesn&#8217;t). Finally, <code>sourcePaths<\/code> can be used to specify the paths where DUB should look for the D source files if neither of the default directories is used. Fortunately, we want to include all the files in the <code>src\/ddmd\/root<\/code>, so using <code>sourcePaths<\/code> works perfectly fine.<\/p>\n<p>We can verify that the subpackage works and builds by invoking:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dub build :root\r\nBuilding package dmd:root in \/Users\/jacob\/development\/d\/dlang\/dmd\/\r\nPerforming \"debug\" build using dmd for x86_64.\r\ndmd:root ~master: building configuration \"library\"...<\/pre>\n<p><code>:package-name<\/code> is shorthand that tells DUB to build the <code>package-name<\/code> subpackage of the current package, in our case the <code>root<\/code> subpackage.<\/p>\n<p>After removing all the modules from the <code>root<\/code> package from the initial list of dependencies, the following modules remain:<\/p>\n<pre class=\"prettyprint lang-plain_text\">ddmd.console\r\nddmd.entity\r\nddmd.errors\r\nddmd.globals\r\nddmd.id\r\nddmd.identifier\r\nddmd.lexer\r\nddmd.tokens\r\nddmd.utf<\/pre>\n<p>The next step is to create a subpackage for the lexer containing the remaning modules.<\/p>\n<pre class=\"prettyprint lang-plain_text\">subPackage {\r\n  name \"lexer\"\r\n  targetType \"library\"\r\n  sourcePaths<\/pre>\n<p>Again we start by specifying the name of the subpackage and that the target type is a library. Specifying <code>sourcePaths<\/code> without any value will set it to an empty list, i.e. no source paths. This is done because there are more files than we want to include in this subpackage in the source directory.<\/p>\n<pre class=\"prettyprint lang-plain_text\">sourceFiles \\\r\n    \"src\/ddmd\/console.d\" \\\r\n    \"src\/ddmd\/entity.d\" \\\r\n    \"src\/ddmd\/errors.d\" \\\r\n    \"src\/ddmd\/globals.d\" \\\r\n    \"src\/ddmd\/id.d\" \\\r\n    \"src\/ddmd\/identifier.d\" \\\r\n    \"src\/ddmd\/lexer.d\" \\\r\n    \"src\/ddmd\/tokens.d\" \\\r\n    \"src\/ddmd\/utf.d\"<\/pre>\n<p>The above specifies all source files that should be included in this subpackage. The difference between <code>sourcePaths<\/code> and <code>sourceFiles<\/code> is that <code>sourcePaths<\/code> expects a whole directory of source files that should be included, where <code>sourceFiles<\/code> lists only the individual files that should be included. A list in SDLang is written by separating the items with a space. The backslash (<code>\\<\/code>) is used for line continuation, making it possible spread the list across multiple lines.<\/p>\n<p>The final step of the <code>lexer<\/code> subpackage is to add a dependency on the <code>root<\/code> subpackage. This is done with the <code>dependency<\/code> directive:<\/p>\n<pre class=\"prettyprint lang-plain_text\">dependency \"dmd:root\" version=\"*\"\r\n}<\/pre>\n<p>The first parameter for the <code>dependency<\/code> directive is the name of another DUB package. The colon is used to separate the package name from the subpackage name. The <code>version<\/code> attribute is used to specify which version the package should depend on. The <code>*<\/code> is used to indicate that any version of the dependency matches, i.e. the latest version should always be used. When implementing subpackages in any given package, this is generally what should be used. External projects that depend on any DUB package should specify a <a href=\"http:\/\/semver.org\/\">SemVer<\/a> version number corresponding to a known release version.<\/p>\n<p>If we build the <code>lexer<\/code> subpackage now it will result in an error:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dub build :lexer\r\nBuilding package dmd:lexer in \/Users\/jacob\/development\/d\/dlang\/dmd\/\r\nPerforming \"debug\" build using dmd for x86_64.\r\ndmd:lexer ~master: building configuration \"library\"...\r\nsrc\/ddmd\/globals.d(339,21): Error: need -Jpath switch to import text file VERSION\r\ndmd failed with exit code 1.<\/pre>\n<p>Looking at the file and line of the error shows that it contains the following code:<\/p>\n<pre class=\"prettyprint lang-plain_text\">_version = (import(\"VERSION\") ~ '\\0').ptr;<\/pre>\n<p>This code contains an <a href=\"http:\/\/dlang.org\/spec\/expression.html#import_expressions\">import expression<\/a>. Import expressions differ from import statements (e.g. <code>import std.stdio;<\/code>) in that they take a file from the file system and insert its contents into the current module. It&#8217;s just as if you copied and pasted the contents yourself. Using an import expression requires that the path where the file is imported from be passed to the compiler as a security mechanism. This can be done using the <code>-J<\/code> flag. In this case, we want to use the package root, where we are executing DUB, so we can use a single dot: &#8220;<code>.<\/code>&#8220;. Passing arbitrary flags to the compiler can be done with the <code>dflags<\/code> build setting, as follows:<\/p>\n<pre class=\"prettyprint lang-plain_text\">dflags \"-J.\"<\/pre>\n<p>Add that to the <code>lexer<\/code> subpackage configuration and it will compile correctly:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dub build :lexer\r\nBuilding package dmd:lexer in \/Users\/jacob\/development\/d\/dlang\/dmd\/\r\nPerforming \"debug\" build using dmd for x86_64.\r\ndmd:lexer ~master: building configuration \"library\"...<\/pre>\n<p>For the final subpackage, we have the parser. The parser is located in <a href=\"https:\/\/github.com\/dlang\/dmd\/blob\/master\/src\/ddmd\/parse.d\"><code>src\/ddmd\/parse.d<\/code><\/a>. To get its dependencies we can use the same approach we used for the lexer. But we will filter out all files that are part of the other subpackages:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dmd -deps=deps.txt -Isrc -J. -o- src\/ddmd\/parse.d\r\n$ cat deps.txt | cut -d ' ' -f 1 | grep ddmd | grep -E -v '(root|console|entity|errors|globals|id|identifier|lexer|tokens|utf)' | sort | uniq\r\nddmd.parse<\/pre>\n<p>Here, we&#8217;re supplying the <code>-v<\/code> flag to <code>grep<\/code> to filter the results and the <code>-E<\/code> flag to enable extended regular expressions. All modules from the <code>root<\/code> package and all modules from the <code>lexer<\/code> subpackage are filtered out and the only remaining module is the <code>ddmd.parse<\/code> module.<\/p>\n<p>The subpackage for the parser will look similar to the other subpackages:<\/p>\n<pre class=\"prettyprint lang-plain_text\">subPackage {\r\n  name \"parser\"\r\n  targetType \"library\"\r\n  sourcePaths\r\n\r\n  sourceFiles \"src\/ddmd\/parse.d\"\r\n\r\n  dependency \"dmd:lexer\" version=\"*\"\r\n}<\/pre>\n<p>Again, we can verify that it&#8217;s working by building the subpackage:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dub build :parser\r\nBuilding package dmd:parser in \/Users\/jacob\/development\/d\/dlang\/dmd\/\r\nPerforming \"debug\" build using dmd for x86_64.\r\ndmd:parser ~master: building configuration \"library\"...<\/pre>\n<p>Currently we have three subpackages in the DUB recipe file, but no way to use the main package as a whole. To fix this we add the <code>parser<\/code> subpackage as a dependency of the main package. We pick the <code>parser<\/code> subpackage as a dependency because it will include the other two subpackages through its own dependencies.<\/p>\n<pre class=\"prettyprint lang-plain_text\">license \"BSL-1.0\"\r\n\r\ntargetType \"none\"\r\ndependency \":parser\" version=\"*\"\r\n\r\nsubPackage {\r\n  name \"root\"<\/pre>\n<p>In addition to specifying <code>parser<\/code> as a dependency, we also specify the target type to be <code>none<\/code>. This will avoid building an empty library out of the main package, since it doesn&#8217;t contain any source files of its own.<\/p>\n<p>As a final step, we&#8217;ll verify that the whole library is working by creating a separate project that uses the DMD DUB package as a dependency. We create a new DUB project in the <code>test<\/code> directory, called <code>dub_package<\/code>:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ cd test\r\n$ mkdir dub_package\r\n$ cd dub_package\r\n$ cat &gt; dub.sdl &lt;&lt;EOF\r\n&gt; name \"dmd-dub-test\"\r\n&gt; description \"Test of the DMD Dub package\"\r\n&gt; license \"BSL 1.0\"\r\n&gt;\r\n&gt; dependency \"dmd\" path=\"..\/..\/\"\r\n&gt; EOF\r\n$ mkdir source<\/pre>\n<p>We create a new file, <code>source\/app.d<\/code>, with the following content:<\/p>\n<pre class=\"prettyprint lang-d\">void main()\r\n{\r\n}\r\n\r\n\/\/ lexer\r\nunittest\r\n{\r\n    import ddmd.lexer;\r\n    import ddmd.tokens;\r\n\r\n    immutable expected = [\r\n        TOKvoid,\r\n        TOKidentifier,\r\n        TOKlparen,\r\n        TOKrparen,\r\n        TOKlcurly,\r\n        TOKrcurly\r\n    ];\r\n\r\n    immutable sourceCode = \"void test() {} \/\/ foobar\";\r\n    scope lexer = new Lexer(\"test\", sourceCode.ptr, 0, sourceCode.length, 0, 0);\r\n    lexer.nextToken;\r\n\r\n    TOK[] result;\r\n\r\n    do\r\n    {\r\n        result ~= lexer.token.value;\r\n    } while (lexer.nextToken != TOKeof);\r\n\r\n    assert(result == expected);\r\n}\r\n\r\n\/\/ parser\r\nunittest\r\n{\r\n    import ddmd.astbase;\r\n    import ddmd.parse;\r\n\r\n    scope parser = new Parser!ASTBase(null, null, false);\r\n    assert(parser !is null);\r\n}<\/pre>\n<p>The above file contains two unit tests, one for the lexer and one for the parser. We can run <code>dub test<\/code> to run the unit tests for this package:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dub test\r\nNo source files found in configuration 'library'. Falling back to \"dub -b unittest\".\r\nPerforming \"unittest\" build using dmd for x86_64.\r\ndmd:root ~issue-17392-dub: building configuration \"library\"...\r\ndmd:lexer ~issue-17392-dub: building configuration \"library\"...\r\n..\/..\/src\/ddmd\/globals.d(339,21): Error: file \"VERSION\" cannot be found or not in a path specified with -J\r\ndmd failed with exit code 1.<\/pre>\n<p>Which gives us the error that it cannot find the <code>VERSION<\/code> file in any string import paths, even though we added the correct directory to the string import paths. If we run the tests with verbose output enabled, using the <code>--verbose<\/code> flag we get a hint (the output has been reduced to save space):<\/p>\n<pre class=\"prettyprint lang-plain_text\">dmd:lexer ~issue-17392-dub: building configuration \"library\"...\r\ndmd -J. -lib<\/pre>\n<p>Here we see that the compiler is invoked with the <code>-J<\/code>. flag, which is what we previously specified in the <code>lexer<\/code> subpackage. The problem is that the current directory is now of the <code>dmd-dub-test<\/code> DUB package instead of the <code>dmd<\/code> DUB package. Looking at the documentation of DUB we can see there&#8217;s an environment variable, <a href=\"http:\/\/code.dlang.org\/package-format?lang=sdl#environment-variables\"><code>$PACKAGE_DIR<\/code><\/a>, that we can use as the string import path instead of hardcoding it to use a single dot. We update the dflags setting of the <code>lexer<\/code> subpackage to use the <code>$PACKAGE_DIR<\/code> environment variable:<\/p>\n<pre class=\"prettyprint lang-plain_text\">dflags \"-J$PACKAGE_DIR\"\r\n}<\/pre>\n<p>Running the tests again shows that the error is fixed, but now we get a new error, a long list of undefined symbols (shortened here):<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dub test\r\nNo source files found in configuration 'library'. Falling back to \"dub -b unittest\".\r\nPerforming \"unittest\" build using dmd for x86_64.\r\ndmd:root ~issue-17392-dub: building configuration \"library\"...\r\ndmd:lexer ~issue-17392-dub: building configuration \"library\"...\r\ndmd:parser ~issue-17392-dub: building configuration \"library\"...\r\ndmd-dub-test ~master: building configuration \"application\"...\r\nLinking...\r\nUndefined symbols for architecture x86_64:\r\n  \"_D4ddmd7astbase12__ModuleInfoZ\", referenced from:\r\n      _D3app12__ModuleInfoZ in dmd-dub-test.o<\/pre>\n<p>The reason for this is that we&#8217;re importing the <code>ddmd.astbase<\/code> module in the test of the parser, but it&#8217;s never compiled. We can solve that problem by adding it to the <code>parser<\/code> subpackage in the <code>dmd<\/code> DUB package. Running <code>dmd<\/code> again to show all its dependencies shows that it also depends on the <code>ddmd.astbasevisitor<\/code> module. We add these two modules as follows:<\/p>\n<pre class=\"prettyprint lang-plain_text\">sourceFiles \\\r\n  \"src\/ddmd\/astbase.d\" \\\r\n  \"src\/ddmd\/astbasevisitor.d\" \\\r\n  \"src\/ddmd\/parse.d\"<\/pre>\n<p>Finally, running the tests again shows that everything is working correctly:<\/p>\n<pre class=\"prettyprint lang-plain_text\">$ dub test\r\nNo source files found in configuration 'library'. Falling back to \"dub -b unittest\".\r\nPerforming \"unittest\" build using dmd for x86_64.\r\ndmd:root ~issue-17392-dub: building configuration \"library\"...\r\ndmd:lexer ~issue-17392-dub: building configuration \"library\"...\r\ndmd:parser ~issue-17392-dub: building configuration \"library\"...\r\ndmd-dub-test ~master: building configuration \"application\"...\r\nLinking...\r\nRunning .\/dmd-dub-test<\/pre>\n<p>After verifying that both the lexer and parser are working in a separate DUB package, this is the final result of the package recipe for the <code>dmd<\/code> DUB package:<\/p>\n<pre class=\"prettyprint lang-plain_text\">name \"dmd\"\r\ndescription \"The DMD compiler\"\r\nauthors \"Walter Bright\"\r\ncopyright \"Copyright \u00a9 1999-2017, Digital Mars\"\r\nlicense \"BSL-1.0\"\r\n\r\ntargetType \"none\"\r\ndependency \":parser\" version=\"*\"\r\n\r\nsubPackage {\r\n  name \"root\"\r\n  targetType \"library\"\r\n  sourcePaths \"src\/ddmd\/root\"\r\n}\r\n\r\nsubPackage {\r\n  name \"lexer\"\r\n  targetType \"library\"\r\n  sourcePaths\r\n\r\n  sourceFiles \\\r\n    \"src\/ddmd\/console.d\" \\\r\n    \"src\/ddmd\/entity.d\" \\\r\n    \"src\/ddmd\/errors.d\" \\\r\n    \"src\/ddmd\/globals.d\" \\\r\n    \"src\/ddmd\/id.d\" \\\r\n    \"src\/ddmd\/identifier.d\" \\\r\n    \"src\/ddmd\/lexer.d\" \\\r\n    \"src\/ddmd\/tokens.d\" \\\r\n    \"src\/ddmd\/utf.d\"\r\n\r\n  dflags \"-J$PACKAGE_DIR\"\r\n\r\n  dependency \"dmd:root\" version=\"*\"\r\n}\r\n\r\nsubPackage {\r\n  name \"parser\"\r\n  targetType \"library\"\r\n  sourcePaths\r\n\r\n  sourceFiles \\\r\n    \"src\/ddmd\/astbase.d\" \\\r\n    \"src\/ddmd\/astbasevisitor.d\" \\\r\n    \"src\/ddmd\/parse.d\"\r\n\r\n  dependency \"dmd:lexer\" version=\"*\"\r\n}<\/pre>\n<p>All this has now been merged into <a href=\"https:\/\/github.com\/dlang\/dmd\/commit\/196760c92aeb05fd1df9789214a1d5805c004d03\">master<\/a> and the DUB package is available here: <a href=\"http:\/\/code.dlang.org\/packages\/dmd\">http:\/\/code.dlang.org\/packages\/dmd<\/a>. Happy hacking!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In his day job, Jacob Carlborg\u00a0is a Ruby backend developer for\u00a0Derivco\u00a0Sweden, but he&#8217;s been using D on his own time since 2006. He is the maintainer of numerous open source projects, including DStep, a utility that generates D bindings from\u00a0C and Objective-C headers, DWT, a port of the Java GUI library SWT, and\u00a0DVM, the topic [&hellip;]<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[26,12,9],"tags":[],"_links":{"self":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1026"}],"collection":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/comments?post=1026"}],"version-history":[{"count":6,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1026\/revisions"}],"predecessor-version":[{"id":1032,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1026\/revisions\/1032"}],"wp:attachment":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/media?parent=1026"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/categories?post=1026"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/tags?post=1026"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}