Symmetry Autumn of Code is Underway

Earlier this year, Laeeth Isharc brought an idea to the D Foundation for Symmetry Investments to sponsor a summer of code. He was eager to provide a few motivated individuals the incentive to get some great work done for D and the D community. Sporadic email discussions preceded some chatting at DConf 2018 Munich and the idea subsequently began to pick up steam. By the time the details were sketched out, it had transformed into the Symmetry Autumn of Code.

The SAoC projects

Eight applicants submitted their resumes and project proposals for three slots. Nearly all of the proposals were taken from the SAoC suggestions page at the D Wiki. Given the limited window, the selection process went fairly quick. Of the three selected, two had mentors attached. It took a little while to find a mentor for the third selection, so we extended the milestone deadline for that participant. Now I can happily say that all three are well underway.

A fork-based concurrent GC for DRuntime

Francesco Mecca proposed this project. The goal is to take Leandro Lucarella’s original D1 fork-based GC, port it to D2, adapt it to DRuntime’s GC interface, and culminate with a pull request for DRuntime to present the work and open discussion. Leandro agreed to mentor this project and is working with Francesco to develop a test suite that the port must pass as part of Milestone 2. They have also included documentation in their milestone list, which is good news.

vibe.d HTTP/2 implementation

This one came from Francesco Galla, who is currently pursuing a MSc in Network and Security. The goal here is to enhance the vibe-http library to support HTTP/2. Fittingly, Sönke Ludwig, the maintainer of vibe.d, agreed to mentor the project, but requested someone to share the load due to his schedule. Sebastian Wilzbach stepped up as co-mentor.

This project involves rewriting the current HTTP/1 API, ensuring it works as expected, then incrementally adding support for HTTP/2. Portions of the rewrite were already completed before SAoC came along, but had not yet been tested. As such, testing and bug fixing will be a significant portion of the first milestone.

Porting the Mago debugger to D

This project was proposed by László Szerémi. He’s a heavy user of debuggers in D and wants to see the situation improved. He believes that porting the Mago debugger to D is a major step in that direction. His first two milestones are concerned with translating, testing, and bug fixing. To cap off the SAoC event, he intends to get a GUI frontend up and running with some basic features.

László had no mentor when he applied, and no one had specifically volunteered to mentor any debugger projects, so we put out a call in the forums and I reached out directly to a few people. In the end, Stefan Koch agreed to take it on.

SAoC is not the end

I know that one of the goals Laeeth had behind his initial suggestion for this event was to enhance the D ecosystem. None of the selected projects are simple, one-shot tasks. These are projects which will all require attention, care, and effort beyond SAoC. The participants are getting them started and we hope they’ll continue to maintain them for some time to come, but in the end, these are projects for the community. When SAoC 2018 is behind us, it will ultimately be up to the community to determine if the projects live long and prosper or die young.

I’ll post more about SAoC and the participants as the event goes on. We wish them the best in meeting their milestones!

DMD 2.082.0 Released

DMD 2.082.0 was released over the weekend. There were 28 major changes and 76 closed Bugzilla issues in this release, including some very welcome improvements in the toolchain. Head over to the download page to pick up the official package for your platform and visit the changelog for the details.

Tooling improvements

While there were several improvements and fixes to the compiler, standard library, and runtime in this release, there were some seemingly innocuous quality-of-life changes to the tooling that are sure to be greeted with more enthusiasm.

DUB gets dubbier

DUB, the build tool and package manager for D that ships with DMD, received a number  of enhancements, including better dependency resolution, variable support in the build settings, and improved environment variable expansion.

Arguably the most welcome change will be the removal of the regular update check. Previously, DUB would check for dependency updates once a day before starting a project build. If there was no internet connection, or if there were any errors in dependency resolution, the process could hang for some time. With the removal of the daily check, upgrades will only occur when running dub upgrade in a project directory. Add to that the brand new --dry-run flag to get a list of any upgradable dependencies without executing the upgrades.

Signed binaries for Windows

For quite some time users of DMD on Windows have had the annoyance of seeing a warning from Windows Smartscreen when running the installer, and the occasional false positive from AntiVirus software when running DMD.

Now those in the Windows D camp can do a little victory dance, as all of the binaries in the distribution, including the installer, are signed with the D Language Foundation’s new code signing certificate. This is one more quality-of-life issue that can finally be laid to rest. On a side note, the cost of the certificate was the first expense entered into our Open Collective page.

Compiler and libraries

Many of the changes and updates in the compiler and library department are unlikely to compel anyone to shout from the rooftops, but a handful are nonetheless notable.

The compiler

One such is an expansion of the User-Defined Attribute syntax. Previously, these were only allowed on declarations. Now, they can be applied to function parameters:

// Previously, it was illegal to attach a UDA to a function parameter
void example(@(22) string param)
{
    // It's always been legal to attach UDAs to type, variable, and function declarations.
    @(11) string var;
    pragma(msg, [__traits(getAttributes, var)] == [11]);
    pragma(msg, [__traits(getAttributes, param)] == [22]);
}

Run this example online

The same goes for enum members (it’s not explicitly listed in the highlights at the top of the changelog, but is mentioned in the bugfix list):

enum Foo {
@(10) one,
@(20) two,
}

void main()
{
pragma(msg, [__traits(getAttributes, Foo.one)] == [10]);
pragma(msg, [__traits(getAttributes, Foo.two)] == [20]);
}

Run this example online

The DasBetterC subset of D is enhanced in this release with some improvements. It’s now possible to use array literals in initializers. Previously, array literals required the use of TypeInfo, which is part of DRuntime and therefore unavailable in -betterC mode. Moreover, comparing arrays of structs is now supported and comparing arrays of byte-sized types should no longer generate any linker errrors.

import core.stdc.stdio;
struct Sint
{
    int x;
    this(int v) { x = v;}
}

extern(C) void main()
{
    // No more TypeInfo error in this initializer
    Sint[6] a1 = [Sint(1), Sint(2), Sint(3), Sint(1), Sint(2), Sint(3)];
    foreach(si; a1) printf("%i\n", si.x);

    // Arrays/slices of structs can now be compared
    assert(a1[0..3] == a1[3..$]);

    // No more linker error when comparing strings, either explicitly
    // or implicitly such as in a switch.
    auto s = "abc";
    switch(s)
    {
        case "abc":
            puts("Got a match!");
            break;
        default:
            break;
    }

    // And the same goes for any byte-sized type
    char[6] a = [1,2,3,1,2,3];
    assert(a[0..3] >= a[3..$]);

    puts("All the asserts passed!");
}

Run this example online

DRuntime

Another quality-of-life fix, this one touching on the debugging experience, is a new run-time flag that can be passed to any D program compiled against the 2.082 release of the runtime or later, --DRT-trapException=0. This allows exception trapping to be disabled from the command line.

Previously, this was supported only via a global variable, rt_trapExceptions. To disable exception trapping, this variable had to be set to false before DRuntime gained control of execution, which meant implementing your own extern(C) main and calling _d_run_main to manually initialize DRuntime which, in turn, would run the normal D main—all of which is demonstrated in the Tip of the Week from the August 7, 2016, edition of This Week in D (you’ll also find there a nice explanation of why you might want to disable this feature. HINT: running in your debugger). A command-line flag is sooo much simpler, no?

Phobos

The std.array module has long had an array function that can be used to create a dynamic array from any finite range. With this release, the module gains a staticArray function that can do the same for static arrays, though it’s limited to input ranges (which includes other arrays). When the length of a range is not knowable at compile time, it must be passed as a template argument. Otherwise, the range itself can be passed as a template argument.

import std.stdio;
void main()
{
    import std.range : iota;
    import std.array : staticArray;

    auto input = 3.iota;
    auto a = input.staticArray!2;
    pragma(msg, is(typeof(a) == int[2]));
    writeln(a);
    auto b = input.staticArray!(long[4]);
    pragma(msg, is(typeof(b) == long[4]));
    writeln(b);
}

Run this example online

September pumpkin spice

Participation in the #dbugfix campaign for this cycle was, like last cycle, rather dismal. Even so, I’ll have an update on that topic later this month in a post of its own.

Three of eight applicants were selected for the Symmetry Autumn of Code, which officially kicked off on September 1. Stay tuned here for a post on that topic as well.

The blog has been quiet for a few weeks, but the gears are slowly and squeakily starting to grind again. Other posts lined up for this month include the next long-overdue installment in the GC Series and the launch of a new ‘D in Production’ profile.

Funding code-d

At the DConf 2018 Hackathon, I announced that the D Language Foundation would be raising money to further the development of the suite of tools comprising code-d, the Visual Studio Code plugin. The project is developed and maintained by Jan Jurzitza, a.k.a. Webfreak001.

The plan was that if we reached $3,000 in donations on our Open Collective page, we’d make the money available to Jan. When we finally reached the goal, I was quite happy to tweet about it:

Jan and I still hadn’t finalized the details at that point, but we finally got it all sorted a few days later. Before I show how we’ll be using the money, I’d first like to talk about why we’re doing this.

The motivation

As the D programming language community has grown, so has the amount of work that needs to be done. Since D is a community-driven language, much of the work that gets completed is done by volunteers. Unfortunately, the priorities for those who actively contribute aren’t necessarily aligned with those of everyone else, and this can lead to some users feeling that there is little progress at all, or that their concerns are being deliberately ignored. That’s just the nature of the beast. More potential pain points and unfulfilled wishlists means there’s a continual need for more manpower.

Therein lies one of the most fundamental issues that has plagued D’s development for years: how can more people be motivated to directly participate so we can keep pace with the consequences of progress?

To be clear, everyone involved in the core team is aware that lack of manpower is not the only negative side effect we suffer from growth, but it is certainly a big one. Part of my role these days is to figure out, along with Sebastian Wilzbach, ways to mitigate it. My inbox is littered with conversations exploring this topic, along with recommendations from Andrei to research how other projects handle one thing or another for inspiration.

To that end, in early February I launched the #dbugfix campaign with the goal of increasing community participation in the bug fixing process by providing a simple way for folks to bring attention to their pet peeve issues (keep it going people – participation has slowed and we need to pick it up!). We also announced the State of D Survey, initiated by Seb, at the end of February to get an idea of what some of the personal pain points are and where people think things are going swell. We’ve got other ideas in development, one of which I’ll be telling you about in the coming days.

Among the data that jumped out at us from the survey is that Visual Studio and Visual Studio Code were the most popular IDEs/editors among participants. The plugins for both Visual Studio (Visual D) and VS Code (code-d/serve-d/workspace-d) are each maintained by solo developers.

That’s a heavy workload for each of them. Look at the issues list for code-d and you’ll find several there (39 open/132 closed as I write), but the PR list is much shorter (1 open/21 closed as I write). What I infer from that is that the demand for improvements is higher than the supply for implementing them. And from personal experience, I know there are projects like code-d whose maintainers would love to add new features, or improve existing ones, but simply can’t (or aren’t motivated to) make the time for it.

Here then is a good place for a little experiment. What if the D Language Foundation started raising money to help fund the development of specific projects like code-d? By creating the opportunity for those with extra money but little time to directly contribute toward fixing their personal pain points and shrinking their wishlists, can development be focused toward getting bugs fixed and improvements implemented more quickly?

Long story short, I picked the VS Code plugin for our trial run because it was prominent in the survey data, it’s cross-platform, and the serve-d component can be beneficial to other IDE/editor plugins. Open Collective launched their goal system just as I was trying to determine how best to go about it, so it seemed the obvious choice (though we didn’t learn until after our announcement that it wasn’t the system we thought it was, i.e. it’s not a system of multiple, targeted fund raising goals but instead a milestone on the global donation progress bar at the top of our Open Collective page). I contacted Jan just before DConf to gauge his interest, then finally had a meeting with the Foundation board the night before the Hackathon to get approval and decide how to go about doling out the money (we decided the money should be paid out in increments based on completion of milestones, with the option to provide money as needed for targeted development costs, e.g. a Windows license, bug bounties, etc). It all came together fairly quickly.

We plan to continue this initiative and raise money for more work in the future. Some will be like this trial run, with the goal of motivating project maintainers to focus development and spend that extra hour or two a day to get things done. Some will be aimed at specific development tasks, as bounties or as payment for a single part-time or full-time developer. We’ll experiment and see what works.

We won’t be using Open Collective’s goals for this (or any other service) going forward. Instead, we’re going to add a page to the web site where we can define targets, allow donations through Open Collective or PayPal, and track donation progress. Each target will allow us to lay out exactly what the donations are being used for, so potential donors can see in advance where their money is going. We’ll be using the State of D Survey as a guide to begin with, but we’ll always be open to suggestions, and we’ll adapt to what works over what doesn’t as we go along.

As long as there are people willing to put either time or money into the D language, community, and ecosystem, this will surely pay off in the long run with more visible progress, hopefully solving a wider range of pain points than currently happens in our “what-I-want-to-work-on” system of community development. We’re looking forward to see the synergy that results from this and other initiatives going forward.

The milestones

Jan has some specific ideas about how he wants to improve his plugin tools and was happy to take this opportunity to put them all front and center. I asked him to make a list, then estimate how much time it would take to complete each task if he were working on it full time, then divide that list into roughly equal milestones of 3–4 weeks each. That does not mean he will be working full time, nor that each milestone will take 3–4 weeks; payment is not dependent on deadlines. It was simply a means of organizing his task list.

We ended up with two sets of tasks as a starting point and agreed to a $500 payout upon completion of both milestones. Once all the tasks in a milestone are completed, the $500 will be paid. At the end of the second milestone, we’ll determine what to do next – another milestone of feature improvements, a heavy round of bug fixing, or something else – and how to make use of the remaining $2000.

Following is a list of the tasks in both of the initial milestones, with a brief description of each directly from Jan (lightly edited).

Improvements & maintainence

  • add automated tests for windows 7, 10, archlinux, ubuntu, fedora, osx for startup, from scratch project initialization, reloading, adding dependencies
    There are automated tests in the workspace-d repo which test different scenarios and combinations with dub, DCD, and d-scanner, ranging from general usage to weird setups. Identifying more issues (especially on windows), creating more tests and especially running all the tests on several platforms would be a good first step towards a less buggy version of serve-d which should work on all platforms equally well.
  • Single file mode
    A requested feature and one which makes the usage of the plugin a lot easier would be single file usage. Currently, single file usage is very limited and without an open workspace it doesn’t work at all. Adding this (with quick startup time and low resource usage) will make code-d a good alternative to vim or notepad for quickly editing a D file.
  • fix unknown DDoc macro parsing
    Currently, if any unknown DDoc macros are used, they are omitted from the documentation renderer, which makes some documentation unreadable and weird. For this, libddoc needs to be worked on by identifying where exactly the macros are discarded, how they could be kept track of, and how to make it more obvious in the API without breaking backwards compatibility.
  • project template creator as separate plugin (with standardized template format if possible)
    Currently the “Create Project” command is a very static command reading from a directory in the plugin. Making this a separate VS Code plugin with an API for other plugins to extend with more functionality would make it more flexible and open it to new platforms and APIs. Also, research on if there are already some standards for this and if we can parse other template formats.
  • detect used imports & add command to remove unused ones & add Remove unused imports + sort command
    Add a way to find out if imports are used and add a command to remove unused ones. DCD could probably be extended for this.
  • code folding on grammar
    The expand/collapse feature on arrays & functions (bracket based) and lambdas. (Instead of indentation based)
  • parse dub commandline arguments
    Dub command line arguments should be able to be passed into the build buttons and future tasks. This will require parsing them because dub is used as an API and not as a program.
  • experiment with adding a mode to DCD to make all symbols available
    A mode to always give all public symbols regardless of imports and additionally provide the import where the symbol comes from (so when completing it, the IDE adds an import)

UX & Tools for beginners

  • improve UI/UX for building and running projects (implement tasks api) 
    As the tasks API is now actually released, I can incorporate the different build types into the task system of VS Code. This would include having build (debug, release), run, test for each detected dub project and submodule.
  • make workspace symbol search show more symbols
    The workspace symbol search (Ctrl-P -> #) currently is using just DCD to find exact matches for types. I would include DScanner ctag/etag results in this response to make the search better.
  • code action: improve implement interface (parse existing functions and not include them)
    Test cases need to be written, existing interfaces must be parsed, cross-file it should all still work, inherit documentation, etc. Need to check in real world applications where interfaces and classes are used. Also add support for abstract classes.
  • auto complete in diet templates (both HTML tags and D code)
    Add static HTML auto completion for tags & attributes and create a virtual D file for D code for DCD for autocompletion.
  • code action: create getter/setter for variable
    For code like int _foo;, create the property functions int foo() @property const { return _foo; } int foo(int value) @property { return _foo = value; } with automatic detection if const can be used. This makes writing classes easier (snippets already exist, but this is more intuitive) and shows beginners how properties are created. The property should be added where other properties are. It could check the comments for “Getters”, “Setters” and “Properties” if possible, otherwise just put it after a block of members.
  • code action: convert boolean parameter to Flag (+import)
    This should change the definition of a function void foo(bool createDirectory) to alias CreateDirectory = Flag!"createDirectory"; void foo(CreateDirectory createDirectory) and add the appropriate import if it doesn’t exist yet.
  • code action: make foreach parallel (with import if neccessary)
    Replace a foreach (a; array) with foreach (a; parallel(array)) and import std.parallelism.
  • code action: convert enum to bitflag enum (isBitFlagEnum assert + modify values)
    Convert any enum enum Foo { a, b, c } to enum Foo { a = 1 << 0, b = 1 << 1, c = 1 << 2 }, add static assert(isBitFlagEnum!Foo); and import std.typecons if neccessary.
  • search for unknown symbol or import online
    Use dpldocs.info. Example: import mongoschema; -> Error importing -> Search online code fix -> Add dub package mongoschemad, etc. For undefined symbols the same: check local projects for symbols, otherwise see if online providers know anything.
  • show local references from DCD & local rename
    Highlight a symbol in the document when hovering on a member and provide the ability to rename it (not cross-file, because DCD doesn’t support that).

Thank you

Thanks again to all of the donors who pushed us over the goal line for this. Though we realize not everyone will be entirely pleased with the milestone list, and that it would have been better if we had finalized everything before asking for donations rather than coming at it haphazardly like we did, we hope that most of you find more here to like than not.

It will be a little while yet before we set up the new system and announce the next goal, as we’ve got other initiatives taking priority at the moment. So please be patient. We haven’t forgotten about the State of D Survey, nor are we going to let the results sit and bitrot. This train may be slow sometimes, but it’s rolling on.

DMD 2.081.0 Released

DMD 2.081.0 is now ready for download. Things that stand out in this release are a few deprecations, the implementation of a recently approved DIP (D Improvement Proposal), and quite a bit of work on C++ compatibility. Be sure to check the changelog for details.

Improving C++ interoperability

D has had binary compatibility with C from the beginning not only because it made sense, but also because it was relatively easy to implement. C++ is a different beast. Its larger-than-C feature set and the differences between it and D introduce complexities that make implementing binary compatibility across all supported platforms a challenge to get right. Because of this, D’s extern(C++) feature has been considered a work in progress since its initial inception.

DMD 2.081.0 brings several improvements to the D <-> C++ story, mostly in the form of name mangling bug fixes and improvements. The mangling of constructors and destructors in extern(C++) now properly match the C++ side, as does that of most of D’s operator overloads (where they are semantically equivalent to C++).

Proper mangling of nullptr_t is implemented now as well. On the D side, use typeof(null):

alias nullptr_t = typeof(null);
extern(C++) void fun(nullptr_t);

The alias in the example is not required, but may help with usability and readability when interfacing with C++. As typing null everywhere is likely reflexive for most D programmers, nullptr_t may be easier to keep in mind than typeof(null) for those special C++ cases.

Most of the D operator overloads in an extern(C++) class will now correctly mangle. This means it’s now possible to bind to operator overloads in C++ classes using the standard D opBinary, opUnary, etc. The exceptions are opCmp, which has no compatible C++ implementation, and the C++ operator!, which has no compatible D implementation.

In addition to name mangling improvements, a nasty bug where extern(C++) destructors were being placed incorrectly in the C++ virtual table has been fixed, and extern(C++) constructors and destructors now semantically match C++. This means mixed-language class hierarchies are now possible and you can now pass extern(C++) classes to object.destroy when you’re done with them.

Indirectly related,  __traits(getLinkage, ...) has been updated to now tell you the ABI with which a struct, class, or interface has been declared, so you can now filter out your extern(C++) aggregates from those which are extern(D) and extern(Objective-C).

The following shows some of the new features in action. First, the C++ class:

#include <iostream>
class CClass {
private:
    int _val;
public:
    CClass(int v) : _val(v) {}
    virtual ~CClass() { std::cout << "Goodbye #" << _val << std::endl; }
    virtual int getVal() { return _val; }
    CClass* operator+(CClass const * const rhs);
};

CClass* CClass::operator+(CClass const * const rhs) {
    return new CClass(_val + rhs->_val);
}

And now the D side:

extern(C++) class CClass
{
    private int _val;
    this(int);
    ~this();
    int getVal();
    CClass opBinary(string op : "+")(const CClass foo);
}

class DClass : CClass
{
    this(int v)
    {
        super(v);
    }
    ~this() {}
    override extern(C++) int getVal() { return super.getVal() + 10; }
}

void main()
{
    import std.stdio : writeln;

    writeln("CClass linkage: ", __traits(getLinkage, CClass));
    writeln("DClass linkage: ", __traits(getLinkage, DClass));

    DClass clazz1 = new DClass(5);
    scope(exit) destroy(clazz1);
    writeln("clazz1._val: ", clazz1.getVal());

    DClass clazz2 = new DClass(6);
    scope(exit) destroy(clazz2);
    writeln("clazz2._val: ", clazz2.getVal());

    CClass clazz3 = clazz1 + clazz2;
    scope(exit) destroy(clazz3);
    writeln("clazz3._val: ", clazz3.getVal);
}

Compile the C++ class to an object file with your C++ compiler, then pass the object file to DMD on the command line with the D source module and Bob’s your uncle (just make sure on Windows to pass -m64 or -m32mscoff to dmd if you compile the C++ file with the 64-bit or 32-bit Microsoft Build Tools, respectively).

This is still a work in progress and users diving into the deep end with C++ and D are bound to hit shallow spots more frequently than they would like, but this release marks a major leap forward in C++ interoperability.

DIP 1009

Given the amount of time DIP 1009 spent crawling through the DIP review process, it was a big relief for all involved when it was finally approved. The DIP proposed a new syntax for contracts in D. For the uninitiated, the old syntax looked like this:

int fun(ref int a, int b)
in
{
    // Preconditions
    assert(a > 0);
    assert(b >= 0, "b cannot be negative");
}
out(result) // (result) is optional if you don't need to test it
{
    // Postconditions
    assert(result > 0, "returned result must be positive");
    assert(a != 0);
}
do
{
 	// The function body
    a += b;
    return b * 100;
}

Thanks to DIP 1009, starting in DMD 2.081.0 you can do all of that more concisely with the new expression-based contract syntax:

int fun(ref int a, int b)
    in(a > 0)
    in(b >= 0, "b cannot be negative")
    out(result; result > 0, "returned result must be positive")
    out(; a != 0)
{
    a += b;
    return b * 100;
}

Note that result is optional in both the old and new out contract syntaxes and can be given any name. Also note that the old syntax will continue to work.

Deprecations

There’s not much information to add here beyond what’s already in the changelog, but these are cases that users should be aware of:

The #dbugfix campaign

The inaugural #dbugfix round prior to the release of DMD 2.080 was a success, but Round 2 has been much, much quieter (few nominations, very little discussion, and no votes).

One of the two nominated bugs selected from Round 1 was issue #18068. It was fixed and merged into the new 2.081.0 release. The second bug selected was issue #15984, which has not yet been fixed.

In Round 2, the following bugs were nominated with one vote each:

I’ll hand this list off to our team of bug fixing volunteers and hope there’s something here they can tackle.

Round 3 of the #dbugfix campaign is on now. Please nominate the bugs you want to see fixed! Create a thread in the General Forum with #dbugfix and the issue number in the title, or send out a tweet containing #dbugfix and the issue number. I’ll tally them up at the end of the cycle (September 28).

And please, if you do use the #dbugfix in a tweet, remember that it’s intended for nominating bugs you want fixed and not for bringing attention to your pull requests!

How an Engineering Company Chose to Migrate to D

Bastiaan Veelo is the lead developer of a specialised program for the
computer aided geometric design of ship hulls called Fairway, for the
company SARC in the Netherlands.


Imagine there is this little-known programming language in which you enjoy programming in your free time. You know it is ready for prime time and you dream about using it at work everyday. This is the story about how I made a dream like that come true.

My early acquaintance with D

Back when “google” was not yet a common verb, I was doing a web search for “parsing C++”. The reason was that writing a report for an assignment had derailed into writing a syntax highlighter for noweb using bison and flex, and I found out firsthand that C++ is not easy to parse. That web search brought up this page (present version) with an overview of the D Programming Language, and the following statement has had me hooked ever since:

D’s lexical analyzer and parser are totally independent of each other and of the semantic analyzer. This means it is easy to write simple tools to manipulate D source perfectly without having to build a full compiler. It also means that source code can be transmitted in tokenized form for specialized applications.

“Genius,” I thought, “here we have someone who knows what he’s doing.” This is representative of the pragmatic professionalism that still radiates from the D community, and it combines with an unpretentious flair that makes it pleasant to be around. This funny quote decorated its homepage for many years:

“Great, just what I need.. another D in programming.” – Segfault

Nevertheless, I didn’t have many opportunities to use the language and I largely remained sitting on the fence, observing its development.

Programming professionally

With mostly academic programming experience, I started programming professionally in 2006 for SARC, a Dutch engineering company serving the maritime industry. Since the early ’80s they have been developing software for ship design and onboard loading calculations, which today amounts to roughly half a million lines of code. I think their success can partly be attributed to their choice of programming language: Extended Pascal (the ISO 10206 standard, not one of the many proprietary extensions of Pascal).

Extended Pascal was a great improvement over ISO 7185 Pascal. Its compiler, by Prospero Software from England, was fast and well documented. The language is small enough and its syntax appropriately verbose to make engineering professionals quickly productive in programming. Personally though, I spent most of my time programming in C++, modernizing their system for computer aided design of ship hulls using Qt and Coin3D.

When your company outlives a programming language

Although selecting an ISO standard in favor of a proprietary Pascal dialect seemed wise at the time, it is apparent now that the company has outlived the language. Prospero Development Software Ltd was officially dissolved 15 years ago. Still, its former director, Tony Hetherington, continued giving support many years after, but he’d be close to 86 years old now and can no longer be reached. Its website is gone, last archived in 2013. There’s GNU Pascal, which also supports ISO 10206, but that project has stopped moving and long ago lost synchrony with gcc. Although there is no immediate crisis, it is clear that something needs to happen sometime if the company wants to continue its activities in the coming decades.

Changing the odds

A couple of years ago, I secretly started playing with the fantasy of replacing Extended Pascal with D. Even though D’s syntax is somewhat different from Pascal, it shares at least four important similarities: support for nested functions, boundary checking, modules, and compilation speed. In addition, it has many traits that make the language attractive to engineers: good focus on performance and numerics, garbage collection, dynamic arrays, easy parallelization, understandable templates, contract programming, memory safety, unit tests, and even wysiwyg strings and formatted numerals. D’s language features encourage experimentation, which resonates well with engineers.

So I wondered what I could do to highlight D’s significance to my employer and show it’s an attractive language to switch to. I thought I could make a compelling case if I could write a parser in D that would take Extended Pascal source and transpile it to D source. At least I would have fun trying!

So I went over to code.dlang.org to see if there were any D alternatives to flex and bison. There, I found Pegged, and instantly the fun began. Pegged combines the functionality of flex and bison in one incredibly easy to use package, for which its creator Philippe Sigaud obviously enjoyed writing excellent documentation. Nowadays, Pegged is part of the D language tour and you can try it out on-line without having to install a thing. The beauty is that the grammar from the Extended Pascal language specification maps almost linearly to the PEG from which Pegged generates the parser. For this it makes heavy use of D’s generic programming capabilities and compile-time function evaluation — it can generate a parser at compile time if you want it to!

However, it wasn’t smooth sailing all along. As I was testing D, I suddenly found myself being tested as well. I learned the hard way that there is a phenomenon called left-recursion, from which a PEG parser typically cannot break out of. And the Extended Pascal grammar is left-recursive in several ways. Consequently, I spent many evenings and weekends researching parsing theory, until eventually I managed to extend Pegged with support for all kinds of left-recursion! From one thing came another, and I added longest match alternation, case insensitive literals, the toHTML() method for dynamically browsing the syntax tree, and a tracer for logging the parsing process.

Obviously, I was having fun. But more importantly, I was demonstrating that the D programming language is accessible enough that a naval architect can understand other people’s code and expand it in non-trivial ways. The icing on the cake came when I was asked to present my experiences at DConf 2017 in Berlin, which you can watch here (and here’s the extra bit I presented at lunch time for the livestream audience).

At this time, I was able to automatically translate the following trivial example:

program hello(output);

begin
    writeln('Hello D''s "World"!');
end.

into D:

import std.stdio;

// Program name: hello
void main(string[] args)
{
    writeln("Hello D's \"World\"!");
}

Language competition

Having come this far, the founder of SARC agreed that it was time to investigate the merits of various alternative programming languages. We would do a thorough and objective comparison based on trial translations of a comprehensive set of language features. Due to the amount of manual labor that this requires, we had to drastically prune the space of programming languages in an initial review round. Note that what I am about to present does not declare which programming language is the best in our industry. What we are looking for is a language that allows an efficient transition from Extended Pascal without interrupting our business, and which enables us to take advantage of modern insights and tools.

In the initial review round we looked at general language characteristics. Here I’ll just highlight what fell through the sieve and why.

Performance is important to us, which is why we did not consider interpreted languages. C++ is in use for one component of our software, but that was written from the ground up. We feel that the options for translation are not favorable, that its long compile times are a serious hindrance to productivity, and that there are too many ways in which one can shoot one’s self in the foot. We cannot require our expert naval architects to also become experts in C++.

Nowadays, whenever D is publicly evaluated, the younger languages Go and Rust are often brought up as alternatives. Here, we need not go into an in-depth comparison of these languages because both Rust and Go lack one feature that we rely on heavily: nested functions with access to variables in their enclosing scope. Solutions for eliminating nested functions, like bringing them into global scope and passing extra variables, or breaking files up into smaller modules, we find unattractive because it would complicate automated translation, and we’d like to preserve the structure and style of our current code. GNU C does offer nested functions, but it is a non-standard extension and it has been predicted that many will move away from C due to its unsafe features. After this initial pruning, three languages remained on our shortlist: Free Pascal, Ada and D.

As a basis for our detailed comparison, we wrote fifteen small programs that each used a specific feature of Extended Pascal that is important in our current code base. We then translated those programs into each language on our shortlist. We kept a simple score board on how well these features were represented in each language: +1 if the feature is supported or can be implemented, 0 if the lack of the feature can be worked around, and -1 if it can’t. This is what came out of that evaluation:

Test Free Pascal Ada D
Arrays beginning at arbitrary indices +1 +1 +1
Sets 0 0 +1
Schema types 0 0 +1
Types with custom initial values -1 0 +1
Classes +1 +1 +1
Casts +1 +1 +1
Protection against use of dangling pointers -1 +1 +1
Thread safe memory [de]allocation +1 +1 +1
Calling into Windows API +1 +1 +1
Forwarding Windows callbacks to nested functions +1 +1 +1
Speed of calculations +1 +1 +1
Calling procedures written in assembly +1 0 +1
Calling procedures in a DLL +1 +1 +1
Binary compatibility of strings 0 +1 +1
Binary compatible file i/o -1 0 0
Score 6 10 14

So, Free Pascal is the only candidate with negative scores, Ada positions itself in the middle, and D achieves an almost perfect score. Not effortlessly, though; we’ll talk about some of the technical challenges later. Because Free Pascal is, like D, fully Open Source and written in itself, extending the language and filling in the gaps is theoretically possible. Although some of its deficiencies could certainly be resolved that way, others would be quite complicated and/or unlikely to be accepted upstream.

We also estimated the productivity of the languages. Free Pascal scored high because it is closest to what we are used to. Despite its dissimilar syntax, D scored high because of its expressiveness and flexibility. Ada scored lowest because of its rigidity and because of the extra work the programmer has to put in (most importantly casts and conversions). Ada is more verbose than Pascal which was disliked by some of us because it can somewhat obscure the essence of what a piece of code tries to express, and frequently the code became not only verbose but cryptic, which was unanimously disliked.

Third, we estimated the future prospects and the advantages each language could bring to the table. Although Free Pascal has a more active community than we expected it to have, we do not see great potential for growth. Ada is renowned for its support for writing reliable code (although it has no monopoly in that field) but it does come at a cost and requires real effort. D has a dynamic and open community, supports both script-like productivity and high performance, includes various features for writing reliable software (approaching Ada but at a much lower cost), and offers some unique advanced features with which wonders can be accomplished.

Finally, we estimated the effort of translation. Although Free Pascal is very similar to Extended Pascal, missing features pose a real problem and would require a high degree of manual translation and rewriting. Although p2ada exists, it only works partially in our case and does not fully support Extended Pascal. Because Ada frequently requires additional code (casting to the correct type, pulling in a package, instantiating a generic type, adding a pragma, splitting up Put_Lines etc.), writing or extending a reliable transpiler into Ada would be more difficult than doing the same into D.

Selecting a winner

I gave away the winner in the title, but we landed at that conclusion as follows. Ada was the first language to be dropped. We really felt that the extra work that the programmer has to put in is a brake on productivity and creativity. Although it barely played a role in our evaluation, illustrative is the difference between the Ada and D equivalents to the Expressive C++17 Challenge. The D solution is both concise and expressive, the Ada solution is hardly expressive and consists of more lines than I want to write or read. Also of secondary importance, but difficult to ignore, is the difference between the communities surrounding the languages, which in Ada’s case is AdaCore Support, who has no problems demanding annual five-figure subscription fees.

Although akin to our current language, Free Pascal was mainly dropped due to its porting challenges and our estimation that its potential is lower and its future outlook is less optimistic than that of D. If we were to choose Free Pascal, we would basically invest a lot of effort only to arrive at a technological solution that we felt would be of lower quality than we currently have.

And that’s were I saw a dream come true: A clap on the table by the company founder and it was decided to commit to the effort of bringing twenty-five years worth of Extended Pascal code to D!

What makes a difference

In short, my experience is that if a feature is not present in the language, D is powerful enough that the feature can be implemented in a library. Translating each sample program by hand has really helped to focus on replicating functionality, leaving the translation process for later concern. This has led to writing a compatibility library with types and functions that are vital for the conversion. Now that equivalents are known and the parser is done, I just have to implement code generation.

Below follows another example that currently translates automatically and executes identically. It iterates over a fixed length array running from 2 to 20 inclusive, fills it with values, prints the memory footprint and writes it to binary file:

program arraybase(input,output);

type t = array[2..20] of integer;
var a : t;
    n : integer;
    f : bindable file of t;

begin
  for n := 2 to 20 do
    a[n] := n;
  writeln('Size of t in bytes is ',sizeof(a):1); { 76 }
  if openwrite(f,'array.dat') then
    begin
      write(f,a);
      close(f);
    end;
end.

Transpiled to D (or should I say Dascal?) and post-processed by dfmt to fix up formatting:

import epcompat;
import std.stdio;

// Program name: arraybase
alias t = StaticArray!(int, 2, 20);

t a;
int n;
Bindable!t f;

void main(string[] args)
{
    for (n = 2; n <= 20; n++)
        a[n] = n;
    writeln("Size of t in bytes is ", a.sizeof); // 76
    if (openwrite(f, "array.dat"))
    {
        epcompat.write(f, a);
        close(f);
    }
}

Of course this is by no means idiomatic D, but the fact that it is recognizable and readable is nice, especially for my colleagues who will have to go through an unusual transition. By the way, did you notice that code comments are preserved?

One very-nice-to-have feature is binary file compatibility; In fact it may have been the killer feature, without which D might not have been so victorious. The case is that whenever a persistent data structure is extended in our software, we make sure that we can still read and convert that structure from its prior format. That way, if a client pulls out an old design from its archives and runs it through our current software, it will still work without the user even being aware that conversion occurs, possibly in multiple steps. Not having to give up that ability is very attractive.

But it wasn’t easy to get there. The main difficulty is the difference in how strings are represented in D and the Prospero implementation of Extended Pascal, in memory and on file. This presented the challenge of how to preserve binary compatibility in file I/O with data structures that contain string members.

Strings

In Prospero Extended Pascal, strings are implemented as a schema type, which is a parameterized type that can be used in the following ways:

type string80 = string(80);
var str1 : string80;
    str2 : string(60);
procedure foo(s : string);

This defines string80 to be an alias for a string type discriminated to have a capacity of 80 characters. Discriminated string variables, like str1 and str2, can be passed to functions and procedures that take undiscriminated strings as arguments, like foo, which thereby work on strings of any capacity. In memory, str1 is laid out as a sequence of 80 chars, followed by a ushort that encodes the length of the string. I say encodes because a shorter string is padded with \0s up to the capacity and the ushort actually contains the length of that padding. This way, when a pointer to the string is passed to a C function and the contents of the string occupy its full capacity, the 0 in the padding length doubles as the terminating \0 of the C string.

My first thought was to mimic this data representation with a D template. But that would require procedures like foo to be turned into templates as well, which would escalate horribly into template bloat, a problem with multiple string arguments and argument ordering, and would complicate translation. Besides, schema types can also be discriminated at run time, which does not translate to a template.

Could some sort of inheritance scheme be the solution? Not really, because instances of D classes live on the heap, so a string embedded in a struct would just be a pointer instead of the char array and ushort.

But binary layout is actually only relevant in files, and in a stroke of insight I realized that this must be why user-defined attributes, or UDAs, exist. If I annotate the string with the correct capacity for file I/O, then I can just use native D strings everywhere, which genuinely must be the best possible translation and solves the function argument issue. Annotation can be done with an instance of a struct like

struct EPString
{
    ushort capacity;
}

The above Pascal snippet then translates to D like so:

@EPString(80) struct string80 { string _; alias _ this; }
string80 str1;
@EPString(60) string str2;
void foo(string s);

Notice how the string80 alias is translated into the slightly convoluted struct instead of a normal D alias, which would have looked like

@EPString(80) alias string80 = string;
</code>

Although that compiles, there is no way to retrieve the UDA in that case because plain alias does not introduce a symbol. Then hasUDA!(typeof(str1), EPString) would have been equivalent to hasUDA!(string, EPString) which evaluates to false. By using the struct, string80 is a symbol so typeof(str1) gives string80, and hasUDA!(string80, EPString) evaluates to true in this example.

There is one side effect that we will have to learn to accept, and that is that taking a slice of a string does not produce the same result in D as it does in Extended Pascal. That is because string indices start at 1 in Extended Pascal and at 0 in D. My strategy is to eliminate slices from the source and replace them with a call to the standard substr function, which I can implement with index correction. Finding all string slices can be accomplished with a switch in the transpiler that makes it insert a static if to test if the slice is being taken on a string, and abort compilation if it is. (Arrays are transpiled into a custom array type that handles slices and indices compatibly with Extended Pascal.)

Binary compatible file I/O

Now, to write structs to file and handle any embedded @EPString()-annotated strings specially, we can use compile-time introspection in an overload to toFile that acts on structs as shown below. I have left out handling of aliased strings for clarity, as well as shortstring, which is a legacy string type with yet a different binary format.

void toFile(S)(S s, File f) if (is(S == struct))
{
    import std.traits;
    static if (!hasIndirections!S)
        f.lockingBinaryWriter.put(s);
    else
        // TODO unions
        foreach(field; FieldNameTuple!S)
        {
            // If the member has itself a toFile method, call it.
            static if (hasMember!(typeof(__traits(getMember, s, field)), "toFile") &&
                       __traits(compiles, __traits(getMember, s, field).toFile(f)))
                __traits(getMember, s, field).toFile(f);
            // If the member is a struct, recurse.
            else static if (is(typeof(__traits(getMember, s, field)) == struct))
                toFile(__traits(getMember, s, field), f);
            // Treat strings specially.
            else static if (is(typeof(__traits(getMember, s, field)) == string))
            {
                // Look for a UDA on the member string.
                static if (hasUDA!(__traits(getMember, s, field), EPString))
                {
                    enum capacity = getUDAs!(__traits(getMember, s, field), EPString)[0].capacity;
                    static assert(capacity > 0);
                    writeAsEPString(__traits(getMember, s, field), capacity, f);
                }
                else static assert(false, `Need an @EPString(n) in front of ` ~ fullyQualifiedName!S ~ `.` ~ field );
            }
            // Just write other data members.
            else static if(!isFunction!(__traits(getMember, s, field)))
                f.lockingBinaryWriter.put(__traits(getMember, s, field));
        }
}

At the time of writing, I still have work to do for unions, which are used in the translation of variant records (including considering the use of one of the seven existing library solutions 1, 2, 3, 4, 5, 6, 7).

Currently, detecting unions is a bit involved . Also, there is a complication in the determination of the size of a union when the largest variant contains strings: the D version of that variant may not be the largest because D strings are just slices. I’ll probably work around this by adding a dummy variant that is a fixed size array of bytes to force the size of the union to be compatible with Extended Pascal. This is the reason why D scored a mere 0 in file format compatibility. It is amazing what D allows you to do though, so I may be able to do all of that automatically and award D a perfect score retroactively. On the other hand, it is probably easiest to just add the dummy variant in the Pascal source at the few places where it matters and be done with it.

The way forward

Obviously, this is long term planning. It has taken years to grow into D; it will possibly take a year, and probably longer, to migrate to D. Unless others turn up who are in the same boat as us (please contribute!) it’ll be me who has to row this ship to D-land and I still have my regular duties to attend to. My colleagues will continue to develop in Extended Pascal as usual, and once my transpiler is able to translate all or almost all of it, we will make the switch to D overnight. From then on, we’ll be in it for the long run. We trust to be with D and D to be with us for decades to come!

DasBetterC: Converting make.c to D

Walter Bright is the BDFL of the D Programming Language and founder of Digital Mars. He has decades of experience implementing compilers and interpreters for multiple languages, including Zortech C++, the first native C++ compiler. He also created Empire, the Wargame of the Century. This post is the third in a series about D’s BetterC mode


D as BetterC (a.k.a. DasBetterC) is a way to upgrade existing C projects to D in an incremental manner. This article shows a step-by-step process of converting a non-trivial C project to D and deals with common issues that crop up.

While the dmd D compiler front end has already been converted to D, it’s such a large project that it can be hard to see just what was involved. I needed to find a smaller, more modest project that can be easily understood in its entirety, yet is not a contrived example.

The old make program I wrote for the Datalight C compiler in the early 1980’s came to mind. It’s a real implementation of the classic make program that’s been in constant use since the early 80’s. It’s written in pre-Standard C, has been ported from system to system, and is a remarkably compact 1961 lines of code, including comments. It is still in regular use today.

Here’s the make manual, and the source code. The executable size for make.exe is 49,692 bytes and the last modification date was Aug 19, 2012.

The Evil Plan is:

  1. Minimize diffs between the C and D versions. This is so that if the programs behave differently, it is far easier to figure out the source of the difference.
  2. No attempt will be made to fix or improve the C code during translation. This is also in the service of (1).
  3. No attempt will be made to refactor the code. Again, see (1).
  4. Duplicate the behavior of the C program as exactly and as much as possible,
    bugs and all.
  5. Do whatever is necessary as needed in the service of (4).

Once that is completed, only then is it time to fix, refactor, clean up, etc.

Spoiler Alert!

The completed conversion. The resulting executable is 52,252 bytes (quite comparable to the original 49,692). I haven’t analyzed the increment in size, but it is likely due to instantiations of the NEWOBJ template (a macro in the C version), and changes in the DMC runtime library since 2012.

Step By Step

Here are the differences between the C and D versions. It’s 664 out of 1961 lines, about a third, which looks like a lot, but I hope to convince you that nearly all of it is trivial.

The #include files are replaced by corresponding D imports, such as replacing #include <stdio.h> with import core.stdc.stdio;. Unfortunately, some of the #include files are specific to Digital Mars C, and D versions do not exist (I need to fix that). To not let that stop the project, I simply included the relevant declarations in lines 29 to 64. (See the documentation for the import declaration.)

#if _WIN32 is replaced with version (Windows). (See the documentation for the version condition and predefined versions.)

extern (C): marks the remainder of the declarations in the file as compatible with C. (See the documentation for the linkage attribute.)

A global search/replace changes uses of the debug1, debug2 and debug3 macros to debug printf. In general, #ifdef DEBUG preprocessor directives are replaced with debug conditional compilation. (See the documentation for the debug statement.)

/* Delete these old C macro definitions...
#ifdef DEBUG
-#define debug1(a)       printf(a)
-#define debug2(a,b)     printf(a,b)
-#define debug3(a,b,c)   printf(a,b,c)
-#else
-#define debug1(a)
-#define debug2(a,b)
-#define debug3(a,b,c)
-#endif
*/

// And replace their usage with the debug statement
// debug2("Returning x%lx\n",datetime);
debug printf("Returning x%lx\n",datetime);

The TRUE, FALSE and NULL macros are search/replaced with true, false, and null.

The ESC macro is replaced by a manifest constant. (See the documentation for manifest constants.)

// #define ESC     '!'
enum ESC =      '!';

The NEWOBJ macro is replaced with a template function.

// #define NEWOBJ(type)    ((type *) mem_calloc(sizeof(type)))
type* NEWOBJ(type)() { return cast(type*) mem_calloc(type.sizeof); }

The filenamecmp macro is replaced with a function.

Support for obsolete platforms is removed.

Global variables in D are placed by default into thread-local storage (TLS). But since make is a single-threaded program, they can be inserted into global storage with the __gshared storage class. (See the documentation for the __gshared attribute.)

// int CMDLINELEN;
__gshared int CMDLINELEN

D doesn’t have a separate struct tag name space, so the typedefs are not necessary. An
alias can be used instead. (See the documentation for alias declarations.) Also, struct is omitted from variable declarations.

/*
typedef struct FILENODE
        {       char            *name,genext[EXTMAX+1];
                char            dblcln;
                char            expanding;
                time_t          time;
                filelist        *dep;
                struct RULE     *frule;
                struct FILENODE *next;
        } filenode;
*/
struct FILENODE
{
        char            *name;
        char[EXTMAX1]  genext;
        char            dblcln;
        char            expanding;
        time_t          time;
        filelist        *dep;
        RULE            *frule;
        FILENODE        *next;
}

alias filenode = FILENODE;

macro is a keyword in D, so we’ll just use MACRO instead.

Grouping together multiple pointer declarations is not allowed in D, use this instead:

// char *name,*text;
// In D, the * is part of the type and 
// applies to each symbol in the declaration.
char* name, text;

C array declarations are transformed to D array declarations. (See the documentation for D’s declaration syntax.)

// char            *name,genext[EXTMAX+1];
char            *name;
char[EXTMAX+1]  genext;

static has no meaning at module scope in D. static globals in C are equivalent to private module-scope variables in D, but that doesn’t really matter when the module is never imported anywhere. They still need to be __gshared and that can be applied to an entire block of declarations. (See the documentation for the static attribute)

/*
static ignore_errors = FALSE;
static execute = TRUE;
static gag = FALSE;
static touchem = FALSE;
static debug = FALSE;
static list_lines = FALSE;
static usebuiltin = TRUE;
static print = FALSE;
...
*/

__gshared
{
    bool ignore_errors = false;
    bool execute = true;
    bool gag = false;
    bool touchem = false;
    bool xdebug = false;
    bool list_lines = false;
    bool usebuiltin = true;
    bool print = false;
    ...
}

Forward reference declarations for functions are not necessary in D. Functions defined in a module can be called at any point in the same module, before or after their definition.

Wildcard expansion doesn’t have much meaning to a make program.

Function parameters declared with array syntax are pointers in reality, and are declared as pointers in D.

// int cdecl main(int argc,char *argv[])
int main(int argc,char** argv)

mem_init() expands to nothing and we previously removed the macro.

C code can play fast and loose with arguments to functions, D demands that function prototypes be respected.

void cmderr(const char* format, const char* arg) {...}

// cmderr("can't expand response file\n");
cmderr("can't expand response file\n", null);

Global search/replace C’s arrow operator (->) with the dot operator (.), as member access in D is uniform.

Replace conditional compilation directives with D’s version.

/*
 #if TERMCODE
    ...
 #endif
*/
    version (TERMCODE)
    {
        ...
    }

The lack of function prototypes shows the age of this code. D requires proper prototypes.

// doswitch(p)
// char *p;
void doswitch(char* p)

debug is a D keyword. Rename it to xdebug.

The \n\ line endings for C multiline string literals are not necessary in D.

Comment out unused code using D’s /+ +/ nesting block comments. (See the documentation for line, block and nesting block comments.)

static if can replace many uses of #if. (See the documentation for the static if condition.)

Decay of arrays to pointers is not automatic in D, use .ptr.

// utime(name,timep);
utime(name,timep.ptr);

Use const for C-style strings derived from string literals in D, because D won’t allow taking mutable pointers to string literals. (See the documentation for const and immutable.)

// linelist **readmakefile(char *makefile,linelist **rl)
linelist **readmakefile(const char *makefile,linelist **rl)

void* cannot be implicitly cast to char*. Make it explicit.

// buf = mem_realloc(buf,bufmax);
buf = cast(char*)mem_realloc(buf,bufmax);

Replace unsigned with uint.

inout can be used to transfer the “const-ness” of a function from its argument to its return value. If the parameter is const, so will be the return value. If the parameter is not const, neither will be the return value. (See the documentation for inout functions.)

// char *skipspace(p) {...}
inout(char) *skipspace(inout(char)* p) {...}

arraysize can be replaced with the .length property of arrays. (See the documentation for array properties.)

// useCOMMAND  |= inarray(p,builtin,arraysize(builtin));
useCOMMAND  |= inarray(p,builtin.ptr,builtin.length)

String literals are immutable, so it is necessary to replace mutable ones with a stack allocated array. (See the documentation for string literals.)

// static char envname[] = "@_CMDLINE";
char[10] envname = "@_CMDLINE";

.sizeof replaces C’s sizeof(). (See the documentation for the .sizeof property).

// q = (char *) mem_calloc(sizeof(envname) + len);
q = cast(char *) mem_calloc(envname.sizeof + len);

Don’t care about old versions of Windows.

Replace ancient C usage of char * with void*.

And that wraps up the changes! See, not so bad. I didn’t set a timer, but I doubt this took more than an hour, including debugging a couple errors I made in the process.

This leaves the file man.c, which is used to open the browser on the make manual page when the -man switch is given. Fortunately, this was already ported to D, so we can just copy that code.

Building make is so easy it doesn’t even need a makefile:

\dmd2.079\windows\bin\dmd make.d dman.d -O -release -betterC -I. -I\dmd2.079\src\druntime\import\ shell32.lib

Summary

We’ve stuck to the Evil Plan of translating a non-trivial old school C program to D, and thereby were able to do it quickly and get it working correctly. An equivalent executable was generated.

The issues encountered are typical and easily dealt with:

  • Replacement of #include with import
  • Lack of D versions of #include files
  • Global search/replace of things like ->
  • Replacement of preprocessor macros with:
    • manifest constants
    • simple templates
    • functions
    • version declarations
    • debug declarations
  • Handling identifiers that are D keywords
  • Replacement of C style declarations of pointers and arrays
  • Unnecessary forward references
  • More stringent typing enforcement
  • Array handling
  • Replacing C basic types with D types

None of the following was necessary:

  • Reorganizing the code
  • Changing data or control structures
  • Changing the flow of the program
  • Changing how the program works
  • Changing memory management

Future

Now that it is in DasBetterC, there are lots of modern programming features available to improve the code:

Action

Let us know over at the D Forum how your DasBetterC project is coming along!

Driving Continuous Improvement in D

Jack Stouffer is a member of the Phobos team and a contributor to dlang.org. You can check out more of his writing on his blog.


In my previous article, I went over the techniques we use in the D Standard Library (a.k.a Phobos) to develop a wide variety of testing mechanisms. I also briefly mentioned our style checker, dscanner. In this article, I’ll detail how we use dscanner to prevent style, documentation, and best practices regressions in the code, none of which can be covered by standard unit tests.

Keeping a level of quality in software projects is a neverending battle. Bad practices and shortsighted design decisions make their way into code over time, whether from poor oversight, rushing things through, or simple code rot. There are three things we do in Phobos, other than tests, to fight this entropy.

First, Pull Requests are required to be about one thing and one thing only. What counts as “one thing” is subjective, but, for example, a PR to fix a bug in a specific piece of code mustn’t also edit the documentation of that function. This allows Phobos reviewers to merge the documentation change quickly if there’s an issue in the bug fix preventing it from being merged (or vice versa). At the same time, it keeps the reviewers focused on one set of changes.

Second, PRs need to be small; ideally less than 100 lines of code changed. If that’s not possible, they need to be broken into multiple commits of smaller changes. This really helps reviewers keep D best practices in mind, while also fully understanding and internalizing the new code.

Third, we continuously improve the existing code with dscanner, which, among other things, is D’s official linting tool.

What dscanner Can Do

As an example of the checks that dscanner can provide, let’s take a look at some code that contains a very hard-to-spot bug. The following code creates a type that mimics an int, but allows a null state:

struct NullableInt
{
    private int value;
    bool isNull;

    int get()
    {
        assert(!isNull, "can't get a null");
        return value;
    }

    void nullify() { isNull = true; }

    bool opEquals(T rhs) // D's equality overloading function
    {
        if (isNull && rhs.isNull)
            return true;
        if (isNull != rhs.isNull)
            return false;
        return value == rhs.value;
    }
}

The bug is not in the code that’s there, it’s in the code that’s not there. All structs in D have default versions of the standard operator overloading functions if they aren’t defined by the user. One of those functions provides a hash to represent the value to use in D’s built-in associative arrays. The default version uses all the type fields to make the hash, which is a problem for NullableInt, as we’ve decided that all instances of the type that are null are equal. Here’s an illustration of the bug:

void main()
{
    auto a = NullableInt(10, false);
    auto b = NullableInt(10, false);
    auto c = NullableInt(10, true);
    auto d = NullableInt(0, true);

    assert(a == b);
    assert(b != c);
    assert(c == d);

    import core.internal.hash : hashOf; // D's internal hashing function
    assert(c.hashOf != d.hashOf);
}

dscanner will emit a message every time it finds a type that defines a custom opEquals, but doesn’t define a custom toHash as well, alerting us to this bug.

Continuous Improvement and “Ratcheting” Quality

dscanner ties into continuous improvement, the philosophy that improvements to processes or designs should be made in a periodic, timely manner, rather than as one-time “breakthroughs”. Such large breakthrough changes tend to be pushed back infinitely; in a phrase, it’s letting the perfect be the enemy of the good. This easily fits with the continuous delivery or rapid release philosophies, and can vastly reduce bugs in software given enough time.

By using the warnings from dscanner, we can ratchet Phobos’s quality over time. If you’ve never used a socket wrench, a ratchet is a mechanism that allows the user to freely move the wrench back and forth, but allows the bolt to spin only when the wrench moves clockwise. Similarly, we can move the quality of Phobos forward, while not letting it ever slip backwards, with dscanner. It works as follows:

  • Run dscanner with every check but one turned off on all files.
  • Populate a black list for that check in dscanner’s configuration file containing each of the files that issued a warning.
  • Repeat until the current code passes with all checks turned on with the black-list enabled.

Once we have our list-of-blacklists, new PRs will trigger warnings for issues detected by dscanner for files that weren’t included in the blacklists, thereby keeping the status quo of quality.

Next, we ratchet quality by making a PR that fixes one issue in one file, and then removing that blacklist entry from the configuration. This way, that file will be checked in the future for every new PR. Over time, quality issues will be removed from Phobos, and they won’t crop up again. For example, from the release of 2.076 to now, Phobos has gone from 624 public symbols without examples, to 328.

Currently, we’re using this ratchet technique with dscanner to

  • remove unused variables.
  • add const or immutable to variables that aren’t modified after their construction.
  • make sure every public symbol is documented.
  • make sure every symbol in Phobos that has documentation, has a code example in that documentation.
  • force every assert to have an error message to print in case it fails.
  • make sure only Exceptions, and not Errors, are caught in try/catch blocks.

among other things.

This process also has the added benefit of dogfooding dscanner, which helps us find bugs and know which features would be helpful to add from a user perspective. If you have a project that isn’t using a linting tool as part of your test suite, it’s only a matter of time before code rot creeps in.

DConf 2018 Ex Post Facto

When I was ten years old, I broke a small hand mirror. For some years after, my moderately superstitious grandmother would remind me of it any time I suffered a bit of misfortune, providing me with periodic updates on how many of my seven years of bad luck I had yet to face. Two days before DConf 2018, I couldn’t help but think of her when my wife and I, having encountered a black cat the evening before, found ourselves in the wrong train car at Frankfurt station.

The train

It was an easy mistake to make. We were supposed to be in car 28, so we boarded at a door that was branded “38/28”. Inside, 2nd class was to the right and 1st class to the left. Our reservation was for the latter. We found our seats, stowed our luggage, and settled in for our trip to Munich and DConf. It was a few minutes before I noticed the big “38” on the screen of the monitor hanging from the ceiling.

I asked the first uniformed person I could find, “Are we in the right place?”. Of course, we weren’t. He informed me we couldn’t just cut through the cars to get to the correct place — we had to disembark and get back on at the other end of the train. Only, there was no time to do so, as the train was leaving in a few seconds.

“Don’t worry,” he said. “It happens.”

When the conductor came around, he assured us that all was well. The seats weren’t reserved and the whole train was going to Munich anyway. I laughed, thinking the last bit was a joke. I later learned that two trains are sometimes linked to share part of a journey and will split somewhere along the way to head to different final destinations. He was being serious!

The pre-conf jitters

A Toastmasters class I won in an essay contest after high school cured me of my fear of public speaking (prior to that, I couldn’t say two words in front of five people without going into convulsions), and after 24 years of teaching in Korea I’m perfectly comfortable speaking to groups of a few people in a small class or a few hundred at an assembly. I always know what I want to say and I’m confident in my ability to read the crowd. So when I accepted the offer to be the DConf emcee this year, I wasn’t concerned at all. Short bursts of speaking in front of 80 people? I could do that in my sleep.

But the train incident was the first in a series of unfortunate events that continued right up until a few minutes before the conference, including two embarrassing instances of my mishearing introductions and thinking a person I’d just met was someone else. By the time I had the mic in my hand, I could hear my grandmother’s voice in my head, reiterating every bit of “bad luck” I’d had since that cat had crossed my path in Frankfurt (it was actually a kitten, not entirely black, and my wife had found it adorable enough that she briefly entertained ideas on how to take it with us to Korea).

Of course, I’m not superstitious. At all. But apparently my lizard brain felt these were extraordinary circumstances. As Walter and I were chatting while we waited for the clock to countdown, I was attacked by the worst case of jitters I’ve had since high school.

It was going to be the worst DConf ever.

The conference

After the first few words were out of my mouth, the jitters evaporated. My grandmother left me alone for the rest of the conference. Barring a few minor glitches and one rather severe one, it all went rather well, though I couldn’t really tell from my perspective.

This was my first time being on the other side. Everything went by in a blur. During the talks, there were emails and Facebook messages to respond to, IRC & the livestream feed to check in on (though Sebastian Wilzbach kept a fulltime eye on them), tweets to write, and the constant concern about how to politely keep speakers in their time budget. Without the Foundation’s intern, Maria Marginean, running the mic to audience members with questions, I would have been frazzled.

Between talks, there were problems getting speakers’ laptops to display properly, questions about things I didn’t immediately have an answer for, and people to track down to resolve one issue or another. Then there were the numerous things I had planned to say each time I got up in front of everyone, only to realize after I sat back down that I had completely forgotten.

In all, I had a great time. I had less time to mingle this year, but I still caught up with some familiar folks and met several new ones. That’s always the best part of DConf for me, and that held true this year, even though many of the conversations I had were brief.

The display issues turned out not to be as troublesome as they threatened to be on the first day. The A/V team was using a bluetooth device. Plug it in to a USB port, install a driver, press the big button on the device, and you’re projecting. But a few speakers had no USB ports, some were using Linux (for which there was no driver), and some struggled to get the proper desktop to display. On the second day, the A/V guys were prepared with every adapter imaginable and, where those failed, they simply unplugged the HDMI cable from the bluetooth receiver and plugged it directly into the laptops. We still had problems sometimes with the wrong desktop showing, but in the end it all worked out.

The biggest glitch, the one we regret most of all, was the loss of the video of the first three talks. It’s something that could have been prevented with a bit of forethought. The best thing I can say about it is that it taught a valuable lesson that will be applied at future DConfs. No matter who is organizing or who is managing the A/V, the Foundation will ensure steps are taken to minimize the chance of this happening again.

In the periods that I was able to pay attention to the talks, there was a lot to enjoy.

Vang Le presented on D in genomic bioinformatics.

We’ve heard in the forums about D being used in bioinformatics, but this year was the first time we’ve witnessed a DConf talk about it. The talk sparked enough interest in the subject that Vang Le was able to get some collaboration at the Hackathon that will hopefully continue beyond as he works to expand D in the field. I look forward to seeing more talks related to this industry at future conferences.

In contrast, the topic of game development is a DConf staple (DConf 2015 was the only edition without a gamedev talk). But this year was the first time someone from Ubisoft presented (Igor Česi), and the first we’ve learned about a project some folks from the company took on as a proof-of-concept, porting D to a platform Igor wasn’t allowed to name just yet. I’ve been interested in game development as a hobby for over a couple of decades now and I don’t get to meet too many game developers in my normal line of work (the first time I met Ethan Watson in Berlin I grilled him mercilessly about his job throughout dinner), so the conversation the trio from Ubisoft had with Andrei and me (mostly Andrei) about the company and their interest in D was one of the highlights of the conference for me.

Another highlight for me was meeting Martin Odersky, our invited keynote speaker this year. I’ve followed Scala from a distance for some time; it was the first place I turned when I needed to correct my lack of experience with functional programming after std.algorithm and friends came along. Our introduction was a bit rushed, though. Five minutes before he was supposed to speak, he wasn’t anywhere in or around the conference room. Walter hadn’t seen him. Andrei hadn’t seen him. Someone reported seeing him in the restaurant. That’s where I found him, banging away at the keys on his laptop. He had realized at breakfast that his talk about abstracting over context needed a bit more context to make it more understandable, given how most of us wouldn’t have the Scala background he referenced. He was just finishing his edits as I arrived, and we got him set up right on time. I ignored my emails and FB messages for most of that talk (and actually understood quite a lot of it).

Martin Odersky was the invited keynote speaker on Day 3.

Some DConf veterans talked about work they’ve been doing, like Jonathan Davis’s dxml library, Ethan Watson’s progress on Binderoo, Dimitry Olshansky’s report on his attempt to create a unified concurrent D runtime, and Steven Schveighoffer’s experience porting a LAMP application to vibe.d. Johan Engelen expanded on a blog post he wrote earlier this year, presenting a talk on some LLVM-backed goodies in LDC that people might not be aware of.

Kai Nacke talked about using D for the blockchain.

Kai Nacke presented another first for DConf, a talk about D for the blockchain, where he did a great job simplifying a complex topic. Luís Marques’s talk on breaking away from OOP orthodoxy was immediately followed by first-time DConf speaker Jean-Louis Leroy’s related talk on open methods. And live-ask.com, the subject of Stefan Dilly’s presentation on scalable webapp development in D with Angular and vibe.d, was actually put to use during the conference — it became our primary source for taking questions online, particularly during Walter and Andrei’s Ask Us Anything session at the end of Day One.

The D Language Foundation’s scholarship recipients (a.k.a. Andrei’s Students, a.k.a. The Romanian Crew) were back to present updates on their work. We heard from Razvan Nitu on the trials and tribulations of contributing to DMD, Eduard Staniloiu on a new approach to generic collections in D, and Alexandru Jercaianu on Project Blizzard, a means of performing safe memory allocation and deallocation in D. The first two of those talks came on Day Two, which was opened by Andrei’s keynote. He had been scheduled to present on static introspection, but changed his mind and instead did a deep dive on things that are vital to D’s future in a talk titled, “Up and Coming”.

Of the three talks for which we have no video, a version of Walter’s keynote on using D in existing C codebases will be available eventually – he gave the same talk at Code Europe Kraków not long after DConf. The picture is not as rosy for the other two talks. Mario Kröplin and Stefan Rohe presented an entertaining report on their experience using D for 10 years at Funkwerk and Jon Degenhardt used his first DConf talk to update us on the performance of eBay’s TSV utilities, following up on a blog post he wrote for the D Blog on the same topic. Sadly, the videos for those talks are forever lost.

Evidence that Jon Degenhardt really did give a talk about eBay’s TSV utilities.

We did another round of lightning talks this year. It was organized as the conference progressed, and given that everyone was using their own laptops, I was sure that trying to get them shifted on and off the lectern and properly configured would be a disaster. In the end, it all came off successfully. A couple of presenters went over their five-minute time budget and two were unable to get their displays configured, but we had just enough time to get everyone in and the display-less folks improvised and presented without their slides anyway.

Liran Zvibel (whose name I can now absolutely properly pronounce) wrapped up the last full day of talks with a closing keynote. He regaled us with a report on Weka.IO’s experience with D in the development of their storage system. It was followed the next day by Shachar Shemesh’s announcement of Mecca, Weka.IO’s container/reactor library, to kick off the Hackathon. Actually, after Shachar saw that a significant number of the audience didn’t raise their hands when I asked how many would be sticking around for the Hackathon, he requested a slot in the lightning talks for an early, abbreviated announcement.

At the Hackathon, we had audio in the room for Shachar’s talk but no video, so I set up a livestream from my MacBook (though it has poor audio quality). Ali Çehreli made sure it kept rolling and he repositioned it as needed. After the talk and the group photo (which Andrei missed!), I saw a lot of folks with their heads together, but I only know of what a couple of them were doing (maybe we’ll hear more about that in future blog posts).

The big announcement at the Hackathon was that the D Language Foundation will now start funding projects via targeted donations. The VS Code plugin code-d (and its companion, serve-d) is the first project selected. Once we achieve $3000 in donations, the project maintainer will be eligible to get that money as he meets specific milestones. Unfortunately, the new goals system at Open Collective isn’t what we expected it to be. We’ll make do with it for now, but we’ll likely be moving to an approach that allows us to set up multiple fundraising targets and count donations through Open Collective, PayPal, and elsewhere. Until then, if you’re a code-d user (or just want to support the project), head to our Open Collective page to show your appreciation and make a financial contribution to its development.

Videos of the talks are starting to appear on HLMC’s YouTube channel. Once they’re all online, we’ll set up a playlist on the D Language Foundation’s channel (you can find playlists of all the previous conferences there now).

Until next year…

Thanks to everyone who attended DConf 2018 in Munich, to everyone who supported it with time or money, and to QA Systems and HLMC Events for putting it all together.

Some of the DConf 2018 crew.

It’s too early to say where DConf 2019 will be hosted, but plans regarding some of the generic organization details are already in the works. I expect some announcements will start coming out a little bit earlier than in the past. Stay tuned!

Complicated Types: Prefer “alias this” Over “alias” For Easier-To-Read Error Messages

Nick Sabalausky is a long-time D user and contributor. He is the maintainer of mysql-native and Scriptlike. In this post, he presents a way to use a specific D language feature to improve error messages involving aliased types.


In the D programming language, alias is a common and handy feature that can be used to provide a simple name for a complex and verbose templated type.

As an example, consider the case of an algebraic type or tagged union:

// A type that can be either an int or a string
Algebraic!(int, string) someVariable;

That’s a fairly simple example. Much more complicated type names are common in D. This sort of thing can be a pain to repeat everywhere it’s used, and can make code difficult to read. alias is often used in situations like this to create a simpler shorthand name:

// A type that can be either an int or a string
alias MyType = Algebraic!(int, string);

// Ahh, much nicer!
MyType someVariable;

There’s one problem this still doesn’t solve. Anytime a compiler error message shows a type name, it shows the full original name, not the convenient easy-to-read alias. Instead of errors saying MyType, they’ll still say Algebraic!(int, string). This can be especially unfriendly if MyType is in the public API of a library and happens to be constructed using some private, internal-only template.

That can be fixed, and error messages forced to provide the customized name, by creating MyType as a separate type on its own, rather than an alias. But how? If this was C or C++, typedef would do the job nicely. There is a D equivalent, std.typecons.Typedef!T, which will create a separate type. But naming the type still involves alias, which just leads back to the same problem.

Luckily, D has another feature which can help simulate a C-style typedef: alias this. Used inside a struct (or class), alias this allows the struct to be implicitly converted to and behave just like any one of its members.

Incidentally, although alias and alias this are separate features of the language, they do have a shared history as their names suggest. Originally, alias was intended to be a variation on C’s typedef, one which would result in two names for the same type instead of two separate types. At the time, D had typedef as well, but it was eventually dropped as a language feature in favor of a standard library solution (the aforementioned std.typecons.Typedef template). As a variant of typedef, alias used the same syntax (alias TypeName AliasName;). Later, alias spawned the alias this feature, which was given a similar syntax: alias memberName this. When alias gained its modern syntax (alias AliasName = TypeName), a lengthy debate resulted in keeping the existing syntax for alias this.

Here is how alias this can be used to solve our problem:

// A type that can be either an int or a string
struct MyType {
    private Algebraic!(int, string) _data;
    alias _data this;
}

// Ahh, much nicer! And now error messages say "MyType"!
MyType someVariable;

There’s an important difference to be aware of, though. Before, when using alias, MyType and Algebraic!(int, string) were considered the same type. Now, they’re not. Is that a problem? What does that imply? Mainly, it means two things:

    1. Although this doesn’t affect any actual code, it can mean the compiler generates extra, duplicate template instantiations. If MyType is passed to one template, and somewhere else Algebraic!(int, string) is passed to the same template, the compiler will now generate two separate template instantiations instead of just one.
      In practice though, this shouldn’t be a problem unless you’re already in a genuine template-bloat situation and are trying to reduce template instantiations. Usually, this won’t be an issue.
    2. Although the alias this means MyType can still be implicitly converted to Algebraic!(int, string), the other way around no longer works. An Algebraic!(int, string) can no longer be implicitly converted to a MyType.
      Arguably, this can be considered a good thing if you believe, as I do, in using domain-specific types. But in any case, you can still manually convert the original type to your MyType with the basic built-in struct constructor:

      Algebraic!(int, string) algebVar;
      auto myVar = MyType(algebVar);

    So when you’re aliasing a large, complicated type name to a simpler name, consider using a struct and alias this instead, especially if it’s a type on offer in a library. There’s little downside, and it will greatly improve the readability of error messages for both yourself and your library’s users.

The #dbugfix Campaign: Round 1 Report

The D Foundation released version 2.080.0 of DMD on May 2nd. That normally would have been accompanied by a blog post highlighting some of the items from the changelog. This time, it should also have included the first update on the #dbugfix campaign. I was on vacation with my wife in the days before and after DConf and never got around to writing the post. It’s a bit late to announce the DMD release now, so this post is instead all about the first round of #dbugfix.

I admit, my initial expectations of participation were low. I suppose I didn’t want to be disappointed. Happily, in the end I had no reason to be pessimistic. There were several issues “nominated” on Twitter and in the forums. I was also pleased that some of the forum posts led to substantial discussion. In fact, at least one of the forum threads led to one of the nominated bugs being fixed well before the period ended, and the fix was included in the 2.080 release.

The results

The following issues were nominated, but were fixed before the nomination period ended. #5227 is the only one I’m certain was fixed as a result of discussion from the #dbugfix campaign. The others might have been, or they may have been fixed in the normal course of activity.

The following issues were open at the time the bug fixers selected which two issues to focus on, along with the “vote” count (+1’s, retweets, or anything which indicated support for fixing the issue) as best as I could determine:

The mandate for the bug fixers is to select at least two of the top five. Of course there has to be some leeway here, given that some issues are extremely difficult to resolve, so they can look beyond the top five if they need to. In this case, given that all of the issues share only four distinct vote counts, the entirety of the list is in play anyway.

I got the bug fixers together before I headed off to my (spectacular) German vacation and asked them to come to a consensus by May 1st on two bugs to fix (I must note that they met their deadline, even though I did not). The two bugs selected were:

There are no time constraints on when these will be fixed. It is hoped that they will be fixed and merged in time for 2.081.0, but they could come in a later release. The point is, each of these is now a subject of attention.

Some details regarding a few of the issues not selected:

  • re: 15692 – a while back, Sebastian Wilzbach revived an old DIP for in-place struct initialization and is part way through an implementation.
  • re: 1983 – there’s an old PR waiting for a champion to revive it.
  • re: 18493 – Mike Franklin has a PR that’s waiting on clarification about whether or not nothrow should be implicit with -betterC.

Keep it going

Now that it’s done, I’m happy to declare the inaugural round of the #dbugfix campaign a success. I had hoped that asking for nominations in the forums would as an alternative to Twitter would spark discussions and am pleased to see that happened. So let’s build on this and keep it going.

The slate was wiped clean and the votes reset from the day I declared the first round finished on April 24th. I haven’t noticed any nominations since then (though I haven’t searched for any yet), but please keep them coming. If there’s an issue on the above list that’s not yet fixed but that you feel strongly about, nominate it again. Or nominate something not on the list that’s stuck in your craw. If you see a nomination on Twitter that you support, retweet it to add your vote. In the forums, give a +1 reply. Keep it going!

And while you’re nominating your issues, keep in mind that fixing bugs is only part of the story. If you really care about seeing issues fixed, please try your hand at reviewing pull requests. Contributors whose fixes languish in the PR queue are demotivated from contributing more, but without more eyes reviewing the PRs, the bandwidth to speed up the reviews isn’t there. More PR reviews beget more merges beget more closed issues beget more motivated contributors and satisfied users.

Barring a change in release schedule, DMD 2.081.0 should be released July 1. This round of nominations will close at the beginning of the last week of June. So tweet out or open a new forum thread for your #dbugfix nominee today. Please include both the #dbugfix hashtag and the bugzilla issue number in the tweet or post title, along with a link to the issue report. That will make my job considerably easier.

And review some PRs!

Thanks to Sebasitan Wilzbach and Mike Franklin for their help with this, particularly in keeping me updated with info about PR and DIPs!