Category Archives: Community

Symmetry Autumn of Code is Underway

Earlier this year, Laeeth Isharc brought an idea to the D Foundation for Symmetry Investments to sponsor a summer of code. He was eager to provide a few motivated individuals the incentive to get some great work done for D and the D community. Sporadic email discussions preceded some chatting at DConf 2018 Munich and the idea subsequently began to pick up steam. By the time the details were sketched out, it had transformed into the Symmetry Autumn of Code.

The SAoC projects

Eight applicants submitted their resumes and project proposals for three slots. Nearly all of the proposals were taken from the SAoC suggestions page at the D Wiki. Given the limited window, the selection process went fairly quick. Of the three selected, two had mentors attached. It took a little while to find a mentor for the third selection, so we extended the milestone deadline for that participant. Now I can happily say that all three are well underway.

A fork-based concurrent GC for DRuntime

Francesco Mecca proposed this project. The goal is to take Leandro Lucarella’s original D1 fork-based GC, port it to D2, adapt it to DRuntime’s GC interface, and culminate with a pull request for DRuntime to present the work and open discussion. Leandro agreed to mentor this project and is working with Francesco to develop a test suite that the port must pass as part of Milestone 2. They have also included documentation in their milestone list, which is good news.

vibe.d HTTP/2 implementation

This one came from Francesco Galla, who is currently pursuing a MSc in Network and Security. The goal here is to enhance the vibe-http library to support HTTP/2. Fittingly, Sönke Ludwig, the maintainer of vibe.d, agreed to mentor the project, but requested someone to share the load due to his schedule. Sebastian Wilzbach stepped up as co-mentor.

This project involves rewriting the current HTTP/1 API, ensuring it works as expected, then incrementally adding support for HTTP/2. Portions of the rewrite were already completed before SAoC came along, but had not yet been tested. As such, testing and bug fixing will be a significant portion of the first milestone.

Porting the Mago debugger to D

This project was proposed by László Szerémi. He’s a heavy user of debuggers in D and wants to see the situation improved. He believes that porting the Mago debugger to D is a major step in that direction. His first two milestones are concerned with translating, testing, and bug fixing. To cap off the SAoC event, he intends to get a GUI frontend up and running with some basic features.

László had no mentor when he applied, and no one had specifically volunteered to mentor any debugger projects, so we put out a call in the forums and I reached out directly to a few people. In the end, Stefan Koch agreed to take it on.

SAoC is not the end

I know that one of the goals Laeeth had behind his initial suggestion for this event was to enhance the D ecosystem. None of the selected projects are simple, one-shot tasks. These are projects which will all require attention, care, and effort beyond SAoC. The participants are getting them started and we hope they’ll continue to maintain them for some time to come, but in the end, these are projects for the community. When SAoC 2018 is behind us, it will ultimately be up to the community to determine if the projects live long and prosper or die young.

I’ll post more about SAoC and the participants as the event goes on. We wish them the best in meeting their milestones!

Funding code-d

At the DConf 2018 Hackathon, I announced that the D Language Foundation would be raising money to further the development of the suite of tools comprising code-d, the Visual Studio Code plugin. The project is developed and maintained by Jan Jurzitza, a.k.a. Webfreak001.

The plan was that if we reached $3,000 in donations on our Open Collective page, we’d make the money available to Jan. When we finally reached the goal, I was quite happy to tweet about it:

Jan and I still hadn’t finalized the details at that point, but we finally got it all sorted a few days later. Before I show how we’ll be using the money, I’d first like to talk about why we’re doing this.

The motivation

As the D programming language community has grown, so has the amount of work that needs to be done. Since D is a community-driven language, much of the work that gets completed is done by volunteers. Unfortunately, the priorities for those who actively contribute aren’t necessarily aligned with those of everyone else, and this can lead to some users feeling that there is little progress at all, or that their concerns are being deliberately ignored. That’s just the nature of the beast. More potential pain points and unfulfilled wishlists means there’s a continual need for more manpower.

Therein lies one of the most fundamental issues that has plagued D’s development for years: how can more people be motivated to directly participate so we can keep pace with the consequences of progress?

To be clear, everyone involved in the core team is aware that lack of manpower is not the only negative side effect we suffer from growth, but it is certainly a big one. Part of my role these days is to figure out, along with Sebastian Wilzbach, ways to mitigate it. My inbox is littered with conversations exploring this topic, along with recommendations from Andrei to research how other projects handle one thing or another for inspiration.

To that end, in early February I launched the #dbugfix campaign with the goal of increasing community participation in the bug fixing process by providing a simple way for folks to bring attention to their pet peeve issues (keep it going people – participation has slowed and we need to pick it up!). We also announced the State of D Survey, initiated by Seb, at the end of February to get an idea of what some of the personal pain points are and where people think things are going swell. We’ve got other ideas in development, one of which I’ll be telling you about in the coming days.

Among the data that jumped out at us from the survey is that Visual Studio and Visual Studio Code were the most popular IDEs/editors among participants. The plugins for both Visual Studio (Visual D) and VS Code (code-d/serve-d/workspace-d) are each maintained by solo developers.

That’s a heavy workload for each of them. Look at the issues list for code-d and you’ll find several there (39 open/132 closed as I write), but the PR list is much shorter (1 open/21 closed as I write). What I infer from that is that the demand for improvements is higher than the supply for implementing them. And from personal experience, I know there are projects like code-d whose maintainers would love to add new features, or improve existing ones, but simply can’t (or aren’t motivated to) make the time for it.

Here then is a good place for a little experiment. What if the D Language Foundation started raising money to help fund the development of specific projects like code-d? By creating the opportunity for those with extra money but little time to directly contribute toward fixing their personal pain points and shrinking their wishlists, can development be focused toward getting bugs fixed and improvements implemented more quickly?

Long story short, I picked the VS Code plugin for our trial run because it was prominent in the survey data, it’s cross-platform, and the serve-d component can be beneficial to other IDE/editor plugins. Open Collective launched their goal system just as I was trying to determine how best to go about it, so it seemed the obvious choice (though we didn’t learn until after our announcement that it wasn’t the system we thought it was, i.e. it’s not a system of multiple, targeted fund raising goals but instead a milestone on the global donation progress bar at the top of our Open Collective page). I contacted Jan just before DConf to gauge his interest, then finally had a meeting with the Foundation board the night before the Hackathon to get approval and decide how to go about doling out the money (we decided the money should be paid out in increments based on completion of milestones, with the option to provide money as needed for targeted development costs, e.g. a Windows license, bug bounties, etc). It all came together fairly quickly.

We plan to continue this initiative and raise money for more work in the future. Some will be like this trial run, with the goal of motivating project maintainers to focus development and spend that extra hour or two a day to get things done. Some will be aimed at specific development tasks, as bounties or as payment for a single part-time or full-time developer. We’ll experiment and see what works.

We won’t be using Open Collective’s goals for this (or any other service) going forward. Instead, we’re going to add a page to the web site where we can define targets, allow donations through Open Collective or PayPal, and track donation progress. Each target will allow us to lay out exactly what the donations are being used for, so potential donors can see in advance where their money is going. We’ll be using the State of D Survey as a guide to begin with, but we’ll always be open to suggestions, and we’ll adapt to what works over what doesn’t as we go along.

As long as there are people willing to put either time or money into the D language, community, and ecosystem, this will surely pay off in the long run with more visible progress, hopefully solving a wider range of pain points than currently happens in our “what-I-want-to-work-on” system of community development. We’re looking forward to see the synergy that results from this and other initiatives going forward.

The milestones

Jan has some specific ideas about how he wants to improve his plugin tools and was happy to take this opportunity to put them all front and center. I asked him to make a list, then estimate how much time it would take to complete each task if he were working on it full time, then divide that list into roughly equal milestones of 3–4 weeks each. That does not mean he will be working full time, nor that each milestone will take 3–4 weeks; payment is not dependent on deadlines. It was simply a means of organizing his task list.

We ended up with two sets of tasks as a starting point and agreed to a $500 payout upon completion of both milestones. Once all the tasks in a milestone are completed, the $500 will be paid. At the end of the second milestone, we’ll determine what to do next – another milestone of feature improvements, a heavy round of bug fixing, or something else – and how to make use of the remaining $2000.

Following is a list of the tasks in both of the initial milestones, with a brief description of each directly from Jan (lightly edited).

Improvements & maintainence

  • add automated tests for windows 7, 10, archlinux, ubuntu, fedora, osx for startup, from scratch project initialization, reloading, adding dependencies
    There are automated tests in the workspace-d repo which test different scenarios and combinations with dub, DCD, and d-scanner, ranging from general usage to weird setups. Identifying more issues (especially on windows), creating more tests and especially running all the tests on several platforms would be a good first step towards a less buggy version of serve-d which should work on all platforms equally well.
  • Single file mode
    A requested feature and one which makes the usage of the plugin a lot easier would be single file usage. Currently, single file usage is very limited and without an open workspace it doesn’t work at all. Adding this (with quick startup time and low resource usage) will make code-d a good alternative to vim or notepad for quickly editing a D file.
  • fix unknown DDoc macro parsing
    Currently, if any unknown DDoc macros are used, they are omitted from the documentation renderer, which makes some documentation unreadable and weird. For this, libddoc needs to be worked on by identifying where exactly the macros are discarded, how they could be kept track of, and how to make it more obvious in the API without breaking backwards compatibility.
  • project template creator as separate plugin (with standardized template format if possible)
    Currently the “Create Project” command is a very static command reading from a directory in the plugin. Making this a separate VS Code plugin with an API for other plugins to extend with more functionality would make it more flexible and open it to new platforms and APIs. Also, research on if there are already some standards for this and if we can parse other template formats.
  • detect used imports & add command to remove unused ones & add Remove unused imports + sort command
    Add a way to find out if imports are used and add a command to remove unused ones. DCD could probably be extended for this.
  • code folding on grammar
    The expand/collapse feature on arrays & functions (bracket based) and lambdas. (Instead of indentation based)
  • parse dub commandline arguments
    Dub command line arguments should be able to be passed into the build buttons and future tasks. This will require parsing them because dub is used as an API and not as a program.
  • experiment with adding a mode to DCD to make all symbols available
    A mode to always give all public symbols regardless of imports and additionally provide the import where the symbol comes from (so when completing it, the IDE adds an import)

UX & Tools for beginners

  • improve UI/UX for building and running projects (implement tasks api) 
    As the tasks API is now actually released, I can incorporate the different build types into the task system of VS Code. This would include having build (debug, release), run, test for each detected dub project and submodule.
  • make workspace symbol search show more symbols
    The workspace symbol search (Ctrl-P -> #) currently is using just DCD to find exact matches for types. I would include DScanner ctag/etag results in this response to make the search better.
  • code action: improve implement interface (parse existing functions and not include them)
    Test cases need to be written, existing interfaces must be parsed, cross-file it should all still work, inherit documentation, etc. Need to check in real world applications where interfaces and classes are used. Also add support for abstract classes.
  • auto complete in diet templates (both HTML tags and D code)
    Add static HTML auto completion for tags & attributes and create a virtual D file for D code for DCD for autocompletion.
  • code action: create getter/setter for variable
    For code like int _foo;, create the property functions int foo() @property const { return _foo; } int foo(int value) @property { return _foo = value; } with automatic detection if const can be used. This makes writing classes easier (snippets already exist, but this is more intuitive) and shows beginners how properties are created. The property should be added where other properties are. It could check the comments for “Getters”, “Setters” and “Properties” if possible, otherwise just put it after a block of members.
  • code action: convert boolean parameter to Flag (+import)
    This should change the definition of a function void foo(bool createDirectory) to alias CreateDirectory = Flag!"createDirectory"; void foo(CreateDirectory createDirectory) and add the appropriate import if it doesn’t exist yet.
  • code action: make foreach parallel (with import if neccessary)
    Replace a foreach (a; array) with foreach (a; parallel(array)) and import std.parallelism.
  • code action: convert enum to bitflag enum (isBitFlagEnum assert + modify values)
    Convert any enum enum Foo { a, b, c } to enum Foo { a = 1 << 0, b = 1 << 1, c = 1 << 2 }, add static assert(isBitFlagEnum!Foo); and import std.typecons if neccessary.
  • search for unknown symbol or import online
    Use dpldocs.info. Example: import mongoschema; -> Error importing -> Search online code fix -> Add dub package mongoschemad, etc. For undefined symbols the same: check local projects for symbols, otherwise see if online providers know anything.
  • show local references from DCD & local rename
    Highlight a symbol in the document when hovering on a member and provide the ability to rename it (not cross-file, because DCD doesn’t support that).

Thank you

Thanks again to all of the donors who pushed us over the goal line for this. Though we realize not everyone will be entirely pleased with the milestone list, and that it would have been better if we had finalized everything before asking for donations rather than coming at it haphazardly like we did, we hope that most of you find more here to like than not.

It will be a little while yet before we set up the new system and announce the next goal, as we’ve got other initiatives taking priority at the moment. So please be patient. We haven’t forgotten about the State of D Survey, nor are we going to let the results sit and bitrot. This train may be slow sometimes, but it’s rolling on.

Communal Benevolence Required

DConf 2018 is coming up fast. The final pieces of preparation are soon to fall into place and then the day will be upon us. Already, we’re looking at what comes next.

We recently put out a survey to gather a good deal of feedback from the community. At that time, we promised that the survey results would be used to guide some initiatives going forward. Though we’ve been silent on that front, it’s still in the works. You should be hearing an announcement at DConf about the first such initiative, and a subsequent blog post will lay out the framework we’ll be following going forward.

One thing all of our initiatives will have in common is that they will require resources. That, unfortunately, is not something the D Language Foundation has in large quantities. Whether we’re talking about time, money, or manpower, D’s development is constrained by the limited resources available. Fortunately for us, one can be used to drum up the other two, so in the absence of enough folks able to donate their time and manpower, we’ll be relying on donations of money to buy what we need to meet the community’s goals.

The same holds for DConf. In order to keep the registration fees relatively affordable, reimburse the speakers, buy the T-shirts, and fund all of the costs associated with the event, the Foundation uses money from the donation pool to cover what the registrations don’t. That’s exactly the sort of thing the donations are for, of course, but the more we dip into the pool for the conference, the longer it takes to replenish after.

So if you’re looking to help the D community, this is an excellent way to do it. You can head over to our OpenCollective page, or choose one of the alternatives on our donation page at dlang.org, and throw a few dollars at us. Leave a note on the donation form letting us know you’re doing this to support DConf 2018.

As a reminder, if you haven’t registered for DConf and aren’t planning to, the Hackathon on May 5th is open to all. So if you’re in the area that day, come on in and see us!

Announcing the DConf 2018 Hackathon Pass

As I write, General Admission for DConf 2018 in Munich has been open for one week. The conference runs from May 2–5, with three days of talks capped off by a day-long Hackathon. We’ve heard there are some in the area who would like to attend, but just can’t get away from work for three days for the full conference. So we’ve decided to do something about it.

This year,  the Hackathon is open, free of charge, to anyone interested in hanging out with us on Saturday, May 5th, the final day of the conference. You’ll have to bring your own lunch, but you’ll get access to the full day. This time around, it’s more than just hacking on and talking about D code.

We’ve added one more new item to the list this year: the Hackathon will be kicked off with a talk by Shachar Shemesh from WekaIO. His talk, Announcing Mecca, introduces the company’s open source D project to the world, a perfect way to get us started on an event intended to motivate hacking on open source D projects.

Last year, our inaugural Hackathon was a great success. It’s not a hackathon in the traditional sense, but an opportunity for D programmers to come together and spend the day working on the projects that they think matter, improve the D ecosystem, or learn more about D programming.

Any attendee with an open source D project is welcome to seek out collaborators to help them get work done on site, or to hunker down and help others hack features and bug fixes into their own projects, or any project in the D ecosystem. Some may simply want to sit and code by themselves, getting instant feedback from others on site when it’s needed, and that’s fine, too. Last year’s event proved to be quite a productive day, particularly for the core D projects.

Those not in the mood for coding, or who aren’t yet up to speed on D programming, are covered, too. Last year, groups gathered to hold brainstorming sessions on ideas for new projects, or for solving complex issues in the core. Others, D’s BDFL Walter Bright among them, took some time during the day to tutor less experienced D programmers, on specific language features and other programming topics, in informal sessions that came about because a few people sitting near each other started asking questions.

In short, the DConf Hackathon is a day of getting things done in a friendly and collaborative environment full of people with shared interests. It’s a loosely organized event, the direction of which is determined by those participating. We’ll throw some ideas out there for you to work on if you need the cues, but you’re free to go your own way.

Last year’s Hackathon fell on a Sunday, and some of the conference attendees had to skip it or leave early in order to get back home for work on Monday. This year, it’s on a Saturday, so we hope that if you weren’t able to stick around last year you can do so this time around.

If you’ve never been to a DConf or don’t know much about the D language, the Hackathon Pass is a great remedy for both ailments. So if you’re in or near Munich Saturday, May 5th, we invite you come on by and hear about a great new open source project, spend a day with a friendly crowd of interesting people, and learn a little about the D programming language in the process.

User Stories: Funkwerk

The deadline for the early-bird registration for DConf 2018 in Munich is coming up on March 17th. The price will go up from $340 to $400. If you’d like to go, hurry and sign up to save yourself $60. And remember, the NH Munich Messe hotel, the conference venue, is offering a special deal on single rooms plus breakfast for attendees.


A few of the DConf attendees are coming from a local company called Funkwerk. They’re a D shop that we’ve highlighted here on this blog in a series of posts about their projects (you’ll see one of their products in action if you take the subway or local train service in Munich).

In this post, we cap off the Funkwerk series with the launch of a new feature we creatively call “User Stories”. Now and again, we’ll publish a post in which D users talk of their experiences with D, not about specific projects, but about the language itself. They’ll tell of things like their favorite features, why they use it, how it has changed the way they write code, or anything they’d like to say that expresses how they feel about programming in D.

For this inaugural post, we’ve got three programmers from Funkwerk. First up, Michael Schnelle talks about the power of ranges. Next, Ronny Spiegel tells why generated code is better code. Finally, Stefan Rohe enlightens us on Funkwerk’s community outreach.

The power of ranges

Michael Schnelle has been working as a software developer for about 5 years. Before starting with D 3 years ago, he worked in (Web)Application Development, mostly with Java, Ruby on Rails, and C++, and did Thread Modeling for Applications. He enjoys coding in D and likes how it helps programmers write clean code.

In my experience, no matter what I am programming, I always end up applying functions to a set of data and filter this set of data. Occasionally I also execute something with side effects in between. Let’s look at a simplified use case: the transformation of a given set of data and filtering for a condition afterwards. I could simply write:

foreach(element; elements) {
  auto transformed = transform(element);
  if (metCondition(transformed) {
     results ~= transformed
  } 
}

Using the power from std.algorithm, I can instead write:

filter!(element => metCondition(element))
       (map!(element => transform(element))(elements));

At this point, we have a mixture of functional and object-oriented code, which is quite nice, but still not quite as readable or easy to understand as it could be. Let’s combine it with UFCS (Uniform Function Call Syntax):

elements.map!(element => element.transform)
        .filter!(element => element.metCondition);

I really like this kind of code, because it is clearly self-explanatory. The foreach loop, on the other hand, only tells me how it is being done. If I look through our code at Funkwerk, it is almost impossible to find traditional loops.

But this only takes you one step further. In many cases, there happen to be side effects which need to be executed during the workflow of the program. For this kind of thing, the library provides functions like std.range.tee. Let’s say I want to execute something external with the transformed value before filtering:

elements
  .map!(element => element.transform)
  .tee!(element => operation(element))
  .filter!(element => element.metCondition)
  .array;

It is crucial that operations with side effects are only executed with higher-order functions that are built for that purpose.

int square(int a) { writefln("square value"); return a*a; }

[4, 5, 8]
  .map!(a => square(a))
  .tee!(a => writeln(a))
  .array;

The code above would print out the square value six times, because tee calls range.front twice. It is possible to avoid this by using functions like std.algorithm.iteration.cache, but in my opinion, the nice way would be to avoid side effects in functions that are not meant for that.

In the end, D gives you the possibility to combine the advantages of object-oriented and functional programming, resulting in more readable and maintainable code.

Generated code is better code

Ronny Spiegel has worked as a professional software developer for almost 20 years. He started out using C and C++, but when he joined Funkwerk he really started to love the D language and the tools it provides to introspect code and to automate things at compile time.

In a previous blog post, I gave a short overview of the evolution of the accessors library. As you might imagine, I really like the idea of using the compiler to generate code; in the end this usually results in less work for me and, as a direct result, causes fewer errors.

The establishment of coding guidelines is crucial for a team in order to create maintainable software, and so we have them here at Funkwerk. There is a rule that every value object (or entity) has to implement the toString method in order to provide diagnostic output. The provided string shall be unambiguous so that it’s more like Python’s __repr__ than __str__.

Example:

StationMessage(GeneralMessage(4711, 2017-12-12T10:00:00Z), station="BAR", …)

The generated string should follow some conventions:

  • provide a way to uniquely reconstruct data from a string
    • start with the class name
    • continue with any potential superclasses
    • list all fields providing their name and value separated by a comma
  • be compact but still human readable (for developers)
    • skip the name where it matches the type (e.g. a field of type SysTime is called time)
    • skip the name if the field is called id (usually there’s an IdType used for type safety)
    • there’s some special output format defined for types like Date and SysTime
    • Nullable!T’s will be skipped if null etc.

To format output in a consistent manner, we implemented a SinkWriter wrapping formattedWrite in a way that follows the listed conventions. If this SinkWriter is used everywhere, this is the first step to fully generate the toString method.

Unfortunately that’s not enough; it’s very common to forget something when adding a new field to a class. Today I stumbled across some code where a field was missing in the diagnostics output and that led to some confusion.

Using (template) mixins together with CTFE (Compile Time Function Execution) and the provided type traits, D provides a powerful toolset which enables us to generate such functions automatically.

We usually implement an alternative toString method which uses a sink delegate as described in https://wiki.dlang.org/Defining_custom_print_format_specifiers. The implementation is a no-brainer and looks like this:

public void toString(scope void delegate(const(char)[]) sink) const
{
    alias MySelf = Unqual!(typeof(this));

    sink(MySelf.stringof);
    sink("(");

    with (SinkWriter(sink))
    {
        write("%s", this.id_);
        write("station=%s", this.station_);
        // ...
    }

    sink(")");
}

This code seems to be so easy that it might be generalized like this:

public void toString(scope void delegate(const(char)[]) sink) const
{
    import std.traits : FieldNameTuple, Unqual;

    alias MySelf = Unqual!(typeof(this));

    sink(MySelf.stringof);
    sink("(");

    with (SinkWriter(sink))
    {
        static foreach (fieldName; FieldNameTuple!MySelf)
        {{
            mixin("const value = this." ~ fieldName ~ ";");
            write!"%s=%s"(fieldName, value);
        }}
    }

    sink(")");
}

The above is just a rough sketch of how such a generic function might look. For a class to use this generation approach, simply call something like

mixin(GenerateToString);

inside the class declaration, and that’s it. Never again will a field be missing in the class’s toString output.

Generating the toString method automatically might also help us to switch from the common toString method to an alternative implementation. If there will be more conventions over time, we will only have to extend the SinkWriter and/or the toString-template, and that’s it.

As a summary: Try to generate code if possible – it is less error prone and D supports you with a great set of tools!

Funkwerk and the D-Community

Stefan Rohe started the D-train at Funkwerk back in 2008. They have loved DLang since then and replaced D1-Tango with D2-Phobos in 2013. They are strong believers in open source and local communities, and are thrilled to see you all in Munich at DConf 2018.

Funkwerk is the largest D shop in south Germany, so we hire D-velopers, mainly just through being known for programming in D. In order to give a little bit back to the D community at large and help the local community grow, Funkwerk hosted the foundational edition of the Munich D Meetup.

The local community is important …

Munich Meetup at Brainlab

The meetup was founded in August 2016, 8 years after the first line of D code at Funkwerk was written. Since then, the Meetup has grown steadily to ~350 members. At that number, it is still not the biggest D Meetup, but it is the most visited and the most active. It provides a chance for locals in Munich to interact with like-minded D-interested people each month. And with an alternating level of detail and a different location each month, it stays interesting and attracts different participants.

… and so is the global community

To engage with the global community, Funkwerk is willing to open source some of its general-purpose D libraries. They can all be found under github.com/funkwerk, and some are registered in the DUB registry.

To mention are:

  • accessors – a library to auto generate getters and setters with UDAs
  • depend – a tool that checks actual import dependencies against a UML model of target dependencies
  • d2uml – reverse engineering of D source code into PlantUML class outlines

Feel free to use these and let us know how you like them.

The D Language Foundation at Open Collective

In its work guiding the development of D and promoting its adoption, the D Language Foundation is driven primarily by donations big and small. The money comes in from different sources, the most visible being those listed on the website’s donation page, and is put to use in different ways.

Donors typically receive an email thanking them for their generosity. Recently, we added a sponsors page to shine a light on those who have given and who are willing to have their names on public display. That and a line in the Vision Document about average monthly expenses are the only obvious bits of transparency in the process.

Today, the D Language Foundation is opening a new chapter in the donation story with our Open Collective page. According to OpenCollective.com, an open collective is,

A group of people with a shared mission that operates in full transparency.

The site allows us to set up packages that donors can choose from, with or without rewards, for one-time and recurring donations, at levels within reach of individuals and those more suited for corporate budgets. Donors can leave notes with their donations to tell us what they think of our work or what’s important to them. We can submit expenses to show how the money is being used, and set up fund drives for specific targets.

In short, Open Collective gives us new possibilities in raising money, spending it, and showing how it’s spent, while also providing more opportunities for D community members to participate. We could, for example, designate a specific amount to put toward the development of a particular language or ecosystem feature of importance to the community and ask community members to help us meet that goal. A perfect opportunity to contribute for those eager to see progress in areas that matter to them. Crowd-sourcing for a niche crowd.

Those who wish to remain in the shadows can still donate behind the scenes via our other sources. Additionally, I wouldn’t expect all Foundation expenses to be listed at Open Collective. We’re inexperienced with this platform yet, so it will take us a bit of time to learn how to make it work best for all of us, but we have high hopes that it will prove beneficial in the long run.

On a related note, if you shop at Amazon, you can help us out by making your purchases via smile.amazon.com and choosing the D Language Foundation as your charity. A small percentage of your purchases through smile.amazon.com will go to the Foundation as long as it is selected as your charity. From now until March 31, the donation percentage is tripled.

Amazon Smile

Don’t forget, the State of D Survey is still open for a couple more days. If you haven’t completed it yet, please take the time to do so. We’re looking forward to see what comes from it.

The New New DIP Process

When I took on the role of DIP Manager last year, my number one goal was to clear out the queue. I made a few revisions to the process and got busy. Over the next few months, things went along fairly well, not so much from anything I did as from the quality of the submissions. But at some point, things broke down and the process stalled.

Near the end of the year, Andrei asked me to make two specific changes to the process. One of them was to come up with a different approach for handling the final review. To that end, he suggested I look at how other languages handle the review of their enhancement proposals.

Before I started, I identified some other areas of the process that were problematic so that I could keep an eye out for ideas on how to shore them up. Might as well overhaul the whole process rather than one small part of it. So I put the entire DIP process on hold until the new process was ready to go.

The main thing I learned from looking at other language processes was that about the only thing they have in common is that they use GitHub repositories to store their proposals and that new submissions are made through PRs. Beyond that, there’s a quite a bit of variety in how they handle review and evaluation.

Ultimately, I decided the basic framework we already had was well-suited for our circumstances. It just needed some serious tweaking to iron out the problem spots. So I set out to write up a new procedure document to address the things that needed addressing. When it was done, it took a while to get the seal of approval, but it finally came through and the process is no longer stalled.

So first, before describing the major changes, it will help to understand their motivation.

What went wrong

One of the earliest stumbles I had as DIP Manager was a mix up over DIP 1006. I had gotten it in my head that the author intended to rewrite it before moving forward. The reality was that I had informed him before DConf that I would get it going at the end of May. The result was that it sat there for several months before I realized my error.

Another problem came with DIP 1009. There was an issue with the way it was written – the style didn’t meet the standard laid out in the Guidelines document. This led to multiple email exchanges, with massive delays and more misunderstandings, that resulted in the DIP being stuck in limbo for quite a while.

The communication problem over DIP 1009 was what prompted the process revision. For the Final Review of each DIP, I was acting as the middle-man between Walter and Andrei on one side and the DIP Author on the other. It worked well enough as long as little further effort on the DIP was needed, but as soon as there were questions that needed answering or more work to do, it became inefficient, cumbersome, and prone to misunderstanding.

Perhaps the biggest issue of all was time. DIP 1006 sat in Draft Review with nothing from me for several months. DIPs 1006, 1009, and 1011 have been in the Final Review stage for ages. There’s no reason any DIP author should have to wait for months on end with no feedback, or vague promises, no matter which stage of the process a DIP is in. It’s discouraging and demotivating. The process should require some motivation and effort from DIP authors, but it should also require a commitment from the other side to keep the authors informed and to get each DIP through from beginning to end as efficiently as possible.

Some of these problems could have been avoided if I had taken a different view of my role. I saw the role of DIP Manager more as that of a Shepherd than a Gatekeeper. The ultimate fate of a DIP rested on the Author’s shoulders, not mine. The Guidelines were “more what you call guidelines than actual rules”. After I made my revisions to the Guidelines document early on, they fell right out of my head and I never looked at the document again.

Righting the wrongs

The new Procedure document outlines the new process. Following is a summary of the big ones.

A minor issue is that there was some confusion about the existing review-stage names. There are now four review stages rather than three: Draft Review, Community Review, Final Review, and Formal Assessment. The Draft Review is the same as before. The Community Review is the new name for the old Preliminary Review. The old Final Review, which had two parts, has been split out into the Final Review and the Formal Assessment – the former is the last chance for the community to leave feedback, and the latter is Walter and Andrei’s decision round.

For all but the Draft Review, each stage specifies a maximum amount of time that a DIP can go without progress. For example, a DIP may remain in the Post-Community Round N state for 180 days, and a DIP in Formal Assessment should receive a final disposition within 30 days. The document defines the steps that must be taken when these deadlines are not met.

Related, though not in the document, is what I will do to keep to the deadlines. I’ll be making use of the calendar in the D Foundation’s Google account to post the start and finish date of each stage for each DIP. When a DIP is between stages, I’ll set milestone dates so that the DIP Author and I can have a clear target to aim for. If we’re on the same page, there will be less opportunity for uncertainty and misunderstanding.

The document provides for a new process for handling the Formal Assessment. No longer will I be a middleman between two email chains. Now, Walter and Andrei will provide their feedback on a private gist, with direct participation by the DIP Author. This should help things move more quickly and will eliminate (or greatly reduce) the chance of anyone (me in particular) causing more delay by getting things mixed up.

Another change is the requirement for a Point of Contact (POC). From here on out, every DIP must have a POC. For a single-author DIP, the DIP Author is the POC. If there are multiple authors, they must select one from among themselves. The need for this came to light after a misunderstanding that arose from the communication problem. The POC must commit to seeing the DIP through to the end of the process. The document outlines what happens to a DIP when the POC becomes unavailable.

Another change that’s not outlined in the document is in how I view my role as DIP Manager. From here on out, I will consider the guidelines as actual rules. I’ll do my best to make sure a DIP meets the standards expected in terms of language and style before it leaves the Draft Review stage. We can tweak it as we go, of course, but never again should a DIP be sent back for revision because it’s too informal.

Open to refinement

The new Procedure document and the undocumented tweaks to my process are the result of lessons learned over several months. That doesn’t mean they’re perfect. We’ll always be open to suggestions on how to patch up any holes that are identified. Not every change was mentioned above, so please read the document for the details.

Hopefully, the three DIPs currently awaiting a final disposition will be resolved before too much longer. After that, DIP 1012 will be moved forward for the Final Review and Formal Assessment to become the first DIP to go through the new gist-based review. DIP 1013 (which will likely be the one introducing binary assignment operators for properties), will be the first test-case for the new process in its entirety. Let’s all keep an eye open for what works and what needs work.

And to everyone, thanks for your patience while I went through my growing pains and we got the mess sorted out. Now that the train is back on the tracks, I’ll do my best to keep it moving.

The State of D 2018 Survey

NOTE: The survey is closed. Thanks to everyone who participated!


Strange things are afoot at the D Language Foundation. Odd noises and varicolored lights have been reported emanating from the cellar into the wee hours of the morning. Foundation members have been sighted, stumbling dazed and bleary-eyed in and out of the front door, arms full of mysterious black boxes. Neighbors whisper, and rumor has it that the spawn of so much secretive activity is only one arcane ritual away from seeing the light of day.

How right they are! For the past few weeks, the initiate Sebastian Wilzbach has devoted his energies to studying the Book of Modern Arcana in preparation for the ritual known as the State of D 2018 Survey. With feedback from those already steeped in the Dark arts, he has been refining the incantations of the ritual so that they prove most effective. Now, at long last, his preparations are complete and the ritual has been unleashed upon the world!

Upon the D community anyway.

And now it’s on you. This is your chance to turn your praise, complaints, and nitpicks into action. By participating in the State of D survey, you’ll be providing guidance to the D Language Foundation to help identify both short and long-term goals for the future development of D and its ecosystem.

A handful of initiatives are already coming together in the cellar. You’ll be able to read about them in more detail here as they are announced in the coming days, weeks, and months. The 2018 H1 document, presenting an overview of the current focus, will be announced soon. This survey will help build on existing plans, fill in the details of the general goals, and identify any course corrections that are necessary for 2018 H2 and beyond.

The Foundation is eager to address the issues that matter most to the community. Members frequently make their thoughts known in the forums, but the conversations can be long, sprawling, wandering, and hard to follow. The State of D Survey will allow the core team to see at a glance what’s going right and what’s not, and to focus their attention where it’s needed most. It could take 15 minutes or more of your time to complete, depending on the amount of thought you put into your answers. If you care about the D programming language, it’s well worth every minute.

We’ll leave the survey open for at least two weeks. A short while after it closes, we’ll publish the results here. From then on, the blog will cover what’s happening in response, providing updates on progress at reasonable intervals. Then next year we’ll do it all again.

Remember, D is a community-driven language, but not everyone is able to contribute as much as they would like. Whether you’re a frequent contributor or just getting started writing D programs, this is your chance to help make D an even better language than it is today.

Make your opinion count and take the survey!

Project Highlight: The D Community Hub

As has been stressed on this blog before, D is a community-driven language. Because of that, the ecosystem depends on the work of volunteers who are willing to contribute their time and open their projects to the community at large. From IDE and editor plugins to libraries in the DUB registry, it’s all about the efforts of people who are (usually) getting no monetary reward for their efforts.

There are some inherent downsides to that reality. Sometimes projects are abandoned. Sometimes they aren’t updated as frequently as users would like. This can become an issue for those who depend upon these projects, but it’s alleviated by the fact that most D projects are open source and their repositories are publicly available. To keep a project alive and up-to-date only requires more volunteers willing to pitch in.

That’s the motivation behind the D Community Hub (dlang-community) at GitHub. According to Sebastian Wilzbach, it started with Brian Schott’s popular tools used by several IDE and editor plugins:

There were maintenance issues with Brian’s (aka Hackerpilot) awesome projects. He has a full-time job and often could only respond to simple issues every few weeks. This meant that simple bug fix PRs sat in the queue for quite a while. For example, there was one case where the same PR to fix the Windows build script was submitted by three different people (there was no Windows CI at the time).

Brian’s projects weren’t the only ones that motivated the idea. Sebastian and Petar Kirov maintain the DLang Tour, and some of the projects they depend upon were either inactive or slow to update. However, Brian’s tools are widely used, so they started with him. Eventually, they convinced him to move some of his projects to the new organization and others followed.

Sebastian lays out the following benefits that have come from moving multiple projects from disparate developers under an umbrella group:

  • Common best policies (e.g. all repositories have GitHub branch protection)
  • No need to fork an inactive repository – work can be shared easily.
  • No dependence on a single person who might be busy or on vacation (this is especially important for swiftly pulling and releasing bug fixes )
  • One common location whenever updates are required (e.g. package bumps or deprecation fixes)
  • Many of the projects are enabled on the Project Tester (their test suite is run on every PR for the DMD, DRuntime, Phobos, Dub, and tools repositories to prevent regressions) – this is possible because many people have merge rights in case an improvement in the compiler finds critical bugs or deprecations are moved forward
  • Shared knowledge (e.g. all projects support “auto-merge” like the dlang repositories)
  • Automation with bots – Mark Rz (@skl131313) created a bot that automatically triggers update PRs whenever dependencies are updated (some of the projects in dlang-community still support builds with only git submodules and make)
  • Less overhead for automation with CIs (everyone can connect a repo to a third-party provider or restart a failing CI job)

It has also resulted in increased participation. For example, other D users have joined the group, and Sociomantic Labs (the D shop in Berlin that hosted the 2016 and 2017 editions of DConf) has taken over the release process for dfmt, Brian’s tool for formatting D source code.

There are currently 22 repositories in the dlang-community organization, including the following:

  • DCD (the D Completion Daemon) – an autocomplete program that is used by several D IDE and editor plugins
  • dfmt – a formatter for D source code, also used by many IDE and editor plugins
  • D-Scanner – a tool for analyzing D source code
  • dfix – a tool for automatically upgrading D source code
  • libparse – a library for lexing and parsing D source code
  • drepl – a DMD-based REPL for D
  • stdx-allocator – a frozen version of std.experimental.allocator (which is due for an overhaul)
  • containers – a set of containers backed by stdx.allocator to easily switch between different allocation strategies, with or without the GC
  • D-YAML – A YAML parser and emitter
  • harbored-mod – a documentation generator that supports both D’s built-in Ddoc syntax and Markdown

In addition to other D projects, there’s a repository set up specifically to discuss the dlang-community organization via GitHub issues, and repositories that contain artwork. If you decide to use any of these projects, the discussion repository is the place to ask for help when you need it.

Other projects may be added in the future. According to Sebastian, there are a few questions that form a set of loose criteria for inclusion under the dlang-community umbrella.

  • Is there enough interest from the general public so that it is “worth maintaining”?
  • Is there a similar library with active development out there?
  • Is at least one DLang community member competent for the domain covered by the project? If no, is there anyone who’s willing to fill the role?

Sebastian and the others are looking to add a few features over time. These include:

  • More automatic documentation builds
  • Automatic build of binaries on new tags (especially for Windows)
  • d-apt: Sociomantic is working on moving d-apt to GitHub and enabling full automatic CI builds for it.
  • dfmt: Leandro Lucarella / Sociomantic is introducing neptune and a proper release process

For anyone interested in joining the dlang-community organization, there are two options. If you are already a well-known participant in the D community, simply ping one of the existing members for merge rights. For anyone else, the best approach is to start contributing to one or more of the dlang-community projects to build up trust. At some point, frequent trustworthy contributors will be welcomed into the fold.

As for the current contributors, Sebasitian says:

There are many people working behind the scenes on the dlang-community libraries. A special thanks goes to the active reviewers who make it possible that your potential PR gets merged in a timely manner.

  • Basile Burg
  • Brian Schott
  • Jan Jurzitza
  • Leandro Lucarella
  • Martin Nowak
  • Petar Kirov
  • Richard Andrew Cattermole
  • skl131313
  • Stefan Koch

If you have or know of a D project that is suffering from a lack of attention, bringing it to the dlang-community might be the way to breathe new life into it. Don’t be shy in asking for help.

The #dbugfix Campaign

Why so many bugs?

Every major release of DMD comes with a list of closed issues from Bugzilla. For example, looking at the changelog for DMD 2.078.0 shows the following counts for closed regressions, bugs, and enhancements: 51 for the compiler, 37 for the standard library, 6 for the runtime, 17 for the website, and 1 for the linker. That’s 112 total issues, the majority related to the compiler. The total number of closed issues fluctuates between releases, but the compiler and standard library normally get the lion’s share.

This isn’t news to anyone who regularly follows DMD releases. But spend enough time on the forums and you’ll eventually see someone under the impression that bugs aren’t getting fixed. They cite the number of open issues in the database, or the age of some of the open issues, or the fact that they can’t find any formal process for fixing bugs. In reaction, it’s easy to point to the changelogs, or cite the number of closed issues in the database, or bring up the number of open issues in other language compilers. And, of course, to explain once again that this is a volunteer community where people work on the things that matter to them, and organizing groups to complete specific tasks is like herding cats.

That’s all quite reasonable, but really isn’t going to matter to someone who found the motivation to check D out, but is still looking for the motivation to stay. For me personally, I really don’t care how many issues are in the database, or the age of the oldest. All I care about is that it works for me. But I’m already invested in D. I don’t need to be motivated to stick around. And while I wouldn’t use a bug database as criteria to judge a new language, I can see that others do. It’s akin to looking at a stable repository on GitHub and dismissing it as abandoned because of its lack of recent activity. If you don’t see the whole picture, you can’t make an informed judgement.

If perception were the only issue, then it would simply be a matter of web design and PR. However, there have been, and are, people invested in D who have become frustrated because issues they reported, or that directly affect them, have languished in Bugzilla for months or even years. This can’t simply be dismissed as not seeing the whole picture. This is a matter of manpower and process. A number of issues are still open because there isn’t a simple fix, or perhaps because no one has taken an interest. The set of people who can solve complex issues is small. The set of people willing to work on issues that aren’t interesting to them is smaller. But again, how do you get a disparate group of volunteers of varying skill levels to devote their free time to fixing other peoples’ problems?

This is something the D community has struggled with as it has grown. There are no easy, comprehensive solutions without a full-time team of dedicated personnel, something we simply don’t have. However, it’s possible that there are opportunities to take baby steps and chip away at some of these issues without the complications inherent in herding cats.

The #dbugfix campaign

To recap, there are two primary complaints about the D bug-fixing process (such as it is):

  • Too many (old) bugs in the database
  • Bugs you care about aren’t getting fixed

In an effort to alleviate these problems, one baby step to chip away at it, I’m announcing the #dbugfix campaign.

It works like this. If there is an issue in D’s Bugzilla that you want to see fixed, whatever the reason (maybe it’s blocking you, or it’s annoying you, or it’s an enhancement you want, or you think it’s too old – it doesn’t matter), then either tweet out the issue number with #dbugfix in the tweet, or create a topic in the general forum with the issue number and #dbugfix in the subject line. I’ll monitor both Twitter and the forums and keep a running tally of issue numbers.

A week before a major version of DMD is released (starting with 2.080.0, which is slated for May 1), I’ll look at the tally and find the top five issues. I’ve already gotten people to commit to fixing at least two of the top five. That doesn’t mean only two. It could well be more. It depends on the complexity of the issues and how many other volunteers we can scrounge up. Hopefully, the two (or more) fixed bugs will be ready to be merged in the subsequent major release.

In the blog post announcing each major release, I’ll report on which bugs in the current release were fixed as a result of the campaign and announce the two selected for the subsequent release. If any of the top five from the previous release were not fixed, I’ll call for volunteers to help so that they can be squashed as soon as possible.

Yes, I know. We enabled voting on Bugzilla issues and that didn’t change anything. That’s because there was no real commitment to fixing any of the highest-voted issues. The votes simply served a guideline for the people browsing the database, looking for issues to fix. For this campaign, there really are people committed to fixing at least two of the issues that float to the top for every major release.

But two is not a lot! No, it isn’t. But it also isn’t zero. As I mentioned at the top of this post, dozens of issues are already fixed with each major DMD release. The problem (for those who see it as such) is that there’s currently next to zero community involvement in deciding which issues get fixed. This campaign gives the community more input into the selection process and it provides public updates on the status of that process. It is my hope that, in addition to changing perception and chipping away at the bug count, it encourages more people to help fix bugs.

If you would like to volunteer your time and knowledge to helping out with this campaign and increase the number of #dbugfix bugs fixed in each release, please email me at aldacron@gmail.com. For everyone else, I’ve got a search for #dbugfix set up in my Twitter client, so start tweeting!