Category Archives: News

Bugzilla Reward System

Digital Mars D logo

The Dlang bot has been updated to track Bugzilla issues
that have been fixed. It went live for testing on the 2nd of July. Each GitHub user who fixes a bug via a merged pull request is awarded a number of points depending on the severity of the issue. The current results can always be seen on the contributor stats page. This blog post covers all of the details regarding the implementation, rules, and prizes of the reward system.

Raison d’être

I want to start by saying that the motivation of this system is not to start a fierce competition between contributors to fix as many issues as possible. The primary reasons: we see this as a means for the D Language Foundation to reward committed contributors and to channel their efforts towards more important bugs. If, as a side effect, the system motivates people to fix more bugs, that’s great! We won’t complain.

There are some negative side effects that are possible with any sort of gamification system, and we’ll be keeping an eye out for them. We think we have one of the best online communities out there. Our members are generally friendly and helpful, and we don’t want to do anything that causes tension or proves negatively disruptive. We think this will be a fun way to reward our contributors, but we will pull the plug if it proves otherwise.

Scoring system

The scoring is designed to reward contributors based on the importance of the issues they fix, rather than the total number fixed. As such, issues are awarded points based on severity:

  • enhancement: 10
  • trivial: 10
  • minor: 15
  • normal: 20
  • major: 50
  • critical/blocker: 75
  • regression: 100

Of course, the severity of an issue does not necessarily reflect the complexity of the solution. There might be regressions that are trivial to solve, and enhancements that require an extremely complicated fix. The message that we are trying to send is that complexity is secondary to need. That is why regressions are given top priority and critical/blocker/major issues aren’t far behind.

Rules

The following rules will guide how points are awarded from the initial launch of the reward system. They are not set in stone and are open to revision over time.

Rule #1: The severity of an issue will be decided by the reviewers of a proposed patch.

Severity levels are not always accurately set when issues are first reported and may not have been updated since. The reviewer of a pull request that closes a Bugzilla issue will evaluate the issue’s severity level and may change it if he or she determines it is inaccurate. I will moderate any disagreements that may arise about severity levels.

Rule #2: A PR fixing a bug may not be merged by the same person that proposed the patch.

This is already an unwritten rule that applies to the DLang repositories, so it should not surprise anyone.

Rule #3: Anyone who adopts an orphaned PR that fixes a bug may be awarded its associated points.

To avoid any authorship conflicts, it is best if the adopter contacts the original author to ask if it is okay to adopt the PR. Rule #3 will apply if there is no response or if the response is affirmative. Otherwise, no points will be awarded.

Rule #4: Only one person may receive points per fixed issue.

This rule is specifically designed for reverted PRs. Imagine that a PR that presumably fixes an issue is merged and the author gets points for it. Later on, it is decided that the fix is incorrect and the PR is reverted. If someone else proposes the correct fix, the points will be subtracted from the original contributor and awarded to the new author. Hopefully, this will motivate the original contributor to propose the correct fix after the reversion.

Rule #5: Incomplete fixes still get points.

A Bugzilla report usually includes a snippet of code that reproduces the issue. A frequent pattern is that the bug is correctly fixed for the provided snippet, then someone comes up with a slightly modified example that does not work and reopens the issue. Since the original fix was correct, but not complete, the procedure here is that the original issue should be left closed and a new one should be opened. The original author keeps the points awarded for the original issue.

Implementation

Since most of you are die-hard geeks and are eagerly awaiting the code, here’s the database implementation hosted on the dlang-bot, and here’s the web page implementation. You will notice that the web page is extremely minimal. That is because I am a total n00b when it comes to web programming, so if anyone has the skills and the time to make a cooler web page, feel free to make a PR :D.

In short, for each of the issues that are fixed, the database stores the Bugzilla issue number, the GitHub ID of the person who fixed it, the date when the fix was merged, and the severity of the issue. Every time the leaderboard page is accessed, a query is issued to the database to compute the total points for all of the contributors and sort them in descending order. Easy peasy.

I would like to thank Vladimir Pantaleev for his continued support and assistance throughout the period that I implemented the system.

Prizes

As Mike briefly described in this forum post, we are going to have quarterly competitions. The quarterly prizes will vary. At the end of the year, the person who has acquired the most points will be awarded a bigger prize.

For the inaugural competition, which will officially start on the 20th of September 2021 and will last until the start of DConf Online (the 20th of November), the prizes will be:

  • First Place: a $300 Amazon eGift Card
  • Second Place: a $200 Amazon eGift Card
  • Third Place: a $100 Amazon eGift Card

The next set of prizes will be announced at DConf Online, so stay tuned!

That’s all folks!

If there are any questions or suggestions regarding any aspect of the bugfix reward system, please contact me at razvan.nitu1305@gmail.com. Also, feel free to directly propose changes to the existing infrastructure.

Happy coding everyone!

SAOC 2021 Projects

Digital Mars D logo

The applications have been reviewed, the results decided, and the applicants notified. Five coders will be participating in the 2021 edition of the Symmetry Autumn of Code, one of whom will be the first to take part in SAOC two times.

Following is a brief introduction to each participant and an equally brief summary of their projects. The project planning phase officially kicks off on September 1st, so any details I could provide from their applications would likely change by the time they finalize their initial milestones with their mentors. If you’re eager for more detail, please hold out a little while longer. The participants will start posting updates in the forums once their projects are underway. Their first updates should include more information.

Rethinking the default class hierarchy

If you followed SAOC 2020, you may recall that Robert Aron was a fourth-year student at University POLITEHNICA of Bucharest who worked on implementing D client libraries for the Google APIs, along with a tool to generate client libraries for said APIs (all of which can be found in his GitHub repositories). He also was a recipient of the final SAOC payment (one of two last year, where usually we have only one) and is owed a free trip to a future real-world DConf.

Robert is now working toward an MSc in Security of Complex Networks at the same university, and he’s back with us for SAOC 2021. His project this time around is a DIP for and implementation of the ProtoObject concept that Eduard Staniloiu described in his DConf 2019 talk. This will set a ProtoObject class as the root class of D’s object hierarchy and the ancestor of the existing Object class. It will allow users to opt-in to features currently provided by default through Object, such as the inclusion of a monitor to support synchronization.

Once again, Robert will be working with Eduard Staniloiu and Razvan Nitu as his mentors.

Welcome back, Robert!

Replace DRuntime hooks with templates

Teodor Dutu is also at university in Bucharest working on a master’s degree in Advanced Cybersecurity. He has experience in C and Java, and it’s the low-level experience he gained working on projects like a file system, a kernel module, and an asynchronous HTTP server that he wants to apply toward improving the D ecosystem. The D language grabbed his interest when he participated in Razvan and Edi’s D Summer School, and he is eager to help out where he can.

To that end, Teodor is entering SAOC to work on a change to DRuntime. Currently, certain operations in user code are rewritten to call functions in the runtime known as runtime hooks (if you’ve ever seen a linker error mentioning something like _d_newArrayT or a symbol with a similar name, that was a runtime hook). There are some significant downsides to this approach, such as code bloat (the entire DRuntime library is linked in when linking statically), negative performance impact (due to the use of TypeInfo to pass runtime information to the hooks), and code that’s hard to maintain (the hooks are inserted at the IR level, a component of the compiler that’s difficult to understand).

Teodor’s plan is to replace each of the runtime hooks with templates. Dan Printzell already did some work on this, and Teodor will be following in his footsteps intending to take it all the way.

Eduard Staniloiu and Razvan Nitu will be Teodor’s mentors.

Implement support for D in LLVM Debugger (LLDB)

Luís Ferreira has extensive experience with C, C++, and D. He has contributed to DMD, DRuntime, and Phobos, and has a WIP implementation of DIP 1029 (Add throw as a Function Attribute) underway.

One of the projects Luís has been working on in his free time is a rewrite of DRuntime’s demangler to avoid exceptions, taken on because of his interest in mangling and demangling. He also has an interest in LLVM. The combination sparked the idea for his SAOC project. His rough goals for the project are to add support to LLDB for demangling D symbols, recognizing D-specific data structures, and parsing D expressions.

Mathias Lang has taken on the role of mentor for this project.

Light Weight DRuntime (LWDR)

Dylan Graham made waves on this blog when he wrote about the custom gearbox controller he built, using D for its firmware. That project led him to a new one: D needs a runtime that is suitable for embedded, Internet of Things, and real-time operating systems. That’s when he started work on LWDR.

You can see from that link that LWDR is not a port of DRuntime, but “a completely new implementation for low-resource environments”, and you’ll find a list of features that are currently supported. For SAOC, Dylan will be working on expanding feature support, shoring up what’s already there and adding new features along the way.

Dylan is a university student in Australia, currently pursuing a Bachelor of Computer Science through Monash University. He’s been programming since he was 11 years old, starting with C on the Arduino Uno and BASIC on the Maximite. His courses have exposed him to several other languages, and he has shown he’s a good hand with D.

His mentor for SAOC 2021 is Adam D. Ruppe.

Improve DUB: solve dependency hell

Ahmet Sait Koçak is a Computer Engineering student from Turkey. He has a strong background in C#, but considers D his second-most comfortable language. Some might be familiar with his work maintaining bindbc-harfbuzz.

For his SAOC project, he made use of our Projects repository and settled on the idea of solving the “dependency hell” problem that can arise when using DUB. Essentially, if library A depends on libraries B and C, which in turn depend on two different versions of library D, dub will error out without any effort to resolve the version conflict.

In reviewing the application, the judges identified some issues with the project as proposed, but it was still accepted with the understanding that Ahmet may need to take a different approach. His project subsequently gained the distinction of being the first SAOC project application discussed in a D Language Foundation meeting. The goal was to determine if there might be another way.

Ahmet’s mentor is Max Haughton, who was present for the meeting. He will be working with Ahmet to investigate the solution arrived at in the meeting and, if that proves infeasible, to move forward with the initial idea. Either way, you’ll hear the details from Ahmet in his weekly forum updates.

Onward!

The SAOC judges (Átila Neves, Robert Schadek, and John Colvin) were impressed with the quality of the applications this year and are eager to see how the projects turn out. Please keep an eye out for the weekly updates that should start arriving in the forums around September 22nd, a week after Milestone 1 begins. This will help you keep abreast of the progress of each project and also provide an opportunity for suggestions that might help our SAOC 2021 coders along their paths.

Milestone 1 kicks off on September 15th, and Milestone 4 will end on January 15th. The D Language Foundation and our sponsors, Symmetry Investments, wish these five coders well in all they do over those four months. Their success is the D community’s success, so we hope everyone will join us in ensuring they have all the support and help they need to get through their four milestones and see this thing through to the end.

D News Roundup

Version 2.097.0 of DMD, the D programming language reference compiler, was released on June 5th in the middle of new GDC and LDC release announcements, while preparations for two major D community events were underway: the Symmetry Autumn of Code 2021 and DConf Online 2021. We’ll cover it all in this post, with a focus first on the events.

Symmetry Autumn of Code 2021

Symmetry Investments logo

As I write, Symmetry Investments employs in the neighborhood of 180 full-time workers and manages over US$8 billion of capital, and they’re always on the lookout for more employees, including programmers to work with D and other languages. They sponsored DConf 2019 in London and have sponsored the annual Symmetry Autumn of Code since 2018, in which a handful of programmers are paid to work for four months on projects of benefit to the D ecosystem.

This year marks the fourth annual SAoC, and we are now accepting applications. Participants will plan four milestones for projects that benefit the D ecosystem and will be expected to work at least 20 hours per week on each milestone. Each participant will be rewarded US$1000 for the successful completion of each of the first three milestones. At the end of the final milestone, the SAoC committee will review the overall progress of each of the remaining participants. One will be rewarded with a final $US1000 payment and a free pass to the next real-world DConf, with reimbursement for travel and lodging. In last year’s event, a second participant was also awarded a fourth US$1000 payment.

Participation in SAoC has led to jobs for some lucky coders and has generally been a valuable learning experience for those who have completed it. Students currently enrolled in graduate or postgraduate university programs will be given priority, but applications are open to all. The application deadline is August 18th. Project ideas can be found in the D community’s projects repository at GitHub. See the Symmetry Autumn of Code page here at the D Blog for all the details on how to apply as a participant or as a mentor.

DConf Online 2021

For the second consecutive year, we were unable to hold a real-world DConf. Last year we launched the first annual DConf Online. And when I say annual, I mean annual! We’re doing it again this year and will continue to do it going forward even after the real-world DConfs are back on.

DConf Online 2021 will take place November 20 and 21 on the D Language Foundation’s YouTube channel. Once again, we’re looking for pre-recorded talks, livestream panels, and livecoding sessions. If you’d like to propose something in one of those categories, the application deadline is September 5. Please visit the DConf Online 2021 homepage for all the details.

And if you haven’t seen them yet, the DConf Online 2020 and DConf Online 2020 Q & A playlists are available on the same channel. You can also find a full list of talks and all the links (talk videos, slides, and Q & A videos) on the DConf Online 2020 homepage.

New compiler releases

D 2.097.0 is live in the latest release of DMD and the beta release of LDC, the LLVM-based D compiler. The new version of GDC also came into the world as part of GCC 11.1 at the end of April.

DMD 2.097.0

Digital Mars D logo

This version of DMD comes with 29 major changes and 144(!) fixed Bugzilla issues courtesy of 54 contributors. Changes include a few deprecations and several improvements to the standard library. Two things stand out:

  • while(auto n = expression) has been on a few wishlists for a while. Now it’s a reality. The same syntax that was already possible with if statements is considered idiomatic in certain circumstances (such as when checking if an item exists in an associative array). Expect the while condition assignment to start popping up in open-source D projects soon.
  • std.sumtype is another wishlist item that is a wish no more. The new SumType is a replacement for std.variant.Algebraic. It’s a discriminated union that makes good use of Design by Introspection with a nice match syntax for those looking for that sort of thing. It’s been quite a while since the last time a new module was added to the D standard library. Many thanks to Paul Backus for putting in the effort to see it through, and a very big Congratulations!

LDC 1.27.0-beta1

LDC logo

On the same day the new DMD was released, the first beta of LDC 1.27.0, which also supports D 2.097.0, was announced in the D forums.

On top of 2.097.0 support, this version of LDC provides greatly improved DLL support on Windows. The prebuilt Windows packages ship with DRuntime and Phobos DLLs. This is big news for D developers on Windows. We’ve long had issues with D DLLs that have prevented heavy use outside of simple interfaces (with APIs exported as extern(C) being the most reliable).

There are some limitations to be aware of, such as the inability to directly access TLS variables across DLL boundaries (though it’s fine with accessor functions). Please see the release page for the details.

Thanks to Martin Kinkelin and all the LDC maintainers and contributors for their continued work on LDC. They aren’t getting paid for this. If you are a happy LDC user or just like the idea of the project, you can support their work by sponsoring Martin Kinkelin on GitHub.

GDC 11.1

In the GCC world, Iain Buclaw continues to make strides on the GDC compiler.

GDC 11.1 still uses the old C++ version of the D frontend, which feature-wise is mostly (see below) at D 2.076.1. There were significant issues in upstream DMD that prevented Iain from making the switch to the D version of the frontend in time to make the release window. He is currently aiming to make the switch in time for GDC 12. As a consolation, this release has support for three BSDs, Mac OS X, and MinGW!

Despite the older frontend, Iain has backported several fixes and optimizations, and even a few features, so it isn’t your grandfather’s D 2.076.1 that GDC supports. For example, the new bottom type that recently made its way through the D Improvement Proposal review process has found its way into this GDC release. See the forum announcement for details of all the new D goodness in GDC 11.1 and Please consider sponsoring his work on GitHub.

One-off donations

If you aren’t up for sponsoring Martin or Iain but would still like to support them financially, you can make one-time donations through the D Language Foundation. You can send money to the D General Fund, the D Open Collective, or to our PayPal account. Whichever method you choose, please be sure to leave a note that the donation is intended for LDC, GDC, or any D project you would like to support. We’ll make sure the appropriate person receives the money.

Other options for supporting the D programming language: visit the D Language Foundation donation page and donate to one of our funds, head to the DLang Swag Emporium and purchase any items that catch your eye (the D Rocket stuff rocks, and DConf Online 2021 swag will be available shortly), or consider using smile.amazon.com and selecting the D Language Foundation as your charity the next time you shop at Amazon.com (we are only available through the .com domain; browser extensions like SmartAmazonSmile for Firefox and AmazonSmileRedirect for Chrome make it easy to do).

Thanks to everyone who has, will, or continues to support the D programming language, either through donations of time or money. We’ve gotten where we are through community effort, and community effort will keep pushing us forward. D rocks!

Symmetry Investments and the D Language Foundation are Hiring

Digital Mars D logo

The D Language Foundation is hiring! Thanks to generous funding from Symmetry Investments, we are looking to fill two (mostly) non-programming positions geared toward improving the D ecosystem. Symmetry is also offering a bounty for a specific improvement to DUB, the D build tool and package manager. And on top of all of that, they are hiring D programmers.

D Pull Request/Issue Manager

A lot of good work goes into the D Programming Language GitHub repositories. Unfortunately, some of that good work sometimes gets left behind. A similar story can be told for our Bugzilla database, where some issues are fixed almost as soon as they’re reported and others fall victim to a lack of attention. Efforts have been made in the past to tidy things up, but without someone in a position to permanently keep at it, it’s a task that is never complete.

The D Language Foundation is looking for one or two motivated individuals to take on that permanent position, get the work done, and keep things running smoothly. Symmetry Investements is generously funding this role with $50,000 per year for one person, or $25,000 per year for each of two.

The ideal candidate is someone who:

  • is familiar with git, GitHub, and Bugzilla;
  • is familiar enough with D to be able to review simple pull requests;
  • is able to recognize when more specialized reviews are required and
  • is able to proofread English text (for reviewing documentation and web site pull requests).

Examples of the role’s responsibilities include:

  • ensuring all pull requests follow procedure;
  • reviewing simple pull requests;
  • finding appropriate reviewers for more complex pull requests;
  • ensuring that pull requests are reviewed in a timely manner;
  • reviving stale pull requests;
  • coordinating between pull request submitters and reviewers to prevent pull requests from going stale;
  • closing pull requests that are no longer valid;
  • identifying Bugzilla issues that are duplicates or invalid;
  • identifying Bugzilla issues that are candidates for bounties;
  • publicizing Bugzilla issues in need of a champion and
  • other related tasks.

We are hoping to hire from within the D community, though we will accept queries from anyone. If you are interested in taking on the role, please send your resume to social@dlang.org. You should also indicate if you are willing to do the job full time (just you) or part time (share the responsibilities with someone else).

Community Relations Assistant

I’ve been working with the D Language Foundation for the past three years. Much of what I do falls loosely in the category of Community Relations. These days, I’m in need of an assistant. Symmetry Investments is providing $600 per month for the role.

The job will involve a number of different activities as the need arises, such as:

  • seeking out guest authors and projects to highlight for the D Blog;
  • monitoring our social media accounts;
  • sending out messages from the D Language Foundation (such as thank you notes to new donors);
  • assisting with maintenance of pages at dlang.org and dconf.org;
  • assisting with the organization of events like DConf and SAOC and
  • any odd jobs that pop up now and again.

If you have good communication skills, an optimistic disposition, and enthusiasm for the D Programming Language, I’d like to talk to you. I don’t need a resume. Instead, please send an email to social@dlang.org explaining why you’re the right person for the job.

DUB Bounty

Symmetry Investments logoDUB has become a critical component in the D ecosystem. A significant number of projects depend on it and we need it to be able to meet a wide range of project needs. To that end, there are certainly improvements to be made. One such is in how DUB determines which of a project’s source files are in need of recompilation. Currently, DUB follows in the tradition of the venerable make and uses timestamp comparisons to make that determination.

A new generation of version control and build tools (git, buck, bazel, scons, waf, plz, and more) rely on file checksums to assess the need for action. This is a much more robust approach because it detects actual changes in file content. Timestamps can change in any number of irrelevant ways. Robustness is important if one is to depend on a build working properly even when files are moved, copied, and shared across people, machines, and teams. As hashes are fast to compute on modern hardware, the impact on speed is very low.

Symmetry Investments is offering a $2,000 bounty to the programmer who either converts DUB’s use of timestamp-dependent builds to use SHA-1 hashing throughout, or implements it as a global option to preserve the current behavior.

For inspiration, see this clip from Linus Torvald’s Google talk, and the article Build-Systems Should Use Hashes Over Timestamps. Note that shasum $(git ls-files) in Phobos takes 0.05 seconds on a warm SSD drive in a desktop machine.

Anyone interested in taking on this bounty should contact social@dlang.org beforehand. Anyone interested in contributing to the bounty amount can do so via the bounty card Support for Hash-Based Recompilation in DUB at our Task Bounties page.

SAOC 2020 and Other News

Symmetry Autumn of Code 2020

Symmetry Investments logo

The 3rd annual Symmetry Autumn of Code (SAoC) is on!

From now until August 16th, we’re accepting applications from motivated coders interested in getting paid to improve the D ecosystem. The SAoC committee will review all submissions and, based on the quality of the applications received, select a number of applicants to complete four milestones from September 15th to January 15th. Each participant will receive $1000 for the successful completion of each of the first three milestones, and one of them will receive an additional $1000 and a free trip (reimbursement for transportation and accommodation, and free registration) to the next real-world DConf (given the ongoing pandemic, we can’t yet be sure when that will be).

Anyone interested in programming D is welcome to apply, but preference will be given to those who can provide proof of enrollment in undergraduate or postgraduate university programs. For details on how to apply, see the SAoC 2020 page here at the D Blog.

The participants will need mentors, so we invite experienced D programmers interested in lending a hand to get in touch and to keep an eye out in the forums for any SAoC applicants in search of a mentor. As with the previous edition of SAoC, all mentors whose mentee completes the event will be guaranteed a one-time payment of $500 after the final milestone (mentors of unsuccessful mentees may still be eligible for the payment at the discretion of the SAoC committee). Potential mentors can follow the same link for details on their responsibilities and how to make themselves available.

We’re also looking for community input on potential SAoC projects. If there’s any work you’re aware of that needs doing in the D ecosystem and which may keep a lone coder occupied for 20 hours per week over four months, please let us know! Once again, details on how submit your suggestions and what sort of information we’re looking for can be found on the SAoC 2020 page.

Our SAoC 2019 selectee, Roberto Rosmaninho, was all set to attend DConf 2020 and we were all looking forward to meeting him. He’ll still be eligible to claim his free DConf trip at the next available opportunity.

SAoC would not be possible without the generosity of Symmetry Investments. A big thanks to them for once again funding this event and for the other ways, both financial and otherwise, they contribute back to the D programming language community.

Finances

Thanks to everyone who has shopped in the DLang Swag Emporium! To date, the D Language Foundation has received over $177 in royalties and referral fees. Thanks are also in order to those who have supported the foundation through smile.amazon.com. Your purchases have brought over $288 into the General Fund. Amazon Smile is perhaps the easiest way to support D financially if you shop through Amazon’s .com domain (the D Language Foundation is unavailable in other Amazon domains). If you’ve never done so, you can select a charitable foundation (the D Language Foundation, of course) on your first visit to smile.amazon.com. Then, every time you shop through that link, the foundation will receive a small percentage of your total purchase. Check your browser’s extension market for plugins that convert every amazon.com link to a smile.amazon.com link!

On the Task Bounties front, we may have closed out a big bounty for bringing D to iOS and iPadOS, but there are still several other bounties waiting to be claimed. The latest, currently at $220, is a bounty to improve DLL support on Windows by closing two related Bugzilla issues; 50% of the total bounty will be paid for the successful closure (merged PR and DMD release) of each issue. We welcome anyone interested in fixing these issues to either up the bounty or roll up their sleeves and start working toward claiming it. If you’d like to contribute to multiple bounties with a single credit card payment, or seed one or more new bounties with a specific amount, visit the Task Bounty Catch-All and follow the instructions there.

Finally, the question was recently raised in the forums about how to view the D Language Foundation’s finances. Because the foundation is a 501(3)(c) non-profit public charity, the Form 990 that the organization is required to submit to the IRS every year is publicly available. There are different ways you can obtain the documents for multiple years, such as searching online databases or contacting the IRS directly. Several websites, such as grantspace.org, provide details on how to do so. The Form 990 does not break down specific expenditures or sources of income except for special circumstances (like scholarship payments). With Andrei’s help, I’m currently working on gathering up more information on the past five years of the foundation’s finances so that we can put up an overview page at dlang.org. It won’t be at line-item detail, but we hope to provide a little more detail than the Form 990. I can’t provide a timeline on when it will be available (I don’t consider it a high priority task, so I’m working on it sporadically), but expect it sometime in the next few months.

DConf Online?

Rumor has it that online conferences are actually a thing. Voices in the wind speak of the potential for an annual event related to D. I don’t usually listen to voices I hear in the wind, but this time I’m intrigued…

D 2.091.0 Released

Digital Mars D logoThe latest release of DMD, the D reference compiler, ships with 18 major changes and 66 bugfixes from 55 contributors. This release contains, among other goodies, improvements to the Windows experience and enhancements to C and C++ interoperability. As fate would have it, the initial release announcement came in the aftermath of some unfortunate news regarding DConf 2020.

DMD on Windows

Over the years, some D users have remarked that the development of D is Linux-centric, that Windows is the black sheep or red-headed stepchild of D platforms. For anyone familiar with D’s early history, that seems an odd thing to say, given that DMD started out as a Windows-only compiler that could only output 32-bit objects in the OMF format. But it’s also understandable, as anyone not familiar with that history could only see that DMD on Windows lagged behind the Linux releases.

64-bit

One place where the official DMD releases on Windows have continued to differ from the releases on other platforms is the lack of 64-bit binaries in the release packages. Again, there’s a historical reason for this. The default output of the compiler is determined by how it is compiled, e.g., 32-bit versions output 32-bit binaries by default. When Walter first added support to DMD for 64-bit output on Windows, it required giving the back end the ability to generate object files in Microsoft’s version of the COFF format and also requiring users to install the Microsoft Build Tools and Platform SDK for access to the MS linker and system link libraries. This is quite a different experience from other platforms, where you can generally expect a common set of build tools to have been installed via the system package manager on any system set up for C and C++ development.

For a Windows developer who chooses GCC for their C and C++ development (or who does no C or C++ development at all), it’s a big ask to require them to download and install several GBs they might not already have installed and probably will never use for anything else. So D releases on Windows continued to ship with 32-bit binaries and the OPTLINK linker in order to provide a minimum out-of-the-box experience. That was a perfectly fine solution, unless you happened to be someone who really wanted 64-bit output (posts from disgruntled Windows users who didn’t want to install the MS tools can be found sprinkled throughout the forum archives).

Eventually, the LLVM linker (LLD) was added to the DMD Windows release packages, along with system link libraries generated from the MinGW definitions. This allowed users to compile 64-bit output out of the box and, once the kinks were worked out, eliminated the dependency on the MS linker. Yet, the official release packages still did not include a 64-bit version of DMD and still did not support 64-bit output by default.

With DMD 2.091.0, the black sheep has come back into the fold. The official DMD releases on Windows now ship with 64-bit binaries, so those of you masochists out there who cling to Makefiles and custom build scripts can expect the default output be what you expect it to be (for the record, DUB, the build tool and package manager that ships with DMD, has been instructing the compiler to compile 64-bit output by default on 64-bit systems for the past few releases).

Windows gets even more love

There are lots of goodies for Windows in this release. Another biggie is that DMD is now 30-40% faster on Windows. It’s no secret that LDC, the LLVM-based D compiler, generates faster binaries than DMD (for some D users, the general rule of thumb is to develop with DMD for its fast compile times and release with LDC for its faster binaries, though others argue that LDC is plenty fast for development and DMD is fine for production). There have been requests for some time to stop compiling DMD with DMD and start doing it with LDC instead. This release is the first to put that into practice.

There are a number of smaller enhancements to the Windows experience: the install.sh script available on the DMD downloads page that some people prefer now supports POSIX environments on Windows; the system link libraries that ship with the compiler have been upgraded from MinGW  5.0.2 to 7.0.0; LLD has been upgraded to 9.0.0; and there’s plenty more in the changelog.

C++ Header Generation

With just about every major release of DMD, D’s interoperability with C and C++ sees some kind of improvement. This release brings a huge one.

Over the years, some have speculated that it would be excellent if the D compiler could generate headers for C and C++ for D libraries intended to be usable in C or C++ programs. Now that wishful thinking has become a(n experimental) reality. Given a set of extern(C) or extern(C++) functions, DMD can generate header files that contain the appropriate C or C++ declarations. Three compiler switches get the job done:

  • -HC will cause the header to be generated and printed to standard output
  • -HCf=fileName will cause the header to be generated and printed to the specified file
  • -HCd=directoryname will (once it’s implemented) cause the header to be printed to a file in the specified directory

See the changelog for example output.

Other News

While the Corona virus was initially ramping up out of sight from most of the world, plans for DConf 2020 were ramping up online from different locations around the world. Planning began in November, the venue was secured in late December, and the website launched with the announcement in early January.

As news of the virus outbreak spread, the conference organizers grew concerned. Would we be okay in June? In late February, that concern manifested as a discussion of possible contingency plans. Two weeks later, it resulted in the decision to cancel DConf 2020. Thankfully, the D community has been supportive of the decision.

As part of the discussion of contingency plans, the possibility was raised of hosting an online conference. The idea of course came up in the discussion of the cancellation in the forums, and a few people reached out shortly after the initial announcement offering to provide help in setting something up. Walter created a forum thread to discuss the topic for anyone interested.

No one involved with organizing DConf has any experience with hosting an online conference. We’re currently exploring options and looking at what the organizers of other Conferences in the Time of COVID-19 are doing. We want to do it, and we want to do it well. Experience with organizing DConf in the real world has taught us not to jump on any old technology without first having a fallback (ahem, DConf 2018 livestream) and making sure the tech does what we expect it to (ahem, DConf 2019 livestream). So don’t expect a quick announcement. We want to find the right tech that fits our requirements and explore how it works before we move forward with setting dates. But do expect that DConf 2020 Online is looking more and more likely to become a thing.

News Update: Swag, Platforms, Documentation Help and More

Here are a few updates on things that have been going on both in front of and behind the scenes of the D Programming Language community.

New D Swag

We’ve got some new items in the DLang Swag Emporium: t-shirts, coffee mugs, and stickers sporting the Royal D logo. (If all Royal D items aren’t showing up for you in the Royal D category, check the D Rocket category. Everything should be in the correct location in a day or two).

You may notice that there are fewer options on the product page than for the other items, i.e. only one mug and sticker, and no dark tee option. They are available, though! When you select one of the existing products, you can change the style of the selection to one of several options. Beware! This may also change the price.

Remember, a small percentage of every item you order from the DLang Swag Emporium goes into the D Language Foundation’s General Fund. Plus, if you click through the link above or on the blog’s sidebar, we’ll get an additional referral fee on top of the item royalty. It’s an easy way to both get some D swag and contribute a few bucks to the Foundation.

Expanded Platform Progress

You maybe aware that some work has been ongoing in getting D onto more platforms. Adam Ruppe was working on contract to get LDC’s Android support to the finish line. He wrapped things up a few weeks back and has been paid out of the Foundation’s HR Fund.

Sebastiaan Koppe has been working on contract to get DRuntime ported to WebAssembly. Progress is ongoing and we currently expect it to be mostly wrapped up by the end of March. Like Adam, he’ll be out of the HR Fund when the contract is complete.

Work is also underway to bring LDC to iOS and iPadOS. We had been hoping to get someone to work on contract for this, but there are few people we know who are familiar enough with the platform to get it done and we were unable to find anyone then with the time to work on it. So we put up a bounty for it and kept our fingers crossed.

Recently, you may have seen forum posts from Jacob Carlborg indicating he’s been working on it in his spare time. Some preliminary support was merged in the LDC 1.20.0 release. Although he isn’t working under contract, he is working toward the bounty. That means anyone who wants to support him can contribute by increasing the bounty. Two contributors have already done so. The base amount of $3000 will be taken from the HR Fund when the work is complete.

And speaking of bounties, there are several others waiting for someone to claim them!

The HR Fund

With one payout from the fund and two coming up, we need to replenish it so we can always have cash earmarked for more contract work and bounties. You can make one-time or recurring donations of any amount directly and receive the same rewards available on our Open Collective page, or you can use a different link to make a $60 donation and get a DConf 2019 t-shirt in return. We’ve still got a few shirts available, so help us get rid of them and boost the HR Fund at the same time!

Documentation Event

Behind-the-scenes discussions about ideas to improve the D ecosystem in one way or another are frequently cycling through the inboxes of the people who can make them happen. Most never see the light of day, but there is one that has great potential. If it all comes together, I’ll be able to announce it in the coming weeks. We need your help to make that happen.

We need some specifics regarding areas where the documentation for D and items in the the D ecosystem is lacking. For example, people often complain about inconsistencies in the D spec, and missing info or examples in the DUB and vibe.d docs.

I’ve started a thread in the D forums where you can post your gripes about incomplete/missing/lackluster documentation. Remember, we need you to be specific. Just saying “the DUB docs are incomplete” doesn’t help. What specifically is missing? Or what specifically is wrong? The more information you can provide the better. And the more examples we can collect the better. The goal is to be able to define specific documentation tasks that anyone with the requisite knowledge can complete.

If we can get enough examples with enough detail, then I should be able to announce a new event sponsored by one of our generous benefactors. And I really want to be able to announce it!

DConf 2020

We really want to see a flood of talk submissions this year. If you’ve never been to DConf, or never presented at any conference, don’t let that stop you! Send us your submission and you may end up with a free trip to the conference.

Also, if you pay for an early-bird registration now (a 15% discount over the regular registration rate) and your talk is selected later, we’ll reimburse your registration fee. So if you’re planning to attend the conference even if your talk isn’t selected, it’s a good idea to register now and avoid the risk of missing the early-bird deadline.

We’re also offering once again the Open Source and Academic Discount; if you are a major open source contributor, a student, or an academic, we’ll give you a 50% discount on the regular registration rate. If you think you qualify, please don’t hesitate to take advantage of it by contacting social@dlang.org (or you can contact me directly at aldacron@gmail.com) for details on how to take advantage.

Finally, we never want to leave anyone out of DConf because they can’t afford to pay. This has been a policy of Walter’s from the beginning. If you are in or around London June 17 – 20 and would like to attend DConf but are unable to afford the registration and/or don’t qualify for the special discount, please email one of the addresses above and we’ll work something out.

DIP Reviews: Discussion vs. Feedback

Digital Mars D logoFor a while now, I’ve been including a link to the DIP Reviewer Guidelines in the initial forum post for every DIP review. My hope was that it would encourage reviewers to keep the thread on topic and also to provide more focused feedback. As it turns out, a link to reviewer guidelines is not quite enough. Recent review threads have turned into massive, 20+ page discussions touching on a number of tangential topics.

The primary purpose of the DIP review process, as I’ve tried to make clear in blog posts, forum discussions, and the reviewer guidelines, is to improve the DIP. It is not a referendum on the DIP. In every review round, the goal is to strengthen the content where it is lacking, bring clarity and precision to the language, make sure all the bases are covered, etc.

At the same time, we don’t want to discourage discussion on the merits of the proposal. Opinions about the necessity or the validity of a DIP can raise points that the language maintainers can take into consideration when they are deciding whether to approve or reject it, or even cause the DIP author to withdraw the proposal. It’s happened before. That’s why such discussion is encouraged in the Community Review rounds (though it’s generally discouraged in Final Review, which should be focused wholly on improving the proposal).

The problem

One issue with allowing such free-form discussion in the review threads is that there is a tremendous amount of noise drowning out the signal. Finding specific DIP-related feedback requires trawling through every post, digging through multiple paragraphs of mixed discussion and feedback. Sometimes, one or more people will level a criticism that spawns a long discussion and results in a changing of minds. This makes it time consuming for me as the DIP manager when I have to summarize the review. It also increases the likelihood that I’ll overlook something.

My summary isn’t just for the ‘Reviews’ section at the bottom of the DIP. It’s also my way of ensuring that the DIP author is aware of and has considered all the unique points of feedback. More than once I have found something the DIP author missed or had forgotten about. But if I overlook something and the DIP author also overlooks it, then we may have missed an opportunity to improve the DIP.

I have threatened to delete posts that go off topic in these threads,  but I can count on one hand the number of posts I’ve actually deleted. In reality, these discussions branch off in so many directions that it’s not easy to say definitively that a post that isn’t focused on the DIP itself is actually off topic. So I tend to let the posts stand rather than risk derailing the thread or removing information that is actually relevant.

The Solution

Starting with the upcoming Final Review of DIP 1027, I’m going to take a new approach to soliciting feedback. Rather than one review thread, I’ll be launching two for each DIP.

The Discussion Thread will be much the same as the current review thread. Opinions and discussion will be welcome and encouraged. I’ll still delete posts that are completely off topic, but other than that I’ll let the discussion flow where it may.

The Feedback Thread will be exclusively for feedback on the document and its contents. There will be no discussion allowed. Every post must contain specific points of feedback (preferably actionable items) intended to improve the proposal. Each post should be a direct reply to my initial post. There are only two exceptions: when a post author who has decided to retract feedback they made in a previous post, said poster can reply to the post in which they made the original feedback in order to make the retraction; and the DIP author may reply directly to any feedback post in order to indicate agreement or disagreement.

Posts in the feedback thread should contain answers to the questions posed in the DIP Reviewer Guidelines. It would be great if reviewers could take the time to do what Joseph Rushton Wakeling did in the Community Review for DIP 1028, where he explicitly listed and answered each question, but we won’t be requiring it. Feedback as bullet points is also very welcome.

Opinions on the validity of the proposed feature will be allowed in the feedback thread as long as they are backed with supporting arguments. In other words, “I’m against this! This is a terrible feature.” is not valid for the feedback thread. That sort of post goes in the discussion thread. However, “I’m against this. This is a terrible feature because <reasoned argument goes here>” is acceptable.

The rules of the feedback thread will be enforced without prejudice. Any post that is not a reply to my initial post, retraction of previous feedback, or a response by the DIP author will be deleted. Any post that does not provide the sort of feedback described above will be deleted. If I do delete a post, I won’t leave a new post explaining why. I’m going to update the DIP Reviewer Guidelines and each opening post in a feedback thread will include a link to that document as well as a paragraph or two summarizing the rules.

I’ll require DIP authors to follow both threads and to participate in the discussion thread. When it comes time to summarize the review, the feedback thread will be my primary source. I will, of course, follow the discussion thread as well and take notes on anything relevant. But if you want to ensure any specific criticisms you may have about a DIP are accounted for, be sure to post them in the feedback thread.

Hopefully, this new approach won’t be too disruptive. We’ll see how it goes.

 

Recent D Compiler Releases

Digital Mars D logoThe LDC team closed out the old year with release 1.19.0 of the LLVM-based D compiler, and the core D team opened the new year with version 2.090.0 of the reference D compiler, DMD. And if you haven’t yet heard, there was some big news about the GCC-based D compiler, GDC, a while back. Time to catch up!

LDC 1.19.0

This release updates the LDC compiler to D front end version 2.089.1, which was the current version when the compiler was released on the day after Christmas. The prebuilt packages are based on LLVM 9.01.

Among the big items in this release is some love for Android. The prebuilt DRuntime/Phobos library is now available for all supported Android targets. This release can be used in conjunction with Adam Ruppe’s D Android project, a collection of helper programs and interfaces, currently in beta, to facilitate D development on Android with LDC.

Windows users will find that the bundled MinGW-based link libraries for Windows development have been upgraded. They are now derived from .def files from the MinGW-w64 7.0.0 package. These libraries allow you to use the Windows system libraries without needing to install the Windows SDK.

DMD 2.090.0

The latest version of DMD was announced on January 7th. It ships with 10 major changes and 71 closed issues courtesy of 48 contributors.

With this release, it’s now possible to do more with lazy parameters. D has long supported lazy parameters:

An argument to a lazy parameter is not evaluated before the function is called. The argument is only evaluated if/when the parameter is evaluated within the function. Hence, a lazy argument can be executed 0 or more times.

Under the hood, they are implemented as delegates. Now, it’s possible to get at the underlying delegate by taking the address of the parameter, an operation which was previously illegal.

import std.stdio;

void chillax(lazy int x)
{
    auto dg = &x;
    assert(dg() == 10);
    writeln(x);
}

void main()
{
    chillax(2 * 5);
}

This release also renders obsolete a D idiom used by those who find themselves with a need to distinguish between finalization (non-deterministic object destruction usually initiated by the garbage collector) and normal destruction (deterministic object destruction) from inside a class or struct destructor.

With the current GC implementation, it’s illegal to perform some GC operations during finalization. However, D does not provide for separate finalizers and destructors. There is only ~this, which is referred to as a destructor even though it fills both roles. This sometimes presents difficulties when implementing destructors for types that are intended to be used with both GC and non-GC allocation. Any cleanup activity that touches the GC could throw an InvalidMemoryOperationError. Hence the need for the aforementioned workaround.

Now it’s possible to call the static GC member function, core.memory.GC.inFinalizer, to get your bearings in a destructor. It returns true if the current thread is performing object finalization, in which case you don’t want to be taking any actions that touch on GC operations. (I’ve been waiting for something like this before writing the next article in my GC series.)

GDC

Thanks to the hard work of Iain Buclaw, Johannes Pfau, and all of the volunteers who have maintained and contributed to it over the years, GDC was accepted into GCC 9 in late 2018 and made available as part of the GCC 9.1 package released in May of last year. GCC 9.2 was released last August. This version of GDC implements version 2.076 of the D front end. You can build it yourself or install it from the same place you install the GCC 9.x series.

DConf 2020: Double Decker Edition

To kick off the year of double 20’s (or double X’s if you prefer), the D Language Foundation is excited to announce that DConf 2020 will return to 99 City Road for a second round in London! We had such a great time last year that we were over the moon when we heard that our DConf 2019 hosts and sponsors at Symmetry Investments were willing to do it all again in 2020. The venue’s Sinisa Poznanovic will be back live streaming the talks on the D Language Foundation’s YouTube channel, and all of the talks will once again be recorded in HD via the capable hands of the Stage Engage crew.

Since DConf 2013, our annual D gathering has taken place each year in May. This time, we’re breaking tradition by running the conference in the middle of June. Our usual three days of talks will take place June 17th – 19th, followed by our annual DConf Hackathon on the 20th. There’s a reason we picked these dates, but it’s tied to an announcement I hope to make some time in the next few weeks. My fingers are crossed that things work out the way we intend and that I can make that announcement sooner rather than later.

Early-bird registration will open in the near future. From now, we’re accepting submissions. If you’ve got an idea for a talk or a panel, don’t be shy. It doesn’t matter if you’re a D veteran or a D noob, if you’ve spoken at multiple DConfs or spoken at none (or never spoken in public at all). We’re eager to see submissions from any and all, but we’re particularly interested in seeing some new faces at the lectern this year. If your talk gets selected, you can plan your trip to London and, as a speaker, you’ll be eligible for reimbursement for the cost of your transportation and lodgings. If your talk doesn’t make the cut, you lose nothing. So head to the DConf 2020 web site for the details and send us your submission!

Additionally, we’re currently working out the details of a potential event peripheral to the conference itself. If all goes well and the plans come to fruition, I’ll announce it here as soon as I’m able (otherwise, I’ll have just teased you for no reason whatsoever). We’ve also got an eye out for opportunities like the walking tours we organized before the conference last year. I don’t know which ideas or opportunities will materialize this year, but I do want you to know that we’re looking.

So start making your plans, send your thanks to Laeeth and Symmetry Investments (if you’re so inclined) for taking on a second DConf and for everything they’ve done and continue to do for the D community, and send us your submissions!