Author Archives: Michael Parker

Deadlines and New Swag

SAOC 2020 Application Deadline

Symmetry Investments logoThe deadline for Symmetry Autumn of Code (SAOC) 2020 applications is on August 16th. There’s work to be done and money to be paid (courtesy of Symmetry Investments). If you know of a project that can keep an eager programmer busy for at least 20 hours a week over the course of four months, please advertise it in the forums and add it to the project ideas list if it isn’t already there.

As for potential applicants, remember that experience with D is not necessary. Experience with another language can be transferred to D “on the job”, with a mentor to guide the way. Tell your friends and spread the word to other programming communities. This is a great way to bring new faces to the D community and the new ideas they may bring with them. All the information on how to apply and how to become a mentor is available on the SAOC 2020 page.

DConf Online 2020 Swag

DConf Online 2020 Logo

The DConf Online 2020 submission deadline of August 31 will be here before we know it. If you haven’t put a submission together yet, head over to the DConf Online 2020 home page (and/or the announcement here on the blog) for the details on what we’re looking for and how to go about it. Everyone whose submission is accepted for the event schedule will receive a t-shirt and coffee mug to commemorate the occasion.

For everyone else, those t-shirts and mugs are on sale now at the DLang Swag Emporium along with tote bags and stickers. Remember, all of the money we raise through the store goes straight into the General Fund, which we’ll dip into to provide DConf Online 2020 speakers with their free stuff and a few lucky viewers with prizes. Donations made directly to the General Fund, or to any of our ongoing campaigns, are also greatly appreciated.

The ABC’s of Templates in D

D was not supposed to have templates.

Several months before Walter Bright released the first alpha version of the DMD compiler in December 2001, he completed a draft language specification in which he wrote:

Templates. A great idea in theory, but in practice leads to numbingly complex code to implement trivial things like a “next” pointer in a singly linked list.

The (freely available) HOPL IV paper, Origins of the D Programming Language, expands on this:

[Walter] initially objected to templates on the grounds that they added disproportionate complexity to the front end, they could lead to overly complex user code, and, as he had heard many C++ users complain, they had a syntax that was difficult to understand. He would later be convinced to change his mind.

It did take some convincing. As activity grew in the (then singular) D newsgroup, requests that templates be added to the language became more frequent. Eventually, Walter started mulling over how to approach templates in a way that was less confusing for both the programmer and the compiler. Then he had an epiphany: the difference between template parameters and function parameters is one of compile time vs. run time.

From this perspective, there’s no need to introduce a special template syntax (like the C++ style <T>) when there’s already a syntax for parameter lists in the form of (T). So a template declaration in D could look like this:

template foo(T, U) {
    // template members here
}

From there, the basic features fell into place and were expanded and enhanced over time.

In this article, we’re going to lay the foundation for future articles by introducing the most basic concepts and terminology of D’s template implementation. If you’ve never used templates before in any language, this may be confusing. That’s not unexpected. Even though many find D’s templates easier to understand than other implementations, the concept itself can still be confusing. You’ll find links at the end to some tutorial resources to help build a better understanding.

Template declarations

Inside a template declaration, one can nest any other valid D declaration:

template foo(T, U) {
    int x;
    T y;

    struct Bar {
        U thing;
    }

    void doSomething(T t, U u) {
        ...
    }
}

In the above example, the parameters T and U are template type parameters, meaning they are generic substitutes for concrete types, which might be built-in types like int, float, or any type the programmer might implement with a class or struct declaration. By declaring a template, it’s possible to, for example, write a single implementation of a function like doSomething that can accept multiple types for the same parameters. The compiler will generate as many copies of the function as it needs to accomodate the concrete types used in each unique template instantiation.

Other kinds of parameters are supported: value parameters, alias parameters, sequence (or variadic) parameters, and this parameters, all of which we’ll explore in future blog posts.

One name to rule them all

In practice, it’s not very common to implement templates with multiple members. By far, the most common form of template declaration is the single-member eponymous template. Consider the following:

template max(T) {
    T max(T a, T b) { ... }
}

An eponymous template can have multiple members that share the template name, but when there is only one, D provides us with an alternate template declaration syntax. In this example, we can opt for a normal function declaration that has the template parameter list in front of the function parameter list:

T max(T)(T a, T b) { ... }

The same holds for eponymous templates that declare an aggregate type:

// Instead of the longhand template declaration...
/* 
template MyStruct(T, U) {
    struct MyStruct { 
        T t;
        U u;
    }
}
*/

// ...just declare a struct with a type parameter list
struct MyStruct(T, U) {
    T t;
    U u;
}

Eponymous templates also offer a shortcut for instantiation, as we’ll see in the next section.

Instantiating templates

In relation to templates, the term instantiate means essentially the same as it does in relation to classes and structs: create an instance of something. A template instance is, essentially, a concrete implementation of the template in which the template parameters have been replaced by the arguments provided in the instantiation. For a template function, that means a new copy of the function is generated, just as if the programmer had written it. For a type declaration, a new copy of the declaration is generated, just as if the programmer had written it.

We’ll see an example, but first we need to see the syntax.

Explicit instantiations

An explicit instantiation is a template instance created by the programmer using the template instantiation syntax. To easily disambiguate template instantiations from function calls, D requires the template instantiation operator, !, to be present in explicit instantiations. If the template has multiple members, they can be accessed in the same manner that members of aggregates are accessed: using dot notation.

import std;

template Temp(T, U) {
    T x;
    struct Pair {
        T t;
        U u;
    }
}

void main()
{
    Temp!(int, float).x = 10;
    Temp!(int, float).Pair p;
    p.t = 4;
    p.u = 3.2;
    writeln(Temp!(int, float).x);
    writeln(p);            
}

Run it online at run.dlang.io

There is one template instantiation in this example: Temp!(int, float). Although it appears three times, it refers to the same instance of Temp, one in which T == int and U == float. The compiler will generate declarations of x and the Pair type as if the programmer had written the following:

int x;
struct Pair {
    int t;
    float u;
}

However, we can’t just refer to x and Pair by themselves. We might have other instantiations of the same template, like Temp!(double, long), or Temp(MyStruct, short). To avoid conflict, the template members must be accessed through a namespace unique to each instantiation. In that regard, Temp!(int, float) is like a class or struct with static members; just as you would access a static x member variable in a struct Foo using the struct name, Foo.x, you access a template member using the template name, Temp!(int, float).x.

There is only ever one instance of the variable x for the instantiation Temp!(int float), so no matter where you use it in a code base, you will always be reading and writing the same x. Hence, the first line of main isn’t declaring and initializing the variable x, but is assigning to the already declared variable. Temp!(int, float).Pair is a struct type, so that after the declaration Temp!(int, float).Pair p, we can refer to p by itself. Unlike x, p is not a member of the template. The type Pair is a member, so we can’t refer to it without the prefix.

Aliased instantiations

It’s possible to simplify the syntax by using an alias declaration for the instantiation:

import std;

template Temp(T, U) {
    T x;
    struct Pair {
        T t;
        U u;
    }
}
alias TempIF = Temp!(int, float);

void main()
{
    TempIF.x = 10;
    TempIF.Pair p = TempIF.Pair(4, 3.2);
    writeln(TempIF.x);
    writeln(p);            
}

Run it online at run.dlang.io

Since we no longer need to type the template argument list, using a struct literal to initialize p, in this case TempIF.Pair(3, 3.2), looks cleaner than it would with the template arguments. So I opted to use that here rather than first declare p and then initialize its members. We can trim it down still more using D’s auto attribute, but whether this is cleaner is a matter of preference:

auto p = TempIF.Pair(4, 3.2);

Run it online at run.dlang.io

Instantiating eponymous templates

Not only do eponymous templates have a shorthand declaration syntax, they also allow for a shorthand instantiation syntax. Let’s take the x out of Temp and rename the template to Pair. We’re left with a Pair template that provides a declaration struct Pair. Then we can take advantage of both the shorthand declaration and instantiation syntaxes:

import std;

struct Pair(T, U) {
    T t;
    U u;
}

// We can instantiate Pair without the dot operator, but still use
// the alias to avoid writing the argument list every time
alias PairIF = Pair!(int, float);

void main()
{
    PairIF p = PairIF(4, 3.2);
    writeln(p);            
}

Run it online at run.dlang.io

The shorthand instantiation syntax means we don’t have to use the dot operator to access the Pair type.

Even shorter syntax

When a template instantiation is passed only one argument, and the argument’s symbol is a single token (e.g., int as opposed to int[] or int*), the parentheses can be dropped from the template argument list. Take the standard library template function std.conv.to as an example:

void main() {
    import std.stdio : writeln;
    import std.conv : to;
    writeln(to!(int)("42"));
}

Run it online at run.dlang.io

std.conv.to is an eponymous template, so we can use the shortened instantiation syntax. But the fact that we’ve instantiated it as an argument to the writeln function means we’ve got three pairs of parentheses in close proximity. That sort of thing can impair readability if it pops up too often. We could move it out and store it in a variable if we really care about it, but since we’ve only got one template argument, this is a good place to drop the parentheses from the template argument list.

writeln(to!int("42"));

Whether that looks better is another case where it’s up to preference, but it’s fairly idiomatic these days to drop the parentheses for a single template argument no matter where the instantiation appears.

Not done with the short stuff yet

std.conv.to is an interesting example because it’s an eponymous template with multiple members that share the template name. That means that it must be declared using the longform syntax (as you can see in the source code), but we can still instantiate it without the dot notation. It’s also interesting because, even though it accepts two template arguments, it is generally only instantiated with one. This is because the second template argument can be deduced by the compiler based on the function argument.

For a somewhat simpler example, take a look at std.utf.toUTF8:

void main()
{    
    import std.stdio : writeln;
    import std.utf : toUTF8;
    string s1 = toUTF8("This was a UTF-16 string."w);
    string s2 = toUTF8("This was a UTF-32 string."d);
    writeln(s1);
    writeln(s2);
}

Run it online at run.dlang.io

Unlike std.conv.to, toUTF8 takes exactly one parameter. The signature of the declaration looks like this:

string toUTF8(S)(S s)

But in the example, we aren’t passing a type as a template argument. Just as the compiler was able to deduce to’s second argument, it’s able to deduce toUTF8’s sole argument.

toUTF8 is an eponymous function template with a template parameter S and a function parameter s of type S. There are two things we can say about this: 1) the return type is independent of the template parameter and 2) the template parameter is the type of the function parameter. Because of 2), the compiler has all the information it needs from the function call itself and has no need for the template argument in the instantiation.

Take the first call to the toUTF8 function in the declaration of s1. In long form, it would be toUTF8!(wstring)("blah"w). The w at the end of the string literal indicates it is of type wstring, with UTF-16 encoding, as opposed to string, with UTF-8 encoding (the default for string literals). In this situation, having to specify !(wstring) in the template instantiation is completely redundant. The compiler already knows that the argument to the function is a wstring. It doesn’t need the programmer to tell it that. So we can drop the template instantiation operator and the template argument list, leaving what looks like a simple function call. The compiler knows that toUTF8 is a template, knows that the template is declared with one type parameter, and knows that the type should be wstring.

Similarly, the d suffix on the literal in the initialization of s2 indicates a UTF-32 encoded dstring, and the compiler knows all it needs to know for that instantiation. So in this case also, we drop the template argument and make the instantiation appear as a function call.

It does seem silly to convert a wstring or dstring literal to a string when we could just drop the w and d prefixes and have string literals that we can directly assign to s1 and s2. Contrived examples and all that. But the syntax the examples are demonstrating really shines when we work with variables.

wstring ws = "This is a UTF-16 string"w;
string s = ws.toUTF8;
writeln(s);

Run it online at run.dlang.io

Take a close look at the initialization of s. This combines the shorthand template instantiation syntax with Uniform Function Call Syntax (UFCS) and D’s shorthand function call syntax. We’ve already seen the template syntax in this post. As for the other two:

  • UFCS allows using dot notation on the first argument of any function call so that it appears as if a member function is being called. By itself, it doesn’t offer much benefit aside from, some believe, readability. Generally, it’s a matter of preference. But this feature can seriously simplify the implementation of generic templates that operate on aggregate types and built-in types.
  • The parentheses in a function call can be dropped when the function takes no arguments, so that foo() becomes foo. In this case, the function takes one argument, but we’ve taken it out of the argument list using UFCS, so the argument list is now empty and the parentheses can be dropped. (The compiler will lower this to a normal function call, toUTF8(ws)—it’s purely programmer convenience.) When and whether to do this in the general case is a matter of preference. The big win, again, is in the simplification of template implementations: a template can be implemented to accept a type T with a member variable foo or a member function foo, or a free function foo for which the first argument is of type T.

All of this shorthand syntax is employed to great effect in D’s range API, which allows chained function calls on types that are completely hidden from the public-facing API (aka Voldemort types).

More to come

In future articles, we’ll explore the different kinds of template parameters, introduce template constraints, see inside a template instantiation, and take a look at some of the ways people combine templates with D’s powerful compile-time features in the real world. In the meantime, here are some template tutorial resources to keep you busy:

  • Ali Çehreli’s Programming in D is an excellent introduction to D in general, suitable even for those with little programming experience. The two chapters on templates (the first called ‘Templates’ and the second ‘More Templates’) provide a great introduction. (The online version of the book is free, but if you find it useful, please consider throwing Ali some love by buying the ebook or print version linked at the top of the TOC.)
  • More experienced programmers may find Phillipe Sigaud’s ‘D Template Tutorial’ a good read. It’s almost a decade old, but still relevant (and still free!). This tutorial goes beyond the basics into some of the more advanced template features. It includes a look at D’s compile-time features, provides a number of examples, and sports an appendix detailing the is expression (a key component of template constraints). It can also serve as a great reference when reading open source D code until you learn your way around D templates.

There are other resources, though other than my book ‘Learning D’ (this is a referral link that will benefit the D Language Foundation), I’m not aware of any that provide as much detail as the above. (And my book is currently not freely available). Eventually, we’ll be able to add this blog series to the list.

Thanks to Stefan Koch for reviewing this article.

DConf Online 2020: Call For Submissions

DConf Online 2020 LogoDConf Online 2020 is happening November 21 & 22, 2020 in your local web browser! We are currently taking submissions for pre-recorded talks, livstreamed panels, and livecoding events. See the DConf Online 2020 web site for details on how you can participate. Keep reading here for more info on how it came together and what we hope to achieve, as well as for a reminder about the 2020 edition of the Symmetry Autumn of Code (the SAOC 2020 registration deadline is just over three weeks away!).

Maybe Next Time, London!

Due to the onset of COVID-19, the D Language Foundation and Symmetry Investments decided in early March to cancel DConf 2020, which had been scheduled to take place June 17–20 in London. DConf has been the premiere D programming language event every year since 2013, the one chance for members of the D community from around the world to gather face-to-face outside of their local meetups for four days of knowledge sharing and comradery. It was a painful decision, but the right one. As of now, we can’t say for sure there will be a DConf 2021, but it’s looking increasingly unlikely.

Immediately upon reaching the decision to cancel DConf, the obvious question arose of whether we should take the conference online. It was something none of the DConf organizers had any experience with, so we were unwilling to commit to anything until we could figure out a way to go about it that makes sense for our community. As time progressed and we explored our options, the idea became more attractive. Finally, we settled on an approach that we think will work for our community while still allowing outsiders to easily drop by to get a look at our favorite programming language.

We also decided that this is not going to be an online substitute for the real-world DConf. That’s why we’ve named it DConf Online 2020 and not DConf 2020 Online. We’re planning to make this an annual event. The real-world DConf will still take place in spring or summer (barring pandemics or other unforeseen circumstances), and DConf Online six months or so later. Without the DConf cancellation, we never would have reached this point, so for us that’s a bit of a bright side to these dark days.

DConf on YouTube

DConf Online will take place on the D Language Foundation’s YouTube Channel. The event will kick off with a pre-recorded keynote from Walter Bright, the creator and co-maintainer of D, on November 21, scheduled to premiere at a yet-to-be-determined time. Other pre-recorded talks will be scheduled to premiere throughout the weekend, including a Day Two keynote on November 22 from co-maintainer Átila Neves. Presenters from the pre-recorded talks will be available for livestreamed question and answer sessions just as they would be in the real-world DConf.

We’ll also be livestreaming an Ask Us Anything session, a DConf tradition, with Walter and Átila. We’re looking for other ideas for livestream panels. Anyone submitting a panel proposal should either be willing to moderate the panel or have already found someone to commit to the position.

And we really, really want to have at least two livecoding sessions. Anyone familiar with D who has experience livecoding is welcome to submit a proposal. Ideally, we’re looking for sessions that present a solid demonstration of D in use, preferably a small project designed exclusively for the livestream, something that can be developed from start to finish in no more than 90 minutes. We aren’t looking for tutorial style sessions that go into great detail on a feature or two (though that sort of thing is great for a pre-recorded talk submission!), but something that shows how a D program comes together and what D features look like in action.

Everything you need to know to submit a pre-recorded talk, panel, or livecoding session to the D Language Foundation can be found at the DConf Online 2020 web site. We’ll have more details here and on the web site in the coming weeks as our plans solidify. Oh, and everyone whose submission is accepted will receive some swag from the DLang Swag Emporium (DConf Online 2020 swag is coming soon).

BeerConf

It’s a DConf tradition that a gathering spot is selected where attendees can get together each evening for drinks, food, and conversation. For many attendees, this is a highlight of the conference. The opportunity to engage in conversation with so many smart, like-minded people is not one to be missed. Ethan Watson dubbed these evening soirees “BeerConf”, and the name has stuck.

Recently, Ethan and other D community members have been gathering for a monthly online #BeerConf. Given that it’s such an integral part of the DConf experience, we hope to make use of the lessons they’re learning to run a BeerConf in parallel to DConf Online, starting on the 20th. Despite the name, no one will be expected to drink alcohol of any kind. It’s all about getting together to socialize as close to face-to-face as we can get online.

More details regarding BeerConf will be announced closer to the conference dates, so keep an eye on the blog!

SAOC 2020

Symmetry Autumn of Code is an annual event where a handful of lucky programmers get paid to write some D code. Sponsored by Symmetry Investments, SAOC 2020 is the third edition of the event. Although priority is given to university students, SAOC is open to anyone over 18.

Applicants send a project proposal and a short bio to the D Language Foundation. Those who are selected will be required to work on their project at least 20 hours per week from September 15, 2020, until January 15, 2021. The event consists of four milestones. Participants who meet their goals for the first three milestones will each receive a payment of $1000. For the fourth milestone, the SAOC Committee will evaluate each participant’s progress for the entire event. On that basis, one will be selected to receive a final $1000 payment and a free trip to the next real-world DConf (no registration fee; travel and lodging expenses reimbursed on the same terms as offered to DConf speakers). The lucky participant will be asked to submit a proposal for the same DConf they attend, but their proposal will be evaluated in the same manner as all proposals (i.e., acceptance is not guaranteed), but they are guaranteed free registration and reimbursement regardless.

Although Roberto Romaninho, our SAOC 2019 selectee, was robbed of the opportunity to attend DConf 2020, he will still be eligible to make use of his reward at our next real-world event along with the 2020 selectee. Francesco Gallà, who was selected in the inaugural SAOC 2018, gave a presentation about his project and the SAOC experience at DConf 2019. The runner up, Franceso Mecca, wrote about his own project for the D Blog.

SAOC 2020 applications are open until August 16. See the SAOC 2020 page for all the details on how to apply.

SAOC 2020 and Other News

Symmetry Autumn of Code 2020

Symmetry Investments logo

The 3rd annual Symmetry Autumn of Code (SAoC) is on!

From now until August 16th, we’re accepting applications from motivated coders interested in getting paid to improve the D ecosystem. The SAoC committee will review all submissions and, based on the quality of the applications received, select a number of applicants to complete four milestones from September 15th to January 15th. Each participant will receive $1000 for the successful completion of each of the first three milestones, and one of them will receive an additional $1000 and a free trip (reimbursement for transportation and accommodation, and free registration) to the next real-world DConf (given the ongoing pandemic, we can’t yet be sure when that will be).

Anyone interested in programming D is welcome to apply, but preference will be given to those who can provide proof of enrollment in undergraduate or postgraduate university programs. For details on how to apply, see the SAoC 2020 page here at the D Blog.

The participants will need mentors, so we invite experienced D programmers interested in lending a hand to get in touch and to keep an eye out in the forums for any SAoC applicants in search of a mentor. As with the previous edition of SAoC, all mentors whose mentee completes the event will be guaranteed a one-time payment of $500 after the final milestone (mentors of unsuccessful mentees may still be eligible for the payment at the discretion of the SAoC committee). Potential mentors can follow the same link for details on their responsibilities and how to make themselves available.

We’re also looking for community input on potential SAoC projects. If there’s any work you’re aware of that needs doing in the D ecosystem and which may keep a lone coder occupied for 20 hours per week over four months, please let us know! Once again, details on how submit your suggestions and what sort of information we’re looking for can be found on the SAoC 2020 page.

Our SAoC 2019 selectee, Roberto Rosmaninho, was all set to attend DConf 2020 and we were all looking forward to meeting him. He’ll still be eligible to claim his free DConf trip at the next available opportunity.

SAoC would not be possible without the generosity of Symmetry Investments. A big thanks to them for once again funding this event and for the other ways, both financial and otherwise, they contribute back to the D programming language community.

Finances

Thanks to everyone who has shopped in the DLang Swag Emporium! To date, the D Language Foundation has received over $177 in royalties and referral fees. Thanks are also in order to those who have supported the foundation through smile.amazon.com. Your purchases have brought over $288 into the General Fund. Amazon Smile is perhaps the easiest way to support D financially if you shop through Amazon’s .com domain (the D Language Foundation is unavailable in other Amazon domains). If you’ve never done so, you can select a charitable foundation (the D Language Foundation, of course) on your first visit to smile.amazon.com. Then, every time you shop through that link, the foundation will receive a small percentage of your total purchase. Check your browser’s extension market for plugins that convert every amazon.com link to a smile.amazon.com link!

On the Task Bounties front, we may have closed out a big bounty for bringing D to iOS and iPadOS, but there are still several other bounties waiting to be claimed. The latest, currently at $220, is a bounty to improve DLL support on Windows by closing two related Bugzilla issues; 50% of the total bounty will be paid for the successful closure (merged PR and DMD release) of each issue. We welcome anyone interested in fixing these issues to either up the bounty or roll up their sleeves and start working toward claiming it. If you’d like to contribute to multiple bounties with a single credit card payment, or seed one or more new bounties with a specific amount, visit the Task Bounty Catch-All and follow the instructions there.

Finally, the question was recently raised in the forums about how to view the D Language Foundation’s finances. Because the foundation is a 501(3)(c) non-profit public charity, the Form 990 that the organization is required to submit to the IRS every year is publicly available. There are different ways you can obtain the documents for multiple years, such as searching online databases or contacting the IRS directly. Several websites, such as grantspace.org, provide details on how to do so. The Form 990 does not break down specific expenditures or sources of income except for special circumstances (like scholarship payments). With Andrei’s help, I’m currently working on gathering up more information on the past five years of the foundation’s finances so that we can put up an overview page at dlang.org. It won’t be at line-item detail, but we hope to provide a little more detail than the Form 990. I can’t provide a timeline on when it will be available (I don’t consider it a high priority task, so I’m working on it sporadically), but expect it sometime in the next few months.

DConf Online?

Rumor has it that online conferences are actually a thing. Voices in the wind speak of the potential for an annual event related to D. I don’t usually listen to voices I hear in the wind, but this time I’m intrigued…

Interfacing D with C: Arrays and Functions (Arrays Part 2)

Digital Mars D logo

This post is part of an ongoing series on working with both D and C in the same project. The previous post explored the differences in array declaration and initialization. This post takes the next step: declaring and calling C functions that take arrays as parameters.

Arrays and C function declarations

Using C libraries in D is extremely easy. Most of the time, things work exactly as one would expect, but as we saw in the previous article there can be subtle differences. When working with C functions that expect arrays, it’s necessary to fully understand these differences.

The most straightforward and common way of declaring a C function that accepts an array as a parameter is to to use a pointer in the parameter list. For example, this hypothetical C function:

void f0(int *arr);

In C, any array of int can be passed to this function no matter how it was declared. Given int a[], int b[3], or int *c, the function calls f0(a), f0(b), and f0(c) are all the same: a pointer to the first element of each array is passed to the function. Or using the lingo of C programmers, arrays decay to pointers

Typically, in a function like f0, the implementer will expect the array to have been terminated with a marker appropriate to the context. For example, strings in C are arrays of char that are terminated with the \0 character (we’ll look at D strings vs. C strings in a future post). This is necessary because, without that character, the implementation of f0 has no way to know which element in the array is the last one. Sometimes, a function is simply documented to expect a certain length, either in comments or in the function name, e.g., a vector3f_add(float *vec) will expect that vec points to exactly 3 elements. Another option is to require the length of the array as a separate argument:

void f1(int *arr, size_t len);

None of these approaches is foolproof. If f0 receives an array with no end marker or which is shorter than documented, or if f1 receives an array with an actual length shorter than len, then the door is open for memory corruption. D arrays take this possibility into account, making it much easier to avoid such problems. But again, even D’s safety features aren’t 100% foolproof when calling C functions from D.

There are other, less common, ways array parameters may be declared in C:

void f2(int arr[]);
void f3(int arr[9]);
void f4(int arr[static 9]);

Although these parameters are declared using C’s array syntax, they boil down to the exact same function signature as f0 because of the aforementioned pointer decay. The [9] in f3 triggers no special enforcement by the compiler; arr is still effectively a pointer to int with unknown length. The [9] serves as documentation of what the function expects, and the implementation cannot rely on the array having nine elements.

The only potential difference is in f4. The static added to the declaration tells the compiler that the function must take an array of, in this case, at least nine elements. It could have more than nine, but it can’t have fewer. That also rules out null pointers. The problem is, this isn’t necessarily enforced. Depending on which C compiler you use, if you shortchange the function and send it less than nine elements you might see warnings if they are enabled, but the compiler might not complain at all. (I haven’t tested current compilers for this article to see if any are actually reporting errors for this, or which ones provide warnings.)

The behavior of C compilers doesn’t matter from the D side. All we need be concerned with is declaring these functions appropriately so that we can call them from D such that there are no crashes or unexpected results. Because they are all effectively the same, we could declare them all in D like so:

extern(C):
void f0(int* arr);
void f1(int* arr, size_t len);
void f2(int* arr);
void f3(int* arr);
void f4(int* arr);

But just because we can do a thing doesn’t mean we should. Consider these alternative declarations of f2, f3, and f4:

extern(C):
void f2(int[] arr);
void f3(int[9] arr);
void f4(int[9] arr);

Are there any consequences of taking this approach? The answer is yes, but that doesn’t mean we should default to int* in each case. To understand why, we need first to explore the innards of D arrays.

The anatomy of a D array

The previous article showed that D makes a distinction between dynamic and static arrays:

int[] a0;
int[9] a1;

a0 is a dynamic array and a1 is a static array. Both have the properties .ptr and .length. Both may be indexed using the same syntax. But there are some key differences between them.

Dynamic arrays

Dynamic arrays are usually allocated on the heap (though that isn’t a requirement). In the above case, no memory for a0 has been allocated. It would need to be initialized with memory allocated via new or malloc, or some other allocator, or with an array literal. Because a0 is uninitialized, a0.ptr is null and a0.length is 0.

A dynamic array in D is an aggregate type that contains the two properties as members. Something like this:

struct DynamicArray {
    size_t length;
    size_t ptr;
}

In other words, a dynamic array is essentially a reference type, with the pointer/length pair serving as a handle that refers to the elements in the memory address contained in the ptr member. Every built-in D type has a .sizeof property, so if we take a0.sizeof, we’ll find it to be 8 on 32-bit systems, where size_t is a 4-byte uint, and 16 on 64-bit systems, where size_t is an 8-byte ulong. In short, it’s the size of the handle and not the cumulative size of the array elements.

Static arrays

Static arrays are generally allocated on the stack. In the declaration of a1, stack space is allocated for nine int values, all of which are initialized to int.init (which is 0) by default. Because a1 is initialized, a1.ptr points to the allocated space and a1.length is 9. Although these two properties are the same as those of the dynamic array, the implementation details differ.

A static array is a value type, with the value being all of its elements. So given the declaration of a1 above, its nine int elements indicate that a1.sizeof is 9 * int.sizeof, or 36. The .length property is a compile-time constant that never changes, and the .ptr property, though not readable at compile time, is also a constant that never changes (it’s not even an lvalue, which means it’s impossible to make it point somewhere else).

These implementation details are why we must pay attention when we cut and paste C array declarations into D source modules.

Passing D arrays to C

Let’s go back to the declaration of f2 in C and give it an implementation:

void f2(int arr[]) {
    for(int i=0; i<3; ++i)
        printf("%d\n", arr[i]);
}

A naïve declaration in D:

extern(C) void f2(int[]);

void main() {
    int[] a = [10, 20, 30];
    f2(a);
}

I say naïve because this is never the right answer. Compiling f2.c with df2.d on Windows (cl /c f2.c in the “x64 Native Tools” command prompt for Visual Studio, followed by dmd -m64 df2.d f2.obj), then running df2.exe, shows me the following output:

3
0
1970470928

There is no compiler error because the declaration of f2 is pefectly valid D. The extern(C) indicates that this function uses the cdecl calling convention. Calling conventions affect the way arguments are passed to functions and how the function’s symbol is mangled. In this case, the symbol will be either _f2 or f2 (other calling conventions, like stdcallextern(Windows) in D—have different mangling schemes). The declaration still has to be valid D. (In fact, any D function can be marked as extern(C), something which is necessary when creating a D library that will be called from other languages.)

There is also no linker error. DMD is calling out to the system linker (in this case, Microsoft’s link.exe), the same linker used by the system’s C and C++ compilers. That means the linker has no special knowledge about D functions. All it knows is that there is a call to a symbol, f2 or _f2, that needs to be linked with the implementation. Since the type and number of parameters are not mangled into the symbol name, the linker will happily link with any matching symbol it finds (which, by the way, is the same thing it would do if a C program tried to call a C function which was declared with an incorrect parameter list).

The C function is expecting a single pointer as an argument, but it’s instead receiving two values: the array length followed by the array pointer.

The moral of this story is that any C function with array parameters declared using array syntax, like int[], should be declared to accept pointers in D. Change the D source to the following and recompile using the same command line as before (there’s no need to recompile the C file):

extern(C) void f2(int*);

void main() {
    int[] a = [10, 20, 30];
    f2(a.ptr);
}

Note the use of a.ptr. It’s an error to try to pass a D array argument where a pointer is expected (with one very special exception, string literals, which I’ll cover in the next article in this series), so the array’s .ptr property must be used instead.

The story for f3 and f4 is similar:

void f3(int arr[9]);
void f4(int arr[static 9]);

Remember, int[9] in D is a static array, not a dynamic array. The following do not match the C declarations:

void f3(int[9]);
void f4(int[9]);

Try it yourself. The C implementation:

void f3(int arr[9]) {
    for(int i=0; i<9; ++i)
        printf("%d\n", arr[i]);
}

And the D implementation:

extern(C) void f3(int[9]);

void main() {
    int[9] a = [10, 20, 30, 40, 50, 60, 70, 80, 90];
    f3(a);
}

This is likely to crash, depending on the system. Rather than passing a pointer to the array, this code is instead passing all nine array elements by value! Now consider a C library that does something like this:

typedef float[16] mat4f;
void do_stuff(mat4f mat);

Generally, when writing D bindings to C libraries, it’s a good idea to keep the same interface as the C library. But if the above is translated like the following in D:

alias mat4f = float[16];
extern(C) void do_stuff(mat4f);

The sixteen floats will be passed to do_stuff every time it’s called. The same for all functions that take a mat4f parameter. One solution is just to do the same as in the int[] case and declare the function to take a pointer. However, that’s no better than C, as it allows the function to be called with an array that has fewer elements than expected. We can’t do anything about that in the int[] case, but that will usually be accompanied by a length parameter on the C side anyway. C functions that take typedef’d types like mat4f usually don’t have a length parameter and rely on the caller to get it right.

In D, we can do better:

void do_stuff(ref mat4f);

Not only does this match the API implementor’s intent, the compiler will guarantee that any arrays passed to do_stuff are static float arrays with 16 elements. Since a ref parameter is just a pointer under the hood, all is as it should be on the C side.

With that, we can rewrite the f3 example:

extern(C) void f3(ref int[9]);

void main() {
    int[9] a = [10, 20, 30, 40, 50, 60, 70, 80, 90];
    f3(a);
}

Conclusion

Most of the time, when interfacing with C from D, the C API declarations and any example code can be copied verbatim in D. But most of the time is not all of the time, so care must be taken to account for those exceptional cases. As we saw in the previous article, carelessness when declaring array variables can usually be caught by the compiler. As this article shows, the same is not the case for C function declarations. Interfacing D with C requires the same care as when writing C code.

In the next article in this series, we’ll look at mixing D strings and C strings in the same program and some of the pitfalls that may arise. In the meantime, Steven Schveighoffer’s excellent article, “D Slices”, is a great place to start for more details about D arrays.

Thanks to Walter Bright and Átila Neves for their valuable feedback on this article.

D 2.091.0 Released

Digital Mars D logoThe latest release of DMD, the D reference compiler, ships with 18 major changes and 66 bugfixes from 55 contributors. This release contains, among other goodies, improvements to the Windows experience and enhancements to C and C++ interoperability. As fate would have it, the initial release announcement came in the aftermath of some unfortunate news regarding DConf 2020.

DMD on Windows

Over the years, some D users have remarked that the development of D is Linux-centric, that Windows is the black sheep or red-headed stepchild of D platforms. For anyone familiar with D’s early history, that seems an odd thing to say, given that DMD started out as a Windows-only compiler that could only output 32-bit objects in the OMF format. But it’s also understandable, as anyone not familiar with that history could only see that DMD on Windows lagged behind the Linux releases.

64-bit

One place where the official DMD releases on Windows have continued to differ from the releases on other platforms is the lack of 64-bit binaries in the release packages. Again, there’s a historical reason for this. The default output of the compiler is determined by how it is compiled, e.g., 32-bit versions output 32-bit binaries by default. When Walter first added support to DMD for 64-bit output on Windows, it required giving the back end the ability to generate object files in Microsoft’s version of the COFF format and also requiring users to install the Microsoft Build Tools and Platform SDK for access to the MS linker and system link libraries. This is quite a different experience from other platforms, where you can generally expect a common set of build tools to have been installed via the system package manager on any system set up for C and C++ development.

For a Windows developer who chooses GCC for their C and C++ development (or who does no C or C++ development at all), it’s a big ask to require them to download and install several GBs they might not already have installed and probably will never use for anything else. So D releases on Windows continued to ship with 32-bit binaries and the OPTLINK linker in order to provide a minimum out-of-the-box experience. That was a perfectly fine solution, unless you happened to be someone who really wanted 64-bit output (posts from disgruntled Windows users who didn’t want to install the MS tools can be found sprinkled throughout the forum archives).

Eventually, the LLVM linker (LLD) was added to the DMD Windows release packages, along with system link libraries generated from the MinGW definitions. This allowed users to compile 64-bit output out of the box and, once the kinks were worked out, eliminated the dependency on the MS linker. Yet, the official release packages still did not include a 64-bit version of DMD and still did not support 64-bit output by default.

With DMD 2.091.0, the black sheep has come back into the fold. The official DMD releases on Windows now ship with 64-bit binaries, so those of you masochists out there who cling to Makefiles and custom build scripts can expect the default output be what you expect it to be (for the record, DUB, the build tool and package manager that ships with DMD, has been instructing the compiler to compile 64-bit output by default on 64-bit systems for the past few releases).

Windows gets even more love

There are lots of goodies for Windows in this release. Another biggie is that DMD is now 30-40% faster on Windows. It’s no secret that LDC, the LLVM-based D compiler, generates faster binaries than DMD (for some D users, the general rule of thumb is to develop with DMD for its fast compile times and release with LDC for its faster binaries, though others argue that LDC is plenty fast for development and DMD is fine for production). There have been requests for some time to stop compiling DMD with DMD and start doing it with LDC instead. This release is the first to put that into practice.

There are a number of smaller enhancements to the Windows experience: the install.sh script available on the DMD downloads page that some people prefer now supports POSIX environments on Windows; the system link libraries that ship with the compiler have been upgraded from MinGW  5.0.2 to 7.0.0; LLD has been upgraded to 9.0.0; and there’s plenty more in the changelog.

C++ Header Generation

With just about every major release of DMD, D’s interoperability with C and C++ sees some kind of improvement. This release brings a huge one.

Over the years, some have speculated that it would be excellent if the D compiler could generate headers for C and C++ for D libraries intended to be usable in C or C++ programs. Now that wishful thinking has become a(n experimental) reality. Given a set of extern(C) or extern(C++) functions, DMD can generate header files that contain the appropriate C or C++ declarations. Three compiler switches get the job done:

  • -HC will cause the header to be generated and printed to standard output
  • -HCf=fileName will cause the header to be generated and printed to the specified file
  • -HCd=directoryname will (once it’s implemented) cause the header to be printed to a file in the specified directory

See the changelog for example output.

Other News

While the Corona virus was initially ramping up out of sight from most of the world, plans for DConf 2020 were ramping up online from different locations around the world. Planning began in November, the venue was secured in late December, and the website launched with the announcement in early January.

As news of the virus outbreak spread, the conference organizers grew concerned. Would we be okay in June? In late February, that concern manifested as a discussion of possible contingency plans. Two weeks later, it resulted in the decision to cancel DConf 2020. Thankfully, the D community has been supportive of the decision.

As part of the discussion of contingency plans, the possibility was raised of hosting an online conference. The idea of course came up in the discussion of the cancellation in the forums, and a few people reached out shortly after the initial announcement offering to provide help in setting something up. Walter created a forum thread to discuss the topic for anyone interested.

No one involved with organizing DConf has any experience with hosting an online conference. We’re currently exploring options and looking at what the organizers of other Conferences in the Time of COVID-19 are doing. We want to do it, and we want to do it well. Experience with organizing DConf in the real world has taught us not to jump on any old technology without first having a fallback (ahem, DConf 2018 livestream) and making sure the tech does what we expect it to (ahem, DConf 2019 livestream). So don’t expect a quick announcement. We want to find the right tech that fits our requirements and explore how it works before we move forward with setting dates. But do expect that DConf 2020 Online is looking more and more likely to become a thing.

News Update: Swag, Platforms, Documentation Help and More

Here are a few updates on things that have been going on both in front of and behind the scenes of the D Programming Language community.

New D Swag

We’ve got some new items in the DLang Swag Emporium: t-shirts, coffee mugs, and stickers sporting the Royal D logo. (If all Royal D items aren’t showing up for you in the Royal D category, check the D Rocket category. Everything should be in the correct location in a day or two).

You may notice that there are fewer options on the product page than for the other items, i.e. only one mug and sticker, and no dark tee option. They are available, though! When you select one of the existing products, you can change the style of the selection to one of several options. Beware! This may also change the price.

Remember, a small percentage of every item you order from the DLang Swag Emporium goes into the D Language Foundation’s General Fund. Plus, if you click through the link above or on the blog’s sidebar, we’ll get an additional referral fee on top of the item royalty. It’s an easy way to both get some D swag and contribute a few bucks to the Foundation.

Expanded Platform Progress

You maybe aware that some work has been ongoing in getting D onto more platforms. Adam Ruppe was working on contract to get LDC’s Android support to the finish line. He wrapped things up a few weeks back and has been paid out of the Foundation’s HR Fund.

Sebastiaan Koppe has been working on contract to get DRuntime ported to WebAssembly. Progress is ongoing and we currently expect it to be mostly wrapped up by the end of March. Like Adam, he’ll be out of the HR Fund when the contract is complete.

Work is also underway to bring LDC to iOS and iPadOS. We had been hoping to get someone to work on contract for this, but there are few people we know who are familiar enough with the platform to get it done and we were unable to find anyone then with the time to work on it. So we put up a bounty for it and kept our fingers crossed.

Recently, you may have seen forum posts from Jacob Carlborg indicating he’s been working on it in his spare time. Some preliminary support was merged in the LDC 1.20.0 release. Although he isn’t working under contract, he is working toward the bounty. That means anyone who wants to support him can contribute by increasing the bounty. Two contributors have already done so. The base amount of $3000 will be taken from the HR Fund when the work is complete.

And speaking of bounties, there are several others waiting for someone to claim them!

The HR Fund

With one payout from the fund and two coming up, we need to replenish it so we can always have cash earmarked for more contract work and bounties. You can make one-time or recurring donations of any amount directly and receive the same rewards available on our Open Collective page, or you can use a different link to make a $60 donation and get a DConf 2019 t-shirt in return. We’ve still got a few shirts available, so help us get rid of them and boost the HR Fund at the same time!

Documentation Event

Behind-the-scenes discussions about ideas to improve the D ecosystem in one way or another are frequently cycling through the inboxes of the people who can make them happen. Most never see the light of day, but there is one that has great potential. If it all comes together, I’ll be able to announce it in the coming weeks. We need your help to make that happen.

We need some specifics regarding areas where the documentation for D and items in the the D ecosystem is lacking. For example, people often complain about inconsistencies in the D spec, and missing info or examples in the DUB and vibe.d docs.

I’ve started a thread in the D forums where you can post your gripes about incomplete/missing/lackluster documentation. Remember, we need you to be specific. Just saying “the DUB docs are incomplete” doesn’t help. What specifically is missing? Or what specifically is wrong? The more information you can provide the better. And the more examples we can collect the better. The goal is to be able to define specific documentation tasks that anyone with the requisite knowledge can complete.

If we can get enough examples with enough detail, then I should be able to announce a new event sponsored by one of our generous benefactors. And I really want to be able to announce it!

DConf 2020

We really want to see a flood of talk submissions this year. If you’ve never been to DConf, or never presented at any conference, don’t let that stop you! Send us your submission and you may end up with a free trip to the conference.

Also, if you pay for an early-bird registration now (a 15% discount over the regular registration rate) and your talk is selected later, we’ll reimburse your registration fee. So if you’re planning to attend the conference even if your talk isn’t selected, it’s a good idea to register now and avoid the risk of missing the early-bird deadline.

We’re also offering once again the Open Source and Academic Discount; if you are a major open source contributor, a student, or an academic, we’ll give you a 50% discount on the regular registration rate. If you think you qualify, please don’t hesitate to take advantage of it by contacting social@dlang.org (or you can contact me directly at aldacron@gmail.com) for details on how to take advantage.

Finally, we never want to leave anyone out of DConf because they can’t afford to pay. This has been a policy of Walter’s from the beginning. If you are in or around London June 17 – 20 and would like to attend DConf but are unable to afford the registration and/or don’t qualify for the special discount, please email one of the addresses above and we’ll work something out.

DConf 2020: Submission Deadline, Early-Bird Registration, and Invited Keynote

In early January, I announced that Symmetry Investments is bringing DConf back to London for our 2020 edition. At the same time, I said we’d start taking submissions from anyone who wanted to send them in. In the interim, we’ve fixed our deadlines and prepared to start accepting reservations. There was only one thing remaining before I was ready for the formal call for submissions and opening of early-bird registrations: confirming our invited keynote speaker. Now that he has confirmed, it’s all official!

Invited Keynote

We’re excited to welcome Roberto Ierusalimschy to DConf 2020! You may know him from his work as the leading architect of the Lua programming language. He’s the author of Programming in Lua and an Associate Professor of Computer Science at PUC-Rio (the Pontifical Catholic University of Rio de Janeiro).

We don’t know yet what his talk will about, but it can be about any topic he wants. We’ll have more information on that for you when we publish the schedule of all selected talks after April 19.

Call for Submissions

We are accepting submissions for DConf 2020 until April 12. Authors will be notified of their final status by April 19.

We’re eager to see some new faces on the stage this year. If you’ve never presented at a DConf before, please don’t hesitate to send us one or more submissions. One person has already sent in seven!

Unless you’re Roberto Ierusalimschy, we prefer topics that are directly or indirectly related to D. We aren’t intransigent, though, so we’re willing to consider other topics. If someone sends us a proposal that isn’t about D but piques our collective interest, we’ll certainly give it serious consideration.

Having a talk selected is a great way to get to DConf if you’re on a budget. You’ll pay no registration fee, plus we’ll reimburse your transportation and lodging costs (within reason—five-star hotels and business- or first-class plane tickets aren’t on the menu). That’s a pretty good deal.

You can find instructions for writing and submitting your submissions on the DConf 2020 homepage.

Early-Bird Registration

Early-bird registration is available at $340, which is 15% off the regular $400 rate. Because we’re being sponsored by Symmetry in London once more, we once again must include a 20% VAT. So the total early-bird rate is $408 (similarly, the regular rate with VAT will be $480). We’re required by UK law to show you the basic rate and VAT in GBP based on the current HMRC exchange rate. That changes every month, so you can see the latest GPB rates in the registration section of the DConf 2020 homepage.

There, you’ll find options for Flipcause and PayPal. From our perspective, we prefer you use our Flipcause form. That gives you the option to cover the credit card processing fee for us so that 100% of your payment can be put toward DConf expenses. If you choose to uncheck that option, that’s fine, too! It will still save us from paying other fees. Every penny we can put toward the expenses helps.

If you do choose to go through PayPal, you have an option for USD and one for GBP. Some registrants told me last year that they get a GBP option even when clicking the USD button. And of course, some register with GBP-based credit cards. However, the GBP button on the DConf 2020 homepage is a fixed amount based on the current HMRC exchange rate. It changes, but only once a month. It may turn out to be cheaper for you than the rate you get from PayPal or your credit card provider. Of course, it could turn out to be more expensive, so if you’re looking to save a few pounds, you may want to investigate the different exchange rates if they apply to your situation.

And Now For Something Completely Different

DConf isn’t the only event Symmetry Investments is sponsoring these days. We recently wrapped up the 2019 edition of the Symmetry Autumn of Code.

This year, we started with five participants working on five interesting projects. Each participant was to complete a total of four milestones over four months with guidance from a mentor. At the successful completion of the first three milestones, each participant would receive $1000. At the end of the fourth and final milestone, one participant would be selected to receive one more $1000 payment and an all-expense paid trip to DConf.

As the event played out, we lost one of the participants at the end of Milestone 2. Two more were unable to fully commit to the Milestone 4 deadline (though they promised to continue working on their projects after SAOC). That left two participants for the SAOC review committee to select from. It was a very difficult decision, as both participants did excellent work and received glowing evaluations from their mentors.

Now I can announce that the SAOC 2019 finalist was Roberto Rosmaninho!

Roberto, with his mentor Nicholas Wilson, worked on adding support for Multi-Level Intermediate Representation (MLIR) to LDC, the LLVM-based D compiler. He is currently working on putting together pull requests for LDC and intends to work on optimizations going forward. He has also confirmed that he will take advantage of his reward so that we will have at least two Robertos at DConf this year.

As we did last year with Francesco Gallà, the SAOC 2018 finalist, we’ve asked Roberto to submit a talk this year. He promised to do so. We can’t promise his talk will be selected (though the odds are high out of the gate), but he still gets a free trip if it isn’t! Besides, we’re looking forward to meeting him.

On behalf of the D Language Foundation and Symmetry Investments, I want to thank everyone who participated in SAOC 2019. Keep an eye on this blog for news about future events.

Now go prep your DConf 2020 submissions!

DIP Reviews: Discussion vs. Feedback

Digital Mars D logoFor a while now, I’ve been including a link to the DIP Reviewer Guidelines in the initial forum post for every DIP review. My hope was that it would encourage reviewers to keep the thread on topic and also to provide more focused feedback. As it turns out, a link to reviewer guidelines is not quite enough. Recent review threads have turned into massive, 20+ page discussions touching on a number of tangential topics.

The primary purpose of the DIP review process, as I’ve tried to make clear in blog posts, forum discussions, and the reviewer guidelines, is to improve the DIP. It is not a referendum on the DIP. In every review round, the goal is to strengthen the content where it is lacking, bring clarity and precision to the language, make sure all the bases are covered, etc.

At the same time, we don’t want to discourage discussion on the merits of the proposal. Opinions about the necessity or the validity of a DIP can raise points that the language maintainers can take into consideration when they are deciding whether to approve or reject it, or even cause the DIP author to withdraw the proposal. It’s happened before. That’s why such discussion is encouraged in the Community Review rounds (though it’s generally discouraged in Final Review, which should be focused wholly on improving the proposal).

The problem

One issue with allowing such free-form discussion in the review threads is that there is a tremendous amount of noise drowning out the signal. Finding specific DIP-related feedback requires trawling through every post, digging through multiple paragraphs of mixed discussion and feedback. Sometimes, one or more people will level a criticism that spawns a long discussion and results in a changing of minds. This makes it time consuming for me as the DIP manager when I have to summarize the review. It also increases the likelihood that I’ll overlook something.

My summary isn’t just for the ‘Reviews’ section at the bottom of the DIP. It’s also my way of ensuring that the DIP author is aware of and has considered all the unique points of feedback. More than once I have found something the DIP author missed or had forgotten about. But if I overlook something and the DIP author also overlooks it, then we may have missed an opportunity to improve the DIP.

I have threatened to delete posts that go off topic in these threads,  but I can count on one hand the number of posts I’ve actually deleted. In reality, these discussions branch off in so many directions that it’s not easy to say definitively that a post that isn’t focused on the DIP itself is actually off topic. So I tend to let the posts stand rather than risk derailing the thread or removing information that is actually relevant.

The Solution

Starting with the upcoming Final Review of DIP 1027, I’m going to take a new approach to soliciting feedback. Rather than one review thread, I’ll be launching two for each DIP.

The Discussion Thread will be much the same as the current review thread. Opinions and discussion will be welcome and encouraged. I’ll still delete posts that are completely off topic, but other than that I’ll let the discussion flow where it may.

The Feedback Thread will be exclusively for feedback on the document and its contents. There will be no discussion allowed. Every post must contain specific points of feedback (preferably actionable items) intended to improve the proposal. Each post should be a direct reply to my initial post. There are only two exceptions: when a post author who has decided to retract feedback they made in a previous post, said poster can reply to the post in which they made the original feedback in order to make the retraction; and the DIP author may reply directly to any feedback post in order to indicate agreement or disagreement.

Posts in the feedback thread should contain answers to the questions posed in the DIP Reviewer Guidelines. It would be great if reviewers could take the time to do what Joseph Rushton Wakeling did in the Community Review for DIP 1028, where he explicitly listed and answered each question, but we won’t be requiring it. Feedback as bullet points is also very welcome.

Opinions on the validity of the proposed feature will be allowed in the feedback thread as long as they are backed with supporting arguments. In other words, “I’m against this! This is a terrible feature.” is not valid for the feedback thread. That sort of post goes in the discussion thread. However, “I’m against this. This is a terrible feature because <reasoned argument goes here>” is acceptable.

The rules of the feedback thread will be enforced without prejudice. Any post that is not a reply to my initial post, retraction of previous feedback, or a response by the DIP author will be deleted. Any post that does not provide the sort of feedback described above will be deleted. If I do delete a post, I won’t leave a new post explaining why. I’m going to update the DIP Reviewer Guidelines and each opening post in a feedback thread will include a link to that document as well as a paragraph or two summarizing the rules.

I’ll require DIP authors to follow both threads and to participate in the discussion thread. When it comes time to summarize the review, the feedback thread will be my primary source. I will, of course, follow the discussion thread as well and take notes on anything relevant. But if you want to ensure any specific criticisms you may have about a DIP are accounted for, be sure to post them in the feedback thread.

Hopefully, this new approach won’t be too disruptive. We’ll see how it goes.

 

Recent D Compiler Releases

Digital Mars D logoThe LDC team closed out the old year with release 1.19.0 of the LLVM-based D compiler, and the core D team opened the new year with version 2.090.0 of the reference D compiler, DMD. And if you haven’t yet heard, there was some big news about the GCC-based D compiler, GDC, a while back. Time to catch up!

LDC 1.19.0

This release updates the LDC compiler to D front end version 2.089.1, which was the current version when the compiler was released on the day after Christmas. The prebuilt packages are based on LLVM 9.01.

Among the big items in this release is some love for Android. The prebuilt DRuntime/Phobos library is now available for all supported Android targets. This release can be used in conjunction with Adam Ruppe’s D Android project, a collection of helper programs and interfaces, currently in beta, to facilitate D development on Android with LDC.

Windows users will find that the bundled MinGW-based link libraries for Windows development have been upgraded. They are now derived from .def files from the MinGW-w64 7.0.0 package. These libraries allow you to use the Windows system libraries without needing to install the Windows SDK.

DMD 2.090.0

The latest version of DMD was announced on January 7th. It ships with 10 major changes and 71 closed issues courtesy of 48 contributors.

With this release, it’s now possible to do more with lazy parameters. D has long supported lazy parameters:

An argument to a lazy parameter is not evaluated before the function is called. The argument is only evaluated if/when the parameter is evaluated within the function. Hence, a lazy argument can be executed 0 or more times.

Under the hood, they are implemented as delegates. Now, it’s possible to get at the underlying delegate by taking the address of the parameter, an operation which was previously illegal.

import std.stdio;

void chillax(lazy int x)
{
    auto dg = &x;
    assert(dg() == 10);
    writeln(x);
}

void main()
{
    chillax(2 * 5);
}

This release also renders obsolete a D idiom used by those who find themselves with a need to distinguish between finalization (non-deterministic object destruction usually initiated by the garbage collector) and normal destruction (deterministic object destruction) from inside a class or struct destructor.

With the current GC implementation, it’s illegal to perform some GC operations during finalization. However, D does not provide for separate finalizers and destructors. There is only ~this, which is referred to as a destructor even though it fills both roles. This sometimes presents difficulties when implementing destructors for types that are intended to be used with both GC and non-GC allocation. Any cleanup activity that touches the GC could throw an InvalidMemoryOperationError. Hence the need for the aforementioned workaround.

Now it’s possible to call the static GC member function, core.memory.GC.inFinalizer, to get your bearings in a destructor. It returns true if the current thread is performing object finalization, in which case you don’t want to be taking any actions that touch on GC operations. (I’ve been waiting for something like this before writing the next article in my GC series.)

GDC

Thanks to the hard work of Iain Buclaw, Johannes Pfau, and all of the volunteers who have maintained and contributed to it over the years, GDC was accepted into GCC 9 in late 2018 and made available as part of the GCC 9.1 package released in May of last year. GCC 9.2 was released last August. This version of GDC implements version 2.076 of the D front end. You can build it yourself or install it from the same place you install the GCC 9.x series.