When Andrei Alexandrescu stepped down as deputy leader of the D programming language, I was asked to take over the role going forward. It’s needless to say, but I’ll say it anyway, that those are some pretty big shoes to fill.
I’m still settling into my new role in the community and figuring out how I want to do things and what those things even are. None of this happens in a vacuum either, since Walter needs to be on board as well.
I was asked on the D forums to write a blog post on my “dreams and way forward for D”, so here it is. What I’d like to happen with D in the near future:
Memory safety
“But D has a GC!”, I hear you exclaim. Yes, but it’s also a systems programming language with value types and pointers, meaning that today, D isn’t memory safe. DIP1000 was a step in the right direction, but we have to be memory safe unless programmers opt-out via a “I know what I’m doing” @trusted block or function. This includes transitioning to @safe by default.
Safe and easy concurrency
We’re mostly there—using the actor model eliminates a lot of problems that would otherwise naturally occur. We need to finalize shared and make everything @safe as well.
Make D the default implementation language
D’s static reflection and code generation capabilities make it an ideal candidate to implement a codebase that needs to be called from several different languages and environments (e.g. Python, Excel, R, …). Traditionally this is done by specifying data structures and RPC calls in an Interface Definition Language (IDL) then translating that to the supported languages, with a wire protocol to go along with it.
With D, none of that is necessary. One can write the production code in D and have libraries automagically make that code callable from other languages. Add to all of this that it’s possible and easy to write D code that runs as fast or faster than the alternatives, and it’s a win on all fronts.
Second to none reflection.
Instead of disparate ways of getting things done with fragmented APIs (__traits, std.traits, custom code), I’d like for there to be a library that centralizes all reflection needs with a great API. I’m currently working on it.
Easier interop with C++.
As I mentioned in my DConf 2019 talk, C++ has had the success it’s enjoyed so far by making the transition from C virtually seamless. I want current C++ programmers with legacy codebases to just as easily be able to start writing D code. That’s why I wrote dpp, but it’s not quite there yet and we might have to make language changes to accommodate this going forward.
Fast development times.
I think we need a ridiculously fast interpreter so that we can skip machine code generation and linking. To me, this should be the default way of running unittest blocks for faster feedback, with programmers only compiling their code for runtime performance and/or to ship binaries to final users. This would also enable a REPL.
String interpolation
I was initially against this, but the more I think about it the more it seems to make sense for D. Why? String mixins. Code generation is one of D’s greatest strengths, and token strings enable visually pleasing blocks of code that are actually “just strings”. String interpolation would make them vastly easier to use. As it happens, there’s a draft DIP for it in the pipeline.
That’s what I came up with after a long walk by Lake Geneva. I’d love to know what the community thinks of this, what their pet peeves and pet features would be, and how they think this would help or hinder D going forward.
I was hoping Tooling (IDE, Debugger) would be on that “wish list”
Anyways.. all the things mentioned are super interesting, the one i’m looking forward to the most is string interpolation
I can’t lead on the tooling front, because I don’t understand what it is that people who want better tooling need or want. For me, I have everything I need with Emacs, gdb, DCD, and dub. I understand that’s not enough for others, but I don’t understand why.
dub is bad in the incremental build of really big projects.
dub is bad as packaging meson is better
Also a tools like jhipster witten in D would be awesome (https://www.jhipster.tech/)
It’s the learning curve. And the commitment you need to put it to make it worth learning. In general, the juice has to be worth the squeeze (in relative terms) I can get everything I need to get done with vscode. Manual setup is much less. Better UX. Plugins that just work. Immediate productivity without any hoop jumping on even esoteric languages.
This,
Everything needs to be accessible and easy to use, choice nowadays is insane for languages, what will differentiate them will be on tooling part, it makes language easier to learn, teach and use
Dlang lacks a good IDE, dlang told us gdb lldb ….etc,
But I can’t config it well in a week on linux(I can’t debug and I don’t how to config it).
I think everybody want to use Dlang will come across this problem,so
I suggest you give us a detail help document.
I think most newer drop dlang for this reason.
I can’t lead on the tooling front, because I don’t understand what it is that people who want better tooling need or want. For me, I have everything I need with Emacs, gdb, DCD, and dub. I understand that’s not enough for others, but I don’t understand why.
————————–
Dear sir,you are a great man,but we are newer,we need you help us how to config a IDE just you have ( Emacs, gdb, DCD, and dub)
Wow, sounds very promising! 🙂
What about “natural” inheritance for structures? Which seems to be a minor things compared to the topics you wish to go on with.
Ah I forgot to express another wish: thorough documentation; e.g but not only for betterC
How about a ridiculously fast interpreter that generates machine code. (making it ridiculously fast). This technology exists in some Forth systems. (MPE, SwiftForth, iForth)
Does this “@safe by default” philosophy risk turning D into something more like Rust?
Rust’s goals are noble, but the end result is, in my opinion and experience, absolutely awful to work with.
I can write nearly-as-safe code in numerous other languages, in much less time and with much less pain than when using Rust.
What will be done to prevent D from acquiring the same problems that Rust exhibits?
What will be done to ensure that this quest for safety doesn’t make D impractical to use?
It’s easy to make D easier than Rust when it comes to memory safety: the GC. Use that, and you’ll be fine.
“Easier interop with C++” – As better interop with C++ is good I don’t think that it is a major showstopper for adopting the D by C++ programmers. For me (as a C++ developer) the main showstopper is GC (so no RAII). Ok, you can write some code without GC, by mainly just simple examples, cannot use STD lib, etc. AFAIK. So until the optional GC (or some RAII constructs) I wouldn’t even consider moving to D (even if it has lot of other useful features).
The first question is “why is the GC a showstopper?”. In my experience, it hardly ever is.
When it is, it’s not true that one can only write simple examples without using GC allocated memory. I’ve written a lot of @nogc code myself.
D has RAII like C++ does, so even if there weren’t libraries to use @nogc code they’re not hard to write. But there are! I wrote automem and nogc, for instance.
What do you mean by no RAII ? Of course D has RAII !
D destructors are called whenever a struct goes out of stack scope, which enabled Atila Neves to write a smart pointers library (https://github.com/atilaneves/automem)
D also has the scope statement that facilitates a lot handling C resources without having the burden to wrap each of them in a struct like in C++ in order to be exception safe.
You can use part of the std library without the GC, not all of it, sure but a growing part of it.
I will answer to both of you Atila and Olivier.
As first, I’m not experienced D and it was long time since I tried something with it (but I’m pinging the dlang forums/blogs from time to time) so I might have outdated informatiosn.
It isn’t strictly of GC, but mainly the lack of RAII.
From what I have read, you cannot create a smart pointer for classes, only for structs (which has its own limitations). Scope guards are OK for some local stuff, but even then if I’m working with some external resources I wrap it in the class because there would be also some functions implented so it’s better to use RAII there. But in most instances I’m passing these resource down to the other classes and I need to track it’s lifetime, so no scope guards there.
AFAIK destructors for classes in D are “useless”, because you are very limited what you can do here and no member access could be done.
Yes you can write @nogc code, but then the memory management must be manual or use some others means (as https://github.com/caraus-ecms/tanya) but you couldn’t use usual syntax which is suboptimal. It’s shame that the WHOLE STD lib isn’t @nogc (but I don’t think that it is even possible, now).
So in short, what I would like is to have full RAII support (so for instance I can make my own “smart pointers” without GC interference) but without the syntax sacrifice.
Or have some another GC implementation which would use some RC pointers and have some API to use it somehow.
> From what I have read, you cannot create a smart pointer for classes,
This is incorrect: https//github.com/atilaneves/automem
> only for structs (which has its own limitations)
Having written smart pointers in D, I’m not aware of any limitations compared to, say, C++. What limitations do you mean?
> AFAIK destructors for classes in D are “useless”, because you are very limited what you can do here and no member access could be done
This is also incorrect. D destructors can do anything a regular member function can.
> Yes you can write @nogc code, but then the memory management must be manual or use some others means
Yes. This is the same situation C++ is in.
> but you couldn’t use usual syntax which is suboptimal
To my knowledge, the only usual syntax which you wouldn’t be able to use is `new T`. Everything else, including concatenation operations, look the same.
> It’s shame that the WHOLE STD lib isn’t @nogc
On the contrary, using the GC is the easiest way to write code and be memory safe.
> So in short, what I would like is to have full RAII support
You do!
> (so for instance I can make my own “smart pointers” without GC interference) but without the syntax sacrifice.
You can! Please take a look at automem linked above.
> Or have some another GC implementation which would use some RC pointers and have some API to use it somehow.
This is being worked on.
I was looking for the automem library with online D, but couldn’t make to compile simple example.
I have an interface ITable and then concrete class TableRef which implements ITable. Now a want to use RefCounted with ITable as template parameter but use TableRef instance, something like this in C++:
std::shared_ptr p_table(new TableRef{});
Is this possible?
Limitation of structs are no inheritance for example (but if RefCounted could work with classes then it doesn’t matter).
> This is also incorrect. D destructors can do anything a regular member function can.
From what I’ve read in the past in documentation, it was stated that when destructor of CLASS is called any of the members could be already deallocated by GC so you shall not touch it. So this isn’t true?
>On the contrary, using the GC is the easiest way to write code and be memory safe.
True, but if the STD lib would be @nogc then building the GC version atop of @nogc should be possible, shouldn’t it?
Thanks for your answers
The code should be like this (the comment has been stripped of template angle brackets):
std::shared_ptr<ITable> p_table(new TableRef{});
There’s a limit of replies on here, but Trigve:
If your use case doesn’t work with automem, please file a bug. Otherwise, yes, what you want to do should be possible.
Destructors and GC-allocated objects don’t mix very well. Then again, GC allocated objects aren’t usually used with RAII!
Ok, thanks for the info.
I’m not confident enough that it is a bug, probably using it wrong. I’ll look at it later. Anyway, will play will the automem libray for sure, looks promising.
DCD is fine, but it does not adequately support several of D’s great features, such as generics.
Also, there is a myriad of “simple” things that good IDEs do for a developer, that no current tool properly does. I’m thinking of tooltips with documentation, code navigation, refactoring (renaming symbols and such), linting, signature help, basically a lot of the features the language server protocol specifies.
There are already a couple of D language servers around, but all of them AFAIK are based on DCD, and run into a wall for more advanced functionality, as DCD wasn’t intended as a basis for other functionalities than autocompletion.
> I’m thinking of tooltips with documentation, code navigation, refactoring (renaming symbols and such), linting, signature help
Other than refactoring, I have all of this in Emacs using DCD.
> Safe and easy concurrency
Let’s please get D on board with structured concurrency. This includes a having a standard cancellation mechanism.
https://forum.dlang.org/thread/ndirsnblaktyfellckil@forum.dlang.org
https://medium.com/@belm0/concurrency-made-easy-d3fdb0382c58
Futures –> bad. Quoting Nathaniel Smith:
> Language design happens by stepwise incremental improvements. And like any form of gradient descent, it can get stuck in sub-optimal local minima. Futures are a really attractive local minima that have trapped C#, JS, Python, now Rust…
https://trio.discourse.group/t/structured-concurrency-in-rust/73/14
>>On the contrary, using the GC is the easiest way to write code and be >>memory safe.
While your statement is undeniably true, it is also a red herring. The issue is , in practice, every-time you use betterC or no-GC you are unable to use the stdlib of the language (or swaths of it anyways). So, no batteries. The stdlib is largely **unusable** with nonGC. This makes D almost a non-starter for any project which doesn’t want to use GC.
The problem I see with D is that it doesn’t really knows what it want to be.
It has GC , but no implementation has a industrial strength garbage collector like Java or .NET does. Seems that Go is getting there slowly as well.
It has a no-GC mode, but you loose the standard library of the language.
It has a better C mode, but again, no standard library usable.
D has a lot of things going for it, but the decision to make std lib garbage collected was a very bad one for the language. Both technically and socially. Remember, large scale adoption of a language is a social phenomena, not a technical one.
If I may make a suggestion, in addition to language richness, we need to concentrate on language reach.
Frrexample, D could and should be a well-known high-performance alternative to such industry-stranded dodos like Java and C#. One way to help here is to author libraries to facilitate porting same to D.
Next, with the banning of Andrrhoid on Huawei devices, some Gurgle fanboy articles claim that this is a serious blow to Huawei – I think they have this backwards, as Huawei is poised for some serious growth outside of the Untied Status where Andrrhoid will thankfully be not present. With the potential development capital, human and otherwise, which Huawei possesses and the relative ease in which an Andrrhoid alternative can be developed and/or acquired, this leaves an app vacuum which could be filled by nice shiny new (OpenGL ES) apps, all written in D of course.
A growing awareness of the true costs of some of the so-called tech companies (e.g. $2.2BN junk bonds for the Nyetflikz Ponzi scheme, followed by a lack of investors) may open up some excellent opportunities for new apps on new platforms, D should be ready.
sir we need a complete development tool Just like you have, please make a docker, our project need a ide, but we. Can’t debug
Dlang is better than Golang,But most people give up Dlang for there is no A easy config IDE!!
You don’t mention bug fixing or improved documentation or filling in the gaps in the std library.
I’m also concerned about “safe by default”. One of the reasons I’m considering D is because I find Rust too painful (except where nothing but performance counts, which isn’t often in my case).
And I also think a response to https://www.reddit.com/r/programming/comments/eyzrm9/d_as_a_c_replacement_the_art_of_machinery/ would be worth giving.
I understand the sentiment that “Rust is too painful”. However (and I really don’t know how this psychological frame came about), D @safe code isn’t hard. It’s actually just “nearly all D”. *Without* DIP1000, you basically can’t:
* Use .ptr on a slice
* Slice a pointer
* Do pointer arithmetic
* Take the address of a local
*With* DIP1000 taking the address of a local is ok if used with scope. That’s it! Given that the GC is the default in D, there’s usually no need to do anything with pointers at all.
As a new D user, this whole “safe by default” thing is concerning. As a systems programmer i want to run away from the garbage collector and not run towards it. I don’t think people truly understand how big of a problem the garbage collector can be in hard real time systems.
This approach looks like how C# deals with safe and unsafe, and honestly, if D is going to do the same as C#, what is even the point ?
I thought D could be a C++ alternative, regarding high performance and low memory usage. Maybe i was wrong.
@nogc is orthogonal to @safe. In fact, what I want to happen is to make it possible to write @safe @nogc code.
My apologies then, i think i misunderstood, thanks for the clarification