Category Archives: Books

The Making of ‘D Web Development’

A long-time contributor to the D community, Kai Nacke is the author of ‘D Web Development‘ and the maintainer of LDC, the LLVM D Compiler. In this post, he tells the story of how his book came together.

At the beginning of 2014, I was asked by Packt Publishing if I wanted to review the D Cookbook by Adam Ruppe. Of course I wanted to!

The review was stressful, but it was a lot of fun. At the end of the year came a surprising question for me: would I be willing to switch sides and write a book myself? Here, I hesitated. Sure, writing your own book is a dream, but is this at all possible on top of a regular job? The proposed topic, D Web Development, was interesting. Web technologies I knew, of course, but the vibe.d framework was for me only a large unit test for each LDC release.

My interest was awakened and I created a chapter overview, based solely on my experience as a developer and the online documentation of vibe.d. The result came out well and I was offered a contract. It came with an immediate challenge: I should set up a small project plan. How do you plan to write a book?!?

Without any experience in this area, I stuck to the following rules. For each chapter, I planned a little time frame. Each should include at least one weekend, for the larger chapters perhaps even two. I reserved some time for the Easter holiday, too. The first version of the book would therefore be ready at the beginning of July, when I started writing in mid-February.

Even the first chapter showed that this plan was much too optimistic. The writing went off quickly – as soon as I had something I could write about. But experimenting and testing took a lot of time. For one thing, I didn’t have much experience with vibe.d. There were sample programs that I wanted to develop Saturday to write about on Sunday. However, I was still searching for errors on Monday, without having written a single line!

On the other hand, there were still a few rough edges in vibe.d at the time, but I did not want to write that these would be changed or implemented in later versions of the library. So I developed a few patches for vibe.d, e.g. digest authentication. By the way, there were also new LDC releases to create. Fortunately, the LDC team had expanded, so I just took care of the release itself (thanks so much, folks!). The result was, of course, that I missed many of my milestones.

In May, the first chapters came back from the review process. Other content also had to be written, such as the text for the back of the book. In mid-December, the last chapter was finished and almost all review notes on the other chapters were incorporated. After a little Christmas break, the remaining notes were quickly incorporated and the pre-final version of each chapter was created in January. And then, on February 1, 2016, the news came that my book was now published. I’d done it! Almost exactly one year after I had started with the first chapter.

Was the work worth it? In any case, it was a very special experience. Would I do it again? Yes! Right now, I am playing with the idea of updating the book and expanding a chapter. Let’s see what happens…

The Origins of Learning D

In ealearningdrly 2015, Adam Ruppe asked in the D forums if anyone was interested in authoring a new book for Packt Publishing, the company that published his D Cookbook. I had submitted a book proposal to Packt a few months before, one with a different concept, and had heard nothing back. So I began to mull over the idea of putting my name forward, but I was concerned about the time investment. Before I could decide, Packt contacted me with some details and an offer. With a target publication date of November 2015, I figured I had plenty of time to get it done, so I accepted. It wasn’t long before I learned how my concept of “plenty of time” was quite a bit off base.

Learning D was not the first programming book I had worked on. I coauthored Learn to Tango with D, which was published in 2008 by Apress, with three other D users. It was a book that simply aimed to introduce the language (version 1) and Tango, the community-driven alternative standard library for D1. I wrote two chapters and had nothing to do with the outline. It was a relatively easy experience that left me completely unprepared for the process of authoring an in-depth programming language book on my own. Tango was a bit controversial at the time and, though it’s no longer actively developed, a fork is still maintained and usable with modern D for those so inclined. The recently released Ocean library from Sociomantic Labs is derived from Tango.

When all the legalities were out of the way, work on the book began in earnest. Packt proposed an outline, with a handful of topics they specifically wanted me to cover. I also had to decide how to approach the book and who my target audience would be. Andrei Alexandrescu’s The D Programming Language already served as the language reference and Ali Çehreli’s Programming in D was targeting programming novices. I wanted to hit somewhere in the middle. I envisioned a young college student or recent undergraduate who, having already picked up some level of proficiency with a C-family language, was not a complete beginner, but also did not yet have the level of experience required to easily think in different languages.

Once I had an imaginary reader to talk to, I had a good idea of what I wanted to say. It was fairly easy to create two groups of features: those fundamental to become proficient enough with the language and the standard library to be productive, and those that are not essential or could be more aptly considered advanced. The former group would comprise the major focus of the book. But in addition to teaching the language, I had a general message I wanted to drive home.

Through all my years of visiting the D forums and maintaining a popular set of bindings to C libraries, I had encountered more than one new D user who was trying to program C++ or Java in D. While that approach will certainly enable some progress, it is bound to lead to compiler errors or unexpected results sooner rather than later. I explicitly pushed this message early in the book, and reiterated it each time I discussed one of the features that look like C++ or Java, but behave differently. Thinking in one language while learning another is a mistake that requires experience–it’s not the sort of thing novices do–and, as such, it’s a hard habit to break. My goal was to help minimize the pain.

I also wanted to create a sample program that demonstrated the language and library features I was discussing. My original plan was to create a new version of the program whenever a new paradigm was discussed. After gaining a false sense of confidence from completing Chapter 1 well before the deadline, Chapter 2 quickly disabused me of any thoughts that the whole book would go that way. It went well over the page budget and took longer than I had estimated, which meant the program for Chapter 2 was written over the space of a few hours during an all-night marathon. It was rightly lambasted by the technical reviewers. In the end, I scrapped the sample programs during the revision process and created a single program that evolves with new features as the book progresses. It didn’t turn out the way I had hoped, but it does show working examples of different paradigms in one program. The web app version developed in Chapter 10, which demonstrates the use of vibe.d, went more smoothly. Since it was the focus of the chapter, I was able to develop it in tandem with the text.

Speaking of the reviewers, the book would have been a mess had it not been for their valuable feedback. In the past, I had always believed authors were simply being kind when they said such a thing, but I can now attest that it is absolutely true. Packt asked me to recommend some technical reviewers, so I gave them a shortlist of people whom I knew to have strengths in areas where I was weak. John Colvin, Jonathan M. Davis, David Nadlinger, and Steven Schveighoffer came in early on, and were later joined by Kingsley Hendrickse and Ilya Yaroshenko. I was determined to be discriminating in implementing their suggestions, but they were so good that I agreed with and implemented most of them. And their criticisms were spot-on, catching coding errors, Big-O crimes, incorrect statements, and so much more. They taught me a few things I hadn’t known about D, cleared up a few misconceptions, and even spotted a couple of compiler bugs.

In the Foreword, Walter Bright says that this book, like D itself, is a labor of love. He is entirely correct. I first encountered D in 2003 and have been using it ever since. It’s just such a fun language. Though it’s a cliché to any long-time D user, I will tell anyone who cares to listen that this language is very much the sum of its parts; it’s not any one specific feature that makes D such a pleasure to use, but all of them taken together. Yes, it has its warts. Yes, it’s possible to encounter frustrating scenarios that require less-than-attractive workarounds. But, for me, my list of cons is nowhere near long enough to detract from the overall experience. I simply enjoy it. I want to see the cons list shrink and hope that more people see in the language what I see in it, so that one day ads looking for a D programmer are as common as those for other major languages. I accepted the offer to write this book as a way to contribute to that process. Most gratifyingly, I have received personal messages from a few readers letting me know that it helped them learn D. As a long time teacher, that’s a feeling I will never get tired of.

The Origins of the D Cookbook

Adam Ruppe is the author of D Cookbook and maintainer of This Week in D. Modules from his freely available arsd package are used throughout the D community. He is also known for his legendary DConf 2014 presentation.

dcookbookIn 2013, Packt Publishing approached me to write D Cookbook. Over the next several months, I was tasked with collecting over 100 D programming language “recipes” and writing a couple of pages about each one.

I wanted it to be a mix of practical and fun examples of what D can do over the wide variety of fields in which I have used it, each one illustrative of some language concept that the reader could use in different contexts. As such, for the majority of the book, I avoided the use of libraries, even keeping the presence of Phobos or my own modules to a minimum, allowing me to focus on the implementations of the ideas.

That said, I didn’t come up with a hundred recipes out of thin air! They came from three main sources: my arsd libraries, my support efforts on the D forums, the #D IRC channel, and Stack Overflow (it was Stack Overflow that got Packt’s attention in the first place), and a few other side projects, like my minimal D runtime (zip file).

I first saw D way back in 2002. At the time, I was still fairly new to programming and was going to download another copy of Digital Mars C++ (which I used to compile 16-bit DOS programs!) when I saw a link on the Digital Mars website about this D programming language. My first impression – upon seeing the import keyword – was that it was too Java-like and I wasn’t interested. Oh, how I regret that naive snap decision! I didn’t take my next look at D until 2006, and that time, with far more real world experience and theoretical knowledge under my belt, I quickly fell in love.

Over the next year, I wrote a game library based on SDL and OpenGL and made a few little games for myself. My D code at this point wasn’t terribly unique. Much of it was based on my existing C++ codebases. That fairly uninteresting fact would later become one of my most recurring D tips: a great deal of your C, C++, and Java knowledge can carry over directly to D! Thanks to the similarities between the languages, it’s easy to learn. While your code is unlikely to work perfectly if simply copy/pasted into a .d file, porting it probably isn’t hard. I found the D language very comfortable right off the bat. Even today, I tell people with D questions to simply consider how they would do it in C++ and apply that existing knowledge to D. This can also work with C and C++ solutions gleaned from Internet resources like Stack Overflow.

I started working as a full-time web developer in 2008, which put a time constraint on my hobby game development efforts, but didn’t ice my love of D. Indeed, it wasn’t long at all before I worked D into my professional life by writing web libraries and eventually transitioning my work projects away from PHP and onto D!

At the same time, the D language was going through a series of rapid changes. Templates and compile time function evaluation got overhauled, immutable was introduced… Most interestingly to me, compile time reflection got massively expanded and a few features like opDispatch got added. Compile time reflection in older D was limited to the is expression and template tricks, but new D had __traits, making it competitive even with dynamic languages, without sacrificing D’s other strengths.

I was a very early adopter of these features and set out to discover how to combine them in ways to make my work easier, to compete with the other web languages, and to just show off a little 🙂 If someone came on the forum and said D can’t do this, then I’d reply Challenge accepted.

In the following years, I wrote: a DOM and JSON library, taking the most interesting ideas from Javascript into D; a web framework, realizing some dreams I had in rapidly churning out prototypes that could also survive the change process of real world development; OAuth and database libraries to support the needs of the projects; and more. One of the most interesting modules was my web.d, which takes a D class definition and builds web infrastructure around it: an automatically generated web site with CRUD forms from the static type information as well as Javascript and PHP code for API access to that functionality. This stretched D’s reflection capabilities and hit quite a few bugs on the bleeding edge, necessitating creative workarounds or alternative approaches. If I was really desperate, I’d fix the bug in DMD myself!

At the same time, I was regularly seen in the D community, helping other people with their problems, and, of course, reading insights from other members of the community. Every few days, I had another tip, and was also building up a mental picture of people’s common difficulties with D.

By 2013, I had years of experience with almost every corner and combination of D. Now, you can get a good slice of that in just a few days by reading the book!

Programming in D: A Happy Accident

This is a guest post from Ali Çehreli, who not only uses D as a hobby, but also gets to use it as an employee of Weka.io. He is the author of Programming in D and is frequently found in the D Learn forum with ready answers to questions on using the language. He also is an officer of the D Foundation.

progindI consider the book Programming in D a happy accident, because initially it was not intended to be a book.

I used to be a frequent contributor to Turkish C and C++ forums. Although there were many smart and motivated members on those forums, most of them were not well-versed enough to follow programming resources in English. If they were patient enough to wait about ten years and if a publisher decided to have them translated, then they might get their hands on Turkish versions of their favorite books.

In 2009, around the time when my interest in C++ had started to diminish, I read with great excitement Andrei Alexandrescu’s The Case for D article in ACCU’s C Vu magazine (also available at Dr. Dobb’s). To a person coming from a C++ background, D was a fresh breath of air, removing some of C++’s warts and bringing many new features, some unique, some borrowed from other languages.

I was instantly hooked. I immediately created the Turkish D site ddili.org, translated Andrei’s article to Turkish, and published it there. One of the reasons for my excitement was the potential that D could be one software technology that Turkish programmers would not be left in the dark about. Since D was still being designed and implemented, there was time to write fresh Turkish documentation for it. I translated other D articles and started writing an HTML tutorial that would later become the book.

I knew very well that attempting to teach a topic is one of the best ways of learning that topic. I knew that I would be learning D myself. Little did I know then that this project would make me a better software engineer in general as well.

Teaching programming is a notoriously difficult task. According to some academic papers I found when I started the tutorial, one of the difficulties comes from the fact that different people model new concepts in their minds in different ways, rendering particular teaching methods inefficient at least for some students. Encouraged by the lack of one correct way of teaching programming, I picked one that was the easiest for me: introducing concepts in linear fashion with as few forward references as possible, starting with the most basic concepts like the = character confusingly meaning something different than is equal to.

Starting from the basics made it necessary for me to introduce lower-level concepts before higher-level concepts. For example, although the foreach statement is much more commonly used in practice, while, for, and foreach statements are introduced in the book in that order. I think that choice created a better foundation for the reader.

It took me two years to finish writing a flow of chapters from the assignment operator all the way to the garbage collector. It was very challenging and very rewarding to find a natural flow of presentation not only throughout the book but also within each chapter. The method I used for each chapter was to think about the presentation of the topic along with non-trivial examples beforehand, without touching the computer. I then wrote the chapter fairly quickly without much attention to detail, put it aside for a couple of days, then came back to review it from the point of view of a reader. I repeated that process perhaps five to ten times for each chapter until I thought it was fairly acceptable. Likely as a result of that process, a common feedback I receive is about how to-the-point my writing style has been.

Based on feedback from the Turkish community and encouragement from Andrei Alexandrescu, I started translating the book to English in early 2011. The translation continued along with new chapter additions, many corrections, and some chapter rewrites.

I made a PDF version available in January 2012 and the translation was finally completed in July 2014. Not only had I achieved my initial goal of providing fresh Turkish documentation for D, this book might have been the first software resource that was translated in the other direction.

I readily agreed with the suggestion that the book should be available in paper form as well. That decision brought many different challenges related to self-publishing like layout, cover design, pricing, the printing company, etc. The first print publication was in August 2015. Surprisingly, producing an ebook version turned out to be even more challenging. In addition to different kinds of layout issues, all ebook formats require special attention.

I am awestruck that my humble idea of a humble tutorial turned into a well known resource in the D ecosystem. It makes me very happy that people actually find the book useful. I am also happy that, periodically, people express interest in translating it to other languages. As of this writing, in addition to the completed Turkish and English versions, there are ongoing translations by volunteers to French and Chinese (German, Korean, Portuguese, and Russian translations were started but not continued).

As for future directions, I would like to add more chapters; definitely one on allocators once they’re added to the standard library (they currently live in the std.experimental.allocator package).

One thing that bothers me about the book is that most code samples don’t take full advantage of D’s universal function call syntax (UFCS), mainly because that feature was added to the language only after most of the book was already written. I would like to move the UFCS chapter to an earlier point in the book so that more code samples can be in the idiomatic D style.

The book will always be freely available online, allowing me to make frequent updates and corrections. Fortunately, my Inglish leaves a lot to improve on, so there will always be grammar and typo corrections as well.