{"id":1507,"date":"2018-03-29T14:00:40","date_gmt":"2018-03-29T14:00:40","guid":{"rendered":"http:\/\/dlang.org\/blog\/?p=1507"},"modified":"2021-10-08T11:04:46","modified_gmt":"2021-10-08T11:04:46","slug":"std-variant-is-everything-cool-about-d","status":"publish","type":"post","link":"https:\/\/dlang.org\/blog\/2018\/03\/29\/std-variant-is-everything-cool-about-d\/","title":{"rendered":"std.variant Is Everything Cool About D"},"content":{"rendered":"<p><em>Jared Hanson has been involved with the D community since 2012, and an active contributor since 2013. Recently, he joined the Phobos team and devised a scheme to make it look like he&#8217;s contributing by adding at least 1 tag to every new PR. He holds a Bachelor of Computer Science degree from the University of New Brunswick and works as a Level 3 Support Engineer at one of the largest cybersecurity companies in the world.<\/em><\/p>\n<hr \/>\n<p><img loading=\"lazy\" class=\"alignleft size-full wp-image-181\" src=\"http:\/\/dlang.org\/blog\/wp-content\/uploads\/2016\/08\/d6.png\" alt=\"\" width=\"200\" height=\"200\" \/>I recently read a great article by <a href=\"https:\/\/bitbashing.io\/about.html\">Matt Kline<\/a> on how <a href=\"https:\/\/bitbashing.io\/std-visit.html\">std::visit is everything wrong with modern C++<\/a>. My C++ skills have grown rusty from disuse (I have long since left for the greener pastures of D), but I was curious as to how things had changed in my absence.<\/p>\n<p>Despite my relative unfamiliarity with post\u20132003 C++, I had heard about the addition of a library-based sum type in\u00a0<code>std<\/code> for C++17. My curiosity was mildly piqued by the news, although like many new additions to C++ in the past decade, it\u2019s something D has had <a href=\"https:\/\/github.com\/dlang\/phobos\/blob\/eec6be69edec9601f9f856afcd25a797e845c181\/std\/variant.d\">for years<\/a>. Given the seemingly sensational title of Mr. Kline\u2019s article, I wanted to see just what was so bad about <code>std::visit<\/code>, and to get a feel for how well D\u2019s equivalent measures up.<\/p>\n<p>My intuition was that the author was exaggerating for the sake of an interesting article. We\u2019ve all heard the oft-repeated criticism that C++ is complex and inconsistent (even some of its <a href=\"https:\/\/www.youtube.com\/watch?v=KAWA1DuvCnQ\">biggest proponents<\/a> think so), and it <em>is<\/em> true that the ergonomics of templates in D are vastly improved over C++. Nevertheless, the underlying concepts are the same; I was dubious that <code>std::visit<\/code> could be much harder to use than <code>std.variant.visit<\/code>, if at all.<\/p>\n<p>For the record, my intuition was completely and utterly wrong.<\/p>\n<h2 id=\"exploringstd.variant\">Exploring std.variant<\/h2>\n<p>Before we continue, let me quickly introduce D\u2019s <a href=\"https:\/\/dlang.org\/phobos\/std_variant.html\">std.variant<\/a> module. The module centres around the <a href=\"https:\/\/dlang.org\/phobos\/std_variant.html#.Variant\">Variant<\/a> type: this is not actually a sum type like C++\u2019s <code>std::variant<\/code>, but a type-safe container that can contain a value of any type. It also knows the type of the value it currently contains (if you\u2019ve ever implemented a type-safe union, you\u2019ll realize why that part is important). This is akin to C++\u2019s <code>std::any<\/code> as opposed to <code>std::variant<\/code>; very unfortunate, then, that C++ chose to use the name <code>variant<\/code> for its implementation of a sum type instead. <em>C\u2019est la vie.<\/em> The type is used as follows:<\/p>\n<pre class=\"prettyprint lang-d\">import std.variant;\r\n\r\nVariant a;\r\na = 42;\r\nassert(a.type == typeid(int));\r\na += 1;\r\nassert(a == 43);\r\n\r\nfloat f = a.get!float; \/\/convert a to float\r\nassert(f == 43);\r\na \/= 2;\r\nf \/= 2;\r\nassert(a == 21 &amp;&amp; f == 21.5);\r\n\r\na = \"D rocks!\";\r\nassert(a.type == typeid(string));\r\n\r\nVariant b = new Object();\r\nVariant c = b;\r\nassert(c is b); \/\/c and b point to the same object\r\nb \/= 2; \/\/Error: no possible match found for Variant \/ int<\/pre>\n<p>Luckily, <code>std.variant<\/code> <em>does<\/em> provide a sum type as well: enter <a href=\"https:\/\/dlang.org\/phobos\/std_variant.html#.Algebraic\">Algebraic<\/a>. The name <code>Algebraic<\/code> refers to <a href=\"https:\/\/en.wikipedia.org\/wiki\/Algebraic_data_type\">algebraic data types<\/a>, of which one kind is a <a href=\"https:\/\/www.quora.com\/What-is-a-sum-type\">\u201csum type\u201d<\/a>. Another example is the tuple, which is a \u201cproduct type\u201d.<\/p>\n<p>In actuality, <code>Algebraic<\/code> is not a separate type from <code>Variant<\/code>; the former is an <a href=\"https:\/\/dlang.org\/spec\/declaration.html#alias\">alias<\/a> for the latter that takes a compile-time specified list of types. The values which an <code>Algebraic<\/code> may take on are limited to those whose type is in that list. For example, an <code>Algebraic!(int, string)<\/code> can contain a value of type <code>int<\/code> or <code>string<\/code>, but if you try to assign a <code>string<\/code> value to an <code>Algebraic!(float, bool)<\/code>, you\u2019ll get an error at compile time. The result is that we effectively get an in-library sum type for free! Pretty darn cool. It\u2019s used like this:<\/p>\n<pre class=\"prettyprint lang-d\">alias Null = typeof(null); \/\/for convenience\r\nalias Option(T) = Algebraic!(T, Null);\r\n\r\nOption!size_t indexOf(int[] haystack, int needle) {\r\n    foreach (size_t i, int n; haystack)\r\n        if (n == needle)\r\n            return Option!size_t(i);\r\n    return Option!size_t(null);\r\n}\r\n\r\nint[] a = [4, 2, 210, 42, 7];\r\nOption!size_t index = a.indexOf(42); \/\/call indexOf like a method using UFCS\r\nassert(!index.peek!Null); \/\/assert that `index` does not contain a value of type Null\r\nassert(index == size_t(3));\r\n\r\nOption!size_t index2 = a.indexOf(117);\r\nassert(index2.peek!Null);<\/pre>\n<p>The <code>peek<\/code> function takes a <code>Variant<\/code> as a runtime argument, and a type <code>T<\/code> as a compile-time argument. It returns a pointer to <code>T<\/code> that points to the <code>Variant<\/code>\u2019s contained value <em>iff<\/em> the <code>Variant<\/code> contains a value of type <code>T<\/code>; otherwise, the pointer is <code>null<\/code>.<\/p>\n<p><sup><em><strong>Note:<\/strong> I\u2019ve made use of <a href=\"https:\/\/dlang.org\/spec\/function.html#pseudo-member\">Universal Function Call Syntax<\/a> to call the free function <code>indexOf<\/code> as if it were a member function of <code>int[]<\/code>.<\/em><\/sup><\/p>\n<p>In addition, just like C++, D\u2019s standard library has a special <code>visit<\/code> function that operates on <code>Algebraic<\/code>. It allows the user to supply a visitor for each type of value the <code>Algebraic<\/code> may hold, which will be executed <em>iff<\/em> it holds data of that type at runtime. More on that in a moment.<\/p>\n<h2 id=\"torecap:\">To recap:<\/h2>\n<ul>\n<li><code>std.variant.Variant<\/code> is the equivalent of <code>std::any<\/code>. It is a type-safe container that can contain a value of any type.<\/li>\n<li><code>std.variant.Algebraic<\/code> is the equivalent of <code>std::variant<\/code> and is a sum type similar to what you\u2019d find in Swift, Haskell, Rust, etc. It is a thin wrapper over <code>Variant<\/code> that restricts what type of values it may contain via a compile-time specified list.<\/li>\n<li><code>std.variant<\/code> provides a <code>visit<\/code> function akin to <code>std::visit<\/code> which dispatches based on the type of the contained value.<\/li>\n<\/ul>\n<p>With that out of the way, let\u2019s now talk about what\u2019s wrong with <code>std::visit<\/code> in C++, and how D makes <code>std.variant.visit<\/code> much more pleasant to use by leveraging its powerful toolbox of compile-time introspection and code generation features.<\/p>\n<h2 id=\"theproblemwithstd::visit\">The problem with std::visit<\/h2>\n<p>The main problems with the C++ implementation are that &#8211; aside from clunkier template syntax &#8211; metaprogramming is very arcane and convoluted, and there are very few static introspection tools included out of the box. You get the absolute basics in <code>std::type_traits<\/code>, but that\u2019s it (there are a couple third-party solutions, which are appropriately horrifying and verbose). This makes implementing <code>std::visit<\/code> much more difficult than it has to be, and also pushes that complexity down to consumers of the library, which makes <em>using<\/em> it that much more difficult as well. My eyes bled at this code from Mr. Kline\u2019s article which generates a visitor struct from the provided lambda functions:<\/p>\n<pre class=\"prettyprint lang-c_cpp\">template &lt;class... Fs&gt;\r\nstruct overload;\r\n\r\ntemplate &lt;class F0, class... Frest&gt;\r\nstruct overload&lt;F0, Frest...&gt; : F0, overload&lt;Frest...&gt;\r\n{\r\n    overload(F0 f0, Frest... rest) : F0(f0), overload&lt;Frest...&gt;(rest...) {}\r\n\r\n    using F0::operator();\r\n    using overload&lt;Frest...&gt;::operator();\r\n};\r\n\r\ntemplate &lt;class F0&gt;\r\nstruct overload&lt;F0&gt; : F0\r\n{\r\n    overload(F0 f0) : F0(f0) {}\r\n\r\n    using F0::operator();\r\n};\r\n\r\ntemplate &lt;class... Fs&gt;\r\nauto make_visitor(Fs... fs)\r\n{\r\n    return overload&lt;Fs...&gt;(fs...);\r\n}<\/pre>\n<p>Now, as he points out, this can be simplified down to the following in C++17:<\/p>\n<pre class=\"prettyprint lang-c_cpp\">template&lt;class... Ts&gt; struct overloaded : Ts... { using Ts::operator()...; };\r\ntemplate&lt;class... Ts&gt; overloaded(Ts...) -&gt; overloaded&lt;Ts...&gt;;\r\n\r\ntemplate &lt;class... Fs&gt;\r\nauto make_visitor(Fs... fs)\r\n{\r\n    return overload&lt;Fs...&gt;(fs...);\r\n}<\/pre>\n<p>However, this code is still quite ugly (though I suspect I could get used to the elipses syntax eventually). Despite being a massive improvement on the preceding example, it\u2019s hard to get right when writing it, and hard to understand when reading it. To write (and more importantly, <em>read<\/em>) code like this, you have to know about:<\/p>\n<ul>\n<li>Templates<\/li>\n<li>Template argument packs<\/li>\n<li><a href=\"http:\/\/en.cppreference.com\/w\/cpp\/language\/sfinae\">SFINAE<\/a><\/li>\n<li><a href=\"http:\/\/en.cppreference.com\/w\/cpp\/language\/class_template_argument_deduction\">User-defined deduction guides<\/a><\/li>\n<li><a href=\"http:\/\/www.cplusplus.com\/articles\/EhvU7k9E\/\">The ellipsis operator (\u2026)<\/a><\/li>\n<li>Operator overloading<\/li>\n<li>C++-specific metaprogramming techniques<\/li>\n<\/ul>\n<p>There\u2019s a lot of complicated template expansion and code generation going on, but it\u2019s hidden behind the scenes. And boy oh boy, if you screw something up you\u2019d better <em>believe<\/em> that the compiler is going to spit some supremely perplexing errors back at you.<\/p>\n<p><sup><em><strong>Note:<\/strong> As a fun exercise, try leaving out an overload for one of the types contained in your <code>variant<\/code> and marvel at the truly cryptic error message your compiler prints.<\/em><\/sup><\/p>\n<p>Here\u2019s an example from <a href=\"http:\/\/en.cppreference.com\/w\/cpp\/utility\/variant\/visit\">cppreference.com<\/a> showcasing the <em>minimal<\/em> amount of work necessary to use <code>std::visit<\/code>:<\/p>\n<pre class=\"prettyprint lang-c_cpp\">template&lt;class... Ts&gt; struct overloaded : Ts... { using Ts::operator()...; };\r\ntemplate&lt;class... Ts&gt; overloaded(Ts...) -&gt; overloaded&lt;Ts...&gt;;\r\n\r\nusing var_t = std::variant&lt;int, long, double, std::string&gt;;\r\nstd::vector&lt;var_t&gt; vec = {10, 15l, 1.5, \"hello\"};\r\n\r\nfor (auto&amp; v: vec) {\r\n    std::visit(overloaded {\r\n        [](auto arg) { std::cout &lt;&lt; arg &lt;&lt; ' '; },\r\n        [](double arg) { std::cout &lt;&lt; std::fixed &lt;&lt; arg &lt;&lt; ' '; },\r\n        [](const std::string&amp; arg) { std::cout &lt;&lt; std::quoted(arg) &lt;&lt; ' '; },\r\n    }, v);\r\n}<\/pre>\n<p><sup><em><strong>Note:<\/strong> I don\u2019t show it in this article, but if you want to see this example re-written in D, it\u2019s <a href=\"https:\/\/run.dlang.io\/is\/CKnGCk\">here<\/a>.<\/em><\/sup><\/p>\n<p>Why is this extra work forced on us by C++, just to make use of <code>std::visit<\/code>? Users are stuck between a rock and a hard place: either write some truly stigmata-inducing code to generate a struct with the necessary overloads, or bite the bullet and write a new struct every time you want to use <code>std::visit<\/code>. Neither is very appealing, and both are a one-way ticket to Boilerplate Hell. The fact that you have to jump through such ridiculous hoops and write some ugly-looking boilerplate for something that <em>should be<\/em> very simple is just\u2026 ridiculous. As Mr. Kline astutely puts it:<\/p>\n<blockquote><p>The rigmarole needed for <code>std::visit<\/code> is entirely insane.<\/p><\/blockquote>\n<p>We can do better in D.<\/p>\n<h2 id=\"thedsolution\">The D solution<\/h2>\n<p>This is how the typical programmer would implement <code>make_visitor<\/code>, using D\u2019s powerful compile-time type introspection tools and code generation abilities:<\/p>\n<pre class=\"prettyprint lang-d\">import std.traits: Parameters;\r\n\r\nstruct variant_visitor(Funs...)\r\n{\r\n    Funs fs;\r\n    this(Funs fs) { this.fs = fs; }\r\n\r\n    static foreach(i, Fun; Funs) \/\/Generate a different overload of opCall for each Fs\r\n        auto opCall(Parameters!Fun params) { return fs[i](params); }\r\n}\r\n\r\nauto make_visitor(Funs...)(Funs fs)\r\n{\r\n    return variant_visitor!Funs(fs);\r\n}<\/pre>\n<p>And\u2026 that\u2019s it. We\u2019re done. No pain, no strain, no bleeding from the eyes. It is a few more lines than the C++ version, granted, but in my opinion, it is also much simpler than the C++ version. To write and\/or read this code, you have to understand a demonstrably smaller number of concepts:<\/p>\n<ul>\n<li>Templates<\/li>\n<li>Template argument packs<\/li>\n<li><a href=\"https:\/\/github.com\/dlang\/DIPs\/blob\/master\/DIPs\/accepted\/DIP1010.md\">static foreach<\/a><\/li>\n<li>Operator overloading<\/li>\n<\/ul>\n<p>However, a D programmer would not write this code. Why? Because <code>std.variant.visit<\/code> does not take a callable struct. From <a href=\"https:\/\/dlang.org\/phobos\/std_variant.html#.visit\">the documentation<\/a>:<\/p>\n<blockquote><p>Applies a <strong>delegate<\/strong> or <strong>function<\/strong> to the given Algebraic depending on the held type, ensuring that all types are handled by the visiting functions. Visiting handlers are passed in the template parameter list. <strong><em>(emphasis mine)<\/em><\/strong><\/p><\/blockquote>\n<p>So <code>visit<\/code> only accepts a <code>delegate<\/code> or <code>function<\/code>, and figures out which one to pass the contained value to based on the functions\u2019 signatures.<\/p>\n<p>Why do this and give the user fewer options? D is what I like to call an anti-boilerplate language. In all things, D prefers the most direct method, and thus, <code>visit<\/code> takes a compile-time specified list of functions as template arguments. <code>std.variant.visit<\/code> may give the user fewer options, but <em>unlike<\/em> <code>std::visit<\/code>, it does not require them to painstakingly create a new struct that overloads <code>opCall<\/code> for each case, or to waste time writing something like <code>make_visitor<\/code>.<\/p>\n<p>This also highlights the difference between the two languages themselves. D may sometimes give the user fewer options (don\u2019t worry though &#8211; D is a systems programming language, so you\u2019re never <em>completely<\/em> without options), but it is in service of making their lives easier. With D, you get faster, safer code, that combines the speed of native compilation with the productivity of scripting languages (hence, D\u2019s motto: <em>\u201cFast code, fast\u201d<\/em>). With <code>std.variant.visit<\/code>, there\u2019s no messing around defining structs with callable methods or unpacking tuples or wrangling arguments; just straightforward, understandable code:<\/p>\n<pre class=\"prettyprint lang-d\">Algebraic!(string, int, bool) v = \"D rocks!\";\r\nv.visit!(\r\n    (string s) =&gt; writeln(\"string: \", s),\r\n    (int    n) =&gt; writeln(\"int: \", n),\r\n    (bool   b) =&gt; writeln(\"bool: \", b),\r\n);<\/pre>\n<p>And in a puff of efficiency, we\u2019ve completely obviated all this machinery that C++ requires for <code>std::visit<\/code>, and greatly simplified our users\u2019 lives.<\/p>\n<p>For comparison, the C++ equivalent:<\/p>\n<pre class=\"prettyprint lang-c_cpp\">template&lt;class... Ts&gt; struct overloaded : Ts... { using Ts::operator()...; };\r\ntemplate&lt;class... Ts&gt; overloaded(Ts...) -&gt; overloaded&lt;Ts...&gt;;\r\n\r\nstd::variant&lt;std::string, int, bool&gt; v = \"C++ rocks?\";\r\nstd::visit(overloaded {\r\n    [](const std::string&amp; s) { std::cout &lt;&lt; s &lt;&lt; '\\n'; },\r\n    [](int  n) { std::cout &lt;&lt; n &lt;&lt; '\\n'; },\r\n    [](bool b) { std::cout &lt;&lt; b &lt;&lt; '\\n'; }\r\n}, v);<\/pre>\n<p><sup><em><strong>Note:<\/strong> Unlike the C++ version, the error message you get when you accidentally leave out a function to handle one of the types is comprehendable by mere mortals. <a href=\"https:\/\/run.dlang.io\/is\/ROA9Ac\">Check it out for yourself<\/a>.<\/em><\/sup><\/p>\n<p>As a bonus, our D example looks very similar to the built-in pattern matching syntax that you find in many up-and-coming languages that take inspiration from their functional forebears, but is implemented completely <em>in user code<\/em>.<\/p>\n<p>That\u2019s powerful.<\/p>\n<h2 id=\"otherconsiderations\">Other considerations<\/h2>\n<p>As my final point &#8211; if you\u2019ll indulge me for a moment &#8211; I\u2019d like to argue with a strawman C++ programmer of my own creation. In his article, Mr. Kline also mentions the new <a href=\"http:\/\/en.cppreference.com\/w\/cpp\/language\/if\">if constexpr<\/a> feature added in C++17 (which of course, D has had for over a decade now). I\u2019d like to forestall arguments from my strawman friend such as:<\/p>\n<blockquote><p>But you\u2019re cheating! You can use the new <code>if constexpr<\/code> to simplify the code and cut out <code>make_visitor<\/code> entirely, just like in your D example!<\/p><\/blockquote>\n<p>Yes, you <em>could<\/em> use <code>if constexpr<\/code> (and by the same token, <code>static if<\/code> in D), but you shouldn\u2019t (and Mr. Kline explicitly rejects using it in his article).There are a few problems with this approach which make it all-around inferior in both C++ and D. One, this method is error prone and inflexible in the case where you need to add a new type to your <code>variant<\/code>\/<code>Algebraic<\/code>. Your old code will still compile but will now be <em>wrong<\/em>. Two, doing it this way is uglier and more complicated than just passing functions to <code>visit<\/code> directly (at least, it is in D). Three, the D version would <em>still<\/em> blow C++ out of the water on readability. Consider:<\/p>\n<pre class=\"prettyprint lang-d\">\/\/D\r\nv.visit!((arg) {\r\n    alias T = Unqual!(typeof(arg)); \/\/Remove const, shared, etc.\r\n\r\n    static if (is(T == string)) {\r\n        writeln(\"string: \", arg);\r\n    }\r\n    else static if (is(T == int)) {\r\n        writeln(\"int: \", arg);\r\n    }\r\n    else static if (is(T == bool)) {\r\n        writeln(\"bool: \", arg);\r\n    }\r\n});<\/pre>\n<p>vs.<\/p>\n<pre class=\"prettyprint lang-c_cpp\">\/\/C++\r\nvisit([](auto&amp; arg) {\r\n    using T = std::decay_t&lt;decltype(arg)&gt;;\r\n\r\n    if constexpr (std::is_same_v&lt;T, string&gt;) {\r\n        printf(\"string: %s\\n\", arg.c_str());\r\n    }\r\n    else if constexpr (std::is_same_v&lt;T, int&gt;) {\r\n        printf(\"integer: %d\\n\", arg);\r\n    }\r\n    else if constexpr (std::is_same_v&lt;T, bool&gt;) {\r\n        printf(\"bool: %d\\n\", arg);\r\n    }\r\n}, v);<\/pre>\n<p>Which version of the code would <em>you<\/em> want to have to read, understand, and modify? For me, it\u2019s the D version &#8211; no contest.<\/p>\n<hr \/>\n<p>If this article has whet your appetite and you want to find out more about D, you can visit the <em>official<\/em> <a href=\"https:\/\/dlang.org\/\">Dlang website<\/a>, and join us on the <a href=\"https:\/\/forum.dlang.org\/\">mailing list<\/a> or #D on IRC at freenode.net. D is a community-driven project, so we\u2019re also always looking for people eager to jump in and get their hands dirty &#8211; <a href=\"https:\/\/github.com\/dlang\">pull requests welcome!<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I recently read a great article by Matt Kline on how std::visit is everything wrong with modern C++.  I was dubious that std::visit could be much harder to use than std.variant.visit, if at all. For the record, my intuition was completely and utterly wrong.<\/p>\n","protected":false},"author":28,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[26,10],"tags":[],"_links":{"self":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1507"}],"collection":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/users\/28"}],"replies":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/comments?post=1507"}],"version-history":[{"count":9,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1507\/revisions"}],"predecessor-version":[{"id":1520,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1507\/revisions\/1520"}],"wp:attachment":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/media?parent=1507"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/categories?post=1507"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/tags?post=1507"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}