![\"\"](\"https:\/\/dlang.org\/blog\/wp-content\/uploads\/2019\/03\/brain02.png\")
<\/p>\n<\/p>\n
The previous entry in this series<\/a> shows how to use the new DIP1000 rules to have slices and pointers refer to the stack, all while being memory safe. But D can refer to the stack in other ways, too, and that’s the topic of this article.<\/p>\n In Part 1, I said that if you understand how DIP1000 works with pointers, then you understand how it works with classes. An example is worth more than mere words:<\/p>\n The The principle is: if the From the point of view of regular functions, that’s all there is to it. What about member functions of a class or an interface? This is how it’s done:<\/p>\n Alternatively one could replace the The Liskov substitution principle<\/a> states, in simplified terms, that an inherited function can give the caller more guarantees than its parent function, but never fewer. DIP1000-related attributes fall in that category. The rule works like this:<\/p>\n If there is no attribute, the caller can not assume anything; the function might store the address of the argument somewhere. If We still have not uncovered how to use They don’t work like just any pointer. Once you understand The simplest possible way to use What does this mean? instead of …compiles, but crashes at runtime because the assignment inside The address of a There is a With a non- It is also possible to have a returnable Here we return the address of the argument passed to The major difference to classes is that where the Note that Note that we did not use the Part One said that We almost know all there is to know about using structs, unions, and classes with DIP1000. We have two final things to learn today.<\/p>\n But before that, a short digression regarding the Back to the topic. The first thing is not really specific to structs or classes. We discussed what The usual meaning of the Since The second thing to learn is that This feature requires using the This is pretty much all that there is to manual DIP1000 usage. But this blog series shall not be over yet! DIP1000 is not intended to always be used explicitly—it works with attribute inference. That’s what the next post will cover.<\/p>\n It will also cover some considerations when daring to use Thanks to Walter Bright and Dennis Korpel for reviewing this article<\/em><\/p>\n","protected":false},"excerpt":{"rendered":" DIP1000: Memory Safety in a Modern System Programming Language Pt. 2 The previous entry in this series shows how to use the new DIP1000 rules to have slices and pointers refer to the stack, all while being memory safe. But D can refer to the stack in other ways, too, and that’s the topic of […]<\/p>\n","protected":false},"author":48,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[26,39,9,20,30],"tags":[],"_links":{"self":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/3116"}],"collection":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/users\/48"}],"replies":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/comments?post=3116"}],"version-history":[{"count":5,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/3116\/revisions"}],"predecessor-version":[{"id":3126,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/3116\/revisions\/3126"}],"wp:attachment":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/media?parent=3116"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/categories?post=3116"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/tags?post=3116"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Object-oriented instances are the easiest case<\/h2>\n
@safe Object ifNull(return scope Object a, return scope Object b)\n{\n return a? a: b;\n}<\/pre>\nreturn scope<\/code> in the above example works exactly as it does in the following:<\/p>\n@safe int* ifNull(return scope int* a, return scope int* b)\n{\n return a? a: b;\n}<\/pre>\nscope<\/code> or return scope<\/code> storage class is applied to an object in a parameter list, the address of the object instance is protected just as if the parameter were a pointer to the instance. From the perspective of machine code, it is<\/strong> a pointer to the instance.<\/p>\ninterface Talkative\n{\n @safe const(char)[] saySomething() scope;\n}\n\nclass Duck : Talkative\n{\n char[8] favoriteWord;\n @safe const(char)[] saySomething() scope\n {\n import std.random : dice;\n\n \/\/ This wouldn't work\n \/\/ return favoriteWord[];\n\n \/\/ This does\n return favoriteWord[].dup;\n\n \/\/ Also returning something totally\n \/\/ different works. This\n \/\/ returns the first entry 40% of the time,\n \/\/ The second entry 40% of the time, and\n \/\/ the third entry the rest of the time.\n return\n [\n "quack!",\n "Quack!!",\n "QUAAACK!!!"\n ][dice(2,2,1)];\n }\n}<\/pre>\nscope<\/code> positioned either before or after the member function name marks the this<\/code> reference as scope<\/code>, preventing it from leaking out of the function. Because the address of the instance is protected, nothing that refers directly to the address of the fields is allowed to escape either. That’s why return favoriteWord[]<\/code> is disallowed; it’s a static array stored inside the class instance, so the returned slice would refer directly to it. favoriteWord[].dup<\/code> on the other hand returns a copy of the data that isn’t located in the class instance, which is why it’s okay.<\/p>\nscope<\/code> attributes of both Talkative.saySomething<\/code> and Duck.saySomething<\/code> with return scope<\/code>, allowing the return of favoriteWord<\/code> without duplication.<\/p>\nDIP1000 and Liskov Substitution Principle<\/h3>\n
\n
this<\/code> reference) in the parent functions has no DIP1000 attributes, the child function may designate it scope<\/code> or return scope<\/code><\/li>\nscope<\/code> in the parent, it must be designated scope<\/code> in the child<\/li>\nreturn scope<\/code> in the parent, it must be either scope<\/code> or return scope<\/code> in the child<\/li>\n<\/ul>\nreturn scope<\/code> is present, the caller can assume the address of the argument is not stored other than in the return value. With scope<\/code>, the guarantee is that the address is not stored anywhere, which is an even stronger guarantee. Example:<\/p>\nclass C1\n{ double*[] incomeLog;\n @safe double* imposeTax(double* pIncome)\n {\n incomeLog ~= pIncome;\n return new double(*pIncome * .15);\n }\n}\n\nclass C2 : C1\n{\n \/\/ Okay from language perspective (but maybe not fair\n \/\/ for the taxpayer)\n override @safe double* imposeTax\n (return scope double* pIncome)\n {\n return pIncome;\n }\n}\n\nclass C3 : C2\n{\n \/\/ Also okay.\n override @safe double* imposeTax\n (scope double* pIncome)\n {\n return new double(*pIncome * .18);\n }\n}\n\nclass C4: C3\n{\n \/\/ Not okay. The pIncome parameter of C3.imposeTax\n \/\/ is scope, and this tries to relax the restriction.\n override @safe double* imposeTax\n (double* pIncome)\n {\n incomeLog ~= pIncome;\n return new double(*pIncome * .16);\n }\n}<\/pre>\nThe special pointer,
ref<\/code><\/h2>\nstruct<\/code>s and union<\/code>s with DIP1000. Well, obviously we’ve uncovered pointers and arrays. When referring to a struct<\/code> or a union<\/code>, they work the same as they do when referring to any other type. But pointers and arrays are not the canonical way to use structs in D. They are most often passed around by value, or by reference when bound to ref<\/code> parameters<\/a>. Now is a good time to explain how ref<\/code> works with DIP1000.<\/p>\nref<\/code>, you can use DIP1000 in many ways you otherwise could not.<\/p>\nA simple
ref int<\/code> parameter<\/h3>\nref<\/code> is probably this:<\/p>\n@safe void fun(ref int arg) {\n arg = 5;\n}<\/pre>\nref<\/code> is internally a pointer—think int* pArg<\/code>—but is used like a value in the source code. arg = 5<\/code> works internally like *pArg = 5<\/code>. Also, the client calls the function as if the argument were passed by value:<\/p>\nauto anArray = [1,2];\nfun(anArray[1]); \/\/ or, via UFCS: anArray[1].fun;\n\/\/ anArray is now [1, 5]<\/pre>\n
fun(&anArray[1])<\/code>. Unlike C++ references<\/a>, D ref<\/code>erences can be null<\/code>, but the application will instantly terminate with a segmentation fault if a null ref<\/code> is used for something other than reading the address with the &<\/code> operator. So this:<\/p>\nint* ptr = null;\nfun(*ptr);<\/pre>\n
fun<\/code> lands at the null address.<\/p>\nref<\/code> variable is always guarded against escape. In this sense @safe void fun(ref int arg){arg = 5;}<\/code> is like @safe void fun(scope int* pArg){*pArg = 5;}<\/code>. For example, @safe int* fun(ref int arg){return &arg;}<\/code> will not compile, just like @safe int* fun(scope int* pArg){return pArg;}<\/code> will not.<\/p>\nreturn ref<\/code> storage class, however, that allows returning the address of the parameter but no other form of escape, just like return scope<\/code>. This means that @safe int* fun(return ref int arg){return &arg;}<\/code> works.<\/p>\nref<\/code>erence to a reference<\/h3>\nref<\/code>erence to an int<\/code> or similar type already allows much nicer syntax than one can get with pointers. But the real power of ref<\/code> shows when it refers to a type that is a reference itself—a pointer or a class, for instance. scope<\/code> or return scope<\/code> can be applied to a reference that is referenced to by ref<\/code>. For example:<\/p>\n@safe float[] mergeSort(ref return scope float[] arr)\n{\n import std.algorithm: merge;\n import std.array : Appender;\n\n if(arr.length < 2) return arr;\n\n auto firstHalf = arr[0 .. $\/2];\n auto secondHalf = arr[$\/2 .. $];\n\n Appender!(float[]) output;\n output.reserve(arr.length);\n\n foreach\n (\n el;\n firstHalf.mergeSort\n .merge!floatLess(secondHalf.mergeSort)\n ) output ~= el;\n\n arr = output[];\n return arr;\n}\n\n@safe bool floatLess(float a, float b)\n{\n import std.math: isNaN;\n\n return a.isNaN? false:\n b.isNaN? true:\n a<b;\n}<\/pre>\nmergeSort<\/code> here guarantees it won’t leak the address of the float<\/code>s in arr<\/code> except in the return value. This is the same guarantee that would be had from a return scope float[] arr<\/code> parameter. But at the same time, because arr<\/code> is a ref<\/code> parameter, mergeSort<\/code> can mutate the array passed to it. Then the client can write:<\/p>\nfloat[] values = [5, 1.5, 0, 19, 1.5, 1];\nvalues.mergeSort;<\/pre>\n
ref<\/code> argument, the client would have to write values = values.sort<\/code> instead (not using ref<\/code> would be a perfectly reasonable API in this case, because we do not always want to mutate the original array). This is something that cannot be accomplished with pointers, because return scope float[]* arr<\/code> would protect the address of the array’s metadata (the length<\/code> and ptr<\/code> fields of the array), not the address of it’s contents.<\/p>\nref<\/code> argument to a scope<\/code> reference. Since this example has a unit test, remember to use the -unittest<\/code> compile flag to include it in the compiled binary.<\/p>\n@safe ref Exception nullify(return ref scope Exception obj)\n{\n obj = null;\n return obj;\n}\n\n@safe unittest\n{\n scope obj = new Exception("Error!");\n assert(obj.msg == "Error!");\n obj.nullify;\n assert(obj is null);\n \/\/ Since nullify returns by ref, we can assign\n \/\/ to it's return value.\n obj.nullify = new Exception("Fail!");\n assert(obj.msg == "Fail!");\n}<\/pre>\nnullify<\/code>, but still guard both the address of the object pointer and the address of the class instance against being leaked by other channels.<\/p>\nreturn<\/code> is a free keyword that does not mandate ref<\/code> or scope<\/code> to follow it. What does void* fun(ref scope return int*)<\/code> mean then? The spec states<\/a> that return<\/code> without a trailing scope<\/code> is always treated as ref return<\/code>. This example thus is equivalent to void* fun(return ref scope int*)<\/code>. However, this only applies if there is ref<\/code>erence to bind to. Writing void* fun(scope return int*)<\/code> means void* fun(return scope int*)<\/code>. It’s even possible to write void* fun(return int*)<\/code> with the latter meaning, but I leave it up to you to decide whether this qualifies as conciseness or obfuscation.<\/p>\nMember functions and
ref<\/code><\/h2>\nref<\/code> and return ref<\/code> often require careful consideration to keep track of which address is protected and what can be returned. It takes some experience to get confortable with them. But once you do, understanding how struct<\/code>s and union<\/code>s work with DIP1000 is pretty straightforward.<\/p>\nthis<\/code> reference is just a regular class reference in class member functions, this<\/code> in a struct or union member function is ref StructOrUnionName<\/code>.<\/p>\nunion Uni\n{\n int asInt;\n char[4] asCharArr;\n\n \/\/ Return value contains a reference to\n \/\/ this union, won't escape references\n \/\/ to it via any other channel\n @safe char[] latterHalf() return\n {\n return asCharArr[2 .. $];\n }\n\n \/\/ This argument is implicitly ref, so the\n \/\/ following means the return value does\n \/\/ not refer to this union, and also that\n \/\/ we don't leak it in any other way.\n @safe char[] latterHalfCopy()\n {\n return latterHalf.dup;\n }\n}<\/pre>\nreturn ref<\/code> should not be used with the this<\/code> argument. char[] latterHalf() return ref<\/code> fails to parse. The language already has to understand what ref char[] latterHalf() return<\/code> means: the return value is a ref<\/code>erence. The “ref” in return ref<\/code> would be redundant anyway.<\/p>\nscope<\/code> keyword here. scope<\/code> would be meaningless with this union, because it does not contain references to anything. Just like it is meaningless to have a scope ref int<\/code>, or a scope int<\/code> function argument. scope<\/code> makes sense only for types that refer to memory elsewhere.<\/p>\nscope<\/code> in a struct<\/code> or union<\/code> means the same thing as it means in a static array. It means that the memory its members refer to cannot be escaped. Example:<\/p>\nstruct CString\n{\n \/\/ We need to put the pointer in an anonymous\n \/\/ union with a dummy member, otherwise @safe user\n \/\/ code could assign ptr to point to a character\n \/\/ not in a C string.\n union\n {\n \/\/ Empty string literals get optimised to null pointers by D\n \/\/ compiler, we have to do this for the .init value to really point to\n \/\/ a '\\0'.\n immutable(char)* ptr = &nullChar;\n size_t dummy;\n }\n\n \/\/ In constructors, the "return value" is the\n \/\/ constructed data object. Thus, the return scope\n \/\/ here makes sure this struct won't live longer\n \/\/ than the memory in arr.\n @trusted this(return scope string arr)\n {\n \/\/ Note: Normal assert would not do! They may be\n \/\/ removed from release builds, but this assert\n \/\/ is necessary for memory safety so we need\n \/\/ to use assert(0) instead which never gets\n \/\/ removed.\n if(arr[$-1] != '\\0') assert(0, "not a C string!");\n ptr = arr.ptr;\n }\n\n \/\/ The return value refers to the same memory as the\n \/\/ members in this struct, but we don't leak references\n \/\/ to it via any other way, so return scope.\n @trusted ref immutable(char) front() return scope\n {\n return *ptr;\n }\n\n \/\/ No references to the pointed-to array passed\n \/\/ anywhere.\n @trusted void popFront() scope\n {\n \/\/ Otherwise the user could pop past the\n \/\/ end of the string and then read it!\n if(empty) assert(0, "out of bounds!");\n ptr++;\n }\n\n \/\/ Same.\n @safe bool empty() scope\n {\n return front == '\\0';\n }\n}\n\nimmutable nullChar = '\\0';\n\n@safe unittest\n{\n import std.array : staticArray;\n\n auto localStr = "hello world!\\0".staticArray;\n auto localCStr = localStr.CString;\n assert(localCStr.front == 'h');\n\n static immutable(char)* staticPtr;\n\n \/\/ Error, escaping reference to local.\n \/\/ staticPtr = &localCStr.front();\n\n \/\/ Fine.\n staticPtr = &CString("global\\0").front();\n\n localCStr.popFront;\n assert(localCStr.front == 'e');\n assert(!localCStr.empty);\n}\n<\/pre>\n@trusted<\/code> is a terrible footgun with DIP1000. This example demonstrates why. Imagine how easy it’d be to use a regular assert or forget about them totally, or overlook the need to use the anonymous union. I think<\/em> this struct is safe to use, but it’s entirely possible I overlooked something.<\/p>\nFinally<\/h2>\n
scope<\/code> keyword. It is not used for just annotating parameters and local variables as illustrated. It is also used for scope classes<\/a> and scope guard statements<\/a>. This guide won’t be discussing those, because the former feature is deprecated, and the latter is not related to DIP1000 or control of variable lifetimes. The point of mentioning them is to dispel a potential misconception that scope<\/code> always means limiting the lifetime of something. Learning about scope guard statements is still a good idea, as it’s a useful feature.<\/p>\nreturn<\/code>, return ref<\/code>, and return scope<\/code> usually mean, but there’s an alternative meaning to them. Consider:<\/p>\n@safe void getFirstSpace\n(\n ref scope string result,\n return scope string where\n)\n{\n \/\/...\n}<\/pre>\nreturn<\/code> attribute makes no sense here, as the function has a void<\/code> return type. A special rule applies in this case: if the return type is void<\/code>, and the first argument is ref<\/code> or out<\/code>, any subsequent return<\/code> [ref<\/code>\/scope<\/code>] is assumed to be escaped by assigning to the first argument. With struct member functions, they are assumed to be assigned to the struct itself.<\/p>\n@safe unittest\n{\n static string output;\n immutable(char)[8] input = "on stack";\n \/\/Trying to assign stack contents to a static\n \/\/variable. Won't compile.\n getFirstSpace(output, input);\n}<\/pre>\nout<\/code> came up, it should be said it would be a better choice for result<\/code> here than ref<\/code>. out<\/code> works like ref<\/code>, with the one difference that the referenced data is automatically default-initialized at the beginning of the function, meaning any data to which the out<\/code> parameter refers is guaranteed to not affect the function.<\/p>\nscope<\/code> is used by the compiler to optimize class allocations inside function bodies. If a new<\/code> class is used to initialize a scope<\/code> variable, the compiler can put it on the stack. Example:<\/p>\nclass C{int a, b, c;}\n@safe @nogc unittest\n{\n \/\/ Since this unittest is @nogc, this wouldn't\n \/\/ compile without the scope optimization.\n scope C c = new C();\n}<\/pre>\nscope<\/code> keyword explicitly. Inference of scope<\/code> does not work, because initializing a class this way does not normally (meaning, without the @nogc<\/code> attribute) mandate limiting the lifetime of c<\/code>. The feature currently works only with classes, but there is no reason it couldn’t work with new<\/code>ed struct pointers and array literals too.<\/p>\nUntil next time<\/h3>\n
@trusted<\/code> and @system<\/code> code. The need for dangerous systems programming exists and is part of the D language domain. But even systems programming is a responsible affair when people do what they can to minimize risks. We will see that even there it’s possible to do a lot.<\/p>\n