![\"\"](\"https:\/\/dlang.org\/blog\/wp-content\/uploads\/2019\/03\/brain02.png\")
<\/p>\n<\/p>\n
The first entry in this series<\/a> shows how to use the new DIP1000 rules to have slices and pointers refer to the stack, all while being memory safe. The second entry in this series<\/a> teaches about the So far the series has deliberately avoided templates and We will also dig deeper into unsafe code. The previous two posts in this series focused on the Function attribute inference<\/a> means that the language will analyze the body of a function and will automatically add the There are many ways to turn on function attribute inference. One is by omitting the return type in the function signature. Note that the The second way to enable attribute inference is to templatize the function. With our The D spec does not say that a template must have any parameters. An empty parameter list can be used to turn attribute inference on: Another means to turn on attribute inference is to store the function inside another function. These are called nested functions<\/a>. Inference is enabled not only on functions nested directly inside another function but also on most things nested in a type or a template inside the function. Example:<\/p>\n A downside of nested functions is that they can only be used in lexical order (the call site must be below the function declaration) unless both the nested function and the call are inside the same struct, class, union, or template that is in turn inside the parent function. Another downside is that they don’t work with Uniform Function Call Syntax<\/a>.<\/p>\n Finally, attribute inference is always enabled for function literals<\/a> (a.k.a. lambda functions). The halving function would be defined as Aiming for minimal manual typing isn’t always wise. Neither is aiming for maximal attribute bloat.<\/p>\n The primary problem of auto inference is that subtle changes in the code can lead to inferred attributes turning on and off in an uncontrolled manner. To see when it matters, we need to have an idea of what will be inferred and what will not.<\/p>\n The compiler in general will go to great lengths to infer Inference of A D programmer should get into the habit of asking, “What will happen if I mistakenly do something that makes this function unsafe, impure, throwing, garbage-collecting, or escaping?” If the answer is “immediate compiler error”, auto inference is probably fine. On the other hand, the answer could be “user code will break when updating this library I’m maintaining”. In that case, annotate manually.<\/p>\n In addition to the potential of losing attributes the author intends to apply, there is also another risk:<\/p>\n You might think that since the author is manually specifying the attributes, there’s no problem. Unfortunately, that’s wrong. Suppose the author decides to rewrite the function such that all the return values are slices of the Surprise! The parameter Still, for internal functions, auto inference is a great way to save both our fingers when writing and our eyes when reading. Note that it’s perfectly fine to rely on auto inference of the It still sometimes makes sense to manually apply attributes to internal functions, because the error messages generated when they are violated tend to be better with manual attributes.<\/p>\n Auto inference is always enabled for templated functions. What if a library interface needs to expose one? There is a way to block the inference, albeit an ugly one:<\/p>\n However, which attributes a template should have often depends on its compile-time parameters. It would be possible to use metaprogramming to designate attributes depending on the template parameters, but that would be a lot of work, hard to read, and easily as error-prone as relying on auto inference.<\/p>\n It’s more practical to just test that the function template infers the wanted attributes. Such testing doesn’t have to, and probably shouldn’t, be done manually each time the function is changed. Instead:<\/p>\n Thanks to D’s compile-time introspection powers, testing against unwanted attributes is also covered:<\/p>\n If assertion failures are wanted at compile time before the unit tests are run, adding the Let’s not forget that D is a systems programming language. As this series has shown, in most D code the programmer is well protected from memory errors, but D would not be D if it didn’t allow going low-level and bypassing the type system in the same manner as C or C++: bit arithmetic on pointers, writing and reading directly to hardware ports, executing a The difference is that in C and C++ it takes only one mistake to break the type system and cause undefined behavior<\/a> anywhere in the code. A D programmer is only at risk when not in a This is a language-agnostic function that compiles in C, C++, and D. Someone intended to calculate the cube of However, in C and C++, not initializing a local variable means reading it is (sans a few narrow exceptions) undefined behavior<\/em>. This function does not even handle pointers, yet according to the standard, calling this function could just as well have In D, even if you explicitly requested no default initialization with Still, for anyone who cares about memory safety, as they probably should for anything intended for a wide audience, it’s a bad idea to assume that D Some people assume that “Undefined Behavior” simply means “erroneous behavior” or crashing at runtime. While that is often what ultimately happens, undefined behavior is far more dangerous than, say, an uncaught exception or an infinite loop. The difference is that with undefined behavior, you have no guarantees at all about what happens. This might not sound any worse than an infinite loop, but an accidental infinite loop is discovered the first time it’s entered. Code with undefined behavior, on the other hand, might do what was intended when it’s tested, but then do something completely different in production. Even if the code is tested with the same flags it’s compiled with in production, the behavior may change from one compiler version to another, or when making completely unrelated changes to the code. Time for an example:<\/p>\n The idea here is that the function replaces all exceptions in the array with This cast is not a problem, right? Object references are always of the same size regardless of their type, and we’re casting the exceptions to the parent type, Unfortunately, that’s not how the D specification views it<\/a>. Having two class references (or any references, for that matter) in the same memory location but with different types, and then assigning one of them to the other, is undefined behavior. That’s exactly what happens in<\/p>\n if the array does contain the It may seem that requiring a cast to use a function is an obvious warning sign that a good D programmer would not ignore. I wouldn’t be so sure. Casts aren’t that rare even in fine high-level code. Even if you disagree, other cases are provably bad enough to bite anyone. Last summer, this case appeared in the D forums<\/a>:<\/p>\n This problem was encountered by Steven Schveighoffer, a long-time D veteran who has himself lectured about ref<\/code> storage class and how DIP1000 works with aggregate types (classes, structs, and unions).<\/p>\nauto<\/code> functions. This kept the first two posts simpler in that they did not have to deal with function attribute inference, which I have referred to as “attribute auto inference” in earlier posts. However, both auto<\/code> functions and templates are very common in D code, so a series on DIP1000 can’t be complete without explaining how those features work with the language changes. Function attribute inference is our most important tool in avoiding so-called “attribute soup”, where a function is decorated with several attributes, which arguably decreases readability.<\/p>\nscope<\/code> attribute, but this post is more focused on attributes and memory safety in general. Since DIP1000 is ultimately about memory safety, we can’t get around discussing those topics.<\/p>\nAvoiding repetition with attributes<\/h2>\n
@safe<\/code>, pure<\/code>, nothrow<\/code>, and @nogc<\/code> attributes where applicable. It will also attempt to add scope<\/code> or return scope<\/code> attributes to parameters and return ref<\/code> to ref<\/code> parameters that can’t otherwise be compiled. Some attributes are never inferred. For instance, the compiler will not insert any ref<\/code>, lazy<\/code>, out<\/code> or @trusted<\/code> attributes, because very likely they are explicitly not wanted where they are left out.<\/p>\nauto<\/code> keyword is not required for this. auto<\/code> is a placeholder keyword used when no return type, storage class, or attribute is specified. For example, the declaration half(int x) { return x\/2; }<\/code> does not parse, so we use auto half(int x) { return x\/2; }<\/code> instead. But we could just as well write @safe half(int x) { return x\/2; }<\/code> and the rest of the attributes (pure<\/code>, nothrow<\/code>, and @nogc<\/code>) will be inferred just as they are with the auto<\/code> keyword.<\/p>\nhalf<\/code> example, it can be done this way:<\/p>\nint divide(int denominator)(int x) { return x\/denominator; }\nalias half = divide!2;<\/pre>\nint half()(int x) { return x\/2; }<\/code>. Calling this function doesn’t even require the template instantiation syntax at the call site, e.g., half!()(12)<\/code> is not required as half(12)<\/code> will compile.<\/p>\n@safe void parentFun()\n{\n \/\/ This is auto-inferred.\n int half(int x){ return x\/2; }\n\n class NestedType\n {\n \/\/ This is auto inferred\n final int half1(int x) { return x\/2; }\n\n \/\/ This is not auto inferred; it's a\n \/\/ virtual function and the compiler\n \/\/ can't know if it has an unsafe override\n \/\/ in a derived class.\n int half2(int x) { return x\/2; }\n }\n\n int a = half(12); \/\/ Works. Inferred as @safe.\n auto cl = new NestedType;\n int b = cl.half1(18); \/\/ Works. Inferred as @safe.\n int c = cl.half2(26); \/\/ Error.\n}<\/pre>\nenum half = (int x) => x\/2;<\/code> and called exactly as normal. However, the language does not consider this declaration a function. It considers it a function pointer. This means that in global scope it’s important to use enum<\/code> or immutable<\/code> instead of auto<\/code>. Otherwise, the lambda can be changed to something else from anywhere in the program and cannot be accessed from pure<\/code> functions. In rare cases, such mutability can be desirable, but most often it is an antipattern (like global variables in general).<\/p>\nLimits of inference<\/h3>\n
@safe<\/code>, pure<\/code>, nothrow<\/code>, and @nogc<\/code> attributes. If your function can<\/em> have those, it almost always will. The specification says that recursion is an exception: a function calling itself should not be @safe<\/code>, pure<\/code>, or nothrow<\/code> unless explicitly specified as such. But in my testing, I found those attributes actually are inferred for recursive functions. It turns out, there is an ongoing effort to get recursive attribute inference working, and it partially works already.<\/p>\nscope<\/code> and return<\/code> on function parameters is less reliable. In the most mundane cases, it’ll work, but the compiler gives up pretty quickly. The smarter the inference engine is, the more time it takes to compile, so the current design decision is to infer those attributes in only the simplest of cases.<\/p>\nWhere to let the compiler infer?<\/h3>\n
@safe pure nothrow @nogc firstNewline(string from)\n{\n foreach(i; 0 .. from.length) switch(from[i])\n {\n case '\\r':\n if(from.length > i+1 && from[i+1] == '\\n')\n {\n return "\\r\\n";\n }\n else return "\\r";\n\n case '\\n': return "\\n";\n\n default: break;\n }\n\n return "";\n}\n<\/pre>\nfrom<\/code> parameter rather than string literals:<\/p>\n@safe pure nothrow @nogc firstNewline(string from)\n{\n foreach(i; 0 .. from.length) switch(from[i])\n {\n case '\\r':\n if (from.length > i + 1 && from[i + 1] == '\\n')\n {\n return from[i .. i + 2];\n }\n else return from[i .. i + 1];\n\n case '\\n': return from[i .. i + 1];\n\n default: break;\n }\n\n return "";\n}<\/pre>\nfrom<\/code> was previously inferred as scope<\/code>, and a library user was relying on that, but now it’s inferred as return scope<\/code> instead, breaking client code.<\/p>\n@safe<\/code> attribute as long as the function is used in explicitly in @safe<\/code> functions or unit tests. If something potentially unsafe is done inside the auto-inferred function, it gets inferred as @system<\/code>, not @trusted<\/code>. Calling a @system<\/code> function from a @safe<\/code> function results in a compiler error, meaning auto inference is safe to rely on in this case.<\/p>\nWhat about templates?<\/h3>\n
private template FunContainer(T)\n{\n \/\/ Not auto inferred\n \/\/ (only eponymous template functions are)\n @safe T fun(T arg){return arg + 3;}\n}\n\n\/\/ Auto inferred per se, but since the function it calls\n\/\/ is not, only @safe is inferred.\nauto addThree(T)(T arg){return FunContainer!T.fun(arg);}<\/pre>\nfloat multiplyResults(alias fun)(float[] arr)\n if (is(typeof(fun(new float)) : float))\n{\n float result = 1.0f;\n foreach (ref e; arr) result *= fun(&e);\n return result;\n}\n\n@safe pure nothrow unittest\n{\n float fun(float* x){return *x+1;}\n \/\/ Using a static array makes sure\n \/\/ arr argument is inferred as scope or\n \/\/ return scope\n float[5] elements = [1.0f, 2.0f, 3.0f, 4.0f, 5.0f];\n\n \/\/ No need to actually do anything with\n \/\/ the result. The idea is that since this\n \/\/ compiles, multiplyResults is proven @safe\n \/\/ pure nothrow, and its argument is scope or\n \/\/ return scope.\n multiplyResults!fun(elements);\n}<\/pre>\n@safe unittest\n{\n import std.traits : attr = FunctionAttribute,\n functionAttributes, isSafe;\n\n float fun(float* x)\n {\n \/\/ Makes the function both throwing\n \/\/ and garbage collector dependant.\n if (*x > 5) throw new Exception("");\n static float* impureVar;\n\n \/\/ Makes the function impure.\n auto result = impureVar? *impureVar: 5;\n\n \/\/ Makes the argument unscoped.\n impureVar = x;\n return result;\n }\n\n enum attrs = functionAttributes!(multiplyResults!fun);\n\n assert(!(attrs & attr.nothrow_));\n assert(!(attrs & attr.nogc));\n\n \/\/ Checks against accepting scope arguments.\n \/\/ Note that this check does not work with\n \/\/ @system functions.\n assert(!isSafe!(\n {\n float[5] stackFloats;\n multiplyResults!fun(stackFloats[]);\n }));\n\n \/\/ It's a good idea to do positive tests with\n \/\/ similar methods to make sure the tests above\n \/\/ would fail if the tested function had the\n \/\/ wrong attributes.\n assert(attrs & attr.safe);\n assert(isSafe!(\n {\n float[] heapFloats;\n multiplyResults!fun(heapFloats[]);\n }));\n}<\/pre>\nstatic<\/code> keyword before each of those assert<\/code>s will get the job done. Those compiler errors can even be had in non-unittest builds by converting that unit test to a regular function, e.g., by replacing @safe unittest<\/code> with, say, private @safe testAttrs()<\/code>.<\/p>\nThe live fire exercise: @system<\/h2>\n
struct<\/code> destructor on a raw byte blob… D is designed to do all of that.<\/p>\n@safe<\/code> function, or when using dangerous compiler switches such as -release<\/code> or -check=assert=off<\/code> (failing a disabled assertion is undefined behavior), and even then the semantics tend to be less UB-prone. For example:<\/p>\nfloat cube(float arg)\n{\n float result;\n result *= arg;\n result *= arg;\n return result;\n}<\/pre>\narg<\/code> but forgot to initialize result<\/code> with arg<\/code>. In D, nothing dangerous happens despite this being a @system<\/code> function. No initialization value means result<\/code> is default initialized to NaN<\/code> (not-a-number), which leads to the result also being NaN<\/code>, which is a glaringly obvious “error” value when using this function the first time.<\/p>\n*(int*) rand() = 0XDEADBEEF;<\/code> in it, all due to a trivial mistake. While many compilers with enabled warnings will catch this one, not all do, and these languages are full of similar examples where even warnings don’t help.<\/p>\nfloat result = void<\/code>, it’d just mean the return value of the function is undefined, not anything and everything that happens if the function is called. Consequently, that function could be annotated @safe<\/code> even with such an initializer.<\/p>\n@system<\/code> code is safe enough to be the default mode. Two examples will demonstrate what can happen.<\/p>\nWhat undefined behavior can do<\/h3>\n
\/\/ return whether the exception itself is in the array\nbool replaceExceptions(Object[] arr, ref Exception e)\n{\n bool result;\n foreach (ref o; arr)\n {\n if (&o is &e) result = true;\n if (cast(Exception) o) o = e;\n }\n\n return result;\n}<\/pre>\ne<\/code>. If e<\/code> itself is in the array, it returns true<\/code>, otherwise false<\/code>. And indeed, testing confirms it works. The function is used like this:<\/p>\nauto arr = [new Exception("a"), null, null, new Exception("c")];\nauto result = replaceExceptions\n(\n cast(Object[]) arr,\n arr[3]\n);<\/pre>\nObject<\/code>. It’s not like the array contains anything other than object references.<\/p>\nif (cast(Exception) o) o = e;<\/pre>\n
e<\/code> argument. Since true<\/code> can only be returned when undefined behavior is triggered, it means that any compiler would be free to optimize replaceExceptions<\/code> to always return false<\/code>. This is a dormant bug no amount of testing will find, but that might, years later, completely mess up the application when compiled with the powerful optimizations of an advanced compiler.<\/p>\nstring foo(in string s)\n{\n return s;\n}\n\nvoid main()\n{\n import std.stdio;\n string[] result;\n foreach(c; "hello")\n {\n result ~= foo([c]);\n }\n writeln(result);\n}<\/pre>\n@safe<\/code> and @system<\/code><\/a> on more than one occasion<\/a>. Anything that can burn him can burn any of us.<\/p>\n
![\"\"](\"https:\/\/dlang.org\/blog\/wp-content\/uploads\/2023\/01\/unsafe-safe.jpg\")
<\/p>\n