{"id":1950,"date":"2019-02-10T14:12:39","date_gmt":"2019-02-10T14:12:39","guid":{"rendered":"http:\/\/dlang.org\/blog\/?p=1950"},"modified":"2021-10-08T11:03:36","modified_gmt":"2021-10-08T11:03:36","slug":"writing-a-d-wrapper-for-a-c-library","status":"publish","type":"post","link":"https:\/\/dlang.org\/blog\/2019\/02\/10\/writing-a-d-wrapper-for-a-c-library\/","title":{"rendered":"Writing a D Wrapper for a C Library"},"content":{"rendered":"

\"\"In porting to D a program I created for a research project, I wrote a D wrapper of a C library in an object-oriented manner. I want to share my experience with other programmers. This article provides some D tips and tricks for writers of D wrappers around C libraries.<\/p>\n

I initially started my research project<\/a> using the Ada 2012 programming language<\/a> (see my article \u201cExperiences on Writing Ada Bindings for a C Library\u201d in Ada User Journal, Volume 39, Number 1, March 2018<\/a>). Due to a number of bugs that I was unable to overcome, I started looking for another programming language. After some unsatisfying experiments with Java and Python, I settled on the D programming language.<\/p>\n

The C Library<\/h3>\n

We have a C library, written in an object-oriented style (C structure pointers serve as objects, and C functions taking such structure pointers serve as methods). Fortunately for us, there is no inheritance in that C library.<\/p>\n

The particular libraries we will deal with are the Redland RDF Libraries<\/a>, a set of libraries which parse Resource Description Framework (RDF)<\/a> files or other RDF resources, manages them, enables RDF queries, etc. Don\u2019t worry if you don\u2019t know what RDF is, it is not really relevant for this article.<\/p>\n

The first stage of this project was to write a D wrapper over librdf. I modeled it on the Ada wrapper I had already written. One advantage I found in D over Ada is that template instantiation is easier\u2014there\u2019s no need in D to instantiate every single template invocation with a separate declaration. I expect this to substantially simplify the code of XML Boiler, my program which uses this library.<\/p>\n

I wrote both raw bindings and a wrapper. The bindings translate the C declarations directly into D, and the wrapper is a new API which is a full-fledged D interface. For example, it uses D types with constructors and destructors to represent objects. It also uses some other D features which are not available in C. This is a work in progress and your comments are welcome.<\/p>\n

The source code of my library (forked from Dave Beckett\u2019s original multi-language bindings of his libraries) is available at GitHub<\/a> (currently only in the dlang<\/code> branch). Initially, I tried some automatic parsers of C headers which generate D code. I found these unsatisfactory, so I wrote the necessary bindings myself.<\/p>\n

Package structure<\/h3>\n

I put my entire API into the rdf.*<\/code> package hierarchy. I also have the rdf.auxiliary<\/code> package and its subpackages for things used by or with my bindings. I will discuss some particular rdf.auxiliary.*<\/code> packages below.<\/p>\n

My mixins<\/h3>\n

In Ada I used tagged types, which are a rough equivalent of D classes, and derived _With_Finalization<\/code> types from _Without_Finalization<\/code> types (see below). However, tagged types increase variable sizes and execution time.<\/p>\n

In D I use structs instead of classes, mainly for efficiency reasons. D structs do not support inheritance, and therefore have no virtual method table (vtable)<\/a>, but do provide constructors and destructors, making classes unnecessary for my use case (however, see below). To simulate inheritance, I use template mixins<\/a> (defined in the rdf.auxiliary.handled_record<\/code> module) and the alias this<\/code> construct<\/a>.<\/p>\n

As I\u2019ve said above, C objects are pointers to structures. All C pointers to structures have the same format and alignment (ISO\/IEC 9899:2011 section 6.2.5 paragraph 28). This allows the representation of any pointer to a C structure as a pointer to an opaque struct (in the below example, URIHandle<\/code> is an opaque struct declared as struct URIHandle;<\/code>).<\/p>\n

Using the mixins shown below, we can declare the public structs of our API this way (you should look into the actual source for real examples):<\/p>\n

struct URIWithoutFinalize {\r\n    mixin WithoutFinalize!(URIHandle,\r\n                           URIWithoutFinalize,\r\n                           URI,\r\n                           raptor_uri_copy);\r\n    \/\/ \u2026\r\n}\r\nstruct URI {\r\n    mixin WithFinalize!(URIHandle,\r\n                        URIWithoutFinalize,\r\n                        URI,\r\n                        raptor_free_uri);\r\n}<\/pre>\n
The difference between the WithoutFinalize<\/code> and WithFinalize<\/code> mixins is explained below.<\/p>\n
About finalization and related stuff<\/h3>\n
The main challenge in writing object-oriented bindings for a C library is finalization.<\/p>\n
In the C library in consideration (as well as in many other C libraries), every object is represented as a pointer to a dynamically allocated C structure. The corresponding D object can be a struct holding the pointer (aka handle), but oftentimes a C function returns a so-called \u201cshared handle\u201d\u2014a pointer to a C struct which we should not free because it is a part of a larger C object and shall be freed by the C library only when that larger C object goes away.<\/p>\n
As such, I first define both (for example) URIWithoutFinalize<\/code> and URI<\/code>. Only URI<\/code> has a destructor. For URIWithoutFinalize<\/code>, a shared handle is not finalized. As D does not support inheritance for structs, I do it with template mixins instead. Below is a partial listing. See the above URI<\/code> example on how to use them:<\/p>\n
mixin template WithoutFinalize(alias Dummy,\r\n                               alias _WithoutFinalize,\r\n                               alias _WithFinalize,\r\n                               alias copier = null)\r\n{\r\n    private Dummy* ptr;\r\n    private this(Dummy* ptr) {\r\n        this.ptr = ptr;\r\n    }\r\n    @property Dummy* handle() const {\r\n        return cast(Dummy*)ptr;\r\n    }\r\n    static _WithoutFinalize fromHandle(const Dummy* ptr) {\r\n        return _WithoutFinalize(cast(Dummy*)ptr);\r\n    }\r\n    static if(isCallable!copier) {\r\n        _WithFinalize dup() {\r\n            return _WithFinalize(copier(ptr));\r\n        }\r\n    }\r\n    \/\/ ...\r\n}\r\n\r\n\r\nmixin template WithFinalize(alias Dummy,\r\n                            alias _WithoutFinalize,\r\n                            alias _WithFinalize,\r\n                            alias destructor,\r\n                            alias constructor = null)\r\n{\r\n    private Dummy* ptr;\r\n    @disable this();\r\n    @disable this(this);\r\n    \/\/ Use fromHandle() instead\r\n    private this(Dummy* ptr) {\r\n        this.ptr = ptr;\r\n    }\r\n    ~this() {\r\n        destructor(ptr);\r\n    }\r\n    \/*private*\/ @property _WithoutFinalize base() { \/\/ private does not work in v2.081.2\r\n        return _WithoutFinalize(ptr);\r\n    }\r\n    alias base this;\r\n    @property Dummy* handle() const {\r\n        return cast(Dummy*)ptr;\r\n    }\r\n    static _WithFinalize fromHandle(const Dummy* ptr) {\r\n        return _WithFinalize(cast(Dummy*)ptr);\r\n    }\r\n    \/\/ ...\r\n}<\/pre>\n
I\u2019ve used template alias parameters<\/a> here, which allow a template to be parameterized with more than just types. The Dummy<\/code> argument is the type of the handle instance (usually an opaque struct). The destructor<\/code> and copier<\/code> arguments are self-explanatory. For the usage of the constructor<\/code> argument, see the real source (here it is omitted).<\/p>\n
The _WithoutFinalize<\/code> and _WithFinalize<\/code> template arguments should specify the structs we define, allowing them to reference each other. Note that the alias this<\/code> construct makes _WithoutFinalize<\/code> essentially a base of _WithFinalize<\/code>, allowing us to use all methods and properties of _WithoutFinalize<\/code> in _WithFinalize<\/code>.<\/p>\n
Also note that instances of the _WithoutFinalize<\/code> type may become invalid, i.e. it may contain dangling access values. It seems that there is no easy way to deal with this problem because of the way the C library works. We may not know when an object is destroyed by the C library. Or we may know but be unable to appropriately \u201cexplain\u201d it to the D compiler. Just be careful when using this library not to use objects which are already destroyed.<\/p>\n
Dealing with callbacks<\/h3>\n
To deal with C callbacks (particularly when accepting a void*<\/code> argument for additional data) in an object-oriented way, we need a way to convert between C void<\/code> pointers and D class objects (we pass D objects as C \u201cuser data\u201d pointers). D structs are enough (and are very efficient) to represent C objects like librdf library objects, but for conveniently working with callbacks, classes are more useful because they provide good callback machinery in the form of virtual functions.<\/p>\n
First, the D object, which is passed as a callback parameter to C, should not unexpectedly be moved in memory by the D garbage collector. So I make them descendants of this class:<\/p>\n
class UnmovableObject {\r\n    this() {\r\n        GC.setAttr(cast(void*)this, GC.BlkAttr.NO_MOVE);\r\n    }\r\n}<\/pre>\n
Moreover, I add the property context()<\/code> to pass it as a void*<\/code> pointer to C functions which register callbacks:<\/p>\n
abstract class UserObject : UnmovableObject {\r\n    final @property void* context() const { return cast(void*)this; }\r\n}<\/pre>\n
When we create a callback we need to pass a D object as a C pointer and an extern(C)<\/code> function defined by us as the callback. The callback receives the pointer previously passed by us and in the callback code we should (if we want to stay object-oriented) convert this pointer into a D object pointer.<\/p>\n
What we need is a bijective (\u201cback and forth\u201d) mapping between D pointers and C void*<\/code> pointers. This is trivial in D: just use the cast()<\/code> operator.<\/p>\n
How to do this in practice? The best way to explain is with an example. We will consider how to create an I\/O stream class which uses the C library callbacks to implement it. For example, when the user of our wrapper requests to write some information to a file, our class receives write<\/em> message. To handle this message, our implementation calls our virtual function doWriteBytes()<\/code>, which actually handles the user\u2019s request.<\/p>\n
private immutable DispatcherType Dispatch =\r\n    { version_: 2,\r\n      init: null,\r\n      finish: null,\r\n      write_byte : &raptor_iostream_write_byte_impl,\r\n      write_bytes: &raptor_iostream_write_bytes_impl,\r\n      write_end  : &raptor_iostream_write_end_impl,\r\n      read_bytes : &raptor_iostream_read_bytes_impl,\r\n      read_eof   : &raptor_iostream_read_eof_impl };\r\n\r\n\r\nclass UserIOStream : UserObject {\r\n    IOStream record;\r\n    this(RaptorWorldWithoutFinalize world) {\r\n        IOStreamHandle* handle = raptor_new_iostream_from_handler(world.handle,\r\n                                                                  context,\r\n                                                                  &Dispatch);\r\n        record = IOStream.fromNonnullHandle(handle);\r\n    }\r\n    void doWriteByte(char byte_) {\r\n        if(doWriteBytes(&byte_, 1, 1) != 1)\r\n            throw new IOStreamException();\r\n    }\r\n    abstract int doWriteBytes(char* data, size_t size, size_t count);\r\n    abstract void doWriteEnd();\r\n    abstract size_t doReadBytes(char* data, size_t size, size_t count);\r\n    abstract bool doReadEof();\r\n}<\/pre>\n
And for example:<\/p>\n
int raptor_iostream_write_bytes_impl(void* context, const void* ptr, size_t size, size_t nmemb) {\r\n    try {\r\n        return (cast(UserIOStream)context).doWriteBytes(cast(char*)ptr, size, nmemb);\r\n    }\r\n    catch(Exception) {\r\n        return -1;\r\n    }\r\n}<\/pre>\n
More little things<\/h3>\n
I \u201cencode\u201d C strings (which can be null<\/code>) as a D template instance, Nullable!string<\/code>. If the string is null<\/code>, the holder is empty. However, it is often enough to transform an empty D string into a null<\/code> C string (this can work only if we don\u2019t differentiate between empty and null<\/code> strings). See rdf.auxiliary.nullable_string<\/code> for an actually useful code.<\/p>\n
I would write a lot more advice on how to write D bindings for a C library, but you can just follow my source, which can serve as an example.<\/p>\n
Static if<\/h3>\n
One thing which can be done in D but not in Ada is compile-time comparison via static if<\/code><\/a>. This is a D construct (similar to but more advanced than C conditional preprocessor directives) which allows conditional compilation based on compile-time values. I use static if<\/code> with my custom Version<\/code> type to enable\/disable features of my library depending on the available features of the version of the base C library in use. In the following example, rasqalVersionFeatures<\/code> is a D constant defined in my rdf.config package<\/code>, created by the GNU configure script from the config.d.in<\/code> file.<\/p>\n
static if(Version(rasqalVersionFeatures) >= Version(\"0.9.33\")) {\r\n    private extern extern(C)\r\n    QueryResultsHandle* rasqal_new_query_results_from_string(RasqalWorldHandle* world,\r\n                                                             QueryResultsType type,\r\n                                                             URIHandle* base_uri,\r\n                                                             const char* string,\r\n                                                             size_t string_len);\r\n    static create(RasqalWorldWithoutFinalize world,\r\n                  QueryResultsType type,\r\n                  URITypeWithoutFinalize baseURI,\r\n                  string value)\r\n    {\r\n        return QueryResults.fromNonnullHandle(\r\n            rasqal_new_query_results_from_string(world.handle,\r\n                                                 type,\r\n                                                 baseURI.handle,\r\n                                                 value.ptr, value.length));\r\n    }\r\n}<\/pre>\n
Comparisons<\/h3>\n
Order comparisons between structs can be easily done with this mixin:<\/p>\n
mixin template CompareHandles(alias equal, alias compare) {\r\n    import std.traits;\r\n    bool opEquals(const ref typeof(this) s) const {\r\n        static if(isCallable!equal) {\r\n          return equal(handle, s.handle) != 0;\r\n        } else {\r\n          return compare(handle, s.handle) == 0;\r\n        }\r\n    }\r\n    int opCmp(const ref typeof(this) s) const {\r\n      return compare(handle, s.handle);\r\n    }\r\n}<\/pre>\n
Sadly, this mixin has to be called in both the _WithoutFinalization<\/code> and the _WithFinalization<\/code> structs. I found no solution to write it once.<\/p>\n
Conclusion<\/h3>\n
I\u2019ve found that D is a great language for writing object-oriented wrappers around C libraries. There are some small annoyances like using class wrappers around structs for callbacks, but generally, D wraps up around C well.<\/p>\n
\n
Victor Porton is an open source developer, a math researcher, and a Christian writer. He earns his living as a programmer.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"
In porting to D a program I created for a research project, I wrote a D wrapper of a C library in an object-oriented manner. I want to share my experience with other programmers. This article provides some D tips and tricks for writers of D wrappers around C libraries. I initially started my research […]<\/p>\n","protected":false},"author":32,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[26,29,9,30],"tags":[],"_links":{"self":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1950"}],"collection":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/users\/32"}],"replies":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/comments?post=1950"}],"version-history":[{"count":5,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1950\/revisions"}],"predecessor-version":[{"id":1956,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/1950\/revisions\/1956"}],"wp:attachment":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/media?parent=1950"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/categories?post=1950"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/tags?post=1950"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}