![\"\"](\"https:\/\/dlang.org\/blog\/wp-content\/uploads\/2019\/03\/brain02.png\")
<\/p>\n<\/p>\n
I\u2019ve used the D programming language to implement a high-frequency trading (HFT) platform. I\u2019ve been quite satisfied with the experience and thought I\u2019d share how I got here. It wasn\u2019t a direct path.<\/p>\n
In 2008, I stumbled across a book on Amazon called Learn to Tango with D<\/a>. That grabbed my curiosity, so I decided to research D further. That led me to Digital Mars and Walter Bright. I had first heard of Walter when I learned about Zortech C++, the first native C++ compiler<\/a>. His work had been a huge influence on my C++ learning experience. So I was immediately interested in the language just because it was his, and excited to learn that he was working with Andrei Alexandrescu on version 2. Still, I decided to wait until they were further along with the new version before I dove in.<\/p>\n In 2010, I bought Andrei\u2019s The D Programming Language<\/a> as soon as it was published and started reading. At the time, I was working at BNP Paribas<\/a> using C++ to optimize their HFT platform, so high performance was prevalent in my thoughts. When I saw that D\u2019s classes were reference types, with functions that are virtual by default, I was disappointed. I didn\u2019t see how this could be useful for low-latency programming. I became too busy with work to explore further at the time, so I put the book and the language aside.<\/p>\n In 2014, I began preparing for a new adventure. As part of that, I started working on a feed handler framework from scratch in C++, using my own long-maintained C++ library of low-level components useful in low-latency, high-performance applications. Andrei\u2019s book came to my attention again, so I decided to give it another look.<\/p>\n This time, I read the book through to the end and learned that my initial impression had been misplaced. I found that I liked D\u2019s metaprogramming features and its support for programming in a functional style. By the end of the book, I was ready to give D a try.<\/p>\n I started by porting my C++ library and feed handler to D. It wasn\u2019t difficult. I use very little inheritance in my C++ code, preferring composition and concrete classes. I found myself quite productive with D\u2019s structs, templates, and mixins. All the while, I kept a close eye on performance benchmarks. When D turned out to give me the same performance as my C++ code, I was sold. I found D to be much more elegant, cleaner, more readable, and easier to maintain. I made the switch and never looked back.<\/p>\n My goal was to develop a complete HFT system using D. The system would consist of different subsystems:<\/p>\n Ultimately, I found a new data structure and better design for my feed-handler framework. I developed the new version completely in D. This implementation heavily uses templates. I like D\u2019s template syntax and generally find the error messages clearer than the complex error messages I was used to from C++. I needed to drop down to assembly for some specific x86 instructions and it was straightforward to do in D.<\/p>\n Later, I needed to work with configuration files. I prefer to write my config files in Lua, a lightweight scripting language<\/a> that is easy to integrate into a program as an extension via its C API. For this, I found a D Lua binding called DerelictLua. Using, again, D\u2019s metaprogramming facilities, I developed a very easy and practical way to interface with Lua on top of DerelictLua. Editor\u2019s Note:<\/em> DerelictLua has since been discontinued; new projects should use its successor, bindbc-lua, instead<\/a>.<\/p>\n The feed handler on the Bats market comes on 31 simultaneous channels, so it is more efficient to use multithreading. For this, I chose not to use the multithreading facilities provided by Phobos. I felt I needed more control in such a low-latency environment, particularly the ability to map each thread to a specific core. I opted to use the pthreads library and its affinity feature. D\u2019s C ABI compatibility made it a straightforward thing to do.<\/p>\n I\u2019m running on FreeBSD. For my intercommunication needs, I\u2019m using kernel queues and sockets. The same functionality is available on macOS, my preferred development platform. D did not get in my way in using these APIs on either macOS or FreeBSD. It was as seamless as using kernel queues from C.<\/p>\n A few notes about problems and limitations:<\/p>\n I was the sole developer on this project but completed it successfully in a relatively short period. Big credit to D and its productivity, readability, and ease of modifications.<\/p>\n","protected":false},"excerpt":{"rendered":" I\u2019ve used the D programming language to implement a high-frequency trading (HFT) platform. I\u2019ve been quite satisfied with the experience and thought I\u2019d share how I got here. It wasn\u2019t a direct path. In 2008, I stumbled across a book on Amazon called Learn to Tango with D. That grabbed my curiosity, so I decided […]<\/p>\n","protected":false},"author":44,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[9,34],"tags":[],"_links":{"self":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/3012"}],"collection":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/users\/44"}],"replies":[{"embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/comments?post=3012"}],"version-history":[{"count":3,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/3012\/revisions"}],"predecessor-version":[{"id":3015,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/posts\/3012\/revisions\/3015"}],"wp:attachment":[{"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/media?parent=3012"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/categories?post=3012"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dlang.org\/blog\/wp-json\/wp\/v2\/tags?post=3012"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
\n
malloc<\/code> and free<\/code> can be costly, so I\u2019m not going to take a chance on a nondeterministic system with unpredictable latency. Instead, I used my library to handle allocation\/deallocation via free lists, with memory preallocated upfront. As a consequence, I also decided not to use D\u2019s standard library for anything.<\/li>\n