Andy Thomason, code performance consultant and lecturer at the University of London covering programming, physics and AI courses focused on game development, created an open source project to add Rust’s performance, reliability, and productivity to R. Andy created the Extendr package, a safe and user-friendly R extension interface for using Rust. The project was supported by a grant from the R Consortium.
Extendr is a toolkit for building extensions to R using Rust. R users can add high performance features to R without having to worry about crashes due to segmentation faults and other problems associated with C and C++. Extendr manages the lifetime of wrapped R objects so that they are available for garbage collection when they reach the end of their life.
Rust is extremely user-friendly and has been voted one of the most loved languages on Stack Overflow for several years.
ExtendR is available on crates.io.
By joining R with Rust, Extendr makes it possible to write packages and small functions in Rust. What problem are you solving by doing this?
The integration of R and Rust through Extendr addresses a specific challenge encountered by individuals with an R background when it comes to writing code in languages like C and C++. While R users often face difficulties when working with C or C++ code, Rust offers a more accessible and streamlined alternative, focused on multi-threaded programs. The package is meant to help simplify the process from R to Rust.
What sort of use cases provided by R users helped with the design requirements?
I originally started working with a biotech start-up in Oxford, where most of their business logic was written in R. They were working on solving tricky problems with Bayesian Inference in particular. Bayesian inference is a method of statistical inference in which Bayes’ theorem is used to update the probability for a hypothesis as more evidence or information becomes available. I started coding C and C++ extensions to R to solve these problems using improved exponential functions.
For Bayesian Inference, you work in log space. Initially, I was using C++ but later the company I was working for started using Rust instead, and I started liking Rust. Using Rust, I was much more productive than with C++.
My interest then got me to study R internals to the point I could start writing something like this Extendr package. More people joined the community, brought things to it, and it has grown since!
How far along are you on the project?
It works! We have a strong team committing every day (andy-thomason, clauswilke, yutannihilation, multimeric, CGMossa, Ilia-Kosenkov). There are regular contributors to the project, and they work constantly to improve Extendr.
As yet, we don’t have a large number of applications. It would be great if more people knew about and were involved in using this useful language.
If you want to consider using or contributing to Extendr, please contact us through Discord.
How did you get involved?
Having dedicated several years to working with Sony and the PlayStation consoles, I gained valuable experience in compiler development. Interestingly, my spouse, who is also a scientist, uses R. She often collaborates with colleagues learning to use Rust. Furthermore, we have friends who use R for drug trial designs, with a shared objective of cost reduction – drug trials are notoriously expensive.
Also of this means I looked into both programming languages, R and Rust. By utilizing R, significant time and financial resources can be saved. It inspired us to set out to build a bridge between Rust and R, leveraging the strengths of each.
Rust excels when dealing with vast amounts of data, and it proved to be exceptional for storing and retrieving multi-terabyte datasets. Rust acted as a powerhouse that facilitates data processing, while R empowered statisticians to draw meaningful conclusions and perform clustering analyses. In this symbiotic relationship, R served as the cognitive component, and Rust as the practical execution framework. The meticulous engineering nature of Rust ensured that everything aligned flawlessly, allowing R to seamlessly integrate and thrive in this cooperative ecosystem.
What are the next steps with the project?
Our contributors Hiroaki Yutani and Claus Wilke have put quite a lot of effort into plotting – there are also plans for 3D plotting. Ilia Kosenkov in Helsinki is the current maintainer and the project is very active. We also hope to release documentation in book form.
What do you do for your day job?
I am a consultant for a number of different industries. I also teach Rust programming to people all over the world like employees of Google and Fastly. Recently I have been consulting with a blockchain company about data organization and data-related problems. They use R for analyzing stock prices. I am also writing a book on Rust performance – making Rust go fast.
What non-project hobbies do you have?
Outside of my projects, I enjoy spending time with my kids, who provide plenty of entertainment. I also have a fascination with Soviet brutalist architecture, which I share with my son. We also find pleasure in exploring areas on the Thames near Oxford.
What was your experience working with the R Consortium? Would you recommend applying for a grant to others?
The process of working with the R Consortium was brilliant! It was a great project to dive into during my transition from employee to consultant. Some tasks were structured, while others allowed for experimentation and refining the project’s structure. Over the past year, we’ve seen increased interest, especially in the collaboration between Rust and R.
The project has a vibrant Discord group (Extendr – R Adventures in Rust) where scientists seek help and support.
I highly recommend others apply for grants through the R Consortium to contribute to meaningful projects and join a supportive community.
About ISC Funded Projects
A major goal of the R Consortium is to strengthen and improve the infrastructure supporting the R Ecosystem. We seek to accomplish this by funding projects that will improve both technical infrastructure and social infrastructure.