Rust in Model Research and Development

[u/TechnologySubject259]

Hi everyone,

I'm Abinash. For the past few weeks, I have been working on building a neural network in Rust to test the idea of using Rust in model development and research.

Rust is being extensively used in LLM infrastructure and is growing in the inference space as we get official libraries to run CUDA kernels from Rust binaries.

But if you see the model design, research or development space, Rust is non-existent.

So I thought I'd give it a try. I tried to build a fairly easy neural network to solve the MNIST handwritten digit identification problem.

I know it's not a huge milestone. But I tried to give it a try to test my idea how easy it is to develop neural networks in Rust.

The neural network I'm developing has 2 hidden layers with 512 and 128 neurons, respectively. The goal is to achieve 95% accuracy on my local CPU-only system.

I got the input system right; it can now take training and testing datasets and prepare the matrices for the training and testing stages.

After that, I plan to start developing a real transformer-based LLM from scratch in Rust.

I'm not sure how it will play out, but I will give it a try.

I'd love to have your thoughts on it. If you're a senior ML engineer or MTS at a frontier lab, I'd love to have your thoughts on it.

[u/EpochVanquisher]

The reason people do it in Python is because you can just write a few lines of code and you don’t have to wait for the compiler… boom, you’re running. Hard to beat that, for research.

With Rust, you spend additional time writing the code so you can be more sure that the code you’re running is robust and correct. That’s why people use Rust (or Go, or Java, or whatever) for putting their models into production.

[u/ProfesorKindness]

Is there any actual benefit in using Rust (or any kind of compiled language) for this task? I don't see any. Model doesn't get faster and you just spend more time with harder and less configurable language.

[u/jartock → u/ProfesorKindness]

I am not a professional. I am writing an app to run inference for a neural network with Burn. One key advantage I can see from Rust + Burn versus Python + Pytorch is the type system: in Pytorch, past a threshold in complexity, i am often wondering from where stem some shape errors at runtime. In Burn the type system avoid this all together while writing the code. It is much more confortable for me. My various attempts at each new run are more successful than with python (which has a terrible error messaging system to read for a human I found).

My two cents.

[u/diplofocus_ → u/ProfesorKindness]

Actual objective benefit? Probably not. Personal benefit? I get to avoid Python, in a fairly wide arc.