SE Radio 594: Sean Moriarity on Deep Learning with Elixir and Axon

Creation Purpose

Axon was born out of a need to demonstrate the capabilities of the NX project within the Elixir ecosystem. Sean Moriarity explains that the project began as a collaboration with Josef Alim, focusing on creating Elixir bindings for Google's XLA, a linear algebra compiler 1. The goal was to showcase the potential of deep learning by building Axon, which served as a practical application of NX's functionality. Sean emphasizes that deep learning is essentially a way to learn hierarchical representations of inputs, transforming them into structured outputs through layers of linear algebra 2.

Deep learning, I would say, is best described as a way to learn hierarchical representations of inputs.

--- Sean Moriarity

This approach allows for complex problem-solving, such as image classification, by breaking down inputs into manageable components.

   

Functionality Features

Axon simplifies complex machine learning tasks in Elixir by leveraging minimal dependencies and focusing on essential components like Nx and Polaris. Sean Moriarity highlights that Axon uses gradient-based optimization techniques, such as stochastic gradient descent, to train models efficiently 3. The process involves creating a training loop that applies these optimizations to update model parameters iteratively. Sean notes that the majority of machine learning work involves data preparation, with model training being a smaller part of the process 4.

There's like a common trope in machine learning that like 90% of the work is working with data.

--- Sean Moriarity

This approach ensures that Axon remains accessible and efficient for users, allowing them to focus on refining their models.

   

Integration Elixir

Axon integrates seamlessly with Elixir, leveraging its functional programming strengths to offer a robust machine learning framework. Sean Moriarity points out that Elixir's immutability posed challenges but also inspired innovative solutions, drawing from frameworks like Keras and PyTorch 5. This integration allows Axon to be familiar to users transitioning from Python, while still taking advantage of Elixir's unique capabilities. Sean asserts that the Elixir machine learning ecosystem is competitive with Python's, enabling developers to achieve significant results without relying on external libraries 6.

You can do a ton with a little in the Elixir ecosystem.

--- Sean Moriarity

This makes Axon a powerful tool for developers looking to harness the power of machine learning within the Elixir environment.