Chris Mattmann — ML Applications on Earth, Mars, and Beyond

Data Processing

Data processing at NASA JPL involves navigating vast amounts of data, which presents significant challenges. Chris Mattmann explains that while there's enthusiasm for external contributions to data mapping, the sheer size and complexity of the data make it difficult for individuals without substantial resources to participate 1. He highlights the financial and logistical constraints in preserving data, noting that archives can cost hundreds of millions of dollars to maintain 2. This creates opportunities for innovation, as NASA encourages proposals for improved data processing methods, allowing universities and commercial partners to develop better products 2.

NASA doesn't have to control everything. This creates a market and an opportunity downstream for universities, commercial partners or whatever to build better products.

--- Chris Mattmann

The process of converting raw satellite data into usable information involves complex scientific processes, often requiring interpolation and averaging to create comprehensive global maps 3.

   

Level Two Data

Level two data at NASA is both massive and intricate, often reaching petabyte scales, which poses significant storage and processing challenges. Chris Mattmann notes that while many desire access to this data, its size and complexity make it impractical for direct use, leading to decisions about what data to preserve and what can be reproduced through reprocessing 4. The data is stored in formats like HDF5, which originated from NASA's need to represent Earth science data, but these formats aren't always machine learning-ready, requiring additional processing 4.

People sometimes, because they don't understand, these satellites generate these weird u shaped orbital swaths where the data is only valid at certain times.

--- Chris Mattmann

The transition from level two to level three data involves significant data reduction, making it more manageable and useful for scientific analysis 5.

   

Edge ML

Implementing machine learning on edge devices presents unique challenges, particularly in ensuring models perform consistently across different hardware. Chris Mattmann emphasizes that deploying models on devices like Nvidia TX2 or Jetson is not straightforward, as it requires extensive engineering to maintain performance 6. He advises against changing hardware midstream, as even minor differences can necessitate significant reengineering efforts 6.

Stick to what you got and the computing power you have and engineer more optimizations there.

--- Chris Mattmann

The future of MLOps and AutoML is promising, with advancements in automated parameter tuning and model feedback systems. However, Mattmann notes that widespread adoption is hindered by the gap between current practices and emerging standards 7.