Mission Simulation and Machine Learning: A Collaboration To Improve Your Analysis

Artificial intelligence is a powerful tool for solving problems. When coupled with mission simulation, you can explore solutions to novel problems for a fraction of the cost of development and testing in the physical world. Specifically, you can improve your analysis by using a workflow that combines mission simulation and machine learning (ML) models in Ansys Systems Tool Kit (STK) digital mission engineering software.

Understand the Problem

ML is a data-driven approach to solving problems because the problem that you want to solve is modeled in the data used to train and validate your models. When applied thoughtfully, ML can extend your ability to understand and solve problems using that data. For example, an ML model trained on well-designed datasets can learn hidden features and relationships, and it can allow you scale and automate tasks in an operational setting. So the question is, how do you get these datasets? You could start the expensive, multiyear process of conducting an operation, or you can turn to commercial off-the-shelf (COTS) tools like STK software to generate datasets for use in ML models.

While dataset design is beyond the scope of this blog, you can answer a few questions to help you start modeling your mission and generating datasets.

1. What kind of ML problem are you trying to solve? For example, is it a classification or identification problem, or is it a regression problem in which you are trying to predict a numeric value from an input? The image above depicts the typical ML development life cycle, and the process always begins with understanding the problem you are trying to solve.
2. What kind of data do you need to train your model, and what features should be included in the data? For instance, if you are trying to identify a satellite using distinguishing characteristics from an optical sensor, you would need images of the satellite to train and validate your model, as well as labels for each example in the dataset, in which the labels describe the satellite in the example.
3. How many examples do you need in your training dataset, and how closely do these examples represent the problem in the physical world?

Generate the Dataset Workflow

Once you have a handle on the problem, the next step is diving into the tools to generate datasets. Generating datasets with physics-based tools like STK software gives you traceability in understanding where your data comes from, and it gives fine-grained control over how you model the problem in your datasets. Simulation tools also provide the benefit of modeling and exploring solutions for which datasets do not exist. For example, simulation is especially helpful for developments taking place in the future or if a solution is too costly to launch and iterate in real life.

With STK software, you can build a design reference mission (DRM) to model the problem that you are investigating. Your DRM can include all the assets and complex systems in the context of the mission. Whether those be on the ground, in the air, or somewhere in space, you can model your assets in one cohesive tool. There is the added flexibility of making changes to your DRM to generate new results without affecting the operational cost. In a live mission, you might have a wide selection of parameters but only get to test one configuration at a time. With simulation, however, you can sweep through unique configurations to test multiple approaches and generate the data you need.

You can generate results from your DRM by modifying the scenario, but you also can connect your tools to external ones to make automating easier. Ansys ModelCenter model-based systems engineering software or custom scripts are common tools to automate and run trade studies on your DRM. Once the data has been generated, it’s time to explore ML approaches to find the best fit for your problem space.

How to Implement a Machine Learning Pipeline

The ML development life cycle is an iterative process. A substantial part of that process involves implementing a pipeline that includes the following steps:

1. Data ingestion and preprocessing.
2. Exploratory data analysis. This step is all about understanding the dataset that you have generated. Specifically, you want to understand how well this data represents the problem.
3. Data preparation. In this step, you perform any preprocessing required to transform the data into a format for training and model evaluation. This includes splitting the data into train, test, and validation sets, as well as feature reduction.
4. Model selection, training, and evaluation.

Explore the Workflow in Action

When used together, STK software and ML can help you achieve new levels with your mission analysis. Let's talk about some research work that demonstrates the approach we just discussed. At the 2023 and 2024 AIAA SciTech Forums, we presented “Electro-optical Fiducial Markers on Satellites for Identification and Characterization” and “Characterization and Classification of Low-resolution LEO and GEO Satellites With Electro-optical Fiducial Markers,” respectively.

In these experiments, we used STK software to simulate the mission and design our dataset. We wanted to test whether unique spectral markers, also called electro-optical fiducial markers, on the body of a satellite could be detected and characterized by neighboring satellites or from the ground. Could our ML model tell if there is or isn’t a marker present? We modeled the cameras’ systems on each asset and let STK software generate the data for us. In our follow-up work, we made the problem harder by using cameras with decreasing resolution to capture images of our satellite of interest. We wanted to see how far we could degrade an image and challenge our ML classifier model.

STK software scenario views in the 2024 study. On the left, SOI_LEO is indicated in fuchsia and the observing satellite is in teal. At right, SOI_GEO is indicated in green and the observing satellite’s trajectory stages are in orange, yellow, and red.

In both studies, we followed the data generation process with the ML pipeline. We first ingested and preprocessed the data, then we conducted an initial exploratory analysis, prepped the data, and trained and evaluated our model.

Take the Next Steps

Our research shows that collaboration between simulation and ML greatly improves analysis. We hope that this workflow and our research inspire you to model your missions with STK software and explore generating datasets to train your ML models.

Get started now, and join us Aug. 6 to 8 at the Space & Missile Defense Symposium in Huntsville, Alabama.