HEEDS 2604 Enhancing the Neural Network Builder

Neural networks in HEEDS 2510: where we started

Simcenter HEEDS 2510 could already build neural network surrogates directly from study data. The Neural Network builder sits in the Surrogate ribbon of HEEDS POST and trains a network for the variables and responses you choose.

If you have used Claude, ChatGPT or Gemini today, then you have already utilized the advantages of Neural Networks. The motivation for Neural Networks is large data sets; a neural network trains much faster and predicts more accurate than a traditional surrogate, and it can then stand in for a long-running analysis tool like, e.g. of a CFD simulation.

To use a trained network in place of an analysis, you bring the *.onnx file into a HEEDS MDO process through the ONNX Portal, which was conveniently called “Import ROM…”

What changes in HEEDS 2604

The 2510 builder worked, but some restrictions kept it from becoming a go-to part of surrogate-based optimization. First of all, the prediction was flat, literally. The figure below shows variants created during an optimization of the Schwefel´s test function, explained here, based on a Kriging model and a Neural Network in HEEDS version 2510. The Neural Network could not accurately represent the design space. At best, it found a slope towards the corner of the global minimum; not enough.

No way to compare the network against other surrogates

You could inspect the network’s training plots, but you could not put it side by side with Kriging or the other surrogate models in HEEDS POST. Deciding whether the network was the right surrogate for a response meant utilizing Simcenter ROM or comparing manually.

HEEDS 2604 refreshes the Neural Network builder in HEEDS POST and removes both disadvantages. The main changes are one-click surrogate comparisons and straightforward workflow integration. GPU training and early stopping make the new network cheaper to build.

Example 1: Schwefel’s function

To compare 2510 and 2604 on equal terms, we start with an analytical benchmark: Schwefel’s function. It is cheap to evaluate, and the global minimum is known f = 0 at x_i ≈ 420.97 for every dimension in i. The function is highly multimodal, and the second-best minimum lies far from the global one. That makes it a deliberately hard target for both a surrogate and an optimizer.

Setup:

• Dimensions: i = 2
• Sampling: Latin Hypercube with 400 designs over the full domain
• Validation: separate design set of 100 design

We trained a network on the same Latin Hypercube data in both versions. The training on this data set is quite fast, and even faster on the 2510 version which is partly related to the overhead for GPU enablement. It converges steep but it gets stuck with a larger error on the validation data. The underlying calculation of HEEDS 2604 changed which is obvious from the epoch training. The Neural Network Builder trains and saves the best network weights for the final neural network; in the history above at epoch 844.

Neural network vs Kriging on Schwefel’s function

On Schwefel’s function, Kriging is the more accurate surrogate. You can compare the result of the optimization in the figure below, Kriging variants and Neural Network variants. With 2604 we can show this directly in HEEDS POST, which was not possible in 2510. We then ran an optimization on each surrogate and evaluated the best design found with the exact function.

RESULT:

When to use a neural network?

Schwefel’s function plays to Kriging’s strengths: few inputs, a moderate number of samples, and noise-free data. Real engineering studies often look different, and that is where a neural network pays off.

• Large data sets. Fitting Kriging means factorizing a correlation matrix whose size grows with the number of samples, so cost rises roughly with the cube of the sample count. Neural network training scales roughly linearly with the data, and 2604 adds GPU acceleration.
• Many inputs. As the number of design variables grows, Kriging needs more hyperparameters and more samples to stay accurate. Neural networks cope better with high-dimensional inputs, given enough data.
• Several responses at once. One network can predict all outputs of interest together, while Kriging typically needs a separate model per response.
• Portability. The ONNX model can be reused outside HEEDS without retraining.

The right choice depends on the problem. 2604 makes that choice cheap, because both surrogates can now be compared side by side in HEEDS POST.

On a realistic heatsink case, the neural network beats Kriging when both are checked against computed results. The setup of the analysis is outlined here. Which reveals a restriction of the Neural Network Builder: String variables. While HEEDS can handle discrete design variables which can also be text, the Neural Network Builder fails when they are selected för training. The heatsink test case utilizes string formatted values for Manufacturing and Material of the heatsink. But the Neural Network Builder can only pass number to train the network.

So we trained a neural network and a Kriging surrogate on the same designs but excluded manufacturing and material. We then optimized on each surrogate and compared designs with the fully computed optimization run. The Neural Network created designs lie closer to the simulated results than the Kriging predictions.

Conclusion

HEEDS 2604 turns the neural network from a standalone experiment into a surrogate you can compare, choose, and deploy in one place. The examples show why that matters. On Schwefel’s function, Kriging was the better surrogate. On the heatsink case, the neural network matched the computed results more closely. Neither result would have been easy to establish in 2510; in 2604 it is a direct comparison.

Want to see how the new Neural Network builder fits your own workflow? Contact us at Volupe for a demo or a discussion of your application.