Stellar Survivor - PFNN

Phase-Function Neural Network

For our game Stellar Survivor, I worked together with a few programmers to attempt to integrate a phase-function neural network  (PFNN) into Unreal Engine 4. This PFNN generates highly realistic animation data at run-time for the characters in the game. 

Since it has never been done before in the industry, this was a truly groundbreaking experiment for our team and our school/lecturers. Unfortunately, due to the complex math required to get the right rotation and translation values into UE4 on a skeletal mesh, and a lack of time and resources, we couldn't successfully implement the system.

Training

This video shows one of the results that we got from training on a rather small library of motion capture data that we recorded with the BUAS Motion Capture Team. This result is shown in a custom engine in which we did manage to get the system working.

Labelling

After the data has been recorded, several details need to be labelled before the neural network can train on it. To do this, I built a Houdini Digital Asset (HDA) that has scripts that speed up this labelling process significantly. Due to the manual correction that is needed for some parts of the process, I built this HDA around keyframes to make the correction as easy as possible. After the correction is finished, all these scripts generate a few new files per animation file that tell the network what it needs to know.