Table of Contents

Editor: Automating System Tests

Description

As you build an expanding array of Systems tests, running them one by one becomes a tedious operation as well as tracking the result. This guide will go through automating the test so that it can run automatically and provide test results using Unreal’s Automation Testing suite. This will build upon Editor: Scripting a System Test.

As more tests will be built, it’ll become very tedious to open each level and run them to get their results. Unreal’s automation suite can be used to run every test you create and simplify this process. To start, open up a level with an assert and drag a BP_Functional_Test_Empty Content into the viewport. This can be found in the content browser under Content → Nominal → NominalEditorTests → NominalUETestingCore.

Untitled

Select the actor in the World Outliner. The properties in the details panel on the left will be shown. The BP_Functional_Test_Empty has default values already pre-configured and ready for use. These can be changed here to edit them on a level-by-level basis. Hover over the properties to get a tooltip explaining their function.

Automating Tests

Open the Level Blueprint of the level and create a Find Functional Test node after the standard Configure UI function. This should be done at the end of the Begin Play event sequence. This can be promoted to a variable to be stored and used later.

Untitled

After adding Assert function(s), get the Functional Test object and call the Finish Automated Test function. Link the result from the Assert (or multiple asserts using the ‘and’ operators to combine the results into one) to the Finish Automated Test node result.

Untitled

The Finish Automated Test node will check if the Functional BP exists and return success or fail to the Unreal Automation suite along with a simple message of “Test Succeed” or “Test Failed”. Additional alerts can be added by creating custom functionality by using Finish Test.

Untitled

Running the Automated Test

Open the Session Frontend window found in Tools → Session Frontend. This window can take up to a minute to load, depending on your computer's operating speed.

Untitled

Select the Automation tab in the Session Frontend window. From here you should be able to see the BP_FunctionalTest_Empty under Project → Functional Test → MapPath. In this example: /Game/Nominal/Demo/Demo_AttitudeFeedbackBasic.

Note

If the level can’t be found in the tab, make sure to save the level and click Refresh Tests in the automation tab.

Click the checkbox to the left of it and click Start Test. If there were any errors during the test, it will list them in the Automation Test Results section at the bottom right.

Untitled

The test will run exactly as if you hit play on the level. After the test is completed, it will return a pass or fail. If you had multiple tests selected, they will run one after the other and their results will be shown here.

Note

A group of tests can be made into a preset which can be run as a group.

Lastly, the results can be exported in a CSV format. This can be done by selecting the test levels in the test section once they have completed running and by clicking on the Export button.

Untitled

Common Errors

  • If a test fails with “A functional test is already running” the test failed due to Unreal’s end and needs to be rerun.
  • A print string saying “Functional Test Actor is not Valid” will be called if you are calling “Finish Automated Test” without a valid functional test, either it doesn’t exist in the level or it hasn’t been set. It can be ignored if you are not utilizing Unreal’s Automation Tool/Session Frontend such as playing a level through the editor.