Test Job Base Classes

Base classes for implementing test jobs that is compliant with test schedulers.

TestJob

class ave.utils.testjob.TestJob()

May be used as a base class when implementing test jobs to be executed. Implement a class inheriting from this class and at least override the main() function. It is recommended to set the instance variable branch to the CM branch that the test is running on.

Call execute() on the class instance in the test job. This will pick up parameters including the globally unique ID, or “GUID”, used to track the job’s execution.

Note

When executing on a development machine, the GUID is randomized by TestJob. If test results are reported to Decibel, this GUID may be used to find those results in that database.

Raises Exception:
 If the scheduler issued an invalid GUID.

Flocker Support

flocker_session_key

Use this attribute in calls to Workspace.flocker_push_*() functions to get all logs collected in the same Flocker session.

State Machine

The different states of the state machine are:

'Setup'
'Pre'
'Main'
'Post'
'Teardown'
execute(nr_of_iterations)

Drives the whole test job execution state machine. Will run setup-pre-main-post-teardown looping over pre-main-post the specified number of times.

Each call to the functions are contained in a try statement that catches exceptions, logs it to file and exits the execution flow in an ordered way if an exception is caught.

Parameters:nr_of_iterations – A number that states how many times pre-main-post should be iterated in the state machine
setup()

Setup state in execution state machine, should be used for general setup for the test job. Regardless of nr_of_iterations parameter in execute it will be executed once.

pre()

Pre state in execution state machine, should be used for setup that is closely connected to the test performed in the following main function. If execute is called with parameter nr_of_iterations > 1, pre() will be executed that number of times.

main()

The Main state in the state machine. Must be overridden in a subclass. Will be executed nr_of_iteration times.

post()

Post state in execution state machine, should be used for cleanup that is closely connected to the test performed in the previous main function. If execute() is called with parameter nr_of_iterations > 1, post() will be executed that number of times.

teardown(self)

Teardown state in execution state machine, should be used for general cleanup for the test job. Regardless of nr_of_iterations parameter in execute() it will be executed once.

Convenience Methods

allocate_equipment(self)
allocate_equipment()

Retrieves the profiles from vcsjob and assigns equipment with the equipment_assignment() function.

equipment_assignment(profiles)

Must be overridden in subclass.

Subclass function should assign all the equipment with a single call to Broker.get().

Parameters:profiles – List of profiles that should be assigned.
evaluate_results()

Must be overridden in subclass.

Should evaluate results and set a verdict of the test job to one of vcsjob.OK, vcsjob.FAILURES or vcsjob.ERROR.

find_suitable_label()

Gets the build label from the allocated handset. If it is set to "private", then the latest official build label on the current branch is used instead.

Returns:A string with the found label

Prints all info in the handset profile plus build label and SIM information to log.

update_status(test_job_status)

Setter of self.test_job_status. Use this method to avoid overwriting values with higher priority, to get logging, and to not set invalid status values.

Parameters:test_job_status – the status to be set.

Multiple Handset Handling

register_handset_workspace(handset, workspace)

Use to register handset and workspace. Create an HandsetWorkspaceData object and put it into self.handset_workspace_data_dict which is a dict, handset’s serial as key, HandsetWorkspaceData object as value.

Parameters:
  • handset – the handset allocated
  • workspace – the workspace allocated with handset together
get_handset_workspace_data(handset)

Get the HandsetWorkspaceData object linked with given handset

Parameters:handset – the handset allocated
Returns:A HandsetWorkspaceData object

AndroidTestJob

class ave.utils.testjob_android.AndroidTestJob(logcat_active=True)

Inherits from TestJob.

Base class for test jobs that are written for Android handsets.

Parameters:logcat_active – Run logcat for the duration of the test job?

State Machine

setup()
  • Calls the TestJob.setup() function.
  • Removes crashes from handset by calling remove_crashes_from_handset().
  • Initiates logcat if variable logcat_active is set to True.
teardown()
  • Get all logcat data and pushes it to flocker if logcat_active is set to True.
  • Handles crash dumps.
  • Calls the TestJob.teardown() function.

Packages Installation

download_binary(pkg, app, label=None, version=None, package_component=None, handset=None)

Downloads the specified application from C2D or Jenkins depending on how self.build_url is set.

Parameters:
  • pkg – The package to fetch from.
  • app – The application to fetch.
  • label – The label to fetch from. Not used if downloading from Jenkins.
  • package_component – Package component to download from, will not be used if label is set. Not used if downloading from Jenkins.
  • version – Version to download, will not be used if label is set. Not used if downloading from Jenkins.
  • handset – the handset used, must be set. For backwards compatibility, Defaults to None
Returns:

The path to the downloaded files. Not used if downloading from Jenkins.

download_binary_from_jenkins(pkg, app, build_url=None, pkg_path=None, handset=None)

Downloads the specified application from Jenkins.

Parameters:
  • pkg – The package to fetch from.
  • app – The application to fetch.
  • build_url – The path to the Jenkins build.
  • pkg_path – The path to the artifacts created by the Jenkins build. Not necessary to set unless a special Jenkins job is used that does not follow standard BOC setup.
  • handset – the handset used, must be set. For backwards compatibility, Defaults to None
Returns:

The path to the downloaded files.

fetch_binary(label, pkg, app, handset=None)

Deprecated. Use download_binary().

install_binary(path, timeout=30, handset=None)

Wrapper function for AndroidHandset.install()

Parameters:
  • path – Path to the place to find files that should be installed.
  • timeout – Timeout in seconds (defaults to 30 seconds).
  • handset – the handset used, must be set. For backwards compatibility, Defaults to None
reinstall_binary(path, timeout=30, handset=None)

Wrapper function for AndroidHandset.reinstall()

Parameters:
  • path – Path to the place to find files that should be reinstalled.
  • timeout – Timeout in seconds (defaults to 30 seconds).
  • handset – the handset used, must be set. For backwards compatibility, Defaults to None
uninstall_binary(pkg, handset=None)

Checks if a package is installed and if it is it is uninstalled.

Parameters:
  • pkg – The name of the package to uninstall.
  • handset – the handset used, must be set. For backwards compatibility, Defaults to None

Crash Handling

remove_crashes_from_handset()

Removes all found crashes on the allocated handset.

handle_crash_dumps()

If the allocated handset has an official build label, then all crashes are pulled and reported to Goobug.

Removes all crashes from the handset.

AndroidInstrumentationTestJob

class ave.utils.testjob_android.AndroidInstrumentationTestJob(logcat_active=True)

Inherits from AndroidTestJob.

Base class for test jobs that are written for Android handsets, that run instrumentation tests.

Instrumentation

execute_instrumentation_tests(runner, test_name, test_options, timeout=600, handset=None)

Executes an instrumentation test. Stores the JUnit result file in the local results directory and pushes it to flocker, then shouts it to Panotti.

Parameters:
  • runner – The runner to run.
  • test_name – The Name of the test.
  • test_options – Test options that are passed on to run_junit(). Defaults to None.
  • timeout – Timeout in seconds, defaults to 600.
  • handset – the handset used, must be set. For backwards compatibility, Defaults to None
evaluate_results()

Reports the test by using report_instrumentation_tests(). If the reporting succeeds, then the report is evaluated with the function evaluate_instrumentation_report().

report_instrumentation_tests(handset=None)

Reports the Instrumentation test to decibel

Parameters:handset – the handset used, must be set. For backwards compatibility, Defaults to None
Returns:True if the reporting is successful.
evaluate_instrumentation_report(handset=None)

Evaluates the instrumentation test report and sets the test job status to vcsjob.ERROR, vcsjob.FAILURES or vcsjob.OK depending on the result of the tests.

Parameters:handset – the handset used, must be set. For backwards compatibility, Defaults to None

State Machine

setup()

Calls AndroidInstrumentationTestJob.setup().

State Machine

setup()

Calls AndroidTestJob.setup().

HandsetWorkspaceData

class ave.utils.testjob.HandsetWorkspaceData(handset, workspace)

Data structure for the shared variables linked to a handset.

List of variables set::
HandsetWorkspaceData.handset HandsetWorkspaceData.workspace HandsetWorkspaceData.results_directory HandsetWorkspaceData.sw HandsetWorkspaceData.hw HandsetWorkspaceData.serial HandsetWorkspaceData.imei HandsetWorkspaceData.instrumentation_report HandsetWorkspaceData.build_handler HandsetWorkspaceData.logcat_handler HandsetWorkspaceData.crash_handler HandsetWorkspaceData.run HandsetWorkspaceData.junit_test_completed
set(name, value):

The function assigns the value to the attribute.

Parameters:
  • name – A string may name an existing attribute or a new attribute
  • value – an arbitrary value