Test automation – flutter mobile app

testautomation1

Test Automation is always seen as a way to speed up the software development process. At the same time, Test Automation demands attention.

 

What is Automation Testing?

Automation Testing is a software testing technique to test and compare the actual outcome with the expected outcome. This Test Automation process is used to automate a set of tasks that are difficult to perform manually and utilize results to improve the quality of a software product.

As we know that apps nowadays are multi-featured making it rigorously difficult to test apps, but this problem is removed by Automation Testing in Flutter. This Automated Testing makes sure your app is bug-free & Performant.

Automation Testing falls into three categories mainly:

    1. Unit Test tests single class, method, or function
    2. Widget Test tests single widget
    3. Integration Test tests a complete app or large part of App

Why?

 Mobile Developers testing their code is limited to writing unit tests which usually is to test a single method or a class. But, when it comes to a full-fledged end to end testing of an app or a new feature, the dedicated Quality Assurance Team need to write automated tests to validate the flow. For instance, how the app responds to a tap, scroll, input, navigation, and other interactions.

Is it a tap? Yes. In order to like an article or if you want to comment on a post, the first action you perform is to tap on that widget.

Is it scroll? Yes. We scroll to see all the articles and to keep reading an article. 

Whenever some new feature gets added/updated, the QA team must test the whole application to ensure no errors. Performing the same test over and over, not only takes more time but also brings down the overall efficiency of a tester.

On the other hand, Manual testing of an application leads to the following issues:

  1. Testing cannot be thoroughly performed every time the software gets updated because of the time constraints.
  2. Unsurity about whether the tests are performed the same way every time.
  3. The more will be the test cases, the more hands and hours you will need
  4. There may be some test cases that may include dozens of reproducible steps. Such scenarios could lead to a faulty assurance by a mere distraction while testing manually.

With Flutter Driver, we can automate the testing process and it also helps us to drive the application in another process and exposes different helpful methods to test user interaction and to compare actual outcome with the expected outcome of the app.

Flutter Driver is for flutter and it is like Selenium WebDriver and Protractor for Angular JS, Google Espresso for Android and Earl Grey for iOS which Drives a Flutter Application running in another process.  To find more information, check official documentation on flutter_driver.

Enough talking? Let’s Code

  1. Create a flutter application
  2. Add the flutter_driver dependency in pubspec.yaml.
  3. Add test cases

Create a folder named test_driver in the root directory and Add two files.

Let’s name the first file as app.dart which will looks like this:

void  main() {enableFlutterDriverExtension();

app.main();

}

Here enableFlutterDriverExtension() method enables the extension and the line app.main() which call the main() function of your app or call ‘runApp’ with any widget that you are interested in testing.

The Second file ends up with app_test.dart where we create FlutterTestConfiguration.

But Before getting into what is in the second file let’s see one important topic i.e., Behavior Driven Development (BDD). Which is one of the best ones available in the market provides Test Automation Services which includes implementation of wide set of testing and around-testing components.

BDD aims to create a shared understanding of how an application should behave by discovering new features based on concrete examples. Key examples are then formalized with natural language following a Given/When/Then structure. 

Gherkin is the most used syntax for describing examples with Given/When/Then in plain text files, called feature files. Gherkin scenarios can be automated to validate the expected behavior.

Feature: Login into App
Scenario: Initiate App
When the user clicks login Button
Then the user can see homepage of App

As you can see, it is almost plain English. Such document can be used as a product documentation as well as test case documentation.

Let’s create Gherkin document and then write some automated tests with the help of a plugin called flutter_gherkin.

The plugin flutter_gherkin parses Gherkin syntax and allows to run Automated Tests in Flutter applications. For Running the tests, it connects with FlutterDriverExtention under the hood.

Let’s implement this on sample Employee Directory Application in which we can be able to add new employee into Employee Directory and in the home screen it displays list of employees.

[/vc_column_text][vc_row_inner el_class=”images-row”][vc_column_inner][us_single_image image=”13073″ size=”full”][us_single_image image=”13072″ size=”full”][/vc_column_inner][/vc_row_inner][vc_column_text]And this application implemented with a functionality that while tapping the plus button it navigates to a new screen where we can fill employee details and by tapping the Add button, it will add that employee to Employee Directory.

If we documented this in Gherkin, it will look like this

Feature: Employee directory App
Test Application
Scenario: View Employees List
Given Can see Employees in Employee Directory
And Tapped Plus button to Add Employee
And Visible employee add Screen
And Tapped Camera icon to Add Image
And Entered Employee name as “Alex Vebour”
And Entered Employee Designation as “Software Engineer”
And Tapped Add Button to Add Employee
And Swapped to remove employee from list

How this will help in Automated Integration Testing in Flutter
Let’s start by creating folders under test_driver and the folder structure would look like this.

- test_driver/features
- test_driver/steps

And add plugin called flutter_gherkin in to pubspec.yaml file.

flutter_gherkin:

Under feature folder we have all the .feature files written in Gherkin language. As mentioned above add that file inside this folder and name it with extension (. feature).

Feature: Employee directory App
Test Application
Scenario: View Employees List
Given Can see Employees in Employee Directory
And Tapped Plus button to Add Employee
And Visible employee add Screen
And Tapped Camera icon to Add Image
And Entered Employee name as “Alex Vebour”
And Entered Employee Designation as “Software Engineer”
And Tapped Add Button to Add Employee
And Swapped to remove employee from list

Implementation of steps involved
In Gherkin, each scenario has multiple steps which begin with keywords like Given, When, And and Then. when we are writing automated tests, we implement each step.

Let’s implement the first step

Given Can see Employees in Employee Directory

Inside the steps folder create file, for example, employee_directory_steps.dart in which we would write the steps. We must create step for each element in every scenario.

StepDefinitionGeneric viewEmployees() {
return given(
‘ Can see Employees in Employee Directory’,
( _) async {
await Future.delayed(Duration(seconds: 1));
},
);
}

The Regex syntax is required for the plugin to parse the Gherkin syntax into their appropriate step definition. Each step has its own unique implementation.

Let us implement another step…😃

And Tapped Plus button to Add Employee

Here we must access the widget button for tap 😏.
As we know flutter is all about widgets and we can specify each widget with unique key by using that key we can access specific widget. They are four ways to find widget in application, those are

• byValueKey, find widget by Value key of widget
• byTooltip
• byType
• text, find widget by mentioned name

Here we have floating action button we would access it by its tooltip ‘addEmployee’

StepDefinitionGeneric addEmployee() {
return when< FlutterWorld>(
‘Tapped Plus button to Add Employee’, ( context) async {
final imageAddIcon=find.byTooltip(‘addEmployee‘);
await FlutterDriverUtils.tap(context.world.driver, imageAddIcon);},
configuration: StepDefinitionConfiguration()
..timeout = Duration(minutes: 2));
}

Here Flutter Driver make it possible actions on widget like tap, scroll the widgets like that😃😃.

Create FlutterTestConfiguration

Create a file called app_test.dart inside test_driver folder in which we would create FlutterTestConfiguration. It allows many different configuration options.

Future <void> main() async {
final config = FlutterTestConfiguration()
..features = [Glob(r”test_driver/features/**.feature”)]
..reporters = [
ProgressReporter(),
TestRunSummaryReporter(),
JsonReporter(path: ‘./report.json’)
]
..hooks = [HookExample()]
..stepDefinitions = []
..restartAppBetweenScenarios = false
..targetAppPath = “test_driver/app.dart”
..exitAfterTestRun = true
..flutterBuildTimeout = Duration(minutes: 10);
return GherkinRunner().execute(config);
}

In Step Definitions we should define all the steps which were we implemented. like this

..stepDefinitions = [
viewEmployees(),
addEmployee()
]

If we want to restart the app between scenarios, set this flag restartAppBetweenScenarios to true.
Hooks are custom bits of code that can be run at certain points with the test run such as before or after a scenario. Place instances of any custom Hook class instance in this collection. They will then be run at the defined points with the test run.

HookExample extends Hook {
/// The priority to assign to this hook.
/// Higher priority gets run first so a priority of 10 is run before a priority of 2
@override
int get priority => 1;

/// Run before any scenario in a test run have executed
@override
Future <void>onBeforeRun(TestConfiguration config) async {
print(“before run hook”);
}

/// Run after all scenarios in a test run have completed
@override
Future <void>onAfterRun(TestConfiguration config) async {
print(“after run hook”);
}

/// Run before a scenario and it steps are executed
@override
Future <void>onBeforeScenario(TestConfiguration config, String scenario) async {
print(“running hook before scenario ‘$scenario‘”);
}

/// Run after a scenario has executed
@override
Future <void> onAfterScenario(TestConfiguration config, String scenario) async {
print(“running hook after scenario ‘$scenario‘”);
}
}

Yeah😀, we are at end….

Use the following command to run the test cases.

flutter drive –target=test_driver/app.dart –driver=test_driver/app_test.dart

Check report in console, all test cases are passed…😄😄

[info ] FlutterDriver: Connecting to Flutter application at http://127.0.0.1:54995/-Ik3cFJqqNY=/
[trace] FlutterDriver: Isolate found with number: 1411561946577759
[trace] FlutterDriver: Isolate is not paused. Assuming application is ready.
[info ] FlutterDriver: Connected to Flutter application.
Running scenario: View Employees List # test_driver\features\add_employee.feature:4
√ Given Can see Employees List if Employee Exists # test_driver\features\add_employee.feature:5 took 1084ms
√ And Tapped Plus botton to Add Employee # test_driver\features\add_employee.feature:6 took 2167ms
√ And Visible employee add Screen # test_driver\features\add_employee.feature:7 took 1004ms
D/FlutterView( 2652): Detaching from a FlutterEngine: io.flutter.embedding.engine.FlutterEngine@bae5c36
D/FlutterView( 2652): Attaching to a FlutterEngine: io.flutter.embedding.engine.FlutterEngine@bae5c36
D/FlutterView( 2652): Detaching from a FlutterEngine: io.flutter.embedding.engine.FlutterEngine@bae5c36
D/FlutterView( 2652): Attaching to a FlutterEngine: io.flutter.embedding.engine.FlutterEngine@bae5c36
√ And Tapped Camera icon to Add Image # test_driver\features\add_employee.feature:8 took 23701ms
√ And Entered Employee name as “Alex Vebour” # test_driver\features\add_employee.feature:9 took 1849ms
√ And Entered Employee Designation as “Software Engineer” # test_driver\features\add_employee.feature:10 took 1505ms
√ And Tapped Add Button to Add Employee # test_driver\features\add_employee.feature:11 took 4948ms
√ And Swapped to remove employee from list # test_driver\features\add_employee.feature:12 took 1793ms
PASSED: Scenario View Employees List # test_driver\features\add_employee.feature:4
1 scenario (1 passed)
8 steps (8 passed)
0:00:38.695000
Terminating Flutter app under test

Finally, it has been completed…. Let us look at some issues we may face while using Gherkin while implementation of Automation Testing😀

  1.  While using Gherkin if a test is poorly written and is not quite understandable, the test maintenance cost is easily increased in those cases.
  2.  The steps often contain white spaces, parameters included in non-standard locations and a lot of other ambiguities that text editors are not being able to rectify. This leads to the breaking of many features like auto-complete, search etc. ultimately hampering the final product.
  3.  While testing each scenario, if any test fails then all the remaining test cases will be skipped, and it may lead to a retest.

…Hope You will not be stuck into these😄

Jeevan

Related posts

Challenges bring the best out of us. What about you?

We love what we do so much and we're always looking for the next big challenge, the next problem to be solved, the next idea that simply needs the breath of life to become a reality. What's your challenge?