Automating Regression Tests for an IoT Based Smart Home Mobile App

Mobile Application

Our client is a leading product company in the smart home industry harnessing the power of IoT to create world-class smart home products.
Automating Regression Tests are a process of verifying that a piece of software/web application functions as expected, by testing it against known data that might have been changed since the software was originally designed or installed. The software is exercised against a set of known inputs to find any errors or unexpected behaviors that might have developed since the last time it was exercised. Regression testing for software is especially important because changes made to the software during its development or after it is installed could cause it to malfunction or produce unintended results.

The smart home solution mainly offers IOT-based smart sensors like smoke alarms, water leak detectors, garage door sensors, and a mobile app. The mobile app, available on both iOS and Android platforms facilitates the provisioning of the smart sensors to a WiFi network either through the SoftAp mechanism or the Sound-based mechanism. Whenever there is a water leak or if the smoke alarm goes off the user will get a push notification to his smartphone alerting the user about the event. The app also lets users customize the smart sensor’s features according to their needs.

iLeaf Solutions has been associated with the client on a long-term basis providing its services for mobile app and cloud development. Since the client had an increasing number of white-labeled versions of the smart home app, reducing the time required for each release cycle became a challenge that needed to be addressed. We did Automating Regression Tests for this IoT-based Smart Home Mobile App. Please know more about this project from IoT Based Smart Home Mobile App.

Go through our PORTFOLIO to know more about our latest projects.

The Challenge

Scaling the tests as the number of white-labelled apps increased.

The main challenge came with the increasing line of white-labeled apps such that whenever there was a feature addition or feature modification or general update in the smart home app, our team of testers had to manually verify all the changes and perform regression testing on the smart home app and the white-labeled versions of the app as well. This required a tremendous amount of time and effort thus affecting the delivery deadline. We realized the fact that along the way the amount of time and effort required would only be increasing as the number of white-labeled apps increased.

So our expert team of testers came up with the idea of automating the regression tests in order to tackle this challenge.

iLeaf's Process

Communication

We discuss to ensure that we have the exact idea of what is required

Collaboration

There's regular interaction with the client to ensure things are on track

Development

Begins according to the needs of our client

Result

The final output will be a perfect match to our clients requirement

The automation testing team at iLeaf Solutions initially conducted a thorough analysis and created a regression test suite that was suitable for automation, since some of the test cases could only be performed manually we had to differentiate those first.

Automated Testing

To automate the test cases, ileaf’s Automation engineers used the Appium framework. The framework was chosen as it enabled the automated testing of both the iOS and Android versions of the smart home app using a single codebase thus saving time and resources otherwise needed to develop and implement separate automated regression testing for the two platforms. The scripts were written in Python which indeed helped us to speed up the scripting process.

Data-Driven Model

We adopted a Data-driven model of testing which enabled us to improve the test coverage and test with multiple sets of data. We also followed the Page Object Model architecture which enabled us to easily maintain and modify the scripts. On top of that, we created a wrapper class that wrapped all the appium commands so that we were able to increase the stability of the tests.

Some of the test cases were based on database testing, so we had to integrate MongoDB into our scripts and write queries and fetch data and assert them according to the test cases.

Cloud-based Solution

During the course of developing the test scripts, the client wanted a possible solution to scale the automated tests on different testing devices of different OS versions and different screen sizes. After a thorough analysis, we decided to go with a cloud-based solution, eventually choosing AWS Device Farm which had a large device pool and was cost-effective as well. This also helped us to get the video footage of tests that failed, which helped us to reduce the time in reproducing and debugging the issues.

We handled the project management using JIRA and the version control of the automation scripts is done on bitbucket.org

Technology Stack:

Appium, Python, MongoDB, AWS Devicefarm, Xcode, Android Studio, GitKraken

The Result

Smooth Operating Test Automation Scripts:

The Test Automation team was able to deliver a highly stable Regression test automation framework to cover a wide range of test cases on both the iOS and Android platforms.

Minimized The Time For Each Release Cycle:

The client was highly satisfied with the output and the fact that the regression automation test suite delivered by iLeaf Solutions met the client’s requirements and was able to positively impact the delivery deadline.

Marginally Reduced Human Efforts

The automation scripts thus delivered also lessened the manual efforts which enabled the testers to focus more on exploration-based tests.

Our test automation engineers regularly conduct automation tests during each release cycle and also maintain and modify the scripts whenever there is a feature change in the mobile app.