Problem: Many testers do not know how to identify UI elements in mobile apps.
In this post, I explain about different ways to find UI elements in Android’s calculator app. I also show you examples of using Appium’s API with the different locator strategies.
Why this post?
Identifying UI elements is a key component of writing UI automation. Most testers know how to inspect a web application and uniquely identify the UI elements that they want. However, inspecting and identifying UI elements in a mobile application is still somewhat of a dark art for most testers. Case in point – when I was learning Appium, it was hard to find a single article that showed me the various options I have to identify UI elements. This post aims to rectify this problem. In this post you will learn different ways to identify and interact with UI elements in a mobile application.
uiautomatorviewer: Mobile’s Firebug
If you automate web application testing, you are sure to have used either developer tools or Firebug. uiautomatorviewer is a GUI tool to scan and analyze the UI components of an Android app. uiautomatorviewer comes with the Android SDK. Please make sure to install Android SDK to follow along.
To launch uiautomatorviewer, pull up your command prompt, navigate to your $android-sdk/tools directory and issue the command uiautomatorviewer. Launch uiautomatorviewer and connect your Android device to your machine. Launch the calculator app. In uiautomatorviewer, click on the Device Screenshot button (top left) to take the snapshot of the calculator app. You can now inspect the layout hierarchy and view the properties of the individual UI components that are displayed on the device.
NOTE 1: For more details on connecting your Android device to your computer, please refer to the Step 2 of our previous blog.
Different ways of finding UI elements
Figure 1 is your path to happiness. You can use the properties of the elements found in the Node Detail table like text, class, resource-id, content-desc etc to find the UI components of the app. Here are a few locator strategies that map well with Appium’s API:
1. Finding element by xpath in mobile application
2. Finding mobile element by Android UIAutomator
1. Finding element by xpath in mobile applications
To write your xpath:
a. start with the class //class
b. [OPTIONAL] add attributes @attribute="blah"
E.g.: XPath to identify the number 7 in the calculator app is
//android.widget.Button[@text='7'] |
With Python and Appium, your corresponding call would be:
driver.find_element_by_xpath("//android.widget.Button[@text='7']") |
2. Finding mobile elements by Android UIAutomator
Another way to identify UI elements in a mobile application is via UiSelector. To do so,
a. start with a new UiSelector object new UiSelector()
b. dot-add any of the other properties of the element .text()
E.g.: To identify the number 7 in the calculator app:
new UiSelector().text("7") |
With Python and Appium, your corresponding call would be:
driver.find_element_by_android_uiautomator('new UiSelector().text("7")') |
NOTE: This method uses properties such as text, content-desc, resorurce id etc to find elements. Please refer to this link for more details.
Here are a few shortcuts to identifying elements that are specific to Appium:
1. Via accessibility ID: On Android, you can use the string in content-desc.
E.g.: To identify the delete button in the calculator,
driver.find_element_by_accessibility_id('delete') |
2. Finding element by Id: Corresponds to the resource-id field
E.g.: To identify the number 7 in the calculator app
driver.find_element_by_id("com.android.calculator2:id/digit7") |
3. Finding element by class name
E.g.: To find the text/results box in the calculator app
driver.find_element_by_class_name("android.widget.EditText") |
An example Appium + Python script with different locator strategies
I am providing a fully functioning Python script for you to practice your locator strategies. Challenge yourself to identify different elements in the calculator app and make corresponding changes to this script.
""" Qxf2: Example script to run one test against calculator app The test will show you how you can find UI elements by various methods like xpath, id, accessibility_id and android UIautomator on a android calculator app """ import os import unittest, time from appium import webdriver from time import sleep class Android_Calculator(unittest.TestCase): "Class to run tests for android calculator" def setUp(self): "Setup for the test" desired_caps = {} desired_caps['platformName'] = 'Android' desired_caps['platformVersion'] = '4.4' desired_caps['deviceName'] = 'Moto E' # Get the package and activity name of the calculator to launch the calculator app desired_caps['appPackage'] = 'com.android.calculator2' desired_caps['appActivity'] = 'com.android.calculator2.Calculator' self.driver = webdriver.Remote('http://localhost:4723/wd/hub', desired_caps) def tearDown(self): "Tear down the test" self.driver.quit() def test_calculator(self): "Testing android calculator" self.driver.implicitly_wait(10) # Find the UI element using xpath self.driver.find_element_by_xpath("//android.widget.Button[@text='7']").click() self.driver.find_element_by_xpath("//android.widget.Button[@resource-id='com.android.calculator2:id/mul']").click() # Find UI element using id self.driver.find_element_by_id("com.android.calculator2:id/digit7").click() # Find UI element using accessibility_id self.driver.find_element_by_accessibility_id('delete').click() # Find UI element using android UIautomator self.driver.find_element_by_android_uiautomator('new UiSelector().text("3")').click() self.driver.find_element_by_android_uiautomator('new UiSelector().resourceId("com.android.calculator2:id/equal")').click() # Find UI element using class name result=self.driver.find_element_by_class_name("android.widget.EditText") print result.get_attribute('text') #---START OF SCRIPT if __name__ == '__main__': suite = unittest.TestLoader().loadTestsFromTestCase(Android_Calculator) unittest.TextTestRunner(verbosity=2).run(suite) |
If you have read so far, you may be interested in our open-sourced Python-based mobile automation framework.Happy inspecting! Happy locating!
I am a dedicated quality assurance professional with a true passion for ensuring product quality and driving efficient testing processes. Throughout my career, I have gained extensive expertise in various testing domains, showcasing my versatility in testing diverse applications such as CRM, Web, Mobile, Database, and Machine Learning-based applications. What sets me apart is my ability to develop robust test scripts, ensure comprehensive test coverage, and efficiently report defects. With experience in managing teams and leading testing-related activities, I foster collaboration and drive efficiency within projects. Proficient in tools like Selenium, Appium, Mechanize, Requests, Postman, Runscope, Gatling, Locust, Jenkins, CircleCI, Docker, and Grafana, I stay up-to-date with the latest advancements in the field to deliver exceptional software products. Outside of work, I find joy and inspiration in sports, maintaining a balanced lifestyle.
Hi Team,
I have recently started working on Android and IOS App for Appium automation. Your article related to “Ways to Identify Elements” in Appium is excellent and 100% useful for new entrants into Appium.
I have thoughts around this and couple of questions.
a) Can we see same attributes for IOS App like you show in the “Calculator” Android App. I know IOS is different OS but any similarity which can be used to maintain single Object Repository for Automation across two different platforms. May be 100% elements not possible but at-least 75% if we can do , that would be fine. If you have any information around this , please share.
b) As of now I have not configured Mac machine for automation , but curious to know how you maintain this Object Mapping in the Open Source Automation projects two different platforms (Android and IOS) which further needs to be integrated with automation framework, if you have any information please share.
Regards,
Kiran
Thanks for the compliments, Kiran.
a) Short answer is most likely not. You may see some attributes for iOS apps that *may* just match . It will also depend on how your development team wrote the two applications. One side point: I have not re-checked recently, but about a year back (if I remember right), I got tripped up by Appium’s find_element_by_id method. There was some nuance (I do not remember now) with it behaving slightly different for iOS and Android. BTW, the latest recommended way to automate iOS is through Apple’s UI Automation framework: https://developer.apple.com/library/ios/documentation/DeveloperTools/Conceptual/InstrumentsUserGuide/UsingtheAutomationInstrument/UsingtheAutomationInstrument.html
b) I do not know. I have not had to maintain this kind of code before. Take the rest of the answer to be the best guess of someone who is very experienced with automation. Technically you could use Appium and then write your own mapping tool based on the platform. If you are disciplined and put all unique identifiers in one place, it will be easy to pick one set of unique identifiers at the time of initialization based on the platform. Using a page-object-like pattern should help you.
Just based on your questions and immediate needs, it looks like you are in a position to arrive at an answer quicker than us. Whenever you solve your problem, can you please write it up either as a comment or (preferably) on your own blog. We would love to learn how you went about solving this problem.
Good article nice one keep it up 🙂
Please can you also give some examples using java
Very Informative ! Nice Article ..
Very Informative ! Nice Article
Great knowledge supplier …Thanks
Hi ,
Whats the command to upload he profile pic from gallery ?
Vaishali,
Which application are you using? Did you try capturing the screenshot using UI Automator Viewer?
Hi Team,
Am trying to configure Appium Grid,but not able to find relevant information in google.
If u have worked on it plz share the stuff with me @[email protected]
Regards,
Mallikarjunareddy
Hi Mallikarjun,
We do not have any relevant information on Appium Grid as we have not configured it yet.
Oh it’s okay Vrushali, I got the solutions for it
Anyhow thanks for the reply…..Doing great job, keep Up.
Mallikarjun,
Do share the relevant information which helped you get the solution.
Sir,does it work in Java. I do the same thing but it throw NoSuchElementException