Problem: Use Appium and press ‘Enter’ on the soft keyboard.
This post continues our series on intermediate level problems when automating mobile application tests using Appium. We thought of sending a KeyEvent to an input (textbox) element using Appium. Superficially this appears to be trivial, but while writing the script, figured out otherwise. To our surprise, it failed. And there seems to be a good reason for it too! The rest of this post outlines what we were trying to do, what we learnt and what we (reluctantly) ended up doing.
Why this post?
Qxf2’s top goal is to help testers. We identified mobile automation as one area where we could help testers. Over the past few months, we have written several basic tutorials that help testers with mobile automation. We feel like the next stage is to address some common intermediate level problems you may face as you implement mobile automation at your workplace. Over the next few posts, we will tackle some common technical challenges testers are likely to face and our approach to solve them.
New to mobile automation?
If you are new to mobile automation, we suggest the following posts to get started:
a) Run Appium on an emulator
b) Run Appium on mobile devices
c) Identify UI elements on mobile platforms
d) Execute different mobile gestures
Overview
As with our earlier post, we continue to use BigBasket as the application under test. This post deals with an issue we faced while writing the tests scripts for search functionality. We entered a string in the search box. In this particular application, once we enter the text to be searched, a soft keyboard appears with the search icon instead of the enter button. Turns out for BigBasket, the search icon cannot be identified by an xpath or id. And our quest for a solution taught us something about Android.
Our Approaches
1. Search icon (element) could not be identified using xpath or id:
Element related to the search icon could not be identified using the UIAutomatorViewer. Hence we could not write any xpath for it.
As seen in the screenshot above the ‘resource-id’ shown for the search icon is actually the id for big basket ‘checkout’ button. Sigh. BigBasket devs, you can do better!
2. Enter key event:
Next, we attempted to send an ‘ENTER’ keyevent with keycode 66(which simulates the enter key behaviour) after we input the search text but we were not successful.
text_field = self.get_element(self.search_box) text_field.send_keys(value) self.wait(3) self.driver.keyevent(66) |
The Appium logs did not show any error but the event was not triggered and the search results were not shown. This blew our minds. How was a KeyEvent being ignored? Surely we were doing something wrong! We spent many an hour trying to figure out why sending the ‘Enter’ KeyEvent was failing. Finally, while pouring through Android documentation, we hit this:
You should never rely on receiving KeyEvents for any key on a soft input method. In particular, the default software keyboard will never send any key event to any application targetting Jelly Bean or later, and will only send events for some presses of the delete and return keys to applications targetting Ice Cream Sandwich or earlier. Be aware that other software input methods may never send key events regardless of the version. Consider using editor actions like IME_ACTION_DONE if you need specific interaction with the software keyboard, as it gives more visibility to the user as to how your application will react to key presses.
Sigh. Sigh. Sigh.
3. Using Co-ordinates:
So we kicked. We screamed. Tore our hair out. Banged our heads on a wall. And then went about using co-ordinates to solve this issue. These co-ordinates unfortunately vary for different devices. We perform a ‘tap_positon’ for the corresponding co-ordinates (as shown in the top right corner of the above screenshot)
text_field = self.get_element(self.search_box) text_field.send_keys(value) self.wait(3) self.tap_position([(500,840)]) |
And that ended our little adventure. We sincerely hope that we are missing something here. Let us know if you found a better way!
Note: We have open-sourced an intuitive, well-organized, Pythonic test automation framework. Check it out!
My journey as a tester started at Sun Microsystems (now Oracle). I was part of the testing and sustaining team for the Portal Server and Identity Management products. My first assignment was to test the Rewriter module. I enjoyed understanding the big picture, writing test cases, finding bugs and sometimes suggesting the fix too! I was hooked onto testing. Testing felt natural and intuitive to me. I am technically inclined and can write automation in Java, C++, Perl and Python. I am well versed with SilkTest, Selenium, Appium and Selendroid. I am a Computer Science graduate from BITS-Pilani. I love travelling and listening to music.
Hi Vrushali
This is an error which you faced when you need to type something on the soft keyboard .
What if you come across a situation when an element in the UI of an app doesn’t have any strategy by which you can locate it for example resource-id, xpath, class name?
Consider an example when you have an app in which clicking on ellipsis sign shows you more options but there is no way to identify the ellipsis?
Could you think of any other strategy except coordinates?
Any help would be greatly appreciated.
Hi Mohit,
For this case, using co-ordinates seems to be the way out as of now.
If you have access to the dev team you can talk to them about this issue.
Earlier post on a similar issue:
https://qxf2.com/blog/naf-issue-mobile-testing/
Thanks
Vrushali
Hi Vrushali,
Today I came across a situation when I need to press the back soft button which is available in the nexus device.
I used driver.keyevent(4) for this purpose and I was able to do it.
Android version was 4.4.4.
So coming to a conclusion that keyevent is not supported in Appium in Jelly bean and above versions of android would not be a good idea.
However, this may be the case in a soft keyboard as I haven’t tried pressing any key on soft keyboard.
Thanks.
So,
Mohit,
Yes the keyevent for ‘back’ button works fine. We have used this in the code snippet given in our earlier blogpost
The issue highlighted in the current post is for clicking ‘the search icon’.
Oh yeah I see that in this post.
Thanks for providing this information.
Hi there, I’ve been doing a lot of digging on how to use Appium. I happened across this blog post since I couldn’t figure out how to click on suggested search results in an app. So I figured I could use the return key to force to search through. So I was a bit dismayed when I read that you had tried it and found that you needed to use coordinates. Then about an hour later I think I found a way to accomplish pressing the return key. At least it worked for me on an emulator running 4.4.2 – API 19.
driver.deviceKeyEvent(66)
I found a code snippet that was roughly the same as that in the Appium documentation for Key Events here:
http://appium.io/slate/en/v1.3.7-Beta/?javascript#current-context
However, I wasn’t sure why they had wd.SPECIAL_KEYS.Home in the parenthesis. I’m still new to all of this, but I thought it may be helpful for others to know if they happen across this article for the same issue.
Hi Michael,
Thanks for your response. We are using Python here and for us driver.KeyEvent(66) did not work when we tested on actual device for BigBasket app. Does driver.deviceKeyEvent(66) work for you on a actual device?
Regards
Avinash Shetty
We have faced the same issue during our test script creation and resolved it successfully.
Initially when we create the script we used ” driver.sendKeyEvent(66)” and even tried 84 for clicking on search button but it never worked.
Later after doing research we identified the below details,
In the application source when user tap on the search it is used “OnEditorActionListener” to identify the search key pressed and do action based on it, The function definition looks like below
public boolean onEditorAction(TextView v, int actionId, KeyEvent event)
{
— the condition and the action written inside this–
}
Unfortunately the keyevent we are sending will not capture through this listener, So we have added another listener and method to handle that case along with OnEditorActionListener , that is “OnKeyListener” and the method definition looks like below
public boolean onKey(View v, int keyCode, KeyEvent event)
{
— The condition to check for keycode 66 and 84 and if it is satisfied do the search action—
}
This will be used when we send the keyevent from the script and there is no impact on the manual search process.After this the search is working very fine without any issue.
Please note; we havent changed anything in our test script, only change made in the application soruce for search part.
You can check with your app developer to check this case.I hope this will help to resolve the issue.
This is very useful when asking development help to make the product a bit more GUI automation friendly. Testers can now point to your comment when approaching their development team. Thanks so much for the comment, Praveen!
Hi All,
I am new to Appium and using Version 1.4.0.0
and trying to send input to a text field
but unable to do so.
I have tried sendkeys,keycodeevent and normal text enter methods but still unable to do so in Android Version 4.4 the same script is working on lower android versions for the same application.
Can you guys please help me out that how to use Soft Keyboard or Native Phone keyboard to take the input in appium.
Deepesh,Yogendra
Can you please share your code snippet if possible.
In our code :
text_field = self.get_element(self.search_box)
text_field.send_keys(value)
has worked for text input and we used co-ordinates for clicking on the search icon on the soft key board
Hi Deepesh,
Its issue related with android version 4.4 and same issue replicating with my code.
Hopefully we will find answer here.
Hi,
I’m having a problem sending the correct command to an app I’m testing.
The issue is, after entering some text into a text box I need to press done on the keyboard. In a different screen of the app i use the code –
HashMap keycode = new HashMap();
keycode.put(“keycode”, 66);
((JavascriptExecutor)driver).executeScript(“mobile: keyevent”, keycode);
To send the KEYCODE_ENTER.
This works fine for that screen. On the screen thats causing me a problem, however, it does not work.
I looked through the apps code and found it is listening for IME_ACTION_DONE rather than the keycode.
Does anyone know if it is possible to send this command using Appium? And if so any clues as to how?
Thanks in advance.
IME_ACTIONS is what Android recommends developers use. As we stated in the article, we were unable to figure out how to (robustly) automate soft keyboard events that are listening for IME_ACTIONS. We ended up using co-ordinates but that is a lousy and flakey workaround.
If you do find an answer, please post here. I would love to learn how to make our automation more robust in this specific case.
How to scroll down? in appium after applying scrolltoExact() method it is continiously scrolling .please give demo for this
Hi Dhanesh,
You can go through our other post on Python and Appium: Scroll through search result table to see how we used swipe and move_to method to scroll through page.
Did you try to try to:
sendKeys(“someText \\n”);
?
Hi Vic,
We have tried this too but it doesn’t work.