This application is not an official Google product.
Franky is a data driven mobile test automation tool designed for driving applications on Android and iOS. Franky is best suited to projects that need to test stand alone applications by interacting with the applications Graphical User Interface. Because of the nature of GUI testing, Franky is not a good choice for to implement in the presubmit queues of a project build workflow.
Franky can interact with your application and mobile system just as a user can. Consequently anything that can be selected or configured by a user is accessible to a user, including objects that are not part of the application (system settings, user accounts, etc).
Franky is written in Python, and depends on Appium and WebDriverAgent (for iOS device testing).
First, install Appium:
npm install -g appium
Follow their installation instructions for your platform. In particular, pay close attention to iOS requirements. You may need to find the WebDriverAgent copy in the Appium installation and update its WebDriverAgent.xcodeproj
with your provisioning profile and signing credentials.
It is highly recommended to install Franky in a separate virtualenv
environment:
mkdir ~/virtualenv cd ~/virtualenv virtualenv franky source franky/bin/activate cd ~ git clone https://chromium.googlesource.com/chromium/src/tools/franky cd franky ./setup.py install
This will install a new command franky
in ~/virtualenv/franky/bin
. You can test it by running:
franky --help
VPython is a drop-in Python replacement used by Chromium operations to create hermetic Python environments on per-script basis. It relies on the CIPD service.
Make sure cipd
and vpython
executables are in your PATH
. The simplest way to get them is to clone depot_tools
and add it to you PATH
:
cd ~ git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
and add the following line to your ~/.bashrc
file:
PATH="$PATH":"$HOME/depot_tools" export PATH
Alternatively, you can compile both yourself from CIPD and VPython sources and install as you like.
Make sure you have an up-to-date and clean checkout:
git pull git status # There should be no modified or untracked files
Upload a new Franky Python wheel to CIPD (you need to have enough permissions in the CIPD package ACLs; view ACLs by running cipd acl-list infra/python/wheels/franky-py2
):
cipd auth-login # Do this once per machine make release
Create an executable shim script franky
:
#!/usr/bin/env vpython import franky.runner if __name__ == '__main__': franky.runner.main()
and accompany it with the VPython spec file franky.vpython
containing all the Python dependencies, and in particular, a reference to the Franky package:
wheel { name: "infra/python/wheels/franky-py2" version: "git_revision:deadbeefe66bf36e0d50eaaba80ff3d538ba5716" }
See test.py and test.py.vpython for an example.
Franky consists of 4 main components:
Appium: The Test automation framework on top of which Franky is built. Appium is an open source test automation framework developed and supported by Sauce Labs to automate native and hybrid mobile apps. It uses JSON wire protocol internally to interact with iOS and Android native apps using the Selenium WebDriver which is a testing framework for web apps. Appium is designed to encourage a 2-tier architecture: a machine runs the test written in one language (python in this case) and another one (the test server) actually executes it. Furthermore the WebDriver protocol targets scalability (because based on HTTP), which makes Appium very scalable as well.
Franky Core: The core component of Franky, it is a client to the appium server.
Dashboard: The web interface that displays test results output.
Data Source: The UI interface used to construct tests to be automated.
Franky is architectured in such a way that the test data or test plan is completely separated from the script that executes it. This gives the ability to any non technical person to use the system at ease to automate tests.
runner.py
This is the entry point and the main orchestra of Franky. It is the mediator between different components of the system. After getting all global constants including the path to the app to automate and where to locate appium server, runner.py module reads test suites from test_data_reader.py
, gets needed app under test info from app_info.py
and device info from device_info.py
. Runner.py
also instantiates an object driver_provider.py
which will be passed along to each step action method providing here a way to access the Appium driver. The _RunTestSuite
method in runner.py
is responsible for creating unittest test suites where each test is going to be executed as a unittest test with a SetUp
and tearDown
methods. The test method testFrankySteps
maps each action in the test step to its corresponding type and executes the test step. After all the actions are done executing in a suite, runner.py
sends the result to generate_report.py
as a json file.
app_info.py
This script provides the needed information about the application under test. It will gather the information and store it, which includes the name of the applications, the platform, and some version information.
device_info.py
Gathers the needed device information to run tests against it.
driver_provider.py
This class provides a property called driver by returning a lazy initialized instance of DriverProvider
class. This driver is used to run all tests and therefore executeall action methods.
step.py
This module provides all of the needed framework to support a test step. Each test step is composed of a description
, action_type
, action
, by
(name of the first element locator), path
(name of the 2nd element locator), value
, duration
, start_coordinate
, end_coordinate
, device_type
, and device_versions
.
test_data_reader.py
This module automatically reads test suites from csv files in Google Sheet. It also validates each test step according to its device type. A valid test step is that which does not have a device type attribute or if given, matches the device type of the connected device. A non valid test step is that whose device type is different from the device type of the connected device. If a non valid test step action is StartTest
, then the entire set of test steps below that invalid test step will also be ignored until the next test step with action StartTest
is found, marking here the beginning of a new test. If the first step in a test case is not StartTest
then all succeeding test steps are going to be ignored until a step with StartTest
is found.
reports
foldergenerate_report.py
Generates a JSON test report from all of the test results.
html_report.py
Will take all of the test results and write a HTML test result report.
html_report_template.py
The template used by html_report
to generate the html report.
test_util
folderThe test_util
folder contains all of the modules that contains logic of implementation Franky test cases and test suites.
franky_testcase.py
It is a wrapper over unittest.TestCase
that allows to execute test steps. In case of failure franky_testcase
tries to recover test infrastructure with Android/iOS specific. Also franky_testcase
allows to add additional steps before/after test.
franky_testsuite.py
unittest.TestSuite
is a skeleton of Franky that parses test cases from csv files(s) and execute test cases. Test suite supports.
appium_util
FolderThe appium_util
folder contains all of the modules that directly pertain to the Appium product.
appium_driver_util.py
This is the module that starts Appium Server and also create Appium driver. The only client to this module is driver_provider.py
and the only way to get or rest the driver is therefore through driver_provider.py
. All the Appium desired capabilities all well defined in appium_driver_utils.py
among which the app path, launchTimeout
, newCommandTimeout
, waitForAppScript
, backendRetries
etc.
input_formatter.py
Provides methods to format objects before they are input.
path_constants.py
Provides path constants that are used across different modules.
timeout_util.py
Provides ways to manipulate the driver timeouts.
actions
FolderIn the actions folder, resides all type of actions files which each implements a specific user action. An action is the process of doing something passive or active by the user on the screen such as tapping a button, installing an app, or verifying that a button is present or not, typically to achieve an aim which is to make sure there is success. Every action corresponds to a method with arguments driver_provider
and a Step
tuple.
Here are the different types of actions:
alert.py
This module is responsible for handling alerts or pop-ups during test execution. This includes in app alerts, local notification, battery warning and privacy access permission alerts such as location, contacts and photos. System update alerts are not handled here. By default Appium does not handle any alert on the screen and Franky dismisses all alerts unless told not to do so. AutoDismissAlerts
and AutoAcceptAlerts
are methods in this module.
element.py
Implements all non interactive actions only used to verify or compare: VerifyElementIsPresent
, VerifyElementIsAbsent
, VerifyElementIsDisplayed
, VerifyElementIsHidden
, VerifyElementIsEnabled
, VerifyElementIsDisabled
, VerifyElementIsSelected
, VerifyElementNameContains
, VerifyElementNameEquals
, VerifyElementValueContains
, VerifyElementValueEquals
, VerifyElementLabelContains
, VerifyElementLabelEquals
.
franky.py
This module provides a common interface for creating action objects.
menu.py
This provides actions to open and close Chrome tabs and menus.
omnibox.py
For all omnibox related actions: SearchFromOmnibox
, SearchFromOmniboxIncognito
, VerifyOmniboxURLContains
, VerifyOmniboxURLEquals
, VerifyIncognitoOmniboxURLContains
, VerifyIncognitoOmniboxURLEquals
, LoadCurrentURLFromOmnibox
, LoadCurrentURLFromOmniboxIncognito
.
onboarding.py
Provides all onboarding related actions.
scroll.py
For all scrolling, swiping and dragging related actions like: Overscroll
, DragInsideWithOptions
, DragFromToForDuration
, ScrollToVisible
, ScrollDownToElement
, ScrollUpToElement
. Swipe
: Swipes the screen from one direction to the opposite. Direction is passed as a value of the step. So if Swipe has value Left, it will swipe from right to left and vice versa. In some instances, the user could want to swipe starting or ending in a specific place of the screen like swiping left but starting from the omnibox, the x_coordinate
or y_coordinate
should therefore be given. For example if step.y_coordinate
is 0.5 and step.value
is Left
, this method will swipe from right to left and starts from the middle of the screen. Coordinates are optional for Swipe
and if they are omitted then SwipeLeft
will start from the right edge, SwipeUp
will start from the bottom edge, etc.. An example of a use case here is the action to SwipeLeft
from the omnibox to go to another tab. This action will specify the omnibox location to SwipeLeft
with a start_coordinate of (1, 0.5).
settings.py
For all settings related actions: RemoveAccounts
, AddAccount
, and ClearBrowsingData
.
system.py
For all system or app related actions like: InstallApp
, RemoveApp
, LaunchApp
, CloseApp
, ResetAppiumSession
, ReinstallApp
, ReinstallAndLaunchApp
, DesactivateAppForDuration
, SetDeviceOrientation
, VerifyDeviceOrientation
, EnableWiFi
, LaunchSplitViewApp
, ReinstallAndLaunchAndByPassFirstRun
.
tap.py
For all tapping related actions like: Tap
, DoubleTap
, TapIfPresent
, TapHiddenElement
, TouchAndHold
, TapWithOptions
: Performs the specified gesture on the specified element using a dictionary to specify gesture attributes. You can use offsets to achieve finer precision in specifying the hitpoint within the rect for the specified element. The offset comprises a pair of x
and y
values, each ranging from 0.0 to 1.0. These values represent, respectively, relative horizontal and vertical positions within the rect, with {x:0.0, y:0.0}
as the top left and {x:1.0, y:1.0}
as the bottom right. Thus, {x:0.3, y:0.6}
specifies a position just below and to the left of center, and {x:1.0, y:0.5}
specifies a position centered vertically at the far right.
timeout.py
This module provides ways to manipulates the driver timeouts. It is very important to note that as well as supporting Selenium webdriver elements, Appium also supports UIAlements
and there are some differences in the way these 2 types of elements are handled. Handling timeout for UIAlement
is done right after the app is launched and PushTimeout()
method will push a timeout to wait for a UIAlement
to be present before failing. This timeout will only affect all UIAelements
during the test and will not affect Selenium webdriver elements. Selenium webdriver timeout to wait for an element to be present before failing is only set when finding an element and is done individually for each Selenium webdriver element using the selenium.webdriver.support.expect_conditions
module.
user_input.py
Provides all user input related actions like inputting characters.
utilities
Folderutilities
contains common helpers that allows to download/upload reports and test suites, store and delete temporary files, run external process, filter and collect logcat data from Android device.
Appium throws Selenium errors (NoSuchElementException
, TimeoutException
, WebDriverException
, etc…) when an element is not found on the screen or is non interactive, like a hidden element. Franky catches these exceptions in order to recover from the test failure (make sure wi-fi is ON and by-pass first run), leaving the app in the appropriate state for the next test to run.
ActionError.py
is Franky’s base class for handling Selenium errors thrown by Appium such that an ActionError
is thrown instead of Selenium
error. VerificationError
, TapError
, NoElementFoundError
are subclasses of ActionError
and each error is thrown when an expected condition is unmet.
Test cases data provider has ten attributes:
Description: This is used to give test case as well as a test step a description. It describes in a very brief and succinct way what the test step does in a human friendly language. An example will be: “Tap on the Back button to return to the home screen”. A test step description could be null but it is very recommended to always. This description is later used in the html report after the steps have been executed.
Action type: Type of action to use for the test step. Any action belongs to a category based on its function and the way it interacts with the app. tap, scroll, sendkeys, verify, etc... are examples of types of actions.
Action: Test step action is the actual Appium command to be sent to the Appium server. An action is encapsulated in a method that calls Appium webdriver for its execution. DoubleTap
, Type
, ScrollUpToElement
, VerifyPresenceOfElement
are example of actions.The list of all actions from Actions.py
is mapped to this field and serves as a data validator. Any action used to build a test should be part of the list of these actions.
By: This field is the first element of the locator tuple. This field may or may not be required depending on the type of action used. In Appium, elements can be found on the screen by id
, xpath
, uiautomation
, classname
, etc...
Path: This field is the second element of the locator tuple. A path could be the accessibility label or accessibility identifier of an element. This field is only required if by
field is provided.
Value: Test step value. The value is one of the attributes of an element and could be the text that populates a text field. A test step value depends on the type of actions used and can just serve as an argument to an action method. This attribute depends may or may not be compulsory depending of the type of action used.
Duration: This field is the time in seconds a test step action needs to accomplish a specific task. For example DeactivateAppForDuration
action uses duration field as the time for the app to remain inactive. A test step duration field may or may not be required depending on the type of action used.
start_coordinate: This field is the horizontal value in a pair of coordinates used to find a point in an element on the screen. A test step start_coordinate field may or may not be required depending on the type of action used. Coordinate range is [0.0;1.0], where 0.0 is left and 1.0 is right.
end_coordinate: This field is the vertical value in a pair of coordinates used to find a point in an element on the screen. A test step end_coordinate field may or may not be required depending on the type of action used. Coordinate range is [0.0;1.0], where 0.0 is top and 1.0 is bottom.
Device Type: A string used to disable an entire test case or a set of test steps. If given at the beginning of a test case, the test case will not be executed on a device configuration different from the one specified. If used on a test step, that test step will not be executed. A test case that has a blank device type attribute will run on all devices.
Device Versions: A list of strings delimited by a comma for all the device versions a test can be executed on. If specified at the beginning of a test case, the test will not be executed on a device whose version is different from the one(s) listed. A test case that has a blank device version attribute will run on all device versions.
Each line in the data provider Spreadsheet represents a test step (Step tuple made up of all the 10 attributes above). The beginning of a new test case is tagged with StartTest
. A test case is composed of one or more test steps and any lines below StartTest
is the set of test steps that is part of that test until the next line containing StartTest
is encountered. A test Suite is composed of many test cases and is also represented by an entire sheet in the Data Provider Spreadsheet.
Generate_report.py
module is responsible for generating a report in a json or html format out of the test results data at the end of each test suite run. If HTML, the report will be generated by using html_report.py which will in turn use the structure of the HTML template which resides in html_report_template.py
. Tests results are generated after each run in a form of a list that is passed to the report module which processes the data. An HTML report will be generated by default.