README.md

# Franky: Android and iOS Test Automation Tool

_This application is not an official Google product._

Franky is a data driven mobile test automation tool designed for driving
applications on Android and iOS. Franky is best suited to projects that need to
test stand alone applications by interacting with the applications Graphical
User Interface. Because of the nature of GUI testing, Franky is not a good
choice for to implement in the presubmit queues of a project build workflow.

Franky can interact with your application and mobile system just as a user can.
Consequently anything that can be selected or configured by a user is accessible
to a user, including objects that are not part of the application (system
settings, user accounts, etc).

Franky is written in Python, and depends on [Appium] and
[WebDriverAgent] (for iOS device testing).

[Appium]: https://github.com/appium/appium
[WebDriverAgent]: https://github.com/facebook/WebDriverAgent

[TOC]

## Installation

First, install [Appium]:

    npm install -g appium

Follow their installation instructions for your platform. In
particular, pay close attention to iOS requirements. You may need to
find the [WebDriverAgent] copy in the Appium installation and update
its `WebDriverAgent.xcodeproj` with your provisioning profile and
signing credentials.

It is highly recommended to install Franky in a separate `virtualenv`
environment:

    mkdir ~/virtualenv
    cd ~/virtualenv
    virtualenv franky
    source franky/bin/activate
    cd ~
    git clone https://chromium.googlesource.com/chromium/src/tools/franky
    cd franky
    ./setup.py install

This will install a new command `franky` in
`~/virtualenv/franky/bin`. You can test it by running:

    franky --help

### Installation using VPython

[VPython] is a drop-in Python replacement used by Chromium operations to create
hermetic Python environments on per-script basis. It relies on the [CIPD] service.

Make sure `cipd` and `vpython` executables are in your `PATH`. The simplest way
to get them is to clone
[`depot_tools`](https://chromium.googlesource.com/chromium/tools/depot_tools.git)
and add it to you `PATH`:

    cd ~
    git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git

and add the following line to your `~/.bashrc` file:

    PATH="$PATH":"$HOME/depot_tools"
    export PATH

Alternatively, you can compile both yourself from [CIPD] and [VPython] sources and install as
you like.

Make sure you have an up-to-date and clean checkout:

    git pull
    git status  # There should be no modified or untracked files

Upload a new Franky Python wheel to CIPD (you need to have enough permissions in
the CIPD package ACLs; view ACLs by running `cipd acl-list
infra/python/wheels/franky-py2`):

    cipd auth-login  # Do this once per machine
    make release

Create an executable shim script `franky`:

```python
#!/usr/bin/env vpython

import franky.runner

if __name__ == '__main__':
  franky.runner.main()
  ```

and accompany it with the [VPython] spec file `franky.vpython` containing all
the Python dependencies, and in particular, a reference to the Franky package:

```
wheel {
  name: "infra/python/wheels/franky-py2"
  version: "git_revision:deadbeefe66bf36e0d50eaaba80ff3d538ba5716"
}
```

See [test.py](./test.py) and [test.py.vpython](./test.py.vpython) for an example.

[VPython]: https://chromium.googlesource.com/infra/luci/luci-go/+/master/vpython/
[CIPD]: https://chromium.googlesource.com/infra/luci/luci-go/+/master/cipd/

## Components

Franky consists of 4 main components:

__Appium:__ The Test automation framework on top of which Franky is built. Appium is
an open source test automation framework developed and supported by Sauce Labs
to automate native and hybrid mobile apps. It uses JSON wire protocol internally
to interact with iOS and Android native apps using the Selenium WebDriver which
is a testing framework for web apps. Appium is designed to encourage a 2-tier
architecture: a machine runs the test written in one language (python in this
case) and another one (the test server) actually executes it. Furthermore the
WebDriver protocol targets scalability (because based on HTTP), which makes
Appium very scalable as well.

__Franky Core:__ The core component of Franky, it is a client to the appium server.

__Dashboard:__ The web interface that displays test results output.

__Data Source:__ The UI interface used to construct tests to be automated.

Franky is architectured in such a way that the test data or test plan is
completely separated from the script that executes it. This gives the ability to
any non technical person to use the system at ease to automate tests.


## FRANKY MODULES

### `runner.py`

This is the entry point and the main orchestra of Franky. It is the
mediator between different components of the system. After getting all global
constants including the path to the app to automate and where to locate appium
server, runner.py module reads test suites from `test_data_reader.py`, gets needed
app under test info from `app_info.py` and device info from `device_info.py`.
`Runner.py` also instantiates an object `driver_provider.py` which will be passed
along to each step action method providing here a way to access the Appium
driver. The `_RunTestSuite` method in `runner.py` is responsible for creating
unittest test suites where each test is going to be executed as a unittest test
with a `SetUp` and `tearDown` methods. The test method `testFrankySteps` maps each
action in the test step to its corresponding type and executes the test step.
After all the actions are done executing in a suite, `runner.py` sends the result
to `generate_report.py` as a json file.

### `app_info.py`

This script provides the needed information about the application under test. It
will gather the information and store it, which includes the name of the
applications, the platform, and some version information.

### `device_info.py`

Gathers the needed device information to run tests against it.

### `driver_provider.py`

This class provides a property called driver by returning a lazy initialized
instance of `DriverProvider` class. This driver is used to run all tests and
therefore executeall action methods.

### `step.py`

This module provides all of the needed framework to support a test step. Each
test step is composed of a `description`, `action_type`, `action`, `by` (name of the
first element locator), `path` (name of the 2nd element locator), `value`, `duration`,
`start_coordinate`, `end_coordinate`, `device_type`, and `device_versions`.

### `test_data_reader.py`

This module automatically reads test suites from csv files in Google Sheet. It
also validates each test step according to its device type. A valid test step is
that which does not have a device type attribute or if given, matches the device
type of the connected device. A non valid test step is that whose device type is
different from the device type of the connected device. If a non valid test step
action is `StartTest`, then the entire set of test steps below that invalid test
step will also be ignored until the next test step with action `StartTest` is
found, marking here the beginning of a new test. If the first step in a test
case is not `StartTest` then all succeeding test steps are going to be ignored
until a step with `StartTest` is found.

## `reports` folder

### `generate_report.py`

Generates a JSON test report from all of the test results.

### `html_report.py`

Will take all of the test results and write a HTML test result report.

### `html_report_template.py`

The template used by `html_report` to generate the html report.

## `test_util` folder

The `test_util` folder contains all of the modules that contains logic of
implementation Franky test cases and test suites.

### `franky_testcase.py`

It is a wrapper over `unittest.TestCase` that allows to execute test steps.
In case of failure `franky_testcase` tries to recover test infrastructure
with Android/iOS specific. Also `franky_testcase` allows to add additional
steps before/after test.

### `franky_testsuite.py`

`unittest.TestSuite` is a skeleton of Franky that parses test cases from
csv files(s) and execute test cases. Test suite supports.


## `appium_util` Folder

The `appium_util` folder contains all of the modules that directly pertain to the
Appium product.

### `appium_driver_util.py`

This is the module that starts Appium Server and also create Appium driver. The
only client to this module is `driver_provider.py` and the only way to get or rest
the driver is therefore through `driver_provider.py`. All the Appium desired
capabilities all well defined in `appium_driver_utils.py` among which the app
path, `launchTimeout`, `newCommandTimeout`, `waitForAppScript`, `backendRetries` etc.

### `input_formatter.py`

Provides methods to format objects before they are input.

### `path_constants.py`

Provides path constants that are used across different modules.

### `timeout_util.py`

Provides ways to manipulate the driver timeouts.


## `actions` Folder

In the actions folder, resides all type of actions files which each implements a
specific user action. An action is the process of doing something passive or
active by the user on the screen such as tapping a button, installing an app, or
verifying that a button is present or not, typically to achieve an aim which is
to make sure there is success. Every action corresponds to a method with
arguments `driver_provider` and a `Step` tuple.

Here are the different types of actions:

### `alert.py`

This module is responsible for handling alerts or pop-ups during test execution.
This includes in app alerts, local notification, battery warning and privacy
access permission alerts such as location, contacts and photos. System update
alerts are not handled here. By default Appium does not handle any alert on the
screen and Franky dismisses all alerts unless told not to do so.
`AutoDismissAlerts` and `AutoAcceptAlerts` are methods in this module.

### `element.py`

Implements all non interactive actions only used to verify or compare:
`VerifyElementIsPresent`, `VerifyElementIsAbsent`,
`VerifyElementIsDisplayed`, `VerifyElementIsHidden`, `VerifyElementIsEnabled`,
`VerifyElementIsDisabled`, `VerifyElementIsSelected`, `VerifyElementNameContains`,
`VerifyElementNameEquals`, `VerifyElementValueContains`, `VerifyElementValueEquals`,
`VerifyElementLabelContains`, `VerifyElementLabelEquals`.

### `franky.py`

This module provides a common interface for creating action objects.

### `menu.py`

This provides actions to open and close Chrome tabs and menus.

### `omnibox.py`

For all omnibox related actions: `SearchFromOmnibox`,
`SearchFromOmniboxIncognito`, `VerifyOmniboxURLContains`, `VerifyOmniboxURLEquals`,
`VerifyIncognitoOmniboxURLContains`, `VerifyIncognitoOmniboxURLEquals`,
`LoadCurrentURLFromOmnibox`, `LoadCurrentURLFromOmniboxIncognito`.

### `onboarding.py`

Provides all onboarding related actions.

### `scroll.py`

For all scrolling, swiping and dragging related actions like:
`Overscroll`, `DragInsideWithOptions`, `DragFromToForDuration`, `ScrollToVisible`,
`ScrollDownToElement`, `ScrollUpToElement`. `Swipe`: Swipes the screen from one
direction to the opposite. Direction is passed as a value of the step. So if
Swipe has value Left, it will swipe from right to left and vice versa. In some
instances, the user could want to swipe starting or ending in a specific place
of the screen like swiping left but starting from the omnibox, the `x_coordinate`
or `y_coordinate` should therefore be given. For example if `step.y_coordinate` is
0.5 and `step.value` is `Left`, this method will swipe from right to left and
starts from the middle of the screen. Coordinates are optional for `Swipe` and if
they are omitted then `SwipeLeft` will start from the right edge, `SwipeUp` will
start from the bottom edge, etc.. An example of a use case here is the action to
`SwipeLeft` from the omnibox to go to another tab. This action will specify the
omnibox location to `SwipeLeft` with a start_coordinate of (1, 0.5).

### `settings.py`

For all settings related actions: `RemoveAccounts`, `AddAccount`, and
`ClearBrowsingData`.

### `system.py`

For all system or app related actions like: `InstallApp`, `RemoveApp`,
`LaunchApp`, `CloseApp`, `ResetAppiumSession`, `ReinstallApp`, `ReinstallAndLaunchApp`,
`DesactivateAppForDuration`, `SetDeviceOrientation`, `VerifyDeviceOrientation`,
`EnableWiFi`, `LaunchSplitViewApp`, `ReinstallAndLaunchAndByPassFirstRun`.

### `tap.py`

For all tapping related actions like: `Tap`, `DoubleTap`, `TapIfPresent`,
`TapHiddenElement`, `TouchAndHold`, `TapWithOptions`: Performs the specified gesture
on the specified element using a dictionary to specify gesture attributes. You
can use offsets to achieve finer precision in specifying the hitpoint within the
rect for the specified element. The offset comprises a pair of `x` and `y` values,
each ranging from 0.0 to 1.0. These values represent, respectively, relative
horizontal and vertical positions within the rect, with `{x:0.0, y:0.0}` as the
top left and `{x:1.0, y:1.0}` as the bottom right. Thus, `{x:0.3, y:0.6}` specifies
a position just below and to the left of center, and `{x:1.0, y:0.5}` specifies a
position centered vertically at the far right.

### `timeout.py`

This module provides ways to manipulates the driver timeouts. It is very
important to note that as well as supporting Selenium webdriver elements, Appium
also supports `UIAlements` and there are some differences in the way these 2 types
of elements are handled. Handling timeout for `UIAlement` is done right after the
app is launched and `PushTimeout()` method will push a timeout to wait for a
`UIAlement` to be present before failing. This timeout will only affect all
`UIAelements` during the test and will not affect Selenium webdriver elements.
Selenium webdriver timeout to wait for an element to be present before failing
is only set when finding an element and is done individually for each Selenium
webdriver element using the `selenium.webdriver.support.expect_conditions` module.

### `user_input.py`

Provides all user input related actions like inputting characters.

## `utilities` Folder

`utilities` contains common helpers that allows to download/upload reports and
test suites, store and delete temporary files, run external process, filter
and collect logcat data from Android device.


## ERROR HANDLING

Appium throws Selenium errors (`NoSuchElementException`, `TimeoutException`,
`WebDriverException`, etc…) when an element is not found on the screen or is non
interactive, like a hidden element. Franky catches these exceptions in order to
recover from the test failure (make sure wi-fi is ON and by-pass first run),
leaving the app in the appropriate state for the next test to run.

`ActionError.py` is Franky’s base class for handling Selenium errors thrown by
Appium such that an `ActionError` is thrown instead of `Selenium` error.
`VerificationError`, `TapError`, `NoElementFoundError` are subclasses of `ActionError`
and each error is thrown when an expected condition is unmet.

## DATA PROVIDER

Test cases data provider has ten attributes:

1. __Description:__ This is used to give test case as well as a test step a
description. It describes in a very brief and succinct way  what the test step
does in a human friendly language. An example will be: “Tap on the Back button
to return to the home screen”. A test step description could be null but it is
very recommended to always. This description is later used in the html report
after the steps have been executed.

2. __Action type:__ Type of action to use for the test step. Any action belongs to a
category  based on its function and the way it interacts with the app. tap,
scroll,  sendkeys, verify, etc... are examples of types of actions.

3. __Action:__ Test step action is the actual Appium command to be sent to the
Appium server. An action is encapsulated in a method that calls Appium webdriver
for its execution. `DoubleTap`, `Type`, `ScrollUpToElement`, `VerifyPresenceOfElement`
are example of actions.The list of all actions from `Actions.py` is mapped to this
field and serves as a data validator. Any action used to build a test should be
part of the list of these actions.

4. __By:__ This field is the first element of the locator tuple.  This field may or
may not be required depending on the type of action used. In Appium, elements
can be found on the screen by `id`, `xpath`, `uiautomation`, `classname`, etc...

5. __Path:__ This field is the second element of the locator tuple. A path could be
the accessibility label or accessibility identifier of an element. This field is
only required if `by` field is provided.

6. __Value:__ Test step value. The value is one of the attributes of an element and
could be the text that populates a text field. A test step value depends on the
type of actions used and can just serve as an argument to an action method. This
attribute depends may or may not be compulsory depending of the type of action
used.

7. __Duration:__ This field is the time in seconds a test step action needs to
accomplish a specific task. For example `DeactivateAppForDuration` action uses
duration field as the time for the app to remain inactive. A test step duration
field may or may not be required depending on the type of action used.

8. __start\_coordinate:__ This field is the horizontal value in a pair of coordinates
used to find a point in an element on the screen. A test step start_coordinate
field may or may not be required depending on the type of action used.
Coordinate range is [0.0;1.0], where 0.0 is left and 1.0 is right.

9. __end\_coordinate:__ This field is the vertical value in a pair of coordinates
used to find a point in an element on the screen. A test step end_coordinate
field may or may not be required depending on the type of action used.
Coordinate range is [0.0;1.0], where 0.0 is top and 1.0 is bottom.

10. __Device Type:__ A string used to disable an entire test case or a set of test
steps. If given at the beginning of a test case, the test case will not be
executed on a device configuration different from the one specified. If used on a
test step, that test step will not be executed. A test case that has a blank
device type attribute will run on all devices.

11. __Device Versions:__  A list of strings delimited by a comma for all the device
versions a test can be executed on. If specified at the beginning of a test case,
the test will not be executed on a device whose version is different from the
one(s) listed. A test case that has a blank device version attribute will run on
all device versions.

Each line in the data provider Spreadsheet represents a test step (Step tuple
made up of all the 10 attributes above). The beginning of a new test case is
tagged with `StartTest`. A test case is composed of one or more test steps and
any lines below `StartTest` is the set of test steps that is part of that test
until the next line containing `StartTest` is encountered. A test Suite is
composed of many test cases and is also represented by an entire sheet in the
Data Provider Spreadsheet.

## REPORT

`Generate_report.py` module is responsible for generating a report in a json or
html format out of the test results data at the end of each test suite run. If
HTML, the report will be generated by using html_report.py which will in turn
use the structure of the HTML template which resides in `html_report_template.py`.
Tests results are generated after each run in a form of a list that is passed to
the report module which processes the data. An HTML report will be generated by
default.