| # Chrome Accessibility on Android | |
| This document covers some of the technical details of how Chrome | |
| implements its accessibility support on Android. | |
| Chrome plays an important role on Android - not only is it the default | |
| browser, but Chrome powers WebView, which is used by many built-in and | |
| third-party apps to display all sorts of content. Android includes a lightweight | |
| implementation called Chrome Custom Tabs, which is also powered by Chrome. | |
| All of these implementations must be accessible, and the Chrome & Chrome OS Accessibility | |
| team provides the support to make these accessibility through the Android API. | |
| Accessibility on Android is heavily used. There are many apps that hijack the | |
| Android accessibility API to act on the user's behalf (e.g. password managers, | |
| screen clickers, anti-virus software, etc). Because of this, roughly **16%** of all | |
| Android users are running the accessibility code (even if they do not realize it). | |
| As background reading, you should be familiar with | |
| [Android Accessibility](https://developer.android.com/guide/topics/ui/accessibility) | |
| and in particular | |
| [AccessibilityNodeInfo](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo) | |
| objects, [AccessibilityEvent](https://developer.android.com/reference/android/view/accessibility/AccessibilityEvent) and | |
| [AccessibilityNodeProvider](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeProvider). | |
| ## WebContentsAccessibility | |
| The main Java class that implements the accessibility protocol in Chrome is | |
| [WebContentsAccessibilityImpl.java](https://cs.chromium.org/chromium/src/content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java). This | |
| class acts as the AccessibilityNodeProvider (see above) for a given tab, and will | |
| provide the virtual tree hierarchy, preform actions on the user's behalf, and | |
| send events to downstream services for changes in the web contents. | |
| This class differs in a few key ways from other platforms. First, it represents | |
| and entire web page, including all frames. The ids in the java code are unique IDs, | |
| not frame-local IDs. They are typically referred to as `virtualViewId` in the code | |
| and Android framework/documentation. Another key difference is the construction of | |
| the native objects for nodes. On most platforms, we create a native object for every | |
| AXNode in a web page, and we implement a bunch of methods on that object that assistive | |
| technology can query. Android is different - it's more lightweight in one way, in that we only | |
| create a native AccessibilityNodeInfo object when specifically requested, when | |
| an Android accessibility service is exploring the virtual tree. In another | |
| sense it's more heavyweight, though, because every time a virtual view is | |
| requested we have to populate it with every possible accessibility attribute, | |
| and there are quite a few. | |
| ### WebContentsAccessibilityImpl is "lazy" and "on demand" | |
| Every Tab in the Chrome Android browser will have its own instance of | |
| WebContentsAccessibilityImpl. The WebContentsAccessibilityImpl object is created | |
| using a static Factory method with a parameter of the WebContents object for | |
| that tab. See [constructor](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22protected%20WebContentsAccessibilityImpl%22). | |
| (Note: There are a few exceptions to this pattern, for example when constructing | |
| the object without access to a WebContents instance, such as in the case of PaintPreview.) | |
| Although the WebContentsAccessibilityImpl object has been constructed (and | |
| technically instantiated), it will not do anything until an accessibility service | |
| is active and queries the system. The base class of Java widgets [View.java](https://developer.android.com/reference/android/view/View), | |
| has a method [getAccessibilityNodeProvider](https://developer.android.com/reference/android/view/View#getAccessibilityNodeProvider\(\)). Custom views, such as the [ContentView.java](https://source.chromium.org/chromium/chromium/src/+/main:components/embedder_support/android/java/src/org/chromium/components/embedder_support/view/ContentView.java) | |
| class in Chrome (which holds all the web contents for a single Tab), can override | |
| this method to return an instance of AccessibilityNodeProvider. If an app returns | |
| its own instance of AccessibilityNodeProvider, then AT will leverage this instance | |
| when querying the view hierarchy. The WebContentsAccessibilityImpl acts as Chrome's | |
| custom instance of AccessibilityNodeProvider so that it can serve the virtual view | |
| hierarchy of the web to the native accessibility framework. | |
| The first time that `getAccessibilityNodeProvider` is called by the Android system, | |
| the WebContentsAccessibilityImpl will be initialized. This is why we consider it | |
| "lazy" and "on demand", because although it has technically been constructed and | |
| instantiated, it does not perform any actions until AT triggered its initialization. | |
| See [WebContentsAccessibilityImpl#getAccessibilityNodeProvider](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22public%20AccessibilityNodeProvider%20getAccessibilityNodeProvider%22) and the | |
| associated [onNativeInit](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22protected%20void%20onNativeInit%22) methods. The getAccessibilityNodeProvider method | |
| will only be called when an accessibility service is enabled, and so by lazily | |
| constructing only after this call, we ensure that the accessibility code is not | |
| being leveraged for users without any services enabled. | |
| Once initialized, the WebContentAccessibilityImpl plays a part in handling nearly | |
| all accessibility related code on Android. This object will be where AT sends | |
| actions performed by users, it constructs and serves the virtual view hierarchy, | |
| and it dispatches events to AT for changes in the web contents. The | |
| WebContentsAccessibilityImpl object has the same lifecycle as the Tab for which | |
| it was created, and although it won't fire events or serve anything to downstream | |
| AT if the tab is backgrounded/hidden, the object will continue to exist and will | |
| not be destroyed until the Tab is destroyed/closed. | |
| ## AccessibilityNodeInfo | |
| The [AccessibilityNodeInfo](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo) | |
| object is at the core of Android accessibility. This is a rather heavy object | |
| which holds every attribute for a given node (a virtual view element) as defined | |
| by the Android API. (Note: The Android accessibiltiy API has different attributes/standards | |
| than the web or other platforms, so there are many special cases and considerations, | |
| more on that below). | |
| As an AccessibilityNodeProvider, the WebContentsAccessibilityImpl class must | |
| override/implement the [createAccessibilityNodeInfo](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeProvider#createAccessibilityNodeInfo\(int\)) method. This is the | |
| method that the accessibility framework will query on behalf of AT to understand | |
| the current virtual view hierarchy. On other platforms, the native-side code may | |
| contain the entire structure of the web contents in native objects, but on Android | |
| the objects are created "on demand" as requested by the framework, and so they are | |
| typically generated synchronously on-the-fly. | |
| The information to populate the AccessibilityNodeInfo objects is contained in | |
| the accessibility tree in the C++ code in the shared BrowserAccessibilityManager. | |
| For Android there is the usual BrowserAccessibilityManagerAndroid, and the | |
| BrowserAccessibilityAndroid classes, as expected, but there is also an additional | |
| [web\_contents\_accessibility\_android.cc](https://cs.chromium.org/chromium/src/content/browser/accessibility/web_contents_accessibility_android.cc) class. | |
| This class is what allows us to connect the Java-side WebContentsAccessibilityImpl | |
| with the C++ side manager, through the Java Native Interface (JNI). | |
| When [WebContentsAccessibilityImpl#createAccessibilityNodeInfo](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22public%20AccessibilityNodeInfoCompat%20createAccessibilityNodeInfo%22) is called for | |
| a given virtual view (web node), the WebContentsAccessibilityImpl object calls into the native | |
| C++ code through JNI, connecting to `web_contents_accessibility_android.cc`. The | |
| web\_contents\_accessibility\_android object in turn compiles information about the | |
| requested node from BrowserAccessibilityAndroid and BrowserAccessibilityManagerAndroid | |
| and then calls back into WebContentsAccessibilityImpl, again through the JNI, to | |
| send this information back to the Java-side to be populated into the | |
| `AccessibilityNodeInfo` object that is being constructed. | |
| These roundtrips across the JNI come with an inherent cost. It is minuscule, but for | |
| thousands of nodes on a page, each with 25+ attributes, it would be too costly to | |
| make so many trips. However, passing all the attributes in one giant function call | |
| is also not ideal. We try to strike a balance by grouping like attributes together | |
| (e.g. all boolean attributes) into a single JNI trip, and make just a few JNI | |
| trips per AccessibilityNodeInfo object. These trips can be found in the | |
| [WebContentsAccessibilityAndroid::PopulateAccessibilityNodeInfo](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/web_contents_accessibility_android.cc?q=%22WebContentsAccessibilityAndroid::PopulateAccessibilityNodeInfo%22) method, | |
| which is called from WebContentsAccessibilityImpl#createAccessibilityNodeInfo, and | |
| is the core method for compiling a node's attributes and passing them to the Java-side. | |
| ### Java-side caching mechanism | |
| One of the most significant performance optimizations in the Android accessibility | |
| code is the addition of a caching mechanism for the Java-side AccessibilityNodeInfo | |
| objects. The cache is built as a simple `SparseArray` of AccessibilityNodeInfo objects. | |
| We use a SparseArray instead of a HashMap because on Java the HashMap requires | |
| Objects for both the key and value, and ideally we would use the `virtualViewId` | |
| of any given node as the key, and this ID is an int (primitive type) in Java. So the | |
| SparseArray is more light weight and is as efficient as using the HashMap in this | |
| case. The array contains AccessibilityNodeInfo objects at the index of the node's | |
| corresponding `virtualViewId`. If an invalid ID is requested, `null` is returned. | |
| In WebContentsAccessibilityImpl's implementation of createAccessibilityNodeInfo, | |
| the cache is queried first, and if it contains a cached version of an object for | |
| this node, then we update that reference and return it. Otherwise the object is | |
| created anew and added to the cache before returning. The rough outline of the | |
| code is: | |
| ``` | |
| private SparseArray<AccessibilityNodeInfoCompat> mNodeInfoCache = new SparseArray<>(); | |
| @Override | |
| public AccessibilityNodeInfoCompat createAccessibilityNodeInfo(int virtualViewId) { | |
| if (mNodeInfoCache.get(virtualViewId) != null) { | |
| cachedNode = mNodeInfoCache.get(virtualViewId); | |
| // ... update cached node through JNI call ... | |
| return cachedNode; | |
| } else { | |
| // Create new node from scratch | |
| AccessibilityNodeInfo freshNode = // ... construct node through JNI call. | |
| mNodeInfoCache.put(virtualViewId, freshNode); | |
| return freshNode; | |
| } | |
| } | |
| ``` | |
| When returning a cached node, there are some fields that we always update. This | |
| requires a call through the JNI, but it still much more efficient than constructing | |
| a full node from scratch. Rather than calling `PopulateAccessibilityNodeInfo`, we | |
| call [WebContentsAccessibilityAndroid::UpdateCachedAccessibilityNodeInfo](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/web_contents_accessibility_android.cc?q=%22WebContentsAccessibilityAndroid::UpdateCachedAccessibilityNodeInfo%22). | |
| This method updates the bounding box for the node so that AT knows where to draw | |
| outlines if needed. (Note: it also technically updates RangeInfo on some nodes to | |
| get around a bug in the Android framework, more on that below.) | |
| We clear nodes from the cache in [BrowserAccessibilityManager::OnNodeWillBeDeleted](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_manager_android.cc?q=%22BrowserAccessibilityManagerAndroid::OnNodeWillBeDeleted%22). | |
| We also clear the parent node of any deleted node so that the AccessibilityNodeInfo | |
| object will receive an updated list of children. We also clear any node that has a focus | |
| change during [FireFocusEvent](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_manager_android.cc?q=%22BrowserAccessibilityManagerAndroid::FireFocusEvent%22). | |
| ### Bundle extras / API gaps | |
| Much of the richness of the web cannot be captured by the Android accessibility API, | |
| which is designed from a native (Java) widget perspective. When there is a piece of | |
| information that an AT would like to have access to, but there is no way to include | |
| it through the standard API, we put that info in the AccessibilityNodeInfo's | |
| [Bundle](https://developer.android.com/reference/android/os/Bundle). | |
| The Bundle, accessed through `getExtras()` is a map of key\-value pairs which can | |
| hold any arbitrary data we would like. Some examples of extra information for a node: | |
| - Chrome role | |
| - roleDescription | |
| - targetUrl | |
| We also include information unique to Android, such as a "clickableScore", which is | |
| a rating of how likely an object is to be clickable. The boolean "offscreen" is | |
| used to denote if an element is "visible" to the user, but off screen (see more below). | |
| We include unclipped bounds to give the true bounding boxes of a node if we were | |
| not clipping them to be only onscreen. The full list can be seen in the | |
| [list of constants](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22Constants%20defined%20for%20AccessibilityNodeInfo%20Bundle%20extras%20keys%22) | |
| at the top of WebContentsAccessibilityImpl.java. | |
| ### Asynchronously adding "heavy" data | |
| Sometimes apps and downstream services will request we add additional information | |
| to the AccessibilityNodeInfo objects that is too computationally heavy to compute | |
| and include for every node. For these cases, the Android API has a method that | |
| can be called by AT, [addExtraDataToAccessibilityNodeInfo](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeProvider#addExtraDataToAccessibilityNodeInfo\(int,%20android.view.accessibility.AccessibilityNodeInfo,%20java.lang.String,%20android.os.Bundle\)). The method is | |
| part of the AccessibilityNodeProvider, and so WebContentsAccessibilityImpl has | |
| its [own implementation](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22public%20void%20addExtraDataToAccessibilityNodeInfo%22) of this for Chrome. When called with valid arguments, | |
| this will start an asynchronous process to add this extra data to the given | |
| AccessibilityNodeInfo object. The two current implementations of this are to add | |
| text character locations, and raw image data. | |
| ### AccessibilityNodeInfoCompat | |
| The Android framework includes a [support library](https://developer.android.com/topic/libraries/support-library) for backwards compatibility. | |
| This is a mechanism that allows Android to add new attributes or methods to their | |
| API while also including backwards compatibility across previously released versions | |
| of Android. The WebContentsAccessibilityImpl class makes heavy use of this to | |
| ensure all features are supported on older versions of Android. This was a recent | |
| change, and the rest of the Chrome code base still uses the non-Compat version of | |
| the accessibility code because there is no use-case yet to switch. To make the | |
| change as minimal as possible, the WebContentsAccessibilityImpl uses the Compat | |
| version internally, but when communicating with other parts of Chrome, it will | |
| unwrap the non-Compat object instead. For this entire document, whenever | |
| AccessibilityNodeInfo is mentioned, technically speaking we are using an | |
| [AccessibilityNodeInfoCompat](https://developer.android.com/reference/androidx/core/view/accessibility/AccessibilityNodeInfoCompat) object, and we use the | |
| [AccessibilityNodeProviderCompat](https://developer.android.com/reference/androidx/core/view/accessibility/AccessibilityNodeProviderCompat) version as well. This library is designed to | |
| be transparent to the end-user though, and for simplicity we generally do not | |
| include the word 'Compat' in documentation, conversation, etc. | |
| ## Responding to user actions | |
| As the AccessibilityNodeProvider, the WebContentsAccessibilityImpl is responsible for | |
| responding to user actions that come from downstream AT. It is also responsible | |
| for telling downstream AT of Events coming from the web contents, to ensure that | |
| AT is aware of any changes to the web contents state. | |
| ### performAction | |
| One of the most important methods in WebContentsAccessibilityImpl is the | |
| implementation of [performAction](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeProvider#performAction\(int,%20int,%20android.os.Bundle\)). | |
| This method is called by the Android framework on behalf of downstream AT for | |
| any user actions, such as focus changes, clicks, scrolls, etc. For most actions, | |
| we call through the JNI to web\_contents\_accessibility\_android, which will call a | |
| corresponding method in BrowserAccessibilityManager to send this request to the | |
| underlying accessibility code. The performAction method parameters include a | |
| virtualViewId, the action, and a Bundle of args/extras. The Bundle may be null, | |
| but sometimes carries necessary information, such as text that a user is trying | |
| to paste into a field. If the accessibility code is able to successfully act on | |
| this performAction call, it will return `true` to the framework, otherwise `false`. | |
| ### AccessibilityEventDispatcher | |
| The other direction of communication is from the accessibility code into the framework | |
| and downstream AT. For this we dispatch an [AccessibilityEvent](https://developer.android.com/reference/android/view/accessibility/AccessibilityEvent). | |
| Often times a call to performAction is paired with one or more AccessibilityEvents | |
| being dispatched in response, but AccessibilityEvents can also be sent without | |
| any user interaction, but instead from updates in the web contents. The AccessibilityEvents | |
| are relatively lightweight, and they are constructed following the same model as | |
| the AccessibilityNodeInfo objects (i.e. calling into web\_contents\_accessibility\_android and | |
| being populated through a series of JNI calls). However, the events can put significant | |
| strain on downstream AT, and this is where another important performance optimization | |
| was added. | |
| Traditionally an app would realize it needs to generate and send an AccessibilityEvent, | |
| it would generate it synchronously and send it on the main thread. The web is more | |
| complicated though, and at times could be generating so many events that downstream | |
| AT is strained. To alleviate this, we have implemented a throttling mechanism, | |
| the [AccessibilityEventDispatcher](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/AccessibilityEventDispatcher.java). | |
| Whenever WebContentsAccessibilityImpl requests to send an AccessibilityEvent, it | |
| first goes through this dispatcher. For most events, they are immediately passed | |
| along to the Android framework (e.g. user clicks, focus changes). Some events are | |
| instead put in a queue and dispatched or dropped based on future user actions. | |
| For events that we wish to throttle, we will immediately send the first event received | |
| of that type. We record the system time of this event. If another event request of | |
| the same type is received within a set time limit, which we call the `throttleDelay`, | |
| then we will wait until to send that event until after the delay. | |
| For example, scroll events can only be sent at most once every 100ms, otherwise we would attempt to send | |
| them on every frame. Consider we have sent a scroll event and recorded the system | |
| time to start a throttle delay. If we try to dispatch another scroll event in that delay | |
| window, it will be queued to release after the delay. If during that throttleDelay period another scroll event | |
| is added to the queue, it will replace the previous event and only the last one added | |
| in the 100ms window is dispatched. The timer would then restart for another 100ms. | |
| The delay times can be specific to a view, or specific to an event. That is, we could | |
| say "only dispatch scroll events every 100ms for any given view", meaning two different nodes | |
| could send scroll events in close succession. Or we can say "only dispatch scroll | |
| events every 100ms, regardless of view", in which case all views trying to send that | |
| event type will enter the same queue. | |
| The event types and their delays can be found [in the constructor](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22Define%20our%20delays%20on%20a%20per%20event%20type%20basis%22) | |
| of the WebContentsAccessibilityImpl, which also includes the construction of the | |
| AccessibilityEventDispatcher. | |
| ### Delayed AccessibilityEvent construction | |
| The final performance optimization related to events (that is released on 100% stable), | |
| was to delay the construction of the AccessibilityEvent until it is about to be | |
| dispatched. Implementing the Dispatcher helps a significant amount, but there are | |
| many events that can be dropped, and there is no reason to construct an event until | |
| we are sure it will be dispatched. So we do not construct the AccessibilityEvents | |
| until the moment the Dispatcher has started the request to send the event to the | |
| Android framework. | |
| ## Testing on Android | |
| Testing on Android happens through a couple of build targets depending on what it | |
| is that we want to test. Android has tests present in the **content\_browsertests** | |
| target, same as the other platforms, which tests the BrowserAccessibilityManagerAndroid | |
| and BrowserAccessibilityAndroid through the various DumpAccessibilityTreeTests | |
| and DumpAccessibilityEventsTests. However, these tests do not cover the | |
| web\_contents\_accessibility\_android layer, or any of the Java-side code. The | |
| web\_contents\_accessibility\_android object and the associated WebContentsAccessibilityImpl | |
| object are not created for content\_browsertests and require a full browser instance | |
| to be available (or at least the content shell). To handle these types of tests | |
| we must use the **content\_shell\_test\_apk** target, which will run an instance of a | |
| web contents and allow the creation/execution of WebContentsAccessibilityImpl and | |
| the corresponding native object. And finally there is the **chrome\_public\_test\_apk**, | |
| which is used to test the Chrome Android UI, outside the web contents, which is | |
| necessary for testing accessibility features that have a user-facing Android UI, such as | |
| image descriptions, the accessibility settings pages, and page zoom. | |
| ### Testing the "missing layer" | |
| The "missing layer" in testing refers to the gap in testing for WebContentsAccessibilityImpl, | |
| and namely web\_contents\_accessibility\_android mentioned above. There are three main | |
| classes we use to test these. They are: | |
| - WebContentsAccessibilityTest | |
| This test suite is used to test the methods of WebContentsAccessibilityImpl.java. It tests | |
| the various actions of performAction, construction of AccessibilityEvents, and | |
| various helper methods we use throughout the code. | |
| - WebContentsAccessibilityTreeTest | |
| This class is the Java-side equivalent of the DumpAccessibilityTreeTests. This test suite | |
| opens a given html file (shared with the content\_browsertests), generates an | |
| AccessibilityNodeInfo tree for the page, and then dumps this tree and compares with | |
| an expectation file (denoted with the `...-expected-android-external.txt` suffix). We continue | |
| to keep around the content\_browsertests because a failure in one and not the other | |
| would provide insight into a potential bug location. | |
| - WebContentsAccessibilityEventsTest | |
| This class is the Java-side equivalent of the DumpAccessibilityEventsTests. Same as the | |
| suite above, it shares the same html files as the content\_browsertests, opens them, | |
| runs the Javascript, and records the AccessibilityEvents that are dispatched to | |
| downstream AT. There is no Android version of the DumpAccessibilityEventsTests though, | |
| so these expectation files are suffixed with the usual `...-expected-android.txt`. | |
| When new tests are added for content_browsertests, the associated test should also | |
| be added in WebContentsAccessibility\*Test, and there are PRESUBMIT warnings to | |
| remind developers of this (although they are non-blocking). | |
| ### Writing new tests | |
| Adding tests is as easy on Android as it is on the other platforms because the | |
| mechanism is in place and only a single new method needs to be added for the test. | |
| If you are adding a new events test, "example-test.html", you would | |
| first create the html file as normal (content/test/data/accessibility/event/example-test.html), | |
| and add the test to the existing `dump_accessibility_events_browsertests.cc`: | |
| ``` | |
| IN_PROC_BROWSER_TEST_P(DumpAccessibilityEventsTest, AccessibilityEventsExampleTest) { | |
| RunEventTest(FILE_PATH_LITERAL("example-test.html")); | |
| } | |
| ``` | |
| To include this test on Android, you would add a similar block to the | |
| `WebContentsAccessibilityEventsTest.java` class: | |
| ``` | |
| @Test | |
| @SmallTest | |
| public void test_exampleTest() { | |
| performTest("example-test.html", "example-test-expected-android.txt"); | |
| } | |
| ``` | |
| Some tests on Android won't produce any events. For these you do not need to | |
| create an empty file, but can instead make the test line: | |
| ``` | |
| performTest("example-test.html", EMPTY_EXPECTATIONS_FILE); | |
| ``` | |
| The easiest approach is to use the above line, run the tests, and if it fails, | |
| the error message will give you the exact text to add to the | |
| `-expected-android.txt` file. The `-expected-android.txt` file should go in the | |
| same directory as the others (content/test/data/accessibility/event). | |
| For adding a new WebContentsAccessibilityTreeTest, you follow the same method but | |
| include the function in the corresponding Java file. | |
| Note: Writing WebContentsAccessibilityTests is much more involved and there is no | |
| general approach that can be encapsulated here, same with the UI tests. For those | |
| there are many existing examples to reference, or you can reach out to an Android developer. | |
| ### Running tests and tracking down flakiness | |
| Running tests for Android can seem a bit daunting because it requires new build | |
| targets, an emulator, and different command-line arguments and syntax. But Android | |
| has a few nifty tricks that don't exist on every platform. For example, Android | |
| tests can be run on repeat indefinitely, but set to break on their first failure. This | |
| is great for tracking down flakiness. It is also possible to use a local repository | |
| to test directly on the build bots, which is great when a test works locally but flakes | |
| or fails during builds. Let's look at some basic examples. | |
| First ensure that you have followed the basic [Android setup](https://chromium.googlesource.com/chromium/src/+/master/docs/android_build_instructions.md) guides and can | |
| successfully build the code. You should not proceed further until you can | |
| successfully run the command: | |
| ``` | |
| autoninja -C out/Debug chrome_apk | |
| ``` | |
| One of the most important things to remember when building for unit tests is to use | |
| the `x86` architecture, because most emulators use this. (Note: For running on try | |
| bots however, you'll want `arm64`, more on that below). Your gn args should contain | |
| at least: | |
| ``` | |
| target_os = "android" | |
| target_cpu = "x86" | |
| ``` | |
| To run the types of tests mentioned above, you'll build with a command similar to: | |
| ``` | |
| autoninja -C out/Debug content_shell_test_apk | |
| ``` | |
| The filtering argument for tests is `-f` rather than the `--gtest_filter` that it is | |
| used with content\_browsertests. So to run an example WebContentsAccessibilityTreeTest test, | |
| you may use a command such as: | |
| ``` | |
| out/Debug/bin/run_content_shell_test_apk --num_retries=0 -f "*WebContentsAccessibilityTreeTest*testExample" | |
| ``` | |
| This would look for an x86 phone to deploy to, which should be your emulator. You can | |
| choose to setup an emulator in Android Studio, or you can use some of the emulators | |
| that come pre-built in the repo. [More information here](https://chromium.googlesource.com/chromium/src/+/refs/heads/main/docs/android_emulator.md). In general it is best to run on one of the | |
| newer Android versions, but sometimes the newest is unstable. To specify an | |
| emulator to use, you include the `--avd-config` argument, along with the desired | |
| emulator (see link above for full list). This will run the test without opening a | |
| window, but if you'd like to see an emulator window you can add the `--emulator-window` | |
| argument. The `--repeat=#` argument allows repeats, and if set to `-1` along with | |
| the `--break-on-failure` argument, the test will run repeatedly until it fails once. | |
| Putting this all together, to run the example test with no retries per run, run | |
| repeatedly until failure, on the Android 11 emulator, with a window available, you would | |
| use the command: | |
| ``` | |
| out/Debug/bin/run_content_shell_test_apk \ | |
| --num_retries=0 \ | |
| --repeat=-1 \ | |
| --break-on-failure \ | |
| --emulator window \ | |
| --avd-config tools/android/avd/proto/generic_android30.textpb \ | |
| -f "*WebContentsAccessibilityTreeTest*testExample" | |
| ``` | |
| All of this information also applies to the UI tests, which use the target: | |
| ``` | |
| autoninja -C out/Debug chrome_public_test_apk | |
| ``` | |
| In this case we would have a similar command to run an ImageDescriptions or Settings | |
| related test: | |
| ``` | |
| out/Debug/bin/run_chrome_public_test_apk \ | |
| --num_retries=0 \ | |
| --repeat=-1 \ | |
| --break-on-failure \ | |
| --emulator window \ | |
| --avd-config tools/android/avd/proto/generic_android30.textpb \ | |
| -f "*ImageDescriptions*" | |
| ``` | |
| #### Running on try bots | |
| For more information you should reference the `mb.py` [user guide](https://source.chromium.org/chromium/chromium/src/+/main:tools/mb/docs/user_guide.md). | |
| Note: When running on the trybots, you often need to use `target_cpu = "arm64"`, since | |
| these are actual devices and not emulators. | |
| It is not uncommon when working on the Android accessibility code to have a test that | |
| works fine locally, but consistently fails on the try bots. These can be difficult | |
| to debug, but if you run the test directly on the bots it is easier to gain insights. | |
| This is done using the `mb.py` script. You should first build the target exactly as | |
| outlined above (or mb.py will build it for you), and then you use the `mb.py run` command | |
| to start a test. You provide a series of arguments to specify which properties you | |
| want the try bot to have (e.g. which OS, architecture, etc), and you also can include | |
| arguments for the test apk, same as above. Note: It is recommended to at least provide | |
| the argument for the test filter to save time. | |
| With `mb.py`, you use `-s` to specify swarming, and `-d` to specify dimensions, which | |
| narrow down the choice in try bot. The dimensions are added in the form: `-d dimension_name dimension_value`. | |
| You should specify the `pool` as `chromium.tests`, the `device_os_type` as `userdebug`, | |
| and the `device_os` for whichever Android version you're interested in (e.g. `M`, `N`, `O`, etc). | |
| After specifying all your arguments to `mb.py`, include a `--`, and after this `--` | |
| all further arguments are what is passed to the build target (e.g. content\_shell\_test\_apk). | |
| Putting this all together, to run the same tests as above, in the same way, but | |
| on the Android M try bots, you would use the command: | |
| ``` | |
| tools/mb/mb.py run -s --no-default-dimensions \ | |
| -d pool chromium.tests \ | |
| -d device_os_type userdebug \ | |
| -d device_os M \ | |
| out/Debug \ | |
| content_shell_test_apk \ | |
| -- \ | |
| --num_retries=0 \ | |
| --repeat=-1 \ | |
| --break-on-failure \ | |
| -f "*WebContentsAccessibilityTreeTest*testExample" | |
| ``` | |
| Piece of cake! | |
| ## Common Android accessibility "gotchas" | |
| - "name" vs. "text" | |
| On other platforms, there is a concept of "name", and throughout the accessibility | |
| code there are references to name. In the Android API, this is referred to as | |
| "text" and is an attribute in the AccessibilityNodeInfo object. In another platform | |
| you may `setName()` for a node, the equivalent on Android is `info.setText(text)`. | |
| In the BrowserAccessibilityAndroid class, the relevant method that provides | |
| this information is [GetTextContentUTF16](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_android.cc?q=BrowserAccessibilityAndroid::GetTextContentUTF16). | |
| - ShouldExposeValueAsName | |
| The AccessibilityNodeInfo objects of Android do not have a concept of "value". | |
| This makes some strange cases for nodes that have both a value and text or | |
| label, and a challenge for how exactly to expose this data through the API. | |
| The [ShouldExposeValueAsName](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_android.cc?q=BrowserAccessibilityAndroid::ShouldExposeValueAsName) | |
| method of BrowserAccessibilityAndroid returns a boolean of whether or not the | |
| node's value should be returned for its name (i.e. text, see above). If this | |
| value is false, then we concatenate the value with the node's text and | |
| return this from GetTextContentUTF16. In the cases where ShouldExposeValueAsName | |
| is true, we expose only the value in the text attribute, and use the "hint" | |
| attribute of AccessibilityNodeInfo to expose the rest of the information (text, | |
| label, description, placeholder). | |
| - stateDescription | |
| The [stateDescription](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo#setStateDescription\(java.lang.CharSequence\)) | |
| attribute of the AccessibilityNodeInfo objects is a recent addition to the API | |
| which allows custom text to be added for any node. This text is usually read at | |
| the end of a node's announcement, and does not replace any other content and is | |
| purely additional information. We make heavy use of the state description | |
| to try and capture the richness of the web. For example, the Android API has a | |
| concept of checkboxes being checked or unchecked, but it does not have the concept | |
| of 'partially checked' as we have on the web (kMixed). When a checkbox is partially checked, | |
| we include that information in the stateDescription attribute. For some nodes like | |
| lists we include a stateDescription of the form "in list, item x of y". The full | |
| list of stateDescriptions can be found in the BrowserAccessibilityAndroid method | |
| [GetStateDescription](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_android.cc?q=BrowserAccessibilityAndroid::GetStateDescription). | |
| - CollectionInfo and CollectionItemInfo | |
| The AccessibilityNodeInfo object has some child objects that do not always | |
| need to be populated, for example [CollectionInfo](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo.CollectionInfo) | |
| and [CollectionItemInfo](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo.CollectionItemInfo). | |
| Collections work differently on Android than other platforms, namely that a given | |
| node does not carry all necessary information to make a determination about the | |
| overall collection. As the names might suggest, an item in a collection will have | |
| the CollectionItemInfo populated, but not CollectionInfo, whereas the container | |
| that holds all the items of the collection will have a CollectionInfo object but | |
| not a CollectionItemInfo. When a CollectionItemInfo object is present, it is up | |
| to the downstream AT to walk up the tree and gather information about the full | |
| Collection. This information is not included on every node. | |
| These collections are used for any table-like node on | |
| Android, such as lists, tables, grids, trees, etc. If a node is not table like or | |
| an item of a table, then these child objects would be `null`. For this example | |
| tree, the objects present for each node would be: | |
| ``` | |
| kGenericContainer - CollectionInfo=null; CollectionItemInfo=null | |
| kList - CollectionInfo=populated; CollectionItemInfo=null | |
| ++kListItem - CollectionInfo=null; CollectionItemInfo=populated | |
| ++kListItem - CollectionInfo=null; CollectionItemInfo=populated | |
| ++kListItem - CollectionInfo=null; CollectionItemInfo=populated | |
| ``` | |
| - contentInvalid | |
| The Android accessibility API has a boolean field isContentInvalid, however this | |
| does not play well with downstream AT, so the Chrome code has some special | |
| implementation details. The accessibility code reports a page exactly as it is, | |
| so if a text field is labeled as contentInvalid, we report the same to all platforms. | |
| There are use-cases where a field may be contentInvalid for each character typed | |
| until a certain regex is met, e.g. when typing an email, empty or a few characters | |
| would be reported as contentInvalid. When isContentInvalid is true on a node's | |
| AccessibilityNodeInfo object, then AT (e.g. TalkBack) will proactively announce | |
| "Error" or "Content Invalid", which can be jarring and unexpected for the user. | |
| This announcement happens on any change, so every character typed would make | |
| this announcement and give a bad user experience. It is the opinion of the Chrome | |
| accessibility team that this ought to be fixed by TalkBack, and reporting an invalid | |
| node as invalid is the pedantically correct approach. However, in the spirit of | |
| giving the best user experience possible, we added two workarounds: | |
| - The contentInvalid boolean is always false if the number of characters in | |
| the field is less than [kMinimumCharacterCountForInvalid](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_android.cc?q=kMinimumCharacterCountForInvalid), currently set to 7. This is done at the BrowserAccessibilityAndroid level. | |
| - The WebContentsAccessibilityImpl includes a further workaround. contentInvalid | |
| will only be reported for a currently focused node, and it will be reported | |
| at most once every [CONTENT\_INVALID\_THROTTLE\_DELAY](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=CONTENT_INVALID_THROTTLE_DELAY) seconds, currently set to 4.5s. | |
| See the [setAccessibilityNodeInfoBooleanAttributes](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=setAccessibilityNodeInfoBooleanAttributes) | |
| method for the full implementation. | |
| - isVisibleToUser vs. "offscreen" | |
| The Android accessibility API includes only one boolean for setting whether or | |
| not a node is "visible", the isVisibleToUser attribute. In general this conflicts with | |
| the way the accessibility code treats [offscreen](https://source.chromium.org/chromium/chromium/src/+/main:docs/accessibility/browser/offscreen.md). | |
| The name isVisibleToUser may suggest that it reflects whether or not the node | |
| is currently visible to the user, but a more apt name would be: isPotentiallyVisibleToUser, | |
| or isNotProgrammaticallyHidden. Nodes that are scrolled off the screen, and thus | |
| not visible to the user, must still report true for isVisibleToUser. The main use-case | |
| for this is for AT to allow navigation by element type. For example, if a user wants to | |
| navigate by Headings, then an app like TalkBack will only navigate through nodes with a | |
| true value for isVisibleToUser. If any node offscreen has isVisibleToUser as false, | |
| then it would effectively remove this navigation option. So, the Chrome Android | |
| accessibility code reports most nodes as isVisibleToUser, and if the node is actually | |
| offscreen (not programmatically hidden but scrolled offscreen), then we include a | |
| Bundle extra boolean, "offscreen" so that downstream AT can differentiate between | |
| the nodes truly on/off screen. | |
| - RangeInfo, aria-valuetext, and caching | |
| The [RangeInfo](https://developer.android.com/reference/android/view/accessibility/AccessibilityNodeInfo.RangeInfo) | |
| object is another child object of AccessibilityNodeInfo. Unfortunately this | |
| object is rather limited in its options, and can only provide float values | |
| for a min, max, and current value. There is no concept of a text description, or | |
| steps or step size. This clashes with nodes such as sliders with an aria-valuetext, | |
| or an indeterminate progress bar, for which we have to add special treatment. | |
| As a further complication, AccessibilityEvents also require information on range | |
| values when there is a change in value, however the event only allows integer | |
| values between 0 and 100 (an integer percentage of the sliders position). | |
| BrowserAccessibilityAndroid has a method [IsRangeControlWithoutAriaValueText](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_android.cc?q=BrowserAccessibilityAndroid::IsRangeControlWithoutAriaValueText) | |
| which we use to separate these cases when populating AccessibilityNodeInfo | |
| objects and AccessibilityEvents (see web\_contents\_accessibility\_android.cc). | |
| Similar to the Collection related objects above, RangeInfo is `null` for any | |
| non-range related nodes. | |
| This RangeInfo object plays a small role in updating the cached AccessibilityNodeInfo | |
| objects above. There is a small bug in the Android framework (which has been fixed | |
| on newer versions) which breaks our caching mechanism for range objects. So the | |
| `UpdateCachedAccessibilityNodeInfo` method also updates the RangeInfo object of a node | |
| if it has one. | |
| - Leaf nodes and links | |
| Android has slightly different IsLeaf logic than other platforms, and this can | |
| cause confusion, especially around links. On Android, links are **never** leafs. | |
| See [IsLeaf](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/browser_accessibility_android.cc?q=BrowserAccessibilityAndroid::IsLeaf). | |
| This is for similar reasons to the isVisibleToUser section above. If a link | |
| were a leaf node, and it were to contain something like a Heading, then AT would | |
| not be able to traverse that link when navigating by headings because it would only | |
| see it is a link. For this reason we always expose the entire child structure | |
| of links. | |
| - Refocusing a node Java-side | |
| There is a strange bug in Android where objects that are accessibility focused | |
| sometimes do not visually update their outline. This does not really block any | |
| user flows per se, but we would ideally have the outlines drawn by AT like TalkBack | |
| to reflect the correct bounds of the node. There is a simple way to get around | |
| this bug, which is to remove focus from the node and refocus it again, which | |
| triggers the underlying Android code necessary to update the bounds. In | |
| WebContentsAccessibilityImpl we have a method | |
| [moveAccessibilityFocusToIdAndRefocusIfNeeded](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22void%20moveAccessibilityFocusToIdAndRefocusIfNeeded%22) | |
| which handles this. | |
| - liveRegions and forced announcements | |
| There is a boolean for liveRegion in the AccessibilityNodeInfo object that the | |
| Chrome accessibility code will never set. This is because TalkBack will read | |
| the entirety of the liveRegion node when there is a change. Instead we force | |
| a custom announcement with an AccessibilityEvent in the WebContentsAccessibilityImpl's | |
| [announceLiveRegionText](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22private%20void%20announceLiveRegionText%22) | |
| method. | |
| - sendAccessibilityEvent vs. requestSendAccessibilityEvent | |
| In the Android framework, there are two methods for sending AccessibiltiyEvents, | |
| [sendAccessibilityEvent](https://developer.android.com/reference/android/view/View#sendAccessibilityEvent\(int\)) and | |
| [requestSendAccessibilityEvent](https://developer.android.com/reference/android/view/ViewParent#requestSendAccessibilityEvent\(android.view.View,%20android.view.accessibility.AccessibilityEvent\)). | |
| Technically speaking, requestSendAccessibilityEvent will ask the system to | |
| send an event, but it doesn't have to send it. For all intents and purposes, | |
| we assume that it is always sent, but as a small detail to keep in mind, this | |
| is not a guarantee. | |
| - TYPE\_WINDOW\_CONTENT\_CHANGED events | |
| The TYPE\_WINDOW\_CONTENT\_CHANGED type AccessibilityEvent is used to tell the | |
| framework that something has changed for the given node. We send these events | |
| for any change in the web, including scrolling. As a result, this is generally the | |
| most frequently sent event, and we can often send too many and put strain on | |
| downstream AT. We have included this event as part of our event throttling in | |
| the AccessibilityEventDispatcher. We also include a small optimization for a | |
| given atomic update. If an atomic update sends more than | |
| [kMaxContentChangedEventsToFire](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/web_contents_accessibility_android.h?q=%22kMaxContentChangedEventsToFire%22) | |
| events (currently set to 5), then any further events are dropped and a single | |
| event on the root node is sent instead. This has proven useful for situations such | |
| as many nodes being toggled visible at once. See [HandleContentChanged](https://source.chromium.org/chromium/chromium/src/+/main:content/browser/accessibility/web_contents_accessibility_android.cc?q=%22WebContentsAccessibilityAndroid::HandleContentChanged%22) | |
| in web\_contents\_accessibility\_android.cc. | |
| - Text selection | |
| Static text selection exists in Android, but not when a service like TalkBack | |
| is enabled. TalkBack allows for text selection inside an editable text field, | |
| but not static text inside the web contents. Text selection is also tied to a | |
| specific node with a start and end index. This means that we cannot select | |
| text across multiple nodes. Ideally the implementation would allow a start | |
| and end index on separate nodes, but this work is still in development. | |
| - Touch exploration | |
| The way touch exploration works on Android is complicated. The process that happens | |
| during any hover event is: | |
| 1. User begins dragging their finger | |
| 2. Java-side View receives a hover event and passes this through to C++ | |
| 3. Accessibility code sends a hit test action to the renderer process | |
| 4. The renderer process fires a HOVER accessibility event on the accessibility | |
| node at that coordinate | |
| 5. [WebContentsAccessibilityImpl#handleHover](https://source.chromium.org/chromium/chromium/src/+/main:content/public/android/java/src/org/chromium/content/browser/accessibility/WebContentsAccessibilityImpl.java?q=%22private%20void%20handleHover%22) is called for that node. | |
| 6. We fire a TYPE\_VIEW\_HOVER\_ENTER AccessibilityEvent on that node. | |
| 7. We fire a TYPE\_VIEW\_HOVER\_EXIT AccessibilityEvent on the previous node. | |
| 8. TalkBack sets accessibility focus to the targeted node. | |
| - WebView and Custom Tabs | |
| As mentioned at the start of this document, Chrome Android also plays an important | |
| role by providing the WebView, which is used by any third party apps | |
| that want to include web content in their app. Android also has a unique feature | |
| of Custom Tabs, which is a lightweight implementation of Chrome that is somewhere | |
| between a WebView and a full browser. The WebView and custom tabs must also be accessible, | |
| and so most of this document applies to them as well. Occasionally there | |
| will be an edge case or small bug that only happens in WebView, or a feature that | |
| needs to be turned off only for WebView/Custom tab (e.g. image descriptions). There is a | |
| WebView and Chrome Custom Tabs test app on the chrome-accessibility appspot page, | |
| and there are methods on the Java-side that can give signals of whether the current | |
| instance is a WebView or Chrome custom tab. [Example: isCustomTab](https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/tab/java/src/org/chromium/chrome/browser/tab/Tab.java?q=isCustomTab). | |
| ## Recent Progress and Features | |
| The Chrome Android accessibility code continues to evolve each quarter. We have | |
| strengthened our testing and the stability of the code, but we also continue to | |
| add new features and improvements. Beyond the usual bug fixes, below is a quick | |
| summary of some features in the pipeline. | |
| ### OnDemand AT | |
| We have recently implemented a feature we refer to as "OnDemand AT" for short. | |
| This feature is still rolling out and we intend to eventually have it enabled | |
| on 100% stable by default. The feature modifies the AccessibilityEventDispatcher | |
| that is explained above. If the feature is enabled, then WebContentsAccessibilityImpl | |
| will query the Android system to determine the currently enabled accessibility | |
| services, as well as the types of data they are interested in, namely the types of | |
| AccessibilityEvents they want to know about. When the AccessibilityEventDispatcher | |
| is sent an event to add to its queue or dispatch, if that event type is not in | |
| the list of AccessibilityEvents relevant to currently enabled accessibility services, | |
| the Dispatcher simply drops/ignores the request. Preliminary data shows that this | |
| has created a noticeable improvement for accessibility services that do not require | |
| the entire suite to function. | |
| ### ComputeAXMode | |
| Loosely related to the OnDemand feature above, the "ComputeAXMode" feature is also | |
| a recent addition to improve overall performance. This feature uses the same | |
| mechanism as OnDemand to query the currently enabled services and the information | |
| they are interested in. ComputeAXMode then takes this information and uses a different | |
| [AXMode](https://source.chromium.org/chromium/chromium/src/+/main:ui/accessibility/ax_mode.h) | |
| based on the situation. This effectively does the same thing as | |
| OnDemand, but further left/up-the-chain, giving a more significant performance | |
| improvement. This feature is still rolling out, and it currently only has two | |
| AXModes (full or basic). As it rolls out and we gather more data we will potentially | |
| add more AXModes in the future. | |
| ### AutoDisableAccessibility | |
| The "AutoDisable" accessibility feature has also been ported to Android. This feature | |
| tracks timing between user inputs and accessibility actions to make a determination | |
| of whether or not accessibility services are still required. If they are no longer | |
| needed by the user, then the accessibility code is disabled. Before this feature, | |
| once the code was enabled it would continue to run for the life of the current | |
| browser session. This feature is still being rolled out to stable. | |
| ## Accessibility code in the ClankUI | |
| Most of this document is focused on the accessibility code and work as it relates | |
| to the web contents, which is where the Chrome & Chrome OS Accessibility team | |
| focuses most of its work. However, some features require a native UI in the browser | |
| app, outside the web contents. When these features are added, the line between | |
| the accessibility team and the Clank UI team becomes blurred. We traditionally | |
| are the owners of this code, but seek regular guidance and approvals from the | |
| Clank team as the front-end code must conform to the Clank standards. | |
| ### AccessibilitySettings | |
| The AccessibilitySettings page is found under the overflow/3-dot menu (Menu\>Settings\>Accessibility). | |
| The page currently contains a slider to change the font scaling of the web contents, | |
| options to force enable zoom, show simplified pages, enable image descriptions (see below), | |
| and live captions. | |
| The main entry point for this code is [here](https://source.chromium.org/chromium/chromium/src/+/main:components/browser_ui/accessibility/android/java/src/org/chromium/components/browser_ui/accessibility/AccessibilitySettings.java). | |
| The code leverages the PreferenceFragment of Android, and so much of the UI and | |
| navigation is available out of the box, and the code is relatively simple in that | |
| it only needs to respond to user actions/changes and pass this information to the | |
| native C++ code. | |
| The settings code is heavily unit tested and stable, so it is rare to have to | |
| work in this area. | |
| ### Image Descriptions | |
| The Clank-side code for the image descriptions feature is a bit more involved. | |
| The image descriptions has to track state, determine whether or not to display the | |
| option in the overflow menu, show dialogs, and provide toasts to the user. This | |
| code is mostly controlled by the [ImageDescriptionsController](https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/image_descriptions/android/java/src/org/chromium/chrome/browser/image_descriptions/ImageDescriptionsController.java). | |
| The image descriptions feature is written using Clank's 'component' model, and so | |
| almost all the code exists in the directory: | |
| [chrome/browser/image_descriptions](https://source.chromium.org/chromium/chromium/src/+/main:chrome/browser/image_descriptions/) | |
| (The exception being the few hooks that connect this code to the other parts of the | |
| Clank UI). | |
| The image descriptions code is heavily unit tested and stable, so it is rare to | |
| have to work in this area. | |
| ### (Upcoming) Page Zoom | |
| An upcoming feature is the page zoom feature, which will allow a more robust way | |
| to zoom web contents than the currently existing text scaling of AccessibilitySettings | |
| (which will be replaced). | |
| The Clank UI code for this feature has not been developed. More to come. |