Tast: Testing Isolated Web Applications (IWAs) on ChromeOS (go/tast-iwa)

This document assumes that you are familiar with Tast test writing, execution, and debugging, and have already gone through Codelab #1 and Codelab #2. You should also know what Isolated Web Application is.

Testing Isolated Web Apps in Tast

This document provides guidance on how to test Isolated Web Apps (IWAs) within the Tast testing framework.

Testing Pyramid and Lower-Level Testing

• Adopt a testing pyramid approach, which emphasizes a balance of different test types. Read more about it at Google Testing Blog.
• There should be a significant number of tests checking the application‘s code, focusing on individual components and their interactions, including components’ integrations. These tests should be prioritized to ensure a robust and reliable application.
• Testing the integration of the IWA with the Chrome browser is currently under development (crbug.com/337872319).

Setting up the Test Environment

1. Test Device:

   • Recommended Approach: VM Testing

     • It is highly recommended to start testing with a ChromeOS VM. This allows for faster iteration, easier debugging, and a more controlled environment.
     • VM Setup:
       1. Linux Chromium Checkout: Ensure you have a Linux Chromium checkout with depot_tools installed.
       2. Virtualization Enabled: Your system firmware (BIOS) must have virtualization features enabled, and KVM must be enabled in your kernel.
       3. Simple Chrome Setup: You should have Simple Chrome set up.
       4. Choose a Board: Select a suitable board. betty is recommended for Googlers, while amd64-generic-vm is suitable for open-source contributors. Set the board using export BOARD=betty
       5. Launch the VM: Use cros vm start or cros_vm --board ${BOARD} to launch the VM.
          Refer to the official ChromeOS VM documentation for detailed instructions.
     • Why VM First? VMs offer a faster, more controlled environment for initial testing and development. They allow for quick modifications and easier debugging before moving to physical hardware.

   • Physical Device Testing:

     • While VMs are ideal for initial testing, it's crucial to test on physical devices.
     • Recommended Devices:
       • Primary Device: Target the most used device within your customer base. This ensures the IWA functions well on the most common hardware.
       • Low-End Device: Include a low-end device in your testing to ensure the IWA remains performant even under resource constraints.
     • Why Physical Devices? Physical devices provide a real-world view of performance, hardware interactions, and user experience.

2. Update manifest: The application should be deployed with its update manifest accessible.

Steps to Writing Tast Tests for IWAs

Here's a general outline for writing Tast tests for IWAs:

Define application details

const (
	kitchenSinkIWAUpdateManifestURL = "https://github.com/chromeos/iwa-sink/releases/latest/download/update.json"
	kitchenSinkIWAWebBundleID       = "aiv4bxauvcu3zvbu6r5yynoh4atkzqqaoeof5mwz54b4zfywcrjuoaacai"
)

Prepare the policies

pb := policy.NewBlob()
policies := []policy.Policy{
	&policy.IsolatedWebAppInstallForceList{
		Val: []*policy.IsolatedWebAppInstallForceListValue{
			{
				UpdateManifestUrl: kitchenSinkIWAUpdateManifestURL,
				WebBundleId:       kitchenSinkIWAWebBundleID,
				PinnedVersion:     "0.17.0",
			},
		},
	},
}

NOTE: It is recommended to use PinnedVersion and the latest ChromeOS to ensure that ChromeOS changes do not impact the IWA's functionality. When testing an unpinned IWA version, use a stable, unchanging ChromeOS version. This will help isolate whether failures are due to changes in the application or in ChromeOS.

Add policies and load them

if err := pb.AddPolicies(policies); err != nil {
	s.Fatal("Failed to add policies for public account setup: ", err)
}

if err := policyutil.ServeBlobAndRefresh(ctx, fdms, cr, pb); err != nil {
	s.Fatal("Failed to update policies: ", err)
}

Create a test connection

tconn, err := cr.TestAPIConn(ctx)
if err != nil {
	s.Fatal("Failed to create test API connection: ", err)
}

Create uiauto object

ui := uiauto.New(tconn)
createSocketConnButton := nodewith.Name("Create new socket connection").Role(role.Button)
sendMessageTextField := nodewith.Name("Send a message").Role(role.TextField)
engageMessage := nodewith.Name("Foo").Role(role.InlineTextBox)
responseMessage := nodewith.Name("Bar").Role(role.InlineTextBox)

Create virtual keyboard object

kb, err := input.VirtualKeyboard(ctx)
if err != nil {
	s.Fatal("Failed to get keyboard: ", err)
}
defer kb.Close(ctx)

Start the application

if err := uiauto.Combine("Launch Kitchen Sink IWA",
		// Launch Kitchen Sink IWA.
		launcher.SearchAndLaunch(tconn, kb, "Kitchen Sink IWA"),
		// Wait till the IWA is launched and the Create Socket button appears.
	ui.WithTimeout(30*time.Second).WaitUntilExists(createSocketConnButton),
	)(ctx); err != nil {
		s.Fatal("Failed to launch Kitchen Sink IWA: ", err)
	}

Interact with the application

if err := uiauto.Combine("Interact with Kitchen Sink IWA UI",
		// Create a new socket connection.
		ui.LeftClick(createSocketConnButton),
		ui.WaitUntilExists(sendMessageTextField.Nth(1)),
		// Send messages to the TCP Server.
		ui.LeftClickUntil(sendMessageTextField.First(), ui.Exists(sendMessageTextField.Focused())),
		kb.TypeAction("Foo\n"),
		ui.WaitUntilExists(engageMessage),
		// Send a message from the TCP Server.
		ui.LeftClickUntil(sendMessageTextField.Nth(1), ui.Exists(sendMessageTextField.Focused())),
		kb.TypeAction("Bar\n"),
		ui.WaitUntilExists(responseMessage),
	)(ctx); err != nil {
		s.Fatal("Failed to interact with the Kitchen Sink IWA: ", err)
	}

The full code of the example is available in the LaunchIWA test.

Important Considerations

• Test Stability: Write robust tests that can handle potential network issues, UI changes, and other factors that might impact the IWA's behavior.
• Error Handling: Include proper error handling to identify and address issues quickly.
• Version Control: Test against different versions of the IWA to ensure compatibility.
• Clear Test Descriptions: Provide descriptive test names and comments to make the tests easier to understand.
• Follow design principles to make your test more robust.
• Prioritize VM testing for initial development and debugging.
• Include testing on the most used and a low-end physical device for optimal coverage.

Commercial setup considerations

• If a Chrome restart is required (e.g., for auto-starting the IWA), use fixture.FakeDMS. Chrome restart needs depend on the type of policies you are trying to apply. For example, the MultiScreenCaptureAllowedForUrls policy requires a restart (Dynamic Policy Refresh: No).
• To launch and manually interact with the IWA, use fixture.ChromePolicyLoggedIn.

Examples

All IWA tests are held within the iwa package. Examples include:

• Screen capture test.
• Starting IWA app from launcher test.
• Autostart and prevent closing test.

Passing Variables to the Test

This is useful when you need to pass sensitive information, such as server credentials, without hardcoding them into the test itself.

Using Run Command

During runtime you can pass variables to a Tast test using the -var flag with the tast run command. This may apply to you if you want to connect to a server and login, and not store those secrets in the test code. To store and access private data such as credentials:

Syntax:

tast run -var=name=value <dut> <tests>

• name: The name of the variable.
• value: The value to assign to the variable.
• <dut>: The target device.
• <tests>: The test(s) to run.

Multiple variables can be passed by repeating the -var flag.

Example:

tast run -var=serverUrl=https://validTestEndpoint -var=userName=foo -var=userPassword=bar <dut> <tests>

All details are available on the following runtime variables documentation.

In your Tast test, you can access the variable using the s.Var method. Make sure to declare the variable in the Vars field of the test's struct.

Example:

package mytestpackage


var exampleStrVar = testing.RegisterVarString(
        "mytestpackage.ServerUrl",
        "Default value",
        "An example variable of string type",
)

func init() {
testing.AddTest(&testing.Test{
	Func: MyTest,
	Desc: "Test that will read the variable from the command line argument",
// ...
// ...
})
}

func MyTest(ctx context.Context, s *testing.State) {
strVal := exampleStrVar.Value()
// ...
}

A full code example of this is in this test.

Using Secret Variable

This way can be used by an internal developer, who has access to the tast-tests-private package. Learn more about it from this article.

Testing IWAs in Kiosk Mode

The main difference between a regular IWA test and a Kiosk IWA test lies in the setup. Instead of using policies to force-install the IWA, you will configure the device to launch directly into the IWA in a Kiosk session.

Steps to Writing Tast Tests for IWAs in Kiosk Mode

Here’s a general outline for writing Tast tests for IWAs in Kiosk mode:

1. Use the fixture.FakeDMSEnrolled fixture in your test definition to simulate a managed device.

   func init() {
       testing.AddTest(&testing.Test{
           // ...
           Fixture: fixture.FakeDMSEnrolled,
       })
   }
   

2. Define the application details, including the update manifest URL and the web bundle ID for your IWA.

   var updateManifestURL string = "https://github.com/chromeos/iwa-sink/releases/latest/download/update.json"
   var webBundleID string = "aiv4bxauvcu3zvbu6r5yynoh4atkzqqaoeof5mwz54b4zfywcrjuoaacai"
   

3. Start Chrome in Kiosk mode using kioskmode.New(). You will need to define a DeviceLocalAccount for the IWA and configure it to auto-launch.

   iwaKioskAccountType := policy.AccountTypeKioskIWA
   
   kiosk, cr, err := kioskmode.New(
       ctx,
       fdms,
       s.RequiredVar("ui.signinProfileTestExtensionManifestKey"),
       kioskmode.CustomLocalAccounts(
           &policy.DeviceLocalAccounts{
               Val: []policy.DeviceLocalAccountInfo{
                   {
                       AccountID:   &kioskmode.KioskAppAccountID,
                       AccountType: &iwaKioskAccountType,
                       IsolatedWebAppKioskInfo: &policy.IsolatedWebAppKioskInfo{
                           WebBundleId: &webBundleID,
                           ManifestUrl: &updateManifestURL,
                       },
                   },
               },
           },
       ),
       kioskmode.AutoLaunch(kioskmode.KioskAppAccountID),
   )
   if err != nil {
       s.Fatal("Failed to start Chrome in Kiosk mode: ", err)
   }
   defer kiosk.Close(cleanupContext)
   

4. Wait for the Kiosk app to launch using kiosk.WaitLaunchLogs().

   if err := kiosk.WaitLaunchLogs(ctx); err != nil {
       s.Fatal("Failed to launch Kiosk: ", err)
   }
   

5. Interact with the application using the uiauto library, just as you would in a regular IWA test. You can create a test API connection, define UI nodes, and perform actions like clicking buttons and typing text.

   tconn, err := cr.TestAPIConn(ctx)
   if err != nil {
       s.Fatal("Failed to create test API connection: ", err)
   }
   
   ui := uiauto.New(tconn)
   createSocketConnButton := nodewith.Name("Create new socket connection").Role(role.Button)
   // ...
   
   if err := uiauto.Combine("Interact with Kitchen Sink IWA UI",
       ui.WithTimeout(30*time.Second).WaitUntilExists(createSocketConnButton),
       // ...
   )(ctx); err != nil {
       s.Fatal("Failed to interact with the Kitchen Sink IWA: ", err)
   }
   

Example

For a complete example of an IWA test running in Kiosk mode, see the launch_iwa.go test.