setup/BUNDLE.md

# Chromium OS Factory Bundle

[TOC]

## What is a factory bundle?

A *factory bundle* is an archive file containing all files you need to setup
and deploy Chromium OS factory environment.

Here is a list of files in a typical factory bundle:

    complete/
      complete_script_sample.sh
    factory_shim/
      factory_install_shim.bin
    firmware/
      chromeos-firmwareupdate
    firmware_images/
      bios.bin
      ec.bin
      pd.bin
    hwid/
      hwid_v3_bundle_SAMUS.sh
    release_image/
      chromeos_9334.72.0_samus_recovery_stable-channel_mp-v3.bin
    test_image/
      ChromeOS-test-R58-9334.72.0-samus.bin
    toolkit/
      install_factory_toolkit.run
    netboot/
      tftp/
        chrome-bot/
          samus/
            cmdline.sample
            vmlinuz
      dnsmasq.conf
      image.net.bin
    setup/
      .default_board
      cgpt
      cros_docker.sh
      cros_payload
      futility
      image_tool
      NETBOOT.md
      README.md
      setup_tools.par
    README

### release_image/

There should be exactly one file in this folder, with arbitrary file name.

* This file will be imported by factory server as release image.
* `factory_installer` will deploy release image into DUTs.

You can get release images from [CPFE] (select image type `RECOVERY_IMAGE`).

### test_image/

There should be exactly one file in this folder, with arbitrary file name.

* This file will be imported by factory server as test image.
* `factory_installer` will deploy test image into DUTs.

You can get test images from [CPFE] (select image type `TEST_IMAGE_ARCHIVE`).

### toolkit/

There should be exactly one file in this folder, with arbitrary file name.

* This file will be imported by factory server as factory toolkit installer.
* `factory_installer` will deploy factory toolkit into DUTs.
* Goofy will try to update factory toolkit when running pytest `sync_shopfloor`.

You can get factory toolkit installers from [CPFE] (select image type
`FACTORY_IMAGE_ARCHIVE`)

### hwid/

There should be exactly one file in this folder, with arbitrary file name.

* This file will be imported by factory server as HWID bundle.
* `factory_installer` will deploy HWID bundle into DUTs.
* Goofy will try to update HWID bundle when running some specific pytests.

You can get HWID bundles from [Chrome OS Device Lifecycle Manager][DLM] (select
your device -> `Files` -> `Add a filter` -> `HWID_BUNDLE`).

### firmware/

There could be at most one file in this folder, with arbitrary file name.

* This file (if exists) will be imported by factory server as firmware updater.
* `factory_installer` will run firmware updater to flash firmware (EC, AP, and
  possibly PD) when imaging finished.
* Goofy will try to update firmware when running pytest `update_firmware`.

You can get a firmware updater from a release image by running
`factory/setup/image_tool get_firmware -i /path/to/release_image`

### complete/

There could be at most one file in this folder, with arbitrary file name.

* This file (if exists) will be imported by factory server as complete script.
* `factory_installer` will run the complete script after firmware updater.

You can find a complete script template in
`factory/setup/complete_script_sample.sh`.

### firmware_images/

`finalize_bundle` will extract firmware images from firmware updater and put
them into this folder.

### factory_shim/

`finalize_bundle` will extract factory shim image from `FACTORY_IMAGE_ARCHIVE`,
patch its server address, and put it into this folder.

### netboot/

`finalize_bundle` will extract netboot firmware `image.net.bin` and netboot
kernel `vmlinuz` from `FACTORY_IMAGE_ARCHIVE`, patch their server address, and
put them into this folder.

Also, there will be two files, `cmdline.sample` and `dnsmasq.conf`, which are
generated by `finalize_bundle` to help you setup TFTP service for netboot.

When importing a bundle:

* `netboot/image.net.bin` will be imported as netboot_firmware.
* `netboot/tftp/chrome-bot/*/vmlinu*` will be imported as netboot_kernel.
* `netboot/tftp/chrome-bot/*/cmdline` will be imported as netboot_cmdline.
(Note that `cmdline.sample` will not be imported)

### setup/

This folder is extracted from `FACTORY_IMAGE_ARCHIVE` by `finalize_bundle`,
contains many useful scripts, such as:

* `cros_docker.sh`
* `image_tool`

## Creating a factory bundle

There are several ways to create a bundle.

### Using command "image_tool bundle"
This is the recommended way for Google Partners (ODM, OEM, SOC) with access to
[CPFE](https://www.google.com/chromeos/partner/fe/).

1. Download the factory Zip file (select Image Type as `FACTORY_IMAGE_ARCHIVE`)
   from CPFE Release / Image Files, extract to somewhere, for example
   `my_bundle` directory. This will provide the toolkit, factory shim, netboot
   and setup.

2. Download the test image (select Image Type as `TEST_IMAGE_ARCHIVE` from CPFE
   Release / Image files, extract all files to the `test_image` sub directory,
   for example `my_bundle/test_image`.

3. Download the release (FSI) image (select Image Type as `RECOVERY_IMAGE` from
   CPFE Release / Image files, save the files to the `release_image` sub
   directory, for example `my_bundle/release_image`.

4. If you want to use a different version of firmware, download the right
   `RECOVERY_IMAGE` that contains the right version of firmware updater,
   and extract the updater from it by `image_tool get_firmware`:

       image_tool get_firmware -i PATH_TO/image.bin

   And copy the generated `chromeos-firmwareupdate` file to the `firmware` sub
   directory, for example `my_bundle/firmware`. If you need to use a firmware
   that is not included in any recovery images, download the firmware image
   archive file with Image Type `FIRMWARE_IMAGE_ARCHIVE` from CPFE Release /
   Image files, extract the `updater.sh` and use it.
   Please be aware this will be a developer-key signed firmware and can't be
   used in PVT or MP builds.

   If you want to skip updating firmware in the imaging process, add
   `--no-firmware` when you invoke `image_tool bundle` command.

5. If available, download the HWID bundle from CPFE Device File Repo / Manage
   Files (Type `HWID_BUNDLE`) and save to the `hwid` sub directory, for example
   `my_bundle/hwid`.

6. Enter the folder with all resource you've extracted and run `image_tool
   bundle` command, and specify the build phase (from one of `proto`, `evt`,
   `dvt`, `pvt`, `mp`) and an optional release note. If you plan to use network
   based installation (netboot or factory shim), also specify the factory server
   URL. For example:

       cd my_bundle;
       setup/image_tool bundle -p evt -n "First Release" -s http://192.168.0.1:8080

   The host IP in server URL will also be used for TFTP server IP (since TFTP
   port is fixed to 69).

   This should generate a factory bundle as
   `factory_bundle_${BOARD}_${DATETIME}_${PHASE}.tar.bz2`, for example
   `factory_bundle_eve_201804120907_evt.tar.bz2`.

7. If you need to use any resources in different path, each of them can be
   specified by `image_tool` command options. Run `image_tool help bundle`
   to see a full list of available options. For example:

       setup/image_tool bundle --release_image PATH_TO/image.bin \
        --test_image PATH_TO/image2.bin --toolkit PATH_TO/my_toolkit \
        --firmware PATH_TO/updater.sh

### Using Easy Bundle Creation Service
This is the recommended way for non-engineers with access to CPFE/cpcon, for
example Google Partners (ODM, OEM, SOC) or Google device PMs.

Open [cpcon](https://www.google.com/chromeos/partner/console), select
Tools / Factory Bundle, and in the menu select your board, device (project),
build phase, versions of toolkit (which implies netboot and factory shim), test
and release image, then click the CREATE button.

Wait for few minutes and you should see a mail notifying the address to download
the generated factory bundle.

### Create bundle manually
This is the recommended way for people who does not have access to CPFE nor
GoldenEye.

If you just want a bundle for [Factory Server](FACTORY_SERVER.md) to import, you
can manually create an archive file (zip file or {gzip|bzip2|xz} compressed
tarball) containing att least the following folders with corresponding files
inside:

* `release_image`
* `test_image`
* `toolkit`

Additionally, if you want to provide the optional components or override with
some more components, create the corresponding folders. For example,

* `complete`
* `firmware`
* `hwid`

Or you can just enter Factory Server web user interface, and partial update
the individual resource.
No need to always create a factory bundle for network installation.

### Using command "finalize_bundle"
This is the recommended way for Google developers with access to GoldenEye.

To create a complete factory bundle, you need to run `finalize_bundle`.
`finalize_bundle` takes a manifest file `MANIFEST.yaml` and produces a factory
bundle.

Here is a template `MANIFEST.yaml`:

    board: samus
    project: samus
    bundle_name: 20170628_dvt
    server_url: http://12.34.56.78:8080/
    firmware: local
    release_image: 9334.72.0
    test_image: 9334.72.0
    toolkit: 9688.0.0

#### Field `board`

Board name.

#### Field `project`

The Public Code Name (can be found on Golden Eye), or the "Google Name". Also
should be same as the "Model Name" displayed in firmware screen (which is
actually HWID), or reported on by command `cros_config / name`.

The `project` is made for projects sharing one board (overlay), especially for
Zerg projects.

For example, the projects `electro` and `basking` both share images built using
`BOARD=reef`, so you will set:

    board: reef
    project: electro

For projects having its own overlay, `board` and `project` may be same name.
For example, `pyro` is using `reef` as reference design but it does have its
own overlay, so you will need to set:

    board: pyro
    project: pyro

The `project` will be used to decide which HWID config file to use, and to
find right firmware (and signing keys).

#### Field `bundle_name`

`{YYYYMMDD}_{proto|evt|dvt|pvt}`

#### Field `server_url`

`http://{server ip}:{server port}/`

`finalize_bundle` will inject this address into netboot firmware and factory
shim.

This is an optional field.

#### Field `firmware`

There are three kinds of valid values:

* `local`: User manually prepares a firmware updater in `firmware/`.
* `release_image`: Use firmware updater in the release image.
* `release_image/xxxx.y.z`: Version of the **Release image** which contains the
  firmware. Note that this is not the version of firmware itself.

This is an optional field. Default value is `release_image`.

#### Field `hwid`

There are two valid values:

* `real`: There should be a real HWID bundle in `hwid/`.
* `none`: `finalize_bundle` will create a dummy HWID bundle.

This is an optional field. Default value is `real`.

#### Field `release_image`

There are two kinds of valid values:

* `local`: User manually prepares a release image in `release_image/`.
* `{xxxx.y.z}`: Release image version for `finalize_bundle` to download.
  If there is already one file in `release_image/`, `finalize_bundle` will
  check whether the version matches or not.

#### Field `test_image`

There are two kinds of valid values:

* `local`: User manually prepares a test image in `test_image/`.
* `{xxxx.y.z}`: Test image version for `finalize_bundle` to download.
  If there is already one file in `test_image/`, `finalize_bundle` will
  check whether the version matches or not.

#### Field `toolkit`

There are two kinds of valid values:

* `local`: User manually extracts `FACTORY_IMAGE_ARCHIVE` in bundle directory.
* `{xxxx.y.z}`: `FACTORY_IMAGE_ARCHIVE` version for `finalize_bundle` to
  download. `finalize_bundle` will always download `FACTORY_IMAGE_ARCHIVE` if
  toolkit is set to non-local.

#### Non-local resources

Currently non-local resources can only be accessed by Google internally.
Partners can only use local resources.

#### Running finalize_bundle command

With `MANIFEST.yaml`, we can run `finalize_bundle` now.

**We strongly recommend you to install `lbzip2`,
which can help you reduce `finalize_bundle` execution time by a lot.**

To get `finalize_bundle`, please checkout factory repository:

    git clone https://chromium.googlesource.com/chromiumos/platform/factory

Create a directory with name `factory_bundle_{board}_{bundle_name}`:

    mkdir factory_bundle_samus_20170628_dvt
    cd factory_bundle_samus_20170628_dvt/

Prepare firmware updater if you want to use local firmware updater:

    ln /path/to/chromeos-firmwareupdate firmware/

Prepare release image if you want to use local release image:

    ln /path/to/chromeos_9334.72.0_samus_recovery_stable-channel_mp-v3.bin release_image/

Prepare test image if you want to use local test image:

    tar -JxvC test_image/ </path/to/ChromeOS-test-R58-9334.72.0-samus.tar.xz

Extract factory zip (`FACTORY_IMAGE_ARCHIVE`) if you want to use a local one:

    unzip /path/to/ChromeOS-factory-R61-9688.0.0-samus.zip

Prepare HWID bundle if you want to use local HWID bundle:

    ln /path/to/hwid_v3_bundle_SAMUS.sh hwid/

Add your `MANIFEST.yaml` if you don't have it in factory zip or you want to use
a new one:

    cp /path/to/MANIFEST.yaml MANIFEST.yaml

Finally, run `finalize_bundle`:

    /path/to/factory/bin/finalize_bundle .

After minutes, you will get a complete factory bundle. Cheers!

[CPFE]: https://www.google.com/chromeos/partner/fe/
[DLM]: https://chromeos.google.com/partner/dlm/device/list