Introduction

The sym_upload tool is able to operate in sym-upload-v2 protocol mode, in addition to the legacy protocol (which will be referred to as sym-upload-v1 for the rest of this document). For now sym-upload-v2 is HTTP/REST-based but it could be extended to operate over gRPC instead, in the future.

Table of Contents

Why

Using sym_upload in sym-upload-v2 protocol mode has the following features beyond sym-upload-v1:

  • Authentication via key (arbitrary secret).
  • Symbol identifier (product of debug_file and debug_id, as recorded in output from dump_syms) can be checked against existing symbol information on server. If it's present, then the upload is skipped entirely.

How

Uploading

Uploading with sym_upload

Uploading in sym-upload-v2 protocol mode is easy. Invoke sym_upload like

$ ./sym_upload -p sym-upload-v2 [-k <API-key>] <symbol-file> <API-URL>

Where symbol-file is a symbol file created by dump_syms, API-URL is the URL of your sym-upload-v2 API service (see next section for details), and API-key is a secret known to your uploader and server.

For more options see sym_upload --help.

Uploading with curl

As an example, if:

  • Your API's URL was “https://sym-upload-api”.
  • Your service has assigned you key “myfancysecret123”.
  • You wanted to upload the symbol file at “path/to/file_name”, with debug_file being “file_name” and debug_id being “123123123123123123123123123”. Normally you would read these values from “path/to/file_name”, which in turn was generated by dump_syms.

Then you might run:

$ curl https://sym-upload-api/symbols/file_name/123123123123123123123123123:checkStatus?key=myfancysecret123

And, upon seeing that this debug_file/debug_id combo is missing from symbol storage then you could run:

$ curl --request POST https://sym-upload-api/uploads:create?key=myfancysecret123

Which returns upload_url “https://upload-server/42?creds=shhhhh” and upload_key “42”. Next you upload the file directly like:

$ curl -T path/to/file_name "https://upload-server/42?creds=shhhhh"

Once the HTTP PUT is complete, run:

$ curl --header "Content-Type: application/json" \
    --request POST \
    --data '{symbol_id:{"debugFile":"file_name",'\
        '"debugId":"123123123123123123123123123"}}' \
    https://sym-upload-api/uploads/42:complete?key=myfancysecret123

Serving the sym-upload-v2 Protocol

The protocol is currently defined only in HTTP/REST. There are three necessary REST operations to implement in your service:

  • /symbols/<debug_file>/<debug_id>:checkStatus?key=<key>
  • /uploads:create?key=<key>
  • /uploads/<upload_key>:complete?key=<key>

Authenticate Using key

The query string arg key contains some secret that both the uploader and server understand. It is up to the service implementer to decide on what constitutes a valid key, how the uploader acquires one, and how to handle requests made with invalid ones.

Symbol checkStatus

/symbols/<debug_file>/<debug_id>:checkStatus?key=<key>

This operation expects an empty (or no) JSON payload in the request.

This operation should return the status of the symbol file uniquely identified by the given debug_file and debug_id. JSON schema:

{
    "type": object",
    "properties": {
        "status": {
            "type": "string",
            "enum": ["STATUS_UNSPECIFIED", "MISING", "FOUND"],
            "required": true
        }
    }
}

Where MISSING denotes that the symbol file does not exist on the server and FOUND denotes that the symbol file exists on the server.

Upload create

/uploads:create?key=<key>

This operation expects an empty (or no) JSON payload in the request.

This operation should return a URL that uploader can HTTP PUT their symbol file to, along with an “upload key” that can be used to notify the service once the file upload is completed. JSON schema:

{
    "type": "object",
    "properties": {
        "upload_url": {
            "type: "string",
            "required": true
        },
        "upload_key": {
            "type": "string",
            "required": true
        }
    }
}

Since this REST API operation can be authenticated via the key query string arg, the service can return a URL that encodes permission delegation to the upload endpoint resource and thereby constrain the ability to upload to those with valid keys.

Uploading the Symbol File

Note that the actual symbol upload step is not part of the REST API. The upload URL obtained in the above operation is meant to be used as the endpoint for a normal HTTP PUT request for the contents of the symbol file. Once that HTTP PUT request is completed use the upload complete operation.

Upload complete

/uploads/<upload_key>:complete?key=<key>

This operation expects a JSON payload in the HTTP request body with the following schema:

{
    "type": "object",
    "properties": {
        "symbol_id": {
            "type": "object",
            "properties": {
                "debug_file": {
                    "type": "string",
                    "required": true
                },
                "debug_id": {
                    "type": "string",
                    "required": true
                }
            }
        }
    }
}

This operation should cause the symbol storage back-end (however implemented) to consume the symbol file identified by upload_key. It is up to the service implementation to decide how uploads are assigned upload_keys and how to retrieve a completed upload by its upload_key. If the symbol file cannot be found, is malformed, or the operation cannot be completed for any other reason then an HTTP error will be returned. JSON schema of non-error responses:

{
    "type": "object",
    "properties": {
        "result": {
            "type": string,
            "enum": ["RESULT_UNSPECIFIED", "OK", "DUPLICATE_DATA"],
            "required": true
        }
    }
}

Where OK denotes that the symbol storage was updated with the new symbol file and DUPLICATE_DATA denotes that the symbol file data was identical to data already in symbol storage and therefore nothing changed.