ALSA Conformance Test is a tool to verify the correctness and performance of audio drivers. It can be used to verify the quality of audio bringup and prevent regression. The tool has found many audio issues on different devices.
Make sure these packages are on the host to compile the tool.
sudo apt-get update sudo apt-get upgrade sudo apt-get install git pkg-config build-essential libasound2-dev
Make sure these packages are on the device under test(DUT) for testing.
sudo apt-get update sudo apt-get upgrade sudo apt-get install alsa-utils libasound2-dev
NOTE: ChromeOS has already had these packages.
The tool is a built-in binary in the ChromeOS test image. There are two ways to build ALSA conformance test manually.
(inside the cros chroot) emerge-${BOARD} audiotest
The binary will be in /build/${BOARD}/usr/bin/alsa_conformance_test.
The command cros deploy ${DUT_IP} audiotest can deploy this tool to a device. The binary will be in /usr/bin/.
NOTE: The built-in binary is in /usr/local/bin where the developer root is mounted at. The location is different from building by yourself. Make sure that you use the correct binary before testing.
git clone https://chromium.googlesource.com/chromiumos/platform/audiotest cd audiotest make alsa_conformance_test
The default binary is in ./alsa_conformance_test/alsa_conformance_test. You can set the variable OUT to change the build directory. For example make alsa_conformance_test OUT=/path/to/builddir changes the output to /path/to/builddir/alsa_conformance_test/alsa_conformance_test.
alsa_conformance_test [OPTIONS]
Notice that some devices may not work before the correct UCM (Use Case Manager) is set. In ChromeOS, the UCM should be set automatically when selecting audio nodes from UI. Users can also find the UCM file for each supported sound card in /usr/share/alsa/ucm/ and set it manually by alsaucm.
Example: To enable the headphone specified on kbl_r5514_5663_max.
alsaucm -c kbl_r5514_5663_max set _verb HiFi set _enadev "Headphone"
[name] [type] [channels] [format] [rate] [period] [block_size] [durations] # comment [type] could be either `PLAYBACK` or `CAPTURE` # comment eg: hw:0,0 PLAYBACK 2 S16_LE 48000 240 240 10 # Example
These are the functions that ALSA conformance test covers.
Sometimes audio device claims that it can support some params but it actually does not. In this case, snd_pcm_hw_params fails. Therefore, verifying all params can be set correctly is necessary.
---------PRINT PARAMS--------- PCM name: hw:0,0 card: acpd7219m98357 [acpd7219m98357] device: Playback da7219-hifi-0 [] stream: PLAYBACK access type: MMAP_INTERLEAVED format: S16_LE channels: 2 rate: 48000 fps period time: 5000 us period size: 240 frames buffer time: 160000 us buffer size: 7680 frames
A specific UCM configuration file is sometimes required for a USB audio device. If the card name is the default USB card name, like ‘USB Audio Device’ it will not be possible to add a UCM configuration file because UCM files are matched by card name.
Device Information Name: hw:3,0 Card: Device [USB Audio Device] Device: USB Audio [USB Audio] Stream: PLAYBACK Format: ['S16_LE'] Channels: [2] Rate: [44100, 48000] Period_size range: [45, 48000] Buffer_size range: [90, 96000] Mixer: Mixer name: PCM Mixer index: 0 Mixer has_volume: 1 Mixer db_range: [-9999999, 0] Mixer volume_range: [0, 37]
If the device is a USB audio device, it is very important that the USB audio mixer control exposes reasonable values. Otherwise, applications may rely on incorrect values and lead to undefined behavior.
The following are the requirements for the USB audio mixer control name for playback:
The following are the requirements for the USB audio mixer control name for capture:
Device can only have one playback/capture mixer control system can recognize. Otherwise system will fail to know which control should be select in the system.
If the mixer control does not have the ability to control volume, there is likely something wrong with it. The main function of a mixer control for a USB audio device is to control volume.
The volume db range must be within a reasonable range. If the volume db range is not reasonable, the system may use it as a basis for its actions, which could cause incorrect behavior.
The system will use the playback volume range to calculate the percentage of volume that should be increased with one volume up/down event. If the value is less than 10, it will cause incorrect behavior. The minimum value for the volume range is 10.
This headset is Cyber Acoustics, and the device info is
Device Information Name: hw:3,0 Card: Device [USB Audio Device] Device: USB Audio [USB Audio] Stream: PLAYBACK Format: ['S16_LE'] Channels: [2] Rate: [44100, 48000] Period_size range: [45, 48000] Buffer_size range: [90, 96000] Mixer: Mixer name: PCM Mixer index: 0 Mixer has_volume: 1 Mixer db_range: [-9999999, 0] Mixer volume_range: [0, 37]
The number of samples consumed / time must be the same as the sampling rate. For short-term, it can be checked every time when writing a block into the device by calculating samples_diff / time_diff.
TIME_DIFF(s) HW_LEVEL READ RATE 0.000780757 32 0 40985.863719 0.000791478 72 0 50538.359879 0.000834111 112 0 47955.248162 0.000838265 152 0 47717.607201
For long-term, the tool can record all data when hw_level changes. And then, it will use linear regression to find out the measured rate. If measured rate is not close to the sampling rate, or standard error from linear regression is too large, the device doesn’t have enough stability of rate.
rate: 48001.729135 rate error: 7.050352
The script only uses the long-term rate to determine pass or fail because an unstable short-term rate may not affect the whole stability.
Explaination of fields
Explaination of point merge
TIME_DIFF(s) HW_LEVEL PLAYED DIFF RATE 0.001000772 376 1064 48 47962.972585 0.001000089 328 1112 48 47995.728380 0.001000082 280 1160 48 47996.064323 0.001016092 232 1208 48 47239.816867 0.000983772 424 1256 48 48791.793220 0.000998709 400 1280 24 24031.024052 0.000082259 376 1304 24 291761.387812 [Merged]
In the last two lines above, the device consumes 24 samples twice in a short time instead of 48 samples in the fixed period and result in higher rate error. To reduce the error, points with TIME_DIFF less than merge_threshold_t and SAMPLES_DIFF less than merge_threshold_sz will be merged. The merge_threshold_sz is determined automatically by the test and merge_threshold_t can be set by --merge_threshold option. With the --merge_threshold set to 0.0001, the last two points will be merged, and only the latter point is counted in linear regression. The result of point merge is the same as below:
TIME_DIFF(s) HW_LEVEL PLAYED DIFF RATE 0.001000772 376 1064 48 47962.972585 0.001000089 328 1112 48 47995.728380 0.001000082 280 1160 48 47996.064323 0.001016092 232 1208 48 47239.816867 0.000983772 424 1256 48 48791.793220 0.001080968 376 1304 48 44404.6447258
If a device consumes too large a block at once, we cannot maintain the buffer level at low. This affects continuous latency. To measure this, the tool can monitor hw_ptr and see whether the largest step during monitoring is small enough.
step min: 8 step max: 288
The runtime of APIs is also important. If it needs a long time to open PCM device, it might affect stream efficiency or cause long latency.
---------TIMER RESULT---------
Total_time(s) Counts Averages(s)
snd_pcm_open 0.003351390 1 0.003351
snd_pcm_hw_params 0.039304997 1 0.039305
snd_pcm_hw_params_any 0.000055113 1 0.000055
snd_pcm_sw_params 0.000017586 1 0.000018
snd_pcm_prepare 0.000007613 1 0.000008
snd_pcm_start 0.000774880 1 0.000775
snd_pcm_avail 0.807074662 246177 0.000003
precision: 0.000000001
Without any help from users, we can verify the correctness of recorded samples by checking whether all of them are zeros. The tool shows whether each channel is a zero channel, which means all samples in this channel are zeros. Usually the recorded samples should not be all zeros.
zero channels: 0 1 // Channel 0 is not a zero channel but channel 1 is.
The basic request of the audio device is not causing any underrun or overrun.
Underrun happens when there are no frames to playback on a running device. It makes the device play obsolete samples in the buffer, which sounds like glitches.
Overrun happens when frames in a device buffer are more than buffer size. It makes the device have a long delay and the latest recorded samples will overlay the old ones.
The block size is the size ALSA conformance test puts data into or gets data from a device buffer. For playback, it puts data into the device once the remaining frames in the device buffer are smaller than the block size. For capture, it gets data from a device once the remaining frames in the device buffer are larger than the block size. Users can adjust the block size to detect whether this size might cause underrun or overrun.
number of underrun: 0 number of overrun: 0
These are some examples to show how to use ALSA conformance test.
hw:{CARD},{DEVICE}. Users can get it by the command aplay -l and arecord -l.> aplay -l **** List of PLAYBACK Hardware Devices **** card 0: sofglkda7219max [sof-glkda7219max], device 0: Speakers (*) [] Subdevices: 1/1 Subdevice #0: subdevice #0 card 0: sofglkda7219max [sof-glkda7219max], device 1: Headset (*) [] Subdevices: 1/1 Subdevice #0: subdevice #0 card 0: sofglkda7219max [sof-glkda7219max], device 5: HDMI1 (*) [] Subdevices: 1/1 Subdevice #0: subdevice #0 card 0: sofglkda7219max [sof-glkda7219max], device 6: HDMI2 (*) [] Subdevices: 1/1 Subdevice #0: subdevice #0 card 0: sofglkda7219max [sof-glkda7219max], device 7: HDMI3 (*) [] Subdevices: 1/1 Subdevice #0: subdevice #0
From this result, the PCM name for internal speaker is hw:0,0 and for headset is hw:0,1. We will use hw:0,0 as a playback and hw:0,1 as capture device for the below examples.
alsa_conformance_test -P hw:0,0 --dev_info_only
alsa_conformance_test -P hw:0,0
alsa_conformance_test -C hw:0,1
alsa_conformance_test -P hw:0,0 -r 44100 -f S32_LE
alsa_conformance_test -P hw:0,0 -d 0.1 --debug
alsa_conformance_test -P hw:0,0 -C hw:0,1
alsa_conformance_test -P hw:0,0 --iterations 10
> cat ./device hw:0,0 PLAYBACK 2 S16_LE 48000 240 240 1 hw:0,1 CAPTURE 2 S32_LE 44100 240 240 2 > alsa_conformance_test --device_file ./device
For more convenience, this is a script that can test the basic funtions of ALSA PCM device automatically.
alsa_conformance_test.py [-h] [-C INPUT_DEVICE] [-P OUTPUT_DEVICE]
[--rate-criteria-diff-pct RATE_CRITERIA_DIFF_PCT]
[--rate-err-criteria RATE_ERR_CRITERIA]
[--json] [--log-file LOG_FILE]
[--test-suites [...]]
The result will show pass or fail.
> alsa_conformance_test.py -P hw:0,0
11 passed, 0 failed
Device Information
Name: hw:0,0
Card: sof-glkda7219max
Stream: PLAYBACK
Format: ['S16_LE', 'S32_LE']
Channels: [2]
Rate: [44100, 48000, 96000, 192000]
Period_size range: [16, 8192]
Buffer_size range: [32, 16384]
Test card name
Test card name is not in the block list: pass
Test Params
Set channels 2: pass
Set format S16_LE: pass
Set format S32_LE: pass
Set rate 44100: pass
Set rate 48000: pass
Set rate 96000: pass
Set rate 192000: pass
Test Rates
Set rate 44100: pass
Set rate 48000: pass
Set rate 96000: pass
Set rate 192000: pass
Test All Pairs
Set channels 2, format S16_LE, rate 44100: pass
Set channels 2, format S16_LE, rate 48000: pass
Set channels 2, format S16_LE, rate 96000: pass
Set channels 2, format S16_LE, rate 192000: pass
Set channels 2, format S32_LE, rate 44100: pass
Set channels 2, format S32_LE, rate 48000: pass
Set channels 2, format S32_LE, rate 96000: pass
Set channels 2, format S32_LE, rate 192000: pass
Test USB mixer - skip
> alsa_conformance_test.py -P hw:1,0
20 passed, 0 failed
Device Information
Name: hw:1,0
Card: Wired [Zone Wired]
Device: USB Audio [USB Audio]
Stream: PLAYBACK
Format: ['S16_LE', 'S24_3LE']
Channels: [2]
Rate: [44100, 48000, 96000]
Period_size range: [45, 131072]
Buffer_size range: [90, 262144]
Mixer:
Mixer name: PCM
Mixer index: 0
Mixer has_volume: 1
Mixer db_range: [-6562, 0]
Mixer volume_range: [0, 175]
Test card name
Test card name is not in the block list: pass
Test Params
Set channels 2: pass
Set format S16_LE: pass
Set format S24_3LE: pass
Set rate 44100: pass
Set rate 48000: pass
Set rate 96000: pass
Test Rates
Set rate 44100: pass
Set rate 48000: pass
Set rate 96000: pass
Test All Pairs
Set channels 2, format S16_LE, rate 44100: pass
Set channels 2, format S16_LE, rate 48000: pass
Set channels 2, format S16_LE, rate 96000: pass
Set channels 2, format S24_3LE, rate 44100: pass
Set channels 2, format S24_3LE, rate 48000: pass
Set channels 2, format S24_3LE, rate 96000: pass
Test USB mixer
Test usb mixer number correctness: pass
Test usb mixer has_volume correctness: pass
Test usb mixer playback db range correctness: pass
Test usb mixer playback volume range correctness: pass