NAME¶
squishyball - perform sample comparison testing on the command line
SYNOPSIS¶
squishyball [
options] fileA [fileB [
fileN...]] [>
results.txt]
DESCRIPTION¶
squishyball is a simple command-line utility for performing double-blind
A/B, A/B/X or X/X/Y testing on the command line. The user specifies two input
files to be compared and uses the keyboard during playback to flip between the
randomized samples to perform on-the-fly comparisons. After a predetermined
number of trials,
squishyball prints the trial results to stdout and
exits. Results (stdout) may be redirected to a file without affecting
interactive use of the terminal.
squishyball can also be used to perform casual, non-randomized
comparisons of groups of up to ten samples; this is the default mode of
operation.
TEST TYPES¶
- -a --ab
- Perform A/B test on two input samples.
A/B testing randomizes the order of two input samples and presents them,
unnamed, as sample 'A' and sample 'B'. In each trial the user selects A or
B as the preferred sample. The samples are then re-randomized for the next
trial. This test is useful for establishing relative or preferred quality
between two samples.
- -b --abx
- Perform A/B/X test on two input samples.
A/B/X presents two input samples, unrandomized, as sample 'A' and sample
'B'. A third sample 'X' is chosen randomly from either 'A' or 'B'. In each
trial, the user selects A or B as the sample believed to be the same as X.
X is then re-randomized for the next trial. This test is useful for
determining if any differences are audible between two samples and to what
confidence level.
Note that because the A and B samples are not randomized (they are presented
in the order given on the command line as per standard industry practice),
an A/B/X test does not eliminate ordering bias. A stronger version of this
test that randomizes all samples is the X/X/Y test below.
- -c --casual
- Perform casual comparison of up to ten samples (default).
Casual comparison mode does not randomize the input samples or perform
multiple trials. It simply provides a convenient way to rapidly flip back
and forth within a group of up to ten samples.
- -x --xxy
- Perform randomized X/X/Y test on two input samples.
X/X/Y testing is a form of A/B/X testing in which the order of all samples
is randomized and the position of the 'X' sample is not known ahead of
time to be in the third position. In each trial, the user selects which of
sample 1, 2 or 3 is believed to be the sample that is different from the
other two. This test is useful for determining if any differences are
audible between two samples and to what confidence level. It is a stronger
version of the A/B/X test that eliminates sample order bias.
OTHER OPTIONS¶
- -B --beep-flip
- Mark transitions between samples with a short beep.
- -d --device N|device
- If a number, output to Nth available sound device. If a device name, use
output device matching that device name. The backend audio driver is
selected automatically based on the device name provided.
- -D --force-dither
- Always use dither when down-converting to 16-bit samples for playback on
audio devices that do not support 24-bit playback. By default,
uncompressed samples are always dithered, but lossy formats (such as
Vorbis and Opus) are simply rounded. See the section CONVERSION AND
DITHER below for more details.
- -e --end-time
[[hh:]mm:]ss[. ff]
- Set sample end time for playback.
- -g --gabbagabbahey | --score-display
- Show running score and probability figures of trials so far while testing.
Can only be used with -a, -b, or -x.
- -h --help
- Print usage summary to stdout and exit.
- -M --mark-flip
- Mark transitions between samples with a short period of silence
(default).
- -n --trials n
- Set desired number of comparison trials (default: 20).
- -N --do-not-normalize
- Do not perform autonormalization to avoid clipping when sample values
exceed the maximum playback range in floating point, lossy, and downmixed
samples.
- -r --restart-after
- Set 'restart-after mode', where sample playback restarts from start point
after every trial.
- -R --restart-every
- Set 'restart-every mode', where sample playback restarts from start point
after 'flip' as well as after every trial.
- -s --start-time
[[hh:]mm:]ss[. ff]
- Set start time within sample for playback
- -S --seamless-flip
- Do not mark transitions between samples; flip with a seamless
crossfade.
- -t --force-truncate
- Always round/truncate (never dither) when down-converting samples to
16-bit for playback on audio devices that do not support 24-bit output.
See the section CONVERSION AND DITHER below for more details.
- -v --verbose
- Produce more and more detailed progress information and warnings.
- -V --version
- Print version and exit.
- -1 --downmix-to-mono
- Downmix all multichannel samples to mono at load time.
- -2 --downmix-to-stereo
- Downmix all surround samples to stereo at load time.
KEYBOARD INTERACTION¶
- a, b, x
- Switch between A and B samples (A/B mode), or A, B and X samples (A/B/X
mode).
- A, B
- Select A or B as preferred sample (A/B mode), or sample A or sample B as
match to sample X (A/B/X testing mode).
- 1, 2, 3...
- Switch between first, second, third [etc] samples (X/X/Y testing mode,
casual comparison mode).
- !, @, #
- Indicate the 'odd sample out' as sample 1, 2, or 3 (X/X/Y testing
mode).
- <del>, <ins>
- Undo/redo previous trial result selection.
- <enter>
- Choose current sample for this trial.
- <-, ->
- Seek back/forward two seconds, +shift for ten seconds.
- <up/down>
- Select sample in sample list (casual mode).
- <space>
- Pause/resume playback.
- <backspace>
- Reset playback to start point.
- e
- Set end playback point to current playback time (see also -e above).
- E
- Reset end playback time to end of sample.
- f
- Toggle through beep-flip/mark-flip/seamless-flip modes (see -B,
-M, and -S above).
- r
- Toggle through restart-after/restart-every/no-restart modes (see -r
and -R above).
- s
- Set start playback point to current playback time (see also -s
above).
- S
- Reset start playback time to beginning of sample.
- ?
- Print this keymap. The keymap will not be printed if the terminal has
insufficient rows to do so.
- ^c
- Abort testing early.
SUPPORTED FILE TYPES¶
- WAV/WAVEX
- 8-, 16-, 24-bit linear integer PCM (format 1), 32-bit float (format
3)
- AIFF/AIFF-C
- 8-, 16-, 24-bit linear integer PCM, 32-bit floating point
- FLAC/OggFLAC
- 16- and 24-bit
- SW
- Mono signed 16-bit little endian 48000Hz raw with a .sw extension
- OggVorbis
- all Vorbis I files
- OggOpus
- all Opus files
CONVERSION¶
squishyball 'reconciles' files to identical channel ordering, length and
bit-depth before playback begins so that CPU and memory resource usage during
playback should be identical for all samples. When 24-bit playback is
available and at least one sample is 24-bit or greater (ie, 32-bit or float),
all samples are converted/promoted to 24 bits. If 24-bit playback is
unavailable, all samples are demoted to 16 bits. Note that Opus and Vorbis
files are both considered to be natively float formats.
NORMALIZATION¶
squishyball checks files for clipping at load time. By default,
squishyball will automatically normalize all float inputs by the amount
needed to avoid clipping any one. Automatic normalization can be disabled with
the
-N option. Integer samples are checked for clipping heuristically;
two or more consecutive full-range values in a channel count as clipped.
Out-of-range integer values cannot be recovered; in this case,
squishyball issues a warning and performs no normalization based on the
integer clipping.
Downmixing samples to mono with
-1 or stereo with
-2 will also
likely require normalization to avoid clipping; as above,
squishyball
will automatically normalize all inputs by the amount necessary to avoid
clipping in any one unless
-N is specified.
DITHER¶
Down-conversions of uncompressed and lossless samples (WAV, AIF[C], FLAC, SW) to
16-bit are dithered using a simple white TPDF. Lossy-encoded samples (Vorbis
and Opus) are dithered to 16-bit only if one or more uncompressed/lossless
inputs are also being dithered. Normalization also triggers dithering of all
input samples (uncompressed, lossless and lossy) upon conversion to 16 bit.
-D overrides the default behavior and forces unconditional dithering of
all 16-bit down-conversions. Similarly,
-t forces unconditional rounded
truncation in all cases, disabling dither completely.
Conversions to 24-bit are never dithered.
IMPORTANT USAGE NOTES¶
- Playback Depth and Rate
-
Many modern audio playback systems (such as PulseAudio or the ALSA 'default'
device) give no means of determining if the requested playback paramters
are actually being used by the hardware, or if the audio system is
helpfully converting everything to some other supported depth/rate. When
using these systems, squishyball has no way of knowing if
16-/24-bit playback or sample rate is being honored. Automatic conversion
can affect audible playback quality; be careful to verify actual system
behavior.
- Flip-Mode Choice
-
Silent Mode smoothly transitions between samples. It allows the most
direct comparison between signals without any intervening auditory
distraction. However, the temporary combination of different signals may
cause unintended cancellation and comb-filtering effects that can give
away the 'unknown' sample just as a 'pop' from an instantaneous transition
would.
Mark Mode quickly fades to silence before flipping to another
sample, marking the transition. Because the samples never overlap,
crosslap artifacts cannot contaminate trial results. However, the audible
dip between samples may distract from listening, potentially making it
slightly more difficult to detect legitimate artifacts.
Beep Mode is similar to mark mode but adds a soft 'beep' to mark
where the transition occurs. It makes the transition point especially
obvious. It does not crosslap the samples; one sample is faded completely
before the second is mixed in as in mark mode.
AUTHORS¶
Monty <monty@xiph.org>
SEE ALSO¶
abx-comparator(1),
rateit(1),
ogg123(1),
oggdec(1),
opusdec(1),
flac(1)