Scroll to navigation

vd(1) 1 (quick reference guide) vd(1)

NAME

VisiDataa terminal utility for exploring and arranging tabular data

SYNOPSIS

vd [options] [input ...]

vd [options] --play cmdlog [-w waitsecs] [--batch] [-o output] [field=value]

vd [options] [input ...] +toplevel:subsheet:col:row

DESCRIPTION

VisiData is an easy-to-use multipurpose tool to explore, clean, edit, and restructure data. Rows can be selected, filtered, and grouped; columns can be rearranged, transformed, and derived via regex or Python expressions; and workflows can be saved, documented, and replayed.

REPLAY MODE

, --play=cmdlog
cmdlog within the interface
, --replay-wait=seconds
seconds between commands
, --batch
replay in batch mode (with no interface)
, --output=file
save final visible sheet to file as .tsv
--play to move cursor cell-by-cell
field=value
field}" in cmdlog contents with value

Commands During Replay

pause/resume replay
execute next row in replaying sheet
cancel current replay

GLOBAL COMMANDS

All keystrokes are case sensitive. The ^ prefix is shorthand for .

Keystrokes to start off with

^Q
abort program immediately
^C
cancel user input or abort all async threads on current sheet
abort all secondary threads
q
quit current sheet
gq
quit all sheets (clean exit)

^H
view this man page
view sheet of command longnames and keybindings
longname
longname

U
(requires enabled options.undo)
R
(requires enabled options.undo)

Cursor Movement

go as expected
h j k l
go left/down/up/right
go all the way to the left/bottom/top/right of sheet
G gg
go all the way to the bottom/top of sheet
scroll one page back/forward
scroll current row to center of screen

(Ctrl+^)
jump to previous sheet (swaps with current sheet)

regex forward/backward in current column
regex
regex forward/backward over all visible columns
expr
expr forward/backward in current column (with column names as variables)
n N
go to next/previous match from last regex search

< >
go up/down current column to next value
go up/down current column to next null value
{ }
go up/down current column to next selected row

c regex
regex
r regex
regex
number
number (0-based)

H J K L
slide current row/column left/down/up/right
slide current row/column all the way to the left/bottom/top/right of sheet
number
number positions to the left/down/up/right

scroll one left/down/up/right

Column Manipulation

_ (underscore)
toggle width of current column between full and default width
g_
toggle widths of all visible columns between full and default width
z_ number
number
number
adjust widths of all visible columns to Ar number

- (hyphen)
hide current column
reduce width of current column by half
unhide all columns

toggle/unset current column as a key column
set type of current column to str/int/float/currency/date/len
^
edit name of current column
g^
set names of all unnamed visible columns to contents of selected rows (or current row)
z^
set name of current column to combined contents of current cell in selected rows (or current row)
set name of all visible columns to combined contents of current column for selected rows (or current row)

= expr
expr, with column names, and attributes, as variables
g= expr
set current column for selected rows to result of Python expr
expr
set current column for selected rows to the items in result of Python sequence expr
z= expr
expr

i (iota)
add column with incremental values
gi
set current column for selected rows to incremental values
zi step
add column with values at increment step
gzi step
set current column for selected rows at increment step

' (tick)
add a frozen copy of current column with all cells evaluated
open a frozen copy of current sheet with all visible columns evaluated
add/reset cache for current/all visible column(s)

: regex
add new columns from regex split; number of columns determined by example row at cursor
; regex
add new columns from capture groups of regex (also requires example row)
; expr
expr, with columnNames as variables
* regex/subst
add column derived from current column, replacing regex with subst (may include \1 backrefs)
regex/subst
regex with subst (may include \1 backrefs)

( g(
[3]) or dicts (e.g. {3}) fully
depth
[3]) or dicts (e.g. {3}) to given depth (0= fully)
)
unexpand current column; restore original column and remove other columns at this level
[3]) or dicts (e.g. {3}) within that column

Row Selection

s t u
select/toggle/unselect current row
gs gt gu
select/toggle/unselect all rows
zs zt zu
select/toggle/unselect all rows from top to cursor
select/toggle/unselect all rows from cursor to bottom
| \ regex
regex in current column
regex
regex in any visible column
expr
expr in any visible column
, (comma)
select rows matching display value of current cell in current column
select rows matching display value of current row in all visible columns
z, gz,
select rows matching typed value of current cell/row in current column/all visible columns

Row Sorting/Filtering

[ ]
sort ascending/descending by current column; replace any existing sort criteria
g[ g]
sort ascending/descending by all key columns; replace any existing sort criteria
z[ z]
sort ascending/descending by current column; add to existing sort criteria
sort ascending/descending by all key columns; add to existing sort criteria
"
open duplicate sheet with only selected rows
open duplicate sheet with all rows
open duplicate sheet with deepcopy of selected rows

Editing Rows and Cells

a za
append a blank row/column; appended columns cannot be copied to clipboard
ga gza number
number blank rows/columns
d gd
delete (cut) current/selected row(s) and move to clipboard
y gy
yank (copy) current/all selected row(s) to clipboard
zy gzy
yank (copy) contents of current column for current/selected row(s) to clipboard
zd gzd
delete (cut) contents of current column for current/selected row(s) and move to clipboard
p P
paste clipboard rows after/before current row
zp gzp
set cells of current column for current/selected row(s) to last clipboard value
Y gY
options.clipboard_copy_cmd)
zY gzY
options.clipboard_copy_cmd)
f
fill null cells in current column with contents of non-null cells up the current column
e text
edit contents of current cell
ge text
set contents of current column for selected rows to text

Commands While Editing Input

Enter ^C
accept/abort input
open external $EDITOR to edit contents
reload initial value
go to beginning/end of line
go back/forward one character
(arrow)
go back/forward one word
delete previous/current character
transpose previous and current characters
^U ^K
clear from cursor to beginning/end of line
paste from cell clipboard
delete previous/current character
toggle insert mode
set contents to previous/next in history
Tab Shift+Tab
autocomplete input (when available)
Arrow and re-enter edit mode

Data Toolkit

o input
open input in VisiData
filename
save current/all sheet(s) to filename in format determined by extension (default .tsv)
filename ends in a /, a directory will be created.
filename
save current column only to filename in format determined by extension (default .tsv)
filename.vd
CommandLog to filename.vd file
open new blank sheet with one column
open new sheet that has rows and columns of current sheet transposed

+ aggregator
aggregator to current column (see Frequency Table)
aggregator
aggregator over values in selected rows for current column
&
Sheets Stack
Sheets Stack

Data Visualization

. (dot)

If rows on the current sheet represent plottable coordinates (as in .shp or vector .mbtiles sources), . plots the current row, and g. plots all selected rows (or all rows if none selected).

Canvas-specific Commands

+ -
increase/decrease zoom level, centered on cursor
_ (underscore)
zoom to fit full extent
(underscore)
set aspect ratio
x xmin xmax
xmin/xmax on graph
y ymin ymax
ymin/ymax on graph
s t u
select/toggle/unselect rows on source sheet contained within canvas cursor
select/toggle/unselect rows on source sheet visible on screen
d
delete rows on source sheet contained within canvas cursor
delete rows on source sheet visible on screen
Enter
open sheet of source rows contained within canvas cursor
open sheet of source rows visible on screen
1 - 9
toggle display of layers
redraw all pixels on canvas
v
show_graph_labels option
zoom in/out of canvas
set canvas cursor
scroll canvas

Split Screen

Z

Split Window specific Commands

Other Commands

CommandLog
toggle sheet-specific visibility (multi-line rows on Sheet, legends/axes on Graph)

^E g^E
view traceback for most recent error(s)
view traceback for error in current cell

^L
refresh screen
^R
reload current sheet
clear cache for current column
^Z
suspend VisiData process
^G
show cursor position and bounds of current sheet on status line
^V
show version and copyright information on status line
^P
Status History
m keystroke
keystroke No, and complete recording. Macro can then be executed everytime provided keystroke is used. Will override existing keybinding. Macros will run on current row, column, sheet.
gm
open an index of all existing macros. Can be directly viewed with Enter, and then modified with ^S.

^Y z^Y g^Y
open current row/cell/sheet as Python object
^X expr
expr and opens result as Python object
expr
expr, in context of current row, and open result as Python object
stmt
stmt in the global scope

Internal Sheets List

.
(Shift+V) browse list of core sheets
.
browse properties of files in a directory
.
browse, install, and (de)activate plugins
 
 
.
(Shift+C) edit column properties
.
(Shift+S) jump between sheets or join them together
.
(Shift+O) edit configuration options
.
(Shift+D) modify and save commands for replay
.
(Ctrl+E) view last error
.
(Ctrl+P) view history of status messages
.
(Ctrl+T) view, cancel, and profile asynchronous threads

 
.
(Shift+F) group rows by column value, with aggregations of other columns
.
(Shift+I) view summary statistics for each column
.
(Shift+W) group rows by key and summarize current column
.
(Shift+M) unpivot non-key columns into variable/value columns
.
(Shift+T) open new sheet with rows and columns transposed

INTERNAL SHEETS

VisiDataMenu (Shift+V)

(sheet-specific commands)

Directory Sheet

(global commands)
open-dir-current
open the Directory Sheet for the current directory
(sheet-specific commands)
Enter gEnter
open current/selected file(s) as new sheet(s)
^O g^O
open current/selected file(s) in external $EDITOR
^R z^R gz^R
reload information for all/current/selected file(s)

Plugins Sheet

Browse through a list of available plugins. VisiData needs to be restarted before plugin activation takes effect. Installation may require internet access.
(global commands)
open-plugins
open the Plugins Sheet
(sheet-specific commands)
install and activate current plugin
deactivate current plugin

METASHEETS

Columns Sheet (Shift+C)

Properties of columns on the source sheet can be changed with standard editing commands () on the Columns Sheet. Multiple aggregators can be set by listing them (separated by spaces) in the aggregators column. The 'g' commands affect the selected rows, which are the literal columns on the source sheet.
(global commands)
Columns Sheet with all visible columns from all sheets
(sheet-specific commands)
&
add column from concatenating selected source columns
toggle/unset selected columns as key columns on source sheet
aggregator
add Ar aggregator No to selected source columns
(hyphen)
hide selected columns on source sheet
set type of selected columns on source sheet to str/int/float/currency/date/len/floatsi
Enter
open a Frequency Table sheet grouped by column referenced in current row

Sheets Sheet (Shift+S)

open Sheets Stack, which contains only the active sheets on the current stack
(global commands)
Sheets Sheet, which contains all sheets from current session, active and inactive
number
number
(sheet-specific commands)
Enter
jump to sheet referenced in current row
push selected sheets to top of sheet stack
a
add row to reference a new blank sheet
gC gI
Columns Sheet/Describe Sheet with all visible columns from selected sheets
abort async threads for current/selected sheets(s)
save selected or all sheets
& jointype
merge selected sheets with visible columns from all, keeping rows according to jointype:
keep only rows which match keys on all sheets
keep all rows from first selected sheet
keep all rows from all sheets (union)
keep only rows NOT in all sheets
keep all rows from all sheets (concatenation)
copy first selected sheet, keeping all rows and sheet type, and extend with columns from other sheets
mostly keep all rows from first selected sheet, except prioritise cells with non-null/non-error values

Options Sheet (Shift+O)

(global commands)
(sheet-specific commands)
Enter e
edit option at current row
remove option override for this context

CommandLog (Shift+D)

(global commands)
open current sheet's CommandLog with all other loose ends removed; includes commands from parent sheets
open global CommandLog for all commands executed in the current session
open current sheet's CommandLog with the parent sheets commands' removed
(sheet-specific commands)
x
replay command in current row
gx
replay contents of entire CommandLog
^C
abort replay

DERIVED SHEETS

Frequency Table (Shift+F)

A Frequency Table groups rows by one or more columns, and includes summary columns for those with aggregators.
(global commands)
open Frequency Table, grouped by all key columns on source sheet
open one-line summary for all rows and selected rows
(sheet-specific commands)
s t u
select/toggle/unselect these entries in source sheet
Enter gEnter
open copy of source sheet with rows that are grouped in current cell / selected rows

Describe Sheet (Shift+I)

A Describe Sheet contains descriptive statistics for all visible columns.
(global commands)
Describe Sheet for all visible columns on all sheets
(sheet-specific commands)
select/unselect rows on source sheet that are being described in current cell
!
toggle/unset current column as a key column on source sheet
Enter
open a Frequency Table sheet grouped on column referenced in current row
open copy of source sheet with rows described in current cell

Pivot Table (Shift+W)

Set key column(s) and aggregators on column(s) before pressing on the column to pivot.
(sheet-specific commands)
Enter
open sheet of source rows aggregated in current pivot row
open sheet of source rows aggregated in current pivot cell

Melted Sheet (Shift+M)

Open Melted Sheet (unpivot), with key columns retained and all non-key columns reduced to Variable-Value rows.
(global commands)
regex
open Melted Sheet (unpivot), with key columns retained and regex capture groups determining how the non-key columns will be reduced to Variable-Value rows.

Python Object Sheet (^X ^Y g^Y z^Y)

(sheet-specific commands)
Enter
dive further into Python object
v
toggle show/hide for methods and hidden properties
gv zv
show/hide methods and hidden properties

COMMANDLINE OPTIONS

Add / to make subsequent CLI options sheet-specific (applying only to paths specified directly on the CLI). By default, CLI options apply to all sheets.

Options can also be set via the Options Sheet or a .visidatarc (see FILES).

=longname
longname before replay or regular launch; limited to bound commands
toplevel:subsheet:col:row
subsheet of toplevel at top-of-stack, and cursor at col and row; all arguments are optional

, --filetype=filetype
tsv set loader to use for filetype instead of file extension
, --confirm-overwrite=F
overwrite existing files without confirmation
=int 1
max time between press/release for click (ms)
=NoneType None
a value to be counted as null
=bool True
enable undo/redo
=int 0
max number of cache entries in each cached column
False
clean column/sheet names to be valid Python identifiers
=int 20
default column width
=int 10
default column height
=bool True
wordwrap text for multiline rows
False
confirm before quitting last sheet
False
exit on error and display stacktrace
=int 0
skip N rows before header
=int 1
parse first N rows as column names
False
load subsheets always (False) or lazily (True)
False
use 256 colors even if curses reports fewer
False
curses use default terminal colors
=str
note to display for pending cells
=str ?
cell note for an exception during formatting
=str !
cell note for an exception during computation
=str !
cell note for an exception during type conversion
=int 3
amount to scroll with scrollwheel
=str _
string to join sheet or column names
=str
string to join display values
False
wrap text to fit window width on TextSheet
=str tsv
specify default file type to save as
=str
filename to save binary profiling data
=int 0
minimum memory to continue loading and async processing
=str
basename of file to store persistent input history
=str utf-8
encoding passed to codecs.open
=str surrogateescape
encoding_errors passed to codecs.open
False
clear selected rows before new bulk selections
False
if no rows selected, if True, someSelectedRows returns all rows; if False, fails
=str
field delimiter to use for tsv/usv filetype
=str
" row delimiter to use for tsv/usv filetype
=str ?
replacement for newline character when saving to tsv
=str ?
replacement for tab character when saving to tsv
=int 0
visibility level (0=low, 1=high)
=int 1000
number of rows to check when expanding columns (0 = all)
=NoneType None
indent to use when saving json
False
sort object keys when saving to json
=str
column name to use for non-dict rows
=str
specify file type
=bool True
whether to prompt for overwrite confirmation on save
=str #ERR
error string to use while saving
=str
command to copy stdin to system clipboard
=str
command to get contents of system clipboard
False
a nicer selection interface for aggregators and jointype
=str mean stdev
numeric aggregators to calculate on Describe sheet
=int 0
number of bins for histogram of numeric columns
False
bin numeric columns into ranges
=float 0.0
time to wait between replayed commands, in seconds
False
insert movements during replay
=str ~/.visidata/
directory to load and store additional files
=str
string prefix for rowkey in the cmdlog
=str
file to autorecord each cmdlog action to
=str I
flags to pass to re.compile() [AILMSUX]
=int 0
maxsplit to pass to regex.split
=int 100
number of rows to sample for regex.split
=bool True
show axes and legend on graph
=str
list of distinct colors to use for plotting distinct objects
=float 2.0
amount to multiply current zoomlevel when zooming
=str
source of randomized startup messages
False
walk source path recursively on DirSheet
False
load hidden files on DirSheet
=str ~/.visidatarc
config file to exec in Python
=str
file.vd to replay
False
replay in batch mode (with no interface and all status sent to stdout)
=NoneType None
save the final visible sheet to output at the end of replay
=str
longnames to preplay before replay
=str plugins
imports to preload before .visidatarc (command-line only)
=float 1.0
start value for column increments
=str excel
dialect passed to csv.reader
=str ,
delimiter passed to csv.reader
=str "
quotechar passed to csv.reader
=bool True
skipinitialspace passed to csv.reader
=NoneType None
escapechar passed to csv.reader
=str
" lineterminator passed to csv.writer
False
sanitize input/output to handle edge cases, with a performance cost
=int 1000
number of rows to check for fixed width columns
=int 0
max number of fixed-width columns to create (0 is no max)
=str public
The desired schema for the Postgres database
=int 0
max next.url pages to follow in http response
=str <h2>{sheet.name}</h2>
table header when saving to html
=str n
(y/s/n) if save_dot includes all internet hosts separately (y), combined (s), or does not include the internet (n)
=bool True
whether to include edge labels on graphviz diagrams
False
parse PDF for tables instead of pages of text
=str https://visidata.org/plugins/plugins.jsonl
source of plugins sheet

DISPLAY OPTIONS

Display options can only be set via the Options Sheet or a .visidatarc (see FILES).

0
height of second sheet on screen
%.02f
default fmtstr to format for currency values
{:.02f}
default fmtstr to format for float values
{:.0f}
default fmtstr to format for int values
%Y-%m-%d
default fmtstr to strftime for date values
visible contents of a cell whose value is None
indicator that the contents are only partially visible
·
displayable character for odd whitespace
<
header note indicating more columns to the left
>
header note indicating more columns to the right
displayed contents for computation exception
1
width to use for unicode chars marked ambiguous
string to display in pending cells
bold magenta
color of note in pending cells
226 yellow
color of cell note for non-str types in anytype columns
220 yellow
color of row note on left edge
|
separator between columns
separator between key columns and rest of columns
|

↑↟⇞⇡⇧⇑
characters for ascending sort
↓↡⇟⇣⇩⇓
characters for descending sort
normal
the default color
bold
color of the column headers
underline
color of the bottom header row
reverse
color of the cursor row
bold
color of the cursor column
bold reverse
color of the header for the cursor column
246 blue
color of column separators
81 cyan
color of key columns
8
color of hidden columns on metasheets
215 yellow
color of selected rows
{sheet.longname} {sheet.nRows:9d} {sheet.rowtype}
right-side status format string
{sheet.shortcut}› {sheet.name}|
status line prefix
0
maximum length of left status line
|
separator between statuses
white
color of input keystrokes on status line
bold
status line color
red
error message color
yellow
warning message color
underline
top window status bar color
bold

active window status bar color
8
inactive window status bar color
green
color of system running smoothly
normal
cell color to use when editing cell
_
edit field fill character
·
substitute character for unprintables
*
histogram element character
50
width of histogram column
status indicator for active replay
status indicator for paused replay
green
color of replay status indicator
False
randomly choose attr from set of pixels instead of most common
238 blue
color of legend for hidden attribute
bold
color of selected graph points
bold
color for graph axis labels
green
color for rows pending add
reverse yellow
color for cells pending modification
red
color for rows pending delete
green
color of active clue

EXAMPLES

vd foo.tsv

open the file foo.tsv in the current directory

vd -f sqlite bar.db
open the file bar.db as a sqlite database

vd foo.tsv -n -f sqlite bar.db
open foo.tsv as tsv and bar.db as a sqlite database

vd -f sqlite foo.tsv bar.db
open both foo.tsv and bar.db as a sqlite database

vd -b countries.fixed -o countries.tsv
convert countries.fixed (in fixed width format) to countries.tsv (in tsv format)

vd postgres://username:passwordhostname:port/database
open a connection to the given postgres database

vd --play tests/pivot.vd --replay-wait 1 --output tests/pivot.tsv
replay tests/pivot.vd, waiting 1 second between commands, and output the final sheet to test/pivot.tsv

ls -l | vd -f fixed --skip 1 --header 0
parse the output of ls -l into usable data

ls | vd | lpr
interactively select a list of filenames to send to the printer

vd newfile.tsv
open a blank sheet named newfile if file does not exist

vd sample.xlsx +:sheet1:2:3
launch with at top-of-stack, and cursor at column and row

vd -P open-plugins
preplay longname before starting the session

FILES

At the start of every session, VisiData looks for $HOME/.visidatarc, and calls Python exec() on its contents if it exists. For example:

   options.min_memory_mb=100  # stop processing without 100MB free

   bindkey('0', 'go-leftmost')   # alias '0' to go to first column, like vim

   def median(values):
       L = sorted(values)
       return L[len(L)//2]

   aggregator('median', median)

Functions defined in .visidatarc are available in python expressions (e.g. in derived columns).

SUPPORTED SOURCES

Core VisiData includes these sources:

(tab-separated value)
Plain and simple. VisiData writes tsv format by default. See the option.

(comma-separated value)
.csv files are a scourge upon the earth, and still regrettably common.
See the --csv-dialect, --csv-delimiter, --csv-quotechar, and --csv-skipinitialspace options.
Accepted dialects are excel-tab, unix, and excel.

(fixed width text)
Columns are autodetected from the first 1000 rows (adjustable with --fixed-rows).

(single object) and jsonl/ndjson/ldjson (one object per line).
Cells containing lists (e.g. [3]) or dicts ({3}) can be expanded into new columns with and unexpanded with ).

May include multiple tables. The initial sheet is the table directory;
loads the entire table into memory. z^S saves modifications to source.

URL schemes are also supported:

(requires ); can be used as transport for with another filetype

For a list of all remaining formats supported by VisiData, see https://visidata.org/formats.

In addition, , , , and files are decompressed on the fly.

SUPPORTED OUTPUT FORMATS

These are the supported savers:

(tab-separated value)
(comma-separated value)
(one object with all rows)
/ndjson/ldjson (one object per line/row)
All expanded subcolumns must be closed (with )) to retain the same structure.
(save to source with z^S)
(markdown table)

AUTHOR

VisiData was made by
Saul Pwanson <vd@saul.pw>.

February 7, 2021 Linux/MacOS