csv and delimited

Comma-separated and other delimited text: the csv format and delimited format, from the csv package. Declare the package once in dependencies.yml; DRE installs it on demand:

dependencies.yml

plugins:
  - csv

csv format: options

Set in a report’s output: (or format_options.csv in dre_project.yml).

Option	Type	Default	Notes
delimiter	char	,	Separates fields.
quote	char	"	Wraps quoted fields; doubled inside them.
quoting	string	minimal	Which fields are quoted: only where needed, all, text columns, or none. One of minimal, all, strings, none.
header	boolean	true	Write the column names first.
line_ending	string	"\r\n"	Ends each line. One of "\n", "\r\n".
encoding	string	utf-8	Text encoding, e.g. utf-8, latin1, windows-1252.
null	string	""	Written for a null value.
byte_order_mark	boolean	false	Start a UTF-8 file with a byte order mark.

delimited format: options

Set in a report’s output: (or format_options.delimited in dre_project.yml).

Option	Type	Default	Notes
delimiter	char	,	Separates fields.
quote	char	"	Wraps quoted fields; doubled inside them.
quoting	string	minimal	Which fields are quoted: only where needed, all, text columns, or none. One of minimal, all, strings, none.
header	boolean	true	Write the column names first.
line_ending	string	"\r\n"	Ends each line. One of "\n", "\r\n".
encoding	string	utf-8	Text encoding, e.g. utf-8, latin1, windows-1252.
null	string	""	Written for a null value.
byte_order_mark	boolean	false	Start a UTF-8 file with a byte order mark.

Writing

• quoting (csv, delimited) picks which fields are wrapped in quote:

  • minimal (the default): only fields holding the delimiter, the quote or a line break;
  • all: every field but nulls;
  • strings: every value of a text, date, time or timestamp column, and the header; numbers, booleans and nulls stay bare unless they hold the delimiter;
  • none: no field. A value that can’t be written without quotes fails the run, naming the row and column.

  A doubled quote escapes a quote inside a quoted field. For tab- or pipe-separated text, use delimited with delimiter: "\t" and, say, extension: tsv.

• null: "NULL" (csv, delimited) writes that marker for nulls instead of an empty field. It can be written unquoted as above: DRE reads a YAML null: key as the option null.

Export for a downstream importer

Use CSV for a standard table export, or delimited for a receiver requiring tabs, pipes or a different extension. Starting with the tutorial project, replace its report output with:

output:
  format: delimited
  extension: tsv
  delimiter: "\t"
  header: true
  quoting: minimal
  quote: '"'
  null: "NULL"
  encoding: utf-8
  line_ending: "\n"
  byte_order_mark: false

Declare csv in dependencies for either format. dre run <report> --preview 5 writes a sample without delivery. Check the importer with a null, an embedded delimiter and a quoted value, not only plain numbers. Use byte_order_mark: true when a receiver needs a UTF-8 BOM; use encoding: windows-1252 only when every required character fits that encoding.

With quoting: none, the actual error is row <row>, column <col>: the value contains the delimiter, quote or a line break, which quoting: none can't represent (the message quotes the column and option with backticks). Restore minimal rather than removing characters silently. An unsupported option yields invalid-output-option; move a destination option out of the output map if it belongs to delivery. See output keys, error codes, and format and output.