PDF and HTML documents

PDF and HTML documents from an HTML + CSS template: the html format and pdf format, from the document package. Declare the package once in dependencies.yml; DRE installs it on demand:

dependencies.yml

plugins:
  - document

html format: options

Set in a report’s output: (or format_options.html in dre_project.yml).

Option	Type	Default	Notes
max_rows	integer	50000	The most rows across the output’s queries; more fails the run.
embed_assets	boolean	true	Inline images, stylesheets and fonts as data URIs, so the file works wherever it’s delivered.

pdf format: options

Set in a report’s output: (or format_options.pdf in dre_project.yml).

Option	Type	Default	Notes
max_rows	integer	50000	The most rows across the output’s queries; more fails the run.
page_size	string	A4	Paper size, used when the template’s CSS sets no @page size. One of A3, A4, A5, Letter, Legal.
orientation	string	portrait	Used when the template’s CSS sets no @page size. One of portrait, landscape.
margins	string	20mm	Page margins as in CSS, like 20mm or 15mm 20mm; used when the CSS sets no @page margin.
author	string		The PDF’s author metadata.
subject	string		The PDF’s subject metadata.
password	string		An open password: the PDF is encrypted with AES-256 and opens only with it; Jinja like {{ each.pdf_password }}
missing_fonts	string	warn	When a script’s font can’t be fetched (offline, nothing cached): warn makes the PDF with boxes for its characters and warns once per script per run; error fails the output (each row, per row) instead. One of warn, error.

A client statement, a board pack or a compliance summary: one HTML + CSS template, filled with the report’s query results, written as an .html file or laid out into a .pdf. The PDF is made inside the plugin: no browser, no Playwright, no conversion service and no network call. New in DRE 0.6. The package needs DRE 0.6.0 or later.

A statement

reports/clients/statement/statement.yml

queries: [summary, holdings]
vars:
  client: Acme Pty Ltd
sets:
  - {name: acme}
  - {name: beta, vars: {client: Beta Ltd}}
output:
  format: pdf
  template:
    file: statement.html          # found next to this YAML (see "Finding a template")
  author: Acme Funds

reports/clients/statement/statement.html

{% extends "base.html" %}
{% block title %}Statement for {{ var('client') }}{% endblock %}
{% block body %}
<h1>{{ var('client') }}</h1>
<p>Balance on {{ run.date | date('d mmmm yyyy') }}:
   <strong>{{ results.summary.first.balance | currency('AUD', 2) }}</strong></p>
{{ table(results.holdings) }}
{% include "terms.html" %}
{% endblock %}

<!-- reports/clients/base.html: shared by every report in the folder -->
<!doctype html>
<html lang="en-AU">
<head>
  <title>{% block title %}{% endblock %}</title>
  <link rel="stylesheet" href="assets/brand.css">
  <style>
    @page { size: A4; margin: 20mm 15mm;
            @bottom-center { content: "Page " counter(page) " of " counter(pages); } }
  </style>
</head>
<body>
  <img src="assets/logo.svg" alt="Acme">
  {% block body %}{% endblock %}
</body>
</html>

dre run statement --set all writes statement.pdf for each Set. Change format: pdf to format: html for a web page from the same template. With no template:, the output is a plain document with a table per query.

What a template reads

Everything a message template reads, the same way:

results.<query>	each query of the output, in YAML order: value, first.<column>, rows, row_count, columns, column_types (each column’s type, {amount: Decimal128(18, 2)}), sets[n]
var('name')	every resolved var: --var, the schedule’s, the Set’s, the report’s, its folders’ and the project’s, whether or not a query uses it
run.*, target.name	the report, Set, date, timezone and the rest of run
locale	the run’s locale: tag, for <html lang>

Values keep their types: numbers stay numbers, dates are DRE dates (shown in the run’s timezone), nulls are none. Unlike messages, rows holds every row: a document is never cut short (see max_rows). env_var() isn’t available: a document is data that’s delivered.

Filters and functions:

• The number filters: number, percent, signed, currency and compact, in the run’s locale:, as in messages.
• date(format): a date or timestamp in an Excel-style format (dd/mm/yyyy, d mmm yyyy, yyyy-mm-dd hh:mm); default yyyy-mm-dd.
• row.fmt('amount') (on a row of rows, or first) shows a value in its column’s YAML format, the xlsx format codes ($#,##0.00, d mmm yyyy); a column with no format is shown as table() shows it. row.amount stays the raw value, for comparisons and arithmetic.
• table(results.<query>) writes a styled <table class="dre-table"> of the query’s rows, each value shown with its column’s YAML format (the xlsx format codes, such as $#,##0.00 or d mmm yyyy). table(results.q, ['fund', 'amount']) picks and orders the columns. Style it with CSS on .dre-table.
• The date functions (date(), period(), …) of every template.

Values are HTML-escaped, so a client called A&B <Ltd> shows as written; | safe inserts deliberate markup. A value that isn’t there (results.holdings.first.amout) fails the run, naming the template line and the queries and columns there are, so a statement never goes out with a silent blank. Project macros (macros/) aren’t available here: share layout with {% include %} and {% extends %}.

Finding a template

template.file, {% include %}, {% extends %} and {% import %} find a file the same way:

• A name with / is a path from the project root, taken exactly: templates/statement.html.
• A bare file name is looked for next to the report’s YAML (for an include: next to the including template), then in each parent folder up to the project root; the nearest wins.
• Failing that, anywhere in the project, where it must be unique. Two matches are an error that lists both; name one by its path.

Never outside the project, and never a URL. The same rule finds an xlsx template. A Set can override template.file like any other output key, so one client can get a different layout.

Images, stylesheets and fonts

<img src>, <link rel="stylesheet" href> and CSS url() are relative to the template that writes them (a shared base.html links its logo next to itself), and must be inside the project. http(s):// is refused: rendering is offline and gives the same document every time. PNG, JPEG, GIF and SVG images work, and so does inline <svg>, so a logo or a simple chart made by a macro appears in the PDF.

An html output is one self-contained file: its images, stylesheets and fonts are inlined as data URIs, so it still shows its logo after email, S3 or SFTP delivery. embed_assets: false keeps them as links, for a file published next to its assets.

The PDF uses only fonts it’s given, never the machine’s, so it looks the same everywhere. The plugin bundles IBM Plex Sans (regular, bold, italic) and IBM Plex Mono, under the SIL Open Font License. Add your own with @font-face, from a .ttf or .otf file in the project:

@font-face { font-family: Brand; src: url(fonts/Brand-Regular.ttf); }
@font-face { font-family: Brand; src: url(fonts/Brand-Bold.ttf); font-weight: bold; }
body { font-family: Brand, sans-serif; }

A font-family that’s neither bundled nor declared falls back to IBM Plex Sans, with a warning naming it. Fonts are subset: the PDF holds only the characters it uses.

Fonts for other languages

From document 1.3.0, PDF text in Japanese, Simplified and Traditional Chinese, Korean, Arabic, Hebrew, Devanagari (Hindi, Marathi, Nepali) and Thai needs no project fonts when used with a core that supports font requests. Before laying out the PDF, the plugin checks its characters against Plex and the project’s @font-face fonts. For each script they leave uncovered, core fetches that script’s Noto Sans regular and bold, and only those: a Hindi report downloads Devanagari and nothing else. Arabic and Hebrew run right to left, Arabic letters join, and Devanagari forms its conjuncts (नमस्ते).

Project fonts stay first when explicitly selected. Other declared project families are available as fallbacks after Plex, including outside elements that select them; Noto is the final fallback. HTML behavior stays the same: the reader’s browser supplies missing fonts, and DRE never downloads Noto for HTML.

Han characters follow lang

Chinese, Japanese and Korean share the Han characters (直, 東京) but draw many of them differently. As in a browser, the HTML lang attribute of an element, or of its nearest ancestor that has one, picks the font: ja Japanese, ko Korean, zh Simplified Chinese, and zh-Hant, zh-TW, zh-HK or zh-MO Traditional Chinese. Set it once for the whole document, or per element:

<html lang="ja">
  <p>東京本社</p>
  <td lang="zh-TW">臺北</td>
</html>

In a per-row report, set it from the row, with no new YAML key:

<td lang="{{ each.lang }}">{{ each.city }}</td>

Text with no Chinese, Japanese or Korean lang (none, or another language such as en) is guessed from the document: kana (です) means Japanese, Hangul (한국어) means Korean, and Han characters alone mean Simplified Chinese. That last guess adds a one-line note suggesting lang. CSS generated content (content:) follows the lang of <html>.

Pinned versions and the cache

The registry pins each package with a SHA-256 checksum for each weight and its SIL Open Font License 1.1 text:

Script	Package	Version	Upstream
Japanese	noto_sans_jp	2.4.0	Noto Sans CJK JP 2.004
Simplified Chinese	noto_sans_sc	2.4.0	Noto Sans CJK SC 2.004
Traditional Chinese	noto_sans_tc	2.4.0	Noto Sans CJK TC 2.004
Korean	noto_sans_kr	2.4.0	Noto Sans CJK KR 2.004
Arabic	noto_sans_arabic	2.13.0	Noto Sans Arabic 2.013
Hebrew	noto_sans_hebrew	3.1.0	Noto Sans Hebrew 3.001
Devanagari	noto_sans_devanagari	2.7.0	Noto Sans Devanagari 2.007
Thai	noto_sans_thai	2.2.0	Noto Sans Thai 2.002

Font files are not bundled in the plugin or added to the project. Core keeps verified files and the licence in ~/.dre/fonts/<package>/<version>/; a second run uses this cache without contacting the registry. DRE_REGISTRY_URL mirrors cover fonts as well as plugins. Older cores keep their previous PDF behavior, and older document plugins keep working with newer cores.

Offline servers

A run fetches each font once, and only when the cache doesn’t have it. On a machine that can’t reach the registry, any of these makes the fonts available:

1. An offline copy of the registry. Copy packages.json and its font files, point their urls at the copies, and set DRE_REGISTRY_URL to the index (a plain path, a file:// URL or an internal https:// mirror), as for plugins (registry).
2. dre fonts install on a connected machine. It downloads and verifies fonts into ~/.dre/fonts ahead of time, by script or language: dre fonts install ja zh-Hans ar, or --all. Copy ~/.dre/fonts to the offline machine, or run it in the Docker image build. dre fonts ls lists what’s cached. It installs each package’s newest version in the registry, which the current document plugin pins (the table above); an older plugin may pin an older one.
3. The project’s own fonts, declared with @font-face (above). They always come first, and a script they cover never needs a download.

RUN dre fonts install ja ko ar

Otherwise the PDF is still made, with boxes for the characters of each script DRE has no font for, and the run warns once per script, however many PDFs or rows need it:

   Warning    row `1`: pdf format: no Japanese font for this PDF, so its Japanese text shows as boxes (can't read /srv/registry/packages.json: No such file or directory (os error 2)). To fix it, set `DRE_REGISTRY_URL` to an offline copy of the registry, run `dre fonts install ja` on a connected machine and copy its `~/.dre/fonts` here, or add a font covering Japanese to the project with `@font-face`

A run tries the registry once per font: after a failure, its other PDFs don’t wait for it again.

Failing instead: missing_fonts

For PDFs that must never show boxes, missing_fonts: error fails the output instead of making the PDF (each row’s, for a per-row output); the error names every script without a font and the same fixes. The default is warn. dre validate rejects other values, and the option is for format: pdf only: HTML uses the reader’s fonts.

output:
  format: pdf
  template: {file: statement.html}
  missing_fonts: error

Text inside an SVG is the exception: it’s drawn with the machine’s fonts. For a logo that looks the same everywhere, convert its text to outlines (paths) in your design tool, or write the name in the html next to an icon-only SVG.

Page setup

Set the page in CSS, as for print: @page { size: A4 landscape; margin: 15mm }, page-margin boxes for running headers and footers (@top-right, @bottom-center with counter(page) and counter(pages)), and break-before: page, break-after, break-inside: avoid. A table that runs over several pages repeats its <thead> on each. The page_size, orientation and margins options apply only where the CSS sets nothing.

The PDF’s title comes from <title>, its language from <html lang>, author and subject from the options, and its creation date is the run’s run.now, so the same data makes the same file.

How big a document can be

max_rows (default 50,000) caps the rows across every query of one document. More fails the run with a hint, before anything is laid out: aggregate in SQL, filter, or raise it. A message instead keeps its first max_rows rows; a document fails, because it must never be cut short. A single table of more than about 30,000 rows is more than the layout engine handles: split it, or aggregate. Laying out 5,000 table rows takes about a second.

Checks before a run

dre validate (and dre run, before any query) parses the template and every file it includes by a literal name, checks its images and stylesheets exist, its filters exist, and that each results.<name> is a query of the output, with a “did you mean” hint. An include whose name is computed ({% include "terms/" ~ var('client') ~ ".html" %}) gets a warning that it isn’t checked until the run.

Designing a template

A template takes many tweaks. Save the results once, then re-render from them, with no query and no connection, in about a second:

dre run statement --set acme --no-deliver --output-json design   # runs the queries once
dre run statement --set acme --from-results design               # each tweak: re-render only

--from-results reads the results --output-json saved (design/<report>/<binding>/, see the manifest page), renders every output of the report (xlsx and messages too), and implies --no-deliver. It fails when a query has no saved result, and warns when a query’s SQL changed since they were saved.

What the PDF engine draws differently

The PDF is laid out by fulgur, not a browser, so check a design in the PDF itself. One difference: a box’s borders share one colour, its top border’s. For a coloured line on one side only, set the colour for every side and the width for that side:

header { border: 0 solid #e8873a; border-bottom-width: 2pt; }

Password-protected PDFs

password: on a pdf output encrypts the file after it’s laid out: it opens only with that password. AES-256 (PDF 2.0), with a random owner password nobody keeps, so the encryption can’t be lifted with a blank owner password. New in document 1.1.0.

output:
  - name: statement
    format: pdf
    template: {file: statement.html}
    for_each: {query: members, key: member_id}
    password: "{{ each.pdf_password }}"        # or "{{ env_var('STATEMENT_PW') }}"

• The password is Jinja like other output values: each.* for one per row, env_var(), var(). A templated password needs DRE 0.7.0 or later; a literal one works on 0.6.
• An empty password is an error, never an unencrypted file. html takes no password: zip it with a password on the destination instead (zip packs).
• Keep passwords to ASCII letters, digits and punctuation: macOS Preview and Safari can’t open a PDF whose password has other characters (Acrobat, Chrome, Edge and Firefox can). The run warns.
• Mark the password’s column in secret_columns: so it never shows in DRE’s logs, results or previews, and send the password by another route than the file.
• Only an open password: no print, copy or edit restrictions (readers honour those voluntarily).

Delivery

File destinations send a .pdf as application/pdf and an .html as text/html; an email attachment opens in the recipient’s mail client. To email the document, attach the output to a message with attach:.