# Spec — Markdown documents

**Status:** current
**Phase:** 14
**Source:** https://github.com/datagripe/datagripe/blob/main/docs/spec/markdown-documents.md

> An engineering document, not user documentation. It records what
> a subsystem does and why the alternatives were rejected, and some
> of it describes things that are not built. For using DataGripe,
> read [the documentation](/docs.md) instead.

**Supersedes:** the "the language stays `sql`" paragraph in
[`docs/spec/datasource-paths.md`](/specs/datasource-paths.md) "The file as a document" (extends
[`docs/spec/editor-workspace.md`](/specs/editor-workspace.md), [`docs/spec/query-execution.md`](/specs/query-execution.md),
[`docs/spec/gripes.md`](/specs/gripes.md))

## Goal

[`docs/spec/datasource-paths.md`](/specs/datasource-paths.md) shipped with a wart it named out loud: a
`.md` opened out of a datasource path gets SQL highlighting, "and the fix
is a language field on the document rather than anything here". This is
that fix, and the reason it is worth a spec rather than a one-line enum
change is what a `.md` file in a database repository actually contains.

It contains a runbook. `maintenance.md` explains what the nightly job
does, and then — in a fenced `sql` block — the query you run when it
fails. Today that query is read in one window and retyped in another,
which is how it drifts from the prose that explains it. DataGripe can
run it where it is written.

So: markdown is the second document language, decided by extension, in
every place DataGripe shows files. It **opens rendered**, because a
runbook is something you read. Its SQL blocks are executable in place,
against the document's connection, through exactly the execution path a
selection goes through. And a button in the bottom right flips to the
editor, where the fences get the SQL editor's completion, formatting and
gripes, and flips back.

## Non-goals

- **Not a markdown authoring tool.** No preview-beside-editor, no
  toolbar, no WYSIWYG, no paste-image. It is two modes and one button.
- **Not a general document store.** Markdown arrives because a file is
  called `.md`. There is no "new markdown document" verb beyond naming
  one — see "Where a language comes from".
- **Not HTML.** Raw HTML in markdown is escaped and rendered as text,
  never passed through. A file from a checkout is content somebody else
  may have written.
- **Not a syntax-highlighting project.** JSON, YAML, TOML and the rest
  are `plaintext`: a name for "no formatting", not a feature. Markdown
  earns a language because it has runnable content; a highlighter per
  extension has no end. (This was "they stay `sql`", which meant a
  `.csv` opened out of a datasource path was a query with a thousand
  findings in it.)
- **Not a notebook.** Blocks do not share state, do not run in order,
  have no persisted outputs, and produce no artifact. Running one is
  running a query.

## Design

### Where a language comes from

One rule, everywhere: **the extension of the name decides.**

| Name | Language |
| --- | --- |
| `maintenance.md`, `README.markdown` | `markdown` |
| `blah.sql`, `query22.sql` | `sql` |
| `notes`, `Makefile`, `.env` | `sql` |
| `analysis.json`, `rows.csv`, `notes.txt` | `plaintext` |

A name with no extension is `sql` because a document nobody has named
yet is a query — `query-1.sql` without the suffix is still the thing it
was. A name *with* an extension that is neither is plain text, which is
the honest answer: DataGripe opens the file and does not pretend to
understand it.

The rule applies to file-backed documents (`origin.filePath`), to shared
workspace files and to local scratchpads, because all three are named by
a person and all three show up in a files area. Renaming a document from
`notes.sql` to `notes.md` changes its language, which is the behaviour
somebody renaming it is asking for.

`documentSchema.language` is `z.enum(["sql", "markdown", "plaintext"])`;
migration 0015 added `documents.language text NOT NULL DEFAULT 'sql'`
with a check constraint and 0026 widened it. It is stored rather than
derived on read because the client needs it before it has the content —
the language is recomputed on save, with the title, and broadcast like
any other document change.

**A rename retunes the open model rather than rebuilding it.** Monaco
cannot change a model's URI, and rebuilding the model to get a new one
would throw away the undo history the person just typed, so
`modelRegistry.setLanguage` calls `setModelLanguage` on the live model
and the URI keeps the extension it was born with. Everything a reader
sees — the highlighting, the markdown pane, the dialect the gripes
engine uses — reads the store's `language`, so the change is immediate
and a reload is not part of it.

The default for `document.create` stays `sql`: a new scratchpad is a
scratchpad. Name it `notes.md` and it is markdown on the next save.

### Two modes

A markdown document's editor panel has a mode, `view` or `edit`, and
opens in `view`.

```
┌ maintenance.md ─────────────────────────────────────────┐
│                                                         │
│  Nightly reconciliation                                 │
│  ─────────────────────                                  │
│  When the job reports a gap, find the orphans:          │
│                                                         │
│  ┌───────────────────────────────────── sql ──── ▶ ──┐  │
│  │ select * from ledger.entries e                    │  │
│  │  where not exists (…)                             │  │
│  └───────────────────────────────────────────────────┘  │
│                                                         │
│  Then re-run the job with `--from`.                     │
│                                                         │
│                                            [ edit ]     │
└─────────────────────────────────────────────────────────┘
```

The button sits in the bottom right of the pane and says `edit`; in edit
mode it stays in the same place and says `view`. One control, one
position, two labels — a control that moves when you press it costs
people the second press.

Mode is per **view**, not per document. Splitting a markdown document to
read it beside its own source is a thing people do, and it is free if
mode is a view property. It rides on the Dockview panel's parameters —
which are already serialised with the layout — rather than a store of
its own, so a tab you left in edit mode reopens in edit mode with no new
persistence to keep in sync.

A `sql` document has no button and no mode. Nothing in the pane moves
for the language that does not need it.

### Rendering

`marked`, configured with GitHub-flavoured markdown, no raw HTML, and no
external resource loading:

- **HTML is escaped, not sanitized.** A `<script>` in the file renders
  as the text `<script>`. Escaping is a property of the parser
  configuration; sanitizing is a filter somebody has to keep ahead of an
  attacker. The first one has no bypass to find.
- **Links** render, open in a new tab, and carry
  `rel="noreferrer noopener"`. A link to a relative path in the
  repository resolves against the document's origin directory and opens
  that file in DataGripe when it is a file the editor can open — this is
  the case that makes a `docs/` folder in a checkout worth reading here
  at all.
- **Images** are not fetched. A remote image would be a beacon and a
  relative one is a file the server would have to serve; both render as
  the link text with the source beside it.
- **Tables, task lists, footnotes and headings** render. Headings get no
  anchor links: there is no URL to link to.
- Rendering is memoised on document content, because a markdown document
  under a debounced gripe runner re-renders on every keystroke otherwise.

The rendered pane uses the same type scale and colour tokens as the rest
of the app (`apps/web/src/styles/tokens.css`). It is not a document
theme; it is DataGripe with markdown in it.

### SQL blocks

Fenced blocks whose info string is `sql` (case-insensitive, and
`postgresql`/`mysql`/`sqlite` alias to it) are extracted before render
and replaced by a block component. Every other fence renders as a plain
code block, monospaced and unstyled — highlighting eleven languages is a
different project.

A SQL block is:

- **A read-only Monaco instance**, not a `<pre>`. It costs a model and
  buys the exact tokenizer, theme and font the editor uses, which is
  what makes the block read as *the same SQL* as the tab next door.
  Blocks are created lazily as they scroll into view and disposed when
  the panel unmounts, because a 40-block runbook should not be 40
  editors on open.
- **Runnable.** The `▶` in the block's top right runs its contents
  through `execution.start` — the same action, the same registry, the
  same limits, the same results panel bound to this document, the same
  cross-user audit trail ([`docs/spec/query-execution.md`](/specs/query-execution.md),
  [`docs/spec/multiplayer.md`](/specs/multiplayer.md) 6d/6e). A block is a selection that happens
  to be delimited by backticks.
- **Bound to the document's connection**, resolved by
  `connectionIdForDocument` exactly as the editor resolves it, falling
  back to the workspace default. A markdown document with no connection
  shows the run button disabled with the reason, in the same words the
  editor uses.
- **Analysed.** The gripes runner ([`docs/spec/gripes.md`](/specs/gripes.md)) sees the
  document with everything that is not a `sql` fence **blanked out,
  character for character** — same length, same line breaks. So a
  `delete` with no `where` in a runbook is a blocker in the gripes panel
  and in the status bar count, at the offset it actually occupies.
  Blanking beats extracting: an extract needs an offset map, and an
  offset map is a thing that can be wrong.

Multiple blocks do not share a session. Running the third block does not
run the first two, and there is no "run all" — a runbook is a set of
things you might do, not a script.

### Editing

Edit mode is the editor the app already has, with Monaco's `markdown`
language on the model, plus one thing: **the SQL features work inside
`sql` fences.**

Monaco does not do embedded languages, so this is done at the provider
seam rather than the grammar seam. Completion, formatting and hover are
registered for `markdown` as thin wrappers that:

1. find the fence containing the position, by scanning fence delimiters
   from the top of the model;
2. return nothing at all when the position is not inside a `sql` fence —
   markdown prose must not get table-name completion;
3. otherwise delegate to the existing SQL provider with the position and
   offsets translated into fence-local coordinates, and translate the
   results back.

That gives, inside a fence and nowhere else: schema-aware completion
([`docs/spec/editor-workspace.md`](/specs/editor-workspace.md)), Ctrl+Alt+L formatting scoped to the
fence rather than the document, and the gripe squiggles and gutter
glyphs already described above. Formatting the whole document when the
caret is in prose does nothing rather than running the SQL formatter
over a paragraph.

Save, revision guard, presence, followed cursors and the disk-conflict
banner are unchanged — a markdown document is a document
([`docs/spec/multiplayer.md`](/specs/multiplayer.md), [`docs/spec/datasource-paths.md`](/specs/datasource-paths.md)).

Followed cursors are an edit-mode thing. A follower in view mode simply
does not see the leader's caret — see "What is not built".

### The sidebar and the tabs

- File trees show `.md` files with their own glyph, so a runbook is
  visibly not a query.
- The tab title is the filename, as it already is. The editor tab's
  dirty dot, the gripe severity marks and middle-click-to-close all work
  unchanged.
- Executing from a markdown document records history rows exactly as the
  editor does, so `history.list` shows the query with the document it
  came from.

## Testing

- **Extension rule.** `a.md`, `a.MD`, `a.markdown` are markdown;
  `a.sql`, `a`, `a.md.sql`, `a.json` are sql. Renaming `notes.sql` to
  `notes.md` changes the language on save and broadcasts it.
- **No HTML.** A file containing `<script>alert(1)</script>`,
  `<img onerror=…>` and an `<iframe>` renders three pieces of visible
  text and produces no element of those types in the pane.
- **Link handling.** An `http` link gets `rel="noreferrer noopener"`; a
  relative link to a sibling `.sql` opens that file; a relative link
  that climbs out of the datasource path is inert.
- **Block extraction.** Fences with info `sql`, `SQL`, `postgresql` and
  ` sql ` become blocks; ```` ```bash ```` and an unfenced indented block
  do not. An unterminated fence at end of file renders as a block and
  does not eat the rest of the document.
- **Execution parity.** Running a block and running the same text as a
  selection in a `.sql` document produce the same request payload,
  including the connection ref and the limits.
- **Offsets.** A gripe in the second of three blocks reports a document
  offset that lands inside that block's text, asserted against the raw
  file content and not against the rendered output.
- **Provider scoping.** Completion at a position in prose returns
  nothing; at a position inside a `sql` fence it returns the same
  suggestions the SQL provider returns for the fence-local position.
  Ctrl+Alt+L with the caret in a fence rewrites only that fence's bytes.
- **Mode persistence.** A view left in edit mode reopens in edit mode
  after reload; a second view of the same document keeps its own mode.
- **Disposal.** Opening and closing a 40-block document twice leaves no
  Monaco models behind.

## What is not built

- **Preview beside editor.** Two modes, one pane. Splitting the tab
  gives you both, and that already works.
- **Highlighting for non-SQL fences.**
- **Front matter.** A leading `---` block renders as a horizontal rule
  and a paragraph, like anywhere else that has not special-cased it.
- **Mermaid, math, admonitions.**
- **Creating a markdown file from the sidebar.** The path tree is not a
  file manager ([`docs/spec/datasource-paths.md`](/specs/datasource-paths.md) "Non-goals"), so a new
  `.md` in a checkout is made by the tool that owns the checkout. A
  workspace file or scratchpad named `.md` is the way in for now.
- **Followed cursors in view mode.** A leader's caret is a position in
  the source, and the rendered pane has no such position to put it in.
  Showing "the block they are in" was specified and then dropped: it is
  a second, weaker notion of following that would have to be explained
  every time somebody noticed the difference. Switching to edit gives a
  follower the real thing.
- **A dirty dot in the rendered pane.** The mode button is the only
  chrome the pane has; the tab already carries the dot.

## Open questions

- Whether a block should be editable in view mode. It is the obvious
  next request and it is a trap: an edit in a read pane has to write
  back into the source at an offset, and the pane has no dirty state to
  hang it on. Deferred until somebody actually asks.
- Whether run results belong under the block rather than in the results
  panel. Under the block reads better and reflows the document under a
  running query; the results panel is where every other execution goes.
- Whether [`docs/spec/gripes.md`](/specs/gripes.md) should get a markdown-specific rule
  class — "this runbook's query references an object that no longer
  exists" is the most valuable gripe in the catalogue and needs a
  catalog the runner does not have at analysis time.
- Whether the language should be a per-document override rather than
  strictly derived from the extension, for the `README` with no
  extension. Cheap to add, and one more piece of state that can disagree
  with the filename.