# Texts, interlinear glossing, and grammar

Connected material rather than isolated words: stories and passages with ordered
sentences, interlinear glossed text (IGT), and the dictionary's structured grammar
description. Read `api-basics` first if you haven't.

## Texts vs. example sentences

Two different objects, easy to confuse:

- An **example sentence** belongs to a sense — it illustrates a word. Written
  nested inside the entry (`senses[].example_sentences`) or standalone via
  `POST …/sentences` and linked by id.
- A **text** is a connected passage — a story, a narrative, a procedural
  description — with its own ORDERED list of sentences (each able to start a new
  paragraph). Independent of entries. Use the `…/texts` routes.

A sentence can be both: a text's sentence linked to a sense also serves as that
sense's example.

```
POST …/texts                    create the text with its ordered sentences
GET  …/texts                    list (filter with ?tag=<name> and/or ?collection=<id>)
GET  …/texts/{textId}           the text + sentences + audio + speakers in ONE call
POST …/texts/{textId}/tags      genre / motif / tale-type classification
```

## Collections (organizing a large corpus)

A dictionary with hundreds or thousands of texts (a story anthology, a hymnal,
multi-chapter works, session transcripts) needs structure, not one flat list.
**Collections** are self-nesting groups — series › book › chapter, up to 3
levels — and every text can belong to one:

```
GET    …/texts/collections                  the whole forest (nest via parent_id)
POST   …/texts/collections                  create ({ title, parent_id?, sort_key? })
PATCH  …/texts/collections/{collectionId}   rename / re-parent / reorder
DELETE …/texts/collections/{collectionId}   children + member texts DETACH (never deleted)
```

Attach a text by setting `collection_id` on text create or PATCH; order within
the collection with `sort_key` (a fractional index — an opaque string that
sorts lexicographically; omit it to append, or read the siblings and pick any
string between two neighbors to insert between them). A parent assignment that
would make a collection its own ancestor (or nest past 3 levels) is rejected
with `422`. Tags and collections are complementary: collections hold the
corpus's SHAPE (what contains what, in order), tags classify across it
(genre/motif in any collection).

When importing a structured corpus, mirror its own organization: one collection
per source volume/section, texts ordered as the source orders them.

A collection's `description` is a **preface**: markdown of any length, shown
above its texts (folded after a screenful). Use it to introduce what the
collection is and how to read it.

A text can also be an **article**: send `body` (markdown per language) and no
`sentences`. The reader shows the body as prose where a sentence list would be,
in its collection next to the other texts; it can carry its own recording like
any text. Use it for introductions, essays and lessons that stand as texts of
their own. Everything below about `[[word]]` applies, and `[@source-slug, p. 12]`
cites a page of a registered source (it links to that page when the source's
PDF is open to readers, and is listed under the text's sources).

## Vernacular words inside prose (`[[word]]`)

Descriptions, prefaces, text summaries, grammar sections, titles, section
headings and translations often mention vernacular words. Write each one as an
**entry link** instead of bare text, so readers can tap it for its card and so
it follows the reader's script switch (in another orthography it shows the
entry's lexeme there):

```
[[word]]                    the entry whose lexeme (any orthography) is "word"
[[word|Word]]               shown as "Word" in the orthography it was written in
[[word#2]]                  the entry with homograph number 2
[[word#<entry-id>]]         exactly that entry
[[a phrase|alt=its form]]   no entry: give its spelling per orthography code
[[s:<sentence-id>]]         alone in a paragraph: quote a sentence from a text
                            (its words, translation, a play button for its span
                            of the recording, and a link to its place)
```

Matching ignores case and treats a hyphen and a space as the same joiner. A
running vernacular line that is in no text goes in an `example` fence; its words
link like `[[word]]` (use `[[…]]` inside it for multi-word or pinned words),
lines labelled with another orthography code are used verbatim when their word
count matches, and lines labelled with a language code are translations:

````
```example
default: The first line, words linked one by one.
alt: The same line in another orthography.
en: A translation.
```
````

Every write that carries `[[…]]` answers with `warnings` for refs that match
several entries (pin them), refs with no entry, and quotes of missing sentences.
The write still succeeds: an unresolved word shows as written, marked, and links
as soon as its entry exists. Bare vernacular in prose is never auto-converted,
so link it explicitly.

## Interlinear glossed text (IGT)

Supply gold `tokens` on any sentence write shape (`SentenceInput`,
`TextSentenceInput`, `SentencePatch`). It's a map of **orthography code → ordered
token list**, usually just `default` (the vernacular line):

```json
{
  "text": "Mbwa wangu anakula",
  "translation": { "en": "My dog is eating" },
  "tokens": {
    "default": [
      { "form": "Mbwa",    "gloss": { "en": "dog" } },
      { "form": "wangu",   "gloss": { "en": "my", "default": "1SG.POSS" } },
      { "form": "anakula", "gloss": { "en": "eat", "default": "3SG.PRS" } }
    ]
  }
}
```

That token list is the **shared index** everything else aligns to 1:1 — the gloss
line, word→entry links, and karaoke audio timings. Get it right and the rest follows.

Rules worth internalizing:

- **`form` must be byte-identical to its appearance in `text`.** `start`/`end`
  offsets are optional on write; the server derives them by walking your ordered
  forms against the text with a left-to-right cursor. Strip a tone mark or footnote
  marker from the form but not the text (or vice-versa) and derivation fails.
- **The `gloss` convention is locked**: language-neutral grammatical category codes
  (`3PL`, `PFV`, `CLF`) go under the reserved `default` key; per-language lexical
  glosses (`tiger`, `虎`) under their language codes. A reader on gloss-language X
  sees `gloss[X] ?? gloss.default`, so a neutral code under `default` survives every
  gloss-language switch — putting it under `en` would make it vanish for a Chinese
  reader.
- **A token needs no entry.** `entry_id` is optional and independent of `gloss` —
  grammatical morphemes and portmanteaux often have no headword but must still be
  glossable.
- **Multi-word → one gloss** = one token spanning the whole char range.
- **Ambiguous match? Send `candidates`, not nothing.** When a form matches several
  entries (homographs — endemic wherever the spelling neutralizes a contrast like
  tone, length, or stress, and worst among the highest-frequency words), write the
  shortlist: `{ "form": "…", "candidates": ["<entry_id>", "<entry_id>"] }`. Those
  tokens land in the `ambiguous` facet of the suggestions queue for a human to
  resolve in one pass. Dropping the link instead loses exactly the words readers
  tap most. Leave `status` absent alongside candidates.
- **Never stamp `status: "confirmed"` on a link no human confirmed.** An ABSENT
  status is the meaningful "unmatched" state: it's what keeps a token reachable by
  the auto-matcher and visible in the review queue. Stamp it and the token is
  invisible to both — permanently unlinked and unreviewable. Use `auto` for a
  machine match you want a human to check.
- `status: "ignored"` marks punctuation, keeping the offset and timing arrays
  aligned.
- **Unsure of the word itself?** Set `uncertain: true` on the token (a machine
  transcription the model doubted, a hard-to-hear field recording). It is about the
  form, not the link: independent of `status`/`entry_id`, kept through every matcher
  pass and token action, and toggled alone by the `mark_uncertain` /
  `clear_uncertain` token actions. Readers set the word in its own colour, with a
  one-line legend.
- **A word no entry can spell in another orthography yet?** Give `forms` on the
  token (`{ "<orthography code>": "<form>" }`), e.g. a rule-converted spelling. A
  reader switched to that orthography sees it in the uncertain colour; a linked
  entry's lexeme there always wins. Mark an entry's own converted (unattested)
  lexeme values with `uncertain_orthographies: ["<code>"]` on the entry.
- Legend codes are matched as a substring anywhere in a gloss cell, so portmanteaux
  like `eat PFV` still render small-caps and tap-to-expand. There's no per-token
  "is this grammatical?" flag to set.
- **Omit `tokens` entirely** and the server tokenizes the sentence and word-matches it
  against the dictionary's entries at write time. Supply them only when your source
  has a real glossed alignment worth preserving. Order does not matter: entries
  created or renamed later fill in the still-unmatched words automatically.
  With `status` omitted, a token with a resolved `entry_id`/`sense_id` (or a linked
  morpheme) defaults to `confirmed`; a bare, gloss-only, or candidates-only token
  stays statusless, i.e. unmatched. A gloss is metadata, not a link — the matcher
  links glossed words against the entries and keeps the gloss verbatim.

A rows-only glossed source (aligned `[form, gloss]` rows with no separate vernacular
line) imports fine: omit `text` for that orthography and send its tokens; the server
joins the forms with spaces to build the text.

Also available per sentence: `citations` (a source ref WITH a page/example
`locator`), `example_label` (the author's own example number, e.g. `"(2a)"`), and
`discourse_role` (information/salience role, so a narrative can render foreground vs.
background). `sources.orthography` declares which script a source's forms use.
`section_heading` (MultiString, keyed by translation language and/or orthography)
opens a new section of the text at that sentence: the reader renders it as a heading
before the sentence and lists the headings in an outline when there are two or more.
PATCH it with `null` to remove it.

## Structured grammar

The grammar surface is live and entry-linked:

| | |
|---|---|
| `…/grammar/sections` | Hierarchical sections: parallel-language markdown prose, entry/sense links, usage conditions. A section may be headless (body-only). |
| `…/grammar/sections/{sectionId}/sentences` | Attach example sentences **by reference** — `POST …/sentences` to create, then attach the id. |
| `…/grammar/clause-slots` | Clause-template slots. |
| `…/grammar/glossing-abbreviations` | The legend that makes gloss codes expandable for readers. |
| `…/entries/{entryId}/grammar` | The reverse lookup: which grammar sections cite this entry. |

There is no separate grammar-intro endpoint — the introductory prose is simply the
first top-level section.

Fill in the **glossing abbreviations legend** whenever you import glossed material —
but ONLY for what the standard Leipzig catalog can't cover. The site already expands
every standard code (`1SG`, `PFV`, `PL` …) in every dictionary, localized into the
reader's UI language; a curated row that just restates "first person singular" adds
nothing and blocks that localization. Register a code when:

- it's outside the standard set (`1SG>2SG`, `URG.FEM`, `Ø`), or
- this language's usage genuinely differs from the standard meaning (a "dual" that
  marks two *or more*).

Wording rules for curated expansions:

- **The dictionary stands alone.** Never write "the book's X" or reference the source
  PDF — readers who never saw the source will just wonder "what book?". Source
  attribution belongs in the import report, not in gloss tooltips.
- Anchor a code to its morpheme when that identification is the point
  (`negative — the suffix -aží 'not'`), but don't echo the morpheme the reader just
  tapped on.

## The word→entry matching queue

Sentences are tokenized and matched against entries on ingest, and every entry
write re-matches the words it names — you never need to import in a particular
order or trigger matching yourself. Forms that matched nothing, or matched
ambiguously, land in the review queue at `…/suggestions`, where they can be linked,
used to create an entry, or ignored. Ignoring a form dictionary-wide writes an
`ignored_forms` row the matcher then skips. Per-token actions go through
`…/sentences/{sentenceId}/tokens/actions`. `POST …/analyze` (`{ text_id }`,
`{ sentence_ids }`, or `{ unanalyzed: true }`) re-runs the matcher on demand — the
API twin of the reader's "Refresh matches" — for the rare catch-up (a sentence
written before 2026-09-18, or an entry edited outside the API).

**Bound morphemes match too.** A lexeme written with a leading hyphen (`-k`) is a
suffix, a trailing hyphen (`ka-`) a prefix. When a word matches no whole lexeme but
has exactly one reading as `stem+suffix` / `stem+suffix+suffix` / `prefix+stem` over
free lexemes, the matcher suggests that segmentation: the token gets `status:'auto'`
and `morphemes[]`, each morpheme carrying `entry_id` (unique) or `candidates` (the
homograph suffixes `-k¹ -k² -k³`). Two readings → the word stays unmatched for a
human. Per-token actions `segment` (`{ morphemes: [{ form, separator?, gloss?,
entry_id? }] }` — forms must spell the word) and `unsegment` store or drop a
segmentation; a `segment` is gold (`confirmed`) and replaces the word-level link.

## Cognate sets (word families)

For comparative material — multi-variety dictionaries, cognate ledgers, etymological
indexes — the dictionary has an N-ary layer above pairwise relationships:
a **cognate set** groups every entry that descends from one ancestor form, and the
reconstruction itself lives on the set (it is not a word in any documented variety,
so it is not an entry). Readers see these as "Word families".

```
GET/POST …/cognate-sets              list; create (single, or batch ≤1000 with per-item results)
GET/PATCH/DELETE …/cognate-sets/{setId}
POST …/cognate-sets/{setId}/members  idempotent upsert by entry_id
DELETE …/cognate-sets/{setId}/members/{entryId}
```

Shape rules worth internalizing:

- **A set needs a head**: a `reconstruction` (send it WITHOUT the asterisk — display
  adds it) or, when no ancestor has been reconstructed yet, a `gloss` ("water"). A
  family can be established on member evidence first and gain its proto-form later.
- **`proto_level`** ("Proto-Polynesian", "Proto-Oceanic") is free text — keep ONE
  consistent spelling per level within a dictionary; the UI suggests previously
  used values.
- **Members are entries** (`entry_id`, optional `sense_id` narrowing) with a
  per-member `note` and an `uncertain` flag. Prefer flagging a debatable reflex
  `uncertain: true` with a note over omitting it — absence is invisible, a flagged
  member is reviewable.
- **Sources are registry refs, never prose**: `citations: [{ slug, locator? }]`
  (`locator` = the set/page number in the comparative source). Register the
  comparative work via `POST …/sources` first.
- **When to use pairwise `cognate` relationships instead**: exactly two entries and
  no reconstruction-level claim. Don't mirror a set's members as pairwise links —
  the set already renders the family on every member's entry page.
- Entry reads embed `entry.cognate_sets` summaries when the entry belongs to any.

## Places (where words, texts and varieties are from)

When material is tied to localities — a survey that elicited the same wordlist in
twenty villages, stories recorded in named towns, varieties described as spoken in
particular counties or valleys — model the localities as **places** and link to them,
instead of copying coordinates onto every entry. Maps read these links.

```
GET/POST …/places                 list (filter ?dialect_id= / ?entry_id= / ?text_id= / ?where_spoken=1); create (single, or batch ≤1000)
GET/PATCH/DELETE …/places/{placeId}
POST/DELETE …/places/{placeId}/links   add/update or remove links to entries, texts, dialects, speakers, the language (≤1000, one transaction)
```

Shape rules worth internalizing:

- **`name` is a MultiString**: one key per language/script — the local name, its
  romanization, the national-language name, English. Readers choose which to show.
- **Say how good the coordinates are.** `precision` is one of `exact`, `village`,
  `town`, `township`, `district`, `county`, `region`, `approximate`. A point you only
  know to the county is the county seat with `precision: "county"` — maps draw it as
  a soft area, not a pin. A place with no coordinates at all is still worth creating.
  Never invent coordinates.
- **Nest with `parent_id`** (village → county → province). In a batch, list parents
  first and give every item your own `id` so children can reference them and
  retries are no-ops.
- **Links carry their meaning.** Entry links have an optional `note`. Text links have
  a `role`: `recorded` (told/recorded here) or `setting` (the story is about this
  place). Dialect links have a `role` (`documented` — surveyed here; `spoken` —
  reported as spoken here; `reported` — a weaker literature claim; `diaspora`) and a
  `weight` 0–1 for how strongly the place evidences the variety; maps fade a
  dialect's presence by it. Speaker links set the speaker's home place.
- **Where the language is spoken** (not tied to one dialect): link the place with
  `{ "dictionary": { "role": "spoken", "weight": 1 } }` (same roles and weight as a
  dialect link; `{}` means plain "spoken here"). The dictionary's home map draws these
  places together with its dialects' places, and the dictionary's catalog location (the
  world map and directory) is derived from them. Remove with `{ "dictionary": true }`
  on `DELETE …/links`.
- **Outlines.** An area place (a county, a valley, a variety's documented extent) may
  carry `boundary`: polygon rings of `[lng, lat]`, outer ring first. Send one only when a
  source actually draws or names the area; without it, maps estimate an outline from the
  linked points (solid where documented and central, dashed where reported or fringe).
- **Sources are registry refs**: `citations: [{ slug, locator? }]` for where the place
  or its coordinates come from (a gazetteer, the survey report's map).
- The old `coordinates` field on entries, dialects and the dictionary is deprecated.
  It is still accepted — each point/region is saved as an `approximate` place (or a
  `region` place with that outline) linked to the row, reusing a place within 150 m,
  and the response carries a `warnings` line — but create places directly: you can name
  them, state their precision and share one place across many entries.

## Etymology records (typed origins per entry)

Where a cognate set groups MANY entries under one ancestor, an **etymology
record** states ONE entry's origin in queryable, typed form:

```
GET/POST …/entries/{entryId}/etymologies
GET/PATCH/DELETE …/entries/{entryId}/etymologies/{etymologyId}
```

Shape rules worth internalizing:

- **`verdict` is the only required field**: `inherited` (carries
  `reconstruction_level` + `reconstruction`, sent WITHOUT the asterisk, and/or a
  `cognate_set_id` pointing at the word family), `loan` (carries
  `donor_language`, `donor_form`, and a free-text `stratum` layer label), or
  `unknown` — an `unknown` verdict with a note ("origin unclear, see
  discussion") is a legitimate, useful record.
- **Multiple records per entry are COMPETING HYPOTHESES** — file each proposal as
  its own record with its own sources rather than merging them into one.
- A `loan` may still carry `reconstruction_level`/`reconstruction` when the word
  was borrowed FROM a reconstructed stage.
- `confidence` is `certain` | `probable` | `possible` (or omitted).
- **The free-prose `linguistic_history` entry field stays as-is** — it's the
  discussion; the typed record is the queryable verdict. When a source states
  both, fill both.
- Sources are registry refs (`citations: [{ slug, locator? }]`), same as
  everywhere else.
- Entry reads embed `entry.etymologies` when any records exist.

## If a shape is awkward

This surface is newer than the rest of the API and still being shaped by real corpus
imports. If your data doesn't fit — `POST …/feedback` with `{ message }`. That
feedback genuinely changes what gets built.