RedlineKnowledge base

B2 Semantic and schema

One block model, shared by the service, the rendered page and the tests, and the contract that describes it.

T2.1 Share the semantic module

  • Wave: 1
  • Depends on: none
  • Size: S
  • Design: state-machine.md (block identity)
  • Touches: semantic.js (moved from service/src/semantic.js), service/src/gitlab.js, service/Dockerfile, index.js, index.d.ts, package.json (files), test/service.test.js (import path), test/semantic.test.js (new), README.md (one line under “Stable API surface”)
  • Why: the remark plugin, the client and the tests need the block model from the package root. It becomes a package module that the service imports.

Steps

  1. git mv service/src/semantic.js semantic.js.
  2. service/src/gitlab.js: import { … } from '../../semantic.js';. test/service.test.js: update its import to ../semantic.js.
  3. service/Dockerfile: the service’s working directory is /app and its source is at /app/src, so ../../semantic.js would resolve outside /app. Change the runtime stage to copy the service under /app/service and the shared module to /app/semantic.js: COPY --chown=node:node semantic.js ./semantic.js, COPY --chown=node:node service/package.json ./service/package.json, COPY --chown=node:node service/src ./service/src, move node_modules to ./service/node_modules, and set CMD ["node", "service/src/server.js"]. The health check path is unchanged.
  4. package.json files: add "semantic.js".
  5. index.js: export { parseDocumentBlocks, diffDocumentBlocks, detectLayerConflicts } from './semantic.js';
  6. index.d.ts: declare SemanticBlock, SemanticOperation, LayerConflict types matching contracts/openapi.yaml and the three function signatures.
  7. npm run build:container must succeed and the container’s health check must pass: run it with the T0.3 environment variables (or any valid set) and curl http://127.0.0.1:8787/v1/health.

Acceptance

test/semantic.test.js:

  • semantic exports are reachable from the package root
  • semantic block ids for the fixture versions match the canonical table (V1, V3 against blockIds from test/fixtures/mock-gitlab/data.js; if T0.1 is not done yet, inline the expected ids from fixture-data.md and switch to the import when it lands)
  • semantic diff of the fixture versions yields the canonical operations (the four diffDocumentBlocks expectations in fixture-data.md)
npm test 2>&1 | grep -c "^✔ semantic"   # prints 3
npm pack --dry-run 2>&1 | grep -c " semantic.js"   # prints 1
npm run check:service                              # exit 0
npm run build:container && docker run --rm -d --name redline-t21 -p 8787:8787 --env-file <a valid env file> fio-redline:local && sleep 3 && curl -sf http://127.0.0.1:8787/v1/health && docker rm -f redline-t21   # health JSON printed

(If Docker is unavailable locally, record it in STATUS.md; the pipeline’s container-build-push job on main is the backstop.)

T2.2 Remark plugin for block ids

  • Wave: 2
  • Depends on: T2.1
  • Size: M
  • Design: state-machine.md (block identity)
  • Touches: remark-redline-blocks.js (new), index.js (integration option blocks), index.d.ts, package.json (files, exports, devDependency @astrojs/markdown-remark exact), test/remark-blocks.test.js (new), test/fixtures/site/astro.config.mjs (gitHistory({ blocks: true })), README.md (short section “Block ids on rendered pages”)
  • Why: comments, redlines and changed-since all need rendered elements to carry the same ids the service computes from source. Today GitComments.astro invents dom:p:N ids that match nothing.

Steps

  1. remark-redline-blocks.js default export is a remark plugin; it imports from ./semantic.js. In the transformer: const blocks = parseDocumentBlocks(String(file.value)); index blocks by startLine. Walk tree.children; for list nodes walk their listItem children instead of the list. For each node, look up blocks.get(node.position.start.line); if absent, fall back to the first unused block whose fingerprint equals the fingerprint of the node’s source slice (file.value.slice(node.position.start.offset, node.position.end.offset)) computed by the same function as semantic.js (export fingerprintBlock(type, content) from semantic.js for this). On a match set node.data = { ...node.data, hProperties: { ...node.data?.hProperties, 'data-redline-block': block.id } } and mark the block used. Never assign one block id twice.
  2. index.js integration: when options.blocks === true, in astro:config:setup call updateConfig({ markdown: { remarkPlugins: [remarkRedlineBlocks] } }). blocks must not reach the serialised virtual module options (strip it).
  3. package.json: add "./remark-redline-blocks.js": "./remark-redline-blocks.js" to exports and the file to files.
  4. Fixture site: gitHistory({ blocks: true }).

Acceptance

test/remark-blocks.test.js using createMarkdownProcessor from @astrojs/markdown-remark:

  • remark plugin stamps every heading paragraph and code block with its semantic id (render V3; assert the HTML contains data-redline-block="<id>" exactly once for each of the four V3 ids)
  • remark plugin ids match parseDocumentBlocks for a document with frontmatter lists and code (a 30-line sample with frontmatter, two heading levels, a bullet list of three items and a fenced block; strip the frontmatter with /^---\n[\s\S]*?\n---\n/ before rendering, as Astro does, and compare against parseDocumentBlocks(fullSource).filter((block) => block.type !== 'frontmatter'); every remaining block of type heading, paragraph, list-item, code appears once in the HTML with its id)
  • remark plugin never assigns one id twice (two identical paragraphs; ids …:paragraph and …:paragraph-2 both present)
npm test 2>&1 | grep -c "^✔ remark plugin"   # prints 3
npm run fixture:build && grep -c 'data-redline-block="title:paragraph:paragraph"' test/fixtures/site/dist/index.html   # prints 1

T2.3 Block history in the service

  • Wave: 2
  • Depends on: T2.1, T0.1
  • Size: M
  • Design: state-machine.md (blockChanges, blockHistory), fixture-data.md
  • Touches: service/src/gitlab.js, test/block-history.test.js (new)
  • Why: “changed since” needs, per block, the commits that changed it.

Steps

  1. In getDocumentHistory, after commits are mapped, compute for each commit (in parallel with mapLimit(..., 4, ...)):
    • after = getFileText(sha) and before = parent ? getFileText(parent) : '', where getFileText(ref) calls getFile with cacheTtl: 86400 and returns '' on a 404 ServiceError. Extend getFile with a fifth argument { cacheTtl = 60 } for this; existing callers are unchanged.
    • blockChanges = diffDocumentBlocks(before, after).operations.map(({ type, blockId }) => ({ blockId, type })).
    • On a 413 (document-too-large) set blockChanges = null.
    • The parent is commit.parent_ids?.[0]; the raw commit list already has it, so carry parentIds through the mapped commit.
  2. Build blockHistory from the commits in order (they are newest first): for each commit with a non-null blockChanges, push { sha, committedAt, type } onto blockHistory[blockId].
  3. Return blockHistory beside commits in the history object. The /documents/state route already embeds history, so no route change.

Acceptance

test/block-history.test.js with the shared mock:

  • document history carries block changes per commit (c1 → two inserts, c2 → one replace of title:paragraph:paragraph, c3 → two inserts)
  • document history aggregates block history newest first (deep-equal to the expected blockHistory in fixture-data.md ignoring committedAt ordering already implied)
  • document state exposes block history (GET /v1/sites/demo/documents/state → history.blockHistory has the four keys)
  • block changes are null for an oversized document (mock returns a file over MAX_DOCUMENT_BYTES for one sha)
npm test 2>&1 | grep -c "^✔ document history\|^✔ document state exposes\|^✔ block changes are null"   # prints 4

T2.4 Failure classification

  • Wave: 2
  • Depends on: T0.5
  • Size: S
  • Design: state-machine.md (classification, messages)
  • Touches: client.js, index.d.ts, index.js (re-export), test/client-state.test.js (new)
  • Why: one function decides which of the three failure states applies; every component and every test uses it.

Steps

  1. client.js: export REDLINE_STATES (array of the eight state names), REDLINE_MESSAGES (object, exact strings from the design; live and error are functions of time and code), classifyRedlineFailure({ error, response, payload }) and deriveRedlineState(payload) exactly as the design specifies.
  2. RedlineClientError: add a kind property set by classifyRedlineFailure where the client throws.
  3. index.js re-exports the four names. index.d.ts declares them.

Acceptance

test/client-state.test.js:

  • classifies a thrown TypeError as offline-network
  • classifies 502 gitlab-unreachable as offline-gitlab
  • classifies 401 as unauthenticated
  • classifies other failures as error
  • derives connect-required from a payload without a GitLab session
  • derives live from a payload with a GitLab session
  • messages exist for every state
npm test 2>&1 | grep -c "^✔ classifies\|^✔ derives\|^✔ messages exist"   # prints 7

T2.5 Contract tests

  • Wave: 3
  • Depends on: T0.5, T2.3
  • Size: M
  • Design: state-machine.md (schema additions)
  • Touches: contracts/openapi.yaml, test/contract.test.js (new), package.json (devDependencies ajv, ajv-formats, yaml, exact)
  • Why: the schema is the written form of the data model. Tests must fail when the service drifts from it.

Steps

  1. openapi.yaml: add a Health schema with gitlab: { reachable, checkedAt, status } and reference it from /v1/health; add blockChanges to Commit (type: [array, 'null'], items { blockId, type: enum }); add blockHistory to History (object with additionalProperties array of { sha, committedAt, type }); add gitlab-unreachable to the error code description; add a note on IDENTITY_MODE in info.description. Bump info.version to 1.1.0.
  2. test/contract.test.js: load the YAML, create new Ajv({ strict: false, allErrors: true }) with addFormats, ajv.addSchema({ $id: 'openapi', components: doc.components }), and validate with { $ref: 'openapi#/components/schemas/<Name>' }. Produce the payloads by calling the service handler with the shared mock.

Acceptance

  • health payload validates against the Health schema
  • document state payload validates against the DocumentState schema
  • error payload validates against the Error schema
  • document state payload with an unknown required field fails validation (delete history from the payload; expect valid === false, proving the validator bites)
npm test 2>&1 | grep -c "validates against\|fails validation"   # prints 4

T2.6 Word-level diff

  • Wave: 2
  • Depends on: T2.1
  • Size: S
  • Design: fixture-data.md (expected diffWords)
  • Touches: semantic.js, index.js, index.d.ts, test/semantic.test.js
  • Why: inline redline renders a replaced block as struck and inserted words, not two whole paragraphs.

Steps

  1. Export diffWords(before, after) from the root semantic.js. Tokenise with text.split(/(\s+)/).filter(Boolean) so whitespace tokens are kept. Run the same LCS as lcsMatches over token strings (refactor lcsMatches to accept a key function rather than duplicating it). Emit segments { type: 'equal' | 'delete' | 'insert', text } in order, merging adjacent segments of the same type. When either side exceeds 2,000 tokens return [{ type: 'delete', text: before }, { type: 'insert', text: after }].
  2. Re-export from index.js; declare in index.d.ts.

Acceptance

Add to test/semantic.test.js:

  • word diff of the fixture paragraphs matches the canonical segments
  • word diff merges adjacent segments and preserves whitespace ("a b c" → "a x y c" gives equal "a ", delete "b", insert "x y", equal " c")
  • word diff falls back to whole-block segments over the token limit
npm test 2>&1 | grep -c "^✔ word diff"   # prints 3
Git history

Loading the page's history…