RedlineKnowledge base

Build-time history index

Problem

getGitMeta(path) runs git log --follow for one path. A static build calls it once per page, so a 300-page site runs about 300 history walks plus a git log -1, a git status and a git ls-files per page. The consuming site measured about 90 seconds for 303 pages and turned the feature off.

Design

Build one index for the whole repository on the first call, then answer every page from memory. The number of git invocations per build becomes a constant.

Git invocations per build

CallPurposeCount
git rev-parse --show-toplevelrepository root1
git rev-parse --abbrev-ref HEAD and --short HEADbranch label1 or 2
git rev-parse --is-shallow-repositorycompleteness1
git rev-parse HEADindex cache key1
git log (below)the index1, skipped on a warm cache
git status --porcelain -- <roots>dirty paths1
git ls-files -- <roots>tracked paths1

Cold: at most 8. Warm: at most 7. Independent of page count.

The log command

git -c core.quotePath=false log --name-status -M --no-color \
  --format='%x1e%H%x1f%aI%x1f%G?%x1f%an%x1f%ae%x1f%s' -- <root>...

Output is a sequence of records. A record starts with the ASCII record separator \x1e, then the six fields separated by \x1f, then zero or more status lines of the form M\tpath, A\tpath, D\tpath, or R<score>\told\tnew (also C<score> which is treated like R). Blank lines are skipped.

%G? asks git to classify each commit’s signature; it is one letter per commit (G good, B bad, U unknown validity, N none, and others). The existing per-path code uses the same field; signed = sig !== 'N' && sig !== ''.

Rename resolution

Walk records newest to oldest, keeping canonical: Map<pathThen, pathNow>.

for each record (newest first):
  for each status line:
    if R/C old new:
      now = canonical.get(new) ?? new
      rows[now].push(record)
      canonical.set(old, now)        # older history of `old` belongs to `now`
    else (M/A/D path):
      now = canonical.get(path) ?? path
      rows[now].push(record)

A commit touching a path produces one row for that path; a commit that touches many paths produces one row per path. Rows for a path are therefore in newest-first order. This matches git log --follow for the cases the tests cover (edits, a rename, a rename followed by edits on both sides). Known difference: --follow can detect renames across commits that also modify the content heavily; -M with the default threshold may not. The acceptance test uses a plain git mv.

From rows to GitMeta

For a path with rows r[0..n-1] (newest first), the same derivation as today:

  • created = r[n-1].date, updated = r[0].date, revisions = n
  • contributors: group by email (fallback name), count, sort by count desc
  • authors = contributors.map(name)
  • signed = r[0].sig !== 'N' && r[0].sig !== ''
  • state = 'committed', trackingSource = 'git', gitlabEnhanced = false
  • historyIncomplete = isShallow
  • dirty = dirtyPaths.has(path), branch as today

Paths that are tracked but have no rows (possible only in a shallow clone) return the existing history-unavailable shape. Paths not tracked return untracked or staged exactly as today, using the status --porcelain set (A prefix means staged, ?? means untracked).

Rows are also kept on the result as localCommits ({ sha, date, sig, name, email, title }[], newest first) so the GitLab enrichment (T1.4) can skip its per-path API call.

Options

getGitMeta(path, opts) and the integration accept:

OptionDefaultMeaning
indextruebuild and use the index; false keeps the per-path walk
contentRoots['.']pathspecs passed to git log, status and ls-files; narrow for large repositories

A path outside contentRoots falls back to the per-path walk and is counted in the stats.

Cache

The existing cache file gains a top-level __index entry:

{ "__index": { "head": "<HEAD sha>", "roots": ["."], "rows": { "<path>": [ ...rows ] } } }

On init, when __index.head equals the current HEAD and roots match, the git log call is skipped and rows are loaded from the file. dirty, branch and the tracked set are always recomputed because they are cheap and volatile.

Observability

_getGitStats() returns { gitCalls, indexSource: 'walk' | 'cache' | 'none', indexPaths, indexMs } and is reset by _resetGitState() (tests only). When the index is built from a walk and silent is not set, one line is printed:

[fio-redline] indexed <indexPaths> paths in <indexMs> ms (<gitCalls> git calls)

The consuming site’s build log is checked for this line in T5.1.

Git history

Loading the page's history…