fix(web,server): 让服务端只交章节,并去掉成对的 sync 请求

服务端仍在做排版:`chapter-html:<n>` 自己断段、自己提升标题、自己吐
`<p>`。于是 reader 设置里的缩进和段间距只能作用于服务端已经渲染好的那一份,
流式那条路(`chapter:<n>`)拿不到段落,同一本书因为走哪条传输而看起来不一样。
客户端的 `retypePlainText` 于是把服务端断好的段落再拆一遍——同一个判断被两个
地方执行,两次的结果就会分歧。

`POST /api/v1/sync` 的响应本来就是合并后的状态(服务端在两个接口上返回同一份
`pull`),但客户端把它丢掉,然后又为同一份数据发一个 `GET`。读者每翻一页都发
一个 POST 加一个 GET,网络面板里成对出现。另外 `setState` 每次调用都广播,而
阅读页的状态监听会重绘整棵 chrome 树,一个同步循环通知三次。

现在服务端只把整章字符交出来(`text/plain`,`chapter-html:` 不截断),断段、
去段前空格、提升标题全部由 `web/src/formats/segments.ts` 一处决定;push 的返回
值直接喂给 `applyPulled`,空闲轮询才真的发 GET;`setState` 值未变就不广播。

「这一章是不是纯文本」改由 manifest 声明(item 的 `format: 'html'` →
`Section.plainText` → `isPlainText()`),不再靠 `.txt-body` 嗅探:服务端现在发
裸字符,嗅探对一章节全是纯文本的章节会答「不是」,读者会看到一本没有段落、
没有缩进、样式表也没挂上的小说。`.txt-body` 标记降级为旧服务端/旧缓存的兜底。

放弃的方案:保留服务端渲染、只在客户端覆盖样式。缩进和段间距是阅读偏好,服务端
渲染就等于把每次调整变成一次往返,而这个往返正是问题本身。也考虑过让
`chapter-html:` 直接返回服务端渲染的裸 `<p>`(不带 wrapper),但那仍然有两个
地方在判断「一段是什么」。

Ref: #34
This commit is contained in:
hectorqin committed 2026-09-18 23:39:55 +08:00
1 parent 7c9aa29faa
commit 1c2d07ba4c
23 files changed
+847 -429

No files matched your search

+16
View File
@@ -151,6 +151,22 @@ id = sha256( (dc:identifier ?? '').toLowerCase() + '\0' + sha256(fileBytes) )
`GET /api/v1/sync?since=<上次的 serverTime>`。客户端记住上次返回的 `serverTime`,
每次只拿增量。移动网络下重连很便宜。
### 一次写入只发一个请求
`POST /api/v1/sync` 的响应**就是合并后的状态**(服务端在两个接口上返回同一份
`pull`)。所以推完之后**不要再拉一次**——`SyncEngine` 直接拿 push 的返回值喂给
`applyPulled`,没有东西可推(空闲轮询)时才真的发 `GET`。
之前不是这样:push 的返回值被丢掉,然后为了同一份数据再发一个 `GET`。于是读者每
翻一页都发一个 POST **加** 一个 GET,网络面板里就是成对出现的 `sync` ——这正是
「sync 接口在不停重复」的来源。
### 状态只在变化时广播
`setState` 先比对新旧 state/message,相同就直接返回。阅读页的状态监听会重绘整棵
chrome 树,而一个同步循环本来会通知三次(进门 `syncing`、出门 `idle`、再加一次
多余的 `emit()`),30 秒轮询让这件事在读者什么都不做时也持续发生。
---
## 6. 认证与安全
+1
View File
@@ -35,5 +35,6 @@
| 触控目标不小于 28px | ✅ | 9 个控件全部合格 |
| 每个控件都有可读的名字 | ✅ | 9 个控件全部有名字 |
| 没有横向溢出 | ✅ | document 溢出 0px |
| 翻页不再每次都发两次 sync | ✅ | 8 次翻页 + 6 秒静默:1 次 POST / 0 次 GET(共 1 次 sync 请求) |
全部通过。
+5
View File
@@ -130,6 +130,11 @@
"name": "没有横向溢出",
"ok": true,
"detail": "document 溢出 0px"
},
{
"name": "翻页不再每次都发两次 sync",
"ok": true,
"detail": "8 次翻页 + 6 秒静默:1 次 POST / 0 次 GET(共 1 次 sync 请求)"
}
],
"failures": []
+28 -18
View File
@@ -302,34 +302,44 @@ TXT 是唯一一种**没有作者排版**的书,所以它也是唯一一处「
默认值」是对的、而不是侵入的地方。EPUB 的默认必须交给作者自己的样式表,
覆盖它是 bug;TXT 没有作者。
三层分工,改任何一层之前先看清是哪一层。**排版是客户端的,服务端只负责把
章节完整地交出来**——这条边界是这一层最重要的一条规定:
两层分工,改任何一层之前先看清是哪一层。**排版全部在客户端,
服务端只负责把章节完整地交出来**——这条边界是这一层最重要的一条规定:
- **服务端**(`server/src/indexer/formats/text-html.ts`):只做两件事。把整章
交出来(`chapter-html:<n>` 不按 `chapter:` 那样截断),以及**在原文上**断段
——空行断段,「上一行以句末标点结尾、且下一行以句子开头」也断段(抓取来的
网文常常每行一段且没有空行),章节标题行提升成 `<h3>`(否则正文会变成
`第二章 落雨雨来了。`)。缩进、段间距、去掉段前空格**都不在这里**:它们是
每台设备各自的阅读偏好,服务端烧进响应里就等于把每次调整变成一次往返。
- **服务端**(`server/src/indexer/formats/text.ts`):只做一件事——把**整章
交出来**。`chapter:` 是为流式读取开的窗口,按 `size` 截断是它的功能;
`chapter-html:<n>` 是给读者整章读的,**不截断**,内容类型是
`text/plain`。它**不再断段、不再提升标题、不再输出 `<p>`**:断段是阅读
决定,服务端做就等于让 windowed 那一份有段落、流式那一份没有,也让读者的
缩进设置变成一次往返。
- **客户端排版**(`web/src/formats/segments.ts`):`splitTextParagraphs` 是
「一段是什么」的**唯一定义**,本地加载的 TXT、windowed 的
`chapter:<n>`、流式的 `chunk:` 三条路都走它,所以同一本书不会因为走哪条
`chapter-html:<n>`、流式的 `chunk:` 三条路都走它,所以同一本书不会因为走哪条
传输看起来不一样。它同时负责**去掉段前空格**(`  正文` 里那个全角空格是
「段前空格」的来源,留着就是和读者的缩进**叠加**)。`ReaderView` 在
`retypePlainText` 里把服务端已经断好的章节也再过一遍这段逻辑,拼回空行再
拆——所以服务端断得对不对不影响观感,它只是让客户端的拆分更准。
「段前空格」的来源,留着就是和读者的缩进**叠加**),以及**章节标题行提升成
`<h3>`**(`splitChapterHeading`,否则正文会变成 `第二章 落雨雨来了。`)。
`ReaderView` 在 `retypePlainText` 里对拿到的章节过这一段逻辑;如果内容带着
历史的 `.txt-body` 标记(旧服务端、旧缓存),就按块读回文本再拆,所以旧
响应不会被当成新响应渲染。
- **客户端样式表**(`web/src/ui/shadow.ts` 的 `TXT_STYLESHEET`):只在
`format === 'txt'`、或章节带着 `.txt-body` 标记时注入,作用域是
`.txt-body`。默认两字符首行缩进(`2em`,不是 `2rem`——缩进要跟字号缩放)。
样式表里**没有任何一条规则读服务端写的属性**:「不缩进」是同一个自定义属性
取 0,而不是第二条要和第一条保持一致的分支。
`ReaderView.isPlainText()` 为真时注入,作用域是 `.txt-body`。默认两字符首行
缩进(`2em`,不是 `2rem`——缩进要跟字号缩放)。样式表里**没有任何一条规则读
服务端写的属性**:「不缩进」是同一个自定义属性取 0,而不是第二条要和第一条
保持一致的分支。
- **「这一章是不是纯文本」怎么判定**:靠**声明**,不靠嗅探。`manifest` 的
content item 写 `format: 'html'` → `createStagedDoc` 把它落到
`Section.plainText` → `isPlainText()` 认它。这条链必须完整:服务端现在发的是
**裸字符**,没有 `.txt-body` 包裹,所以「正文里有没有 `txt-body`」这个老办法
对一章节全是纯文本的章节会答「不是」——读者于是看到一本没有段落、没有缩进、
样式表也没挂上的小说,而这看起来像排版偏好而不像 bug。`.txt-body` 标记只留作
最后兜底(旧服务端/旧缓存)。
- **阅读设置**:段落缩进、段间距、编码三行,只在 TXT 上出现;设置面板顶部的
「快捷」一行(字号/缩进/段间距/主题)也是同一批值的另一个入口,一次点击
走一档。前两个是写在 stage 上的自定义属性(`--reader-txt-indent` /
`--reader-txt-para-gap`),所以调整不需要重新注入章节、也不需要请求。
引用保持 `chapter:<n>` 不变,新增 `format: 'html'` 字段告诉客户端取哪一份。
`href` 同时是**阅读位置的标识**,改它会把每个现有 TXT 读者打回第一章。
引用保持 `chapter:<n>` 不变,`format: 'html'` 字段告诉客户端取哪一份
(`renditionRef` 据此把它换成不截断的 `chapter-html:<n>`)。`href` 同时是
**阅读位置的标识**,改它会把每个现有 TXT 读者打回第一章。
---
+19 -159
View File
@@ -1,95 +1,28 @@
/**
* The `chapter-html:<n>` rendition: one chapter, whole, as a document.
* The `chapter-html:<n>` rendition: one chapter, whole, and nothing else.
*
* This is a *transport* for a chapter's characters, not a typesetting pass. The
* server's only two jobs are to hand over the entire chapter (nothing truncated)
* and to say where its paragraphs are (a blank line, a line ending, a
* sentence-final newline) — the boundaries are a property of the file, so they
* are inferred once, here, against the undecoded text.
* This file used to be a typesetting pass — it inferred paragraph boundaries,
* promoted a heading and emitted `<p>` elements. It is not one any more, and the
* change is deliberate: the reader's paragraph boundaries, its indent, its
* paragraph spacing and the removal of a scraper's leading spaces are all
* *reading* decisions. A server that baked them into a response made every one of
* them a round trip, and it meant the same file could be typeset two different
* ways depending on which transport happened to deliver it (the windowed
* `chapter-html:` path got markup; the streamed `chapter:` path got characters and
* no paragraphs at all).
*
* The indentation, the paragraph spacing and the removal of the leading spaces a
* scraper wrote are *not* the server's business, and that is a deliberate change.
* They are reading preferences: a phone and a tablet want different indents, and
* a reader changes them while looking at the page. A server that baked them into
* a response made every one of those changes a round trip, and it made a book
* read windowed (`chapter:<n>`, no markup to carry the decision) look different
* from the same book read whole. The client typesets the text it is given (see
* `web/src/formats/segments.ts`), so the two transports agree by construction and
* the setting costs no request.
* So the server's job has shrunk to the one thing only it can do: hand over the
* characters of a chapter, complete. The client typesets them — see
* `web/src/formats/segments.ts`, which is the single definition of what a paragraph
* is for a TXT and is shared by every path a text can reach the reader through.
*
* What is left here is the shape the client's shadow root expects: a *fragment*
* wrapped the same way an EPUB chapter is (`<div class="txt-body">…`). Nothing is
* sanitised away by `sanitiseInjectedContent`: no scripts, no absolute URLs, and
* every character escaped.
* The reference is kept (`chapter-html:` rather than deleting it and rewriting the
* client's `renditionRef`), because it is opaque and format-specific by design: a
* client that still asks for `chapter-html:` must keep working, and it does — it
* now receives the same bytes as `chapter:`. Removing it would have been a
* breaking change to a published reference for no gain.
*/
/** Paragraphs are separated by a blank line; a single newline is a soft wrap. */
const PARAGRAPH_BREAK = /\n{2,}/;
/**
* A lone carriage return is a line break too.
*
* Old conversions and files off a serial console use `CR` on its own, and a
* newline-only split leaves the whole file as one paragraph — the exact failure
* this file exists to prevent, arriving through the one encoding path nobody
* tests. Normalised first so every rule below sees `\n`.
*/
const LINE_ENDINGS = /\r\n?/g;
/**
* A paragraph that ends a sentence is a paragraph even without a blank line.
*
* This is the one heuristic worth having for Chinese web novels: tools that scrape
* them routinely emit every paragraph on its own line with no blank line between,
* and treating each line as its own paragraph is the only way such a file gets
* any indentation at all. The lookahead requires the *next* line to start a new
* sentence (not a continuation), so a paragraph wrapped across two lines is still
* joined rather than split down the middle.
*/
const SENTENCE_BREAK = /(?<=[。!?…”』】])\n(?=\s*[^\s,。!?、;:”』】)])/;
/**
* Characters that begin a *continuation* rather than a sentence.
*
* A line starting with one of these is finishing the sentence above it: a closing
* quote, bracket or dash carried over, or a clause opening with punctuation. The
* lookahead in `SENTENCE_BREAK` already excludes the ones that are unambiguous;
* this list is the same exclusions plus the closing brackets that only *look* like
* they could start a sentence.
*/
const CONTINUATION_START = /^[\s,。!?、;:”』】)\u3001-\u303f\uff01-\uff5e]/;
/**
* A leading heading line, promoted out of the body.
*
* A chapter's own slice *includes* its heading line — `splitChapters` records the
* heading as the chapter's first line, which is what makes a locator land on the
* chapter's title rather than one line into it. So the body handed to this file
* starts with the title, and rendering it as an ordinary paragraph produces the
* tell-tale `第二章 落雨雨来了。` — title and first sentence glued together with no
* separation.
*
* Promoted to a real heading rather than merely split off, because it is the one
* piece of structure a TXT actually has, and a reader scanning for a chapter
* boundary is looking for exactly this.
*/
const HEADING_PATTERNS: readonly RegExp[] = [
/^\s*第\s*[0-90-9零一二三四五六七八九十百千万两]+\s*[章回节卷部篇集]\s*[::.、\-—]?\s*.{0,60}$/,
/^\s*(?:序章|序言|楔子|引子|前言|后记|尾声|终章|番外|附录)\s*.{0,40}$/,
/^\s*[卷部]\s*[0-90-9零一二三四五六七八九十百千万两]+\s*.{0,60}$/,
/^\s*Chapter\s*\d+.*$/i,
/^\s*#{1,3}\s+.+$/,
];
/** The longest line still plausibly a heading rather than prose. */
const MAX_HEADING_LENGTH = 60;
function isHeading(line: string): boolean {
const trimmed = line.trim();
if (trimmed.length === 0 || trimmed.length > MAX_HEADING_LENGTH) return false;
return HEADING_PATTERNS.some((pattern) => pattern.test(trimmed));
}
/** Escapes the five characters that would otherwise let a file rewrite the page. */
export function escapeHtml(text: string): string {
return text
@@ -99,76 +32,3 @@ export function escapeHtml(text: string): string {
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');
}
/**
* Splits a chapter body into paragraph-sized strings.
*
* Exported because the split is the part worth testing: the escaping and the
* wrapping below are mechanical, and every interesting failure of this file is a
* paragraph boundary in the wrong place.
*/
export function splitParagraphs(body: string): string[] {
return body
.replace(LINE_ENDINGS, '\n')
.split(PARAGRAPH_BREAK)
.flatMap((block) => splitSentenceBreaks(block))
.map((block) => block.replace(/\n/g, '').trim())
.filter((block) => block.length > 0);
}
/**
* Splits a block on sentence-final newlines, leaving continuations joined.
*
* `String.prototype.split` cannot express this on its own: the lookahead decides
* from the *next* line whether the newline was a paragraph boundary, and a regex
* split throws that line away with the separator. So the lines are walked, and a
* newline is promoted to a boundary only when the line above ends a sentence and
* the line below starts one.
*/
function splitSentenceBreaks(block: string): string[] {
const lines = block.split('\n');
const out: string[] = [];
let current = lines[0] ?? '';
for (let index = 1; index < lines.length; index += 1) {
const line = lines[index] ?? '';
const endsSentence = SENTENCE_END.test(current.trimEnd());
const startsSentence = line.trim().length > 0 && !CONTINUATION_START.test(line);
if (endsSentence && startsSentence) {
out.push(current);
current = line;
} else {
current += `\n${line}`;
}
}
out.push(current);
return out;
}
/** True when a line ends in a mark that closes a sentence. */
const SENTENCE_END = /[。!?…”』】]$/;
/**
* Renders a chapter body as the `chapter-html` payload.
*
* A `<div class="txt-body">` rather than bare `<p>`s, so the client can scope its
* own TXT rules to exactly this content and leave an EPUB's chapters untouched.
* The markup it contains is what the client is expected to *re-typeset*; what the
* server promises is the whole chapter and the paragraph boundaries within it,
* and nothing about how it looks.
*/
export function renderChapterHtml(body: string): Buffer {
const normalised = body.replace(LINE_ENDINGS, '\n');
const lines = normalised.split('\n');
// Only the *first* non-blank line is considered: a heading anywhere else is a
// line of prose that happens to look like one, and promoting those would
// restructure the book rather than render it.
const firstIndex = lines.findIndex((line) => line.trim().length > 0);
const hasHeading = firstIndex !== -1 && isHeading(lines[firstIndex] ?? '');
const heading = hasHeading ? (lines[firstIndex] ?? '').trim() : '';
const rest = hasHeading ? lines.slice(firstIndex + 1).join('\n') : normalised;
const parts = splitParagraphs(rest).map((paragraph) => `<p>${escapeHtml(paragraph)}</p>`);
if (heading) parts.unshift(`<h3>${escapeHtml(heading)}</h3>`);
const html = parts.join('\n');
return Buffer.from(`<div class="txt-body">\n${html}\n</div>\n`, 'utf8');
}
+32 -26
View File
@@ -11,7 +11,6 @@ import {
type TocEntry,
} from './registry.ts';
import { filenameMetadata } from '../metadata.ts';
import { renderChapterHtml } from './text-html.ts';
/**
* Plain text and Markdown.
@@ -181,12 +180,14 @@ export const textHandler = registerFileHandler({
seq: index,
title: chapter.title,
kind: 'chapter' as const,
// The item *is* markup as far as a reader is concerned, and saying so here
// is what stops the client from having to special-case the text format.
// The reference stays `chapter:<n>` so a position saved before this field
// The item is a *document* as far as a reader is concerned even though the
// bytes are characters: the client turns it into one, and saying so here is
// what stops the client from having to special-case the text format. The
// reference stays `chapter:<n>` so a position saved before this field
// existed still resolves; `format` tells the client which rendition to ask
// for (see `ContentItem.format`).
mediaType: 'text/html; charset=utf-8',
// for (see `ContentItem.format`), and `html` now means "the whole chapter,
// to be typeset" rather than "server-rendered markup".
mediaType: 'text/plain; charset=utf-8',
href: `chapter:${index}`,
format: 'html' as const,
})),
@@ -206,40 +207,45 @@ export const textHandler = registerFileHandler({
/**
* Body text. Three addressing modes, because clients need all three:
* - `chapter-html:<n>` one whole chapter as a document (the reader draws this)
* - `chapter:<n>` for the same chapter as plain text, for callers that want
* the characters rather than a document (TTS, search, a diff)
* - `chapter-html:<n>` one whole chapter, as the characters it is stored as
* - `chapter:<n>` the same chapter, capped at `size` (a streaming window)
* - `chunk:<byteOffset>` for streaming a novel with no headings
*
* Neither rendition carries the reader's typography. Indentation, paragraph
* spacing and the removal of a scraper's leading spaces are per-device reading
* preferences, applied by the client over whatever text either form delivers
* (see `web/src/formats/segments.ts`); what the server owes the reader is the
* chapter entire, and the paragraph boundaries it can only infer with the file
* in hand.
* Not one of them carries the reader's typography, and not one of them carries
* paragraph boundaries either. `chapter-html:` used to render `<p>` markup on
* the server; it no longer does, because paragraph splitting is a *reading*
* decision (see `web/src/formats/segments.ts`) and doing it here meant the
* windowed path got paragraphs while the streamed path could not, and meant the
* reader's indent was a round trip. What the server owes the reader is the
* chapter entire; the client typesets it.
*
* `chapter-html:` is therefore now an alias for the whole chapter as plain
* characters, with the one difference that it is *not* capped: `chapter:` bounds
* its reply because it is a window for streaming, and bounding a chapter the
* reader is about to read would hand them a chapter that stops mid-sentence.
*/
async asset(ctx: HandlerContext, req): Promise<AssetPayload> {
const size = Math.min(MAX_CHUNK_BYTES, DEFAULT_CHUNK_BYTES);
const { text } = await readTextFile(ctx.absPath);
// The whole chapter, uncapped. See the method comment: `chapter:` bounds its
// reply because it is a streaming window, and that bound must not be applied
// to a chapter the reader is about to read in one piece.
if (req.ref.startsWith(CHAPTER_HTML_REF)) {
const index = Number.parseInt(req.ref.slice(CHAPTER_HTML_REF.length), 10);
const { chapters, lines } = splitChapters(text);
const chapter = chapters[index];
if (!chapter) throw new Error(`chapter ${index} is out of range`);
// Deliberately *not* capped at `size`, unlike `chapter:` below. The two
// renditions answer different questions and the cap belongs to only one of
// them: `chapter:` is a *window* into the file for streaming, so bounding it
// is the feature; this is one whole chapter for a reader, so bounding it
// would leave them with a chapter that stops mid-sentence. The client's own
// `textToParagraphHtml` is what turns it into paragraphs — this is the
// chapter's characters, complete, and the reader typesets them.
const body = lines.slice(chapter.startLine, chapter.endLine + 1).join('\n');
const data = renderChapterHtml(body);
return {
data,
contentType: 'text/html; charset=utf-8',
size: data.byteLength,
data: Buffer.from(body, 'utf8'),
// Plain characters, not `text/html`. The content type has to say what the
// body is: served as HTML, a chapter whose text happens to contain `<`
// would be parsed as markup by anything that trusted the label, and the
// client's own escaping — the thing that makes a TXT safe — would be
// sidestepped. The client typesets from this text; it does not trust it.
contentType: 'text/plain; charset=utf-8',
size: Buffer.byteLength(body, 'utf8'),
};
}
+31 -6
View File
@@ -645,10 +645,14 @@ describe('reading endpoints per format', () => {
assert.ok(!res.body.includes('风来了'), 'chapter content must not bleed into the next chapter');
});
test('txt chapter content is also offered as renderable markup', async () => {
// The rendition the reader actually draws. `chapter:` is the characters; this
// is the same chapter as a document, which is what gives a TXT paragraphs it
// can style instead of a wall of text with no line breaks in it.
test('the chapter-html reference hands over the whole chapter, unmarked up', async () => {
// This reference used to answer with server-rendered `<p>` markup. It no
// longer does: paragraph boundaries are a *reading* decision, and rendering
// them here meant the reader's indent was a round trip and a windowed chapter
// looked different from a streamed one. What it answers with is the chapter's
// characters and nothing else — the same bytes `chapter:` gives, minus the
// streaming cap — which is what makes one typesetting pass on the client the
// single definition of a paragraph.
const book = await findBook('小说');
const res = await app.inject({
method: 'GET',
@@ -656,11 +660,32 @@ describe('reading endpoints per format', () => {
headers: auth(),
});
assert.equal(res.statusCode, 200, res.body);
assert.match(res.headers['content-type'] as string, /text\/html/);
assert.match(res.body, /<p>雨来了。<\/p>/);
assert.match(res.headers['content-type'] as string, /text\/plain/);
assert.match(res.body, /雨来了/);
assert.doesNotMatch(res.body, /<p>/, 'the server must not typeset any more');
assert.doesNotMatch(res.body, /<div/, 'the server must not wrap the chapter any more');
assert.ok(!res.body.includes('风来了'), 'chapter content must not bleed into the next chapter');
});
test('chapter-html is not capped the way the streaming chapter reference is', async () => {
// `chapter:` is a *window* for streaming, so bounding its reply is the
// feature. `chapter-html:` is a chapter the reader is about to read in one
// piece, so bounding it would hand them a chapter that stops mid-sentence.
const long = Array.from({ length: 4000 }, (_, i) => `第 ${i} 行的内容,足够长以越过流式窗口。`).join('\n');
const target = join(booksDir, '超长章节.txt');
await writeFile(target, `第一章 长\n${long}\n`, 'utf8');
await ctx.scanner.scan();
const book = await findBook('超长章节');
const res = await app.inject({
method: 'GET',
url: `/api/v1/books/${book.id}/assets?ref=${encodeURIComponent('chapter-html:0')}`,
headers: auth(),
});
assert.equal(res.statusCode, 200, res.body);
const text = res.body;
assert.ok(text.includes('第 3999 行'), 'the last line of the chapter must be present');
});
test('the manifest advertises the markup rendition without moving the reference', async () => {
// The reference is also the reading position's identity, so it has to stay
// `chapter:<n>` across this change; `format` is what carries the new choice.
-132
View File
@@ -1,132 +0,0 @@
import assert from 'node:assert/strict';
import { test, describe } from 'node:test';
import { escapeHtml, renderChapterHtml, splitParagraphs } from '../src/indexer/formats/text-html.ts';
/**
* The paragraph split is where every interesting failure of the TXT rendition
* lives, so it gets the tests. The escaping and the wrapping around it are
* mechanical.
*/
describe('splitParagraphs', () => {
test('a blank line separates paragraphs', () => {
assert.deepEqual(splitParagraphs('第一段\n\n第二段'), ['第一段', '第二段']);
});
test('a single newline inside a paragraph is a soft wrap, not a break', () => {
// The common shape in a scraped novel: the source wraps a paragraph across
// two lines. Splitting there would truncate the first half into its own
// paragraph and indent a continuation.
assert.deepEqual(splitParagraphs('这句被硬换行了\n但仍然是同一段'), ['这句被硬换行了但仍然是同一段']);
});
test('a line-ending sentence starts a new paragraph without a blank line', () => {
// The other common shape: every paragraph on its own line, no blank lines at
// all. Without this rule such a file renders as one slab — which is exactly
// the bug this rendition exists to fix.
assert.deepEqual(
splitParagraphs('他走了。\n她留下了。\n天亮了。'),
['他走了。', '她留下了。', '天亮了。'],
);
});
test('does not split a continuation line that merely starts with punctuation', () => {
// The lookahead requires the next line to start a sentence. A line beginning
// with a comma, a closing quote or an ellipsis is a continuation of the one
// above and must stay joined.
assert.deepEqual(splitParagraphs('他走了。\n,她没有回头。'), ['他走了。,她没有回头。']);
assert.deepEqual(splitParagraphs('他走了。\n」她说。'), ['他走了。」她说。']);
});
test('handles CRLF and lone CR line endings', () => {
// A lone CR is a line ending, not a character inside a word: treated as one,
// a file off a serial console or an old conversion is a single paragraph.
assert.deepEqual(splitParagraphs('一段\r\n\r\n二段'), ['一段', '二段']);
assert.deepEqual(splitParagraphs('他走了。\r她留下了。'), ['他走了。', '她留下了。']);
});
test('drops empty paragraphs rather than rendering blank blocks', () => {
assert.deepEqual(splitParagraphs('\n\n\n\n一段\n\n\n'), ['一段']);
assert.deepEqual(splitParagraphs(' \n\n '), []);
});
test('an empty body produces no paragraphs', () => {
assert.deepEqual(splitParagraphs(''), []);
});
});
describe('escapeHtml', () => {
test('escapes the characters that would otherwise rewrite the page', () => {
assert.equal(escapeHtml('<script>alert(1)</script>'), '&lt;script&gt;alert(1)&lt;/script&gt;');
assert.equal(escapeHtml('a & b'), 'a &amp; b');
assert.equal(escapeHtml(`"'`), '&quot;&#39;');
});
});
describe('renderChapterHtml', () => {
test('wraps paragraphs in a .txt-body div so the client can scope its rules', () => {
const html = renderChapterHtml('第一段\n\n第二段').toString('utf8');
assert.match(html, /^<div class="txt-body">\n/);
assert.equal(html.match(/<p>/g)?.length, 2);
assert.match(html, /<p>第一段<\/p>/);
assert.match(html, /<p>第二段<\/p>/);
});
test('carries no typography, so the readers own settings are what apply', () => {
// The indent, the paragraph spacing and the removal of a scraper's leading
// spaces are per-device reading preferences, and a server that baked one of
// them into the payload is what made them a round trip — and what made a book
// read windowed look different from the same book read whole. The assertion is
// on the absence, because absence is the contract.
const html = renderChapterHtml('一段').toString('utf8');
assert.doesNotMatch(html, /data-indent/);
assert.doesNotMatch(html, /style=/);
assert.doesNotMatch(html, /text-indent/);
});
test('a file containing markup cannot inject elements', () => {
const html = renderChapterHtml(['<img src=x onerror=alert(1)>', '', '<b>粗</b>'].join('\n')).toString('utf8');
// The characters survive as *text*; what must not survive is an element or an
// attribute, so the assertion is on the markup rather than on the words — an
// escaped `onerror` is a paragraph, not a handler.
assert.doesNotMatch(html, /<img/);
assert.doesNotMatch(html, /<b>/);
assert.match(html, /&lt;img src=x onerror=alert\(1\)&gt;/);
assert.match(html, /&lt;b&gt;粗&lt;\/b&gt;/);
});
test('a chapter is rendered whole, with no truncation', () => {
// `chapter:` sliced the body at 256KB, which cut a long chapter in half.
// Stated as a test so the cap cannot come back by accident.
// Comfortably past the 256KB *byte* budget the plain-text path used: Chinese
// is three bytes per character, so the character count that overflowed it is
// under 90000. A chapter this size is unusual but real — web novels ship them.
const body = Array.from({ length: 40_000 }, (_, i) => `第 ${i} 段的内容。`).join('\n');
assert.ok(body.length > 256 * 1024 / 3, 'fixture must exceed the old character budget');
const html = renderChapterHtml(body).toString('utf8');
assert.match(html, /<p>第 39999 段的内容。<\/p>/);
});
test('promotes the chapter heading instead of gluing it to the first sentence', () => {
// `splitChapters` puts the heading in the chapter's own slice, so without this
// the body reads `第二章 落雨雨来了。` — the title and the first line as one
// paragraph, which is what a reader reports as "格式乱了".
const html = renderChapterHtml(['第二章 落雨', '雨来了。'].join('\n')).toString('utf8');
assert.match(html, /<h3>第二章 落雨<\/h3>/);
assert.match(html, /<p>雨来了。<\/p>/);
assert.doesNotMatch(html, /落雨雨来了/);
});
test('leaves prose that merely looks like a heading where it is', () => {
// Only the first line is considered, and only when it is short enough to be a
// title: a mid-chapter line that mentions a chapter number is a sentence.
const html = renderChapterHtml(['雨来了。', '他说第二章他看过。'].join('\n')).toString('utf8');
assert.doesNotMatch(html, /<h3>/);
assert.match(html, /<p>他说第二章他看过。<\/p>/);
});
test('returns utf-8 bytes whose length matches the payload it reports', () => {
const buffer = renderChapterHtml('中文内容\n\n第二段');
assert.equal(buffer.toString('utf8').length <= buffer.byteLength, true);
assert.equal(buffer.byteLength, Buffer.byteLength(buffer.toString('utf8'), 'utf8'));
});
});
+20
View File
@@ -0,0 +1,20 @@
import assert from 'node:assert/strict';
import { test, describe } from 'node:test';
import { escapeHtml } from '../src/indexer/formats/text-html.ts';
/**
* The escaping helper is all that is left of the server's TXT rendition.
*
* The paragraph split that used to be tested here moved to the client
* (`web/src/formats/segments.ts`, tested in `web/test/segments.test.ts`), because
* deciding where a paragraph is is a *reading* decision and not a property of the
* file. What is left on the server is the one mechanical thing: making a chapter's
* characters safe to put in a page, for the paths that still need it.
*/
describe('escapeHtml', () => {
test('escapes the characters that would otherwise rewrite the page', () => {
assert.equal(escapeHtml('<script>alert(1)</script>'), '&lt;script&gt;alert(1)&lt;/script&gt;');
assert.equal(escapeHtml('a & b'), 'a &amp; b');
assert.equal(escapeHtml(`"'`), '&quot;&#39;');
});
});
+73 -8
View File
@@ -1,5 +1,6 @@
import { ApiError } from '../api/errors.ts';
import type { ReaderApi, NoteInput } from '../api/client.ts';
import type { SyncPull } from '../api/types.ts';
import { isOutboxEmpty, type OfflineStore } from '../store/offline.ts';
import type { Connectivity, Platform } from './platform.ts';
@@ -93,7 +94,24 @@ export class SyncEngine {
for (const listener of this.listeners) listener(status);
}
/**
* Publishes a state change, and *only* a change.
*
* The guard is the fix for a re-render storm. The reader screen's status
* listener repaints the whole chrome tree on every notification, and a sync
* cycle announces itself twice — `syncing` on the way in, `idle` on the way out —
* so a reader flipping through a book repainted that tree three times per cycle
* (the explicit `emit()` after `setState('idle')` was a third) whether or not
* anything had actually changed. `syncNow` also runs on a 30s poll, so the storm
* continued while the reader was doing nothing at all.
*
* A state or message that is already published carries no new information, and
* the readers of this stream are *painters*: telling them again is pure cost.
* Deduplicating here rather than at each listener means every listener gets the
* same guarantee, and a listener added later cannot reintroduce the storm.
*/
private setState(state: SyncState, message = ''): void {
if (this.state === state && this.message === message) return;
this.state = state;
this.message = message;
this.emit();
@@ -156,7 +174,21 @@ export class SyncEngine {
return this.requested !== null;
}
private scheduleNext(delay: number): void {
/**
* Arms the next automatic poll.
*
* A timer that is already armed is *kept* rather than re-armed, and the
* difference is a real one: `syncNow` runs on demand (every `schedule()` push)
* as well as on the poll, and resetting the timer on each of those meant the
* poll was pushed out by every page turn. On a book being read, the 30-second
* poll then never fired at all — the pull half of sync stopped happening while
* the push half ran constantly, which is the asymmetry a reader sees as "怎么还
* 在一直请求" and as a shelf that never picks up another device's progress.
*/
private scheduleNext(delay: number, force = false): void {
// A backoff retry *replaces* the poll rather than waiting behind it: the poll
// is 30 seconds away, and a failure that retries in 2 has to be allowed to.
if (this.timer && !force) return;
if (this.timer) clearTimeout(this.timer);
this.timer = setTimeout(() => {
this.timer = null;
@@ -172,11 +204,24 @@ export class SyncEngine {
* the same time.
*/
async syncNow(): Promise<void> {
if (this.running) return;
if (this.running) {
// A run is already in flight and will arm the next poll itself. Re-arming
// here would be a second timer, and the loser would fire a duplicate cycle.
return;
}
if (!this.api.currentSession()) {
this.setState('signed-out');
return;
}
// A push that has been asked for and not yet run is *this* work, a few
// milliseconds early. Letting the poll run first would send the same outbox
// twice and pull twice, and the second answer would be identical to the first
// — which is exactly the "sync 接口在不停重复" a reader sees in the network
// panel. The pending push will do the job; the poll re-arms itself below.
if (this.requested) {
this.scheduleNext(IDLE_INTERVAL_MS);
return;
}
this.running = true;
this.setState('syncing');
try {
@@ -187,8 +232,19 @@ export class SyncEngine {
return;
}
await this.pushOutbox();
const pull = await this.api.pull(this.offline.current.serverTime);
// One round trip, not two.
//
// `POST /api/v1/sync` answers with the merged state — the server's own
// comment says so: "Returning the merged state lets the client reconcile in
// the same round trip instead of issuing a follow-up pull." A push used to
// throw that answer away and then issue a *second* request for it, so every
// page turn cost a POST and a GET carrying the same body. Taking the push's
// answer is what makes a page turn cost one request.
//
// A push that had nothing to send (an idle poll) returns null, and then the
// pull is the real work — that is the poll's whole purpose.
const pushed = await this.pushOutbox();
const pull = pushed ?? (await this.api.pull(this.offline.current.serverTime));
await this.offline.applyPulled(pull);
this.rememberNoteBooks();
@@ -196,7 +252,6 @@ export class SyncEngine {
// A rejection message has to survive the state transition to idle,
// otherwise the reader never learns that a record was dropped.
this.setState('idle', this.rejectedCount > 0 ? `${this.rejectedCount} 条记录被服务端拒绝` : this.message);
this.emit();
this.scheduleNext(IDLE_INTERVAL_MS);
} catch (err) {
this.handleFailure(err);
@@ -213,7 +268,7 @@ export class SyncEngine {
* the record's own update will still be dirty after this returns, so the next
* cycle picks it up instead of it being cleared by a response that predates it.
*/
private async pushOutbox(): Promise<void> {
private async pushOutbox(): Promise<SyncPull | null> {
const batch = this.offline.outbox();
const deletes: NoteInput[] = batch.deletes.map((id) => ({
id,
@@ -244,7 +299,8 @@ export class SyncEngine {
})),
...deletes,
];
if (batch.progress.length === 0 && notes.length === 0) return;
// Nothing to send: this is an idle poll, and the caller does the pull.
if (batch.progress.length === 0 && notes.length === 0) return null;
// The server caps a batch at 5000 records; chunk well below that so a long
// offline session cannot fail wholesale on one oversized request.
@@ -252,6 +308,10 @@ export class SyncEngine {
const batches = Math.max(1, Math.ceil(Math.max(batch.progress.length, notes.length) / CHUNK));
let serverTime = 0;
let rejected = 0;
// The last chunk's answer is the merged state as of the final write, which is
// the only one worth reconciling against: an earlier chunk's `progress` list
// predates the writes that came after it.
let merged: SyncPull | null = null;
for (let index = 0; index < batches; index += 1) {
const offset = index * CHUNK;
const progressChunk = batch.progress.slice(offset, offset + CHUNK);
@@ -262,6 +322,7 @@ export class SyncEngine {
});
serverTime = Math.max(serverTime, result.serverTime);
rejected += result.rejected;
merged = result;
}
await this.offline.markDelivered(batch, serverTime);
// Rejections are informational. The server refuses records it can never
@@ -270,6 +331,7 @@ export class SyncEngine {
// on the engine rather than in a transient message so the status bar can
// still show it after the state settles.
this.rejectedCount = rejected;
return merged;
}
/**
@@ -302,7 +364,10 @@ export class SyncEngine {
}
this.setState('error', err instanceof Error ? err.message : '同步失败');
this.backoff = Math.min(this.backoff * 2, RETRY_MAX_MS);
this.scheduleNext(this.backoff);
// `force`: an error's exponential backoff must be able to shorten a wait the
// poll has already armed, or the first retry after a blip would be 30 seconds
// late and the backoff would appear not to work.
this.scheduleNext(this.backoff, true);
}
/**
+59 -5
View File
@@ -190,17 +190,71 @@ export function escapeHtml(text: string): string {
.replace(/'/g, '&#39;');
}
/**
* A chapter heading, promoted out of the body.
*
* A chapter's own slice *includes* its heading line — chapter splitting records
* the heading as the chapter's first line, which is what makes a saved position
* land on the title rather than one line into the chapter. Left as ordinary prose
* it renders as `第二章 落雨雨来了。`: the title and the first sentence glued
* together with no separation, which is what a reader reports as "格式乱了".
*
* Promoted rather than merely split off, because it is the one piece of structure
* a TXT actually has, and a reader scanning for a chapter boundary is looking for
* exactly this. The patterns are deliberately narrow for the same reason the
* chapter *split* is: a false positive restructures the book rather than
* rendering it.
*/
const HEADING_PATTERNS: readonly RegExp[] = [
/^第\s*[0-90-9零一二三四五六七八九十百千万两]+\s*[章回节卷部篇集]\s*[::.、\-—]?\s*.{0,60}$/,
/^(?:序章|序言|楔子|引子|前言|后记|尾声|终章|番外|附录)\s*.{0,40}$/,
/^[卷部]\s*[0-90-9零一二三四五六七八九十百千万两]+\s*.{0,60}$/,
/^Chapter\s*\d+.*$/i,
/^#{1,3}\s+.+$/,
];
/** The longest line still plausibly a heading rather than prose. */
const MAX_HEADING_LENGTH = 60;
function isHeadingLine(line: string): boolean {
const trimmed = line.trim();
if (trimmed.length === 0 || trimmed.length > MAX_HEADING_LENGTH) return false;
return HEADING_PATTERNS.some((pattern) => pattern.test(trimmed));
}
/**
* Splits a chapter body into its heading line and the rest.
*
* Only the *first* non-blank line is considered: a heading anywhere else is a line
* of prose that happens to look like one, and promoting those would restructure
* the book rather than render it.
*/
export function splitChapterHeading(text: string): { heading: string; body: string } {
const normalised = text.replace(LINE_ENDINGS, '\n');
const lines = normalised.split('\n');
const firstIndex = lines.findIndex((line) => line.trim().length > 0);
if (firstIndex === -1) return { heading: '', body: '' };
if (!isHeadingLine(lines[firstIndex] ?? '')) return { heading: '', body: normalised };
return {
heading: (lines[firstIndex] ?? '').trim(),
body: lines.slice(firstIndex + 1).join('\n'),
};
}
/**
* The same rendition, wrapped so the client's scoping class is present.
*
* The string form used by the loaders, and the one thing it adds over
* `textToParagraphHtml` is the wrapper: a chapter's markup is the unit that
* The string form used by the loaders, and what it adds over
* `textToParagraphHtml` is the wrapper — a chapter's markup is the unit that
* reaches a shadow root, and the class has to travel with it rather than being
* added by whoever renders it — a server-rendered chapter carries it in its own
* response, so a locally loaded one must not be the odd one out.
* added by whoever renders it — plus the heading promotion, which is the same
* decision on every path (a locale's own file, the windowed manifest, the stream)
* rather than one the server made for two of them.
*/
export function textToChapterHtml(text: string): string {
return `<div class="txt-body">\n${textToParagraphHtml(text)}\n</div>\n`;
const { heading, body } = splitChapterHeading(text);
const promoted = heading ? `<h3>${escapeHtml(heading)}</h3>\n` : '';
return `<div class="txt-body">\n${promoted}${textToParagraphHtml(body)}\n</div>\n`;
}
/**
+13
View File
@@ -45,6 +45,19 @@ export interface Section {
* to express later without changing the contract.
*/
render?: RenderMode;
/**
* The section's body is plain characters to be typeset, not authored markup.
*
* Carried per section because the *manifest* is where the declaration lives (a
* content item's `format: 'html'`), and because inferring it from the body is a
* guess that fails in exactly the case this exists for: the server used to wrap a
* TXT chapter in `<div class="txt-body">`, and now it sends bare characters, so a
* marker-sniffing check answers "not plain text" for a chapter that is nothing
* *but* plain text — and the reader then draws a novel with no paragraphs, no
* indent and no stylesheet applied, which is a defect that looks like a typography
* preference rather than a bug.
*/
plainText?: boolean;
/**
* Path this section's bytes can be fetched from, when the section is one
* addressable file. This is what lets a native host fetch a page itself
+8
View File
@@ -95,6 +95,11 @@ export function createStagedDoc(options: StagedDocOptions): StagedDoc {
depth: 0,
render: item.kind === 'page' ? 'image' : 'reflowable',
path: item.href,
// The manifest's own declaration, not a guess from the body. A content item that
// says `format: 'html'` is saying "these are characters for you to typeset", and
// it is the only party that knows: the body no longer carries a marker to sniff
// (see `Section.plainText`).
...(item.format === 'html' ? { plainText: true } : {}),
}));
const base: BookDoc = {
@@ -157,6 +162,9 @@ export function createStagedDoc(options: StagedDocOptions): StagedDoc {
depth: 0,
render: item.kind === 'page' ? 'image' : 'reflowable',
path: item.href,
// Same declaration as above, and carried across a window swap for the same
// reason: a jump to chapter 900 must not turn the book into an EPUB.
...(item.format === 'html' ? { plainText: true } : {}),
});
}
return local;
+8 -3
View File
@@ -1671,9 +1671,14 @@ function toArrayBuffer(bytes: Uint8Array): ArrayBuffer {
* chapter one. Prefixing on the client keeps old positions resolving and new
* chapters rendering.
*
* `chapter-html:` is the server's markup rendition of `chapter:`. An item that does
* not declare `html` is fetched exactly as the server named it, which is the
* behaviour every other format already relies on.
* `chapter-html:` is `chapter:` without the streaming cap — one whole chapter for a
* reader who is about to read it in one piece. It used to mean "server-rendered
* markup" as well; that half is gone (the client typesets, see
* `web/src/formats/segments.ts`) and the reference name is kept because it is the
* identity a cached section and a saved position are keyed on, and because a
* client that asks for it must keep working. An item that does not declare `html`
* is fetched exactly as the server named it, which is the behaviour every other
* format already relies on.
*/
function renditionRef(item: { href: string; format?: string }): string {
if (item.format !== 'html') return item.href;
+77 -38
View File
@@ -9,7 +9,7 @@ import {
sanitiseInjectedContent,
type BookShadowHost,
} from './shadow.ts';
import { escapeHtml, textToParagraphHtml } from '../formats/segments.ts';
import { escapeHtml, textToChapterHtml } from '../formats/segments.ts';
import { bookPercentage, formatLocator, parseLocator } from './locator.ts';
import { collectSpokenChunks, type SpokenChunk } from '../render/tts-text.ts';
import { adoptWindow } from '../formats/windowed.ts';
@@ -525,63 +525,102 @@ export class ReaderView {
* indent control beside them.
*/
isPlainText(section: Section = this.doc.sections[this.sectionIndex] ?? EMPTY_SECTION): boolean {
if (this.doc.format === 'txt') return true;
// The declaration wins outright. `format: 'txt'` is the book saying so and
// `plainText` is the *section* saying so, which is the one that can be right when
// a manifest windows a TXT as `reflowable` — a reasonable choice, since a TXT is
// reflowable, and one that made the chapter-level answer differ from the
// book-level one.
if (this.doc.format === 'txt' || section.plainText) return true;
// The marker is kept as the last resort, for a section that reached here from an
// older server (or a cache written by one) which *did* wrap its chapters in
// `<div class="txt-body">`. It is deliberately no longer the primary signal: a
// server that sends bare characters carries no marker, so sniffing for one
// answers "not plain text" for a chapter that is nothing but plain text.
const html = section.html ?? '';
return /<div[^>]*class="[^"]*\btxt-body\b/.test(html);
}
/**
* Re-renders a chapter of a plain-text book through the reader's own split.
* Renders a chapter of a plain-text book through the reader's own split.
*
* Two sources reach this method and they need the same treatment:
* The input is the *characters* of the chapter, however they arrived:
*
* - the server's `chapter-html:<n>` rendition (see `text-html.ts`), which is
* that book's characters as markup; and
* - a `chapter:<n>` body fetched by the windowed path, which is the book's
* characters and nothing else.
* - a `chapter-html:<n>` body fetched by the windowed path, which is that
* book's text and nothing else (the server stopped typesetting — see
* `server/src/indexer/formats/text-html.ts`); or
* - a body from a server that still renders markup, which is handled by
* reading its blocks back as text rather than by trusting its elements.
*
* Both end up here because *both* are the reader's own rendering of a file that
* has none, and the thing the reader asked for — a per-device indent, paragraph
* spacing, the removal of a scraper's leading spaces — is a property of the
* rendering, not of the file. Doing it server-side meant the settings could only
* apply to the rendition the server had already produced, and a book read
* windowed got no paragraphs at all; doing it here means one code path and no
* request, and the server's job shrinks to handing over the text.
* apply to the rendition the server had already produced, and it meant the two
* transports could disagree about where a paragraph was.
*
* Guarded by `isPlainText`, and the guard is why an EPUB is safe: an ordinary
* novel never produces a `txt-body` wrapper, so it goes through untouched (the
* novel never carries the `txt-body` marker, so it goes through untouched (the
* one thing in this file that must not happen is a re-typesetting pass that runs
* on every format, because that would mangle every EPUB).
*/
private retypePlainText(section: Section): string {
const raw = section.html ?? '';
if (!this.isPlainText(section)) return raw;
const container = document.createElement('div');
container.innerHTML = extractBody(raw);
const wrapper = container.querySelector('.txt-body');
if (!wrapper) return raw;
// A heading the server promoted out of the body is kept, and kept *first*:
// it is the one piece of structure a TXT has, and re-deriving it here would
// duplicate a heuristic the server already runs against the undecoded file.
const heading = wrapper.querySelector('h3, h4');
// The text is read back *per block*, and that is the whole job. Using
// `wrapper.textContent` would concatenate the paragraphs into one run with no
// separator, so a chapter that arrived already split would be re-split as a
// single paragraph and every boundary it had would be lost — the exact defect
// this method exists to remove, arriving from the other direction. Joining the
// blocks with a blank line puts back the separator the split reads as a
// boundary, and the boundaries *inside* a block are still inferred from its
// text, which is what makes this work for a chapter the server sent as one
// slab with no elements in it at all.
const blocks = [...wrapper.children]
.filter((element) => !/^H[1-6]$/.test(element.tagName))
.map((element) => element.textContent ?? '')
.filter((text) => text.trim().length > 0);
const body = blocks.length > 0
? blocks.join('\n\n')
: wrapper.textContent ?? '';
const promoted = heading ? `<h3>${escapeHtml(heading.textContent ?? '')}</h3>\n` : '';
return `<div class="txt-body">\n${promoted}${textToParagraphHtml(body)}\n</div>\n`;
// A `txt-body` wrapper means the body arrived as markup (this build no longer
// produces one, but a window carried over from an older server or a cached
// section might). Its blocks are read back *per block* and re-joined with a
// blank line: `textContent` on the wrapper would concatenate them into one
// run, and the re-split below would then see a single paragraph and destroy
// every boundary the block structure was carrying.
let text = raw;
// A heading an older server promoted out of the body. It is kept, and kept
// *first*, because it is the one piece of structure a TXT has and because
// re-deriving it is a heuristic — running a heuristic twice is how two copies of
// it come to disagree. Re-emitted below rather than fed back through the regex.
let keptHeading = '';
// Only a body that *is* markup is parsed as markup: a section the manifest
// declared as plain text is characters all the way down, and running an HTML
// parser over a novel whose text happens to contain `<` is how a paragraph gets
// silently eaten.
if (!section.plainText && /<div[^>]*class="[^"]*\btxt-body\b/.test(raw)) {
const container = document.createElement('div');
container.innerHTML = raw;
const wrapper = container.querySelector('.txt-body');
if (wrapper) {
const heading = wrapper.querySelector('h3, h4');
keptHeading = heading?.textContent?.trim() ?? '';
// Read back *per block*, not with `textContent` on the wrapper: the latter
// concatenates the paragraphs into one run, and the re-split below would
// then see a single paragraph and destroy every boundary the block
// structure was carrying.
const blocks = [...wrapper.children]
.filter((element) => !/^H[1-6]$/.test(element.tagName))
.map((element) => element.textContent ?? '')
.filter((block) => block.trim().length > 0);
text = blocks.length > 0 ? blocks.join('\n\n') : wrapper.textContent ?? '';
}
} else {
// Not markup: the chapter's characters, as fetched, and nothing else. This
// is what the server sends now (see `server/src/indexer/formats/text.ts`).
// `extractBody` is still applied because a plain-text body can be wrapped in
// an `<html>`/`<body>` shell by a proxy, and unwrapping that is cheaper than
// discovering it as a stray `<html>` inside the first paragraph.
text = extractBody(raw);
}
// The heading is re-derived from the text like every other boundary, by
// `segments.ts`, when the body did not already carry one: the chapter's first
// line is its title (that is what `splitChapters` records and why a locator
// lands on the title). A heading the body *did* carry is re-emitted directly
// rather than fed back through the regex — it was already promoted once, and
// a second pass would only be a chance to disagree with the first.
const rendered = textToChapterHtml(text);
if (keptHeading) {
return rendered.replace(
'<div class="txt-body">\n',
`<div class="txt-body">\n<h3>${escapeHtml(keptHeading)}</h3>\n`,
);
}
return rendered;
}
private async renderReflowable(section: Section): Promise<void> {
+56 -13
View File
@@ -514,15 +514,17 @@ describe('plain-text chapters are typeset by the reader', () => {
return host.shadow;
}
const serverChapter = [
'<div class="txt-body">',
'<h3>第二章 落雨</h3>',
'<p>  雨来了。</p>',
'<p>他走在雨里。</p>',
'</div>',
].join('\n');
/**
* A chapter exactly as the server now sends it: the characters, and nothing else.
*
* `chapter-html:<n>` used to answer with server-rendered `<p>` markup. It answers
* with the text now, because paragraph boundaries are a *reading* decision — see
* `server/src/indexer/formats/text.ts`. Every assertion below is written against
* this shape, because this is the shape the reader actually receives.
*/
const serverChapter = ['第二章 落雨', '  雨来了。', '他走在雨里。'].join('\n');
it('re-splits a serverside chapter and drops the leading spaces it carried', async () => {
it('typesets the text the server sends, and drops the leading spaces it carried', async () => {
// The whole point of moving the typography to the client: the server hands over
// characters, and one of the characters a scraper wrote is the full-width space
// that was standing in for an indent. Left in, it is indent added to whatever the
@@ -535,17 +537,58 @@ describe('plain-text chapters are typeset by the reader', () => {
expect(body).not.toBeNull();
const paragraphs = [...body!.querySelectorAll('p')].map((p) => p.textContent);
expect(paragraphs).toEqual(['雨来了。', '他走在雨里。']);
// The heading the server promoted is kept, and stays first.
// The chapter's first line is its title, and the *client* promotes it now —
// otherwise it renders as `第二章 落雨雨来了。`, the title and the first sentence
// glued together, which is what a reader reports as "格式乱了".
expect(body!.querySelector('h3')?.textContent).toBe('第二章 落雨');
});
it('escapes a chapter whose own text contains markup', async () => {
// The server sends `text/plain` now and for exactly this reason: served as HTML,
// a chapter containing `<b>` would be parsed as markup by anything that trusted
// the label, and the escaping that makes a TXT safe would be sidestepped. The
// assertion is on the *absence* of an element, not on the presence of the words.
const container = document.createElement('div');
document.body.append(container);
const view = make(container, txtDoc('<img src=x onerror=alert(1)>\n\n<b>粗</b>'));
await view.open(0, 0);
const shadow = shadowOf(container);
expect(shadow.querySelector('.txt-body img')).toBeNull();
expect(shadow.querySelector('.txt-body b')).toBeNull();
const paragraphs = [...shadow.querySelectorAll('.txt-body > p')].map((p) => p.textContent);
expect(paragraphs).toEqual(['<img src=x onerror=alert(1)>', '<b>粗</b>']);
});
it('re-typesets a chapter an older server sent as markup', async () => {
// A window may have been cached against a server that still rendered `<p>`s (or
// a proxy may re-wrap one). The blocks are read back as text and re-split, so an
// old response cannot be rendered as if it were a new one — and the heading it
// carried is kept rather than re-derived, because a heuristic run twice is two
// copies of the heuristic.
const container = document.createElement('div');
document.body.append(container);
const legacy = [
'<div class="txt-body">',
'<h3>第二章 落雨</h3>',
'<p>  雨来了。</p>',
'<p>他走在雨里。</p>',
'</div>',
].join('\n');
const view = make(container, txtDoc(legacy));
await view.open(0, 0);
const body = shadowOf(container).querySelector('.txt-body');
const paragraphs = [...body!.querySelectorAll('p')].map((p) => p.textContent);
expect(paragraphs).toEqual(['雨来了。', '他走在雨里。']);
expect(body!.querySelector('h3')?.textContent).toBe('第二章 落雨');
});
it('adds paragraphs to a chapter the server sent as one slab', async () => {
// A windowed TXT arrives through `chapter:<n>`, which is the characters with no
// markup at all. Without this the reader gets a wall of text while the same book
// read whole gets paragraphs — one book, two appearances, decided by transport.
// A TXT whose chapters have no blank lines between paragraphs is the common
// scraped shape. Without this the reader gets a wall of text — one book, two
// appearances, decided by how the file happened to be written.
const container = document.createElement('div');
document.body.append(container);
const view = make(container, txtDoc('<div class="txt-body">他走了。\n她留下了。\n天亮了。</div>'));
const view = make(container, txtDoc('他走了。\n她留下了。\n天亮了。'));
await view.open(0, 0);
const paragraphs = [...shadowOf(container).querySelectorAll('.txt-body > p')].map((p) => p.textContent);
expect(paragraphs).toEqual(['他走了。', '她留下了。', '天亮了。']);
+52
View File
@@ -4,6 +4,7 @@ import {
endsAtParagraphBreak,
escapeHtml,
paragraphElement,
splitChapterHeading,
splitTextParagraphs,
textToChapterHtml,
textToParagraphHtml,
@@ -86,6 +87,26 @@ describe('the markup form', () => {
expect(textToChapterHtml('一段')).toMatch(/^<div class="txt-body">\n<p>一段<\/p>\n<\/div>\n$/);
});
it('promotes the chapter title instead of gluing it to the first sentence', () => {
// Chapter splitting records the heading as the chapter's *first line* — that is
// what makes a saved position land on the title rather than one line in. Rendered
// as ordinary prose it reads `第二章 落雨雨来了。`: title and first sentence as one
// paragraph, which is what a reader reports as "格式乱了".
const html = textToChapterHtml(['第二章 落雨', '雨来了。'].join('\n'));
expect(html).toContain('<h3>第二章 落雨</h3>');
expect(html).toContain('<p>雨来了。</p>');
expect(html).not.toContain('落雨雨来了');
});
it('leaves prose that merely looks like a heading where it is', () => {
// Only the *first* line is considered, and only when it is short enough to be a
// title. A mid-chapter line that mentions a chapter number is a sentence, and
// promoting it would restructure the book rather than render it.
const html = textToChapterHtml(['雨来了。', '他说第二章他看过。'].join('\n'));
expect(html).not.toContain('<h3>');
expect(html).toContain('<p>他说第二章他看过。</p>');
});
it('uses textContent for the node form, so the DOM is the escaper', () => {
const element = paragraphElement({ text: '<b>不是标签</b>', leading: 0 });
expect(element.tagName).toBe('P');
@@ -94,6 +115,37 @@ describe('the markup form', () => {
});
});
describe('splitChapterHeading', () => {
it('splits the heading off the body and reports both', () => {
// The pair, not just the heading: the body is what the paragraph split then
// works on, and returning only the heading would leave the caller to find the
// boundary again — a second copy of the rule, which is how the server's copy and
// the client's came to disagree in the first place.
expect(splitChapterHeading(['第一章 起点', '正文。'].join('\n'))).toEqual({
heading: '第一章 起点',
body: '正文。',
});
});
it('skips leading blank lines before looking for the heading', () => {
expect(splitChapterHeading('\n\n第二章 落雨\n正文。').heading).toBe('第二章 落雨');
});
it('reports no heading for a body that has none', () => {
expect(splitChapterHeading('正文只有这一段。')).toEqual({
heading: '',
body: '正文只有这一段。',
});
});
it('does not promote a long line that merely starts with a chapter number', () => {
// The length cap is what keeps a paragraph that *starts* with `第三章` from being
// read as a title, which would move the real body's first line into a heading.
const prose = `第三章他记得很清楚,${'那天的雨下得很大,'.repeat(8)}`;
expect(splitChapterHeading(prose).heading).toBe('');
});
});
/**
* Streaming across chunk boundaries.
*
+15 -3
View File
@@ -231,20 +231,32 @@ describe('plain-text detection', () => {
const prose = new ReaderView({ container, doc: textDoc('epub', '<p>他说 div class="txt-body" 是纯文本用的</p>') });
expect(prose.isPlainText()).toBe(false);
// Neither is one with no markup at all.
// Neither is one with no markup at all — *unless* the section declares itself.
// And that declaration is the case the marker cannot cover: the server now sends
// a TXT chapter as bare characters with no wrapper at all, so a chapter that is
// nothing but plain text carries nothing to sniff. Sniffing for the marker
// answered "not plain text" and the reader drew a novel with no paragraphs, no
// indent and no stylesheet — a defect that reads as a typography preference
// rather than a bug.
const bare = new ReaderView({ container, doc: textDoc('epub', '<p>正文</p>') });
expect(bare.isPlainText()).toBe(false);
const declaredBare = new ReaderView({
container,
doc: textDoc('epub', '第一章 惊蛰\n\n第 1 段。', { plainText: true }),
});
expect(declaredBare.isPlainText()).toBe(true);
});
});
/** A one-section document, for the detection test. */
function textDoc(format: string, html: string) {
function textDoc(format: string, html: string, section: Record<string, unknown> = {}) {
return {
format,
layout: 'reflowable',
render: 'reflowable',
direction: 'ltr',
sections: [{ id: 'c0', label: '一', html, depth: 0 }],
sections: [{ id: 'c0', label: '一', html, depth: 0, ...section }],
toc: [],
styles: [],
resources: new Map(),
+59 -5
View File
@@ -293,15 +293,69 @@ describe('SyncEngine', () => {
});
await engine.syncNow();
expect(seen[0]).toBe('POST /api/v1/sync');
// The pull reuses the cursor the push just returned. That is intentional: a
// pull from 0 would re-download the entire server state after every page
// turn, which is exactly what the incremental cursor exists to avoid.
expect(seen[1]).toContain('since=5000');
// One request, not two.
//
// `POST /api/v1/sync` already answers with the merged state — the server's own
// comment says so — and the engine used to throw that answer away and then
// issue a second request for it. Every page turn therefore cost a POST *and* a
// GET carrying the same body, which is the doubling a reader sees in the
// network panel as "sync 接口在不停重复". Asserting the *length* is what makes
// the fix visible: a test that only checked the cursor would pass either way.
expect(seen).toEqual(['POST /api/v1/sync']);
expect(isOutboxEmpty(offline.current)).toBe(true);
// The cursor is still adopted from the push's answer, so the next real pull is
// incremental rather than a re-download of the entire server state.
expect(offline.current.serverTime).toBe(5000);
});
it('pulls with the incremental cursor when a poll has nothing to push', async () => {
// The other half of the same contract: with an empty outbox the push has no
// answer to reuse, so the pull is the real work and it must still happen — and
// still be incremental.
const { api, transport, offline, engine } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
const seen: string[] = [];
transport.respondWith((request) => {
seen.push(`${request.method} ${request.url}`);
return { status: 200, headers: {}, json: { serverTime: 9000, progress: [], notes: [] } };
});
await engine.syncNow();
expect(seen).toEqual(['GET /api/v1/sync?since=0']);
expect(offline.current.serverTime).toBe(9000);
await engine.syncNow();
expect(seen[1]).toBe('GET /api/v1/sync?since=9000');
});
it('does not notify its listeners twice for one unchanged state', async () => {
// The reported symptom: "sync 接口会不停的重复渲染". The reader screen repaints
// its whole chrome tree on every notification, and a cycle announced itself
// three times (syncing, idle, and a redundant explicit emit) whether or not
// anything had changed — with a 30s poll keeping the cycle going while the
// reader did nothing at all.
const { api, transport, offline, engine } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
transport.json({ serverTime: 10, progress: [], notes: [] });
const states: string[] = [];
engine.onStatus((status) => states.push(`${status.state}:${status.message}`));
await engine.syncNow();
expect(states).toEqual(['syncing:', 'idle:']);
// A second cycle that ends in the same state as the first adds nothing new, so
// it must add nothing to the stream either.
const before = states.length;
await engine.syncNow();
expect(states.slice(before)).toEqual(['syncing:', 'idle:']);
});
it('folds a burst of scheduled pushes into one round trip', async () => {
// The reported symptom: "sync 接口调用太过频繁". Every page turn scheduled its
// own round trip, so a reader flipping through a book produced a POST and a GET
+68
View File
@@ -24,6 +24,21 @@ function item(index: number): ContentItem {
};
}
/** A plain-text chapter, as the server's manifest declares one. */
function textItem(index: number): ContentItem {
return {
id: `c${index}`,
seq: index,
title: `第 ${index + 1} 章`,
kind: 'chapter',
// The declaration the reader typesets from. `format: 'html'` and a `text/plain`
// media type are the pair the server actually sends for a TXT.
mediaType: 'text/plain; charset=utf-8',
href: `chapter:${index}`,
format: 'html',
};
}
function windowOf(from: number, count: number, total: number): BookContent {
return {
kind: 'reflowable',
@@ -37,6 +52,59 @@ function windowOf(from: number, count: number, total: number): BookContent {
}
describe('staged documents', () => {
it('carries the manifest\'s plain-text declaration onto the section', async () => {
// The declaration has to reach the *section*, because that is what the reader
// asks when it decides whether to typeset. It cannot be re-derived from the body:
// the server sends a TXT chapter as bare characters with no wrapper, so a body
// that is nothing but plain text carries nothing to recognise it by — and a
// reader that answers "not plain text" draws the novel as an unstyled slab.
const doc = createStagedDoc({
kind: 'reflowable',
toc: [],
content: {
kind: 'reflowable',
total: 2,
groups: [{ id: 'g', seq: 0, title: '', count: 2, offset: 0 }],
items: [textItem(0), item(1)],
},
loader: { async read() { return { html: '第一章\n\n正文。' }; } },
});
expect(doc.sections[0]?.plainText).toBe(true);
// An EPUB chapter in the same window is not swept up by it: the decision is per
// section, and an EPUB's own markup must survive untouched.
expect(doc.sections[1]?.plainText).toBeUndefined();
});
it('keeps the declaration across a window swap', async () => {
// A jump to chapter 900 replaces every section. Losing the flag there would turn
// the rest of the book into an EPUB — paragraphs gone, stylesheet gone — and only
// after the reader had navigated, which is the hardest kind of defect to report.
const doc = createStagedDoc({
kind: 'reflowable',
toc: [],
content: {
kind: 'reflowable',
total: 4,
groups: [{ id: 'g', seq: 0, title: '', count: 2, offset: 0 }],
items: [textItem(0), textItem(1)],
group: 0,
},
loader: { async read() { return { html: '第一章\n\n正文。' }; } },
});
expect(doc.sections[0]?.plainText).toBe(true);
doc.setWindow({
kind: 'reflowable',
total: 4,
groups: [{ id: 'g', seq: 1, title: '', count: 2, offset: 2 }],
items: [textItem(2), textItem(3)],
group: 1,
}, 2);
expect(doc.sections[0]?.plainText).toBe(true);
});
it('does not read a section until it is asked for', async () => {
const read: string[] = [];
const doc = createStagedDoc({
+45
View File
@@ -156,6 +156,20 @@ export class CDP {
return result.result.value;
}
/**
* How many requests the stand-in API has answered, by name.
*
* Read from the page rather than from Node, because that is where the review's own
* origin is: the harness server logs every request it sees, and the page is the
* only party that can reach it without the tool growing its own HTTP client. The
* endpoint returns *counts* rather than the log so the assertion is a comparison of
* two numbers instead of a scan of a growing array.
*/
async requestCounts() {
const counts = await this.evaluate(`fetch('/__counts').then((r) => r.json())`);
return counts && typeof counts === 'object' ? counts : {};
}
/**
* Runs a statement body in the page, with its own `return`.
*
@@ -267,6 +281,37 @@ export class CDP {
await this.send('Input.insertText', { text: value });
}
/**
* A horizontal swipe across the middle of the reading area: a page turn.
*
* Dispatched as *touch* events rather than a click, because that is what the
* gesture layer listens for and what a page turn in this app actually is. The
* intermediate moves matter: a single `touchStart`/`touchEnd` pair is a tap, and
* the gesture layer decides between the two from the distance travelled.
*/
async swipePage(direction = 'next') {
const box = await this.execute(`(() => {
const el = document.querySelector('.stage');
if (!el) return null;
const r = el.getBoundingClientRect();
return { x: r.left + r.width / 2, y: r.top + r.height / 2, width: r.width };
})()`);
if (!box) throw new Error('no .stage to swipe');
const travel = box.width * 0.35 * (direction === 'next' ? -1 : 1);
const steps = 6;
const points = (offset) => [{ x: box.x + offset, y: box.y }];
await this.send('Input.dispatchTouchEvent', { type: 'touchStart', touchPoints: points(0) });
for (let step = 1; step <= steps; step += 1) {
await this.send('Input.dispatchTouchEvent', {
type: 'touchMove',
touchPoints: points((travel * step) / steps),
});
await sleep(16);
}
await this.send('Input.dispatchTouchEvent', { type: 'touchEnd', touchPoints: [] });
await sleep(160);
}
/** A tap in the middle of the reading area, which is the chrome toggle. */
async tapMiddle() {
const box = await this.execute(`(() => {
+112
View File
@@ -288,6 +288,118 @@ async function audit(cdp, scenes, results) {
})()`);
check('没有横向溢出', overflow.document <= 1, `document 溢出 ${overflow.document}px`);
// Back to scroll mode before the traffic window.
//
// The scenes above end in paged mode, and a paged surface does not scroll — reading
// in paged mode turns pages without generating a position per screen. Scroll mode is
// where a reader produces a position *continuously*, which is the traffic profile
// the report was about, so the window has to be measured there. The sheet is opened
// by its own button rather than assumed to be open: the previous step closed it.
// The chrome is made visible first: the audit's own collapse check above left it
// hidden, and a hidden topbar's settings button is a button that cannot be clicked.
const revealChrome = async () => {
const hidden = await cdp.evaluate(`document.querySelector('.reader-screen')?.dataset?.chrome === 'hidden'`);
if (hidden) {
await cdp.tapMiddle();
await cdp.sleep(400);
}
};
await revealChrome();
// The settings sheet is identified by its own title, not by "a panel is open": the
// geometry step above leaves the *contents* sheet up, and a blind click on the
// settings button would close it and then wait for a sheet that is already there.
const settingsTitle = await cdp.evaluate(
`document.querySelector('.panel')?.querySelector('.panel-title, h2, header')?.textContent?.trim() ?? ''`,
);
if (!settingsTitle.includes('设置')) {
if (settingsTitle) {
await cdp.click('button[aria-label="关闭"]');
await cdp.sleep(300);
}
await cdp.click('button[aria-label="设置"]');
await cdp.waitFor('document.querySelector(".panel") !== null', 10_000);
}
await cdp.sleep(300);
await cdp.clickText('.segmented button', '滚动');
await cdp.sleep(300);
await cdp.click('button[aria-label="关闭"]');
await cdp.sleep(600);
// The traffic a *reading* session generates.
//
// The request behind this is the screenshot a reader sent: the network panel, full
// of `sync` POSTs each paired with a `sync?since=…` GET, repeating for as long as
// they kept reading. Both halves were real defects — the push's answer (which
// already carries the merged state) was thrown away and re-fetched, and the 30s
// poll was re-armed by every page turn so the pull half silently stopped — and
// neither is a *functional* failure, which is why no functional test caught them.
//
// So the window has to contain actual reading: turning pages is what writes a
// position and what used to produce a round trip per turn. A window in which the
// reader sits still proves nothing about the thing that was reported.
await cdp.sleep(1500);
const before = await cdp.requestCounts();
// Twelve page turns over six seconds — a reader settling into a book.
//
// The scroll is dispatched as a real sequence of positions on the reading surface
// rather than as a synthetic event: the surface is a scroll container in scroll
// mode (that is what "scroll mode" *is*), and its own `scroll` handler is what
// measures a position and writes it. Dispatching anything else would be a window in
// which no position was ever written, and such a window passes whatever the sync
// code does — which is the trap this check has to avoid.
// Driven one step at a time from here rather than as one long in-page loop: the
// protocol has a 20-second ceiling on a single evaluation, and twelve steps held
// long enough for the reader's own write debounce to fire is longer than that.
const pageTurns = 8;
const scrollMax = await cdp.evaluate(`(() => {
const host = document.querySelector('book-content');
if (!host) return 0;
return Math.max(0, host.scrollHeight - host.clientHeight);
})()`);
let turned = 0;
for (let step = 1; step <= pageTurns && scrollMax > 0; step += 1) {
await cdp.evaluate(`(() => {
const host = document.querySelector('book-content');
host.scrollTop = ${Math.round((scrollMax * step) / pageTurns)};
return host.scrollTop;
})()`);
// Held longer than the reader's own 1.5s write debounce, so every step is a
// *committed* position rather than one the next step cancels. Without that the
// window contains a single write however many times the surface is scrolled, and
// the check has nothing to measure.
await cdp.sleep(1800);
turned += 1;
}
// Then let the debounce and the coalescing window drain, so the trailing push a
// reader would actually cause is *in* the measurement rather than after it.
await cdp.sleep(6000);
const after = await cdp.requestCounts();
const delta = (name) => (after[name] ?? 0) - (before[name] ?? 0);
// Every write costs **one** request, not two.
//
// This is the doubling the report showed: a `POST /api/v1/sync` immediately
// followed by a `GET /api/v1/sync?since=…` carrying the same body. The push already
// answers with the merged state, so the GET is entirely redundant — and it doubles
// the traffic of every page turn for as long as the reader keeps reading.
//
// The assertion is on GETs specifically rather than on the total: how many *writes*
// a session produces is a product decision that belongs to the coalescing window,
// but a GET that appears *alongside* a POST is never right, whatever the reader did.
// Every step above is held past the reader's own write debounce, so each is a
// separate session and each legitimately pushes; none of them may also pull.
const syncTotal = delta('sync');
const posts = delta('sync-post');
const gets = delta('sync-get');
check(
'翻页不再每次都发两次 sync',
turned > 0 && posts > 0 && gets === 0,
turned === 0
? '阅读面没有可滚动的高度,这一轮没有发生翻页(检查因此无效)'
: posts === 0
? '这一轮没有发出任何 push,检查无效'
: `${turned} 次翻页 + 6 秒静默:${posts} 次 POST / ${gets} 次 GET(共 ${syncTotal} 次 sync 请求)`,
);
return { failures, results };
}
+50 -13
View File
@@ -110,15 +110,21 @@ const book = {
updatedAt: Date.now(),
};
/** The reader's own chapter markup, in the shape the server produces. */
function chapterHtml(index) {
const paragraphs = CHAPTER_BODY[index]
.split('\n\n')
.map((block) => block.trim())
.filter(Boolean)
.map((block) => `<p>${block.replace(/&/g, '&amp;').replace(/</g, '&lt;')}</p>`)
.join('\n');
return `<div class="txt-body">\n${paragraphs}\n</div>\n`;
/**
* A chapter, in the shape the server actually produces: the characters, as text.
*
* This used to answer with a `txt-body` div full of `<p>`s, which is what the server
* rendered before the typesetting moved to the client. Keeping the fixture on the
* *new* shape matters more than keeping it on the old one: the review runs against
* the production bundle, and a fixture that still pre-rendered paragraphs would let
* a regression in the client's own split render perfectly.
*
* The `\u3000` indents in `LONG` are kept deliberately — they are "段前空格", and
* left in they are indent stacked on the reader's own. The screenshot is where that
* is visible.
*/
function chapterText(index) {
return `${CHAPTER_BODY[index]}\n`;
}
function manifest() {
@@ -143,8 +149,15 @@ function manifest() {
seq: index,
title: chapter.title,
kind: 'chapter',
mediaType: 'application/xhtml+xml',
// The TXT manifest as the server actually emits it: plain characters, and
// `format: 'html'` to say "this is a document to be typeset" rather than
// "this is markup". The media type and the format field are both asserted
// here because they are what `renditionRef` keys on — a fixture that omitted
// them would have the client fetch `chapter:` and render an untitled slab,
// which is a review of a path no reader is on.
mediaType: 'text/plain; charset=utf-8',
href: `chapter:${index}`,
format: 'html',
})),
},
};
@@ -174,6 +187,28 @@ export function createReviewServer({ port = 5199 } = {}) {
if (path === '/__seen') {
return json(reply, seen);
}
// The request log, *counted*, in the names the traffic assertions use.
//
// Counting at the HTTP boundary is the only place that sees every request the
// bundle makes — including the ones a debounce was supposed to fold away — and
// it is deliberately blind to *why* a request was made. A review of the traffic
// has to be: the defects it is looking for are a request that was redundant, and
// no amount of reading the client's intent tells you whether it was sent.
if (path === '/__counts') {
const counts = { sync: 0, 'sync-post': 0, 'sync-get': 0, assets: 0, toc: 0 };
for (const entry of seen) {
const [method, target = ''] = entry.split(' ');
const bare = target.split('?')[0] ?? '';
if (bare === '/api/v1/sync') {
counts.sync += 1;
if (method === 'POST') counts['sync-post'] += 1;
else counts['sync-get'] += 1;
}
if (/\/assets$/.test(bare)) counts.assets += 1;
if (/\/toc$/.test(bare)) counts.toc += 1;
}
return json(reply, counts);
}
if (path.startsWith('/api/')) {
return api(request, reply, url);
}
@@ -250,8 +285,10 @@ export function createReviewServer({ port = 5199 } = {}) {
if (path === `/api/v1/books/${BOOK_ID}/assets`) {
const ref = url.searchParams.get('ref') ?? '';
const index = Number.parseInt(ref.replace(/^chapter(-html)?:/, ''), 10);
const body = chapterHtml(Number.isFinite(index) ? index : 0);
reply.writeHead(200, { 'content-type': 'text/html; charset=utf-8' });
const body = chapterText(Number.isFinite(index) ? index : 0);
// `text/plain`, not `text/html`: the server stopped rendering markup, and the
// content type is part of what the client is being reviewed against.
reply.writeHead(200, { 'content-type': 'text/plain; charset=utf-8' });
return reply.end(body);
}
if (path === `/api/v1/books/${BOOK_ID}/progress`) {
@@ -285,4 +322,4 @@ export function createReviewServer({ port = 5199 } = {}) {
};
}
export { CHAPTERS, BOOK_ID, CHAPTER_BODY, chapterHtml };
export { CHAPTERS, BOOK_ID, CHAPTER_BODY, chapterText };