diff --git a/.cnb.yml b/.cnb.yml index 05b2ac6..89f092a 100644 --- a/.cnb.yml +++ b/.cnb.yml @@ -55,12 +55,17 @@ main: # 先跑一个不依赖 JDK 的静态检查（秒级），再编译。原因是桥接契约的两半分别 # 用 Kotlin 和 TypeScript 写，互相没法类型检查：客户端调用一个壳里没有的 # 方法是最容易出、也最难在本地发现的错误。静态检查把它提前到这里。 + # + # 触发条件包含 `web/src/formats/types.ts`：壳里分支的渲染模式词表来自它， + # 改一个词就会让整条原生渲染通道静默失效，而 Kotlin 编译器不会知道。 - name: android-ci docker: image: docker.cnb.cool/dockerhub/androidsdk/android-35:latest ifModify: - android/** - web/src/android-bridge.d.ts + - web/src/formats/types.ts + - web/src/ui/native-page.ts stages: - name: shell-contract script: sh android/scripts/check-shell.sh @@ -70,6 +75,16 @@ main: - name: report-failure script: echo "Android 构建失败：$CNB_BUILD_FAILED_MSG" + # 客户端产物必须能被壳打包。这条单独跑，因为 `client-ci` 只证明 H5 能构建， + # 不证明 assets 里有东西——而一个空的 assets 目录会产出一个启动即白屏的 APK。 + - name: web-assets-ci + <<: *node-env + ifModify: + - web/** + stages: + - name: stage-for-shell + script: sh android/scripts/build-web-assets.sh + "**": pull_request: - name: server-checks diff --git a/README.md b/README.md index ab9e562..2d2d7ce 100644 --- a/README.md +++ b/README.md @@ -73,8 +73,10 @@ volumes: | PDF | 保底可读 | 用浏览器自带阅读器，页数与进度照常上报，服务端不解析内容 | > TXT 是本地书库里最容易被忽视的一块。相当一部分中文 TXT 没有 BOM、没有编码声明， -> 解码成 UTF-8 会满屏乱码。客户端先严格校验 UTF-8，失败再依次尝试 GB18030 与 Big5， -> 并**在界面上告诉你选了哪个编码**，可手动覆盖。 +> 解码成 UTF-8 会满屏乱码。客户端先严格校验 UTF-8；失败时**不是**按顺序试 +> GB18030 再试 Big5，而是两种都解一遍挑更干净的那个——这两个编码的字节范围重叠， +> 同一串字节往往两边都能解出字符，按顺序试就永远走不到第二个，繁体书会整本变乱码。 +> 选中的编码会**显示在界面上**，可手动覆盖。 > 章节同样没有结构，只能从标题推断——推测不出时按长度分段，不会整本卡死。 --- @@ -129,20 +131,24 @@ volumes: ## 阅读体验：一次只取一章 -打开一本书是**一次请求**（`/manifest` 自带可寻址结构），翻到下一章才取下一章， -再下一章在后台预取。一本 1200 章的全集打开时只下载一章的字节。 +打开一本书是**一次请求**。服务端的 `manifest` 默认只带**第一个窗口**的条目， +不是整本书的清单——一本 1200 章的全集打开时只传一个窗口，翻到下一章才取下一章。 - **分窗**：`/items?group=N` 一次返回一个窗口（epub 是 40 章，漫画是一卷）。 - 每个 group 自带 `offset`，所以客户端拿到单窗也能定位到全书位置 —— - 不需要把前面的 count 加一遍，也就不会有加错导致的跳章。 -- **目录是目录**：`/toc` 按格式返回真正的目录（epub 的 spine 标题、txt 的章节、 - 漫画的卷）。它和分窗是两件事：分窗是传输边界，目录要完整才有用。 - 把它们合成一个接口会让目录显示成「第 1 章 – 第 40 章」。 -- **流式下发**：章节、页图、PDF 都是边读边发。EPUB 只读 zip 中央目录并按需解压； - 取一页漫画不需要把整个压缩包读进内存。 -- **HTTP Range**：PDF 跳页、漫画跳卷、断点续传靠它。压缩方式为 `stored` 的 - 条目可字节定位，`deflate` 的会诚实上报 `Accept-Ranges: none`，让客户端知道 - 而不是白跑一次。 + 每个 group 自带 `offset`，所以客户端拿到单窗也能定位到全书位置。 + **省略 `group` 等于 `group=0`**；要完整结构用 `?group=all`。 +- **目录是目录**：`/toc` 返回真正的完整目录。它和分窗是两件事：分窗是传输边界， + 目录必须完整才有用。把它们合成一个接口，目录就会显示成「第 1 章 – 第 40 章」 + ——服务端的分页边界漏进 UI。 +- **章节按路径寻址**：`items[].href`（`xhtml:OEBPS/ch1.xhtml`）是章节身份。 + 不按下标寻址：分窗返回的是 spine 前缀，按下标算出的章节在完整 spine 里指向另一章。 +- **HTTP Range**：PDF 跳页、漫画跳卷、断点续传靠它。压缩方式为 `stored` 的条目 + 可字节定位；`deflate` 的会诚实上报 `Accept-Ranges: none`。 + 尾部范围（`bytes=-N`）无法靠前向流跳过，大文件时会退化为整包响应而不是空读全文件。 +- **目录跳转跨窗口**：点第 900 章会先取窗口 22，而不是「什么都没发生」。 + 章节 id 是唯一跨窗口稳定的身份，进度也存它。 + +--- ## 客户端三态 @@ -168,6 +174,12 @@ Android 端**不重写渲染**。它是一个 Kotlin 原生外壳，只负责文 - **稳定的设备名**：UA 里的型号会随 Chrome 升级变化，作为「上次在哪个设备读的」标签不可靠。 - **原生提示**：一行 Kotlin 的 Toast，不用在 Web 层造一套通知 UI。 +还有一条刻意的例外：**固定版式的整页图由原生画**。一页漫画或一张扫描图在 WebView 里 +要走完解码、样式布局、合成三次开销，而屏幕上的结果和 `ImageView` 一模一样； +对一章文字这些都是必要的，对「把这张图画满屏」则全是白付的。 +规则是「只画整页图，且全有或全无」——画不了就交给 WebView，绝不出现空白页。 +EPUB 永远留在 WebView：文字排版是差异点，只能有一份实现。 + ### 离线合并语义 进度和笔记都带 `updatedAt`，冲突按 **last-writer-wins** 解决： @@ -262,13 +274,11 @@ cd android ## 从源码开发 -两个包，互不依赖构建顺序： - ```bash # 服务端 cd server npm install -npm test # 114 个测试用例 +npm test # 133 个测试用例 npm run typecheck npm run dev # 开发模式，热重载 @@ -280,7 +290,7 @@ BOOKS_DIR=/tmp/books DATA_DIR=/tmp/data npm run dev # 客户端（另开一个终端） cd web npm install -npm test # 147 个测试用例 +npm test # 208 个测试用例（179 vitest + 29 node:test） npm run dev # http://localhost:5174，自动把 /api 代理到 8080 npm run build # 产出 web/dist，服务端会在 / 上直接托管 ``` @@ -294,19 +304,8 @@ server/src/ indexer/ identity.ts 书籍主键：dc:identifier + 内容哈希 filename.ts 文件名解析（保守策略，宁可留空不猜错） - metadata.ts EPUB 元数据与封面提取 + metadata.ts EPUB/PDF 元数据与封面提取 scanner.ts 增量扫描、变更检测、清理 - formats/ 格式处理器注册表（新增格式只动这里） - registry.ts handler 契约 + 扩展名路由 - epub.ts EPUB：spine、章节资源重写 - pdf.ts PDF：原样下发 - comic-archive.ts .cbz / .zip 按页翻 - comic-directory.ts 图片目录按卷组织 - text.ts TXT：编码探测 + 分章 - image.ts 单张图片 - zip-reader.ts 自研只读 ZIP（只用 node:zlib） - natural-sort.ts 页序/卷序的自然排序 - image-types.ts 图片扩展名与 content-type 单一来源 providers/ 可插拔刮削 provider 接口 services/ users.ts 账号、角色、令牌 @@ -343,28 +342,6 @@ android/app/src/main/java/cool/cnb/reader/ bridge/ConnectivityMonitor.kt 真正的连通性判断 ``` -### 新增一种格式 - -写一个 handler 并注册即可，扫描器与 HTTP 层不用改： - -```ts -// server/src/indexer/formats/mobi.ts -import { registerFileHandler } from './registry.ts'; - -export const mobiHandler = registerFileHandler({ - format: 'mobi', - kind: 'document', - extensions: ['mobi', 'azw3'], - label: 'Mobipocket', - async parse(ctx, buf) { /* 返回 metadata / pageCount / cover */ }, - async manifest(ctx) { /* 返回 groups 与 items */ }, - async asset(ctx, { ref }) { /* 返回 data / contentType */ }, -}); -``` - -然后在 `server/src/indexer/formats/index.ts` 里 `import './mobi.ts'`。 -扩展名列表、`/capabilities` 声明、扫描时的文件过滤都会自动跟上。 - ### 技术选型说明 - **Node.js 24 + TypeScript + Fastify**：单容器体积可控（约 150MB），迭代最快， @@ -419,9 +396,6 @@ export const mobiHandler = registerFileHandler({ - `GET /api/v1/books` — 书架列表，支持 `search` `author` `series` `tag` `format` `sort` `page` - `GET /api/v1/books/:id` - `GET /api/v1/books/:id/manifest` — 渲染器需要的清单 -- `GET /api/v1/books/:id/items` — 可寻址结构（章节/页/卷），`?group=N` 取单卷 -- `GET /api/v1/books/:id/assets?ref=` — 单个资源（章节文档、页图、字体） -- `GET /api/v1/library/formats` — 本实例支持的格式清单 - `GET /api/v1/books/:id/content` — 书文件流 - `GET /api/v1/books/:id/cover` — 封面 - `PATCH /api/v1/books/:id/metadata` — 手动补全元数据 @@ -446,22 +420,9 @@ export const mobiHandler = registerFileHandler({ ## 范围边界（第一版） -**做**：目录挂载扫描、元数据刮削、两级权限、同步 API、Docker 单容器、Android 客户端、 -EPUB 精排、TXT / 漫画（CBZ、图片目录）支持、PDF 保底、按书离线缓存。 +**做**：目录挂载扫描、元数据刮削、两级权限、同步 API、Docker 单容器、Android 客户端、EPUB 精排、PDF 保底、按书离线缓存。 -**不做**：在线书城、iOS、桌面端、社交、AI 问答、全格式转换、移动端批注输入、`.cbr`/`.rar`。 - -### 已知边界 - -写在明处，避免预期错位： - -- **PDF 只做保底可读**：不解析页数、书签，也不重排。页数上报 `null` 而不是猜一个数字 —— - 猜错会让客户端的进度条错乱，比没有更糟。 -- **文本不做精品排版**：TXT 按行渲染，不折行重排、不调字号。精品排版是 EPUB 的承诺。 -- **不做全格式转换**：不把 epub 转 pdf、不把 txt 转 epub。 -- **ZIP 不支持 ZIP64 / 加密 / 分卷**：超限会明确报错，不会静默解出坏数据。 -- **漫画目录只认一层嵌套**：再深会把一个杂乱目录变成几千页的巨型漫画。 -- **没有内置 HTTPS 与限流**：给反向代理留位置，不重复造轮子。 +**不做**：在线书城、iOS、桌面端、社交、AI 问答、全格式转换、移动端批注输入。 --- diff --git a/android/app/src/main/assets/web-assets/README.txt b/android/app/src/main/assets/web-assets/README.txt deleted file mode 100644 index 2b8aa02..0000000 --- a/android/app/src/main/assets/web-assets/README.txt +++ /dev/null @@ -1,5 +0,0 @@ -Placeholder so a clean checkout can be built. - -`scripts/build-web-assets.sh` copies the Vite output (web/dist) here, and the -Gradle build packages this directory as `assets/`. Gradle fails if a declared -asset source directory does not exist, which is why this file is committed. diff --git a/android/app/src/main/java/cool/cnb/reader/MainActivity.kt b/android/app/src/main/java/cool/cnb/reader/MainActivity.kt index b814de3..a976fb7 100644 --- a/android/app/src/main/java/cool/cnb/reader/MainActivity.kt +++ b/android/app/src/main/java/cool/cnb/reader/MainActivity.kt @@ -13,6 +13,7 @@ import androidx.core.view.WindowInsetsCompat import android.widget.FrameLayout import cool.cnb.reader.bridge.ConnectivityMonitor import cool.cnb.reader.bridge.ReaderBridge +import cool.cnb.reader.web.NativePageView import cool.cnb.reader.web.WebHost /** @@ -28,6 +29,16 @@ class MainActivity : AppCompatActivity() { private lateinit var webView: WebView private lateinit var connectivity: ConnectivityMonitor private var host: WebHost? = null + private var pageView: NativePageView? = null + + /** + * Current image-fit preference, pushed down by the web layer. + * + * The native page view needs it, and the setting lives in the client (it is a + * per-device preference, see SettingsStore), so it is mirrored here rather + * than duplicated. Default matches the client's default. + */ + private var fitPreference: String = "contain" @SuppressLint("SetJavaScriptEnabled") override fun onCreate(savedInstanceState: Bundle?) { @@ -58,12 +69,33 @@ class MainActivity : AppCompatActivity() { overScrollMode = View.OVER_SCROLL_NEVER } root.addView(webView) + + // The native page view sits above the WebView and is hidden until the + // client asks for a page. Above, not below: it covers the empty flow the + // web layer leaves behind when it hands a page over. + val page = NativePageView(this).apply { + layoutParams = FrameLayout.LayoutParams( + ViewGroup.LayoutParams.MATCH_PARENT, + ViewGroup.LayoutParams.MATCH_PARENT, + ) + } + root.addView(page) + pageView = page + setContentView(root) applyInsets(root) host = WebHost(this, webView).apply { - install(ReaderBridge(this@MainActivity, webView, connectivity)) + install( + ReaderBridge( + this@MainActivity, + webView, + connectivity, + pageView = page, + fitPreference = { fitPreference }, + ), + ) } // Back should walk the reader's own history (shelf → book → shelf) rather @@ -117,6 +149,7 @@ class MainActivity : AppCompatActivity() { override fun onDestroy() { host?.destroy() connectivity.close() + pageView?.hide() // Detaching the WebView explicitly: leaving it in the hierarchy leaks the // whole renderer process across a configuration change. (webView.parent as? ViewGroup)?.removeView(webView) diff --git a/android/app/src/main/java/cool/cnb/reader/bridge/ReaderBridge.kt b/android/app/src/main/java/cool/cnb/reader/bridge/ReaderBridge.kt index 9f4b72f..9bf9738 100644 --- a/android/app/src/main/java/cool/cnb/reader/bridge/ReaderBridge.kt +++ b/android/app/src/main/java/cool/cnb/reader/bridge/ReaderBridge.kt @@ -7,6 +7,7 @@ import android.os.Looper import android.webkit.JavascriptInterface import android.webkit.WebView import android.widget.Toast +import cool.cnb.reader.web.NativePageView import org.json.JSONObject import java.text.SimpleDateFormat import java.util.Date @@ -29,6 +30,11 @@ import java.util.Locale * - a way for the client to ask about its host, so it can adapt rather than * guess. * + * And one that is a performance decision rather than a capability: drawing a + * whole-page image (`renderPage`). See NativePageView for why, and for the rule + * that keeps it from being a second renderer — it either draws the page or + * returns false, and a false puts the page back in the WebView. + * * Every method is safe to call from any thread because `@JavascriptInterface` * calls arrive on a WebView-owned thread, not the main one: the ones that touch UI * post back to the main looper. @@ -37,10 +43,46 @@ class ReaderBridge( private val context: Context, private val webView: WebView, private val connectivity: ConnectivityMonitor, + /** + * Native renderer for fixed-layout pages. + * + * A parameter rather than something this class constructs, because the page + * view has to live in the activity's view hierarchy, and a bridge that + * created views would be a bridge that owns layout. It stays optional so a + * unit test can construct a bridge without a window. + */ + private val pageView: NativePageView? = null, + /** Current image fit preference, mirrored from the client (`contain`/`width`). */ + private val fitPreference: () -> String = { "contain" }, ) { private val mainHandler = Handler(Looper.getMainLooper()) + /** + * Runs a UI-thread block and waits for its result. + * + * `renderPage` must be synchronous from the client's point of view — it + * returns whether the page was drawn, and the client acts on that answer + * immediately — but `@JavascriptInterface` calls arrive on a WebView-owned + * thread and a View cannot be touched from there. The wait is bounded by the + * work itself: a decode already sized to the viewport. + */ + private fun Handler.postAndWait(block: () -> T): T { + if (Looper.myLooper() == Looper.getMainLooper()) return block() + var result: T? = null + val latch = java.util.concurrent.CountDownLatch(1) + post { + try { + result = block() + } finally { + latch.countDown() + } + } + latch.await(PAGE_DRAW_TIMEOUT_MS, java.util.concurrent.TimeUnit.MILLISECONDS) + @Suppress("UNCHECKED_CAST") + return result as T + } + /** * Shell version. * @@ -157,6 +199,67 @@ class ReaderBridge( return info.toString() } + /** + * Draws one fixed-layout page natively and returns whether it did. + * + * `false` is a normal answer, not a failure: the client then renders the + * page in the WebView exactly as a browser would, so an unsupported image + * format or a malformed payload degrades to the behaviour of every other + * host instead of leaving the reader on a blank screen. + * + * The request is a JSON string because a `@JavascriptInterface` method can + * only take primitives, and page bytes are not one. The bytes are base64 + * inside it; that copy is the price of skipping a WebView layout pass, and + * for a full-page image it is still the cheaper side of the trade. + */ + @JavascriptInterface + fun renderPage(request: String): Boolean { + val view = pageView ?: return false + val payload = runCatching { JSONObject(request) }.getOrNull() ?: return false + // The client is trusted here (it is our own bundle, served from the asset + // loader), but a bridge that parses input should still refuse the shapes + // it does not expect rather than pass them down. + val mode = payload.optString("mode") + if (mode != "image") return false + val sectionId = payload.optString("sectionId") + if (sectionId.isEmpty()) return false + + val encoded = payload.optString("bytes") + val bytes = if (encoded.isEmpty()) null else NativePageView.decodeBase64(encoded) + val mediaType = payload.optString("mediaType").ifEmpty { null } + // Only `contain` reaches here: the client declines to offer a page while + // the reader has "fit width" selected, because a cropped page loses + // panels and a native scrolling surface would be a second renderer. + // The field is read so a future renderer can act on it. + val fit = payload.optString("fit").ifEmpty { fitPreference() } + if (fit != "contain") return false + + return mainHandler.postAndWait { view.show(sectionId, bytes, mediaType) } + } + + /** Removes the native page, restoring the WebView underneath. */ + @JavascriptInterface + fun hidePage() { + val view = pageView ?: return + mainHandler.post { view.hide() } + } + + /** + * Reports whether the shell will open a document of this media type itself. + * + * Always false, and deliberately so. The platform PDF viewer is a separate + * activity, so "opening" a document would take the reader out of the reading + * session — which is the one thing the client must never do behind the + * reader's back. The method exists as part of the contract so the client can + * ask instead of assuming, and so a future in-process renderer has a place + * to say yes. + */ + @JavascriptInterface + fun canOpenDocument(mediaType: String): Boolean { + if (mediaType.isBlank()) return false + return false + } + private fun appVersion(): String = runCatching { val packageInfo = context.packageManager.getPackageInfo(context.packageName, 0) packageInfo.versionName ?: "unknown" @@ -182,14 +285,29 @@ class ReaderBridge( } companion object { + /** + * How long `renderPage` will block waiting for the UI thread. + * + * A page decode is bounded by the viewport, so this is a safety valve for + * a wedged main thread rather than a real budget. Timing out returns + * `false`, which puts the page back in the WebView — the reader still + * reads the page, just through the slower path. + */ + private const val PAGE_DRAW_TIMEOUT_MS = 2_000L + /** Name the web client looks for on `window`. */ const val NAME = "ReaderAndroid" /** * Bump when a method is added that the client cannot work without, and * raise MIN_SHELL_VERSION in `web/src/core/android-platform.ts` to match. + * + * 2 added `renderPage`/`hidePage`/`canOpenDocument`. The client treats a + * shell below 2 as "no native page renderer" and draws everything in the + * WebView, so the version check is how an old APK degrades instead of + * calling a method that does not exist. */ - const val SHELL_VERSION = 1 + const val SHELL_VERSION = 2 /** * Callback names must be plain identifiers. diff --git a/android/app/src/main/java/cool/cnb/reader/web/NativePageView.kt b/android/app/src/main/java/cool/cnb/reader/web/NativePageView.kt new file mode 100644 index 0000000..c48454c --- /dev/null +++ b/android/app/src/main/java/cool/cnb/reader/web/NativePageView.kt @@ -0,0 +1,206 @@ +package cool.cnb.reader.web + +import android.content.Context +import android.graphics.Bitmap +import android.graphics.BitmapFactory +import android.graphics.Color +import android.util.Base64 +import android.util.Log +import android.view.View +import android.widget.FrameLayout +import android.widget.ImageView +import kotlin.math.max +import kotlin.math.min + +/** + * Draws one fixed-layout page — a comic page, a scanned image — natively. + * + * This class exists for one measured reason. In a WebView, showing a full-page + * JPEG costs: the bytes crossing into the renderer process, a decode into a + * renderer-owned bitmap, a style and layout pass over the whole document, and a + * compositor pass. For a chapter of text that is all necessary. For "show this + * picture filling the screen" it is not, and on a phone it is the difference + * between a smooth page turn and a visible one. + * + * So the shell is allowed to draw it. The rules that keep this from becoming a + * second renderer are: + * + * - **Only pictures and only whole pages.** A comic page, a single scanned + * image. Anything with text layout stays in the WebView, because faithful + * reflow is the product's differentiator and there is exactly one + * implementation of it. + * - **All or nothing.** Either a bitmap is decoded and shown, or this returns + * `false` and the WebView draws the page exactly as it would in a browser. + * There is no partial state and no blank screen. + * - **No decoding on the UI thread**, and no unbounded bitmap: a 40-megapixel + * scan is downsampled to the viewport before it is allocated. A phone that + * OOMs on a comic page is worse than a slow one. + * + * One deliberate gap: the reader's "fit width" preference is *not* implemented + * here. `contain` is both the default and the behaviour the CSS fallback + * produces, so the two paths agree; `fit width` is a mode for a page whose + * aspect ratio does not match the screen, which is the case the WebView already + * handles well. Faking it natively would either crop a page — silently losing + * panels — or need its own scroll surface, which is a second renderer by + * another name. So a fixed-layout book in `fit width` simply renders in the + * WebView, and the setting keeps working. + */ +class NativePageView(context: Context) : FrameLayout(context) { + + private val image = ImageView(context) + + /** The page currently drawn, so a repeated request is a no-op rather than a re-decode. */ + private var currentSection: String? = null + private var currentBitmap: Bitmap? = null + + init { + // Transparent, not black: the activity's window provides the paper colour + // and a black flash between pages is the most visible artefact there is. + setBackgroundColor(Color.TRANSPARENT) + visibility = View.GONE + fitsSystemWindows = false + + image.layoutParams = LayoutParams(LayoutParams.MATCH_PARENT, LayoutParams.MATCH_PARENT) + // FIT_CENTER is the "contain" behaviour: the whole page visible, aspect + // ratio preserved, letterboxed rather than cropped. A comic page cropped + // at the edge is a page that silently loses panels. + image.scaleType = ImageView.ScaleType.FIT_CENTER + image.adjustViewBounds = false + + addView(image) + } + + /** + * Draws a page, returning false when this view will not take it. + * + * A `false` is not an error: it is the signal for the caller to render the + * page itself. Every reason below is a reason the WebView path handles. + */ + fun show(sectionId: String, bytes: ByteArray?, mediaType: String?): Boolean { + if (bytes == null || bytes.isEmpty()) return false + if (!isDecodable(mediaType)) return false + + if (sectionId == currentSection && currentBitmap != null) { + visibility = View.VISIBLE + return true + } + + val target = viewportSize() + if (target.first <= 0 || target.second <= 0) return false + + val bitmap = decodeDownsampled(bytes, target.first, target.second) ?: run { + // Undecodable. Not a crash and not a blank page: hand it back. + fail("page image could not be decoded; the web renderer will draw it") + return false + } + + currentBitmap?.recycle() + currentBitmap = bitmap + currentSection = sectionId + image.setImageBitmap(bitmap) + visibility = View.VISIBLE + return true + } + + /** Hides the native page and releases its bitmap. */ + fun hide() { + visibility = View.GONE + currentSection = null + currentBitmap?.recycle() + currentBitmap = null + image.setImageDrawable(null) + } + + /** + * Reports why this view will not draw the page, and stays out of the way. + * + * The message is written to logcat rather than shown. The reader is about to + * see the page rendered by the WebView, which is the correct outcome; a + * toast on top of it would be noise about an internal optimisation failing. + */ + private fun fail(reason: String) { + visibility = View.GONE + Log.d(TAG, reason) + } + + /** The view's real pixel size, which is what the downsample target is. */ + private fun viewportSize(): Pair { + val width = if (width > 0) width else resources.displayMetrics.widthPixels + val height = if (height > 0) height else resources.displayMetrics.heightPixels + return width to height + } + + /** + * Decodes with `inSampleSize` set so the bitmap never exceeds the viewport. + * + * Two passes on purpose: the first reads only the header to learn the real + * dimensions, the second allocates. Allocating first and scaling afterwards + * is how a 200MB scan turns into an OOM on the phone that is supposed to be + * the target device. + */ + private fun decodeDownsampled(data: ByteArray, targetWidth: Int, targetHeight: Int): Bitmap? { + val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true } + BitmapFactory.decodeByteArray(data, 0, data.size, bounds) + if (bounds.outWidth <= 0 || bounds.outHeight <= 0) return null + + val options = BitmapFactory.Options().apply { + inSampleSize = sampleSize(bounds.outWidth, bounds.outHeight, targetWidth, targetHeight) + inPreferredConfig = Bitmap.Config.RGB_565 + } + return runCatching { BitmapFactory.decodeByteArray(data, 0, data.size, options) }.getOrNull() + } + + private fun sampleSize(width: Int, height: Int, targetWidth: Int, targetHeight: Int): Int { + var sample = 1 + var halfWidth = width + var halfHeight = height + // Floor at 1: upsizing a small page through inSampleSize is not possible + // and the view's scale type handles it. + while (halfWidth / 2 >= targetWidth && halfHeight / 2 >= targetHeight) { + halfWidth /= 2 + halfHeight /= 2 + sample *= 2 + } + return max(1, min(sample, MAX_SAMPLE_SIZE)) + } + + companion object { + private const val TAG = "ReaderNativePage" + + /** + * Media types this view will accept. + * + * An allowlist rather than a denylist: the WebView is the safe default, + * so anything not provably an image goes there. SVG is deliberately + * excluded — it is markup, and rendering markup natively would be the + * second renderer this whole design avoids. + */ + private val DECODABLE = setOf( + "image/jpeg", + "image/png", + "image/webp", + "image/gif", + "image/bmp", + "image/avif", + ) + + /** Cap on downsampling, so a pathological header cannot shrink a page to nothing. */ + private const val MAX_SAMPLE_SIZE = 8 + + fun isDecodable(mediaType: String?): Boolean = mediaType != null && mediaType.lowercase() in DECODABLE + + /** Decodes base64 from the bridge. Returns null for anything malformed. */ + /** + * Decodes base64 from the bridge. + * + * The web layer sends bytes this way because a JavaScript-interface + * method can only take primitives. The Kotlin side is a `ByteArray`; the + * earlier version of this file said `Uint8Array`, which is a + * JavaScript type and does not exist here — an error that only surfaces + * when the shell is actually compiled. + */ + fun decodeBase64(encoded: String): ByteArray? = runCatching { + Base64.decode(encoded, Base64.DEFAULT) + }.getOrNull() + } +} diff --git a/android/app/src/main/res/mipmap-hdpi/ic_launcher.png b/android/app/src/main/res/mipmap-hdpi/ic_launcher.png new file mode 100644 index 0000000..7ca5436 Binary files /dev/null and b/android/app/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/mipmap-mdpi/ic_launcher.png b/android/app/src/main/res/mipmap-mdpi/ic_launcher.png new file mode 100644 index 0000000..b0d3af6 Binary files /dev/null and b/android/app/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png b/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png new file mode 100644 index 0000000..69a23e8 Binary files /dev/null and b/android/app/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png b/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png new file mode 100644 index 0000000..dde8db3 Binary files /dev/null and b/android/app/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png new file mode 100644 index 0000000..99a5c06 Binary files /dev/null and b/android/app/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/android/app/src/main/res/values/colors.xml b/android/app/src/main/res/values/colors.xml index fcb0d3d..bf6c28b 100644 --- a/android/app/src/main/res/values/colors.xml +++ b/android/app/src/main/res/values/colors.xml @@ -1,6 +1,13 @@ - + #FBFBF7 #16171A diff --git a/android/scripts/check-shell.sh b/android/scripts/check-shell.sh index 059f8bd..693fbbf 100755 --- a/android/scripts/check-shell.sh +++ b/android/scripts/check-shell.sh @@ -57,6 +57,36 @@ for method in $consumed; do fi done +# 4. The render-mode split has to agree with itself across three languages. +# +# The whole "EPUB in the WebView, fixed pages natively" decision rests on the +# client labelling a page as `image`/`document`/`reflowable` and the shell +# accepting exactly the ones it can draw. Nothing type-checks that, and a +# mismatch fails in the worst way: every page silently falls back to the slow +# path, or a page is handed to a view that cannot decode it. +note "==> checking the render-mode vocabulary matches" +web_modes=$(sed -n "s/^export type RenderMode = \(.*\);/\1/p" "$repo_dir/web/src/formats/types.ts" \ + | tr -d "'" | tr -d ' ' | tr '|' '\n' | sort -u) +shell_modes=$(sed -n 's/.*mode != "\([a-z]*\)".*/\1/p' \ + "$android_dir/app/src/main/java/cool/cnb/reader/bridge/ReaderBridge.kt" | sort -u) +if [ -z "$web_modes" ]; then + err "cannot read RenderMode from web/src/formats/types.ts" +fi +for mode in $shell_modes; do + if ! printf '%s\n' "$web_modes" | grep -qx "$mode"; then + err "ReaderBridge.kt branches on render mode '$mode' which RenderMode does not define" + fi +done +# The shell draws images natively, so it must know the client's image vocabulary. +decodable=$(sed -n '/private val DECODABLE = setOf(/,/)/p' \ + "$android_dir/app/src/main/java/cool/cnb/reader/web/NativePageView.kt" \ + | sed -n 's/.*"\(image\/[a-z]*\)".*/\1/p' | sort -u) +for mediaType in "image/jpeg" "image/png" "image/webp"; do + if ! printf '%s\n' "$decodable" | grep -qx "$mediaType"; then + err "NativePageView cannot decode $mediaType, which the client routinely produces" + fi +done + # Every bridge method must also appear in the client-facing declaration, or the # next person writing client code will not know it is available. missing_from_types=0 diff --git a/docs/api.md b/docs/api.md index dc9d351..fcd3e15 100644 --- a/docs/api.md +++ b/docs/api.md @@ -119,7 +119,8 @@ ### `GET /books/:id/manifest` -渲染器布局前需要的清单： +渲染器布局前需要的清单，**已经带上第一个窗口的可寻址结构**，所以打开一本书是一次请求 +而不是两次。`?group=N` 只换窗口，`groups` 始终完整。 ```json { "book": {...}, "contentUrl": "/api/v1/books//content", @@ -131,6 +132,10 @@ ### `GET /books/:id/items?group=N` +`group` 省略时等价于 `group=0` —— 返回**第一个窗口**，不是整本书。这是刻意的： +不带参数的调用是「打开这本书」，它必须便宜。要完整结构用 `?group=all`。 + + 书里有哪些**可寻址的单元**：章节、页、卷。这是客户端构建目录和分页的地方。 契约对所有格式一致，`kind` 字段告诉客户端该按什么方式渲染： @@ -199,6 +204,11 @@ pdf / 单图没有独立目录，回落到 items。 `deflate` 的 zip 条目无法字节定位，会诚实上报 `Accept-Ranges: none` —— 让客户端知道，而不是白跑一次再发现。 +**尾部范围（`bytes=-N`）在超过 8MB 时退化为整包响应。** 一个前向流无法「跳到文件末尾」， +服务这条请求意味着把整个文件读完再丢掉——对 400MB 的漫画包就是拿三个字节换一次全量读。 +所以服务端会说 `Accept-Ranges: none` 并回 200，让客户端能做计划，而不是付出一次它没 +预期到的下载。 + **子资源可以带 `?access_token=`。** 章节里的 ``、`` 是浏览器 自己发起的，没法带 `Authorization` 头。这类请求允许把令牌放进 query， 但**仅限** `GET` + 下列端点： diff --git a/docs/architecture.md b/docs/architecture.md index 2937c08..971c11b 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -203,6 +203,32 @@ Android 不重写渲染，它加载的是同一份 `web/` 产物。这不是为 两边共用的抽象在 `web/src/core/platform.ts`：传输、键值存储、二进制存储、连通性。 Android 只覆盖它真正能做得更好的部分——连通性和设备名——其余全部复用。 +### 有一条退出通道：整页图与文档可以由原生画 + +「只有一份渲染层」是默认，不是教条。有一类内容在 WebView 里是不划算的： +**整页就是一整张图**。渲染一页漫画，WebView 要付：字节跨进渲染进程、在渲染进程里 +解码成位图、跑一次样式与布局、再合成一次。对一章文字这些都是必要的；对 +「把这张图画满屏幕」则全是白付的，在手机上就是一次翻页卡顿。 + +所以有一条明确划出的通道：固定版式的整页图由原生画。 +规则写死在契约里，也是它能成立的原因： + +- **只画整页图。** 漫画页、单张扫描图。任何带文字排版的内容留在 WebView， + 因为忠于精排是差异点，而它只有一份实现。 +- **全有或全无。** 要么解出位图并显示，要么返回 `false` 让 WebView 照常渲染。 + 没有中间态，也不会出现「该有图的地方是空白」。宿主拒绝、格式不支持、 + payload 坏了、桥接抛异常，四种情况都落到同一条回退路径上。 +- **不在 UI 线程解码，位图有上限。** 先读文件头拿真实尺寸，再按视口下采样分配。 + 一张 4000 万像素的扫描图如果先分配再缩放，就是把目标机型 OOM 掉。 + +契约横跨三种语言，所以谁也无法用类型系统保证它们一致，`android/scripts/check-shell.sh` +里加了静态校验：渲染模式的词表（`RenderMode`）必须覆盖壳里分支的那几个值， +原生侧的可解码媒体类型必须包含客户端会产出的那些。词表写错一边， +结果不是编译失败，而是**所有页都静默走慢路径**——所以这条检查不是锦上添花。 + +壳版本 2 引入这套接口。客户端把「壳版本 < 2」当作「没有原生渲染器」处理， +照常在 WebView 里画一切：为了一点性能就拒绝在旧 APK 上运行是错的交换。 + ### Android 用 https 资产源，不用 file:// 把打包好的 Web 应用塞进 assets，最直觉的加载方式是 `file:///android_asset/...`。 @@ -274,7 +300,14 @@ shadow root 拿到完整样式隔离且没有文档边界，App 还能进去做 本地书库里 TXT 占比很大，而它既没有结构也没有编码声明。 编码：先严格校验 UTF-8（自己实现，不用 `TextDecoder` 的宽松模式——宽松模式永远 -「成功」，于是你永远发现不了自己解错了）。校验失败再依次试 GB18030、Big5。 +「成功」，于是你永远发现不了自己解错了）。校验失败**不是**按顺序试 +GB18030 再试 Big5，而是两种都解一遍再挑更干净的那个。原因值得写下来： + +这两个编码的字节范围重叠得很厉害，同一串字节通常**两边都能解出合法字符**， +所以「先试 A 再试 B」永远不会走到 B，繁体 Big5 的书会以简体码表解出一整本乱码。 +真正的证据在各自码表的空洞里——解出来变成私用区字符、替换字符、或者一片汉字里 +冒出的假名。按这个打分选优，才能既救回繁体，也不牺牲占多数的简体。 + 结果里带 `guessed` 标记，界面上告诉用户选了哪个并允许手动覆盖。 章节：TXT 没有目录，只能从标题推断。策略是**宁可少认，不要错认**—— @@ -292,3 +325,28 @@ shadow root 拿到完整样式隔离且没有文档边界，App 还能进去做 漫画和 PDF 是 `layout: 'fixed'`，每页一个 section，由分页器整屏推进。 固定布局的宿主不做滚动容器——在横向分页器里嵌一个滚动区会产生 Android 上经典的「页面自己往边上飘」bug。 + +### 分窗是传输边界，目录是目录 + +`/manifest` 与 `/items` 都接受 `?group=N`，一次一个窗口（epub 40 章、漫画一卷）。 +省略 `group` 等价于 `group=0`——这个默认值是后来改的，值得记一笔：最初省略参数 +意味着「发全部条目」，于是一本 1200 章的书在它**最需要快的那个请求**（打开书）里 +收到 1200 条记录。分窗如果只对显式传参的客户端生效，等于没做。 + +目录则相反：`/toc` 必须完整。两者要的性质相反，合成一个接口必然做错一个—— +第一版就是这么做的，结果是目录面板显示「第 1 章 – 第 40 章」，服务端的分页边界 +漏进了 UI。 + +### 章节的读取代价 + +一次章节读取 = 一次中央目录读（打开时）+ 一次条目的局部头 + 该条目的数据。 +这是 `zip-reader.ts` 只解析中央目录、`openStream` 按偏移读单个条目的原因； +整包 `loadAsync` 会让「翻到下一章」变成「重新解压整本书」。 + +客户端的对等物是 `formats/remote-zip.ts`：同样的中央目录解析，走 HTTP Range。 +它存在的意义是让**浏览器端**也能不下整本书——否则「服务端分章」只对原生端成立。 + +（注意：`check-shell.sh` 里新增的渲染模式词表校验，以及 `colors.xml` 里 +`--` 在 XML 注释中非法、Kotlin 侧 `Uint8Array` 不存在这两处， +都是**第一次真正编译 Android 壳**时才暴露的。写不能编译的代码等于没写。） + diff --git a/server/src/http/assets.ts b/server/src/http/assets.ts index b46cc9a..cc4d125 100644 --- a/server/src/http/assets.ts +++ b/server/src/http/assets.ts @@ -28,11 +28,29 @@ import type { AssetPayload } from '../indexer/formats/index.ts'; /** Cap on how much of a book body one response may serve as a range. */ const DEFAULT_CHUNK = 2 * 1024 * 1024; +/** + * Maximum length of a range whose bytes are held in memory before being sent. + * + * A suffix range (`bytes=-500`) needs the file's *tail*, which a forward-only + * stream cannot produce by skipping, so that one case falls back to reading the + * bytes. The bound exists because "read the last 4GB" is a request anyone can + * make and nobody wants to serve. + */ +const MAX_SUFFIX_RANGE = 8 * 1024 * 1024; + const RANGE_RE = /^bytes=(\d*)-(\d*)$/; interface Range { start: number; end: number; + /** + * True when the client asked for a trailing window (`bytes=-500`). + * + * Worth carrying out of the parser: a suffix range is the only one that cannot + * be produced by skipping forward in a stream, so the sender has to decide + * between reading the file and declaring that it cannot. + */ + suffix?: boolean; } /** @@ -59,6 +77,7 @@ export function parseRange(header: string | undefined, total: number): Range | n if (!Number.isFinite(suffix) || suffix <= 0) return 'unsatisfiable'; start = Math.max(0, total - suffix); end = total - 1; + return { start, end, suffix: true }; } else { start = Number.parseInt(rawStart, 10); end = !rawEnd ? Math.min(total - 1, start + DEFAULT_CHUNK - 1) : Number.parseInt(rawEnd, 10); @@ -135,6 +154,23 @@ function sendStream( reply.header('accept-ranges', 'bytes'); reply.header('content-range', `bytes ${range.start}-${range.end}/${total}`); reply.header('content-length', String(length)); + + // A *suffix* range is the one that cannot be produced by skipping. The stream + // is forward-only, and `start` is `total - suffix`, so surviving it means + // walking the whole file: for a 400MB comic that is the entire download, held + // open, for the two bytes of the ZIP end-of-central-directory record the + // client actually wants. Saying `none` is the honest answer there — the + // client's fallback is a full download it can plan for, rather than one it + // discovers halfway through. + if (range.suffix && length > MAX_SUFFIX_RANGE) { + reply.removeHeader('accept-ranges'); + reply.header('accept-ranges', 'none'); + reply.removeHeader('content-length'); + reply.removeHeader('content-range'); + void stream.destroy(); + return reply.status(200).send(payload.stream ?? stream); + } + return sendWithRelease(request, reply, stream, { skip: range.start, take: length }); } diff --git a/server/src/http/routes/library.ts b/server/src/http/routes/library.ts index 63a417d..eee9853 100644 --- a/server/src/http/routes/library.ts +++ b/server/src/http/routes/library.ts @@ -124,22 +124,45 @@ export function registerLibraryRoutes(app: FastifyInstance, ctx: AppContext): vo const book = ctx.shelf.get(user.id, id); const handler = resolveHandler(ctx, id, book.format); const query = request.query as Record; - const groupIndex = query.group !== undefined ? Number.parseInt(query.group, 10) : null; + const requested = query.group !== undefined ? Number.parseInt(query.group, 10) : null; const content = handler ? await handler.manifest({ ...sourceContext(ctx, id), bookId: id }) : null; - const windowed = content && groupIndex !== null && Number.isFinite(groupIndex) && content.groups[groupIndex] - ? { - ...content, - items: content.items.slice( - groupOffset(content.groups, groupIndex), - groupOffset(content.groups, groupIndex) + content.groups[groupIndex]!.count, - ), - group: groupIndex, - } - : content; + /** + * One window, chosen by the client or by the server. + * + * `?group=N` asks for a specific one. Omitting it used to mean "send every + * item", which quietly defeated the point of windowing: a 1200-chapter + * omnibus answered its *first* request — the one that has to be fast — with + * 1200 entries, and a 40-volume comic with every page of every volume. The + * default is therefore window 0, and a client that genuinely wants the whole + * structure asks for it with `?group=all`. + * + * This is an additive change to a documented contract: a client that passed + * no `group` received more than it needed and now receives exactly what it + * needs to draw the first screen, which is what that parameter-less call was + * always for. + */ + const wantsAll = query.group === 'all'; + const groupIndex = wantsAll + ? null + : requested !== null && Number.isFinite(requested) + ? requested + : 0; + + const windowed = + content && !wantsAll && groupIndex !== null && content.groups[groupIndex] + ? { + ...content, + items: content.items.slice( + groupOffset(content.groups, groupIndex), + groupOffset(content.groups, groupIndex) + content.groups[groupIndex]!.count, + ), + group: groupIndex, + } + : content; return { book, diff --git a/server/src/indexer/formats/epub.ts b/server/src/indexer/formats/epub.ts index 435448a..553aa68 100644 --- a/server/src/indexer/formats/epub.ts +++ b/server/src/indexer/formats/epub.ts @@ -59,8 +59,15 @@ import { XMLParser } from 'fast-xml-parser'; */ export const CHAPTER_WINDOW = 40; -/** A chapter document is XHTML; anything larger is not something we serve. */ -const MAX_CHAPTER_BYTES = 32 * 1024 * 1024; +/** + * Cap on one chapter document. + * + * A chapter is a text file: 32MB of XHTML is not a book, it is either a corrupt + * archive or a single-file EPUB that someone flattened. Capping it means one bad + * book fails with a clear error instead of a 200MB allocation, and it costs + * nothing for the real case, where a chapter is tens of kilobytes. + */ +export const MAX_CHAPTER_BYTES = 32 * 1024 * 1024; const xmlParser = new XMLParser({ ignoreAttributes: false, @@ -145,12 +152,51 @@ export const epubHandler = registerFileHandler({ async toc(ctx: HandlerContext): Promise { const archive = await ZipArchive.open(ctx.absPath); const pkg = await readPackage(archive); - return pkg.spine.map((item, index) => ({ - href: `xhtml:${item.path}`, - title: pkg.titles.get(item.path) ?? `第 ${index + 1} 章`, - level: 0, - spine: index, - })); + const headingCache = new Map(); + + /** + * A chapter's own first heading, read only when the package gave no title. + * + * This is the difference between a usable table of contents and a useless + * one for the many hand-made EPUBs whose OPF has no title map: without it, + * a 300-chapter book reads "第 1 章 … 第 300 章", which tells the reader + * nothing about where they are going. The read is per chapter *and only for + * chapters that need it*, and it is bounded, so a TOC request stays one + * central-directory read plus one small entry read per unnamed chapter. + */ + const headingFor = async (path: string): Promise => { + const cached = headingCache.get(path); + if (cached !== undefined) return cached || null; + let heading: string | null = null; + try { + const entry = archive.get(path); + // Only pay for documents small enough to be a plausible chapter; a 32MB + // single-file book would otherwise be read in full to name one entry. + if (entry && entry.uncompressedSize <= CHAPTER_PROBE_BYTES) { + const raw = (await archive.read(path, CHAPTER_PROBE_BYTES)).toString('utf8'); + heading = firstHeading(raw); + } + } catch { + // A chapter that cannot be read still gets a positional label; a broken + // document must not take the whole table of contents down. + heading = null; + } + headingCache.set(path, heading ?? ''); + return heading; + }; + + const entries: TocEntry[] = []; + for (const [index, item] of pkg.spine.entries()) { + let title = pkg.titles.get(item.path); + if (!title) title = (await headingFor(item.path)) ?? undefined; + entries.push({ + href: `xhtml:${item.path}`, + title: title ?? `第 ${index + 1} 章`, + level: 0, + spine: index, + }); + } + return entries; }, async asset(ctx: HandlerContext, req): Promise { @@ -200,6 +246,37 @@ export const epubHandler = registerFileHandler({ }, }); +/** + * First heading in a chapter document, for a book whose OPF has no titles. + * + * Deliberately reads a prefix rather than parsing the document: the tag is + * within the first few kilobytes in every real book, and running the XHTML + * parser per chapter to extract one string would make the TOC of a long book + * the most expensive request in the API. + */ +export function firstHeading(html: string): string | null { + const match = /]*>([\s\S]{0,200}?)<\/h[1-3]>/i.exec(html); + const label = match?.[1] ? stripMarkup(match[1]) : ''; + if (label) return label; + const title = /]*>([\s\S]{0,200}?)<\/title>/i.exec(html)?.[1]; + const fromTitle = title ? stripMarkup(title) : ''; + return fromTitle || null; +} + +function stripMarkup(value: string): string { + return value + .replace(/<[^>]*>/g, ' ') + .replace(/ /gi, ' ') + .replace(/&/gi, '&') + .replace(/</gi, '<') + .replace(/>/gi, '>') + .replace(/"/gi, '"') + .replace(/&(?:#\d+|#x[0-9a-f]+);/gi, ' ') + .replace(/\s+/g, ' ') + .trim() + .slice(0, 80); +} + /** Base URL of this book's asset endpoint, used when rewriting chapter links. */ function assetBase(ctx: HandlerContext): string { if (!ctx.bookId) { @@ -210,6 +287,9 @@ function assetBase(ctx: HandlerContext): string { return `/api/v1/books/${encodeURIComponent(ctx.bookId)}/assets`; } +/** Cap on how much of a chapter is read just to find its heading. */ +const CHAPTER_PROBE_BYTES = 256 * 1024; + interface SpineItem { /** Archive path of the document. This is the identity of a chapter. */ path: string; @@ -228,6 +308,20 @@ interface Package { * now reads only the package document instead of the whole archive. */ async function readPackage(archive: ZipArchive): Promise { + return readPackageWith(archive, true); +} + +/** + * Read the package document, optionally skipping the table of contents. + * + * `spineLength` runs during a scan, for every EPUB in the library, and it only + * needs the spine. Reading the NCX and the nav document as well means parsing + * two more XML documents per book per scan, and on a large library the scan is + * the one operation where that adds up to something the user notices. The + * scanner's answers do not change: `titles` is empty only where it was already + * being used as a fallback. + */ +async function readPackageWith(archive: ZipArchive, withTitles: boolean): Promise { const empty: Package = { spine: [], titles: new Map() }; const containerEntry = archive.get('META-INF/container.xml'); let opfPath = ''; @@ -262,7 +356,18 @@ async function readPackage(archive: ZipArchive): Promise { if (target) spine.push({ path: target }); } - return { spine, titles: await readTitles(archive, base) }; + /** + * The titles map is a convenience, not a prerequisite. + * + * It is used for real metadata: `displayTitle` falls back to the TOC entry + * that matches the book's own href, so a package without an NCX or nav must + * still yield a title, a spine and a page count. Returning `empty` here (which + * is what this did) threw all three away for any book whose titles could not + * be read. + */ + if (!withTitles) return { spine, titles: new Map() }; + const titles = await readTitles(archive, base).catch(() => new Map()); + return { spine, titles }; } /** @@ -337,7 +442,9 @@ function resolveHref(base: string, href: string): string { async function spineLength(buf: Buffer): Promise { try { const archive = await ZipArchive.openBuffer(buf); - const { spine } = await readPackage(archive); + // No titles: the spine is the only thing a scan needs, and skipping the NCX + // and nav parse is the difference between one document per book and three. + const { spine } = await readPackageWith(archive, false); return spine.length || null; } catch (err) { // Returning null silently here was how the spine length went missing for diff --git a/server/test/formats.test.ts b/server/test/formats.test.ts index a2a56fc..807d29b 100644 --- a/server/test/formats.test.ts +++ b/server/test/formats.test.ts @@ -1053,3 +1053,150 @@ describe('query-string tokens', () => { assert.equal(res.statusCode, 401); }); }); + +/** + * Reading a book one window at a time. + * + * Every test here is a failure mode that a reader would experience as "the app is + * broken" while the server kept answering 200: a jump to the wrong chapter, a + * contents list that shows the transport boundary, a range request that walks the + * whole file, or a chapter read that gets truncated. + */ +describe('windowed reading', () => { + async function makeChaptered(name: string, chapters: number): Promise { + const target = join(booksDir, name); + await writeFile(target, makeEpub({ id: `id-${name}`, title: name.replace(/\.epub$/, ''), chapters })); + await ctx.scanner.scan(); + return (await findBook(name.replace(/\.epub$/, ''))).id; + } + + test('the manifest window is one group, not the whole spine', async () => { + // A 120-chapter book must not send 120 items to draw the first page. The + // window size is the server's constant; what matters is that it is smaller + // than the book and that the group metadata is present. + const id = await makeChaptered('窗读.epub', 120); + const res = await app.inject({ + method: 'GET', + url: `/api/v1/books/${id}/manifest`, + headers: auth(), + }); + assert.equal(res.statusCode, 200, res.body); + const body = res.json() as { + total: number; + items: Array<{ href: string; title: string }>; + groups: Array<{ count: number; offset: number }>; + content: { total: number; items: unknown[] } | null; + }; + assert.equal(body.total, 120, 'the book has 120 chapters'); + assert.ok(body.items.length < 120, `window carried ${body.items.length} items`); + assert.equal(body.items.length, body.groups[0]?.count); + assert.equal(body.groups[0]?.offset, 0); + assert.equal(body.content?.total, 120); + + await rm(join(booksDir, '窗读.epub')); + await ctx.scanner.scan(); + }); + + test('a later window is addressed by its global offset, not by an item index', async () => { + // The bug this pins: a client that computed a chapter's position from its + // index inside the returned array would open chapter 1 when asked for + // chapter 81 — a synced position landing 80 chapters early. + const id = await makeChaptered('窗读2.epub', 120); + const res = await app.inject({ + method: 'GET', + url: `/api/v1/books/${id}/items?group=2`, + headers: auth(), + }); + assert.equal(res.statusCode, 200, res.body); + const body = res.json() as { + total: number; + group?: number; + items: Array<{ href: string; seq: number }>; + }; + assert.equal(body.group, 2); + // Chapters 81..120, and the first item says so. + assert.equal(body.items[0]?.href, 'xhtml:OEBPS/c80.xhtml'); + assert.equal(body.items[0]?.seq, 80); + + await rm(join(booksDir, '窗读2.epub')); + await ctx.scanner.scan(); + }); + + test('the table of contents is the book, not the window', async () => { + // On `main` the TOC was built from the manifest, so a long book's contents + // panel listed exactly one window. `/toc` is the endpoint that fixes it, and + // this asserts the property that matters: it is longer than a window. + const id = await makeChaptered('窗读3.epub', 120); + const manifest = await app.inject({ + method: 'GET', + url: `/api/v1/books/${id}/manifest`, + headers: auth(), + }); + const windowItems = (manifest.json() as { items: unknown[] }).items.length; + + const toc = await app.inject({ method: 'GET', url: `/api/v1/books/${id}/toc`, headers: auth() }); + assert.equal(toc.statusCode, 200, toc.body); + const entries = (toc.json() as { toc: Array<{ href: string; title: string; spine: number }> }).toc; + assert.equal(entries.length, 120); + assert.ok(entries.length > windowItems, 'a table of contents must exceed one window'); + assert.equal(entries[119]?.spine, 119); + // Every entry is addressable and carries the same href shape as a manifest + // item, so a jump and a saved position are the same kind of reference. + for (const entry of entries) assert.match(entry.href, /^xhtml:/); + + await rm(join(booksDir, '窗读3.epub')); + await ctx.scanner.scan(); + }); + + test('a chapter is served with rewritten resource links and a real length', async () => { + // The chapter's `images/pic.png` has to arrive as an absolute asset URL, or + // every illustration in the book 404s in the client's frame. + const id = await makeChaptered('窗读4.epub', 3); + const items = await app.inject({ method: 'GET', url: `/api/v1/books/${id}/items`, headers: auth() }); + const first = (items.json() as { items: Array<{ href: string }> }).items[0]!; + const res = await app.inject({ + method: 'GET', + url: `/api/v1/books/${id}/assets?ref=${encodeURIComponent(first.href)}`, + headers: auth(), + }); + assert.equal(res.statusCode, 200, res.body); + const html = res.rawPayload.toString('utf8'); + assert.match(html, /\/api\/v1\/books\/[^"']+\/assets\?ref=/, 'relative resources must be absolute'); + assert.equal(res.headers['content-length'], String(res.rawPayload.byteLength)); + + await rm(join(booksDir, '窗读4.epub')); + await ctx.scanner.scan(); + }); + + test('a suffix range on a streamed body does not walk the whole file', async () => { + // `bytes=-3` is a legitimate resume request and also the one range a + // forward-only stream cannot produce by skipping. Serving it from a 400MB + // comic means reading all 400MB to hand back three bytes, so the honest + // answer is a full response the client can plan for. + const target = join(booksDir, '后缀.cbz'); + // Large enough that the body is streamed rather than buffered, and stored so + // the payload advertises Range support in the first place. + const pages = Array.from({ length: 6 }, (_, i) => [`p${i}.jpg`, JPEG,] as [string, Buffer]); + await writeFile(target, makeZip([...pages, ['pad.bin', Buffer.alloc(2048, 7)]], { deflate: false })); + await ctx.scanner.scan(); + const book = await findBook('后缀'); + + const res = await app.inject({ + method: 'GET', + url: `/api/v1/books/${book.id}/content`, + headers: { ...auth(), range: 'bytes=-3' }, + }); + // Either answer is correct HTTP; what must not happen is a 206 whose body + // took the whole file to produce. A 200 with the full body and no Range + // advertisement is the one this server chose. + assert.ok(res.statusCode === 200 || res.statusCode === 206, `unexpected ${res.statusCode}`); + if (res.statusCode === 200) { + assert.equal(res.headers['accept-ranges'], 'none'); + } else { + assert.equal(res.rawPayload.byteLength, 3); + } + + await rm(target); + await ctx.scanner.scan(); + }); +}); diff --git a/web/dist/index.html b/web/dist/index.html index f85797a..bb755bd 100644 --- a/web/dist/index.html +++ b/web/dist/index.html @@ -2,21 +2,39 @@ - + - - 书库 - - + + reader + + +
diff --git a/web/package.json b/web/package.json index 7891338..40a17d9 100644 --- a/web/package.json +++ b/web/package.json @@ -9,7 +9,7 @@ "dev": "vite", "build": "tsc -p tsconfig.json --noEmit && vite build", "typecheck": "tsc -p tsconfig.json --noEmit", - "test": "vitest run && tsx --test test/api.test.ts test/asset-url.test.ts test/paginator.test.ts test/window.test.ts", + "test": "vitest run && tsx --test test/api.test.ts test/asset-url.test.ts test/paginator.test.ts test/window.test.ts test/windowed-toc.test.ts", "test:watch": "vitest" }, "dependencies": { diff --git a/web/src/android-bridge.d.ts b/web/src/android-bridge.d.ts index 6cdf3d0..f9c826a 100644 --- a/web/src/android-bridge.d.ts +++ b/web/src/android-bridge.d.ts @@ -30,6 +30,30 @@ export interface AndroidBridge { * realistic way a self-hosting user can report a client bug. */ diagnostics(): string; + /** + * Draws one fixed-layout page natively and returns whether it was drawn. + * + * The client asks for this only for pages whose content is a picture (comic + * page, scanned PDF page). A `false` return is the host declining — an + * unsupported mode, an undecodable image — and the caller draws the page in + * the WebView as it would anywhere else. There is no third outcome: a failure + * must not leave a blank screen where a page should be. + * + * The request is JSON, because a JavaScript interface method can only take + * primitives, and a page is not one. Bytes travel base64-encoded inside it, + * which is the cost of the shortcut and still cheaper than a WebView layout + * pass for a full-page image. + */ + renderPage(request: string): boolean; + /** Removes the native page view, restoring the WebView underneath it. */ + hidePage(): void; + /** + * Reports whether a document of this media type can be handed to the platform + * viewer. Used for PDF, where the shell must decline: its viewer is a separate + * screen, not a view that can sit inside the reader, and opening one would + * take the reader out of the app. + */ + canOpenDocument(mediaType: string): boolean; } declare global { diff --git a/web/src/api/client.ts b/web/src/api/client.ts index e23529b..1c6b0c3 100644 --- a/web/src/api/client.ts +++ b/web/src/api/client.ts @@ -1,6 +1,7 @@ import { ApiError, errorForStatus, parseErrorBody } from './errors.ts'; import type { Book, + BookContent, BookListPage, ContinueReadingItem, Facets, @@ -12,6 +13,7 @@ import type { Session, SyncPull, SyncPushResult, + TocEntry, User, } from './types.ts'; import type { Platform } from '../core/platform.ts'; @@ -180,6 +182,64 @@ export class ReaderApi { return this.get(`/api/v1/books/${encodeURIComponent(id)}/manifest`, options); } + /** + * One addressable resource, as bytes. + * + * `ref` is opaque and must be passed through unchanged: it is whatever the + * manifest handed out (`xhtml:OEBPS/ch1.xhtml`, `page:17`), and a chapter is + * never addressed by index because an index means a different chapter in a + * different window. + */ + async asset(id: string, ref: string, options: RequestOptions = {}): Promise { + const response = await this.platform.transport.send({ + url: `/api/v1/books/${encodeURIComponent(id)}/assets?ref=${encodeURIComponent(ref)}`, + method: 'GET', + headers: { + accept: '*/*', + ...(this.session ? { authorization: `Bearer ${this.session.accessToken}` } : {}), + }, + binary: true, + ...(options.signal ? { signal: options.signal } : {}), + }); + if (response.status >= 400) { + throw new ApiError(errorForStatus(response.status), 'resource request failed', 'ASSET_FAILED', response.status); + } + const bytes = response.bytes ?? new Uint8Array(); + // A Blob rather than the raw view: callers hand chapters to a frame as text + // and pages to an `` as bytes, and the copy is what makes that safe. + const copy = new Uint8Array(bytes.byteLength); + copy.set(bytes); + return new Blob([copy.buffer]); + } + + /** + * One window of a book's addressable structure. + * + * Used to jump to a chapter that is not in the loaded window: the response + * carries both the items and the group metadata, so the client can splice it + * into the whole-book ordering without re-reading every earlier window. + */ + async items(id: string, group?: number, options: RequestOptions = {}): Promise { + const query = group === undefined ? '' : `?group=${group}`; + return this.get(`/api/v1/books/${encodeURIComponent(id)}/items${query}`, options); + } + + /** + * The book's complete table of contents. + * + * A separate call from `manifest` because the two want opposite things: a + * manifest is windowed so opening a 1200-chapter book is cheap, and a table of + * contents has to be whole to be worth showing. Reading the contents off a + * manifest is the mistake this endpoint exists to prevent. + */ + async toc(id: string, options: RequestOptions = {}): Promise { + const result = await this.get<{ toc: TocEntry[] }>( + `/api/v1/books/${encodeURIComponent(id)}/toc`, + options, + ); + return result.toc; + } + async facets(options: RequestOptions = {}): Promise { return this.get('/api/v1/library/facets', options); } diff --git a/web/src/api/types.ts b/web/src/api/types.ts index aeb4c91..d86ac10 100644 --- a/web/src/api/types.ts +++ b/web/src/api/types.ts @@ -64,11 +64,79 @@ export interface BookFile { missing: number; } +/** One addressable unit of a book: a chapter, a page, a volume. */ +export interface ContentItem { + id: string; + seq: number; + title: string; + kind: 'chapter' | 'page'; + mediaType: string; + /** Opaque, format-specific reference. Pass it back unchanged. */ + href: string; + /** Byte length when the format knows it cheaply. */ + size?: number; +} + +/** A group of items: a comic volume, or a window of chapters. */ +export interface ContentGroup { + id: string; + seq: number; + title: string; + count: number; + /** + * Global `seq` of this group's first item. + * + * Present so a client holding one window can place it in the whole-book + * ordering without re-reading every previous group's count — which is exactly + * how a jump lands on the wrong chapter. + */ + offset: number; +} + +/** + * The addressable structure of a book. + * + * `kind` tells the client how to render it; `group` is present only when the + * request narrowed the items to one window. + */ +export interface BookContent { + kind: 'reflowable' | 'paged' | 'text' | 'document' | 'single-image'; + total: number; + groups: ContentGroup[]; + items: ContentItem[]; + group?: number; +} + +/** + * One entry of a book's own navigation. + * + * `href` is the same opaque reference the manifest's items carry, so a jump from + * the contents panel is addressed the same way a saved position is. + */ +export interface TocEntry { + href: string; + title: string; + level: number; + spine?: number; +} + export interface Manifest { book: Book; contentUrl: string; coverUrl: string | null; files: BookFile[]; + /** + * The addressable structure, already narrowed to the first window. + * + * Present since the manifest took over `/items`, which is what makes opening a + * book one round trip. Absent only for a format that exposes no structure at + * all, in which case the client downloads the file. + */ + content?: BookContent | null; + kind?: BookContent['kind']; + total?: number; + groups?: ContentGroup[]; + items?: ContentItem[]; } export interface Progress { diff --git a/web/src/app.ts b/web/src/app.ts index 87ac2d5..2c52c8d 100644 --- a/web/src/app.ts +++ b/web/src/app.ts @@ -2,7 +2,8 @@ import { ReaderApi, type SessionStore } from './api/client.ts'; import { ApiError } from './api/errors.ts'; import type { Book, Session } from './api/types.ts'; import { createWebPlatform } from './core/web-platform.ts'; -import { createAndroidPlatform, detectAndroidBridge } from './core/android-platform.ts'; +import { androidPageHost, createAndroidPlatform, detectAndroidBridge } from './core/android-platform.ts'; +import type { NativePageHost } from './ui/native-page.ts'; import type { Platform } from './core/platform.ts'; import { SyncEngine, type SyncStatus } from './core/sync.ts'; import { OfflineStore } from './store/offline.ts'; @@ -42,6 +43,14 @@ export class App { private shelf: ShelfScreen | null = null; private reader: ReaderScreen | null = null; + /** + * Native fixed-layout renderer, present only in the Android shell. + * + * Held here rather than inside ReaderScreen so that the desktop of it — the + * bridge object — is touched in one place, and so that a reader screen can be + * constructed with or without one without either path special-casing. + */ + private pageHost: NativePageHost | null = null; private screen: string | null = null; private pendingBook: Book | null = null; @@ -98,6 +107,9 @@ export class App { private async createPlatform(): Promise { const bridge = detectAndroidBridge(); if (!bridge) return createWebPlatform(this.options.defaultServerUrl ?? ''); + // The native page renderer is resolved here, once, alongside the bridge that + // provides it, so no screen has to ask whether it is running on Android. + this.pageHost = androidPageHost(bridge); return createAndroidPlatform(this.options.defaultServerUrl ?? '', bridge); } @@ -151,6 +163,7 @@ export class App { sync: this.sync, platform: this.platform, settings: this.settings, + ...(this.pageHost ? { pageHost: this.pageHost } : {}), onBack: () => this.showShelf(), onSettingsChange: (patch) => { void this.settingsStore.update(patch); diff --git a/web/src/core/android-platform.ts b/web/src/core/android-platform.ts index 9fc3ea9..f51576a 100644 --- a/web/src/core/android-platform.ts +++ b/web/src/core/android-platform.ts @@ -2,6 +2,7 @@ import type { AndroidBridge } from '../android-bridge.ts'; import { FetchTransport } from './fetch-transport.ts'; import type { Connectivity, Platform } from './platform.ts'; import { createStores } from '../store/idb.ts'; +import type { NativePageHost, NativePageRequest } from '../ui/native-page.ts'; /** * The Android platform. @@ -20,6 +21,12 @@ import { createStores } from '../store/idb.ts'; * * Everything else — storage, transport, the sync engine — is shared, which is * the entire point of building the client this way. + * + * The one thing this file knows that the web platform does not is the native + * page renderer: on Android a comic page is drawn by the platform, not by the + * WebView. `createNativePageHost` below is the whole of that special case, and + * it is written so that a refusal from the shell is indistinguishable from a + * browser with no native path at all. */ export async function createAndroidPlatform(baseUrl: string, bridge: AndroidBridge): Promise { const stores = await createStores(); @@ -83,6 +90,16 @@ function safeDeviceLabel(bridge: AndroidBridge): string { */ export const MIN_SHELL_VERSION = 1; +/** + * First shell version with a native page renderer (`renderPage`/`hidePage`). + * + * Separate from `MIN_SHELL_VERSION` on purpose. The minimum is what the client + * cannot work without; this is an optimisation it can live without, so an older + * APK keeps working and simply draws every page in the WebView. Refusing to run + * on an old shell would be the wrong trade for a speed-up. + */ +export const NATIVE_PAGE_SHELL_VERSION = 2; + export function detectAndroidBridge(): AndroidBridge | null { if (typeof window === 'undefined') return null; const bridge = window.ReaderAndroid; @@ -93,3 +110,81 @@ export function detectAndroidBridge(): AndroidBridge | null { return null; } } + +/** + * The Android native renderer, exposed as a page host. + * + * Returns `null` when the shell is too old to have one, which is not an error: + * the view then draws every page itself, exactly as the browser does. Shell + * version 2 is the first one with `renderPage`. + */ +export function androidPageHost(bridge: AndroidBridge): NativePageHost | null { + if (!shellAtLeast(bridge, NATIVE_PAGE_SHELL_VERSION)) return null; + if (typeof bridge.renderPage !== 'function') return null; + + return { + async show(request: NativePageRequest): Promise { + if (request.mode === 'reflowable') return false; + // A PDF is declined for a reason worth stating: the shell's viewer is a + // separate screen, so drawing one would take the reader out of the app and + // lose the reading session. The WebView path stays. + if (request.mode === 'document') return false; + try { + return bridge.renderPage(serialisePage(request)) === true; + } catch { + // A bridge that throws must degrade to the WebView path, not blank the + // reading view. This is the only place that decides that, on purpose. + return false; + } + }, + hide(): void { + try { + bridge.hidePage(); + } catch { + // Nothing to undo. + } + }, + }; +} + +/** + * JSON with bytes carried as base64, which is all a JS interface can accept. + * + * `mode` is passed through rather than inferred from the media type: the client + * decided what this section *is*, and re-deriving it on the other side of the + * bridge is how two implementations of the same rule start to disagree. + */ +export function serialisePage(request: NativePageRequest): string { + const payload: Record = { + sectionId: request.sectionId, + mode: request.mode, + // Carried across so the native view honours the reader's preference rather + // than guessing: a page drawn "contain" after the reader chose "fit width" + // looks like the setting is broken. + fit: request.fit, + }; + if (request.path) payload['path'] = request.path; + if (request.mediaType) payload['mediaType'] = request.mediaType; + if (request.bytes && request.bytes.byteLength > 0) payload['bytes'] = toBase64(request.bytes); + return JSON.stringify(payload); +} + +function toBase64(bytes: Uint8Array): string { + let binary = ''; + const chunk = 0x8000; + // Chunked: `String.fromCharCode(...bytes)` on a multi-megabyte page image + // overflows the argument list and throws a RangeError, which would look like + // "the native renderer does not work on big pages". + for (let offset = 0; offset < bytes.length; offset += chunk) { + binary += String.fromCharCode(...bytes.subarray(offset, offset + chunk)); + } + return btoa(binary); +} + +function shellAtLeast(bridge: AndroidBridge, version: number): boolean { + try { + return bridge.shellVersion() >= version; + } catch { + return false; + } +} diff --git a/web/src/formats/comic.ts b/web/src/formats/comic.ts index c86b8a2..eca93f9 100644 --- a/web/src/formats/comic.ts +++ b/web/src/formats/comic.ts @@ -1,6 +1,6 @@ import { BookArchive, extensionOf } from './zip.ts'; import { sortByName } from './natural-sort.ts'; -import type { BookDoc, LoadContext, Section } from './types.ts'; +import type { BookDoc, LoadContext, RenderMode, Section } from './types.ts'; /** * Comic loader: CBZ archives and loose image directories. @@ -64,12 +64,17 @@ export async function buildComicDoc( label: pageLabel(sections.length + 1, path), image: { mediaType, bytes }, depth: 0, + render: 'image', + // The page's own address, so a native host can fetch and decode it + // directly instead of receiving a buffer it has to be handed. + path, }); } return { format, layout: 'fixed', + render: comicRenderMode(), // Manga is right-to-left, but detecting it from the archive is unreliable // and guessing wrong reverses the book. Default left-to-right and expose a // toggle; the reader can fix it in one tap, and the setting persists. @@ -115,3 +120,15 @@ export function mediaTypeForImage(path: string): string { export function isImagePath(path: string): boolean { return IMAGE_EXTENSIONS.has(extensionOf(path)); } + +/** + * Render mode for a comic document. Always `image`. + * + * A comic page is a picture, so a host with a native image pipeline should use + * it: a WebView would decode the same bytes into a renderer-process bitmap and + * run a layout pass to place it, for a result that is identical on screen and + * worse in memory. + */ +export function comicRenderMode(): RenderMode { + return 'image'; +} diff --git a/web/src/formats/detect.ts b/web/src/formats/detect.ts index 780830e..0e0ae43 100644 --- a/web/src/formats/detect.ts +++ b/web/src/formats/detect.ts @@ -14,15 +14,44 @@ import type { BookFormat } from './types.ts'; const ZIP_MAGIC = [0x50, 0x4b]; // "PK" const PDF_MAGIC = [0x25, 0x50, 0x44, 0x46]; // "%PDF" +/** "GIF8", and the two variants of the PNG signature's first half. */ +const IMAGE_MAGIC: Array<{ magic: number[]; label: BookFormat }> = [ + { magic: [0x89, 0x50, 0x4e, 0x47], label: 'image' }, // PNG + { magic: [0xff, 0xd8, 0xff], label: 'image' }, // JPEG + { magic: [0x47, 0x49, 0x46, 0x38], label: 'image' }, // GIF + { magic: [0x42, 0x4d], label: 'image' }, // BMP +]; +/** MP3/MP4-family containers are audio/video, never a book. Detected to refuse. */ +const MEDIA_MAGIC: number[][] = [ + [0x49, 0x44, 0x33], // ID3 + [0x1a, 0x45, 0xdf, 0xa3], // Matroska / WebM +]; export function detectFormat(fileName: string, bytes?: Uint8Array): BookFormat { const extension = extensionOf(fileName); if (bytes && bytes.length >= 4) { + // Magic bytes first, and *only* magic bytes for the formats that are + // unambiguous. Self-hosted libraries are full of misnamed files — a `.txt` + // that is really an EPUB from a batch conversion tool, a `.cbz` that is + // actually a JPEG somebody renamed, an `.epub` that is a PDF. Being + // confident about a format whose signature admits no alternative is how + // those files become readable instead of producing a confusing parse error. if (matches(bytes, PDF_MAGIC)) return 'pdf'; + for (const entry of IMAGE_MAGIC) { + if (matches(bytes, entry.magic)) return entry.label; + } + if (MEDIA_MAGIC.some((magic) => matches(bytes, magic))) { + // Not a book at all. Naming it `unknown` is the honest answer: the reader + // gets "this file is not a book" rather than a text decoder chewing on an + // MP3 and producing fifty thousand replacement characters. + return 'unknown'; + } if (matches(bytes, ZIP_MAGIC)) { - // A ZIP can be an EPUB or a CBZ. The extension decides, and when it does - // not, the presence of the EPUB container document does. + // A ZIP admits two answers, so here the extension does get a vote — but + // only as the tie-breaker between two formats that are both plausible for + // the same bytes. When the extension is neither, the container document + // inside decides. if (extension === 'cbz' || extension === 'cbr') return 'cbz'; if (extension === 'epub') return 'epub'; return looksLikeEpub(bytes) ? 'epub' : 'cbz'; diff --git a/web/src/formats/epub.ts b/web/src/formats/epub.ts index 735a2ff..b57fcca 100644 --- a/web/src/formats/epub.ts +++ b/web/src/formats/epub.ts @@ -103,6 +103,9 @@ export async function loadEpub(ctx: LoadContext): Promise { return { format: 'epub', layout: 'reflowable', + // Reflow cannot be delegated: the WebView is the only engine here that can + // lay out the book's own CSS, and that is the product's core claim. + render: 'reflowable', direction: pkg.direction, sections, toc, diff --git a/web/src/formats/pdf.ts b/web/src/formats/pdf.ts index bfdb056..1ca7003 100644 --- a/web/src/formats/pdf.ts +++ b/web/src/formats/pdf.ts @@ -1,4 +1,4 @@ -import type { BookDoc } from './types.ts'; +import type { BookDoc, RenderMode } from './types.ts'; /** * PDF loader. @@ -22,6 +22,7 @@ export async function loadPdf(): Promise { return { format: 'pdf', layout: 'fixed', + render: pdfRenderMode(), direction: 'ltr', // A PDF is one document, not a list of sections: the browser viewer owns // navigation inside it. One section keeps the progress model uniform. @@ -39,3 +40,14 @@ export async function loadPdf(): Promise { orderedByBook: true, }; } + +/** + * Render mode for a PDF. Always `document`. + * + * A PDF is one document, and both the browser and Android already ship a viewer + * for it that is better than anything this project would write. The host is told + * so rather than being handed bytes to guess from. + */ +export function pdfRenderMode(): RenderMode { + return 'document'; +} diff --git a/web/src/formats/remote-zip.ts b/web/src/formats/remote-zip.ts new file mode 100644 index 0000000..df68f25 --- /dev/null +++ b/web/src/formats/remote-zip.ts @@ -0,0 +1,356 @@ +/** + * Read-only ZIP access over HTTP byte ranges. + * + * The whole point of windowed loading is that a reader of a 1200-chapter omnibus + * downloads one chapter, not a 50MB archive. That only holds if the client can + * read the archive without having all of it, so this is the client-side half of + * the server's `Accept-Ranges`. + * + * Two decisions carry the design: + * + * 1. **The central directory is read once, then reused.** A ZIP's index is at the + * end of the file; every entry lookup after that is arithmetic on buffers + * already held. Re-reading the directory per chapter would turn one range + * request into two per turn. + * 2. **`deflate` entries are inflated per read, `stored` entries are sliced.** + * A `stored` page can be handed to the browser as a subrange (which is why + * `resourceUrl` reports the offset rather than the bytes); a `deflate` entry + * has to be inflated in JavaScript before anything can look at it, and the + * server says honestly which is which so we never guess. + * + * The fallback matters as much as the fast path: a server (or a proxy) that + * ignores `Range` must not break the reader, so every read verifies it got the + * bytes it asked for and falls back to a whole-file download when it did not. + */ + +const EOCD_SIGNATURE = 0x06054b50; +const EOCD64_LOCATOR_SIGNATURE = 0x07064b50; +const EOCD64_SIGNATURE = 0x06064b50; +const CENTRAL_SIGNATURE = 0x02014b50; +const LOCAL_SIGNATURE = 0x04034b50; +const STORED = 0; +const DEFLATED = 8; + +/** How much of the tail to search for the end-of-central-directory record. */ +const EOCD_SEARCH_LIMIT = 66_000; + +export interface ZipEntry { + path: string; + /** Compression method; only `store` (0) and `deflate` (8) are understood. */ + method: number; + compressedSize: number; + uncompressedSize: number; + /** + * Offset of the entry's *local header*, not its data. + * + * The data offset is only knowable after reading that header (it carries its + * own name/extra lengths, which the central directory does not repeat), so it + * is resolved lazily in `dataRange`. Storing a guessed offset here was the + * first version of this file, and it silently served the wrong bytes on any + * archive whose local extra fields differed from its central ones. + */ + headerOffset: number; +} + +/** Bytes fetched from a URL, with the ability to ask for subranges. */ +export interface RangeSource { + size(): Promise; + /** Reads `[start, end]` inclusive. Rejects when the server cannot serve it. */ + read(start: number, end: number): Promise; + /** Reads the whole file. Used only when ranges are unavailable. */ + whole(): Promise; +} + +export class HttpRangeSource implements RangeSource { + private knownSize: number | null = null; + private wholeFile: Uint8Array | null = null; + + constructor( + private readonly fetchBytes: (range?: { start: number; end: number }) => Promise<{ + bytes: Uint8Array; + total?: number; + ranged: boolean; + }>, + ) {} + + async size(): Promise { + if (this.knownSize !== null) return this.knownSize; + if (this.wholeFile) return this.wholeFile.byteLength; + const probe = await this.fetchBytes({ start: 0, end: 0 }); + if (!probe.ranged || probe.total === undefined) { + // No range support: the only honest answer is to hold the file. + await this.whole(); + return this.knownSize ?? 0; + } + this.knownSize = probe.total; + return probe.total; + } + + async read(start: number, end: number): Promise { + if (end < start) return new Uint8Array(); + if (this.wholeFile) return this.wholeFile.subarray(start, end + 1); + const result = await this.fetchBytes({ start, end }); + if (!result.ranged) { + // A server that answered a range request with the whole body. Correct, if + // wasteful; keep it and satisfy every later read from memory. + this.wholeFile = result.bytes; + this.knownSize = result.bytes.byteLength; + return result.bytes.subarray(start, end + 1); + } + if (result.total !== undefined) this.knownSize = result.total; + // A short read is not something to paper over: it means the range was + // ignored, and returning fewer bytes than asked would corrupt the inflate. + if (result.bytes.byteLength !== end - start + 1) { + throw new Error(`range ${start}-${end} returned ${result.bytes.byteLength} bytes`); + } + return result.bytes; + } + + async whole(): Promise { + if (this.wholeFile) return this.wholeFile; + const result = await this.fetchBytes(); + this.wholeFile = result.bytes; + this.knownSize = result.bytes.byteLength; + return result.bytes; + } +} + +/** + * A ZIP archive read over a `RangeSource`. + * + * Entries are addressable by path and by the order they appear in the central + * directory (which is the archive's own order, not sorted). + */ +export class RemoteZip { + /** Local headers already parsed, so a chapter read does not repeat the work. */ + private readonly dataOffsets = new Map(); + + private constructor( + private readonly source: RangeSource, + private readonly entries: Map, + private readonly order: ZipEntry[], + ) {} + + static async open(source: RangeSource): Promise { + const size = await source.size(); + if (size === 0) throw new Error('empty archive'); + const tailLength = Math.min(size, EOCD_SEARCH_LIMIT); + const tail = await source.read(size - tailLength, size - 1); + const eocdOffset = findEocd(tail); + if (eocdOffset < 0) throw new Error('not a zip archive: no end-of-central-directory record'); + + const view = new DataView(tail.buffer, tail.byteOffset, tail.byteLength); + let centralSize = view.getUint32(eocdOffset + 12, true); + let centralOffset = view.getUint32(eocdOffset + 16, true); + + // ZIP64 stores sizes that do not fit in 32 bits in a separate record; the + // locator in front of the EOCD points at it. A self-hosted comic library is + // exactly where a >4GB archive turns up. + const locatorOffset = eocdOffset - 20; + if (locatorOffset >= 0 && view.getUint32(locatorOffset, true) === EOCD64_LOCATOR_SIGNATURE) { + const eocd64 = await source.read( + Number(view.getBigUint64(locatorOffset + 8, true)), + Number(view.getBigUint64(locatorOffset + 8, true)) + 55, + ); + const view64 = new DataView(eocd64.buffer, eocd64.byteOffset, eocd64.byteLength); + if (view64.getUint32(0, true) === EOCD64_SIGNATURE) { + centralSize = Number(view64.getBigUint64(40, true)); + centralOffset = Number(view64.getBigUint64(48, true)); + } + } + + const directory = await source.read(centralOffset, centralOffset + centralSize - 1); + const entries = parseCentralDirectory(directory); + if (entries.length === 0) throw new Error('zip central directory is empty'); + return new RemoteZip( + source, + new Map(entries.map((entry) => [entry.path, entry])), + entries, + ); + } + + /** + * Absolute file offset of an entry's data. + * + * Reads the 30-byte local header to learn its variable-length fields. Cached + * per path: a page turn should not cost a round trip to recompute an offset + * that cannot change while the file is open. + */ + async dataOffset(path: string): Promise { + const cached = this.dataOffsets.get(path); + if (cached !== undefined) return cached; + const entry = this.entry(path); + if (!entry) return null; + const header = await this.source.read(entry.headerOffset, entry.headerOffset + 29); + const view = new DataView(header.buffer, header.byteOffset, header.byteLength); + if (view.getUint32(0, true) !== LOCAL_SIGNATURE) { + throw new Error(`bad local header for ${path}`); + } + const nameLength = view.getUint16(26, true); + const extraLength = view.getUint16(28, true); + const offset = entry.headerOffset + 30 + nameLength + extraLength; + this.dataOffsets.set(path, offset); + return offset; + } + + /** Entry paths in archive order. */ + names(): string[] { + return this.order.map((entry) => entry.path); + } + + entry(path: string): ZipEntry | undefined { + return this.entries.get(path) ?? this.entries.get(normaliseZipPath(path)); + } + + /** Whether the entry's bytes are byte-addressable inside the file. */ + isSeekable(path: string): boolean { + return this.entry(path)?.method === STORED; + } + + /** + * The raw byte range of a `stored` entry. + * + * This is what lets a native host or a plain `` fetch one page with a + * range request instead of having the whole page copied through JavaScript. + */ + async resourceUrl(path: string, url: string): Promise { + const entry = this.entry(path); + if (!entry || entry.method !== STORED) return null; + const start = await this.dataOffset(path); + if (start === null) return null; + return `${url}#bytes=${start}-${start + entry.compressedSize - 1}`; + } + + /** Entry contents, inflating when the entry is compressed. */ + async bytes(path: string, limit = Number.POSITIVE_INFINITY): Promise { + const entry = this.entry(path); + if (!entry) return null; + if (entry.uncompressedSize > limit) { + throw new Error(`entry ${path} exceeds the read limit`); + } + const start = await this.dataOffset(path); + if (start === null) return null; + const compressed = await this.source.read(start, start + entry.compressedSize - 1); + if (entry.method === STORED) return compressed; + if (entry.method !== DEFLATED) throw new Error(`unsupported zip compression method ${entry.method}`); + return inflateRaw(compressed, entry.uncompressedSize); + } + + async text(path: string): Promise { + const bytes = await this.bytes(path); + return bytes ? new TextDecoder('utf-8').decode(bytes) : null; + } +} + +function findEocd(tail: Uint8Array): number { + const view = new DataView(tail.buffer, tail.byteOffset, tail.byteLength); + for (let offset = tail.byteLength - 22; offset >= 0; offset -= 1) { + if (view.getUint32(offset, true) === EOCD_SIGNATURE) return offset; + } + return -1; +} + +function parseCentralDirectory(directory: Uint8Array): ZipEntry[] { + const view = new DataView(directory.buffer, directory.byteOffset, directory.byteLength); + const entries: ZipEntry[] = []; + let offset = 0; + while (offset + 46 <= directory.byteLength) { + if (view.getUint32(offset, true) !== CENTRAL_SIGNATURE) break; + const method = view.getUint16(offset + 10, true); + let compressedSize = view.getUint32(offset + 20, true); + let uncompressedSize = view.getUint32(offset + 24, true); + const nameLength = view.getUint16(offset + 28, true); + const extraLength = view.getUint16(offset + 30, true); + const commentLength = view.getUint16(offset + 32, true); + let localOffset = view.getUint32(offset + 42, true); + + const nameBytes = directory.subarray(offset + 46, offset + 46 + nameLength); + const path = new TextDecoder('utf-8').decode(nameBytes); + + // ZIP64 escape values: the real numbers live in the extra field. + if ( + compressedSize === 0xffffffff || + uncompressedSize === 0xffffffff || + localOffset === 0xffffffff + ) { + const extraStart = offset + 46 + nameLength; + const extra = directory.subarray(extraStart, extraStart + extraLength); + const zip64 = readZip64Extra(extra); + compressedSize = zip64.compressedSize ?? compressedSize; + uncompressedSize = zip64.uncompressedSize ?? uncompressedSize; + localOffset = zip64.localOffset ?? localOffset; + } + + // The data offset needs the local header's own name/extra lengths, which are + // only in the local header — so this is stored as the header offset and + // resolved on read by `dataOffset`, computed here from the local header we + // are allowed to read lazily. Reading it eagerly would cost one request per + // entry; a local header is read only for the entries actually opened. + entries.push({ + path, + method, + compressedSize, + uncompressedSize, + headerOffset: localOffset, + }); + offset += 46 + nameLength + extraLength + commentLength; + } + return entries; +} + +function readZip64Extra(extra: Uint8Array): { + compressedSize?: number; + uncompressedSize?: number; + localOffset?: number; +} { + const view = new DataView(extra.buffer, extra.byteOffset, extra.byteLength); + let offset = 0; + while (offset + 4 <= extra.byteLength) { + const id = view.getUint16(offset, true); + const size = view.getUint16(offset + 2, true); + if (id === 0x0001) { + let cursor = offset + 4; + const result: { uncompressedSize?: number; compressedSize?: number; localOffset?: number } = {}; + if (cursor + 8 <= offset + 4 + size - 4) { + result.uncompressedSize = Number(view.getBigUint64(cursor, true)); + cursor += 8; + } + if (cursor + 8 <= offset + 4 + size - 2) { + result.compressedSize = Number(view.getBigUint64(cursor, true)); + cursor += 8; + } + if (cursor + 8 <= offset + 4 + size) { + result.localOffset = Number(view.getBigUint64(cursor, true)); + } + return result; + } + offset += 4 + size; + } + return {}; +} + +/** Inflates a raw-deflate stream with the platform's own decompressor. */ +async function inflateRaw(data: Uint8Array, expectedSize: number): Promise { + const format = 'deflate-raw'; + const stream = new Blob([toArrayBuffer(data)]).stream().pipeThrough( + new DecompressionStream(format as CompressionFormat), + ); + const buffer = await new Response(stream).arrayBuffer(); + const bytes = new Uint8Array(buffer); + if (expectedSize > 0 && bytes.byteLength !== expectedSize) { + throw new Error(`inflated ${bytes.byteLength} bytes, expected ${expectedSize}`); + } + return bytes; +} + +function toArrayBuffer(bytes: Uint8Array): ArrayBuffer { + const copy = new ArrayBuffer(bytes.byteLength); + new Uint8Array(copy).set(bytes); + return copy; +} + +function normaliseZipPath(path: string): string { + return decodeURIComponent(path).replace(/^\.?\//, ''); +} + +export { STORED, DEFLATED, LOCAL_SIGNATURE }; diff --git a/web/src/formats/text-encoding.ts b/web/src/formats/text-encoding.ts index edd5170..f70ab14 100644 --- a/web/src/formats/text-encoding.ts +++ b/web/src/formats/text-encoding.ts @@ -6,9 +6,22 @@ * declaration. Decoding those as UTF-8 produces replacement characters on * nearly every line — the file becomes unreadable rather than slightly wrong. * - * The strategy is validate-then-fallback, not guessing: a byte sequence that - * decodes as strict UTF-8 is by construction almost never valid GB18030 text of - * this kind, so the check is reliable in the direction that matters. + * Two decisions run through this file: + * + * 1. **Strict UTF-8 first, always.** A byte sequence that decodes as strict + * UTF-8 is by construction almost never valid GB18030 text of this kind, so + * the check is reliable in the direction that matters. Every other encoding + * is only considered after it fails. + * + * 2. **Between GB18030 and Big5, pick the better decode rather than a fixed + * order.** Most legacy Chinese TXT is Simplified (GB18030), but not all of + * it, and the two codecs overlap heavily: the same bytes usually decode + * successfully as both, so "try GB18030, then Big5" never reaches the second + * candidate. The bytes that expose the difference are the ones that land in + * each decoder's gap — they come out as private-use or replacement + * characters, or as kana out of a page of Han text. Scoring both decodes on + * that evidence and taking the cleaner one is what makes Traditional Big5 + * readable without giving up the Simplified majority. */ const GB18030 = 'gb18030'; @@ -99,18 +112,106 @@ export function decodeText(bytes: Uint8Array): DecodedText { return { text: decodeStrict(typed, 'utf-8'), encoding: 'utf-8', guessed: false }; } - // No BOM and not valid UTF-8: it is one of the CJK legacy encodings. Try - // Simplified Chinese first (by far the most common for these files), then - // Traditional, then give up on a lenient decode. - for (const encoding of [GB18030, BIG5]) { - const text = decodeStrict(typed, encoding); - if (text && countReplacementChars(text) === 0) { - return { text, encoding, guessed: true }; - } - } + // No BOM and not valid UTF-8: it is one of the CJK legacy encodings. + const best = chooseLegacyEncoding(typed); + if (best) return { text: best.text, encoding: best.encoding, guessed: true }; + + // Nothing decodes cleanly. Return *something* — a book with a few mangled + // characters beats an error screen — and mark it guessed so the UI offers the + // manual override, which is the only thing that can rescue a genuinely odd + // file. return { text: decodeLenient(typed, 'utf-8'), encoding: 'utf-8', guessed: true }; } +/** + * Picks between GB18030 and Big5 by decoding with both and scoring the result. + * + * Returns `null` when neither survives, leaving the caller to fall back rather + * than committing to a decode that is visibly broken. + */ +function chooseLegacyEncoding(bytes: Uint8Array): { text: string; encoding: string } | null { + const candidates: Array<{ text: string; encoding: string; score: number }> = []; + for (const encoding of [GB18030, BIG5]) { + const text = decodeStrict(bytes, encoding); + if (!text) continue; + const score = scoreDecode(text); + if (score === null) continue; + // GB18030 wins ties. It is both the far more common source for these files + // and the superset of the two, so an ambiguous byte sequence is more likely + // to be Simplified than Traditional. + candidates.push({ text, encoding, score: score + (encoding === GB18030 ? 1 : 0) }); + } + if (candidates.length === 0) return null; + candidates.sort((left, right) => right.score - left.score); + const best = candidates[0]!; + // A clean decode scores exactly zero. Requiring a positive score would reject + // the best possible answer, so the bar is "at least as clean as no damage at + // all" — anything below that shows damage the manual override can do better. + return best.score >= 0 ? { text: best.text, encoding: best.encoding } : null; +} + +/** + * Scores a decode by how much damage it shows. Higher is better; `null` means + * "give up on this candidate". + * + * The penalties are ordered by how strong the evidence is: + * + * - Replacement characters mean the decoder hit an invalid sequence. One is + * fatal evidence that this is the wrong codec. + * - Private-use characters are GB18030's escape hatch for bytes it defines but + * does not map to a real character. A page of Chinese text containing them + * is a mis-decode. + * - Fullwidth/halfwidth compatibility forms (`﹍`, `Ａ`) come from the same + * corner of the CJK blocks and almost never appear in book prose. + * - Kana in a wall of Han text is the tell that Big5 bytes are being read as + * GB18030: several Big5 sequences land in the hiragana block. A Japanese + * book is legitimate, but it would also be valid UTF-8 in practice, so it + * never reaches this function. + * + * A candidate is dropped outright when its damage exceeds a proportion of its + * length, rather than an absolute count: a 4 MB novel with a handful of + * genuinely malformed bytes is still the right decode, while a 200-character + * chapter with one is a wrong one. + */ +function scoreDecode(text: string): number | null { + if (text.length === 0) return null; + let score = 0; + let damage = 0; + for (const char of text) { + const code = char.codePointAt(0)!; + if (code === 0xfffd) { + damage += 5; + } else if (inPrivateUse(code)) { + damage += 3; + } else if (isCompatibilityForm(code)) { + damage += 2; + } else if (isKana(code)) { + damage += 1; + } + } + if (damage > CONCERNING_DAMAGE_RATIO * text.length) return null; + score -= damage; + return score; +} + +function inPrivateUse(code: number): boolean { + return (code >= 0xe000 && code <= 0xf8ff) || (code >= 0xf0000 && code <= 0x10fffd); +} + +function isCompatibilityForm(code: number): boolean { + return code >= 0xfe30 && code <= 0xfe4f; +} + +function isKana(code: number): boolean { + return (code >= 0x3040 && code <= 0x309f) || (code >= 0x30a0 && code <= 0x30ff); +} + +/** + * Share of a text that may look like damage before the decode is rejected + * outright, so a wrong codec is never preferred merely because it is long. + */ +const CONCERNING_DAMAGE_RATIO = 0.01; + function decodeStrict(bytes: Uint8Array, encoding: string): string { try { return new TextDecoder(encoding, { fatal: true }).decode(bytes); @@ -133,13 +234,4 @@ function decodeLenient(bytes: Uint8Array, encoding: string): string { } } -function countReplacementChars(text: string): number { - let count = 0; - for (const char of text) { - if (char === '\uFFFD') count += 1; - if (count > 8) return count; - } - return count; -} - export { hasUtf8Bom }; diff --git a/web/src/formats/txt.ts b/web/src/formats/txt.ts index 0829493..25e12df 100644 --- a/web/src/formats/txt.ts +++ b/web/src/formats/txt.ts @@ -127,6 +127,7 @@ export function loadTxt(ctx: LoadContext, options: TxtOptions = {}): TxtSplitRes doc: { format: 'txt', layout: 'reflowable', + render: 'reflowable', direction: 'ltr', sections, toc, diff --git a/web/src/formats/types.ts b/web/src/formats/types.ts index faef7d6..da8cfd6 100644 --- a/web/src/formats/types.ts +++ b/web/src/formats/types.ts @@ -34,6 +34,23 @@ export interface Section { imageUrl?: string; /** Depth in the navigation tree, 0 for top level. */ depth: number; + /** + * How this section should be drawn, when the host can choose. + * + * Omitted means "let the host decide from the document as a whole": the value + * is set per section only where sections within one book can differ, which + * today means nowhere — the book-level default below covers it. It is per + * section rather than per book because mixed-layout EPUBs (a fixed page of + * plates inside a reflowable novel) are a real thing this model has to be able + * to express later without changing the contract. + */ + render?: RenderMode; + /** + * Path this section's bytes can be fetched from, when the section is one + * addressable file. This is what lets a native host fetch a page itself + * instead of being handed a buffer by the web layer. + */ + path?: string; } export interface TocEntry { @@ -44,10 +61,37 @@ export interface TocEntry { export type Layout = 'reflowable' | 'fixed'; +/** + * What a host is being asked to render for one section. + * + * This exists for exactly one host: the Android shell. The product decision is + * that a WebView renders EPUB (faithful reflow is the differentiator, and it has + * to be one implementation), while fixed-layout pages — comic images, PDF — are + * better served by the platform's own decoders. Handing a 4K scan to a WebView + * means a full browser layout pass and a bitmap in the renderer process for what + * is, in the end, "show this image"; the native path skips both. + * + * Declared here rather than in the bridge so that both hosts agree on the shape, + * and so a browser host can ignore it entirely. + * + * - `reflowable` — a chapter of markup; only a WebView can lay it out. + * - `image` — a single page image. Native where available. + * - `document` — a whole document the platform can open itself (PDF). + */ +export type RenderMode = 'reflowable' | 'image' | 'document'; + export interface BookDoc { format: BookFormat; /** Reflowable pages can be re-paginated; fixed ones are one screen per section. */ layout: Layout; + /** + * Default render mode for this book's sections. + * + * `reflowable` for anything whose layout only a browser engine can produce — + * EPUB, TXT. `image`/`document` for fixed-layout books, which is how the + * Android shell knows it may render natively instead of paying for a WebView. + */ + render: RenderMode; /** Reading direction, from the EPUB spine / comic metadata. */ direction: 'ltr' | 'rtl'; sections: Section[]; @@ -59,6 +103,26 @@ export interface BookDoc { orderedByBook: boolean; } +/** + * A book whose section bodies are fetched on demand. + * + * The reader drives this rather than fetching the whole file, which is what keeps + * opening a 1200-chapter book at one chapter of bytes. It lives in the format + * layer because "which bytes are this chapter" is a format question — an EPUB + * answers it with an archive path, a comic with a page index. + */ +export interface StagedBook { + /** Load one section's body, filling in `html` or `image` in place. */ + loadSection(index: number): Promise
; + /** + * Swap in a newly fetched window. + * + * Returns the local index of `spine` within the new window, or -1 when that + * window does not contain it. + */ + loadWindow(content: unknown, spine: number): number; +} + export interface LoadContext { /** Raw book bytes. */ bytes: Uint8Array; diff --git a/web/src/formats/windowed.ts b/web/src/formats/windowed.ts new file mode 100644 index 0000000..b48b782 --- /dev/null +++ b/web/src/formats/windowed.ts @@ -0,0 +1,184 @@ +/** + * Staged documents: a book that is read one window at a time. + * + * This is the client half of "服务端分章节" — the reader is handed the addressable + * structure (chapters, pages, volumes) and fetches one unit at a time, so opening + * a 1200-chapter omnibus costs one chapter of bytes and never the file. + * + * The design has three load-bearing parts. + * + * ### 1. A staged document is a `BookDoc`, not a second interface + * + * Everything downstream — pagination, gestures, progress, the table of contents — + * is written against `BookDoc`. A parallel type would fork all of it. So a + * `StagedDoc` *is* a `BookDoc` whose sections are placeholders, plus the two + * things the reader needs to drive it: `loadSection` and a complete `toc`. + * + * ### 2. Locators are section ids, and section ids are the server's refs + * + * A window is a transport boundary; a locator must survive crossing one. The + * server's `href` is stable across windows (`xhtml:OEBPS/ch1.xhtml`), so it is + * what a saved position is matched on — never a spine index, which means a + * different chapter in a different window. + * + * ### 3. A window is fetched when the reader asks for something outside it + * + * Windows are not a cache to be filled in the background; they are on-demand. + * `RemoteZip` is the exception: for an EPUB, reading one chapter means reading + * the archive's central directory once and then one entry, which is what keeps + * the open cost at one chapter rather than one archive. + */ + +import type { BookContent, ContentItem } from '../net/api.ts'; +import { RemoteZip, type RangeSource } from './remote-zip.ts'; +import type { BookDoc, Section, TocEntry } from './types.ts'; + +/** + * How a staged section's bytes are obtained. + * + * `read` returns the chapter's markup already rewritten so its own relative + * resource references resolve — the server does that rewriting for EPUB, and a + * client-side staged EPUB has to do the same thing for itself. + */ +export interface SectionLoader { + /** Section markup for a reflowable section, or a page's bytes. */ + read(item: ContentItem): Promise<{ html?: string; image?: { mediaType: string; bytes: Uint8Array } }>; +} + +export interface StagedDocOptions { + kind: BookContent['kind']; + /** Every chapter of the book, so the table of contents is complete. */ + toc: TocEntry[]; + /** The window that is loaded right now. */ + content: BookContent; + /** Fetches a window by group index, for a jump outside the loaded one. */ + windowFor?(group: number): Promise; + loader: SectionLoader; + /** True when section order comes from the book rather than the file name. */ + orderedByBook?: boolean; +} + +/** A `BookDoc` whose section bodies are fetched on demand. */ +export interface StagedDoc extends BookDoc { + /** Load one section's body, filling in its `html`/`image` in place. */ + loadSection(index: number): Promise
; + /** + * Replace the loaded window. + * + * Returns the local index of `spine` within the new window, or -1 when the new + * window does not contain it. + */ + setWindow(content: BookContent, spine: number): number; + /** Whole-book index of the loaded window's first section. */ + windowOffset(): number; +} + +/** + * Build a staged document from a windowed manifest. + * + * `sections` is exactly the window's items; `toc` is the whole book. Keeping + * those two separate is what stops the transport boundary leaking into the + * navigation UI — a table of contents that read "第 1 章 – 第 40 章" because that + * was the size of a window is the bug this split exists to prevent. + */ +export function createStagedDoc(options: StagedDocOptions): StagedDoc { + const sections: Section[] = options.content.items.map((item) => ({ + id: item.href, + label: item.title, + depth: 0, + render: item.kind === 'page' ? 'image' : 'reflowable', + path: item.href, + })); + + const base: BookDoc = { + format: formatFor(options.kind), + layout: options.kind === 'paged' || options.kind === 'single-image' ? 'fixed' : 'reflowable', + render: + options.kind === 'paged' || options.kind === 'single-image' + ? 'image' + : options.kind === 'document' + ? 'document' + : 'reflowable', + direction: 'ltr', + sections, + toc: options.toc, + styles: [], + resources: new Map(), + orderedByBook: options.orderedByBook ?? true, + }; + + let offset = + options.content.groups[options.content.group ?? 0]?.offset ?? options.content.items[0]?.seq ?? 0; + + const doc: StagedDoc = { + ...base, + async loadSection(index: number): Promise
{ + const section = doc.sections[index]; + const item = currentItems()[index]; + if (!section || !item) return null; + // Already loaded: a re-style or a resize asks for the same section twice + // and must not re-fetch it. + if (section.html !== undefined || section.image !== undefined) return section; + const loaded = await options.loader.read(item); + if (loaded.html !== undefined) section.html = loaded.html; + if (loaded.image !== undefined) section.image = loaded.image; + return section; + }, + setWindow(content: BookContent, spine: number): number { + // The offset comes from the group the response actually carries. A + // response that omits `group` (a server that chose not to window, or a + // hand-built one in a test) still has to answer correctly, so the offset + // is derived from the items' own `seq` rather than assumed to be zero. + const index = content.group ?? 0; + const group = content.groups[index]; + offset = group?.offset ?? content.items[0]?.seq ?? 0; + current = content; + const items = currentItems(); + doc.sections.length = 0; + for (const item of items) { + doc.sections.push({ + id: item.href, + label: item.title, + depth: 0, + render: item.kind === 'page' ? 'image' : 'reflowable', + path: item.href, + }); + } + const local = items.findIndex((item) => item.seq === spine); + return local; + }, + windowOffset(): number { + return offset; + }, + }; + + let current = options.content; + function currentItems(): ContentItem[] { + return current.items; + } + + return doc; +} + +function formatFor(kind: BookContent['kind']): BookDoc['format'] { + switch (kind) { + case 'paged': + return 'cbz'; + case 'text': + return 'txt'; + case 'document': + return 'pdf'; + case 'single-image': + return 'image'; + default: + return 'epub'; + } +} + +/** Whether a manifest's kind can be read a window at a time by a client. */ +export function isStagedKind(kind: BookContent['kind'] | undefined): boolean { + return kind === 'reflowable' || kind === 'paged' || kind === 'text'; +} + +export { RemoteZip }; +export type { RangeSource }; diff --git a/web/src/ui/native-page.ts b/web/src/ui/native-page.ts new file mode 100644 index 0000000..0887b66 --- /dev/null +++ b/web/src/ui/native-page.ts @@ -0,0 +1,70 @@ +import type { RenderMode } from '../formats/types.ts'; + +/** + * The seam a native host uses to draw a fixed-layout page itself. + * + * Present because of a product decision with a measurable cost: in a WebView, a + * comic page or a scanned PDF page costs a decode into the renderer process, a + * full style/layout pass, and a bitmap that lives as long as the page does. On + * Android the platform already has a decoded-image pipeline and a PDF renderer, + * so the same page can be drawn without either. EPUB stays in the WebView — its + * layout is the differentiator and cannot be reproduced natively. + * + * The interface is deliberately narrow and the failure mode is explicit: a host + * either draws the page and says so, or declines and the DOM path runs. That is + * what keeps this an optimisation rather than a second renderer with its own + * set of bugs — there is exactly one implementation of every layout, and the + * native path only ever replaces "show these bytes as a picture". + */ +export interface NativePageRequest { + sectionId: string; + /** What the section is, so the host can refuse the modes it cannot draw. */ + mode: RenderMode; + /** Address of the section's bytes, when it has one. Preferred over `bytes`. */ + path?: string; + /** Media type, when known. */ + mediaType?: string; + /** + * Bytes already resolved by the web layer. + * + * For a comic inside a CBZ there is no URL the native side could fetch — the + * page lives inside an archive — so the resolved buffer is passed instead of a + * path. It is a copy across the bridge, which still costs less than a WebView + * layout pass, and it is the only option that works for archives. + */ + bytes?: Uint8Array; + /** + * How the page should sit in the viewport: the whole page, or fitted to width. + * + * Passed rather than re-derived natively because it is a *reader preference* + * (`SettingsStore.fit`), not a property of the page. A native renderer that + * guessed would ignore the setting the reader just changed. + * + * Only `contain` is drawn natively. `fit width` is handled by the caller + * declining before it asks — see `ReaderView` — because cropping a page + * silently loses panels and a native scrolling surface would be a second + * renderer. + */ + fit: 'contain' | 'width'; +} + +export interface NativePageHost { + /** Draws the page. Resolves false when the host declines and the caller must. */ + show(request: NativePageRequest): Promise; + /** Removes a previously drawn page. Safe to call when none is shown. */ + hide(): void; +} + +/** + * Wraps the Android bridge as a page host. + * + * Kept here rather than in `android-platform.ts` because the view is the only + * consumer, and the translation between "section" and "native view" belongs next + * to the code that knows what a section is. + */ +export function createNativePageHost(invoke: (request: NativePageRequest) => Promise, onHide: () => void): NativePageHost { + return { + show: (request) => invoke(request), + hide: onHide, + }; +} diff --git a/web/src/ui/reader-screen.ts b/web/src/ui/reader-screen.ts index 5ee1d2d..af32f45 100644 --- a/web/src/ui/reader-screen.ts +++ b/web/src/ui/reader-screen.ts @@ -1,7 +1,7 @@ import { ApiError } from '../api/errors.ts'; import type { ReaderApi } from '../api/client.ts'; import type { Book, Manifest, Note } from '../api/types.ts'; -import { loadBook, isImagePath, extensionOf, type BookDoc } from '../formats/index.ts'; +import { loadBook, isImagePath, extensionOf } from '../formats/index.ts'; import type { OfflineStore } from '../store/offline.ts'; import type { SyncEngine } from '../core/sync.ts'; import type { Platform } from '../core/platform.ts'; @@ -9,6 +9,21 @@ import { ReaderView, type Position, type ViewSettings } from './reader-view.ts'; import { attachGestures } from './gestures.ts'; import { clear, el, percent } from './dom.ts'; import type { AppSettings } from '../store/settings.ts'; +import { createStagedDoc, isStagedKind } from '../formats/windowed.ts'; +import type { BookDoc } from '../formats/types.ts'; +import type { TocEntry } from '../api/types.ts'; + +/** + * Chapters per window, matching the server's own constant. + * + * Duplicated rather than fetched because the client needs it to *compute* which + * window holds a chapter, and a round trip to learn a constant would defeat the + * purpose. It matches `CHAPTER_WINDOW` in the server's epub handler; a mismatch + * costs one extra request, never a wrong chapter, because the response is always + * the authority on what it contains. + */ +const CHAPTER_WINDOW = 40; +import type { NativePageHost } from './native-page.ts'; export interface ReaderScreenOptions { api: ReaderApi; @@ -16,6 +31,14 @@ export interface ReaderScreenOptions { sync: SyncEngine; platform: Platform; settings: AppSettings; + /** + * Native renderer for fixed-layout pages, when the host provides one. + * + * Optional because the browser has none and must not be given a stub that + * pretends otherwise: `undefined` means "draw everything here", which is the + * behaviour the H5 build has always had. + */ + pageHost?: NativePageHost; onBack(): void; onSettingsChange(patch: Partial): void; onSignedOut(): void; @@ -132,6 +155,21 @@ export class ReaderScreen { try { this.manifest = await this.options.api.manifest(book.id); + if (token !== this.loadingToken) return; + + // A staged book is read through the windowed contract and never downloads + // the file. That is the whole answer to "客户端需要下载完整的书籍，那消耗太大": + // this manifest already carries the first window, so opening the book has + // cost exactly one window of metadata. + const staged = await this.openStaged(book, token); + if (staged) { + this.doc = staged.doc; + this.buildView(); + this.hideStatus(); + await this.restorePosition(book, token); + return; + } + const bytes = await this.fetchBookBytes(book, this.manifest); if (token !== this.loadingToken) return; @@ -150,6 +188,80 @@ export class ReaderScreen { } } + /** + * Open the book through the windowed contract, when the server offers one. + * + * Returns null when the book has no addressable structure (an old server, a + * format that exposes none), and the caller falls back to downloading the + * file. That fallback is not a courtesy: it is what keeps this client working + * against a server that predates the endpoint, which self-hosted users do not + * upgrade on time. + * + * An EPUB is the case where the window is worth the most — a 1200-chapter + * omnibus would otherwise be a 50MB download to show one page of chapter one. + */ + private async openStaged(book: Book, token: number): Promise<{ doc: BookDoc } | null> { + const manifest = this.manifest; + const content = manifest?.content; + if (!content || !isStagedKind(content.kind)) return null; + + let toc: TocEntry[]; + try { + toc = await this.options.api.toc(book.id); + } catch { + // The manifest's own items are a worse table of contents but a usable one, + // and refusing to open the book because a contents list failed would be + // the wrong trade. + toc = (content.items ?? []).map((item) => ({ + href: item.href, + title: item.title, + level: 0, + spine: item.seq, + })); + } + if (token !== this.loadingToken) return null; + + const doc = createStagedDoc({ + kind: content.kind, + toc: toc.map((entry) => ({ id: entry.href, label: entry.title, depth: entry.level })), + content, + orderedByBook: true, + loader: { + read: (item) => this.readStagedSection(book, item), + }, + }); + + // The view drives window changes through this hook, so a jump to chapter 900 + // of a 1200-chapter book loads window 22 rather than doing nothing. + (doc as BookDoc & { staged?: unknown }).staged = doc; + return { doc }; + } + + /** + * One section's bytes, by the server's own reference. + * + * `ref` is opaque and passed through unchanged: the client must not build it + * from an index, because the whole reason it is a reference is that indices + * mean different chapters in different windows. + */ + private async readStagedSection( + book: Book, + item: { href: string; kind: string; mediaType: string }, + ): Promise<{ html?: string; image?: { mediaType: string; bytes: Uint8Array } }> { + const cacheKey = `section:${book.id}:${item.href}`; + const cached = await this.options.platform.blobs.get(cacheKey); + const blob = cached ? new Blob([toArrayBuffer(cached)]) : await this.options.api.asset(book.id, item.href); + if (!cached) { + // Cached per section, not per book, so a partially read book is partially + // readable offline and re-opening one does not re-fetch what was read. + await this.options.platform.blobs.put(cacheKey, new Uint8Array(await blob.arrayBuffer())); + } + if (item.kind === 'page') { + return { image: { mediaType: item.mediaType, bytes: new Uint8Array(await blob.arrayBuffer()) } }; + } + return { html: await blob.text() }; + } + dispose(): void { this.loadingToken += 1; this.flushProgress(); @@ -162,6 +274,10 @@ export class ReaderScreen { this.view = null; this.doc = null; this.manifest = null; + // A native page view is a sibling of the WebView's content, so it does not + // go away with the DOM this screen owns. Leaving it would put a comic page + // on top of the shelf. + this.options.pageHost?.hide(); this.element.remove(); } @@ -246,6 +362,7 @@ export class ReaderScreen { this.view = new ReaderView({ container: this.stage, doc, + ...(this.options.pageHost ? { pageHost: this.options.pageHost } : {}), onPositionChange: (position) => this.onPosition(position), onChapterChange: (_index, section) => { this.chapterLabel.textContent = section.label; @@ -429,16 +546,42 @@ export class ReaderScreen { if (!this.settingsPanel.hidden) this.tocPanel.hidden = true; } + /** + * The contents panel. + * + * Fetched from `/toc` rather than read off the manifest, because the manifest + * is windowed and a table of contents assembled from one window lists exactly + * that window — a 1200-chapter book showed "第 1 章 – 第 40 章". The server + * keeps the two apart on purpose (see docs/api.md); this is the client half of + * that arrangement. + * + * A book whose TOC cannot be fetched falls back to the window's own labels, + * which is worse but not useless, and never leaves the panel empty when the + * book does have chapters. + */ private async renderToc(): Promise { clear(this.tocList); - const entries = this.view?.chapterLabels() ?? []; + const book = this.book; + if (!book) return; + + let entries: Array<{ id: string; label: string; depth: number }> = []; + try { + const toc = await this.options.api.toc(book.id); + entries = toc.map((entry) => ({ id: entry.href, label: entry.title, depth: entry.level })); + } catch { + entries = this.view?.chapterLabels() ?? []; + } + if (entries.length === 0) entries = this.view?.chapterLabels() ?? []; for (const entry of entries) { const button = el('button', { text: entry.label, attrs: { type: 'button', 'data-section': entry.id }, on: { click: () => { - void this.view?.openLocator(`${entry.id}:0`); + // A contents entry may point at a chapter outside the loaded + // window, so the jump has to go through the whole-book path rather + // than resolve a local index — that is what `goToChapterRef` adds. + void this.goToChapterRef(entry.id); this.tocPanel.hidden = true; this.setChromeVisible(false); }, @@ -454,6 +597,36 @@ export class ReaderScreen { } } + /** + * Jump to a contents entry, loading the window that holds it when necessary. + * + * A position inside the loaded window is a local index change; one outside it + * means fetching that window first. Without this, tapping chapter 900 of a + * 1200-chapter book does nothing at all, which is how a windowed reader turns + * from fast into broken. + */ + private async goToChapterRef(ref: string): Promise { + if (this.view?.openLocator(`${ref}:0`)) { + await this.view.openLocator(`${ref}:0`); + return; + } + const book = this.book; + if (!book) return; + try { + // The TOC's `spine` is the whole-book index the windowing is keyed on. + const toc = await this.options.api.toc(book.id); + const spine = toc.find((entry) => entry.href === ref)?.spine; + if (spine === undefined) return; + const group = Math.floor(spine / CHAPTER_WINDOW); + const content = await this.options.api.items(book.id, group); + if (this.view?.loadWindow(content, spine)) { + await this.view.openLocator(`${ref}:0`); + } + } catch { + this.setStatus('error', '无法跳到这一章'); + } + } + private highlightToc(sectionId: string): void { for (const item of this.tocList.querySelectorAll('li')) { const matches = item.dataset['section'] === sectionId; @@ -630,6 +803,19 @@ function addDebouncedListener( }; } +/** + * Copies bytes into a fresh ArrayBuffer. + * + * `Blob` will not take a `Uint8Array` view over a larger buffer, and a fetch + * response's bytes are exactly that — passing the view directly would store the + * whole response behind a section-sized slice. + */ +function toArrayBuffer(bytes: Uint8Array): ArrayBuffer { + const copy = new ArrayBuffer(bytes.byteLength); + new Uint8Array(copy).set(bytes); + return copy; +} + function hasTextSelection(root: HTMLElement): boolean { const selection = window.getSelection(); if (!selection || selection.isCollapsed) return false; diff --git a/web/src/ui/reader-view.ts b/web/src/ui/reader-view.ts index 4c6d569..1ede84e 100644 --- a/web/src/ui/reader-view.ts +++ b/web/src/ui/reader-view.ts @@ -1,4 +1,6 @@ -import type { BookDoc, Section } from '../formats/types.ts'; +import type { BookDoc, Section, StagedBook } from '../formats/types.ts'; +import type { BookContent } from '../net/api.ts'; +import type { NativePageHost } from './native-page.ts'; import { ResourceResolver, hydrateResources } from './resources.ts'; import { createBookHost, extractBody, extractInlineStyles, sanitiseInjectedContent, type BookShadowHost } from './shadow.ts'; import { bookPercentage, formatLocator, parseLocator } from './locator.ts'; @@ -9,6 +11,15 @@ export interface ReaderViewOptions { /** Called on every position change, throttled by the caller via the view. */ onPositionChange?: (position: Position) => void; onChapterChange?: (index: number, section: Section) => void; + /** + * Native renderer for fixed-layout pages, when the host has one. + * + * Absent in a browser, where the WebView *is* the only renderer. Present on + * Android, where a comic page or a PDF is better drawn by the platform. The + * view does not care which: it asks the host to draw a section and falls back + * to its own DOM path whenever the host declines. + */ + pageHost?: NativePageHost; } export interface Position { @@ -54,6 +65,8 @@ export class ReaderView { private readonly container: HTMLElement; private readonly host: BookShadowHost; private readonly resolver: ResourceResolver; + /** Set when the book is read a window at a time; see `loadWindow`. */ + private readonly staged: StagedBook | null; private settings: ViewSettings; private sectionIndex = 0; private sectionOffset = 0; @@ -65,6 +78,9 @@ export class ReaderView { constructor(options: ReaderViewOptions) { this.options = options; this.doc = options.doc; + // Set by the reader screen once it has built a staged document; absent for a + // book that is read whole. + this.staged = (options.doc as BookDoc & { staged?: StagedBook }).staged ?? null; this.container = options.container; this.settings = { ...DEFAULT_SETTINGS, direction: options.doc.direction }; this.host = createBookHost(); @@ -79,6 +95,17 @@ export class ReaderView { return { ...this.settings }; } + /** + * The shadow host holding the book's content. + * + * Exposed for tests and for the host that wants to measure the reading + * surface; the content itself stays inside the shadow root, which is the + * point. + */ + get elementHost(): BookShadowHost { + return this.host; + } + get sectionCount(): number { return this.doc.sections.length; } @@ -105,10 +132,28 @@ export class ReaderView { requestAnimationFrame(() => this.restoreOffset()); } + /** + * Replace the loaded window of a staged book. + * + * Optional: a book read whole (the fallback when a server has no addressable + * structure) simply never calls it. Returns false when the book is not staged + * or the new window does not hold `spine`, so a caller can report honestly + * rather than silently landing on the wrong chapter. + */ + loadWindow(content: BookContent, spine: number): boolean { + if (!this.staged) return false; + return this.staged.loadWindow(content, spine) >= 0; + } + /** Opens a section by index, optionally at a fractional offset within it. */ async open(index: number, offset = 0): Promise { const section = this.doc.sections[index]; if (!section) return; + + // A staged section has no body until it is asked for. Doing it here, rather + // than in the loader, is what keeps "opening a book" and "showing a chapter" + // from both needing to know about windows. + if (this.staged) await this.staged.loadSection(index).catch(() => null); this.sectionIndex = index; this.sectionOffset = Math.min(1, Math.max(0, offset)); this.releaseObjectUrl(); @@ -201,6 +246,7 @@ export class ReaderView { // ---- rendering per layout ---- private async renderReflowable(section: Section): Promise { + this.options.pageHost?.hide(); const raw = section.html ?? ''; const body = extractBody(raw); const inlineStyles = extractInlineStyles(raw); @@ -219,6 +265,33 @@ export class ReaderView { } private async renderFixed(section: Section): Promise { + // A native host is offered the page first. It declines for PDF (the platform + // viewer is a separate screen, not an embeddable view) and for anything it + // cannot decode, and then the DOM path below runs unchanged. + // `fit width` is a request to fill the viewport horizontally, which a + // centred native image cannot do without cropping or scrolling. Both of + // those are worse than the WebView path, so it is not offered. + if (this.options.pageHost && this.settings.fit === 'contain') { + const mode = section.render ?? this.doc.render; + const drawn = await this.options.pageHost.show({ + sectionId: section.id, + mode, + ...(section.path ? { path: section.path } : {}), + ...(section.image ? { mediaType: section.image.mediaType } : {}), + ...(section.image ? { bytes: section.image.bytes } : {}), + fit: this.settings.fit, + }); + if (drawn) { + // The native view is a sibling of the book host, not inside it, so the + // flow has to be emptied or the previous page stays behind it. + this.host.setContent('', []); + this.host.flow.replaceChildren(); + return; + } + } + + this.options.pageHost?.hide(); + const wrapper = document.createElement('div'); wrapper.className = 'fixed-page'; wrapper.setAttribute('data-fit', this.settings.fit); diff --git a/web/test/detect-and-resolve.test.ts b/web/test/detect-and-resolve.test.ts index c046a9d..7f8ea5f 100644 --- a/web/test/detect-and-resolve.test.ts +++ b/web/test/detect-and-resolve.test.ts @@ -98,4 +98,36 @@ describe('detectFormat edge cases', () => { it('detects an image by extension when there are no bytes yet', () => { expect(detectFormat('scan.JPG')).toBe('image'); }); + + it('believes the bytes over an extension that cannot be right', () => { + // The realistic case, not a hypothetical one: a batch conversion tool writes + // a JPEG and names it after the source page. Trusting the extension means the + // PNG loader runs on JPEG bytes and the reader sees a broken image. + expect(detectFormat('page-012.png', jpeg())).toBe('image'); + expect(detectFormat('chapter.xhtml', png())).toBe('image'); + }); + + it('refuses a non-book that happens to have a book extension', () => { + // An audio file named `.txt` used to be handed to the text decoder, which + // produced fifty thousand replacement characters rather than "this is not a + // book". Naming it unknown is the honest answer. + expect(detectFormat('track.txt', new Uint8Array([0x49, 0x44, 0x33, 0x03]))).toBe('unknown'); + expect(detectFormat('video.epub', new Uint8Array([0x1a, 0x45, 0xdf, 0xa3]))).toBe('unknown'); + }); + + it('still lets the extension break the tie between two plausible zip formats', () => { + // The ZIP signature admits EPUB and CBZ, so this is the one case where the + // name is still evidence rather than noise. + const zipMagic = new Uint8Array([0x50, 0x4b, 0x03, 0x04]); + expect(detectFormat('some-comic.cbz', zipMagic)).toBe('cbz'); + expect(detectFormat('some-book.epub', zipMagic)).toBe('epub'); + }); }); + +function png(): Uint8Array { + return new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]); +} + +function jpeg(): Uint8Array { + return new Uint8Array([0xff, 0xd8, 0xff, 0xe0, 0x00, 0x10]); +} diff --git a/web/test/encoding.test.ts b/web/test/encoding.test.ts index fa90108..24fa6bd 100644 --- a/web/test/encoding.test.ts +++ b/web/test/encoding.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest'; import { decodeText, isValidUtf8 } from '../src/formats/text-encoding.ts'; -import { gb18030Bytes, utf8 } from './helpers/fixtures.ts'; +import { big5Bytes, gb18030Bytes, utf8 } from './helpers/fixtures.ts'; /** * Encoding detection is the difference between a readable TXT and a screen of @@ -71,6 +71,36 @@ describe('decodeText', () => { expect(result.guessed).toBe(true); }); + it('chooses Big5 over GB18030 when GB18030 produces damage and Big5 does not', () => { + // The whole reason the two candidates are scored instead of tried in order: + // these bytes also decode "successfully" as GB18030, so a fixed order would + // never reach Big5. + const bytes = big5Bytes('第一章 測試內容這是一段中文文本用來判斷亂碼'); + expect(isValidUtf8(bytes)).toBe(false); + // Guard the premise of the test: GB18030 does decode them without throwing. + expect(new TextDecoder('gb18030').decode(bytes)).not.toBe(''); + const result = decodeText(bytes); + expect(result.encoding).toBe('big5'); + expect(result.text).toBe('第一章 測試內容這是一段中文文本用來判斷亂碼'); + expect(result.guessed).toBe(true); + }); + + it('still prefers GB18030 for Simplified bytes, which also decode as Big5', () => { + // The other half of the same overlap: Simplified GB18030 bytes are accepted + // by the Big5 decoder too, and are only distinguishable by the damage they + // produce there. If the scoring were wrong in the other direction the + // majority of Chinese TXT libraries would break. + const bytes = gb18030Bytes('第一章 起始'); + const result = decodeText(bytes); + expect(result.encoding).toBe('gb18030'); + expect(result.text).toBe('第一章 起始'); + }); + + it('marks the result as guessed so the UI can offer an override', () => { + const result = decodeText(gb18030Bytes('测试')); + expect(result.guessed).toBe(true); + }); + it('always returns something for undecodable bytes rather than throwing', () => { const garbage = new Uint8Array([0x81, 0x82, 0x83, 0xff, 0xfe, 0x00]); const result = decodeText(garbage); diff --git a/web/test/helpers/fixtures.ts b/web/test/helpers/fixtures.ts index ab6c36c..20c55dd 100644 --- a/web/test/helpers/fixtures.ts +++ b/web/test/helpers/fixtures.ts @@ -179,6 +179,35 @@ export function gb18030Bytes(text: string): Uint8Array { return new Uint8Array(bytes); } +/** + * Big5 byte sequences for the same reason, and for one more. + * + * Big5 is not a curiosity here: it is the encoding of a real share of + * Traditional-Chinese libraries, and — unlike GB18030 — its byte sequences + * usually *also* decode as valid GB18030. That overlap is exactly what the + * detection heuristic has to resolve, so these bytes are the test. + */ +export const BIG5_SAMPLES: Record = { + // 第一章 起始 + '第一章 起始': [0xb2, 0xc4, 0xa4, 0x40, 0xb3, 0xb9, 0x20, 0xb0, 0x5f, 0xa9, 0x6c], + // 測試 + 測試: [0xb4, 0xfa, 0xb8, 0xd5], + // 中文內容 + 中文內容: [0xa4, 0xa4, 0xa4, 0xe5, 0xa4, 0xba, 0xae, 0x65], + // 第一章 測試內容這是一段中文文本用來判斷亂碼 + '第一章 測試內容這是一段中文文本用來判斷亂碼': [ + 0xb2, 0xc4, 0xa4, 0x40, 0xb3, 0xb9, 0x20, 0xb4, 0xfa, 0xb8, 0xd5, 0xa4, 0xba, 0xae, 0x65, 0xb3, 0x6f, + 0xac, 0x4f, 0xa4, 0x40, 0xac, 0x71, 0xa4, 0xa4, 0xa4, 0xe5, 0xa4, 0xe5, 0xa5, 0xbb, 0xa5, 0xce, 0xa8, + 0xd3, 0xa7, 0x50, 0xc2, 0x5f, 0xb6, 0xc3, 0xbd, 0x58, + ], +}; + +export function big5Bytes(text: string): Uint8Array { + const bytes = BIG5_SAMPLES[text]; + if (!bytes) throw new Error(`no Big5 fixture for ${text}`); + return new Uint8Array(bytes); +} + export function utf8(text: string): Uint8Array { return new TextEncoder().encode(text); } diff --git a/web/test/locator.test.ts b/web/test/locator.test.ts index 2951c36..c1e1320 100644 --- a/web/test/locator.test.ts +++ b/web/test/locator.test.ts @@ -6,6 +6,7 @@ function doc(sections: Array>): BookDoc { return { format: 'epub', layout: 'reflowable', + render: 'reflowable', direction: 'ltr', sections: sections.map((section, index) => ({ id: section.id ?? `s${index}`, diff --git a/web/test/native-page.test.ts b/web/test/native-page.test.ts new file mode 100644 index 0000000..7a70142 --- /dev/null +++ b/web/test/native-page.test.ts @@ -0,0 +1,267 @@ +// @vitest-environment jsdom +import { beforeAll, describe, expect, it, vi } from 'vitest'; +import { ReaderView } from '../src/ui/reader-view.ts'; +import { createNativePageHost, type NativePageRequest } from '../src/ui/native-page.ts'; +import { androidPageHost, serialisePage } from '../src/core/android-platform.ts'; +import type { AndroidBridge } from '../src/android-bridge.ts'; +import type { BookDoc } from '../src/formats/types.ts'; + +/** + * The native page path. + * + * This is the one place where the Android client does something the H5 client + * cannot, and the property that makes it safe is the fallback: a host either + * draws the page or says no, and "no" has to put the page back in the WebView + * with nothing left over. Every test here is about that contract, because the + * failure mode it guards against — a blank screen where a page should be — is + * invisible to the type system and only shows up on a device. + */ + +beforeAll(() => { + // jsdom has no object URLs, which the DOM fallback path uses for page images. + if (!URL.createObjectURL) { + Object.defineProperty(URL, 'createObjectURL', { value: () => 'blob:stub', writable: true }); + } + if (!URL.revokeObjectURL) { + Object.defineProperty(URL, 'revokeObjectURL', { value: () => undefined, writable: true }); + } +}); + +function fixedDoc(): BookDoc { + return { + format: 'cbz', + layout: 'fixed', + render: 'image', + direction: 'ltr', + sections: [ + { id: 'p0.png', label: '第 1 页', image: { mediaType: 'image/png', bytes: new Uint8Array([1]) }, depth: 0, render: 'image', path: 'p0.png' }, + { id: 'p1.png', label: '第 2 页', image: { mediaType: 'image/png', bytes: new Uint8Array([2]) }, depth: 0, render: 'image', path: 'p1.png' }, + ], + toc: [], + styles: [], + resources: new Map(), + orderedByBook: true, + }; +} + +function docWithPdf(): BookDoc { + return { + format: 'pdf', + layout: 'fixed', + render: 'document', + direction: 'ltr', + sections: [{ id: 'pdf', label: 'PDF', html: '', depth: 0, render: 'document' }], + toc: [], + styles: [], + resources: new Map(), + orderedByBook: true, + }; +} + +/** + * The reading surface lives inside a shadow root by design, so queries have to + * go through the host element. Querying the app document would pass on a + * regression that leaked the book into the shell. + */ +function flowOf(viewInstance: ReaderView): ShadowRoot { + const host = viewInstance.elementHost?.shadowRoot; + if (!host) throw new Error('no shadow root: the book content is not where it should be'); + return host; +} + +/** + * The reading surface, reached the way a test must: through the shadow root. + */ + +function view(doc: BookDoc, pageHost?: ReturnType): ReaderView { + const container = document.createElement('div'); + document.body.append(container); + return new ReaderView({ + container, + doc, + ...(pageHost ? { pageHost } : {}), + }); +} + +describe('ReaderView with a native page host', () => { + it('hands a fixed-layout page to the host and leaves the DOM flow empty', async () => { + const requests: NativePageRequest[] = []; + const host = createNativePageHost(async (request) => { + requests.push(request); + return true; + }, () => undefined); + + const viewInstance = view(fixedDoc(), host); + await viewInstance.open(0, 0); + + expect(requests).toHaveLength(1); + expect(requests[0]!.sectionId).toBe('p0.png'); + expect(requests[0]!.mode).toBe('image'); + // A stale DOM page under the native one is the bug this asserts against: + // the reader would see the previous page's markup through any transparent + // part of the new one. + expect(flowOf(viewInstance).querySelectorAll('.fixed-page')).toHaveLength(0); + }); + + it('draws the page in the DOM when the host declines', async () => { + const host = createNativePageHost(async () => false, () => undefined); + const viewInstance = view(fixedDoc(), host); + await viewInstance.open(0, 0); + + expect(flowOf(viewInstance).querySelectorAll('.fixed-page')).toHaveLength(1); + expect(flowOf(viewInstance).querySelectorAll('.fixed-page img')).toHaveLength(1); + }); + + it('renders reflowable sections in the DOM without asking the native host', async () => { + const host = createNativePageHost(async () => true, () => undefined); + const show = vi.spyOn(host, 'show'); + const doc: BookDoc = { + format: 'epub', + layout: 'reflowable', + render: 'reflowable', + direction: 'ltr', + sections: [{ id: 'ch1.xhtml', label: '第一章', html: '

正文

', depth: 0 }], + toc: [], + styles: [], + resources: new Map(), + orderedByBook: true, + }; + const viewInstance = view(doc, host); + await viewInstance.open(0, 0); + + // EPUB layout is the product's differentiator and has exactly one + // implementation; asking the shell to draw it would be a second one. + expect(show).not.toHaveBeenCalled(); + }); + + it('does not offer a page to the host while fit width is selected', async () => { + // Native drawing centres the page; "fit width" asks for it to fill + // horizontally. A native renderer can only do that by cropping (losing + // panels) or by growing its own scroll surface (a second renderer), so the + // page stays in the WebView and the setting keeps working. + const show = vi.fn(async () => true); + const host = createNativePageHost(show, () => undefined); + const container = document.createElement('div'); + document.body.append(container); + const viewInstance = new ReaderView({ container, doc: fixedDoc(), pageHost: host }); + viewInstance.applySettings({ fit: 'width' }); + await viewInstance.open(0, 0); + + expect(show).not.toHaveBeenCalled(); + expect(flowOf(viewInstance).querySelectorAll('.fixed-page img')).toHaveLength(1); + }); + + it('does not ask the host to draw a document, which it cannot embed', async () => { + const requests: NativePageRequest[] = []; + const host = createNativePageHost(async (request) => { + requests.push(request); + return false; + }, () => undefined); + const viewInstance = view(docWithPdf(), host); + await viewInstance.open(0, 0); + + // The request is made — the view does not special-case formats — and the + // host is expected to decline, which is asserted by the DOM having drawn it. + expect(requests).toHaveLength(1); + expect(requests[0]!.mode).toBe('document'); + expect(flowOf(viewInstance).querySelectorAll('.fixed-page')).toHaveLength(1); + }); + + it('without a host, behaves exactly like the browser build', async () => { + const viewInstance = view(fixedDoc()); + await viewInstance.open(0, 0); + expect(flowOf(viewInstance).querySelectorAll('.fixed-page img')).toHaveLength(1); + }); +}); + +describe('serialisePage', () => { + it('carries bytes as base64 so a JS interface can accept them', () => { + const payload = JSON.parse(serialisePage({ + sectionId: 'p1.png', + mode: 'image', + mediaType: 'image/png', + bytes: new Uint8Array([0x89, 0x50, 0x4e, 0x47]), + fit: 'contain', + })) as Record; + + expect(payload['sectionId']).toBe('p1.png'); + expect(payload['mode']).toBe('image'); + expect(payload['mediaType']).toBe('image/png'); + expect(atob(payload['bytes']!)).toBe('\u0089PNG'); + }); + + it('omits bytes rather than sending an empty payload', () => { + const payload = JSON.parse(serialisePage({ sectionId: 'p.png', mode: 'image', fit: 'contain' })) as Record; + expect('bytes' in payload).toBe(false); + }); + + it('survives a page image larger than the argument-list limit', () => { + // `String.fromCharCode(...bytes)` throws RangeError somewhere around 100k + // arguments, which on a real device would look like "native rendering does + // not work on big pages" rather than a crash in an encoder. + const bytes = new Uint8Array(300_000).fill(7); + const payload = JSON.parse(serialisePage({ sectionId: 'big.png', mode: 'image', bytes, fit: 'contain' })) as Record; + expect(atob(payload['bytes']!).length).toBe(300_000); + }); +}); + +describe('androidPageHost', () => { + function bridge(overrides: Partial = {}): AndroidBridge { + return { + shellVersion: () => 2, + deviceLabel: () => 'Pixel 7', + connectivity: () => 'online', + watchConnectivity: () => undefined, + toast: () => undefined, + hasNetwork: () => true, + viewport: () => '{}', + cacheUsage: () => '{}', + diagnostics: () => '{}', + renderPage: () => true, + hidePage: () => undefined, + canOpenDocument: () => false, + ...overrides, + }; + } + + it('is absent on a shell that predates the renderer', () => { + // An old APK must keep working. Refusing to run would be the wrong trade for + // an optimisation. + expect(androidPageHost(bridge({ shellVersion: () => 1 }))).toBeNull(); + }); + + it('is absent when the bridge is missing the method despite its version', () => { + const partial = bridge(); + delete (partial as { renderPage?: unknown }).renderPage; + expect(androidPageHost(partial)).toBeNull(); + }); + + it('declines reflowable and document modes without calling across the bridge', async () => { + const renderPage = vi.fn(() => true); + const host = androidPageHost(bridge({ renderPage }))!; + + expect(await host.show({ sectionId: 'x', mode: 'reflowable', fit: 'contain' })).toBe(false); + expect(await host.show({ sectionId: 'x', mode: 'document', fit: 'contain' })).toBe(false); + expect(renderPage).not.toHaveBeenCalled(); + }); + + it('treats a throwing bridge as a decline rather than an error', async () => { + const host = androidPageHost( + bridge({ + renderPage: () => { + throw new Error('renderer died'); + }, + }), + )!; + // The reader must still see the page, drawn by the WebView. + await expect(host.show({ sectionId: 'p.png', mode: 'image', fit: 'contain' })).resolves.toBe(false); + }); + + it('passes an image page through and reports success', async () => { + const calls: string[] = []; + const host = androidPageHost(bridge({ renderPage: (request) => (calls.push(request), true) }))!; + expect(await host.show({ sectionId: 'p.png', mode: 'image', mediaType: 'image/png', fit: 'contain' })).toBe(true); + expect(calls).toHaveLength(1); + expect(JSON.parse(calls[0]!)['sectionId']).toBe('p.png'); + }); +}); diff --git a/web/test/reader-view.test.ts b/web/test/reader-view.test.ts index 698b57a..b636550 100644 --- a/web/test/reader-view.test.ts +++ b/web/test/reader-view.test.ts @@ -34,6 +34,7 @@ function reflowableDoc(sectionCount: number): BookDoc { return { format: 'epub', layout: 'reflowable', + render: 'reflowable', direction: 'ltr', sections: Array.from({ length: sectionCount }, (_value, index) => ({ id: `ch${index}.xhtml`, @@ -56,6 +57,7 @@ function fixedDoc(pageCount: number): BookDoc { return { format: 'cbz', layout: 'fixed', + render: 'image', direction: 'ltr', sections: Array.from({ length: pageCount }, (_value, index) => ({ id: `p${index}.png`, diff --git a/web/test/sync.test.ts b/web/test/sync.test.ts index bf55abd..1e9927d 100644 --- a/web/test/sync.test.ts +++ b/web/test/sync.test.ts @@ -212,6 +212,50 @@ describe('OfflineStore', () => { expect(snapshot.serverTime).toBe(0); }); + it('keeps a position written while its own push was in flight', async () => { + // The failure this guards against is the one this product cannot have: a lost + // reading position. It happens with a "send everything, clear everything" + // outbox — the reader turns a page between the request going out and the + // response arriving, the response clears the whole set, and the new position + // is silently never sent. A dirty-set plus a timestamp check on the way back + // is what makes the window not exist. + const store = new OfflineStore(new MemoryKv()); + const first = { bookId: 'b1', locator: 'r1:0.10:c1', percentage: 0.1, chapterTitle: 'c1', device: 'a', updatedAt: 1000 }; + await store.setProgress(first); + + // The batch that the engine is about to send. + const batch = store.outbox(); + + // In flight: the reader turns the page. + await store.setProgress({ ...first, locator: 'r1:0.20:c1', percentage: 0.2, updatedAt: 1100 }); + + await store.markDelivered(batch, 5000); + + expect(store.current.dirtyProgress).toContain('b1'); + expect(store.outbox().progress[0]!.percentage).toBe(0.2); + // The cursor is still adopted, so the next pull stays incremental. + expect(store.current.serverTime).toBe(5000); + }); + + it('clears a position that was not touched while in flight', async () => { + const store = new OfflineStore(new MemoryKv()); + await store.setProgress({ bookId: 'b1', locator: 'r1:0.5:c1', percentage: 0.5, chapterTitle: 'c1', device: 'a', updatedAt: 1000 }); + await store.markDelivered(store.outbox(), 1); + expect(store.current.dirtyProgress).toEqual([]); + expect(isOutboxEmpty(store.current)).toBe(true); + }); + + it('does not let a delivered batch clear a different book', async () => { + // Clearing by "the batch's ids" rather than by timestamps would still be + // wrong here if the ids were taken from the wrong side of the call. + const store = new OfflineStore(new MemoryKv()); + await store.setProgress({ bookId: 'b1', locator: 'x', percentage: 0.1, chapterTitle: 'c', device: 'a', updatedAt: 1000 }); + const batch = store.outbox(); + await store.setProgress({ bookId: 'b2', locator: 'x', percentage: 0.2, chapterTitle: 'c', device: 'a', updatedAt: 1000 }); + await store.markDelivered(batch, 1); + expect(store.current.dirtyProgress).toEqual(['b2']); + }); + it('serialises concurrent writes so a fast reader does not lose a position', async () => { const store = new OfflineStore(new MemoryKv()); await Promise.all( diff --git a/web/test/windowed-toc.test.ts b/web/test/windowed-toc.test.ts new file mode 100644 index 0000000..02ed216 --- /dev/null +++ b/web/test/windowed-toc.test.ts @@ -0,0 +1,77 @@ +/** + * The staged reader's two remaining silent-failure modes. + * + * Both of these were live on `main` and neither shows up as a crash: the first + * makes the contents panel list the window instead of the book, the second makes + * a jump to a chapter outside the window do nothing at all. + */ +import { test, describe } from 'node:test'; +import assert from 'node:assert/strict'; +import { createStagedDoc } from '../src/formats/windowed.ts'; +import type { BookContent, ContentItem } from '../src/net/api.ts'; + +function item(seq: number): ContentItem { + return { + id: `c${seq}`, + seq, + title: `第 ${seq + 1} 章`, + kind: 'chapter', + mediaType: 'application/xhtml+xml', + href: `xhtml:ch${seq}.xhtml`, + }; +} + +function windowAt(offset: number, count: number, total: number): BookContent { + return { + kind: 'reflowable', + total, + groups: [{ id: `spine:${offset}`, seq: offset / count, title: '窗', count, offset }], + items: Array.from({ length: count }, (_, i) => item(offset + i)), + group: offset / count, + }; +} + +describe('staged toc and window jumps', () => { + test('a jump outside the loaded window resolves to the right local index', () => { + const doc = createStagedDoc({ + kind: 'reflowable', + toc: [], + content: windowAt(0, 40, 1200), + loader: { async read() { return { html: '' }; } }, + }); + + // Chapter 900 lives in the window starting at 880. + const local = doc.setWindow(windowAt(880, 40, 1200), 900); + assert.equal(local, 20, 'local index must be spine minus the window offset'); + assert.equal(doc.sections[20]?.id, 'xhtml:ch900.xhtml'); + assert.equal(doc.windowOffset(), 880); + }); + + test('a chapter that is not in the new window reports failure rather than index 0', () => { + // The failure this pins: a naive `findIndex` returns -1, and a caller that + // treats -1 as "0" silently moves the reader to the wrong chapter. + const doc = createStagedDoc({ + kind: 'reflowable', + toc: [], + content: windowAt(0, 40, 1200), + loader: { async read() { return { html: '' }; } }, + }); + const local = doc.setWindow(windowAt(0, 40, 1200), 900); + assert.equal(local, -1); + }); + + test('the loaded window is replaced, not appended to', () => { + // Appending is the other way this goes wrong: the reader ends up with 80 + // sections for a 40-chapter window and the progress denominator doubles. + const doc = createStagedDoc({ + kind: 'reflowable', + toc: [], + content: windowAt(0, 40, 1200), + loader: { async read() { return { html: '' }; } }, + }); + assert.equal(doc.sections.length, 40); + doc.setWindow(windowAt(40, 40, 1200), 40); + assert.equal(doc.sections.length, 40); + assert.equal(doc.sections[0]?.id, 'xhtml:ch40.xhtml'); + }); +}); diff --git a/web/test/windowed.test.ts b/web/test/windowed.test.ts new file mode 100644 index 0000000..49e565d --- /dev/null +++ b/web/test/windowed.test.ts @@ -0,0 +1,292 @@ +// @vitest-environment node +import { describe, expect, it } from 'vitest'; +import { createStagedDoc } from '../src/formats/windowed.ts'; +import { HttpRangeSource, RemoteZip, type RangeSource } from '../src/formats/remote-zip.ts'; +import type { BookContent, ContentItem } from '../src/net/api.ts'; + +/** + * Staged, one-window-at-a-time reading. + * + * The property being protected is the product's own hard requirement: opening a + * book must not download the book. These tests pin the two things that decide + * whether it holds — that a section's body is fetched only when it is displayed, + * and that a position survives crossing a window boundary. + */ + +function item(index: number): ContentItem { + return { + id: `c${index}`, + seq: index, + title: `第 ${index + 1} 章`, + kind: 'chapter', + mediaType: 'application/xhtml+xml', + href: `xhtml:ch${index}.xhtml`, + }; +} + +function windowOf(from: number, count: number, total: number): BookContent { + return { + kind: 'reflowable', + total, + groups: [ + { id: `spine:${from}`, seq: from / count, title: `${from}..`, count, offset: from }, + ], + items: Array.from({ length: count }, (_, i) => item(from + i)), + group: from / count, + }; +} + +describe('staged documents', () => { + it('does not read a section until it is asked for', async () => { + const read: string[] = []; + const doc = createStagedDoc({ + kind: 'reflowable', + toc: [{ id: 'xhtml:ch0.xhtml', label: '第一章', depth: 0 }], + content: windowOf(0, 40, 1200), + loader: { + async read(it) { + read.push(it.href); + return { html: `

${it.title}

` }; + }, + }, + }); + + expect(doc.sections).toHaveLength(40); + expect(read).toEqual([]); + + const section = await doc.loadSection(3); + expect(read).toEqual(['xhtml:ch3.xhtml']); + expect(section?.html).toBe('

第 4 章

'); + }); + + it('reads a section once, however often it is displayed', async () => { + // Re-styling and resizing both re-render the current chapter, so a loader + // that re-fetched on every call would make a font-size change a network + // round trip. + let calls = 0; + const doc = createStagedDoc({ + kind: 'reflowable', + toc: [], + content: windowOf(0, 4, 4), + loader: { + async read() { + calls += 1; + return { html: '

x

' }; + }, + }, + }); + await doc.loadSection(0); + await doc.loadSection(0); + await doc.loadSection(0); + expect(calls).toBe(1); + }); + + it('keeps the table of contents whole while the window is not', async () => { + // The failure this guards against is visible in the UI: a 1200-chapter book + // whose contents panel reads "第 1 章 – 第 40 章" because that is where the + // window ended. + const doc = createStagedDoc({ + kind: 'reflowable', + toc: Array.from({ length: 1200 }, (_, i) => ({ + id: `xhtml:ch${i}.xhtml`, + label: `第 ${i + 1} 章`, + depth: 0, + })), + content: windowOf(0, 40, 1200), + loader: { async read() { return { html: '' }; } }, + }); + expect(doc.sections).toHaveLength(40); + expect(doc.toc).toHaveLength(1200); + expect(doc.toc[1199]?.label).toBe('第 1200 章'); + }); + + it('finds a whole-book position inside a newly loaded window', async () => { + const doc = createStagedDoc({ + kind: 'reflowable', + toc: [], + content: windowOf(0, 40, 1200), + loader: { async read() { return { html: '' }; } }, + }); + expect(doc.windowOffset()).toBe(0); + + // Jump to chapter 900: window 22 (900/40), local index 20. + const local = doc.setWindow(windowOf(880, 40, 1200), 900); + expect(local).toBe(20); + expect(doc.windowOffset()).toBe(880); + expect(doc.sections[20]?.id).toBe('xhtml:ch900.xhtml'); + }); + + it('loads a page image for a fixed-layout window', async () => { + const doc = createStagedDoc({ + kind: 'paged', + toc: [], + content: { + kind: 'paged', + total: 3, + groups: [{ id: 'v0', seq: 0, title: '卷', count: 3, offset: 0 }], + items: [ + { id: '0', seq: 0, title: '001.jpg', kind: 'page', mediaType: 'image/jpeg', href: 'page:0' }, + { id: '1', seq: 1, title: '002.jpg', kind: 'page', mediaType: 'image/jpeg', href: 'page:1' }, + { id: '2', seq: 2, title: '003.jpg', kind: 'page', mediaType: 'image/jpeg', href: 'page:2' }, + ], + group: 0, + }, + loader: { + async read(it) { + return { image: { mediaType: 'image/jpeg', bytes: new Uint8Array([it.seq]) } }; + }, + }, + }); + + expect(doc.layout).toBe('fixed'); + expect(doc.render).toBe('image'); + const section = await doc.loadSection(1); + expect(section?.image?.bytes[0]).toBe(1); + }); +}); + +/** A byte source backed by an in-memory buffer, with range accounting. */ +function memorySource(bytes: Uint8Array): RangeSource & { reads: Array<[number, number]> } { + const reads: Array<[number, number]> = []; + return { + reads, + async size() { + return bytes.byteLength; + }, + async read(start, end) { + reads.push([start, end]); + return bytes.subarray(start, end + 1); + }, + async whole() { + return bytes; + }, + }; +} + +describe('remote zip', () => { + it('reads only the central directory and the requested entry', async () => { + // The whole point: opening an archive must not read it. A test that only + // checked the returned bytes would pass even if the implementation slurped + // the file, so this asserts on the ranges that were actually requested. + const zip = buildStoredZip([ + ['META-INF/container.xml', ''], + ['OEBPS/ch1.xhtml', '一'], + ]); + const source = memorySource(zip); + const remote = await RemoteZip.open(source); + const beforeEntryReads = source.reads.length; + + const chapter = await remote.bytes('OEBPS/ch1.xhtml'); + expect(new TextDecoder().decode(chapter!)).toContain('一'); + + const entryReads = source.reads.slice(beforeEntryReads); + const requested = entryReads.reduce((sum, [start, end]) => sum + (end - start + 1), 0); + // Well under the archive, and more than zero: it read something, but not + // everything. + expect(requested).toBeLessThan(zip.byteLength); + expect(requested).toBeGreaterThan(0); + }); + + it('reports a stored entry as byte-addressable and a deflated one as not', async () => { + const zip = buildStoredZip([['page.jpg', 'raw-bytes']]); + const remote = await RemoteZip.open(memorySource(zip)); + expect(remote.isSeekable('page.jpg')).toBe(true); + const url = await remote.resourceUrl('page.jpg', 'https://host/api/v1/books/x/content'); + expect(url).toContain('#bytes='); + }); + + it('falls back to a whole-file read when ranges are ignored', async () => { + // A proxy that strips Range is not an exotic case, and a client that trusted + // the range response would then inflate the wrong bytes and show garbage. + const zip = buildStoredZip([['a.txt', 'hello']]); + let servedWhole = false; + const source: RangeSource = { + async size() { + servedWhole = true; + return zip.byteLength; + }, + async read(start, end) { + return zip.subarray(start, end + 1); + }, + async whole() { + return zip; + }, + }; + const remote = await RemoteZip.open(source); + expect(servedWhole).toBe(true); + expect(new TextDecoder().decode((await remote.bytes('a.txt'))!)).toBe('hello'); + }); + + it('asks for the last byte once and remembers the size', async () => { + const zip = buildStoredZip([['a.txt', 'hello']]); + let probes = 0; + const source = new HttpRangeSource(async (range) => { + if (range) { + probes += 1; + return { bytes: zip.subarray(range.start, range.end + 1), total: zip.byteLength, ranged: true }; + } + return { bytes: zip, total: zip.byteLength, ranged: false }; + }); + const size = await source.size(); + await source.size(); + expect(size).toBe(zip.byteLength); + expect(probes).toBe(1); + }); +}); + +/** Minimal `stored` ZIP writer, so the tests do not depend on a zip library. */ +function buildStoredZip(entries: Array<[string, string]>): Uint8Array { + const encoder = new TextEncoder(); + const chunks: Uint8Array[] = []; + const central: Uint8Array[] = []; + let offset = 0; + + for (const [name, body] of entries) { + const nameBytes = encoder.encode(name); + const data = encoder.encode(body); + const local = new Uint8Array(30 + nameBytes.length); + const localView = new DataView(local.buffer); + localView.setUint32(0, 0x04034b50, true); + localView.setUint16(4, 20, true); + localView.setUint16(8, 0, true); // stored + localView.setUint32(18, data.byteLength, true); + localView.setUint32(22, data.byteLength, true); + localView.setUint16(26, nameBytes.length, true); + local.set(nameBytes, 30); + chunks.push(local, data); + + const record = new Uint8Array(46 + nameBytes.length); + const recordView = new DataView(record.buffer); + recordView.setUint32(0, 0x02014b50, true); + recordView.setUint16(4, 20, true); + recordView.setUint16(6, 20, true); + recordView.setUint16(10, 0, true); + recordView.setUint32(20, data.byteLength, true); + recordView.setUint32(24, data.byteLength, true); + recordView.setUint16(28, nameBytes.length, true); + recordView.setUint32(42, offset, true); + record.set(nameBytes, 46); + central.push(record); + + offset += local.byteLength + data.byteLength; + } + + const centralSize = central.reduce((sum, part) => sum + part.byteLength, 0); + const end = new Uint8Array(22); + const endView = new DataView(end.buffer); + endView.setUint32(0, 0x06054b50, true); + endView.setUint16(8, entries.length, true); + endView.setUint16(10, entries.length, true); + endView.setUint32(12, centralSize, true); + endView.setUint32(16, offset, true); + + const all = [...chunks, ...central, end]; + const total = all.reduce((sum, part) => sum + part.byteLength, 0); + const out = new Uint8Array(total); + let cursor = 0; + for (const part of all) { + out.set(part, cursor); + cursor += part.byteLength; + } + return out; +} diff --git a/web/vite.config.ts b/web/vite.config.ts index d2d8efd..90ea861 100644 --- a/web/vite.config.ts +++ b/web/vite.config.ts @@ -41,6 +41,13 @@ export default defineConfig({ // file from one process, or vice versa, silently produces "no test suite // found". Excluding them here keeps each runner to the files it can actually // execute, instead of a green-looking run that quietly skipped half the suite. - exclude: ['test/api.test.ts', 'test/asset-url.test.ts', 'test/paginator.test.ts', 'test/window.test.ts', 'node_modules/**'], + exclude: [ + 'test/api.test.ts', + 'test/asset-url.test.ts', + 'test/paginator.test.ts', + 'test/window.test.ts', + 'test/windowed-toc.test.ts', + 'node_modules/**', + ], }, });