mirror of
https://github.com/hectorqin/reader.git
synced 2026-11-10 21:11:08 +08:00
feat(sources): 增加内置 local、OPDS 与进程插件系统
This commit is contained in:
hectorqin committed
1 parent
847f046724
commit
037bdd6880
39 files changed
+4303
-91
No files matched your search
@@ -12,6 +12,8 @@
|
||||
| 渲染层 | `web/` | TypeScript + Vite，EPUB 精排、TXT、漫画、PDF |
|
||||
| Android 壳 | `android/` | Kotlin + WebView，复用上面那一份渲染层 |
|
||||
|
||||
服务端已提供内置 local / OPDS 来源与受信任进程插件接口。OPDS 下载后复用现有书架和阅读器；插件章节目前提供开发接口，管理界面与章节书架接入仍在后续计划中。见 [插件设计与实施状态](docs/source-plugins.md) 和 [API 与示例插件安装](docs/api.md#来源与插件)。
|
||||
|
||||
---
|
||||
|
||||
## 一条命令跑起来
|
||||
|
||||
+161
-1
@@ -781,6 +781,142 @@ manifest 的每个条目还带 `ref`，即这个文件对应的**资源引用**
|
||||
|
||||
---
|
||||
|
||||
## 来源与插件
|
||||
|
||||
来源接口把本地书、OPDS 和外部章节插件统一成同一组发现与获取操作。来源实例由管理员创建；来源配置在实例级共享，登录凭据通过用户级凭据接口保存。插件运行在独立的受信任 Node 进程中，安装前必须确认它拥有服务端操作系统权限。
|
||||
|
||||
### `GET /sources/types`
|
||||
|
||||
返回当前已注册的内置和外部来源类型：
|
||||
|
||||
```json
|
||||
{
|
||||
"types": [
|
||||
{ "id": "local", "pluginId": "reader.local", "builtin": true, "label": "本地书库", "version": "1.0.0", "capabilities": ["browse", "search", "detail"] },
|
||||
{ "id": "opds", "pluginId": "reader.opds", "builtin": true, "label": "OPDS", "version": "1.0.0", "capabilities": ["browse", "search", "detail", "acquire.file"] }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### `GET /sources`
|
||||
|
||||
登录用户可以看到来源实例，响应为 `{ "sources": [...] }`；每项有 `id/pluginId/sourceType/name/enabled/descriptor`，只有管理员返回 `config`。服务端启动时自动创建 `local` 实例。
|
||||
|
||||
### `POST /sources` — 管理员
|
||||
|
||||
创建来源：
|
||||
|
||||
```http
|
||||
POST /api/v1/sources
|
||||
Authorization: Bearer <admin-token>
|
||||
Content-Type: application/json
|
||||
|
||||
{"pluginId":"reader.opds","sourceType":"opds","name":"家庭 OPDS","config":{"url":"https://books.example.test/opds","username":"reader"}}
|
||||
```
|
||||
|
||||
返回 `201 { "source": {...} }`。可指定 `id`（2～80 位字母、数字、下划线或短横线），省略则自动生成。`sourceType` 和 `pluginId` 从 `/sources/types` 读取。`local` 仅接受空配置，使用宿主 `BOOKS_DIR`。
|
||||
|
||||
`config` 不能包含密码。OPDS 的 `url` 必须是 HTTP(S) 地址；Feed 和下载链接默认限制在该 origin。需要跨 origin 获取文件时，管理员可在 config 中增加 `allowedOrigins`（如 `["https://cdn.example.test"]`）。Basic 凭据不会转发到附加 origin。当前只支持通过 `PATCH /sources/:id` 更新 `enabled`，未提供修改名称/URL/config 或删除来源的接口。
|
||||
|
||||
### 来源浏览、搜索和详情
|
||||
|
||||
```text
|
||||
GET /api/v1/sources/:id/browse?ref=<optional>&cursor=<optional>&limit=<1..200>
|
||||
GET /api/v1/sources/:id/search?q=<query>&cursor=<optional>&limit=<1..200>
|
||||
GET /api/v1/sources/:id/entries?ref=<entry-ref>
|
||||
```
|
||||
|
||||
目录响应形如 `{ "items": [...], "navigation": [...], "nextCursor": "...", "title": "..." }`；可选字段可省略。`ref` 和 `cursor` 是来源拥有的不透明值，客户端只保存并原样回传。条目返回 `ref`、标题、作者、封面和 `options`；不要从 `ref` 推断 URL 或拼接下一页地址。引用和查询参数必须为非空字符串且不超过 16,384 字符；路径中的 `publicationRef` 需要 URL 编码，路由允许最大 16,384 字符的参数。
|
||||
|
||||
### 凭据与来源启停
|
||||
|
||||
```http
|
||||
PUT /api/v1/sources/:id/credentials/password
|
||||
Authorization: Bearer <user-token>
|
||||
Content-Type: application/json
|
||||
|
||||
{"value":"用户自己的 OPDS 密码"}
|
||||
```
|
||||
|
||||
返回 `{ "ok": true }`。`value` 必须为字符串，允许空密码，最大 16,384 字符。密码按 `(sourceId, userId, key)` 隔离并以 AES-256-GCM 加密保存，普通来源列表不会回显。管理员可用 `PATCH /api/v1/sources/:id` 携带 `{ "enabled": false }` 暂停来源；已获取到 `DATA_DIR/acquired` 的书不会被删除。
|
||||
|
||||
### 获取内容、目录和章节资源
|
||||
|
||||
```http
|
||||
POST /api/v1/sources/:id/acquire
|
||||
Authorization: Bearer <user-token>
|
||||
Content-Type: application/json
|
||||
|
||||
{"entryRef":"<entry-ref>","optionId":"<option-id>"}
|
||||
```
|
||||
|
||||
内置 `local` 直接返回已有书的 `publicationId`。OPDS 文件会同步下载，成功响应为 `{ "kind":"ready", "publicationId":"<book-id>" }`；单文件上限 256 MiB，下载完成后计算真实 SHA-256，写入 `DATA_DIR/acquired`，复用现有格式解析和阅读端点，并将书加入当前用户书架。取消、超限或解析失败不会留下临时文件。
|
||||
|
||||
章节插件返回 `{ "kind":"chapters", "publicationRef":"..." }`，随后调用：
|
||||
|
||||
```text
|
||||
GET /api/v1/sources/:id/publications/:publicationRef/manifest
|
||||
GET /api/v1/sources/:id/publications/:publicationRef/resource?ref=<resource-ref>
|
||||
```
|
||||
|
||||
首个进程插件版本的资源接口只返回 JSON：章节文本用 `text`，二进制小资源用 `base64`；不会把插件返回的 HTML 作为应用页面执行。章节 publication 的书架、离线缓存和追更尚未接入。
|
||||
|
||||
### 插件管理（管理员）
|
||||
|
||||
插件包需要管理员预先放入 `DATA_DIR/plugins/<folder>`，当前安装接口只接受目录名，不上传或解压包：
|
||||
|
||||
```http
|
||||
POST /api/v1/plugins
|
||||
Authorization: Bearer <admin-token>
|
||||
Content-Type: application/json
|
||||
|
||||
{"folder":"demo-chapters","trusted":true}
|
||||
```
|
||||
|
||||
启停和卸载：
|
||||
|
||||
```text
|
||||
GET /api/v1/plugins
|
||||
PATCH /api/v1/plugins/:id body: {"enabled":true|false}
|
||||
DELETE /api/v1/plugins/:id 保留来源实例、已获取书和数据
|
||||
```
|
||||
|
||||
安装返回 `201 { "plugin": {...} }`，列表返回 `{ "plugins": [...] }`，启停返回 `{ "plugin": {...} }`，卸载返回 `{ "ok": true }`。插件状态包含 `pluginId/builtin/enabled/sourceTypes/runtime`，外部插件还带 `folder/name/version`，加载失败时有 `error`。卸载会移除注册和安装记录，保留包文件、来源实例、凭据和书籍数据。
|
||||
|
||||
禁用或插件进程故障不会删除已下载文件。当前没有包上传、在线升级或回滚接口；内置 `reader.local` 和 `reader.opds` 不可卸载、禁用或替换。已安装插件失败后可用 `PATCH ... {"enabled":true}` 重新加载。
|
||||
|
||||
### 可运行示例：demo-chapters
|
||||
|
||||
仓库中的 [`examples/plugins/demo-chapters`](../examples/plugins/demo-chapters) 是一个不联网的静态章节插件。按以下步骤验证完整调用链（服务端已启动且已有管理员令牌）：
|
||||
|
||||
```powershell
|
||||
# 在仓库根目录执行；改为服务端实际使用的 DATA_DIR 和管理员令牌。
|
||||
$readerDataDir = 'D:\reader-data'
|
||||
$readerApi = 'http://localhost:8080/api/v1'
|
||||
$readerHeaders = @{ Authorization = 'Bearer <admin-token>' }
|
||||
New-Item -ItemType Directory -Force (Join-Path $readerDataDir 'plugins')
|
||||
Copy-Item -Recurse examples/plugins/demo-chapters (Join-Path $readerDataDir 'plugins/demo-chapters')
|
||||
|
||||
Invoke-RestMethod -Method Post -Uri "$readerApi/plugins" -Headers $readerHeaders `
|
||||
-ContentType 'application/json' -Body '{"folder":"demo-chapters","trusted":true}'
|
||||
$createdSource = Invoke-RestMethod -Method Post -Uri "$readerApi/sources" -Headers $readerHeaders `
|
||||
-ContentType 'application/json' -Body '{"pluginId":"reader.source.demo","sourceType":"demo-chapters","name":"Demo chapters","config":{}}'
|
||||
$readerSourceId = $createdSource.source.id
|
||||
|
||||
Invoke-RestMethod -Uri "$readerApi/sources/$readerSourceId/browse" -Headers $readerHeaders
|
||||
Invoke-RestMethod -Uri "$readerApi/sources/$readerSourceId/entries?ref=demo-book" -Headers $readerHeaders
|
||||
Invoke-RestMethod -Method Post -Uri "$readerApi/sources/$readerSourceId/acquire" -Headers $readerHeaders `
|
||||
-ContentType 'application/json' -Body '{"entryRef":"demo-book"}'
|
||||
Invoke-RestMethod -Uri "$readerApi/sources/$readerSourceId/publications/demo-book/manifest" -Headers $readerHeaders
|
||||
Invoke-RestMethod -Uri "$readerApi/sources/$readerSourceId/publications/demo-book/resource?ref=chapter-one" -Headers $readerHeaders
|
||||
```
|
||||
|
||||
资源响应中的 `text` 应包含“这是通过独立 Node 进程提供的示例章节”。普通成员也可使用自己的令牌调用浏览与章节端点；只有安装插件、创建和启停来源需要管理员。
|
||||
|
||||
宿主来源调用预算为 60 秒，同一来源实例最多 2 个并发调用，宿主总计最多 8 个，超限立即返回 `429 RATE_LIMITED`。插件每次 RPC 预算 30 秒、单条 JSON 消息上限 2 MiB（包含 base64 开销）；超时、取消或协议错误会终止整个插件进程及其在途请求。外部进程暂不支持 `acquire.file`，大文件获取由内置 OPDS 提供。
|
||||
|
||||
---
|
||||
|
||||
## 管理员
|
||||
|
||||
### `GET /admin/users`
|
||||
@@ -807,7 +943,7 @@ manifest 的每个条目还带 `ref`，即这个文件对应的**资源引用**
|
||||
|
||||
→ `{ "providers": [ { "id": "none", "displayName": "...", "enabled": true } ] }`
|
||||
|
||||
第一版只有 `none`。在线源接入后会自动出现在这里。
|
||||
第一版只有 `none`。此处是元数据补全 provider，与 `/sources` 内容来源独立；安装书源插件不会改变该列表。
|
||||
|
||||
---
|
||||
|
||||
@@ -850,6 +986,30 @@ manifest 的每个条目还带 `ref`，即这个文件对应的**资源引用**
|
||||
| `TTS_DISABLED` | 400 | 该实例没有配置 `TTS_URL`，无法使用 HTTP 朗读 |
|
||||
| `TTS_UPSTREAM` | 400 | 上游合成服务不可达、超时、报错，或返回的不是音频 |
|
||||
| `TEXT_TOO_LONG` | 400 | 单条语句超过 800 字，请客户端先切句 |
|
||||
| `INVALID_SOURCE_CONFIG` | 400 | 来源配置不符合内置来源要求 |
|
||||
| `SOURCE_UNSUPPORTED` | 400 | 来源未声明或未实现请求能力 |
|
||||
| `SOURCE_DISABLED` | 400 | 来源实例已停用 |
|
||||
| `SOURCE_NOT_FOUND` | 404 | 来源实例不存在 |
|
||||
| `SOURCE_TYPE_NOT_FOUND` | 404 | 来源类型未安装 |
|
||||
| `ENTRY_NOT_FOUND` | 404 | 来源条目不存在 |
|
||||
| `AUTH_REQUIRED` | 401 | 来源需要凭据或凭据无法解密 |
|
||||
| `OPDS_UNSAFE_URL` | 400 | OPDS 链接跳转到未允许的 origin 或包含 URL 凭据 |
|
||||
| `RESOURCE_TOO_LARGE` | 413 | OPDS Feed 或资源超过大小限制 |
|
||||
| `ACQUISITION_TOO_LARGE` | 413 | 获取文件超过 256 MiB 上限 |
|
||||
| `INVALID_RESOURCE` | 400 | 获取内容为空、媒体类型不匹配或解析失败 |
|
||||
| `PLUGIN_TRUST_REQUIRED` | 400 | 安装受信任进程插件时未显式确认权限 |
|
||||
| `PLUGIN_UNAVAILABLE` | 404 / 503 | 来源插件未注册，或进程不可用 |
|
||||
| `RATE_LIMITED` | 429 | 上游限流或宿主来源并发达到上限 |
|
||||
| `PLUGIN_TIMEOUT` | 504 | 插件单次 RPC 超过 30 秒 |
|
||||
| `PLUGIN_PROTOCOL_ERROR` | 502 | 插件返回了不符合协议的 JSON |
|
||||
| `PLUGIN_INCOMPATIBLE` | 400 | 安装包的 API 版本或运行时不受支持 |
|
||||
| `PLUGIN_PACKAGE_NOT_FOUND` | 404 | 插件目录、清单或入口文件不存在 |
|
||||
| `PLUGIN_INVALID_MANIFEST` | 400 | 插件清单 JSON 或字段不合法 |
|
||||
| `PLUGIN_PATH_ESCAPE` | 400 | 插件目录不在 `DATA_DIR/plugins` 内 |
|
||||
| `PLUGIN_ALREADY_INSTALLED` | 409 | 插件 ID 或目录已安装 |
|
||||
| `BUILTIN_PLUGIN_IMMUTABLE` | 400 | 内置来源不能停用、卸载或替换 |
|
||||
| `SOURCE_TIMEOUT` | 504 | 宿主来源调用超过 60 秒 |
|
||||
| `SOURCE_CANCELLED` | 499 | 客户端断开或主动取消来源调用 |
|
||||
| `INTERNAL` | 500 | 服务端错误 |
|
||||
|
||||
## 客户端应当遵守的约定
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
本文记录**为什么这样设计**，尤其是那些一旦写错就很难回头的决策。
|
||||
产品设计文档（v0.2）里的约束在这里落成了实现，逐条对应。
|
||||
|
||||
来源扩展与插件系统的设计、接口及实施状态见 [来源插件系统设计](source-plugins.md)。该文档区分目标架构和当前开发切片，本页现有本地书籍身份及兼容约束继续适用。
|
||||
|
||||
---
|
||||
|
||||
## 1. 数据边界
|
||||
|
||||
@@ -0,0 +1,465 @@
|
||||
# 来源插件系统设计
|
||||
|
||||
状态：设计已确定，服务端首个切片已实现
|
||||
|
||||
适用接口：`/api/v1`，来源插件协议 1
|
||||
|
||||
更新日期：2026-09-20
|
||||
|
||||
本文定义 reader 对本地书库、OPDS 书库和远程书源的统一扩展架构。目标是让核心程序只维护一套书架、阅读、缓存和任务逻辑；内置 `local`、`opds` 两种来源；远程书源以独立插件开发、安装和升级。
|
||||
|
||||
本文区分目标架构和本轮交付。当前已实现 local、OPDS 下载入库、受信任 Node 来源插件及管理接口；外部章节提供目录和资源开发预览，尚未接入书架、离线缓存和追更。交付状态与验证见 §13，调用示例见 [API 参考](api.md#来源与插件)。
|
||||
|
||||
## 1. 目标与范围
|
||||
|
||||
### 1.1 目标
|
||||
|
||||
- 通过同一套来源接口支持本地书、OPDS 目录和远程章节书源。
|
||||
- 新增远程站点时只安装或更新插件，不修改核心路由、书架和阅读器。
|
||||
- 将来源协议复杂性封装在深模块内：调用方只理解发现、详情和获取内容。
|
||||
- 让文件内容和在线章节都进入现有 `manifest / toc / assets` 阅读协议。
|
||||
- 插件崩溃、超时或禁用时，不破坏书架、阅读进度和已经缓存的内容。
|
||||
- 插件升级保持来源引用和章节身份稳定，并支持可验证的状态迁移。
|
||||
- 通过能力声明、凭据隔离、网络权限和独立进程降低插件风险。
|
||||
|
||||
### 1.2 完整版本目标范围
|
||||
|
||||
包括：
|
||||
|
||||
- 宿主插件注册表、安装/启用/禁用/升级流程。
|
||||
- 内置 `local`：扫描 `BOOKS_DIR`，复用现有格式处理器。
|
||||
- 内置 `opds`：支持 OPDS 1.x Atom、OPDS 2 JSON、导航、分页、搜索（若目录提供）和文件获取。
|
||||
- 外部 Node.js 进程插件及版本化 RPC。
|
||||
- 远程书源的搜索、详情、目录、章节正文和图片资源。
|
||||
- 宿主统一缓存、下载任务、限流、错误码和日志。
|
||||
- 来源配置 JSON Schema 和凭据引用。
|
||||
|
||||
不包括：
|
||||
|
||||
- 插件直接替换阅读器 UI 或注入任意前端代码。
|
||||
- 不可信插件的完整操作系统级沙箱。首个切片只运行管理员明确安装的受信任插件；需要容器或系统沙箱时另行实现。
|
||||
- 自动把不同来源的版本合并为同一作品并迁移进度。先保留独立版本，后续可增加可选 `workId`。
|
||||
|
||||
## 2. 术语与设计原则
|
||||
|
||||
| 术语 | 定义 |
|
||||
| --- | --- |
|
||||
| 插件包（Plugin package） | 可安装、校验、升级的代码与声明，例如 `reader.source.example`。 |
|
||||
| 来源实例（Source instance） | 一个具体配置，例如 `/books`、某个 OPDS 地址或一个站点账号。 |
|
||||
| 来源条目（Source entry） | 来源目录中的书，以 `(sourceInstanceId, externalId)` 标识。 |
|
||||
| 出版物（Publication） | 可阅读的具体版本，拥有自己的目录、章节/文件和定位体系。 |
|
||||
| 资源（Resource） | 文件、章节正文、图片、字体等可读取内容。 |
|
||||
| 宿主（Host） | reader 服务端，负责插件调度和用户数据。 |
|
||||
|
||||
设计遵循 `docs/architecture.md` 的数据边界：`BOOKS_DIR` 是用户书库，`DATA_DIR` 存数据库、下载、缓存和插件状态。插件不直接访问宿主数据库，也不把任意路径或远程 URL 交给客户端。
|
||||
|
||||
来源接口是外部 seam。`local`、`opds` 和远程插件是该 seam 上的 adapter；宿主将鉴权、任务、缓存、资源流和错误语义放在接口之后，调用方不需要知道每个来源的协议细节。
|
||||
|
||||
## 3. 总体架构
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
UI[书库/搜索/书架] --> API[来源 HTTP API]
|
||||
API --> Host[SourceHost]
|
||||
Host --> Registry[PluginRegistry]
|
||||
Registry --> BuiltinRunner[BuiltinRunner]
|
||||
Registry --> ProcessRunner[ProcessRunner]
|
||||
BuiltinRunner --> Local[local adapter]
|
||||
BuiltinRunner --> OPDS[opds adapter]
|
||||
ProcessRunner --> Remote[外部来源插件]
|
||||
Host --> Runtime[Runtime：HTTP/凭据/缓存/任务/日志]
|
||||
Host --> Content[PublicationContent：manifest/toc/assets]
|
||||
Content --> Reader[现有阅读器]
|
||||
Local --> Content
|
||||
OPDS --> Content
|
||||
Remote --> Content
|
||||
```
|
||||
|
||||
### 3.1 模块职责
|
||||
|
||||
| 模块 | 职责 | 不负责 |
|
||||
| --- | --- | --- |
|
||||
| `PluginRegistry` | 发现插件包、校验 manifest、选择版本、记录状态 | 不执行来源业务 |
|
||||
| `SourceHost` | 将来源调用包装成统一接口；管理实例、能力、超时、任务 | 不解析具体网站规则 |
|
||||
| `BuiltinRunner` | 在宿主进程内运行受控的 `local` 和 `opds` adapter | 不加载外部任意代码 |
|
||||
| `ProcessRunner` | 启动和监督外部插件进程，执行 RPC、取消和资源流 | 不拥有书架或用户数据 |
|
||||
| `Runtime` | HTTP、凭据、命名空间存储、缓存、限流、日志、取消 | 不定义来源协议 |
|
||||
| `PublicationContent` | 将文件或章节绑定转换为统一阅读协议 | 不负责搜索和目录发现 |
|
||||
| `JobManager` | 下载、扫描、刷新目录、离线缓存、重试和进度 | 不修改插件内部状态 |
|
||||
| `Book/Shelf` | 持久化 publication、书架、进度、笔记 | 不根据书名猜测来源身份 |
|
||||
|
||||
## 4. 来源接口
|
||||
|
||||
接口以 [`server/src/sources/types.ts`](../server/src/sources/types.ts) 为准。以下摘录首个切片的 SDK 契约；可选能力通过 `capabilities` 声明，注册表拒绝声明能力却缺少对应方法的提供者。
|
||||
|
||||
```ts
|
||||
interface SourceProvider {
|
||||
readonly descriptor: SourceDescriptor;
|
||||
validateConfig?(config: unknown): void | Promise<void>;
|
||||
browse?(ctx: SourceContext, request: BrowseRequest): Promise<CatalogPage>;
|
||||
search?(ctx: SourceContext, request: SearchRequest): Promise<CatalogPage>;
|
||||
detail(ctx: SourceContext, entryRef: string): Promise<CatalogEntry>;
|
||||
acquire(ctx: SourceContext, request: AcquireRequest): Promise<Acquisition>;
|
||||
openFile?(ctx: SourceContext,
|
||||
acquisition: Extract<Acquisition, { kind: 'file' }>): Promise<ResourceResponse>;
|
||||
getManifest?(ctx: SourceContext, publicationRef: string): Promise<ManifestSnapshot>;
|
||||
readResource?(ctx: SourceContext, request: ResourceRequest): Promise<ResourceResponse>;
|
||||
}
|
||||
```
|
||||
|
||||
`SourceProvider` 的接口保持小而稳定。适配器内部可以实现规则解析、登录流程、分页协议和站点兼容逻辑；宿主只依赖这些方法的输入、输出、取消、超时和错误不变量。
|
||||
|
||||
### 4.1 目录和条目
|
||||
|
||||
```ts
|
||||
interface SourceDescriptor {
|
||||
id: string;
|
||||
label: string;
|
||||
version: string;
|
||||
capabilities: SourceCapability[];
|
||||
configSchema?: unknown;
|
||||
}
|
||||
|
||||
interface CatalogPage {
|
||||
items: CatalogEntry[];
|
||||
navigation?: NavigationEntry[];
|
||||
nextCursor?: string;
|
||||
title?: string;
|
||||
}
|
||||
|
||||
interface NavigationEntry {
|
||||
ref: string;
|
||||
title: string;
|
||||
kind?: 'catalog' | 'search' | 'collection';
|
||||
}
|
||||
|
||||
interface CatalogEntry {
|
||||
ref: string;
|
||||
title: string;
|
||||
authors?: string[];
|
||||
description?: string;
|
||||
language?: string;
|
||||
coverUrl?: string;
|
||||
options?: AcquisitionOption[];
|
||||
metadata?: Record<string, unknown>;
|
||||
}
|
||||
```
|
||||
|
||||
`entryRef`、游标和导航引用由来源解释。宿主只做不透明保存和回传，不拼接 URL，也不把它当作数据库主键。
|
||||
|
||||
### 4.2 内容获取
|
||||
|
||||
```ts
|
||||
type Acquisition =
|
||||
| { kind: 'ready'; publicationId: string }
|
||||
| { kind: 'file'; acquisitionRef: string; mediaType: string }
|
||||
| { kind: 'chapters'; publicationRef: string }
|
||||
| { kind: 'action-required'; action: AcquisitionAction };
|
||||
```
|
||||
|
||||
`Acquisition` 是提供者返回给宿主的内容绑定。`local` 可以返回已经存在的 `publicationId`；远程提供者返回文件或章节引用，由宿主分配稳定身份并处理获取流程。任何 `ready` ID 都必须由宿主验证确实存在且调用用户有权访问，不能直接信任外部插件的声明。后续引入持久化任务时，HTTP 响应可新增 `pending/jobId`，它不是首个切片的插件返回值。
|
||||
|
||||
文件内容和章节内容由宿主内部统一为：
|
||||
|
||||
```ts
|
||||
interface PublicationContent {
|
||||
manifest(ctx: ReadContext): Promise<ManifestSnapshot>;
|
||||
toc(ctx: ReadContext): Promise<TocEntry[]>;
|
||||
asset(ctx: ReadContext, ref: string): Promise<ResourceResponse>;
|
||||
}
|
||||
```
|
||||
|
||||
SDK 的 `ResourceResponse` 支持 `mediaType`、`size`（若已知）、`data`、`text` 和 `stream`。内置 OPDS 提供者通过流下载。首个外部进程只支持章节来源，声明 `acquire.file` 会被拒绝；资源使用有大小上限的 `text/base64` JSON，适合章节和小图片。HTTP 开发预览始终返回 JSON，不在应用同源下直接执行插件 HTML。后续资源通道支持背压和取消后，外部插件才适合传输大文件。
|
||||
|
||||
### 4.3 能力
|
||||
|
||||
首个切片的能力枚举：
|
||||
|
||||
```text
|
||||
browse
|
||||
search
|
||||
detail
|
||||
acquire.file
|
||||
acquire.chapters
|
||||
content.manifest
|
||||
content.resource
|
||||
content.update
|
||||
```
|
||||
|
||||
来源级能力决定 UI 是否显示入口；条目级 `options` 决定本书是否支持某种格式、下载或试读。能力缺失统一返回 `CAPABILITY_UNSUPPORTED`。
|
||||
|
||||
## 5. 插件包与 SDK
|
||||
|
||||
### 5.1 manifest
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "reader.source.example",
|
||||
"name": "示例远程书源",
|
||||
"version": "1.0.0",
|
||||
"apiVersion": 1,
|
||||
"runtime": "node",
|
||||
"entry": "dist/main.js",
|
||||
"sourceTypes": [
|
||||
{
|
||||
"id": "example",
|
||||
"label": "示例书源",
|
||||
"capabilities": ["search", "detail", "acquire.chapters", "content.manifest", "content.resource"],
|
||||
"configSchema": { "type": "object", "properties": {} }
|
||||
}
|
||||
],
|
||||
"permissions": {
|
||||
"network": { "domains": ["example.com", "*.example.com"] },
|
||||
"storage": true,
|
||||
"credentials": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
目标安装流程校验插件 ID、版本、宿主 `apiVersion`、入口文件、来源类型唯一性、JSON Schema 和权限。首个切片的 manifest 校验集中在 [`registry.ts`](../server/src/sources/registry.ts)，进程启动另行检查支持的协议版本和入口路径。语义化版本范围、包内容哈希、签名和完整权限 schema 校验属于后续包管理工作。
|
||||
|
||||
### 5.2 宿主能力
|
||||
|
||||
SDK 预留以下能力；首个切片的 `SourceContext` 中能力注入是可选的，独立进程内暂未实现宿主能力的反向 RPC，不能假设以下约束已强制生效：
|
||||
|
||||
| 能力 | 规则 |
|
||||
| --- | --- |
|
||||
| `http` | 超时、重定向检查、Cookie、限流和允许域名均由宿主执行。 |
|
||||
| `credentials` | 按用户和来源实例隔离；插件只能通过引用读取，日志自动脱敏。 |
|
||||
| `storage` | 插件/来源实例命名空间 KV，带 schema 版本；不能访问 SQLite。 |
|
||||
| `cache` | 由宿主控制配额和淘汰；缓存键含来源、账号和资源身份。 |
|
||||
| `logger` | 结构化日志、级别和脱敏由宿主控制。 |
|
||||
| `signal` | 每次调用都带取消信号、截止时间和最大响应大小。 |
|
||||
|
||||
宿主不提供前端 DOM 和数据库接口。首个切片中的受信任 Node 进程仍可使用 Node 的网络、文件和子进程能力，独立进程只提供故障隔离；系统沙箱落地后才可以强制限制这些直接调用。
|
||||
|
||||
### 5.3 RPC
|
||||
|
||||
首个切片采用一行一个 JSON 的 stdio JSON-RPC 2.0；stdout 只输出协议消息，插件日志写 stderr：
|
||||
|
||||
```text
|
||||
request = { jsonrpc: "2.0", id, method,
|
||||
params: { sourceType, context: { instance, userId }, ... } }
|
||||
response = { jsonrpc: "2.0", id, result }
|
||||
error = { jsonrpc: "2.0", id, error: { code, message, data? } }
|
||||
```
|
||||
|
||||
宿主每次来源调用预算为 60 秒，同一来源实例最多并发 2 次，宿主共 8 次，超限返回 `429 RATE_LIMITED`。插件单次 RPC 预算为 30 秒，每条 JSON 消息上限为 2 MiB（包括 base64 和协议字段）。请求取消、超时、协议错误或进程异常退出会终止 worker，全部在途调用失败；取消是硬停止，不是合作式取消。插件进入 `failed` 后可通过管理员重新启用来加载新进程。重试由调用方显式决定，不自动重复 `acquire` 等可能有副作用的调用。后续再加入进度事件、背压流和宿主能力的反向调用。
|
||||
|
||||
## 6. 内置来源
|
||||
|
||||
### 6.1 `local`
|
||||
|
||||
- 自动创建 `local` 实例，配置固定为 `{}`，使用宿主 `BOOKS_DIR`；首个切片不支持每个实例覆盖目录。
|
||||
- 由宿主扫描器负责增量检测，路径必须通过 `resolveInside()`，不跟随越界符号链接。
|
||||
- 文件格式继续复用现有 `server/src/indexer/formats` 处理器。
|
||||
- 本地文件作为 `Publication` 的 `file` 绑定；书籍身份继续遵循现有 `content_hash` 规则。
|
||||
- 文件读取继续保持原接口的缺失错误语义（如 `FILE_MISSING`）；本轮不改已有书架、进度及扫描清理规则。
|
||||
|
||||
### 6.2 `opds`
|
||||
|
||||
- 支持 OPDS 1.x Atom 和 OPDS 2 JSON；解析导航、分页、搜索入口、封面和 acquisition link。
|
||||
- 不假设每个链接都可下载。根据 `rel`、`type` 和条目状态产生 `AcquisitionOption`。
|
||||
- 首个切片支持公开或 Basic 认证的 EPUB、PDF、TXT、CBZ 下载；借阅、购买、试读等无法直接获取的关系返回 `action-required`。
|
||||
- 来源地址、用户名、附加下载 origin 由管理员配置并共享；密码通过凭据接口按用户和来源隔离，以 AES-256-GCM 加密保存。密码不写在普通配置中。
|
||||
- 导航和 Feed 请求限制在配置地址的 origin，附加下载 origin 必须列入 `allowedOrigins`；每次重定向重新校验，跨 origin 不转发 Basic 凭据。此策略允许管理员配置内网 OPDS，不等于系统级网络隔离。
|
||||
- 下载同步完成后返回 `ready`，没有后台任务 ID。临时文件写入 `DATA_DIR/acquired`，单文件最大 256 MiB；完成后计算真实 SHA-256、解析格式、原子改名并事务写入 `books`、`acquired_files`、`source_acquisitions`，加入调用用户书架。超限、取消和入库失败清理本次文件；现有书籍元数据保持不变。
|
||||
- 下载后的 EPUB/PDF 走与 `local` 相同的格式解析和阅读路径。
|
||||
|
||||
### 6.3 外部远程书源
|
||||
|
||||
目标上插件负责搜索、详情、目录快照、正文清洗和图片引用；宿主负责缓存、限流、任务、权限和阅读协议。本轮先实现搜索、详情、获取章节绑定、目录和资源的契约及开发预览；章节 publication 持久化、正文渲染清洗、离线缓存和追更仍待实施。
|
||||
|
||||
## 7. 数据模型
|
||||
|
||||
首个切片使用增量表，保留已有 `books`、`book_files`、`user_books` 和 `reading_progress`，没有新建 `publications` 或重写旧书 ID：
|
||||
|
||||
| 表 | 当前用途 |
|
||||
| --- | --- |
|
||||
| `source_instances` | 共享来源配置：`id/plugin_id/source_type/name/config_json/enabled/created_at`。 |
|
||||
| `source_credentials` | `(source_id, user_id, key)` 对应的加密凭据。 |
|
||||
| `acquired_files` | `book_id` 对应的 `DATA_DIR` 相对文件路径和大小，与扫描目录分离。 |
|
||||
| `source_acquisitions` | `(source_id, user_id, entry_ref, option_id)` 到 `book_id` 的映射。 |
|
||||
| `plugin_storage` | 宿主 KV，按来源和用户命名空间保存；外部进程尚无反向访问接口。 |
|
||||
| `installed_plugins` | 插件 ID、已放置的包目录名及启用状态。 |
|
||||
|
||||
来源实例由管理员配置，对登录用户共享。凭据、获取记录和书架属于当前用户；普通用户的来源列表不返回 `config`。禁用来源不删除已下载文件。
|
||||
|
||||
下面是后续接入在线章节时的逻辑模型，不是本轮实际 schema：
|
||||
|
||||
```text
|
||||
source_instances
|
||||
id, plugin_id, source_type, label, config_json, credential_ref,
|
||||
enabled, state, state_version, created_at, updated_at
|
||||
|
||||
source_entries
|
||||
id, source_instance_id, external_ref, metadata_json,
|
||||
last_seen_at, created_at, updated_at
|
||||
|
||||
publications
|
||||
id, source_entry_id?, content_kind, media_type, content_version,
|
||||
availability, created_at, updated_at
|
||||
|
||||
publication_resources
|
||||
id, publication_id, resource_ref, local_path?, cache_key?,
|
||||
size, media_type, checksum, state, updated_at
|
||||
|
||||
shelf_items
|
||||
user_id, publication_id, hidden, created_at, updated_at
|
||||
|
||||
reading_states
|
||||
user_id, publication_id, locator_json, percent, updated_at
|
||||
```
|
||||
|
||||
约束：
|
||||
|
||||
- `source_entries` 的唯一键是 `(source_instance_id, external_ref)`。
|
||||
- `external_ref`、OPDS URL、章节序号都不是 `publicationId`。
|
||||
- 章节必须有稳定 ID；目录顺序单独保存，插章不会让旧进度指向另一章。
|
||||
- 来源失效或插件禁用后，已缓存 publication 仍可读；卸载插件默认保留用户数据。
|
||||
- 初期不同来源版本不自动合并进度；未来的作品聚合只提供迁移建议。
|
||||
- 插件状态写入自己的 namespace，状态迁移失败时保持旧版本或回滚。
|
||||
|
||||
## 8. 生命周期与故障语义
|
||||
|
||||
### 8.1 插件生命周期
|
||||
|
||||
```text
|
||||
发现 → 校验 → 安装 → 启用 → 健康检查 → 运行
|
||||
↓
|
||||
禁用 / 故障
|
||||
```
|
||||
|
||||
当前安装流程：管理员将包放在 `DATA_DIR/plugins/<folder>`，调用安装接口并显式传 `trusted: true`。宿主校验真实路径、manifest、入口和来源类型，保存安装记录；启动时自动恢复启用的插件。启用、禁用、卸载串行执行，禁用先注销提供者再终止进程；卸载移除安装记录，保留包文件、来源实例和已有用户数据。内置 `reader.local`、`reader.opds` 不允许卸载或替换。
|
||||
|
||||
后续包上传及升级流程（尚未实现）：
|
||||
|
||||
1. 校验包、manifest、API 版本、哈希/签名和权限。
|
||||
2. 将包解压到版本目录，不覆盖当前运行版本。
|
||||
3. 启动健康检查，读取 `descriptor` 并调用 `validateConfig`。
|
||||
4. 停止接收新任务，等待或取消在途调用。
|
||||
5. 执行插件声明的状态迁移；失败则保留旧版本。
|
||||
6. 原子切换当前版本，重新加载来源实例。
|
||||
|
||||
插件禁用不删除来源实例、publication、缓存和进度。删除数据是单独的显式操作。
|
||||
|
||||
### 8.2 统一错误
|
||||
|
||||
```text
|
||||
AUTH_REQUIRED 需要登录或凭据过期
|
||||
RATE_LIMITED 来源限流
|
||||
SOURCE_CHANGED 站点结构或规则失效
|
||||
RESOURCE_GONE 远程或本地资源不存在
|
||||
CAPABILITY_UNSUPPORTED 当前条目不支持该能力
|
||||
PLUGIN_UNAVAILABLE 插件未运行或崩溃
|
||||
TIMEOUT 宿主截止时间已到
|
||||
QUOTA_EXCEEDED 缓存或下载配额不足
|
||||
```
|
||||
|
||||
该列表是完整版本的错误分类目标。当前 HTTP 响应保持 `{ "error": { "code", "message" } }`，具体已实现错误码以 [API 参考](api.md#错误码) 为准，例如 `SOURCE_TIMEOUT`、`PLUGIN_TIMEOUT`、`RATE_LIMITED`。`retryAfter/details` 未在本轮通用错误响应中实现。
|
||||
|
||||
## 9. 安全与资源边界
|
||||
|
||||
以下区分当前强制约束与后续安全工作；不能把 manifest 权限声明当作已生效的系统策略。
|
||||
|
||||
- 外部插件默认独立进程运行；插件进程崩溃不能让宿主退出。
|
||||
- 内置 OPDS 的网络请求经过宿主校验：允许 origin、重定向目标、Basic 凭据转发、超时和响应大小；普通 OPDS 链接不能跳到未配置的 origin。
|
||||
- 外部 Node 插件当前是受信任进程，仍拥有本机 Node 的网络、文件和子进程权限；manifest 权限声明和 `SourceContext` 可选能力尚未构成 OS 级网络沙箱。管理员只能安装信任的包。
|
||||
- 插件凭据不出现在普通配置、前端 DTO 和日志中。
|
||||
- 外部插件资源预览只返回 JSON 文本或 base64，不作为应用同源 HTML 执行；章节 HTML 清洗和受控 `reader-res:` 缓存属于后续阅读接入。
|
||||
- 本地和已获取文件路径通过宿主 `resolveInside()` 校验；外部插件直接读写路径的能力目前依赖管理员信任，系统沙箱属于后续工作。
|
||||
- 当前硬限制是宿主来源调用 60 秒、插件 RPC 30 秒、单条 JSON 2 MiB、文件获取 256 MiB；CPU/内存和域名权限的强制隔离属于后续工作。
|
||||
- 插件安装及来源实例配置需要管理员权限；实例配置共享，凭据按用户隔离。
|
||||
|
||||
manifest 中的权限声明用于宿主授权和展示，不能单独视为安全沙箱；运行不可信插件需要容器或操作系统策略。
|
||||
|
||||
## 10. 当前 HTTP 接口
|
||||
|
||||
均位于 `/api/v1` 并遵循现有认证和错误格式：
|
||||
|
||||
```text
|
||||
GET /plugins 插件及版本状态
|
||||
POST /plugins 安装已放入 plugins 目录的受信任插件
|
||||
PATCH /plugins/:id 启用/禁用（enabled）
|
||||
DELETE /plugins/:id 卸载注册（保留包文件和数据）
|
||||
|
||||
GET /sources/types 当前已注册来源类型
|
||||
GET /sources 来源实例列表
|
||||
POST /sources 创建来源实例
|
||||
PATCH /sources/:id 仅修改 enabled
|
||||
PUT /sources/:id/credentials/:key 设置当前用户凭据
|
||||
GET /sources/:id/browse 浏览目录
|
||||
GET /sources/:id/search 搜索
|
||||
GET /sources/:id/entries?ref=... 详情
|
||||
POST /sources/:id/acquire 获取内容，body 传 entryRef
|
||||
GET /sources/:id/publications/:publicationRef/manifest
|
||||
GET /sources/:id/publications/:publicationRef/resource?ref=...
|
||||
```
|
||||
|
||||
HTTP 返回宿主 DTO，凭据不回显。插件管理均需管理员权限；来源浏览、搜索、获取和当前用户凭据设置需登录。来源的 URL、名称和 config 更新、来源删除、插件包上传及升级尚无接口。请求参数、响应和可运行示例见 [API 参考](api.md#来源与插件)。
|
||||
|
||||
## 11. 分阶段实施
|
||||
|
||||
### 阶段一：内容 seam 与内置 local
|
||||
|
||||
- 抽取 `PublicationContent`，让现有 `/manifest`、`/toc`、`/assets` 不再直接依赖 `book_files → absPath`。
|
||||
- 将现有扫描和格式处理包装为 `local` adapter。
|
||||
- 保留旧 `bookId`、进度、笔记和书架语义，补充 publication 映射。
|
||||
- 验收：已有 EPUB/TXT/PDF/漫画书籍、阅读进度和离线缓存行为不变。
|
||||
|
||||
### 阶段二：SourceHost 与内置 OPDS
|
||||
|
||||
- 实现来源注册、实例配置、能力 DTO 和 `JobManager`。
|
||||
- 实现 OPDS 1.x/2.x 解析、分页、详情和 EPUB/PDF 下载。
|
||||
- 下载文件进入 `DATA_DIR`，复用 local 的内容解析链路。
|
||||
- 验收：OPDS 书可浏览、搜索（若服务支持）、下载、阅读和重启恢复；失败任务可重试。
|
||||
|
||||
### 阶段三：插件管理与 ProcessRunner
|
||||
|
||||
- 定义 manifest、SDK、RPC、插件目录和生命周期。
|
||||
- 增加一个最小章节插件，验证搜索→详情→目录→正文→图片完整链路。
|
||||
- 增加超时、取消、进程崩溃恢复、错误映射和插件状态迁移。
|
||||
- 验收：插件无需修改核心代码即可新增来源；禁用插件不影响已缓存内容。
|
||||
|
||||
### 阶段四：远程书源生态
|
||||
|
||||
- 实现配置表单、凭据动作、章节缓存和目录刷新。
|
||||
- 建立契约测试样例：分页、登录、限流、规则失效、插章和图片资源。
|
||||
- 验收：导入一个书源配置即可创建来源实例；更新规则只升级插件/配置，不改阅读器。
|
||||
|
||||
## 12. 验证与兼容策略
|
||||
|
||||
每个 adapter 和插件都必须通过宿主契约测试：
|
||||
|
||||
- 能力声明与实际方法一致。
|
||||
- 所有调用支持取消、超时和稳定错误码。
|
||||
- 外部引用保持可重复解析；章节 ID 在刷新后稳定。
|
||||
- 资源流不会被 JSON 编码截断或超出大小限制。
|
||||
- 凭据和缓存按用户/来源实例隔离。
|
||||
- 插件崩溃、禁用、升级失败时，书架和已缓存 publication 仍可访问。
|
||||
|
||||
宿主 API 继续遵循现有长期兼容策略：只增加字段和端点，不改变已有字段语义。数据库迁移按可回滚的小步骤提交；旧书先映射为 `local` publication，远程来源不伪造 `content_hash`。
|
||||
|
||||
## 13. 本轮实施状态与后续工作
|
||||
|
||||
本轮交付服务端扩展接口和 OPDS 下载阅读闭环。状态截至 2026-09-20；后续条目不能视为已交付。
|
||||
|
||||
| 工作 | 本轮状态 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| 来源契约 | 已实现并测试 | `server/src/sources/types.ts`，包括目录、获取、章节和资源。 |
|
||||
| 来源注册表及 manifest 校验 | 已实现并测试 | 验证能力与方法一致，校验插件 DTO、章节 ID 和顺序。 |
|
||||
| OPDS 解析、浏览、搜索、获取 | 已实现并测试 | OPDS 1.x/2.x、相对链接、OpenSearch、分页、Basic 认证和获取关系。 |
|
||||
| OPDS 文件下载入库 | 已实现并测试 | 同步下载，真实哈希、256 MiB 上限、失败清理、幂等复用、当前用户上架；复用现有阅读接口。 |
|
||||
| SourceHost、来源实例及凭据 | 已实现 | 实例由管理员配置共享，密码按来源/用户加密，调用预算 60 秒；并发限额每实例 2、全宿主 8。 |
|
||||
| 来源管理 HTTP 接口 | 已实现并测试 | 创建、列表、启停和来源调用；仅 `enabled` 可更新，不含前端配置页面。 |
|
||||
| 内置 local 与本地内容 seam | 已实现并测试 | 保留现有 book ID、书架及阅读接口，集中本地与下载文件解析。 |
|
||||
| 受信任 Node ProcessRunner | 已实现并测试 | JSON-RPC、30 秒调用预算、硬取消、异常退出、2 MiB 消息上限。 |
|
||||
| 插件安装、启停、卸载与启动恢复 | 已实现并测试 | 包由管理员预置到 `DATA_DIR/plugins`；卸载保留数据和包文件。 |
|
||||
| 最小外部来源示例 | 已实现并测试 | `examples/plugins/demo-chapters`，提供静态示例目录和正文，不联网。 |
|
||||
| 插件安装 UI、包上传、在线升级与状态回滚 | 后续 | 本轮不承诺完整插件市场和包管理。 |
|
||||
| 后台下载任务、断点恢复、JobManager | 后续 | 当前完整文件会持久化，下载过程本身没有后台任务及恢复机制。 |
|
||||
| 外部章节写入书架、离线缓存与追更 | 后续 | 先可调用插件目录/资源，再接入持久化出版物。 |
|
||||
| 反向宿主能力 RPC、凭据和配额强制隔离 | 后续 | 类型预留不等于运行策略已生效。 |
|
||||
| 容器/操作系统沙箱 | 后续 | 首个版本仅允许运行管理员信任的插件。 |
|
||||
|
||||
验证结果：在 `server` 目录运行 `npm test`，共 264 项，263 通过、0 失败、1 项因 Windows 不支持 Unix 目录权限位而明确跳过；`npm run typecheck` 与生产 `npm run build` 通过。覆盖新增来源、插件进程、管理接口、OPDS 下载、用户隔离、并发限额、参数校验与既有本地阅读回归。测试均使用本地 HTTP fixture 或示例插件，未验证特定公网 OPDS 服务或第三方采集站点。
|
||||
@@ -0,0 +1,54 @@
|
||||
import { createInterface } from 'node:readline';
|
||||
|
||||
const entry = {
|
||||
ref: 'demo-book',
|
||||
title: '插件示例书',
|
||||
authors: ['Reader'],
|
||||
description: '用于验证外部插件的发现、获取和章节阅读协议。',
|
||||
};
|
||||
|
||||
const methods = {
|
||||
validateConfig: () => null,
|
||||
browse: () => ({ title: '示例书库', items: [entry] }),
|
||||
search: ({ request }) => ({ items: entry.title.includes(request.query) ? [entry] : [] }),
|
||||
detail: ({ entryRef }) => {
|
||||
if (entryRef !== entry.ref) throw Object.assign(new Error('书籍不存在'), { code: 'RESOURCE_GONE' });
|
||||
return entry;
|
||||
},
|
||||
acquire: ({ request }) => {
|
||||
if (request.entryRef !== entry.ref) throw Object.assign(new Error('书籍不存在'), { code: 'RESOURCE_GONE' });
|
||||
return { kind: 'chapters', publicationRef: entry.ref };
|
||||
},
|
||||
getManifest: ({ publicationRef }) => {
|
||||
if (publicationRef !== entry.ref) throw Object.assign(new Error('书籍不存在'), { code: 'RESOURCE_GONE' });
|
||||
return {
|
||||
publicationRef,
|
||||
version: '1',
|
||||
items: [{ id: 'chapter-one', seq: 0, title: '第一章', kind: 'chapter', mediaType: 'text/plain', ref: 'chapter-one' }],
|
||||
};
|
||||
},
|
||||
readResource: ({ request }) => {
|
||||
if (request.publicationRef !== entry.ref || request.ref !== 'chapter-one') {
|
||||
throw Object.assign(new Error('章节不存在'), { code: 'RESOURCE_GONE' });
|
||||
}
|
||||
return { mediaType: 'text/plain; charset=utf-8', text: '第一章\n\n这是通过独立 Node 进程提供的示例章节。' };
|
||||
},
|
||||
};
|
||||
|
||||
const input = createInterface({ input: process.stdin, crlfDelay: Infinity });
|
||||
for await (const line of input) {
|
||||
let request;
|
||||
try {
|
||||
request = JSON.parse(line);
|
||||
if (request.method === '$/cancelRequest') continue;
|
||||
const method = methods[request.method];
|
||||
if (!method) throw Object.assign(new Error('不支持的方法'), { code: 'UNSUPPORTED_OPERATION' });
|
||||
const result = await method(request.params);
|
||||
process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result }) + '\n');
|
||||
} catch (error) {
|
||||
process.stdout.write(JSON.stringify({
|
||||
jsonrpc: '2.0', id: request?.id,
|
||||
error: { code: error.code ?? 'PLUGIN_ERROR', message: error.message },
|
||||
}) + '\n');
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"id": "reader.source.demo",
|
||||
"name": "Demo chapter source",
|
||||
"version": "0.1.0",
|
||||
"apiVersion": 1,
|
||||
"runtime": "node",
|
||||
"entry": "main.mjs",
|
||||
"sourceTypes": [
|
||||
{
|
||||
"id": "demo-chapters",
|
||||
"label": "示例章节书源",
|
||||
"capabilities": ["browse", "search", "detail", "acquire.chapters", "content.manifest", "content.resource"],
|
||||
"configSchema": { "type": "object", "properties": {}, "additionalProperties": false }
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
import { randomBytes } from 'node:crypto';
|
||||
import { mkdirSync, readFileSync, writeFileSync, existsSync } from 'node:fs';
|
||||
import { join, resolve } from 'node:path';
|
||||
import { isAbsolute, join, relative, resolve, sep } from 'node:path';
|
||||
|
||||
/**
|
||||
* Runtime configuration.
|
||||
@@ -98,13 +98,16 @@ function loadOrCreateSecret(dataDir: string, file = 'token.secret'): string {
|
||||
export function loadConfig(): AppConfig {
|
||||
const booksDir = resolve(process.env.BOOKS_DIR ?? '/books');
|
||||
const dataDir = resolve(process.env.DATA_DIR ?? join(process.cwd(), DATA_DIR_DEFAULT_NAME));
|
||||
mkdirSync(dataDir, { recursive: true });
|
||||
|
||||
if (booksDir === dataDir || dataDir.startsWith(`${booksDir}/`)) {
|
||||
const dataRelativeToBooks = relative(booksDir, dataDir);
|
||||
const insideBooks = dataRelativeToBooks === '' || (
|
||||
dataRelativeToBooks !== '..' && !dataRelativeToBooks.startsWith(`..${sep}`) && !isAbsolute(dataRelativeToBooks)
|
||||
);
|
||||
if (insideBooks) {
|
||||
throw new Error(
|
||||
`DATA_DIR (${dataDir}) must not live inside BOOKS_DIR (${booksDir}); the library mount is read-only.`,
|
||||
);
|
||||
}
|
||||
mkdirSync(dataDir, { recursive: true });
|
||||
|
||||
return {
|
||||
booksDir,
|
||||
|
||||
@@ -100,6 +100,48 @@ CREATE TABLE IF NOT EXISTS book_files (
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_files_book ON book_files(book_id);
|
||||
|
||||
-- Configurations contain no credentials. Built-in and installed source types
|
||||
-- share this table; disabling a source never removes acquired content.
|
||||
CREATE TABLE IF NOT EXISTS source_instances (
|
||||
id TEXT PRIMARY KEY,
|
||||
plugin_id TEXT NOT NULL,
|
||||
source_type TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
config_json TEXT NOT NULL DEFAULT '{}',
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
created_at INTEGER NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS source_credentials (
|
||||
source_id TEXT NOT NULL REFERENCES source_instances(id) ON DELETE CASCADE,
|
||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
key TEXT NOT NULL,
|
||||
encrypted_value TEXT NOT NULL,
|
||||
PRIMARY KEY (source_id, user_id, key)
|
||||
);
|
||||
-- Host-managed files are deliberately separate from the scanned books mount.
|
||||
CREATE TABLE IF NOT EXISTS acquired_files (
|
||||
book_id TEXT PRIMARY KEY REFERENCES books(id) ON DELETE CASCADE,
|
||||
rel_path TEXT NOT NULL UNIQUE,
|
||||
size INTEGER NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS source_acquisitions (
|
||||
source_id TEXT NOT NULL REFERENCES source_instances(id),
|
||||
user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
entry_ref TEXT NOT NULL,
|
||||
option_id TEXT NOT NULL DEFAULT '',
|
||||
book_id TEXT NOT NULL REFERENCES books(id) ON DELETE CASCADE,
|
||||
PRIMARY KEY (source_id, user_id, entry_ref, option_id)
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS plugin_storage (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS installed_plugins (
|
||||
plugin_id TEXT PRIMARY KEY,
|
||||
folder TEXT NOT NULL,
|
||||
enabled INTEGER NOT NULL DEFAULT 1
|
||||
);
|
||||
|
||||
-- Per-user library state. Kept separate from books so that a rescan can never
|
||||
-- clobber a user's own edits.
|
||||
CREATE TABLE IF NOT EXISTS user_books (
|
||||
|
||||
@@ -7,6 +7,7 @@ import { registerLibraryRoutes } from './routes/library.ts';
|
||||
import { registerSyncRoutes } from './routes/sync.ts';
|
||||
import { registerTtsRoutes } from './routes/tts.ts';
|
||||
import { registerWebRoutes } from './routes/web.ts';
|
||||
import { registerSourceRoutes } from './routes/sources.ts';
|
||||
import { isOriginAllowed, resolveCorsOrigin } from './cors.ts';
|
||||
|
||||
export function buildApp(ctx: AppContext): FastifyInstance {
|
||||
@@ -20,6 +21,9 @@ export function buildApp(ctx: AppContext): FastifyInstance {
|
||||
// JSON request (a sync batch, a batch metadata edit) hold 400MB of heap.
|
||||
bodyLimit: 8 * 1024 * 1024,
|
||||
trustProxy: true,
|
||||
// Source publications use opaque plugin-owned references, often longer than
|
||||
// the router's default 100-character parameter limit.
|
||||
routerOptions: { maxParamLength: 16_384 },
|
||||
});
|
||||
|
||||
registerErrorHandler(app);
|
||||
@@ -58,6 +62,7 @@ export function buildApp(ctx: AppContext): FastifyInstance {
|
||||
registerLibraryRoutes(app, ctx);
|
||||
registerSyncRoutes(app, ctx);
|
||||
registerTtsRoutes(app, ctx);
|
||||
registerSourceRoutes(app, ctx);
|
||||
// Registered last: the SPA fallback must not shadow an API route.
|
||||
registerWebRoutes(app, ctx);
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ import type { SyncService } from '../services/sync.ts';
|
||||
import type { TtsService } from '../services/tts.ts';
|
||||
import type { BrowseService } from '../services/browse.ts';
|
||||
import type { UploadService } from '../services/uploads.ts';
|
||||
import type { SourceHost } from '../services/source-host.ts';
|
||||
|
||||
export interface AppContext {
|
||||
config: AppConfig;
|
||||
@@ -36,6 +37,7 @@ export interface AppContext {
|
||||
* is to write.
|
||||
*/
|
||||
uploads: UploadService;
|
||||
sources?: SourceHost;
|
||||
log: FastifyBaseLogger;
|
||||
}
|
||||
|
||||
|
||||
@@ -12,32 +12,10 @@ import {
|
||||
capabilities,
|
||||
contentTypeFor,
|
||||
directoryHandlerForFormat,
|
||||
fileHandlerForFormat,
|
||||
type ContentItem,
|
||||
} from '../../indexer/formats/index.ts';
|
||||
import { sendAssetPayload } from '../assets.ts';
|
||||
|
||||
/**
|
||||
* The single live path backing a book, if it is file-backed.
|
||||
*
|
||||
* A directory book has no file to stream, so the caller is told which of the two
|
||||
* shapes it is rather than being handed a path that may not exist.
|
||||
*/
|
||||
function resolveBookSource(
|
||||
ctx: AppContext,
|
||||
bookId: string,
|
||||
format: string,
|
||||
): { relPath: string; isDirectory: boolean } | null {
|
||||
const file = ctx.db.get<{ rel_path: string }>(
|
||||
'SELECT rel_path FROM book_files WHERE book_id = ? AND missing = 0 ORDER BY rel_path LIMIT 1',
|
||||
bookId,
|
||||
);
|
||||
if (!file) return null;
|
||||
// Directory books are recorded under a path with no extension; a file handler
|
||||
// for the format means the path is a real file.
|
||||
const isDirectory = format === 'comic-dir' || fileHandlerForFormat(format) === null;
|
||||
return { relPath: file.rel_path, isDirectory };
|
||||
}
|
||||
import { FilePublications } from '../../publications/files.ts';
|
||||
|
||||
/**
|
||||
* The handler that owns a book, resolved from its recorded format.
|
||||
@@ -45,20 +23,13 @@ function resolveBookSource(
|
||||
* A missing file still has a handler; only the manifest and asset calls fail.
|
||||
*/
|
||||
function resolveHandler(ctx: AppContext, bookId: string, format: string) {
|
||||
void ctx;
|
||||
void bookId;
|
||||
return fileHandlerForFormat(format) ?? directoryHandlerForFormat(format) ?? null;
|
||||
return new FilePublications(ctx.db, ctx.config).handler(format);
|
||||
}
|
||||
|
||||
/** Absolute path plus library-relative path for a book's backing store. */
|
||||
/** Validated storage context, independent of a book's discovery source. */
|
||||
function sourceContext(ctx: AppContext, bookId: string) {
|
||||
const file = ctx.db.get<{ rel_path: string }>(
|
||||
'SELECT rel_path FROM book_files WHERE book_id = ? AND missing = 0 ORDER BY rel_path LIMIT 1',
|
||||
bookId,
|
||||
);
|
||||
if (!file) throw notFound('no available file for this book', 'FILE_MISSING');
|
||||
const relPath = assertSafeRel(file.rel_path);
|
||||
return { relPath, absPath: resolveInside(ctx.config.booksDir, relPath) };
|
||||
return new FilePublications(ctx.db, ctx.config).context(bookId);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -230,21 +201,7 @@ export function registerLibraryRoutes(app: FastifyInstance, ctx: AppContext): vo
|
||||
//
|
||||
// This is a contract, not an inventory: `/books/:id/file` serves exactly the
|
||||
// paths named here, and nothing else.
|
||||
files: content && handler && 'files' in handler && handler.files
|
||||
? (await handler.files({ ...sourceContext(ctx, id), bookId: id })).map((file) => ({
|
||||
rel_path: file.relPath,
|
||||
// The handler's own reference, passed through untouched. It is what
|
||||
// makes the listed path fetchable: a directory book addresses a page
|
||||
// as `page:<volume>:<page>`, and only its handler knows where the
|
||||
// volumes begin.
|
||||
ref: file.ref,
|
||||
size: file.size,
|
||||
missing: file.missing,
|
||||
}))
|
||||
: ctx.db.all<{ rel_path: string; size: number; missing: number }>(
|
||||
'SELECT rel_path, size, missing FROM book_files WHERE book_id = ? ORDER BY rel_path',
|
||||
id,
|
||||
),
|
||||
files: await new FilePublications(ctx.db, ctx.config).files(id, book.format),
|
||||
// `content` is null only for a book whose format exposes no addressable
|
||||
// structure at all; the client then falls back to the raw file.
|
||||
content: windowed,
|
||||
@@ -263,29 +220,7 @@ export function registerLibraryRoutes(app: FastifyInstance, ctx: AppContext): vo
|
||||
const user = currentUser(request);
|
||||
const { id } = request.params as { id: string };
|
||||
const book = ctx.shelf.get(user.id, id);
|
||||
const source = resolveBookSource(ctx, id, book.format);
|
||||
if (!source) throw notFound('no available file for this book', 'FILE_MISSING');
|
||||
|
||||
// Directory books have no file to stream; the client caches page by page.
|
||||
if (source.isDirectory) {
|
||||
throw badRequest(
|
||||
'this book is backed by a directory; fetch pages from the manifest instead',
|
||||
'DIRECTORY_BOOK',
|
||||
);
|
||||
}
|
||||
|
||||
const abs = resolveInside(ctx.config.booksDir, assertSafeRel(source.relPath));
|
||||
if (!existsSync(abs)) throw notFound('file is no longer on disk', 'FILE_MISSING');
|
||||
|
||||
return sendAssetPayload(request, reply, {
|
||||
stream: createReadStream(abs),
|
||||
contentType: contentTypeFor(source.relPath),
|
||||
filename: source.relPath.split('/').pop() ?? 'book',
|
||||
size: (await stat(abs)).size,
|
||||
seekable: true,
|
||||
etag: id,
|
||||
lastModified: (await stat(abs)).mtimeMs,
|
||||
});
|
||||
return sendAssetPayload(request, reply, await new FilePublications(ctx.db, ctx.config).content(id, book.format));
|
||||
});
|
||||
|
||||
/**
|
||||
@@ -399,6 +334,11 @@ export function registerLibraryRoutes(app: FastifyInstance, ctx: AppContext): vo
|
||||
// legitimate one. Without this a traversal inside a directory book passes the
|
||||
// "is it under the folder" test.
|
||||
const wanted = normalizeRel(query.path);
|
||||
const publications = new FilePublications(ctx.db, ctx.config);
|
||||
const managed = publications.managed(id);
|
||||
if (managed && managed.rel_path === wanted) {
|
||||
return sendAssetPayload(request, reply, await publications.content(id, book.format));
|
||||
}
|
||||
|
||||
// Two shapes of book, and each owns a different set of paths.
|
||||
//
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
import type { FastifyInstance, FastifyReply, FastifyRequest } from 'fastify';
|
||||
import type { AppContext } from '../context.ts';
|
||||
import { authenticate, currentUser, requireAdmin } from '../auth.ts';
|
||||
import { badRequest } from '../../lib/errors.ts';
|
||||
import { SourceHost } from '../../services/source-host.ts';
|
||||
|
||||
function textBody(body: unknown, name: string): string {
|
||||
const value = (body as Record<string, unknown> | null)?.[name];
|
||||
return parameter(value, name, true)!;
|
||||
}
|
||||
|
||||
function parameter(value: unknown, name: string, required = false): string | undefined {
|
||||
if (value === undefined && !required) return undefined;
|
||||
if (typeof value !== 'string' || !value.trim() || value.length > 16_384) {
|
||||
throw badRequest(`${name} must be a nonempty string up to 16384 characters`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
/** Source discovery and plugin management. All provider-specific behaviour stays in SourceHost. */
|
||||
export function registerSourceRoutes(app: FastifyInstance, ctx: AppContext): void {
|
||||
const auth = authenticate(ctx);
|
||||
const host = ctx.sources ??= new SourceHost(ctx.db, ctx.config, () => ctx.shelf, app.log);
|
||||
app.addHook('onReady', () => host.plugins.loadInstalled());
|
||||
app.addHook('onClose', () => host.plugins.close());
|
||||
|
||||
app.get('/api/v1/sources/types', { preHandler: auth }, async () => ({
|
||||
types: host.registry.list().map(({ pluginId, builtin, provider }) => ({
|
||||
...provider.descriptor, pluginId, builtin: builtin === true,
|
||||
})),
|
||||
}));
|
||||
app.get('/api/v1/sources', { preHandler: auth }, async (request) => {
|
||||
const user = currentUser(request);
|
||||
return { sources: host.list(user.role === 'admin') };
|
||||
});
|
||||
app.post('/api/v1/sources', { preHandler: auth }, async (request, reply) => {
|
||||
requireAdmin(request);
|
||||
const body = (request.body ?? {}) as Record<string, unknown>;
|
||||
const pluginId = textBody(body, 'pluginId');
|
||||
const sourceType = textBody(body, 'sourceType');
|
||||
const name = textBody(body, 'name');
|
||||
const id = await host.create({
|
||||
id: parameter(body.id, 'id'), pluginId, sourceType, name,
|
||||
config: body.config ?? {},
|
||||
});
|
||||
reply.status(201);
|
||||
return { source: host.list(true).find((source) => source.id === id) };
|
||||
});
|
||||
app.patch('/api/v1/sources/:id', { preHandler: auth }, async (request) => {
|
||||
requireAdmin(request);
|
||||
const { id } = request.params as { id: string };
|
||||
const enabled = (request.body as Record<string, unknown> | null)?.enabled;
|
||||
if (typeof enabled !== 'boolean') throw badRequest('enabled must be boolean');
|
||||
host.setEnabled(id, enabled);
|
||||
return { source: host.list(true).find((source) => source.id === id) };
|
||||
});
|
||||
app.put('/api/v1/sources/:id/credentials/:key', { preHandler: auth }, async (request) => {
|
||||
const user = currentUser(request);
|
||||
const { id, key } = request.params as { id: string; key: string };
|
||||
const value = (request.body as Record<string, unknown> | null)?.value;
|
||||
if (typeof value !== 'string') throw badRequest('credential value must be a string');
|
||||
host.setCredential(id, user.id, key, value);
|
||||
return { ok: true };
|
||||
});
|
||||
app.get('/api/v1/sources/:id/browse', { preHandler: auth }, async (request, reply) => {
|
||||
const user = currentUser(request); const { id } = request.params as { id: string };
|
||||
const q = request.query as { ref?: string; cursor?: string; limit?: string };
|
||||
return withSignal(request, reply, (signal) => host.browse(user.id, id, {
|
||||
ref: parameter(q.ref, 'ref'), cursor: parameter(q.cursor, 'cursor'), limit: pageLimit(q.limit),
|
||||
}, signal));
|
||||
});
|
||||
app.get('/api/v1/sources/:id/search', { preHandler: auth }, async (request, reply) => {
|
||||
const user = currentUser(request); const { id } = request.params as { id: string };
|
||||
const q = request.query as { q?: string; cursor?: string; limit?: string };
|
||||
const query = parameter(q.q, 'q', true)!;
|
||||
return withSignal(request, reply, (signal) => host.search(user.id, id, {
|
||||
query, cursor: parameter(q.cursor, 'cursor'), limit: pageLimit(q.limit),
|
||||
}, signal));
|
||||
});
|
||||
app.get('/api/v1/sources/:id/entries', { preHandler: auth }, async (request, reply) => {
|
||||
const user = currentUser(request); const { id } = request.params as { id: string };
|
||||
const q = request.query as { ref?: string };
|
||||
const ref = parameter(q.ref, 'ref', true)!;
|
||||
return withSignal(request, reply, (signal) => host.detail(user.id, id, ref, signal));
|
||||
});
|
||||
app.post('/api/v1/sources/:id/acquire', { preHandler: auth }, async (request, reply) => {
|
||||
const user = currentUser(request); const { id } = request.params as { id: string };
|
||||
const body = request.body as Record<string, unknown> | null;
|
||||
const entryRef = textBody(body, 'entryRef');
|
||||
const optionId = parameter(body?.optionId, 'optionId');
|
||||
return withSignal(request, reply, (signal) => host.acquire(user.id, id, entryRef, optionId, signal));
|
||||
});
|
||||
app.get('/api/v1/sources/:id/publications/:publicationRef/manifest', { preHandler: auth }, async (request, reply) => {
|
||||
const user = currentUser(request); const params = request.params as { id: string; publicationRef: string };
|
||||
return withSignal(request, reply, (signal) => host.manifest(user.id, params.id, params.publicationRef, signal));
|
||||
});
|
||||
app.get('/api/v1/sources/:id/publications/:publicationRef/resource', { preHandler: auth }, async (request, reply) => {
|
||||
const user = currentUser(request); const params = request.params as { id: string; publicationRef: string };
|
||||
const ref = parameter((request.query as { ref?: string }).ref, 'ref', true)!;
|
||||
const resource = await withSignal(request, reply, (signal) => host.resource(user.id, params.id, params.publicationRef, ref, signal));
|
||||
// Development endpoint: JSON only. Untrusted HTML is never served as an app-origin document.
|
||||
if (resource.stream) { resource.stream.destroy(); throw badRequest('streaming preview is not supported'); }
|
||||
return { mediaType: resource.mediaType, ...(resource.text !== undefined ? { text: resource.text } : {}),
|
||||
...(resource.data ? { base64: Buffer.from(resource.data).toString('base64') } : {}) };
|
||||
});
|
||||
app.get('/api/v1/plugins', { preHandler: auth }, async (request) => {
|
||||
requireAdmin(request);
|
||||
return { plugins: host.plugins.list() };
|
||||
});
|
||||
app.post('/api/v1/plugins', { preHandler: auth }, async (request, reply) => {
|
||||
requireAdmin(request);
|
||||
const body = request.body as Record<string, unknown> | null;
|
||||
if (body?.trusted !== true) throw badRequest('trusted must be true: installed plugins execute with server OS privileges', 'PLUGIN_TRUST_REQUIRED');
|
||||
const plugin = await host.plugins.install(textBody(body, 'folder'));
|
||||
reply.status(201);
|
||||
return { plugin };
|
||||
});
|
||||
app.patch('/api/v1/plugins/:id', { preHandler: auth }, async (request) => {
|
||||
requireAdmin(request);
|
||||
const enabled = (request.body as Record<string, unknown> | null)?.enabled;
|
||||
if (typeof enabled !== 'boolean') throw badRequest('enabled must be boolean');
|
||||
return { plugin: await host.plugins.setEnabled((request.params as { id: string }).id, enabled) };
|
||||
});
|
||||
app.delete('/api/v1/plugins/:id', { preHandler: auth }, async (request) => {
|
||||
requireAdmin(request);
|
||||
await host.plugins.uninstall((request.params as { id: string }).id);
|
||||
return { ok: true };
|
||||
});
|
||||
}
|
||||
|
||||
function pageLimit(value?: string): number | undefined {
|
||||
if (value === undefined) return undefined;
|
||||
if (typeof value !== 'string' || !/^[1-9]\d{0,2}$/.test(value)) throw badRequest('invalid limit');
|
||||
const number = Number(value);
|
||||
if (!Number.isSafeInteger(number) || number < 1 || number > 200) throw badRequest('limit must be an integer between 1 and 200');
|
||||
return number;
|
||||
}
|
||||
|
||||
async function withSignal<T>(request: FastifyRequest, reply: FastifyReply, action: (signal: AbortSignal) => Promise<T>): Promise<T> {
|
||||
const controller = new AbortController();
|
||||
const abort = () => controller.abort();
|
||||
request.raw.once('aborted', abort);
|
||||
reply.raw.once('close', abort);
|
||||
try { return await action(controller.signal); }
|
||||
finally { request.raw.removeListener('aborted', abort); reply.raw.removeListener('close', abort); }
|
||||
}
|
||||
@@ -182,7 +182,7 @@ export type ProgressReporter = (scanned: number) => void;
|
||||
export interface HandlerContext {
|
||||
/** Library-relative path, forward slashes. */
|
||||
relPath: string;
|
||||
/** Absolute path, guaranteed inside the read-only books root. */
|
||||
/** Absolute path validated by the host: the books mount or managed DATA_DIR content. */
|
||||
absPath: string;
|
||||
/**
|
||||
* Book id, available once the book exists. Handlers that emit links back into
|
||||
|
||||
@@ -234,6 +234,7 @@ export class Scanner {
|
||||
this.db.run(
|
||||
`DELETE FROM books
|
||||
WHERE id NOT IN (SELECT DISTINCT book_id FROM book_files WHERE missing = 0)
|
||||
AND id NOT IN (SELECT book_id FROM acquired_files)
|
||||
AND id NOT IN (SELECT DISTINCT book_id FROM user_books)`,
|
||||
);
|
||||
|
||||
|
||||
@@ -30,6 +30,7 @@ async function main(): Promise<void> {
|
||||
shelf: undefined,
|
||||
sync: undefined,
|
||||
log: undefined,
|
||||
sources: undefined,
|
||||
} as unknown as AppContext;
|
||||
|
||||
const app = buildApp(ctx);
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
import { createReadStream, existsSync } from 'node:fs';
|
||||
import { stat } from 'node:fs/promises';
|
||||
import type { AppConfig } from '../config/index.ts';
|
||||
import type { Db } from '../db/index.ts';
|
||||
import { assertSafeRel, resolveInside } from '../lib/paths.ts';
|
||||
import { badRequest, notFound } from '../lib/errors.ts';
|
||||
import {
|
||||
contentTypeFor, directoryHandlerForFormat, fileHandlerForFormat,
|
||||
type AssetPayload, type HandlerContext,
|
||||
} from '../indexer/formats/index.ts';
|
||||
|
||||
/** Resolves stored content independently of where a book was discovered. */
|
||||
export class FilePublications {
|
||||
constructor(private readonly db: Db, private readonly config: AppConfig) {}
|
||||
|
||||
handler(format: string) {
|
||||
return fileHandlerForFormat(format) ?? directoryHandlerForFormat(format);
|
||||
}
|
||||
|
||||
managed(bookId: string): { rel_path: string; size: number } | undefined {
|
||||
return this.db.get('SELECT rel_path, size FROM acquired_files WHERE book_id = ?', bookId);
|
||||
}
|
||||
|
||||
context(bookId: string): HandlerContext {
|
||||
const local = this.db.get<{ rel_path: string }>(
|
||||
'SELECT rel_path FROM book_files WHERE book_id = ? AND missing = 0 ORDER BY rel_path LIMIT 1', bookId,
|
||||
);
|
||||
const managed = this.managed(bookId);
|
||||
const useLocal = local && (!managed || existsSync(resolveInside(this.config.booksDir, assertSafeRel(local.rel_path))));
|
||||
const file = useLocal ? local : managed ?? local;
|
||||
if (!file) throw notFound('no available file for this book', 'FILE_MISSING');
|
||||
const relPath = assertSafeRel(file.rel_path);
|
||||
return { bookId, relPath, absPath: resolveInside(useLocal ? this.config.booksDir : this.config.dataDir, relPath) };
|
||||
}
|
||||
|
||||
async files(bookId: string, format: string) {
|
||||
const handler = this.handler(format);
|
||||
if (handler && 'files' in handler && handler.files) {
|
||||
return (await handler.files(this.context(bookId))).map((file) => ({
|
||||
rel_path: file.relPath, ref: file.ref, size: file.size, missing: file.missing,
|
||||
}));
|
||||
}
|
||||
const local = this.db.all<{ rel_path: string; size: number; missing: number }>(
|
||||
'SELECT rel_path, size, missing FROM book_files WHERE book_id = ? ORDER BY rel_path', bookId,
|
||||
);
|
||||
const managed = this.managed(bookId);
|
||||
return managed ? [...local, { ...managed, missing: 0 }] : local;
|
||||
}
|
||||
|
||||
async content(bookId: string, format: string): Promise<AssetPayload> {
|
||||
if (!fileHandlerForFormat(format)) {
|
||||
throw badRequest('this book is backed by a directory; fetch pages from the manifest instead', 'DIRECTORY_BOOK');
|
||||
}
|
||||
const context = this.context(bookId);
|
||||
if (!existsSync(context.absPath)) throw notFound('file is no longer on disk', 'FILE_MISSING');
|
||||
const info = await stat(context.absPath);
|
||||
return {
|
||||
stream: createReadStream(context.absPath), contentType: contentTypeFor(context.relPath),
|
||||
filename: context.relPath.split('/').pop() ?? 'book', size: info.size,
|
||||
seekable: true, etag: bookId, lastModified: info.mtimeMs,
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -544,14 +544,16 @@ export class BrowseService {
|
||||
throw badRequest('bookIds must contain strings', 'BAD_BOOK_ID');
|
||||
}
|
||||
/*
|
||||
* `book_files` rather than `books`: the path is a property of the *file*, and a
|
||||
* book whose last file is missing has a row in `books` and none to point at.
|
||||
* Reporting that one as a failure is the honest answer — the shelf's own
|
||||
* predicate already says a book with no live file is not on the shelf.
|
||||
* Local files belong to the shared library. Managed downloads can only be
|
||||
* added or removed by a user with an acquisition record, even if another
|
||||
* account supplies their book id. Neither path grants access from an id alone.
|
||||
*/
|
||||
const file = this.db.get<{ book_id: string }>(
|
||||
'SELECT book_id FROM book_files WHERE book_id = ? AND missing = 0 LIMIT 1',
|
||||
input,
|
||||
`SELECT book_id FROM book_files WHERE book_id = ? AND missing = 0
|
||||
UNION SELECT f.book_id FROM acquired_files f
|
||||
JOIN source_acquisitions a ON a.book_id = f.book_id
|
||||
WHERE f.book_id = ? AND a.user_id = ? LIMIT 1`,
|
||||
input, input, userId,
|
||||
);
|
||||
if (!file) {
|
||||
result.failed.push({ path: input, reason: 'NO_LIVE_FILE' });
|
||||
|
||||
@@ -167,7 +167,7 @@ export class ShelfService {
|
||||
const library = scope === 'library';
|
||||
const where: string[] = library
|
||||
? ['EXISTS (SELECT 1 FROM book_files f WHERE f.book_id = b.id AND f.missing = 0)']
|
||||
: ['ub.user_id = ?', 'ub.hidden = 0', 'EXISTS (SELECT 1 FROM book_files f WHERE f.book_id = b.id AND f.missing = 0)'];
|
||||
: ['ub.user_id = ?', 'ub.hidden = 0', '(EXISTS (SELECT 1 FROM book_files f WHERE f.book_id = b.id AND f.missing = 0) OR EXISTS (SELECT 1 FROM acquired_files a WHERE a.book_id = b.id))'];
|
||||
const params: Array<string | number> = library ? [] : [userId];
|
||||
|
||||
if (options.search) {
|
||||
|
||||
@@ -0,0 +1,218 @@
|
||||
import { createCipheriv, createDecipheriv, createHash, randomBytes } from 'node:crypto';
|
||||
import type { FastifyBaseLogger } from 'fastify';
|
||||
import type { AppConfig } from '../config/index.ts';
|
||||
import type { Db } from '../db/index.ts';
|
||||
import type { ShelfService } from './shelf.ts';
|
||||
import { AppError, badRequest, conflict, notFound } from '../lib/errors.ts';
|
||||
import { SourceRegistry } from '../sources/registry.ts';
|
||||
import { createLocalProvider } from '../sources/local.ts';
|
||||
import { createOpdsProvider } from '../sources/opds.ts';
|
||||
import type { SourceContext, SourceInstance, SourceProvider, SourceStorage, CredentialStore } from '../sources/types.ts';
|
||||
import { FileAcquisitions } from '../sources/acquisitions.ts';
|
||||
import { PluginManager } from '../sources/plugin-manager.ts';
|
||||
|
||||
function json(value: unknown): string { return JSON.stringify(value ?? {}); }
|
||||
function parse(value: string): unknown { try { return JSON.parse(value); } catch { return {}; } }
|
||||
|
||||
/** Host-owned registry, instance lifecycle and credential boundary. */
|
||||
export class SourceHost {
|
||||
readonly registry = new SourceRegistry();
|
||||
readonly plugins: PluginManager;
|
||||
private readonly key: Buffer;
|
||||
private readonly acquisitions: FileAcquisitions;
|
||||
private activeCalls = 0;
|
||||
private readonly activeSources = new Map<string, number>();
|
||||
|
||||
constructor(private readonly db: Db, private readonly config: AppConfig, shelf: () => ShelfService, private readonly log?: FastifyBaseLogger) {
|
||||
this.key = createHash('sha256').update('reader/source-credentials/v1\0').update(config.jwtSecret).digest();
|
||||
this.acquisitions = new FileAcquisitions(db, config);
|
||||
this.registry.registerBuiltin('reader.local', createLocalProvider(db, shelf));
|
||||
this.registry.registerBuiltin('reader.opds', createOpdsProvider());
|
||||
this.plugins = new PluginManager(db, config.dataDir, this.registry);
|
||||
this.ensureLocal();
|
||||
}
|
||||
|
||||
private ensureLocal(): void {
|
||||
const found = this.db.get<{ id: string }>("SELECT id FROM source_instances WHERE plugin_id = 'reader.local' AND source_type = 'local' LIMIT 1");
|
||||
if (!found) this.db.run(
|
||||
`INSERT INTO source_instances (id, plugin_id, source_type, name, config_json, enabled, created_at)
|
||||
VALUES ('local', 'reader.local', 'local', '本地书库', '{}', 1, ?)`, Date.now(),
|
||||
);
|
||||
}
|
||||
|
||||
list(includeConfig = false) {
|
||||
return this.db.all<{ id: string; plugin_id: string; source_type: string; name: string; config_json: string; enabled: number }>(
|
||||
'SELECT id, plugin_id, source_type, name, config_json, enabled FROM source_instances ORDER BY name',
|
||||
).map((row) => ({
|
||||
id: row.id, pluginId: row.plugin_id, sourceType: row.source_type, name: row.name,
|
||||
enabled: row.enabled === 1,
|
||||
...(includeConfig ? { config: parse(row.config_json) } : {}),
|
||||
descriptor: this.registry.get(row.plugin_id, row.source_type)?.provider.descriptor ?? null,
|
||||
}));
|
||||
}
|
||||
|
||||
async create(input: { id?: string; pluginId: string; sourceType: string; name: string; config: unknown }): Promise<string> {
|
||||
const id = input.id?.trim() || randomBytes(12).toString('hex');
|
||||
if (!/^[a-zA-Z0-9_-]{2,80}$/.test(id)) throw badRequest('invalid source instance id');
|
||||
if (!input.name.trim() || input.name.length > 128) throw badRequest('invalid source name');
|
||||
if (json(input.config).length > 64 * 1024) throw badRequest('source configuration is too large');
|
||||
const registration = this.registry.get(input.pluginId, input.sourceType);
|
||||
if (!registration) throw notFound('source type is not installed', 'SOURCE_TYPE_NOT_FOUND');
|
||||
await registration.provider.validateConfig?.(input.config);
|
||||
if (this.registry.get(input.pluginId, input.sourceType) !== registration) {
|
||||
throw conflict('source plugin changed during configuration; retry', 'PLUGIN_UNAVAILABLE');
|
||||
}
|
||||
if (this.db.get('SELECT id FROM source_instances WHERE id = ?', id)) throw conflict('source instance already exists');
|
||||
this.db.run(
|
||||
`INSERT INTO source_instances (id, plugin_id, source_type, name, config_json, enabled, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, 1, ?)`, id, input.pluginId, input.sourceType, input.name.trim(), json(input.config), Date.now(),
|
||||
);
|
||||
return id;
|
||||
}
|
||||
|
||||
setEnabled(id: string, enabled: boolean): void {
|
||||
const result = this.db.get<{ id: string }>('SELECT id FROM source_instances WHERE id = ?', id);
|
||||
if (!result) throw notFound('source instance not found', 'SOURCE_NOT_FOUND');
|
||||
this.db.run('UPDATE source_instances SET enabled = ? WHERE id = ?', enabled ? 1 : 0, id);
|
||||
}
|
||||
|
||||
setCredential(sourceId: string, userId: string, name: string, value: string): void {
|
||||
if (!/^[a-z][a-z0-9_.-]{0,63}$/.test(name) || value.length > 16_384) throw badRequest('invalid credential');
|
||||
this.instance(sourceId);
|
||||
this.db.run(
|
||||
`INSERT INTO source_credentials (source_id, user_id, key, encrypted_value) VALUES (?, ?, ?, ?)
|
||||
ON CONFLICT(source_id, user_id, key) DO UPDATE SET encrypted_value = excluded.encrypted_value`,
|
||||
sourceId, userId, name, this.encrypt(value),
|
||||
);
|
||||
}
|
||||
|
||||
async browse(userId: string, id: string, request: { ref?: string; cursor?: string; limit?: number }, signal?: AbortSignal) {
|
||||
return this.call(userId, id, (provider, ctx) => provider.browse
|
||||
? provider.browse(ctx, request) : Promise.reject(badRequest('source does not support browse', 'SOURCE_UNSUPPORTED')), signal);
|
||||
}
|
||||
async search(userId: string, id: string, request: { query: string; cursor?: string; limit?: number }, signal?: AbortSignal) {
|
||||
return this.call(userId, id, (provider, ctx) => provider.search
|
||||
? provider.search(ctx, request) : Promise.reject(badRequest('source does not support search', 'SOURCE_UNSUPPORTED')), signal);
|
||||
}
|
||||
async detail(userId: string, id: string, ref: string, signal?: AbortSignal) {
|
||||
return this.call(userId, id, (provider, ctx) => provider.detail(ctx, ref), signal);
|
||||
}
|
||||
async acquire(userId: string, id: string, entryRef: string, optionId?: string, signal?: AbortSignal) {
|
||||
return this.call(userId, id, async (provider, ctx) => {
|
||||
const acquisition = await provider.acquire(ctx, { entryRef, optionId });
|
||||
if (acquisition.kind === 'ready') {
|
||||
if (ctx.instance.pluginId !== 'reader.local' || ctx.instance.sourceType !== 'local') {
|
||||
throw badRequest('external plugins cannot reference host book identities', 'INVALID_ACQUISITION');
|
||||
}
|
||||
this.db.run(
|
||||
`INSERT INTO user_books (user_id, book_id, added_at, hidden) VALUES (?, ?, ?, 0)
|
||||
ON CONFLICT(user_id, book_id) DO UPDATE SET hidden = 0,
|
||||
added_at = CASE WHEN user_books.hidden = 1 THEN excluded.added_at ELSE user_books.added_at END`,
|
||||
userId, acquisition.publicationId, Date.now(),
|
||||
);
|
||||
}
|
||||
if (acquisition.kind !== 'file') return acquisition;
|
||||
const entry = await provider.detail(ctx, entryRef);
|
||||
const bookId = await this.acquisitions.acquire(provider, ctx, { entryRef, optionId }, acquisition, entry);
|
||||
this.log?.info({ sourceId: id, bookId }, 'source file acquired');
|
||||
return { kind: 'ready' as const, publicationId: bookId };
|
||||
}, signal);
|
||||
}
|
||||
|
||||
async manifest(userId: string, id: string, ref: string, signal?: AbortSignal) {
|
||||
return this.call(userId, id, (provider, ctx) => provider.getManifest
|
||||
? provider.getManifest(ctx, ref) : Promise.reject(badRequest('source has no chapter manifest', 'SOURCE_UNSUPPORTED')), signal);
|
||||
}
|
||||
async resource(userId: string, id: string, publicationRef: string, ref: string, signal?: AbortSignal) {
|
||||
return this.call(userId, id, (provider, ctx) => provider.readResource
|
||||
? provider.readResource(ctx, { publicationRef, ref }) : Promise.reject(badRequest('source has no chapter resources', 'SOURCE_UNSUPPORTED')), signal);
|
||||
}
|
||||
|
||||
private async call<T>(userId: string, id: string, action: (provider: SourceProvider, ctx: SourceContext) => Promise<T>, externalSignal?: AbortSignal): Promise<T> {
|
||||
const row = this.instance(id);
|
||||
if (!row.enabled) throw badRequest('source is disabled', 'SOURCE_DISABLED');
|
||||
const registration = this.registry.get(row.plugin_id, row.source_type);
|
||||
if (!registration) throw notFound('source plugin is not installed', 'PLUGIN_UNAVAILABLE');
|
||||
const active = this.activeSources.get(id) ?? 0;
|
||||
if (active >= 2 || this.activeCalls >= 8) {
|
||||
throw new AppError(429, 'RATE_LIMITED', 'source is busy; retry after current requests complete');
|
||||
}
|
||||
this.activeSources.set(id, active + 1);
|
||||
this.activeCalls += 1;
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), 60_000); timer.unref();
|
||||
const signal = externalSignal ? AbortSignal.any([externalSignal, controller.signal]) : controller.signal;
|
||||
try {
|
||||
signal.throwIfAborted();
|
||||
return await action(registration.provider, this.context(row, userId, signal));
|
||||
} catch (error) {
|
||||
if (controller.signal.aborted) throw new AppError(504, 'SOURCE_TIMEOUT', 'source request exceeded its execution deadline');
|
||||
if (externalSignal?.aborted) throw new AppError(499, 'SOURCE_CANCELLED', 'source request cancelled');
|
||||
throw error;
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
this.activeCalls -= 1;
|
||||
const remaining = (this.activeSources.get(id) ?? 1) - 1;
|
||||
if (remaining) this.activeSources.set(id, remaining);
|
||||
else this.activeSources.delete(id);
|
||||
}
|
||||
}
|
||||
|
||||
private context(row: SourceRow, userId: string, signal: AbortSignal): SourceContext {
|
||||
const instance: SourceInstance = {
|
||||
id: row.id, pluginId: row.plugin_id, sourceType: row.source_type, name: row.name,
|
||||
config: parse(row.config_json), enabled: row.enabled === 1,
|
||||
};
|
||||
return {
|
||||
instance, userId, signal,
|
||||
credentials: this.credentials(row.id, userId),
|
||||
storage: this.storage(row.id, userId),
|
||||
log: this.log ? {
|
||||
debug: (message, fields) => this.log?.debug(fields ?? {}, message),
|
||||
info: (message, fields) => this.log?.info(fields ?? {}, message),
|
||||
warn: (message, fields) => this.log?.warn(fields ?? {}, message),
|
||||
error: (message, fields) => this.log?.error(fields ?? {}, message),
|
||||
} : undefined,
|
||||
};
|
||||
}
|
||||
private credentials(sourceId: string, userId: string): CredentialStore {
|
||||
return {
|
||||
get: async (key) => {
|
||||
const row = this.db.get<{ encrypted_value: string }>('SELECT encrypted_value FROM source_credentials WHERE source_id = ? AND user_id = ? AND key = ?', sourceId, userId, key);
|
||||
return row ? this.decrypt(row.encrypted_value) : undefined;
|
||||
},
|
||||
set: async (key, value) => this.setCredential(sourceId, userId, key, value),
|
||||
delete: async (key) => this.db.run('DELETE FROM source_credentials WHERE source_id = ? AND user_id = ? AND key = ?', sourceId, userId, key),
|
||||
};
|
||||
}
|
||||
private storage(sourceId: string, userId: string): SourceStorage {
|
||||
const prefix = `source:${sourceId}:${userId}:`;
|
||||
return {
|
||||
get: async (key) => { const row = this.db.get<{ value: string }>('SELECT value FROM plugin_storage WHERE key = ?', prefix + key); return row ? parse(row.value) : undefined; },
|
||||
set: async (key, value) => this.db.run('INSERT INTO plugin_storage (key, value) VALUES (?, ?) ON CONFLICT(key) DO UPDATE SET value = excluded.value', prefix + key, json(value)),
|
||||
delete: async (key) => this.db.run('DELETE FROM plugin_storage WHERE key = ?', prefix + key),
|
||||
};
|
||||
}
|
||||
private instance(id: string): SourceRow {
|
||||
const row = this.db.get<SourceRow>('SELECT id, plugin_id, source_type, name, config_json, enabled FROM source_instances WHERE id = ?', id);
|
||||
if (!row) throw notFound('source instance not found', 'SOURCE_NOT_FOUND');
|
||||
return row;
|
||||
}
|
||||
private encrypt(value: string): string {
|
||||
const iv = randomBytes(12); const cipher = createCipheriv('aes-256-gcm', this.key, iv);
|
||||
const encrypted = Buffer.concat([cipher.update(value, 'utf8'), cipher.final()]);
|
||||
return `${iv.toString('base64url')}.${cipher.getAuthTag().toString('base64url')}.${encrypted.toString('base64url')}`;
|
||||
}
|
||||
private decrypt(value: string): string {
|
||||
try {
|
||||
const [iv, tag, body] = value.split('.');
|
||||
const decipher = createDecipheriv('aes-256-gcm', this.key, Buffer.from(iv!, 'base64url'));
|
||||
decipher.setAuthTag(Buffer.from(tag!, 'base64url'));
|
||||
return Buffer.concat([decipher.update(Buffer.from(body!, 'base64url')), decipher.final()]).toString('utf8');
|
||||
} catch {
|
||||
throw new AppError(401, 'AUTH_REQUIRED', 'stored credentials could not be decrypted; save them again');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
interface SourceRow { id: string; plugin_id: string; source_type: string; name: string; config_json: string; enabled: number }
|
||||
@@ -250,7 +250,8 @@ export class SyncService {
|
||||
FROM reading_progress p
|
||||
JOIN user_books ub ON ub.book_id = p.book_id AND ub.user_id = p.user_id
|
||||
WHERE p.user_id = ? AND ub.hidden = 0
|
||||
AND EXISTS (SELECT 1 FROM book_files f WHERE f.book_id = p.book_id AND f.missing = 0)
|
||||
AND (EXISTS (SELECT 1 FROM book_files f WHERE f.book_id = p.book_id AND f.missing = 0)
|
||||
OR EXISTS (SELECT 1 FROM acquired_files a WHERE a.book_id = p.book_id))
|
||||
ORDER BY p.updated_at DESC LIMIT ?`,
|
||||
userId, limit,
|
||||
);
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
import { createHash, randomUUID } from 'node:crypto';
|
||||
import { createWriteStream, existsSync } from 'node:fs';
|
||||
import { mkdir, readFile, rename, rm } from 'node:fs/promises';
|
||||
import { basename, extname, join } from 'node:path';
|
||||
import { Readable, Transform } from 'node:stream';
|
||||
import { pipeline } from 'node:stream/promises';
|
||||
import type { AppConfig } from '../config/index.ts';
|
||||
import type { Db } from '../db/index.ts';
|
||||
import { fileHandlerForExtension, type ParsedSource } from '../indexer/formats/index.ts';
|
||||
import { computeBookId } from '../indexer/identity.ts';
|
||||
import { saveCover } from '../indexer/metadata.ts';
|
||||
import { AppError, badRequest } from '../lib/errors.ts';
|
||||
import { resolveInside } from '../lib/paths.ts';
|
||||
import type {
|
||||
Acquisition, AcquireRequest, CatalogEntry, ResourceResponse, SourceContext, SourceProvider,
|
||||
} from './types.ts';
|
||||
|
||||
export const MAX_ACQUISITION_BYTES = 256 * 1024 * 1024;
|
||||
type FileAcquisition = Extract<Acquisition, { kind: 'file' }>;
|
||||
|
||||
const EXTENSIONS: Readonly<Record<string, string>> = {
|
||||
'application/epub+zip': 'epub',
|
||||
'application/pdf': 'pdf',
|
||||
'text/plain': 'txt',
|
||||
'text/markdown': 'md',
|
||||
'application/vnd.comicbook+zip': 'cbz',
|
||||
'application/x-cbz': 'cbz',
|
||||
'application/zip': 'zip',
|
||||
'image/jpeg': 'jpg',
|
||||
'image/png': 'png',
|
||||
'image/gif': 'gif',
|
||||
'image/webp': 'webp',
|
||||
};
|
||||
|
||||
function mediaType(value: string): string {
|
||||
return value.split(';', 1)[0]!.trim().toLowerCase();
|
||||
}
|
||||
|
||||
function tooLarge(): AppError {
|
||||
return new AppError(413, 'ACQUISITION_TOO_LARGE', 'download exceeds the acquisition size limit');
|
||||
}
|
||||
|
||||
/** Imports complete files into DATA_DIR without changing the scanned library. */
|
||||
export class FileAcquisitions {
|
||||
private readonly queues = new Map<string, Promise<unknown>>();
|
||||
private readonly maxBytes: number;
|
||||
|
||||
constructor(
|
||||
private readonly db: Db,
|
||||
private readonly config: Pick<AppConfig, 'dataDir'>,
|
||||
options: { maxBytes?: number } = {},
|
||||
) {
|
||||
this.maxBytes = options.maxBytes ?? MAX_ACQUISITION_BYTES;
|
||||
if (!Number.isSafeInteger(this.maxBytes) || this.maxBytes <= 0 || this.maxBytes > MAX_ACQUISITION_BYTES) {
|
||||
throw new Error('acquisition maxBytes must be between 1 and 256 MiB');
|
||||
}
|
||||
}
|
||||
|
||||
acquire(
|
||||
provider: SourceProvider,
|
||||
context: SourceContext,
|
||||
request: AcquireRequest,
|
||||
acquisition: FileAcquisition,
|
||||
entry: CatalogEntry,
|
||||
): Promise<string> {
|
||||
const key = JSON.stringify([context.instance.id, context.userId, request.entryRef, request.optionId ?? '']);
|
||||
// Serialize instead of sharing promises: one caller's cancellation must not
|
||||
// cancel another caller's download. The successor reuses a committed file.
|
||||
const previous = this.queues.get(key) ?? Promise.resolve();
|
||||
const result = previous.catch(() => undefined).then(() => this.importFile(provider, context, request, acquisition, entry));
|
||||
this.queues.set(key, result);
|
||||
void result.finally(() => {
|
||||
if (this.queues.get(key) === result) this.queues.delete(key);
|
||||
}).catch(() => undefined);
|
||||
return result;
|
||||
}
|
||||
|
||||
private async importFile(
|
||||
provider: SourceProvider,
|
||||
context: SourceContext,
|
||||
request: AcquireRequest,
|
||||
acquisition: FileAcquisition,
|
||||
entry: CatalogEntry,
|
||||
): Promise<string> {
|
||||
const { signal } = context;
|
||||
signal.throwIfAborted();
|
||||
const existing = this.db.get<{ book_id: string; rel_path: string }>(
|
||||
`SELECT a.book_id, f.rel_path FROM source_acquisitions a
|
||||
JOIN acquired_files f ON f.book_id = a.book_id
|
||||
WHERE a.source_id = ? AND a.user_id = ? AND a.entry_ref = ? AND a.option_id = ?`,
|
||||
context.instance.id, context.userId, request.entryRef, request.optionId ?? '',
|
||||
);
|
||||
if (existing && existsSync(resolveInside(this.config.dataDir, existing.rel_path))) {
|
||||
this.addToShelf(context.userId, existing.book_id, Date.now());
|
||||
return existing.book_id;
|
||||
}
|
||||
if (!provider.openFile) throw badRequest('source does not support file downloads', 'CAPABILITY_UNSUPPORTED');
|
||||
const type = mediaType(acquisition.mediaType);
|
||||
const extension = EXTENSIONS[type];
|
||||
const handler = extension ? fileHandlerForExtension(extension) : null;
|
||||
if (!extension || !handler) throw badRequest('unsupported acquisition media type', 'UNSUPPORTED_FORMAT');
|
||||
if (acquisition.size !== undefined && acquisition.size > this.maxBytes) throw tooLarge();
|
||||
|
||||
await mkdir(join(this.config.dataDir, 'acquired'), { recursive: true });
|
||||
const token = randomUUID();
|
||||
const temporaryPath = resolveInside(this.config.dataDir, `acquired/${token}.part`);
|
||||
let finalPath: string | undefined;
|
||||
let coverPath: string | undefined;
|
||||
let keepFile = false;
|
||||
let keepCover = false;
|
||||
let resource: ResourceResponse | undefined;
|
||||
try {
|
||||
resource = await provider.openFile(context, acquisition);
|
||||
signal.throwIfAborted();
|
||||
const responseType = mediaType(resource.mediaType);
|
||||
if (responseType !== 'application/octet-stream' && responseType !== type) {
|
||||
throw badRequest('download media type does not match its acquisition', 'INVALID_RESOURCE');
|
||||
}
|
||||
if (resource.size !== undefined && resource.size > this.maxBytes) throw tooLarge();
|
||||
const bodies = Number(resource.data !== undefined) + Number(resource.text !== undefined) + Number(resource.stream !== undefined);
|
||||
if (bodies !== 1) throw badRequest('download must contain exactly one body', 'INVALID_RESOURCE');
|
||||
const input = resource.stream ?? Readable.from([
|
||||
resource.data !== undefined ? resource.data : Buffer.from(resource.text!, 'utf8'),
|
||||
]);
|
||||
let size = 0;
|
||||
const hash = createHash('sha256');
|
||||
const maximum = this.maxBytes;
|
||||
const limit = new Transform({
|
||||
transform(chunk: Buffer, _encoding, callback) {
|
||||
size += chunk.byteLength;
|
||||
if (size > maximum) { callback(tooLarge()); return; }
|
||||
hash.update(chunk);
|
||||
callback(null, chunk);
|
||||
},
|
||||
});
|
||||
await pipeline(input, limit, createWriteStream(temporaryPath, { flags: 'wx' }), { signal });
|
||||
signal.throwIfAborted();
|
||||
if (size === 0) throw badRequest('download is empty', 'INVALID_RESOURCE');
|
||||
const bytes = await readFile(temporaryPath);
|
||||
const filename = basename((acquisition.filename ?? resource.filename ?? `${entry.title}.${extension}`).replaceAll('\\', '/'));
|
||||
const displayName = `${filename.slice(0, filename.length - extname(filename).length)}.${extension}`;
|
||||
const parseContext = { relPath: displayName, absPath: temporaryPath };
|
||||
if (type === 'application/pdf' && !bytes.subarray(0, 1024).includes(Buffer.from('%PDF-'))) {
|
||||
throw badRequest('download is not a PDF', 'INVALID_RESOURCE');
|
||||
}
|
||||
if (handler.matches && !await handler.matches(parseContext, bytes.subarray(0, 4096))) {
|
||||
throw badRequest('download does not match its format', 'INVALID_RESOURCE');
|
||||
}
|
||||
const parsed = await handler.parse(parseContext, bytes);
|
||||
if (parsed.metadata.raw.parseError) {
|
||||
throw badRequest('download could not be parsed as its declared format', 'INVALID_RESOURCE');
|
||||
}
|
||||
const contentHash = hash.digest('hex');
|
||||
const bookId = computeBookId(parsed.metadata.identifier, contentHash);
|
||||
signal.throwIfAborted();
|
||||
const relPath = `acquired/${bookId}-${token}.${extension}`;
|
||||
finalPath = resolveInside(this.config.dataDir, relPath);
|
||||
await rename(temporaryPath, finalPath);
|
||||
|
||||
if (parsed.cover && !this.db.get('SELECT id FROM books WHERE id = ?', bookId)) {
|
||||
// A unique cover name lets rollback remove only this import's bytes,
|
||||
// even when another source imports the same publication concurrently.
|
||||
coverPath = await saveCover(this.config.dataDir, `${bookId}-${token}`, parsed.cover);
|
||||
}
|
||||
signal.throwIfAborted();
|
||||
const now = Date.now();
|
||||
this.db.transaction(() => {
|
||||
this.insertBook(bookId, parsed, contentHash, size, coverPath, entry, context, now);
|
||||
const file = this.db.get<{ rel_path: string }>('SELECT rel_path FROM acquired_files WHERE book_id = ?', bookId);
|
||||
if (!file || !existsSync(resolveInside(this.config.dataDir, file.rel_path))) {
|
||||
this.db.run(
|
||||
`INSERT INTO acquired_files (book_id, rel_path, size) VALUES (?, ?, ?)
|
||||
ON CONFLICT(book_id) DO UPDATE SET rel_path = excluded.rel_path, size = excluded.size`,
|
||||
bookId, relPath, size,
|
||||
);
|
||||
}
|
||||
this.db.run(
|
||||
`INSERT INTO source_acquisitions (source_id, user_id, entry_ref, option_id, book_id) VALUES (?, ?, ?, ?, ?)
|
||||
ON CONFLICT(source_id, user_id, entry_ref, option_id) DO UPDATE SET book_id = excluded.book_id`,
|
||||
context.instance.id, context.userId, request.entryRef, request.optionId ?? '', bookId,
|
||||
);
|
||||
this.addToShelf(context.userId, bookId, now);
|
||||
});
|
||||
keepFile = this.db.get<{ rel_path: string }>('SELECT rel_path FROM acquired_files WHERE book_id = ?', bookId)?.rel_path === relPath;
|
||||
keepCover = coverPath !== undefined && this.db.get<{ cover_path: string | null }>('SELECT cover_path FROM books WHERE id = ?', bookId)?.cover_path === coverPath;
|
||||
return bookId;
|
||||
} finally {
|
||||
resource?.stream?.destroy();
|
||||
await rm(temporaryPath, { force: true });
|
||||
if (finalPath && !keepFile) await rm(finalPath, { force: true });
|
||||
if (coverPath && !keepCover) await rm(resolveInside(this.config.dataDir, coverPath), { force: true });
|
||||
}
|
||||
}
|
||||
|
||||
private addToShelf(userId: string, bookId: string, now: number): void {
|
||||
this.db.run(
|
||||
`INSERT INTO user_books (user_id, book_id, added_at, hidden) VALUES (?, ?, ?, 0)
|
||||
ON CONFLICT(user_id, book_id) DO UPDATE SET hidden = 0`,
|
||||
userId, bookId, now,
|
||||
);
|
||||
}
|
||||
|
||||
private insertBook(
|
||||
bookId: string, parsed: ParsedSource, contentHash: string, size: number,
|
||||
coverPath: string | undefined, entry: CatalogEntry, context: SourceContext, now: number,
|
||||
): void {
|
||||
const m = parsed.metadata;
|
||||
const embedded = m.source === 'embedded';
|
||||
// Importing another copy must not refresh or overwrite existing metadata.
|
||||
this.db.run(
|
||||
`INSERT OR IGNORE INTO books (id, identifier, content_hash, format, title, author, publisher, language,
|
||||
isbn, description, series, series_index, tags, pubdate, cover_path, file_size, page_count, meta_json,
|
||||
source, created_at, updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`,
|
||||
bookId, m.identifier, contentHash, parsed.format,
|
||||
(embedded ? m.title : entry.title) || m.title || entry.title,
|
||||
m.author || entry.authors?.join(', ') || '', m.publisher, m.language || entry.language || '',
|
||||
m.isbn, m.description || entry.description || '', m.series, m.seriesIndex,
|
||||
JSON.stringify(m.tags), m.pubdate || entry.publishedAt || '', coverPath ?? null, size,
|
||||
parsed.pageCount, JSON.stringify({ ...m.raw, kind: parsed.kind }),
|
||||
embedded ? m.source : `provider:${context.instance.pluginId}`, now, now,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
import type { Db } from '../db/index.ts';
|
||||
import type { ShelfService, BookDto } from '../services/shelf.ts';
|
||||
import { badRequest, notFound } from '../lib/errors.ts';
|
||||
import type { SourceProvider, CatalogEntry } from './types.ts';
|
||||
|
||||
function entry(book: Pick<BookDto, 'id' | 'title' | 'author' | 'description' | 'format'>): CatalogEntry {
|
||||
return {
|
||||
ref: book.id, title: book.title, authors: book.author ? [book.author] : [],
|
||||
description: book.description,
|
||||
options: [{ id: 'read', label: '加入书架', available: true }], metadata: { format: book.format },
|
||||
};
|
||||
}
|
||||
|
||||
export function createLocalProvider(db: Db, shelf: () => ShelfService): SourceProvider {
|
||||
const detail = async (_ctx: unknown, ref: string): Promise<CatalogEntry> => {
|
||||
const row = db.get<BookDto>(
|
||||
`SELECT id, title, author, description, format FROM books b
|
||||
WHERE b.id = ? AND EXISTS (SELECT 1 FROM book_files f WHERE f.book_id = b.id AND f.missing = 0)`, ref,
|
||||
);
|
||||
if (!row) throw notFound('local book not found', 'ENTRY_NOT_FOUND');
|
||||
return entry(row);
|
||||
};
|
||||
function page(cursor?: string): number {
|
||||
if (cursor && !/^[1-9]\d{0,5}$/.test(cursor)) throw badRequest('invalid catalog cursor');
|
||||
return Number(cursor ?? 1);
|
||||
}
|
||||
return {
|
||||
descriptor: { id: 'local', label: '本地书库', version: '1.0.0', capabilities: ['browse', 'search', 'detail'] },
|
||||
validateConfig(config) {
|
||||
if (!config || typeof config !== 'object' || Array.isArray(config) || Object.keys(config).length) {
|
||||
throw badRequest('local uses the configured BOOKS_DIR and accepts no instance settings');
|
||||
}
|
||||
},
|
||||
async browse(ctx, request) {
|
||||
const result = shelf().list(ctx.userId, {
|
||||
scope: 'library', path: request.ref, page: page(request.cursor), pageSize: request.limit ?? 40,
|
||||
});
|
||||
return { items: result.items.map(entry), ...(result.page * result.pageSize < result.total
|
||||
? { nextCursor: String(result.page + 1) } : {}) };
|
||||
},
|
||||
async search(ctx, request) {
|
||||
const result = shelf().list(ctx.userId, {
|
||||
scope: 'library', search: request.query, page: page(request.cursor), pageSize: request.limit ?? 40,
|
||||
});
|
||||
return { items: result.items.map(entry), ...(result.page * result.pageSize < result.total
|
||||
? { nextCursor: String(result.page + 1) } : {}) };
|
||||
},
|
||||
detail,
|
||||
async acquire(ctx, request) {
|
||||
if (request.optionId && request.optionId !== 'read') throw badRequest('unknown acquisition option');
|
||||
await detail(ctx, request.entryRef);
|
||||
return { kind: 'ready', publicationId: request.entryRef };
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,434 @@
|
||||
import { XMLParser, XMLValidator } from 'fast-xml-parser';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { Readable } from 'node:stream';
|
||||
import { AppError } from '../lib/errors.ts';
|
||||
import type {
|
||||
Acquisition, CatalogEntry, CatalogPage, SourceContext, SourceProvider,
|
||||
ResourceResponse, SourceHttpResponse,
|
||||
} from './types.ts';
|
||||
|
||||
const ACQUISITION = 'http://opds-spec.org/acquisition';
|
||||
const FILE_TYPES = new Set([
|
||||
'application/epub+zip', 'application/pdf', 'text/plain',
|
||||
'application/vnd.comicbook+zip', 'application/x-cbz',
|
||||
]);
|
||||
|
||||
type ObjectValue = Record<string, unknown>;
|
||||
interface OpdsLink { href: string; rel: string[]; type: string; title: string }
|
||||
interface OpdsBook {
|
||||
id: string;
|
||||
title: string;
|
||||
authors: string[];
|
||||
description?: string;
|
||||
coverUrl?: string;
|
||||
detailUrl?: string;
|
||||
acquisitions: OpdsLink[];
|
||||
}
|
||||
interface OpdsDocument {
|
||||
title: string;
|
||||
books: OpdsBook[];
|
||||
navigation: Array<{ title: string; url: string }>;
|
||||
next?: string;
|
||||
search?: OpdsLink;
|
||||
}
|
||||
|
||||
function object(value: unknown): ObjectValue {
|
||||
return value !== null && typeof value === 'object' && !Array.isArray(value) ? value as ObjectValue : {};
|
||||
}
|
||||
|
||||
function list(value: unknown): unknown[] {
|
||||
return value === undefined || value === null ? [] : Array.isArray(value) ? value : [value];
|
||||
}
|
||||
|
||||
function text(value: unknown): string {
|
||||
if (typeof value === 'string') return value;
|
||||
if (typeof value === 'number') return String(value);
|
||||
const record = object(value);
|
||||
return typeof record['#text'] === 'string' ? record['#text'] : '';
|
||||
}
|
||||
|
||||
function url(value: string, base: string): string | undefined {
|
||||
try {
|
||||
const parsed = new URL(value, base);
|
||||
if (!['http:', 'https:'].includes(parsed.protocol) || parsed.username || parsed.password) return undefined;
|
||||
return parsed.href;
|
||||
} catch { return undefined; }
|
||||
}
|
||||
|
||||
function links(value: unknown, base: string, atom = false): OpdsLink[] {
|
||||
return list(value).flatMap((item) => {
|
||||
const link = object(item);
|
||||
const prefix = atom ? '@_' : '';
|
||||
const href = url(text(link[`${prefix}href`]), base);
|
||||
if (!text(link[`${prefix}href`]) || !href) return [];
|
||||
const rel = list(link[`${prefix}rel`]).flatMap((value) => text(value).split(/\s+/)).filter(Boolean);
|
||||
return [{ href, rel, type: text(link[`${prefix}type`]), title: text(link[`${prefix}title`]) }];
|
||||
});
|
||||
}
|
||||
|
||||
function plainType(type: string): string { return type.split(';')[0]!.trim().toLowerCase(); }
|
||||
function hasRel(link: OpdsLink, rel: string): boolean { return link.rel.includes(rel); }
|
||||
function isAcquisition(link: OpdsLink): boolean { return link.rel.some((rel) => rel === ACQUISITION || rel.startsWith(`${ACQUISITION}/`)); }
|
||||
function downloadable(link: OpdsLink): boolean {
|
||||
return (hasRel(link, ACQUISITION) || hasRel(link, `${ACQUISITION}/open-access`)) && FILE_TYPES.has(plainType(link.type));
|
||||
}
|
||||
|
||||
function documentLinks(parsed: OpdsDocument, all: OpdsLink[]): void {
|
||||
parsed.next = all.find((link) => hasRel(link, 'next'))?.href;
|
||||
parsed.search = all.find((link) => hasRel(link, 'search'));
|
||||
}
|
||||
|
||||
function parseJsonPublication(raw: unknown, base: string): OpdsBook | undefined {
|
||||
const item = object(raw);
|
||||
const metadata = object(item.metadata);
|
||||
const all = links(item.links, base);
|
||||
const id = text(metadata.identifier) || all.find((link) => hasRel(link, 'self'))?.href || all.find(isAcquisition)?.href;
|
||||
const title = text(metadata.title);
|
||||
if (!id || !title) return undefined;
|
||||
return {
|
||||
id, title,
|
||||
authors: list(metadata.author).map((author) => text(author) || text(object(author).name)).filter(Boolean),
|
||||
description: text(metadata.description) || undefined,
|
||||
coverUrl: links(item.images, base)[0]?.href,
|
||||
detailUrl: all.find((link) => hasRel(link, 'self') && plainType(link.type) === 'application/opds-publication+json')?.href,
|
||||
acquisitions: all.filter(isAcquisition),
|
||||
};
|
||||
}
|
||||
|
||||
function parseJson(raw: unknown, base: string): OpdsDocument {
|
||||
const feed = object(raw);
|
||||
if (!('metadata' in feed) || !text(object(feed.metadata).title)) throw new Error('Invalid OPDS JSON document');
|
||||
const parsed: OpdsDocument = { title: text(object(feed.metadata).title), books: [], navigation: [] };
|
||||
documentLinks(parsed, links(feed.links, base));
|
||||
const sections = [feed, ...list(feed.groups).map(object)];
|
||||
for (const section of sections) {
|
||||
for (const item of list(section.publications)) {
|
||||
const book = parseJsonPublication(item, base);
|
||||
if (book) parsed.books.push(book);
|
||||
}
|
||||
for (const link of links(section.navigation, base)) {
|
||||
parsed.navigation.push({ title: link.title || link.href, url: link.href });
|
||||
}
|
||||
}
|
||||
// A detail URL can return a single OPDS publication rather than a feed.
|
||||
if (!('publications' in feed) && !('navigation' in feed) && !('groups' in feed)) {
|
||||
const book = parseJsonPublication(feed, base);
|
||||
if (book) parsed.books.push(book);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function parseXml(body: string): ObjectValue {
|
||||
if (/<!DOCTYPE|<!ENTITY/i.test(body) || XMLValidator.validate(body) !== true) {
|
||||
throw new Error('Invalid OPDS XML document');
|
||||
}
|
||||
return object(new XMLParser({
|
||||
ignoreAttributes: false, removeNSPrefix: true, parseTagValue: false,
|
||||
processEntities: true, trimValues: true,
|
||||
}).parse(body));
|
||||
}
|
||||
|
||||
function parseAtom(body: string, base: string): OpdsDocument {
|
||||
const root = parseXml(body);
|
||||
if (!root.feed && !root.entry) throw new Error('Invalid OPDS Atom document');
|
||||
const feed = object(root.feed ?? root.entry);
|
||||
const feedBase = url(text(feed['@_base']), base) ?? base;
|
||||
const parsed: OpdsDocument = { title: text(feed.title), books: [], navigation: [] };
|
||||
documentLinks(parsed, links(feed.link, feedBase, true));
|
||||
for (const raw of root.entry ? [root.entry] : list(feed.entry)) {
|
||||
const entry = object(raw);
|
||||
const entryBase = url(text(entry['@_base']), feedBase) ?? feedBase;
|
||||
const all = links(entry.link, entryBase, true);
|
||||
const acquisitions = all.filter(isAcquisition);
|
||||
const title = text(entry.title);
|
||||
if (acquisitions.length === 0) {
|
||||
const navigation = all.find((link) => (!link.rel.length || hasRel(link, 'subsection') || hasRel(link, 'alternate')) && /atom\+xml|opds\+json/i.test(link.type));
|
||||
if (navigation) parsed.navigation.push({ title: title || navigation.title || navigation.href, url: navigation.href });
|
||||
continue;
|
||||
}
|
||||
const id = text(entry.id) || all.find((link) => hasRel(link, 'self'))?.href || acquisitions[0]!.href;
|
||||
if (!title) continue;
|
||||
parsed.books.push({
|
||||
id, title,
|
||||
authors: list(entry.author).map((author) => text(object(author).name)).filter(Boolean),
|
||||
description: text(entry.summary) || text(entry.content) || undefined,
|
||||
coverUrl: all.find((link) => hasRel(link, 'http://opds-spec.org/image/thumbnail'))?.href
|
||||
?? all.find((link) => hasRel(link, 'http://opds-spec.org/image'))?.href,
|
||||
detailUrl: all.find((link) => hasRel(link, 'alternate') && /atom\+xml/i.test(link.type))?.href,
|
||||
acquisitions,
|
||||
});
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function parseDocument(body: string, base: string): OpdsDocument {
|
||||
return body.trimStart().startsWith('<') ? parseAtom(body, base) : parseJson(JSON.parse(body), base);
|
||||
}
|
||||
|
||||
interface OpdsConfig {
|
||||
url: string;
|
||||
username?: string;
|
||||
allowedOrigins?: string[];
|
||||
}
|
||||
|
||||
export interface OpdsProviderOptions {
|
||||
fetch?: typeof globalThis.fetch;
|
||||
maxFeedBytes?: number;
|
||||
maxFileBytes?: number;
|
||||
}
|
||||
|
||||
function fail(message: string, code = 'OPDS_INVALID_RESPONSE', status = 502): never {
|
||||
throw new AppError(status, code, message);
|
||||
}
|
||||
|
||||
function configuration(raw: unknown): OpdsConfig {
|
||||
const value = object(raw);
|
||||
if (!text(value.url) || !url(text(value.url), text(value.url))) {
|
||||
fail('OPDS url must be an absolute HTTP(S) URL without credentials', 'INVALID_SOURCE_CONFIG', 400);
|
||||
}
|
||||
if ('password' in value) fail('Store the OPDS password in credentials, not public configuration', 'INVALID_SOURCE_CONFIG', 400);
|
||||
if (value.username !== undefined && typeof value.username !== 'string') {
|
||||
fail('OPDS username must be a string', 'INVALID_SOURCE_CONFIG', 400);
|
||||
}
|
||||
if (value.allowedOrigins !== undefined && (!Array.isArray(value.allowedOrigins) || value.allowedOrigins.some((origin) => {
|
||||
if (typeof origin !== 'string') return true;
|
||||
const normalized = url(origin, origin);
|
||||
return !normalized || new URL(normalized).origin !== origin;
|
||||
}))) fail('allowedOrigins must contain HTTP(S) origins without paths', 'INVALID_SOURCE_CONFIG', 400);
|
||||
return { url: new URL(text(value.url)).href, username: value.username as string | undefined, allowedOrigins: value.allowedOrigins as string[] | undefined };
|
||||
}
|
||||
|
||||
function checkedUrl(candidate: string, config: OpdsConfig, file = false): string {
|
||||
const normalized = url(candidate, config.url);
|
||||
if (!normalized) fail('OPDS link must use HTTP(S) without URL credentials', 'OPDS_UNSAFE_URL', 400);
|
||||
const target = new URL(normalized);
|
||||
const allowed = [new URL(config.url).origin, ...(file ? config.allowedOrigins ?? [] : [])];
|
||||
if (!allowed.includes(target.origin)) fail('OPDS link origin is not allowed by this source', 'OPDS_UNSAFE_URL', 400);
|
||||
return target.href;
|
||||
}
|
||||
|
||||
function encodeRef(prefix: string, value: unknown): string {
|
||||
return `${prefix}${Buffer.from(JSON.stringify(value)).toString('base64url')}`;
|
||||
}
|
||||
|
||||
function decodeRef(ref: string, prefix: string): ObjectValue {
|
||||
if (!ref.startsWith(prefix) || ref.length > 16_384) fail('Invalid OPDS reference', 'INVALID_SOURCE_REF', 400);
|
||||
try { return object(JSON.parse(Buffer.from(ref.slice(prefix.length), 'base64url').toString('utf8'))); }
|
||||
catch { return fail('Invalid OPDS reference', 'INVALID_SOURCE_REF', 400); }
|
||||
}
|
||||
|
||||
function optionId(link: OpdsLink): string {
|
||||
return createHash('sha256').update(`${link.href}\n${link.type}\n${link.rel.join(' ')}`).digest('hex').slice(0, 24);
|
||||
}
|
||||
|
||||
function catalogEntry(book: OpdsBook, feedUrl: string): CatalogEntry {
|
||||
return {
|
||||
ref: encodeRef('opds:', { url: feedUrl, id: book.id }),
|
||||
title: book.title, authors: book.authors, description: book.description, coverUrl: book.coverUrl,
|
||||
options: book.acquisitions.map((link) => ({
|
||||
id: optionId(link), mediaType: link.type,
|
||||
label: link.title || (downloadable(link) ? plainType(link.type) : `Unsupported acquisition: ${link.rel.join(', ')}`),
|
||||
available: downloadable(link),
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
function catalogPage(document: OpdsDocument, feedUrl: string): CatalogPage {
|
||||
return {
|
||||
title: document.title,
|
||||
items: document.books.map((book) => catalogEntry(book, feedUrl)),
|
||||
navigation: document.navigation.map((entry) => ({ ref: entry.url, title: entry.title, kind: 'catalog' })),
|
||||
nextCursor: document.next,
|
||||
};
|
||||
}
|
||||
|
||||
function expandSearch(template: string, query: string, limit: number): string {
|
||||
const normalized = template.replace(/%7B/gi, '{').replace(/%7D/gi, '}');
|
||||
if (!normalized.includes('{searchTerms}')) fail('OPDS search template has no searchTerms parameter', 'SOURCE_UNSUPPORTED', 400);
|
||||
return normalized.replace(/\{([^}]+)\}/g, (_match, name: string) => {
|
||||
if (name === 'searchTerms') return encodeURIComponent(query);
|
||||
if (name === 'count' || name === 'count?') return String(limit);
|
||||
if (name === 'startIndex' || name === 'startPage') return '1';
|
||||
if (name.endsWith('?')) return '';
|
||||
return fail(`Unsupported OPDS search parameter: ${name}`, 'SOURCE_UNSUPPORTED', 400);
|
||||
});
|
||||
}
|
||||
|
||||
/** Built-in OPDS 1/2 source. Credentials and downloads stay inside the provider. */
|
||||
export function createOpdsProvider(options: OpdsProviderOptions = {}): SourceProvider {
|
||||
const fetcher = options.fetch ?? globalThis.fetch;
|
||||
const maxFeedBytes = options.maxFeedBytes ?? 4 * 1024 * 1024;
|
||||
const maxFileBytes = options.maxFileBytes ?? 256 * 1024 * 1024;
|
||||
|
||||
async function request(ctx: SourceContext, candidate: string, file = false): Promise<Response | SourceHttpResponse> {
|
||||
const config = configuration(ctx.instance.config);
|
||||
let target = checkedUrl(candidate, config, file);
|
||||
for (let redirects = 0; redirects <= 5; redirects += 1) {
|
||||
ctx.signal.throwIfAborted();
|
||||
const headers: Record<string, string> = { accept: file ? '*/*' : 'application/opds+json, application/atom+xml, application/opds-publication+json, application/opensearchdescription+xml' };
|
||||
if (config.username && new URL(target).origin === new URL(config.url).origin) {
|
||||
const password = await ctx.credentials?.get('password', ctx.signal);
|
||||
if (password === undefined) fail('OPDS credentials are required', 'AUTH_REQUIRED', 401);
|
||||
headers.authorization = `Basic ${Buffer.from(`${config.username}:${password}`).toString('base64')}`;
|
||||
}
|
||||
const response = ctx.http
|
||||
? await ctx.http.request({ url: target, headers }, ctx.signal)
|
||||
: await fetcher(target, { headers, signal: ctx.signal, redirect: 'manual' });
|
||||
const responseUrl = response.url || target;
|
||||
checkedUrl(responseUrl, config, file);
|
||||
const header = (name: string) => response.headers instanceof Headers ? response.headers.get(name) : response.headers[name];
|
||||
if ([301, 302, 303, 307, 308].includes(response.status)) {
|
||||
const location = header('location');
|
||||
if (response instanceof Response) await response.body?.cancel();
|
||||
if (!location || redirects === 5) fail('OPDS redirect limit exceeded or Location missing');
|
||||
target = checkedUrl(new URL(location, target).href, config, file);
|
||||
continue;
|
||||
}
|
||||
if (response.status < 200 || response.status >= 300) {
|
||||
if (response instanceof Response) await response.body?.cancel();
|
||||
if (response.status === 401 || response.status === 403) fail('OPDS authentication is required', 'AUTH_REQUIRED', 401);
|
||||
if (response.status === 404) fail('OPDS resource no longer exists', 'RESOURCE_GONE', 404);
|
||||
if (response.status === 429) fail('OPDS request rate is limited', 'RATE_LIMITED', 429);
|
||||
fail(`OPDS server returned HTTP ${response.status}`);
|
||||
}
|
||||
const size = Number(header('content-length'));
|
||||
if (Number.isFinite(size) && size > (file ? maxFileBytes : maxFeedBytes)) {
|
||||
if (response instanceof Response) await response.body?.cancel();
|
||||
fail('OPDS resource exceeds the configured size limit', 'RESOURCE_TOO_LARGE', 413);
|
||||
}
|
||||
return response;
|
||||
}
|
||||
return fail('OPDS redirect limit exceeded');
|
||||
}
|
||||
|
||||
async function* bytes(response: Response | SourceHttpResponse, maxBytes: number): AsyncGenerator<Uint8Array> {
|
||||
let total = 0;
|
||||
const stream = response instanceof Response
|
||||
? response.body ? Readable.fromWeb(response.body as never) : []
|
||||
: [response.body];
|
||||
for await (const chunk of stream) {
|
||||
total += chunk.byteLength;
|
||||
if (total > maxBytes) fail('OPDS resource exceeds the configured size limit', 'RESOURCE_TOO_LARGE', 413);
|
||||
yield chunk;
|
||||
}
|
||||
}
|
||||
|
||||
async function document(ctx: SourceContext, target: string): Promise<{ parsed: OpdsDocument; url: string }> {
|
||||
const response = await request(ctx, target);
|
||||
const chunks: Uint8Array[] = [];
|
||||
for await (const chunk of bytes(response, maxFeedBytes)) chunks.push(chunk);
|
||||
const finalUrl = response.url || target;
|
||||
try { return { parsed: parseDocument(Buffer.concat(chunks).toString('utf8'), finalUrl), url: finalUrl }; }
|
||||
catch (error) {
|
||||
if (error instanceof AppError) throw error;
|
||||
return fail('OPDS response is not a valid Atom or JSON catalog');
|
||||
}
|
||||
}
|
||||
|
||||
async function lookup(ctx: SourceContext, ref: string): Promise<{ book: OpdsBook; url: string }> {
|
||||
const stored = decodeRef(ref, 'opds:');
|
||||
if (!text(stored.url) || !text(stored.id)) fail('Invalid OPDS entry reference', 'INVALID_SOURCE_REF', 400);
|
||||
const feed = await document(ctx, text(stored.url));
|
||||
let book = feed.parsed.books.find((entry) => entry.id === stored.id);
|
||||
if (!book) fail('OPDS entry no longer exists in its catalog', 'RESOURCE_GONE', 404);
|
||||
if (book.detailUrl && book.detailUrl !== feed.url) {
|
||||
const full = await document(ctx, book.detailUrl);
|
||||
book = full.parsed.books.find((entry) => entry.id === stored.id) ?? book;
|
||||
}
|
||||
return { book, url: feed.url };
|
||||
}
|
||||
|
||||
return {
|
||||
descriptor: {
|
||||
id: 'opds', label: 'OPDS', version: '1.0.0',
|
||||
capabilities: ['browse', 'search', 'detail', 'acquire.file'],
|
||||
configSchema: {
|
||||
type: 'object', required: ['url'], additionalProperties: false,
|
||||
properties: {
|
||||
url: { type: 'string', format: 'uri', title: 'OPDS catalog URL' },
|
||||
username: { type: 'string', title: 'Basic authentication username (password is stored separately)' },
|
||||
allowedOrigins: { type: 'array', items: { type: 'string', format: 'uri' }, title: 'Additional download origins' },
|
||||
},
|
||||
},
|
||||
},
|
||||
validateConfig: (config) => { configuration(config); },
|
||||
async browse(ctx, request) {
|
||||
const config = configuration(ctx.instance.config);
|
||||
const feed = await document(ctx, request.cursor || request.ref || config.url);
|
||||
return catalogPage(feed.parsed, feed.url);
|
||||
},
|
||||
async search(ctx, search) {
|
||||
const config = configuration(ctx.instance.config);
|
||||
if (search.cursor) {
|
||||
const feed = await document(ctx, search.cursor);
|
||||
return catalogPage(feed.parsed, feed.url);
|
||||
}
|
||||
if (!search.query.trim()) fail('Search query must not be empty', 'BAD_REQUEST', 400);
|
||||
const root = await document(ctx, config.url);
|
||||
const link = root.parsed.search;
|
||||
if (!link) fail('This OPDS catalog does not advertise search', 'SOURCE_UNSUPPORTED', 400);
|
||||
let template = link.href;
|
||||
if (plainType(link.type) === 'application/opensearchdescription+xml') {
|
||||
const response = await request(ctx, link.href);
|
||||
const chunks: Uint8Array[] = [];
|
||||
for await (const chunk of bytes(response, maxFeedBytes)) chunks.push(chunk);
|
||||
let description: ObjectValue;
|
||||
try { description = object(parseXml(Buffer.concat(chunks).toString('utf8')).OpenSearchDescription); }
|
||||
catch { return fail('Invalid OPDS OpenSearch description'); }
|
||||
const candidates = list(description.Url).map(object);
|
||||
const candidate = candidates.find((item) => /atom\+xml|opds\+json/i.test(text(item['@_type'])) && (!item['@_method'] || text(item['@_method']).toUpperCase() === 'GET'));
|
||||
const rawTemplate = candidate ? text(candidate['@_template']) : '';
|
||||
if (!rawTemplate) fail('OPDS search has no supported GET template', 'SOURCE_UNSUPPORTED', 400);
|
||||
template = new URL(rawTemplate, response.url || link.href).href;
|
||||
}
|
||||
const target = expandSearch(template, search.query, Math.max(1, Math.min(search.limit ?? 50, 200)));
|
||||
const feed = await document(ctx, target);
|
||||
return catalogPage(feed.parsed, feed.url);
|
||||
},
|
||||
async detail(ctx, entryRef) {
|
||||
const entry = await lookup(ctx, entryRef);
|
||||
return { ...catalogEntry(entry.book, entry.url), ref: entryRef };
|
||||
},
|
||||
async acquire(ctx, request): Promise<Acquisition> {
|
||||
const { book } = await lookup(ctx, request.entryRef);
|
||||
const selected = request.optionId
|
||||
? book.acquisitions.find((link) => optionId(link) === request.optionId)
|
||||
: book.acquisitions.find(downloadable) ?? book.acquisitions[0];
|
||||
if (!selected) fail('OPDS acquisition option no longer exists', 'RESOURCE_GONE', 404);
|
||||
if (!downloadable(selected)) {
|
||||
return { kind: 'action-required', action: {
|
||||
type: hasRel(selected, `${ACQUISITION}/borrow`) ? 'borrow' : 'external',
|
||||
label: 'This release supports direct EPUB, PDF, TXT and CBZ downloads; borrowing, purchases and DRM require an external application.',
|
||||
url: selected.href,
|
||||
} };
|
||||
}
|
||||
checkedUrl(selected.href, configuration(ctx.instance.config), true);
|
||||
return {
|
||||
kind: 'file', mediaType: plainType(selected.type),
|
||||
acquisitionRef: encodeRef('opds-file:', { entryRef: request.entryRef, optionId: optionId(selected) }),
|
||||
};
|
||||
},
|
||||
async openFile(ctx, acquisition): Promise<ResourceResponse> {
|
||||
const ref = decodeRef(acquisition.acquisitionRef, 'opds-file:');
|
||||
if (!text(ref.entryRef) || !text(ref.optionId)) fail('Invalid OPDS acquisition reference', 'INVALID_SOURCE_REF', 400);
|
||||
const { book } = await lookup(ctx, text(ref.entryRef));
|
||||
const selected = book.acquisitions.find((link) => optionId(link) === ref.optionId && downloadable(link));
|
||||
if (!selected) fail('OPDS download no longer exists', 'RESOURCE_GONE', 404);
|
||||
const response = await request(ctx, selected.href, true);
|
||||
const header = (name: string) => response.headers instanceof Headers ? response.headers.get(name) : response.headers[name];
|
||||
const actualType = plainType(header('content-type') ?? '');
|
||||
if (actualType && actualType !== 'application/octet-stream' && actualType !== plainType(selected.type)) {
|
||||
if (response instanceof Response) await response.body?.cancel();
|
||||
fail('OPDS download media type differs from its catalog entry');
|
||||
}
|
||||
const length = header('content-length');
|
||||
return {
|
||||
mediaType: plainType(selected.type), stream: Readable.from(bytes(response, maxFileBytes)),
|
||||
size: length && Number.isFinite(Number(length)) ? Number(length) : undefined,
|
||||
etag: header('etag') ?? undefined,
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export const opdsProvider = createOpdsProvider();
|
||||
@@ -0,0 +1,283 @@
|
||||
import { realpath, stat } from 'node:fs/promises';
|
||||
import { isAbsolute, join, relative, resolve, sep } from 'node:path';
|
||||
import type { Db } from '../db/index.ts';
|
||||
import { AppError } from '../lib/errors.ts';
|
||||
import { ProcessPlugin } from './process-plugin.ts';
|
||||
import { SourceRegistry, SourceRegistryError } from './registry.ts';
|
||||
import type { PluginRuntimeStatus, SourceDescriptor, SourceProvider } from './types.ts';
|
||||
|
||||
interface InstalledPlugin { plugin_id: string; folder: string; enabled: number }
|
||||
interface ManagedPlugin {
|
||||
plugin?: ProcessPlugin;
|
||||
providers: readonly SourceProvider[];
|
||||
active: boolean;
|
||||
error?: { code: string; message: string };
|
||||
}
|
||||
|
||||
export interface PluginInfo {
|
||||
pluginId: string;
|
||||
folder?: string;
|
||||
builtin: boolean;
|
||||
enabled: boolean;
|
||||
name?: string;
|
||||
version?: string;
|
||||
sourceTypes: readonly SourceDescriptor[];
|
||||
runtime: PluginRuntimeStatus | null;
|
||||
error?: { code: string; message: string };
|
||||
}
|
||||
|
||||
function pluginError(status: number, code: string, message: string): AppError {
|
||||
return new AppError(status, code, message);
|
||||
}
|
||||
|
||||
function inside(root: string, target: string): boolean {
|
||||
const rel = relative(root, target);
|
||||
return rel !== '' && rel !== '..' && !rel.startsWith(`..${sep}`) && !isAbsolute(rel);
|
||||
}
|
||||
|
||||
/**
|
||||
* Installs trusted packages already present under DATA_DIR/plugins. All lifecycle
|
||||
* mutations are serialized; unloading removes providers before awaiting workers.
|
||||
*/
|
||||
export class PluginManager {
|
||||
private readonly managed = new Map<string, ManagedPlugin>();
|
||||
private pending: Promise<void> = Promise.resolve();
|
||||
private closed = false;
|
||||
|
||||
constructor(private readonly db: Db, private readonly dataDir: string, private readonly registry: SourceRegistry) {}
|
||||
|
||||
loadInstalled(): Promise<void> {
|
||||
return this.enqueue(async () => {
|
||||
for (const row of this.rows()) {
|
||||
if (row.enabled !== 1 || this.managed.get(row.plugin_id)?.active) continue;
|
||||
try {
|
||||
this.assertExternal(row.plugin_id);
|
||||
await this.activate(row);
|
||||
} catch (error) {
|
||||
// One bad package must not prevent the library or other plugins starting.
|
||||
this.managed.set(row.plugin_id, { active: false, providers: [], error: this.errorInfo(error) });
|
||||
}
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
install(folder: string): Promise<PluginInfo> {
|
||||
return this.enqueue(async () => {
|
||||
const plugin = await this.loadPackage(folder);
|
||||
let managed: ManagedPlugin | undefined;
|
||||
try {
|
||||
this.assertExternal(plugin.manifest.id);
|
||||
if (this.rows().some((row) => row.plugin_id === plugin.manifest.id || row.folder === folder)) {
|
||||
throw pluginError(409, 'PLUGIN_ALREADY_INSTALLED', 'This plugin id or package folder is already installed');
|
||||
}
|
||||
managed = this.prepare(plugin);
|
||||
this.register(managed);
|
||||
this.db.run('INSERT INTO installed_plugins (plugin_id, folder, enabled) VALUES (?, ?, 1)', plugin.manifest.id, folder);
|
||||
this.managed.set(plugin.manifest.id, managed);
|
||||
return this.info({ plugin_id: plugin.manifest.id, folder, enabled: 1 });
|
||||
} catch (error) {
|
||||
if (managed) this.unregister(managed);
|
||||
await plugin.close();
|
||||
throw this.packageInputError(error);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
setEnabled(pluginId: string, enabled: boolean): Promise<PluginInfo> {
|
||||
return this.enqueue(async () => {
|
||||
this.assertExternal(pluginId);
|
||||
const row = this.row(pluginId);
|
||||
const previous = this.managed.get(pluginId);
|
||||
if (!enabled) {
|
||||
this.db.run('UPDATE installed_plugins SET enabled = 0 WHERE plugin_id = ?', pluginId);
|
||||
if (previous) await this.stop(previous);
|
||||
return this.info({ ...row, enabled: 0 });
|
||||
}
|
||||
if (row.enabled === 1 && previous?.active && previous.plugin?.status().state !== 'failed') return this.info(row);
|
||||
if (previous) await this.stop(previous);
|
||||
try {
|
||||
await this.activate(row);
|
||||
this.db.run('UPDATE installed_plugins SET enabled = 1 WHERE plugin_id = ?', pluginId);
|
||||
} catch (error) {
|
||||
const failed = this.managed.get(pluginId);
|
||||
if (failed) await this.stop(failed);
|
||||
this.managed.set(pluginId, { active: false, providers: [], error: this.errorInfo(error) });
|
||||
throw error;
|
||||
}
|
||||
return this.info({ ...row, enabled: 1 });
|
||||
});
|
||||
}
|
||||
|
||||
uninstall(pluginId: string): Promise<void> {
|
||||
return this.enqueue(async () => {
|
||||
this.assertExternal(pluginId);
|
||||
this.row(pluginId);
|
||||
this.db.run('DELETE FROM installed_plugins WHERE plugin_id = ?', pluginId);
|
||||
const managed = this.managed.get(pluginId);
|
||||
if (managed) await this.stop(managed);
|
||||
this.managed.delete(pluginId);
|
||||
// Package files, source instances, credentials and reading state are retained.
|
||||
});
|
||||
}
|
||||
|
||||
list(): PluginInfo[] {
|
||||
const builtins = new Map<string, PluginInfo>();
|
||||
for (const registration of this.registry.list()) {
|
||||
if (!registration.builtin) continue;
|
||||
const item = builtins.get(registration.pluginId) ?? {
|
||||
pluginId: registration.pluginId, builtin: true, enabled: true,
|
||||
name: registration.provider.descriptor.label, version: registration.provider.descriptor.version,
|
||||
sourceTypes: [], runtime: null,
|
||||
};
|
||||
item.sourceTypes = [...item.sourceTypes, registration.provider.descriptor];
|
||||
builtins.set(registration.pluginId, item);
|
||||
}
|
||||
return [...builtins.values(), ...this.rows().filter((row) => !builtins.has(row.plugin_id)).map((row) => this.info(row))];
|
||||
}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.enqueue(async () => {
|
||||
if (this.closed) return;
|
||||
this.closed = true;
|
||||
const outcomes = await Promise.allSettled([...this.managed.values()].map((managed) => this.stop(managed)));
|
||||
const failure = outcomes.find((outcome) => outcome.status === 'rejected');
|
||||
if (failure?.status === 'rejected') throw failure.reason;
|
||||
}, true);
|
||||
}
|
||||
|
||||
private enqueue<T>(action: () => Promise<T>, allowClosed = false): Promise<T> {
|
||||
const work = this.pending.then(() => {
|
||||
if (this.closed && !allowClosed) throw pluginError(503, 'PLUGIN_MANAGER_CLOSED', 'Plugin manager is closed');
|
||||
return action();
|
||||
});
|
||||
this.pending = work.then(() => {}, () => {});
|
||||
return work;
|
||||
}
|
||||
|
||||
private rows(): InstalledPlugin[] {
|
||||
return this.db.all<InstalledPlugin>('SELECT plugin_id, folder, enabled FROM installed_plugins ORDER BY plugin_id');
|
||||
}
|
||||
|
||||
private row(pluginId: string): InstalledPlugin {
|
||||
const row = this.db.get<InstalledPlugin>('SELECT plugin_id, folder, enabled FROM installed_plugins WHERE plugin_id = ?', pluginId);
|
||||
if (!row) throw pluginError(404, 'PLUGIN_NOT_FOUND', 'Plugin is not installed');
|
||||
return row;
|
||||
}
|
||||
|
||||
private assertExternal(pluginId: string): void {
|
||||
if (this.registry.list().some((registration) => registration.pluginId === pluginId && registration.builtin)) {
|
||||
throw pluginError(400, 'BUILTIN_PLUGIN_IMMUTABLE', 'Built-in source providers cannot be replaced, disabled or uninstalled');
|
||||
}
|
||||
}
|
||||
|
||||
private async packageDirectory(folder: string): Promise<string> {
|
||||
if (typeof folder !== 'string' || !/^[a-zA-Z0-9](?:[a-zA-Z0-9._-]{0,126}[a-zA-Z0-9_-])?$/.test(folder)) {
|
||||
throw pluginError(400, 'PLUGIN_INVALID_FOLDER', 'Plugin folder must be a single directory name');
|
||||
}
|
||||
const data = await realpath(this.dataDir);
|
||||
const root = await realpath(resolve(data, 'plugins'));
|
||||
if (!inside(data, root)) throw pluginError(400, 'PLUGIN_PATH_ESCAPE', 'Plugin root must remain within DATA_DIR');
|
||||
const directory = await realpath(join(root, folder));
|
||||
if (!inside(root, directory)) throw pluginError(400, 'PLUGIN_PATH_ESCAPE', 'Plugin package must remain within DATA_DIR/plugins');
|
||||
const manifest = await realpath(join(directory, 'plugin.json'));
|
||||
if (!inside(directory, manifest)) {
|
||||
throw pluginError(400, 'PLUGIN_PATH_ESCAPE', 'Plugin package must remain within DATA_DIR/plugins');
|
||||
}
|
||||
if (!(await stat(directory)).isDirectory()) throw pluginError(400, 'PLUGIN_INVALID_FOLDER', 'Plugin package must be a directory');
|
||||
return directory;
|
||||
}
|
||||
|
||||
private async loadPackage(folder: string): Promise<ProcessPlugin> {
|
||||
try { return await ProcessPlugin.load(await this.packageDirectory(folder)); }
|
||||
catch (error) { throw this.packageInputError(error); }
|
||||
}
|
||||
|
||||
private packageInputError(error: unknown): unknown {
|
||||
if (error instanceof AppError) {
|
||||
if (error.code === 'PLUGIN_INVALID' || error.code === 'PLUGIN_INCOMPATIBLE') {
|
||||
return pluginError(400, error.code, error.message);
|
||||
}
|
||||
return error;
|
||||
}
|
||||
if (error instanceof SyntaxError) return pluginError(400, 'PLUGIN_INVALID_MANIFEST', 'Plugin manifest must contain valid JSON');
|
||||
if (error instanceof SourceRegistryError) return pluginError(400, 'PLUGIN_INVALID_MANIFEST', error.message);
|
||||
const code = (error as NodeJS.ErrnoException | undefined)?.code;
|
||||
if (code === 'ENOENT') return pluginError(404, 'PLUGIN_PACKAGE_NOT_FOUND', 'Plugin folder, manifest or entry file does not exist');
|
||||
if (code === 'ENOTDIR' || code === 'EISDIR') return pluginError(400, 'PLUGIN_INVALID_PACKAGE', 'Plugin package must contain a manifest and an executable entry file');
|
||||
if (code === 'EACCES' || code === 'EPERM') return pluginError(403, 'PLUGIN_PACKAGE_UNREADABLE', 'The server cannot read this plugin package');
|
||||
if (code === 'ELOOP') return pluginError(400, 'PLUGIN_PATH_ESCAPE', 'Plugin package contains a symbolic link loop');
|
||||
return error;
|
||||
}
|
||||
|
||||
private prepare(plugin: ProcessPlugin): ManagedPlugin {
|
||||
if (this.registry.list().some((registration) => registration.pluginId === plugin.manifest.id)) {
|
||||
throw pluginError(409, 'PLUGIN_SOURCE_CONFLICT', 'The plugin already owns a registered source type');
|
||||
}
|
||||
const managed: ManagedPlugin = { plugin, providers: [], active: true };
|
||||
managed.providers = plugin.providers().map((provider) => new Proxy(provider, {
|
||||
get(target, key, receiver) {
|
||||
const value = Reflect.get(target, key, receiver) as unknown;
|
||||
if (typeof value !== 'function') return value;
|
||||
return async (...args: unknown[]) => {
|
||||
// Existing callers may retain providers after registry removal. Do not
|
||||
// let those references restart a worker after disable or uninstall.
|
||||
if (!managed.active) throw pluginError(503, 'PLUGIN_UNAVAILABLE', 'Plugin is disabled or uninstalled');
|
||||
return value.apply(target, args);
|
||||
};
|
||||
},
|
||||
}));
|
||||
const validation = new SourceRegistry();
|
||||
for (const provider of managed.providers) validation.registerPluginManifest(plugin.manifest, provider);
|
||||
return managed;
|
||||
}
|
||||
|
||||
private register(managed: ManagedPlugin): void {
|
||||
for (const provider of managed.providers) this.registry.registerPluginManifest(managed.plugin!.manifest, provider);
|
||||
}
|
||||
|
||||
private unregister(managed: ManagedPlugin): void {
|
||||
managed.active = false;
|
||||
for (const provider of managed.providers) {
|
||||
const pluginId = managed.plugin!.manifest.id;
|
||||
if (this.registry.get(pluginId, provider.descriptor.id)?.provider === provider) {
|
||||
this.registry.unregister(pluginId, provider.descriptor.id);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private async stop(managed: ManagedPlugin): Promise<void> {
|
||||
this.unregister(managed);
|
||||
await managed.plugin?.close();
|
||||
}
|
||||
|
||||
private async activate(row: InstalledPlugin): Promise<void> {
|
||||
const plugin = await this.loadPackage(row.folder);
|
||||
let managed: ManagedPlugin | undefined;
|
||||
try {
|
||||
if (plugin.manifest.id !== row.plugin_id) throw pluginError(409, 'PLUGIN_ID_CHANGED', 'Installed package plugin id no longer matches its installation record');
|
||||
managed = this.prepare(plugin);
|
||||
this.register(managed);
|
||||
this.managed.set(row.plugin_id, managed);
|
||||
} catch (error) {
|
||||
if (managed) this.unregister(managed);
|
||||
await plugin.close();
|
||||
throw this.packageInputError(error);
|
||||
}
|
||||
}
|
||||
|
||||
private info(row: InstalledPlugin): PluginInfo {
|
||||
const managed = this.managed.get(row.plugin_id);
|
||||
return {
|
||||
pluginId: row.plugin_id, folder: row.folder, builtin: false, enabled: row.enabled === 1,
|
||||
name: managed?.plugin?.manifest.name, version: managed?.plugin?.manifest.version,
|
||||
sourceTypes: managed?.providers.map((provider) => provider.descriptor) ?? [],
|
||||
runtime: managed?.plugin?.status() ?? null, error: managed?.error,
|
||||
};
|
||||
}
|
||||
|
||||
private errorInfo(error: unknown): { code: string; message: string } {
|
||||
return error instanceof AppError
|
||||
? { code: error.code, message: error.message }
|
||||
: { code: 'PLUGIN_LOAD_FAILED', message: error instanceof Error ? error.message : 'Plugin failed to load' };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,286 @@
|
||||
import { spawn } from 'node:child_process';
|
||||
import type { ChildProcessWithoutNullStreams } from 'node:child_process';
|
||||
import { readFile, realpath, stat } from 'node:fs/promises';
|
||||
import { isAbsolute, relative, resolve, sep } from 'node:path';
|
||||
import { AppError } from '../lib/errors.ts';
|
||||
import { validatePluginManifest } from './registry.ts';
|
||||
import { decodeCatalogEntry, decodeCatalogPage, decodeChapterAcquisition, decodeManifest } from './protocol.ts';
|
||||
import type {
|
||||
PluginManifest,
|
||||
PluginRuntimeState,
|
||||
PluginRuntimeStatus,
|
||||
ResourceResponse,
|
||||
SourceContext,
|
||||
SourceProvider,
|
||||
} from './types.ts';
|
||||
|
||||
export class PluginError extends AppError {
|
||||
constructor(code: string, message: string) {
|
||||
super(code === 'PLUGIN_TIMEOUT' ? 504 : code === 'PLUGIN_UNAVAILABLE' ? 503 : 502, code, message);
|
||||
this.name = 'PluginError';
|
||||
}
|
||||
}
|
||||
|
||||
export interface ProcessPluginOptions {
|
||||
readonly timeoutMs?: number;
|
||||
/** Applies to each JSON line and limits the first version's inline resource payloads. */
|
||||
readonly maxMessageBytes?: number;
|
||||
}
|
||||
|
||||
interface PendingRequest {
|
||||
resolve(value: unknown): void;
|
||||
reject(reason: unknown): void;
|
||||
cleanup(): void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Trusted Node plugins speak newline-delimited JSON-RPC over stdio. This is
|
||||
* fault isolation, not a sandbox: installed code has this OS user's privileges.
|
||||
*/
|
||||
export class ProcessPlugin {
|
||||
private process: ChildProcessWithoutNullStreams | undefined;
|
||||
private state: PluginRuntimeState = 'stopped';
|
||||
private sequence = 0;
|
||||
private buffer = Buffer.alloc(0);
|
||||
private readonly pending = new Map<number, PendingRequest>();
|
||||
private readonly exits = new Set<Promise<void>>();
|
||||
private readonly timeoutMs: number;
|
||||
private readonly maxMessageBytes: number;
|
||||
|
||||
private constructor(
|
||||
readonly manifest: PluginManifest,
|
||||
private readonly directory: string,
|
||||
private readonly entry: string,
|
||||
options: ProcessPluginOptions,
|
||||
) {
|
||||
this.timeoutMs = options.timeoutMs ?? 30_000;
|
||||
this.maxMessageBytes = options.maxMessageBytes ?? 2 * 1024 * 1024;
|
||||
if (!Number.isSafeInteger(this.timeoutMs) || this.timeoutMs < 1) {
|
||||
throw new Error('plugin timeoutMs must be a positive integer');
|
||||
}
|
||||
if (!Number.isSafeInteger(this.maxMessageBytes) || this.maxMessageBytes < 256) {
|
||||
throw new Error('plugin maxMessageBytes must be at least 256');
|
||||
}
|
||||
}
|
||||
|
||||
static async load(directory: string, options: ProcessPluginOptions = {}): Promise<ProcessPlugin> {
|
||||
const root = await realpath(directory);
|
||||
const manifest = validatePluginManifest(JSON.parse(await readFile(resolve(root, 'plugin.json'), 'utf8')));
|
||||
if (manifest.apiVersion !== 1) {
|
||||
throw new PluginError('PLUGIN_INCOMPATIBLE', `unsupported plugin API version: ${manifest.apiVersion}`);
|
||||
}
|
||||
if (isAbsolute(manifest.entry)) {
|
||||
throw new PluginError('PLUGIN_INVALID', 'plugin entry must be relative to its directory');
|
||||
}
|
||||
const entry = await realpath(resolve(root, manifest.entry));
|
||||
const rel = relative(root, entry);
|
||||
if (!rel || rel === '..' || rel.startsWith(`..${sep}`) || isAbsolute(rel)) {
|
||||
throw new PluginError('PLUGIN_INVALID', 'plugin entry must be inside its directory');
|
||||
}
|
||||
if (!(await stat(entry)).isFile()) {
|
||||
throw new PluginError('PLUGIN_INVALID', 'plugin entry must be a file');
|
||||
}
|
||||
return new ProcessPlugin(manifest, root, entry, options);
|
||||
}
|
||||
|
||||
status(): PluginRuntimeStatus {
|
||||
return { pluginId: this.manifest.id, state: this.state, pendingRequests: this.pending.size };
|
||||
}
|
||||
|
||||
providers(): readonly SourceProvider[] {
|
||||
return this.manifest.sourceTypes.map((type) => {
|
||||
const context = (ctx: SourceContext) => ({ instance: ctx.instance, userId: ctx.userId });
|
||||
const call = <T>(method: string, ctx: SourceContext, args: Record<string, unknown>): Promise<T> =>
|
||||
this.request(method, { sourceType: type.id, context: context(ctx), ...args }, ctx.signal) as Promise<T>;
|
||||
const provider: SourceProvider = {
|
||||
descriptor: { ...type, version: this.manifest.version },
|
||||
validateConfig: async (config) => {
|
||||
await this.request('validateConfig', { sourceType: type.id, config });
|
||||
},
|
||||
detail: async (ctx, entryRef) => decodeCatalogEntry(await call('detail', ctx, { entryRef })),
|
||||
acquire: async (ctx, request) => decodeChapterAcquisition(await call('acquire', ctx, { request })),
|
||||
};
|
||||
if (type.capabilities.includes('browse')) {
|
||||
provider.browse = async (ctx, request) => decodeCatalogPage(await call('browse', ctx, { request }));
|
||||
}
|
||||
if (type.capabilities.includes('search')) {
|
||||
provider.search = async (ctx, request) => decodeCatalogPage(await call('search', ctx, { request }));
|
||||
}
|
||||
if (type.capabilities.includes('content.manifest')) {
|
||||
provider.getManifest = async (ctx, publicationRef) => decodeManifest(await call('getManifest', ctx, { publicationRef }), publicationRef);
|
||||
}
|
||||
if (type.capabilities.includes('content.resource')) {
|
||||
provider.readResource = async (ctx, request) => decodeResource(await call('readResource', ctx, { request }));
|
||||
}
|
||||
// File transfer through this bounded JSON channel is intentionally unsupported.
|
||||
if (type.capabilities.includes('acquire.file')) {
|
||||
throw new PluginError('PLUGIN_INCOMPATIBLE', 'process plugins currently support chapter resources only');
|
||||
}
|
||||
return provider;
|
||||
});
|
||||
}
|
||||
|
||||
async close(): Promise<void> {
|
||||
const child = this.process;
|
||||
this.process = undefined;
|
||||
this.state = 'stopped';
|
||||
this.buffer = Buffer.alloc(0);
|
||||
this.rejectAll(new PluginError('PLUGIN_UNAVAILABLE', 'plugin stopped'));
|
||||
child?.kill('SIGKILL');
|
||||
await Promise.all([...this.exits]);
|
||||
}
|
||||
|
||||
private start(): ChildProcessWithoutNullStreams {
|
||||
if (this.process) return this.process;
|
||||
if (this.state === 'failed') throw new PluginError('PLUGIN_UNAVAILABLE', 'plugin failed; reload it to retry');
|
||||
const child = spawn(process.execPath, [this.entry], {
|
||||
cwd: this.directory,
|
||||
shell: false,
|
||||
windowsHide: true,
|
||||
stdio: ['pipe', 'pipe', 'pipe'],
|
||||
// Do not automatically copy server API tokens and other configuration.
|
||||
env: pluginEnvironment(),
|
||||
});
|
||||
this.process = child;
|
||||
this.state = 'running';
|
||||
const exit = new Promise<void>((done) => child.once('close', () => done()));
|
||||
this.exits.add(exit);
|
||||
void exit.then(() => this.exits.delete(exit));
|
||||
child.stdout.on('data', (chunk: Buffer) => {
|
||||
if (this.process === child) this.onData(chunk);
|
||||
});
|
||||
child.stderr.resume();
|
||||
child.stdin.on('error', () => {
|
||||
if (this.process === child) this.fail(new PluginError('PLUGIN_UNAVAILABLE', 'plugin input pipe failed'));
|
||||
});
|
||||
child.on('error', () => {
|
||||
if (this.process === child) this.fail(new PluginError('PLUGIN_UNAVAILABLE', 'plugin could not start'));
|
||||
});
|
||||
child.on('close', () => {
|
||||
if (this.process === child) this.fail(new PluginError('PLUGIN_UNAVAILABLE', 'plugin process exited'));
|
||||
});
|
||||
return child;
|
||||
}
|
||||
|
||||
private request(method: string, params: Record<string, unknown>, signal?: AbortSignal): Promise<unknown> {
|
||||
if (signal?.aborted) return Promise.reject(new PluginError('PLUGIN_CANCELLED', 'plugin request cancelled'));
|
||||
let child: ChildProcessWithoutNullStreams;
|
||||
try {
|
||||
child = this.start();
|
||||
} catch (error) {
|
||||
return Promise.reject(error);
|
||||
}
|
||||
const id = ++this.sequence;
|
||||
const line = JSON.stringify({ jsonrpc: '2.0', id, method, params }) + '\n';
|
||||
if (Buffer.byteLength(line) > this.maxMessageBytes) {
|
||||
return Promise.reject(new PluginError('PLUGIN_MESSAGE_TOO_LARGE', 'plugin request exceeds the message limit'));
|
||||
}
|
||||
return new Promise((resolveRequest, reject) => {
|
||||
const cancel = () => {
|
||||
if (!this.pending.has(id)) return;
|
||||
// Killing the worker is the only hard cancellation available for trusted
|
||||
// stdio plugins. Other calls in the same process fail with it.
|
||||
this.fail(new PluginError('PLUGIN_CANCELLED', 'plugin process stopped because a request was cancelled'));
|
||||
};
|
||||
const timer = setTimeout(() => {
|
||||
this.fail(new PluginError('PLUGIN_TIMEOUT', 'plugin request exceeded its execution deadline'));
|
||||
}, this.timeoutMs);
|
||||
timer.unref();
|
||||
const cleanup = () => {
|
||||
clearTimeout(timer);
|
||||
signal?.removeEventListener('abort', cancel);
|
||||
};
|
||||
this.pending.set(id, { resolve: resolveRequest, reject, cleanup });
|
||||
signal?.addEventListener('abort', cancel, { once: true });
|
||||
child.stdin.write(line);
|
||||
});
|
||||
}
|
||||
|
||||
private onData(chunk: Buffer): void {
|
||||
this.buffer = Buffer.concat([this.buffer, chunk]);
|
||||
while (true) {
|
||||
const end = this.buffer.indexOf(10);
|
||||
if (end < 0) {
|
||||
if (this.buffer.length > this.maxMessageBytes) {
|
||||
this.fail(new PluginError('PLUGIN_MESSAGE_TOO_LARGE', 'plugin response exceeds the message limit'));
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (end > this.maxMessageBytes) {
|
||||
this.fail(new PluginError('PLUGIN_MESSAGE_TOO_LARGE', 'plugin response exceeds the message limit'));
|
||||
return;
|
||||
}
|
||||
const line = this.buffer.subarray(0, end).toString('utf8');
|
||||
this.buffer = this.buffer.subarray(end + 1);
|
||||
try {
|
||||
this.onMessage(JSON.parse(line));
|
||||
} catch {
|
||||
this.fail(new PluginError('PLUGIN_PROTOCOL_ERROR', 'plugin returned an invalid JSON-RPC message'));
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private onMessage(input: unknown): void {
|
||||
if (!input || typeof input !== 'object') throw new Error('invalid response');
|
||||
const value = input as Record<string, unknown>;
|
||||
if (value.jsonrpc !== '2.0' || !Number.isSafeInteger(value.id)) throw new Error('invalid response');
|
||||
if (('result' in value) === ('error' in value)) throw new Error('invalid response');
|
||||
const pending = this.pending.get(value.id as number);
|
||||
if (!pending) return; // A cooperative cancellation can race an already written response.
|
||||
this.pending.delete(value.id as number);
|
||||
pending.cleanup();
|
||||
if ('error' in value) {
|
||||
const error = value.error as { code?: unknown; message?: unknown } | null;
|
||||
const code = typeof error?.code === 'string' ? error.code : 'PLUGIN_ERROR';
|
||||
pending.reject(new PluginError(code, typeof error?.message === 'string' ? error.message : 'plugin request failed'));
|
||||
} else {
|
||||
pending.resolve(value.result);
|
||||
}
|
||||
}
|
||||
|
||||
private rejectAll(error: Error): void {
|
||||
for (const pending of this.pending.values()) {
|
||||
pending.cleanup();
|
||||
pending.reject(error);
|
||||
}
|
||||
this.pending.clear();
|
||||
}
|
||||
|
||||
private fail(error: PluginError): void {
|
||||
const child = this.process;
|
||||
this.process = undefined;
|
||||
this.state = 'failed';
|
||||
this.buffer = Buffer.alloc(0);
|
||||
this.rejectAll(error);
|
||||
child?.kill('SIGKILL');
|
||||
}
|
||||
}
|
||||
|
||||
function pluginEnvironment(): NodeJS.ProcessEnv {
|
||||
const result: NodeJS.ProcessEnv = {};
|
||||
for (const key of ['PATH', 'Path', 'SystemRoot', 'SYSTEMROOT', 'WINDIR', 'TEMP', 'TMP', 'TMPDIR', 'LANG']) {
|
||||
if (process.env[key] !== undefined) result[key] = process.env[key];
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function decodeResource(input: unknown): ResourceResponse {
|
||||
if (!input || typeof input !== 'object') throw new PluginError('PLUGIN_PROTOCOL_ERROR', 'invalid plugin resource');
|
||||
const value = input as Record<string, unknown>;
|
||||
if (typeof value.mediaType !== 'string' || !value.mediaType.trim()) {
|
||||
throw new PluginError('PLUGIN_PROTOCOL_ERROR', 'plugin resource has no media type');
|
||||
}
|
||||
const hasText = typeof value.text === 'string';
|
||||
const hasBase64 = typeof value.base64 === 'string';
|
||||
if (hasText === hasBase64) {
|
||||
throw new PluginError('PLUGIN_PROTOCOL_ERROR', 'plugin resource must contain exactly one of text or base64');
|
||||
}
|
||||
if (hasText) return { mediaType: value.mediaType, text: value.text as string };
|
||||
const encoded = value.base64 as string;
|
||||
if (!/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(encoded)) {
|
||||
throw new PluginError('PLUGIN_PROTOCOL_ERROR', 'plugin resource contains invalid base64');
|
||||
}
|
||||
const data = Buffer.from(encoded, 'base64');
|
||||
return { mediaType: value.mediaType, data, size: data.length };
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
import { AppError } from '../lib/errors.ts';
|
||||
import type { Acquisition, CatalogEntry, CatalogPage, ManifestSnapshot } from './types.ts';
|
||||
|
||||
function invalid(message: string): never {
|
||||
throw new AppError(502, 'PLUGIN_PROTOCOL_ERROR', message);
|
||||
}
|
||||
|
||||
function object(input: unknown, label: string): Record<string, unknown> {
|
||||
if (!input || typeof input !== 'object' || Array.isArray(input)) invalid(`invalid plugin ${label}`);
|
||||
return input as Record<string, unknown>;
|
||||
}
|
||||
|
||||
function string(value: unknown, label: string): asserts value is string {
|
||||
if (typeof value !== 'string' || !value.trim()) invalid(`plugin ${label} must be a nonempty string`);
|
||||
}
|
||||
|
||||
/** Validate wire data before it can become host-owned catalog or reading state. */
|
||||
export function decodeCatalogEntry(input: unknown): CatalogEntry {
|
||||
const value = object(input, 'catalog entry');
|
||||
string(value.ref, 'entry ref');
|
||||
string(value.title, 'entry title');
|
||||
for (const key of ['description', 'coverUrl', 'language', 'publishedAt']) {
|
||||
if (value[key] !== undefined && typeof value[key] !== 'string') invalid(`invalid plugin entry ${key}`);
|
||||
}
|
||||
if (value.authors !== undefined &&
|
||||
(!Array.isArray(value.authors) || value.authors.some((author) => typeof author !== 'string'))) {
|
||||
invalid('plugin authors must be strings');
|
||||
}
|
||||
if (value.options !== undefined) {
|
||||
if (!Array.isArray(value.options)) invalid('plugin acquisition options must be an array');
|
||||
const ids = new Set<string>();
|
||||
for (const option of value.options) {
|
||||
const acquisition = object(option, 'acquisition option');
|
||||
string(acquisition.id, 'acquisition option id');
|
||||
string(acquisition.label, 'acquisition option label');
|
||||
if (ids.has(acquisition.id)) invalid('duplicate plugin acquisition option id');
|
||||
ids.add(acquisition.id);
|
||||
}
|
||||
}
|
||||
return input as CatalogEntry;
|
||||
}
|
||||
|
||||
export function decodeCatalogPage(input: unknown): CatalogPage {
|
||||
const value = object(input, 'catalog page');
|
||||
if (!Array.isArray(value.items)) invalid('plugin catalog items must be an array');
|
||||
value.items.forEach(decodeCatalogEntry);
|
||||
for (const key of ['title', 'nextCursor']) {
|
||||
if (value[key] !== undefined && typeof value[key] !== 'string') invalid(`invalid plugin catalog ${key}`);
|
||||
}
|
||||
if (value.navigation !== undefined) {
|
||||
if (!Array.isArray(value.navigation)) invalid('plugin navigation must be an array');
|
||||
for (const entry of value.navigation) {
|
||||
const navigation = object(entry, 'navigation');
|
||||
string(navigation.ref, 'navigation ref');
|
||||
string(navigation.title, 'navigation title');
|
||||
}
|
||||
}
|
||||
return input as CatalogPage;
|
||||
}
|
||||
|
||||
export function decodeChapterAcquisition(input: unknown): Acquisition {
|
||||
const value = object(input, 'acquisition');
|
||||
if (value.kind === 'chapters') {
|
||||
string(value.publicationRef, 'publication ref');
|
||||
} else if (value.kind === 'action-required') {
|
||||
const action = object(value.action, 'acquisition action');
|
||||
if (!['login', 'borrow', 'external'].includes(action.type as string)) invalid('invalid plugin acquisition action type');
|
||||
string(action.label, 'action label');
|
||||
if (action.url !== undefined) {
|
||||
string(action.url, 'action URL');
|
||||
let url: URL;
|
||||
try { url = new URL(action.url); } catch { invalid('invalid plugin action URL'); }
|
||||
if (!['http:', 'https:'].includes(url.protocol)) invalid('plugin action URL must use HTTP or HTTPS');
|
||||
}
|
||||
} else {
|
||||
invalid('process plugins must acquire chapter content or request an action');
|
||||
}
|
||||
return input as Acquisition;
|
||||
}
|
||||
|
||||
export function decodeManifest(input: unknown, publicationRef: string): ManifestSnapshot {
|
||||
const value = object(input, 'manifest');
|
||||
if (value.publicationRef !== publicationRef) invalid('plugin manifest publication ref does not match the request');
|
||||
if (!Array.isArray(value.items)) invalid('plugin manifest items must be an array');
|
||||
const ids = new Set<string>();
|
||||
const sequences = new Set<number>();
|
||||
for (const entry of value.items) {
|
||||
const item = object(entry, 'manifest item');
|
||||
string(item.id, 'chapter id');
|
||||
string(item.ref, 'chapter ref');
|
||||
string(item.title, 'chapter title');
|
||||
string(item.mediaType, 'chapter media type');
|
||||
if (item.kind !== 'chapter' && item.kind !== 'page') invalid('invalid plugin chapter kind');
|
||||
if (typeof item.seq !== 'number' || !Number.isSafeInteger(item.seq) || item.seq < 0) invalid('invalid plugin chapter order');
|
||||
if (ids.has(item.id) || sequences.has(item.seq)) invalid('plugin manifest contains duplicate chapter ids or order');
|
||||
ids.add(item.id);
|
||||
sequences.add(item.seq);
|
||||
}
|
||||
return input as ManifestSnapshot;
|
||||
}
|
||||
@@ -0,0 +1,189 @@
|
||||
import type {
|
||||
PluginManifest,
|
||||
SourceCapability,
|
||||
SourceDescriptor,
|
||||
SourceProvider,
|
||||
SourceRegistration,
|
||||
} from './types.ts';
|
||||
|
||||
export class SourceRegistryError extends Error {
|
||||
constructor(message: string) {
|
||||
super(message);
|
||||
this.name = 'SourceRegistryError';
|
||||
}
|
||||
}
|
||||
|
||||
const ID_RE = /^[a-z][a-z0-9._-]{0,127}$/;
|
||||
const CAPABILITIES = new Set<SourceCapability>([
|
||||
'browse', 'search', 'detail', 'acquire.file', 'acquire.chapters',
|
||||
'content.manifest', 'content.resource', 'content.update',
|
||||
]);
|
||||
|
||||
function assertDescriptor(descriptor: SourceDescriptor): void {
|
||||
if (!ID_RE.test(descriptor.id)) throw new SourceRegistryError(`invalid source id: ${descriptor.id}`);
|
||||
if (!descriptor.label.trim()) throw new SourceRegistryError(`source ${descriptor.id} has an empty label`);
|
||||
if (!descriptor.version.trim()) throw new SourceRegistryError(`source ${descriptor.id} has no version`);
|
||||
const capabilities = new Set(descriptor.capabilities);
|
||||
for (const capability of capabilities) {
|
||||
if (!CAPABILITIES.has(capability)) throw new SourceRegistryError(`unsupported capability: ${capability}`);
|
||||
}
|
||||
if (capabilities.size !== descriptor.capabilities.length) {
|
||||
throw new SourceRegistryError(`source ${descriptor.id} declares duplicate capabilities`);
|
||||
}
|
||||
}
|
||||
|
||||
function assertProvider(provider: SourceProvider): void {
|
||||
assertDescriptor(provider.descriptor);
|
||||
if (typeof provider.detail !== 'function') {
|
||||
throw new SourceRegistryError(`source ${provider.descriptor.id} must implement detail()`);
|
||||
}
|
||||
if (typeof provider.acquire !== 'function') {
|
||||
throw new SourceRegistryError(`source ${provider.descriptor.id} must implement acquire()`);
|
||||
}
|
||||
const caps = new Set(provider.descriptor.capabilities);
|
||||
const requireMethod = (capability: SourceCapability, method: keyof SourceProvider): void => {
|
||||
if (caps.has(capability) && typeof provider[method] !== 'function') {
|
||||
throw new SourceRegistryError(
|
||||
`source ${provider.descriptor.id} declares ${capability} but does not implement ${String(method)}()`,
|
||||
);
|
||||
}
|
||||
};
|
||||
requireMethod('browse', 'browse');
|
||||
requireMethod('search', 'search');
|
||||
requireMethod('content.manifest', 'getManifest');
|
||||
requireMethod('content.resource', 'readResource');
|
||||
requireMethod('content.update', 'getManifest');
|
||||
if (caps.has('acquire.chapters')) {
|
||||
if (typeof provider.getManifest !== 'function' || typeof provider.readResource !== 'function') {
|
||||
throw new SourceRegistryError(`source ${provider.descriptor.id} must implement chapter manifest and resources`);
|
||||
}
|
||||
}
|
||||
if (caps.has('acquire.file') && typeof provider.openFile !== 'function') {
|
||||
throw new SourceRegistryError(
|
||||
`source ${provider.descriptor.id} declares acquire.file but does not implement openFile()`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** Registry for built-in and installed source providers. */
|
||||
export class SourceRegistry {
|
||||
private readonly registrations = new Map<string, SourceRegistration>();
|
||||
|
||||
register(registration: SourceRegistration): SourceRegistration {
|
||||
assertProvider(registration.provider);
|
||||
if (!ID_RE.test(registration.pluginId)) {
|
||||
throw new SourceRegistryError(`invalid plugin id: ${registration.pluginId}`);
|
||||
}
|
||||
const key = this.key(registration.pluginId, registration.provider.descriptor.id);
|
||||
if (this.registrations.has(key)) {
|
||||
throw new SourceRegistryError(`source already registered: ${key}`);
|
||||
}
|
||||
this.registrations.set(key, registration);
|
||||
return registration;
|
||||
}
|
||||
|
||||
registerBuiltin(pluginId: string, provider: SourceProvider): SourceRegistration {
|
||||
return this.register({ pluginId, provider, builtin: true });
|
||||
}
|
||||
|
||||
unregister(pluginId: string, sourceType: string): boolean {
|
||||
return this.registrations.delete(this.key(pluginId, sourceType));
|
||||
}
|
||||
|
||||
get(pluginId: string, sourceType: string): SourceRegistration | undefined {
|
||||
return this.registrations.get(this.key(pluginId, sourceType));
|
||||
}
|
||||
|
||||
require(pluginId: string, sourceType: string): SourceRegistration {
|
||||
const registration = this.get(pluginId, sourceType);
|
||||
if (!registration) throw new SourceRegistryError(`source not found: ${pluginId}/${sourceType}`);
|
||||
return registration;
|
||||
}
|
||||
|
||||
list(): readonly SourceRegistration[] {
|
||||
return [...this.registrations.values()];
|
||||
}
|
||||
|
||||
listDescriptors(): readonly SourceDescriptor[] {
|
||||
return this.list().map(({ provider }) => provider.descriptor);
|
||||
}
|
||||
|
||||
registerPluginManifest(manifest: PluginManifest, provider: SourceProvider): SourceRegistration {
|
||||
validatePluginManifest(manifest);
|
||||
const sourceType = manifest.sourceTypes.find((type) => type.id === provider.descriptor.id);
|
||||
if (!sourceType) {
|
||||
throw new SourceRegistryError(
|
||||
`plugin ${manifest.id} does not declare source type ${provider.descriptor.id}`,
|
||||
);
|
||||
}
|
||||
if (sourceType.capabilities.length !== provider.descriptor.capabilities.length ||
|
||||
sourceType.capabilities.some((capability) => !provider.descriptor.capabilities.includes(capability))) {
|
||||
throw new SourceRegistryError(`provider capabilities do not match plugin manifest: ${provider.descriptor.id}`);
|
||||
}
|
||||
return this.register({ pluginId: manifest.id, provider });
|
||||
}
|
||||
|
||||
private key(pluginId: string, sourceType: string): string {
|
||||
return `${pluginId}/${sourceType}`;
|
||||
}
|
||||
}
|
||||
|
||||
export function validatePluginManifest(input: unknown): PluginManifest {
|
||||
if (!input || typeof input !== 'object') throw new SourceRegistryError('plugin manifest must be an object');
|
||||
const value = input as Record<string, unknown>;
|
||||
const requiredString = (name: string): string => {
|
||||
const v = value[name];
|
||||
if (typeof v !== 'string' || !v.trim()) throw new SourceRegistryError(`plugin manifest ${name} is required`);
|
||||
return v;
|
||||
};
|
||||
const id = requiredString('id');
|
||||
if (!ID_RE.test(id)) throw new SourceRegistryError(`invalid plugin id: ${id}`);
|
||||
const manifest: PluginManifest = {
|
||||
id,
|
||||
name: requiredString('name'),
|
||||
version: requiredString('version'),
|
||||
apiVersion: value.apiVersion as number,
|
||||
runtime: value.runtime as 'node',
|
||||
entry: requiredString('entry'),
|
||||
sourceTypes: Array.isArray(value.sourceTypes) ? (value.sourceTypes as PluginManifest['sourceTypes']) : [],
|
||||
permissions: value.permissions as PluginManifest['permissions'],
|
||||
};
|
||||
if (manifest.apiVersion !== 1) {
|
||||
throw new SourceRegistryError('unsupported plugin API version; expected apiVersion: 1');
|
||||
}
|
||||
if (manifest.runtime !== 'node') throw new SourceRegistryError(`unsupported plugin runtime: ${manifest.runtime}`);
|
||||
if (manifest.sourceTypes.length === 0) throw new SourceRegistryError('plugin manifest must declare sourceTypes');
|
||||
const ids = new Set<string>();
|
||||
for (const type of manifest.sourceTypes) {
|
||||
if (!type || typeof type.id !== 'string' || !ID_RE.test(type.id)) {
|
||||
throw new SourceRegistryError('plugin source type has an invalid id');
|
||||
}
|
||||
if (ids.has(type.id)) throw new SourceRegistryError(`duplicate plugin source type: ${type.id}`);
|
||||
ids.add(type.id);
|
||||
if (typeof type.label !== 'string' || !type.label.trim() || !Array.isArray(type.capabilities)) {
|
||||
throw new SourceRegistryError(`invalid plugin source type: ${type.id}`);
|
||||
}
|
||||
assertDescriptor({ ...type, version: manifest.version });
|
||||
if (type.capabilities.includes('acquire.chapters') &&
|
||||
(!type.capabilities.includes('content.manifest') || !type.capabilities.includes('content.resource'))) {
|
||||
throw new SourceRegistryError(`chapter source ${type.id} must declare manifest and resource capabilities`);
|
||||
}
|
||||
}
|
||||
return manifest;
|
||||
}
|
||||
|
||||
/** Descriptors reserved for the two providers shipped by the host. */
|
||||
export const BUILTIN_SOURCE_TYPES = {
|
||||
local: {
|
||||
id: 'local',
|
||||
label: 'Local library',
|
||||
version: '1',
|
||||
capabilities: ['browse', 'search', 'detail'],
|
||||
} as const satisfies SourceDescriptor,
|
||||
opds: {
|
||||
id: 'opds',
|
||||
label: 'OPDS catalog',
|
||||
version: '1',
|
||||
capabilities: ['browse', 'search', 'detail', 'acquire.file'],
|
||||
} as const satisfies SourceDescriptor,
|
||||
};
|
||||
@@ -0,0 +1,239 @@
|
||||
import type { Readable } from 'node:stream';
|
||||
|
||||
/** Stable source capabilities exposed to the host and clients. */
|
||||
export type SourceCapability =
|
||||
| 'browse'
|
||||
| 'search'
|
||||
| 'detail'
|
||||
| 'acquire.file'
|
||||
| 'acquire.chapters'
|
||||
| 'content.manifest'
|
||||
| 'content.resource'
|
||||
| 'content.update';
|
||||
|
||||
export interface SourceDescriptor {
|
||||
/** Stable source type id, e.g. `local`, `opds`, or a plugin-defined id. */
|
||||
readonly id: string;
|
||||
readonly label: string;
|
||||
readonly version: string;
|
||||
readonly capabilities: readonly SourceCapability[];
|
||||
/** JSON Schema for an instance's public configuration, when applicable. */
|
||||
readonly configSchema?: unknown;
|
||||
}
|
||||
|
||||
export interface SourceInstance {
|
||||
readonly id: string;
|
||||
readonly pluginId: string;
|
||||
readonly sourceType: string;
|
||||
readonly name: string;
|
||||
/** Configuration is owned by the host and never persisted by a provider. */
|
||||
readonly config: unknown;
|
||||
readonly enabled: boolean;
|
||||
}
|
||||
|
||||
export interface SourceContext {
|
||||
readonly instance: SourceInstance;
|
||||
/** User id scopes cache and credential access. */
|
||||
readonly userId: string;
|
||||
readonly signal: AbortSignal;
|
||||
readonly http?: SourceHttpClient;
|
||||
readonly credentials?: CredentialStore;
|
||||
readonly storage?: SourceStorage;
|
||||
readonly log?: SourceLogger;
|
||||
}
|
||||
|
||||
export interface SourceHttpRequest {
|
||||
readonly url: string;
|
||||
readonly method?: string;
|
||||
readonly headers?: Record<string, string>;
|
||||
readonly body?: string | Uint8Array;
|
||||
}
|
||||
|
||||
export interface SourceHttpResponse {
|
||||
readonly status: number;
|
||||
readonly headers: Record<string, string>;
|
||||
readonly body: Uint8Array;
|
||||
readonly url: string;
|
||||
}
|
||||
|
||||
export interface SourceHttpClient {
|
||||
request(request: SourceHttpRequest, signal: AbortSignal): Promise<SourceHttpResponse>;
|
||||
}
|
||||
|
||||
export interface CredentialStore {
|
||||
get(key: string, signal?: AbortSignal): Promise<string | undefined>;
|
||||
set(key: string, value: string, signal?: AbortSignal): Promise<void>;
|
||||
delete(key: string, signal?: AbortSignal): Promise<void>;
|
||||
}
|
||||
|
||||
export interface SourceStorage {
|
||||
get(key: string): Promise<unknown | undefined>;
|
||||
set(key: string, value: unknown): Promise<void>;
|
||||
delete(key: string): Promise<void>;
|
||||
}
|
||||
|
||||
export interface SourceLogger {
|
||||
debug(message: string, fields?: Record<string, unknown>): void;
|
||||
info(message: string, fields?: Record<string, unknown>): void;
|
||||
warn(message: string, fields?: Record<string, unknown>): void;
|
||||
error(message: string, fields?: Record<string, unknown>): void;
|
||||
}
|
||||
|
||||
export interface BrowseRequest {
|
||||
readonly ref?: string;
|
||||
readonly cursor?: string;
|
||||
readonly limit?: number;
|
||||
}
|
||||
|
||||
export interface SearchRequest {
|
||||
readonly query: string;
|
||||
readonly cursor?: string;
|
||||
readonly limit?: number;
|
||||
}
|
||||
|
||||
export interface CatalogPage {
|
||||
readonly items: readonly CatalogEntry[];
|
||||
readonly navigation?: readonly NavigationEntry[];
|
||||
readonly nextCursor?: string;
|
||||
readonly title?: string;
|
||||
}
|
||||
|
||||
export interface NavigationEntry {
|
||||
readonly ref: string;
|
||||
readonly title: string;
|
||||
readonly kind?: 'catalog' | 'search' | 'collection';
|
||||
}
|
||||
|
||||
export interface CatalogEntry {
|
||||
readonly ref: string;
|
||||
readonly title: string;
|
||||
readonly authors?: readonly string[];
|
||||
readonly description?: string;
|
||||
readonly coverUrl?: string;
|
||||
readonly language?: string;
|
||||
readonly publishedAt?: string;
|
||||
readonly options?: readonly AcquisitionOption[];
|
||||
readonly metadata?: Readonly<Record<string, unknown>>;
|
||||
}
|
||||
|
||||
export interface AcquisitionOption {
|
||||
readonly id: string;
|
||||
readonly label: string;
|
||||
readonly mediaType?: string;
|
||||
readonly size?: number;
|
||||
readonly available?: boolean;
|
||||
}
|
||||
|
||||
export interface AcquireRequest {
|
||||
readonly entryRef: string;
|
||||
readonly optionId?: string;
|
||||
}
|
||||
|
||||
export type Acquisition =
|
||||
| {
|
||||
readonly kind: 'ready';
|
||||
readonly publicationId: string;
|
||||
}
|
||||
| {
|
||||
readonly kind: 'file';
|
||||
/** Provider-owned reference; host must call openFile rather than fetch arbitrary URLs. */
|
||||
readonly acquisitionRef: string;
|
||||
readonly mediaType: string;
|
||||
readonly filename?: string;
|
||||
readonly size?: number;
|
||||
}
|
||||
| {
|
||||
readonly kind: 'chapters';
|
||||
readonly publicationRef: string;
|
||||
}
|
||||
| {
|
||||
readonly kind: 'action-required';
|
||||
readonly action: AcquisitionAction;
|
||||
};
|
||||
|
||||
export interface AcquisitionAction {
|
||||
readonly type: 'login' | 'borrow' | 'external';
|
||||
readonly label: string;
|
||||
readonly url?: string;
|
||||
readonly fields?: readonly { name: string; label: string; secret?: boolean }[];
|
||||
}
|
||||
|
||||
export interface ManifestSnapshot {
|
||||
readonly publicationRef: string;
|
||||
readonly version?: string;
|
||||
readonly items: readonly ManifestItem[];
|
||||
}
|
||||
|
||||
export interface ManifestItem {
|
||||
readonly id: string;
|
||||
readonly seq: number;
|
||||
readonly title: string;
|
||||
readonly kind: 'chapter' | 'page';
|
||||
readonly mediaType: string;
|
||||
readonly ref: string;
|
||||
}
|
||||
|
||||
export interface ResourceRequest {
|
||||
readonly publicationRef: string;
|
||||
readonly ref: string;
|
||||
readonly rendition?: 'text' | 'html' | 'binary';
|
||||
}
|
||||
|
||||
export interface ResourceResponse {
|
||||
readonly mediaType: string;
|
||||
readonly data?: Uint8Array;
|
||||
readonly text?: string;
|
||||
readonly stream?: Readable;
|
||||
readonly size?: number;
|
||||
readonly filename?: string;
|
||||
readonly etag?: string;
|
||||
}
|
||||
|
||||
export interface SourceProvider {
|
||||
readonly descriptor: SourceDescriptor;
|
||||
validateConfig?(config: unknown): void | Promise<void>;
|
||||
browse?(ctx: SourceContext, request: BrowseRequest): Promise<CatalogPage>;
|
||||
search?(ctx: SourceContext, request: SearchRequest): Promise<CatalogPage>;
|
||||
detail(ctx: SourceContext, entryRef: string): Promise<CatalogEntry>;
|
||||
acquire(ctx: SourceContext, request: AcquireRequest): Promise<Acquisition>;
|
||||
/** Resolve a file acquisition without exposing arbitrary network access to the host. */
|
||||
openFile?(ctx: SourceContext, acquisition: Extract<Acquisition, { kind: 'file' }>): Promise<ResourceResponse>;
|
||||
getManifest?(ctx: SourceContext, publicationRef: string): Promise<ManifestSnapshot>;
|
||||
readResource?(ctx: SourceContext, request: ResourceRequest): Promise<ResourceResponse>;
|
||||
}
|
||||
|
||||
export interface SourceRegistration {
|
||||
readonly pluginId: string;
|
||||
readonly provider: SourceProvider;
|
||||
readonly builtin?: boolean;
|
||||
}
|
||||
|
||||
export interface PluginSourceType {
|
||||
readonly id: string;
|
||||
readonly label: string;
|
||||
readonly capabilities: readonly SourceCapability[];
|
||||
readonly configSchema?: unknown;
|
||||
}
|
||||
|
||||
export interface PluginManifest {
|
||||
readonly id: string;
|
||||
readonly name: string;
|
||||
readonly version: string;
|
||||
readonly apiVersion: number;
|
||||
readonly runtime: 'node';
|
||||
readonly entry: string;
|
||||
readonly sourceTypes: readonly PluginSourceType[];
|
||||
readonly permissions?: {
|
||||
readonly network?: { readonly domains?: readonly string[] };
|
||||
readonly storage?: boolean;
|
||||
readonly credentials?: boolean;
|
||||
};
|
||||
}
|
||||
|
||||
export type PluginRuntimeState = 'stopped' | 'running' | 'failed';
|
||||
|
||||
export interface PluginRuntimeStatus {
|
||||
readonly pluginId: string;
|
||||
readonly state: PluginRuntimeState;
|
||||
readonly pendingRequests: number;
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
import { test, describe } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtemp, readFile, readdir, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { createHash } from 'node:crypto';
|
||||
import { Readable } from 'node:stream';
|
||||
import JSZip from 'jszip';
|
||||
import { Db } from '../src/db/index.ts';
|
||||
import { FileAcquisitions } from '../src/sources/acquisitions.ts';
|
||||
import type { SourceContext, SourceInstance, SourceProvider } from '../src/sources/types.ts';
|
||||
|
||||
function fixture(root: string): { db: Db; config: { dataDir: string }; context: SourceContext } {
|
||||
const db = new Db(join(root, 'reader.db'));
|
||||
db.run(
|
||||
`INSERT INTO users (id, username, display_name, password_hash, role, disabled, created_at, updated_at)
|
||||
VALUES ('u1', 'u1', '', 'x', 'member', 0, 0, 0)`,
|
||||
);
|
||||
db.run(
|
||||
`INSERT INTO source_instances (id, plugin_id, source_type, name, config_json, enabled, created_at)
|
||||
VALUES ('s1', 'demo', 'demo', 'Demo', '{}', 1, 0)`,
|
||||
);
|
||||
const instance: SourceInstance = {
|
||||
id: 's1', pluginId: 'demo', sourceType: 'demo', name: 'Demo', config: {}, enabled: true,
|
||||
};
|
||||
return { db, config: { dataDir: root }, context: { instance, userId: 'u1', signal: new AbortController().signal } };
|
||||
}
|
||||
|
||||
function provider(body: Buffer | Readable, mediaType = 'text/plain'): SourceProvider {
|
||||
return {
|
||||
descriptor: { id: 'demo', label: 'Demo', version: '1', capabilities: ['detail', 'acquire.file'] },
|
||||
async detail() { return { ref: 'entry-1', title: 'Remote title', authors: ['Author'] }; },
|
||||
async acquire() { return { kind: 'file', acquisitionRef: 'file-1', mediaType, filename: 'remote.txt' }; },
|
||||
async openFile() { return { mediaType, data: Buffer.isBuffer(body) ? body : undefined, stream: Buffer.isBuffer(body) ? undefined : body }; },
|
||||
};
|
||||
}
|
||||
|
||||
describe('FileAcquisitions', () => {
|
||||
test('downloads, parses, persists, and shelves a file', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-acquisition-'));
|
||||
const { db, config, context } = fixture(root);
|
||||
try {
|
||||
const importer = new FileAcquisitions(db, config);
|
||||
const bytes = Buffer.from('第一章\nhello\n', 'utf8');
|
||||
const bookId = await importer.acquire(
|
||||
provider(bytes), context, { entryRef: 'entry-1', optionId: 'txt' },
|
||||
{ kind: 'file', acquisitionRef: 'file-1', mediaType: 'text/plain', filename: 'remote.txt' },
|
||||
{ ref: 'entry-1', title: 'Remote title', authors: ['Author'] },
|
||||
);
|
||||
const hash = createHash('sha256').update(bytes).digest('hex');
|
||||
const book = db.get<{ title: string; content_hash: string; source: string }>('SELECT title, content_hash, source FROM books WHERE id = ?', bookId);
|
||||
assert.deepEqual({ ...book }, { title: 'Remote title', content_hash: hash, source: 'provider:demo' });
|
||||
assert.equal(db.get('SELECT book_id FROM user_books WHERE user_id = ? AND book_id = ? AND hidden = 0', 'u1', bookId)?.book_id, bookId);
|
||||
const acquired = db.get<{ rel_path: string; size: number }>('SELECT rel_path, size FROM acquired_files WHERE book_id = ?', bookId);
|
||||
assert.equal(acquired?.size, bytes.byteLength);
|
||||
assert.deepEqual(await readFile(join(config.dataDir, acquired!.rel_path)), bytes);
|
||||
assert.equal(db.get('SELECT book_id FROM source_acquisitions WHERE source_id = ? AND user_id = ? AND entry_ref = ? AND option_id = ?', 's1', 'u1', 'entry-1', 'txt')?.book_id, bookId);
|
||||
} finally {
|
||||
db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('serial repeat reuses the committed file and does not create another copy', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-acquisition-'));
|
||||
const { db, config, context } = fixture(root);
|
||||
try {
|
||||
const importer = new FileAcquisitions(db, config);
|
||||
const body = Buffer.from('same bytes');
|
||||
const args = [provider(body), context, { entryRef: 'entry-1', optionId: '' },
|
||||
{ kind: 'file', acquisitionRef: 'file-1', mediaType: 'text/plain', filename: 'same.txt' },
|
||||
{ ref: 'entry-1', title: 'Same' }] as const;
|
||||
const first = await importer.acquire(...args);
|
||||
const second = await importer.acquire(...args);
|
||||
assert.equal(second, first);
|
||||
assert.equal((await readdir(join(root, 'acquired'))).length, 1);
|
||||
} finally {
|
||||
db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects an oversized response and cleans temporary files', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-acquisition-'));
|
||||
const { db, config, context } = fixture(root);
|
||||
try {
|
||||
const importer = new FileAcquisitions(db, config, { maxBytes: 4 });
|
||||
await assert.rejects(
|
||||
importer.acquire(
|
||||
provider(Buffer.from('12345')), context, { entryRef: 'large' },
|
||||
{ kind: 'file', acquisitionRef: 'large', mediaType: 'text/plain' }, { ref: 'large', title: 'Large' },
|
||||
),
|
||||
(error: unknown) => (error as { code?: string }).code === 'ACQUISITION_TOO_LARGE',
|
||||
);
|
||||
assert.equal((await readdir(join(root, 'acquired'))).length, 0);
|
||||
assert.equal(db.get('SELECT COUNT(*) AS n FROM books')?.n, 0);
|
||||
} finally {
|
||||
db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('cancellation after provider response leaves no acquired file', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-acquisition-'));
|
||||
const { db, config, context: original } = fixture(root);
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const context = { ...original, signal: controller.signal };
|
||||
const source: SourceProvider = {
|
||||
...provider(Buffer.from('cancel me')),
|
||||
async openFile() {
|
||||
controller.abort();
|
||||
return { mediaType: 'text/plain', stream: Readable.from([Buffer.from('cancel me')]) };
|
||||
},
|
||||
};
|
||||
const importer = new FileAcquisitions(db, config);
|
||||
await assert.rejects(importer.acquire(
|
||||
source, context, { entryRef: 'cancel' },
|
||||
{ kind: 'file', acquisitionRef: 'cancel', mediaType: 'text/plain' }, { ref: 'cancel', title: 'Cancel' },
|
||||
), { name: 'AbortError' });
|
||||
assert.equal((await readdir(join(root, 'acquired'))).length, 0);
|
||||
assert.equal(db.get('SELECT COUNT(*) AS n FROM books')?.n, 0);
|
||||
} finally {
|
||||
db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('concurrent requests for one acquisition download once and restore a hidden shelf entry', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-acquisition-'));
|
||||
const { db, config, context } = fixture(root);
|
||||
try {
|
||||
const importer = new FileAcquisitions(db, config);
|
||||
let calls = 0;
|
||||
const source = provider(Buffer.from('same bytes'));
|
||||
source.openFile = async () => {
|
||||
calls += 1;
|
||||
return { mediaType: 'text/plain', stream: Readable.from([Buffer.from('same bytes')]) };
|
||||
};
|
||||
const acquire = () => importer.acquire(source, context, { entryRef: 'same' },
|
||||
{ kind: 'file', acquisitionRef: 'same', mediaType: 'text/plain' }, { ref: 'same', title: 'Same' });
|
||||
const [first, second] = await Promise.all([acquire(), acquire()]);
|
||||
assert.equal(first, second);
|
||||
assert.equal(calls, 1);
|
||||
db.run('UPDATE user_books SET hidden = 1 WHERE user_id = ? AND book_id = ?', context.userId, first);
|
||||
assert.equal(await acquire(), first);
|
||||
assert.equal(db.get('SELECT hidden FROM user_books WHERE user_id = ? AND book_id = ?', context.userId, first)?.hidden, 0);
|
||||
} finally {
|
||||
db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('concurrent sources with identical bytes share one file and preserve existing metadata', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-acquisition-'));
|
||||
const { db, config, context } = fixture(root);
|
||||
try {
|
||||
const importer = new FileAcquisitions(db, config);
|
||||
const body = Buffer.from('the same publication');
|
||||
const acquire = (entryRef: string, title: string) => importer.acquire(
|
||||
provider(body), context, { entryRef }, { kind: 'file', acquisitionRef: entryRef, mediaType: 'text/plain' },
|
||||
{ ref: entryRef, title },
|
||||
);
|
||||
const [first, second] = await Promise.all([acquire('first', 'First'), acquire('second', 'Second')]);
|
||||
assert.equal(first, second);
|
||||
assert.equal((await readdir(join(root, 'acquired'))).length, 1);
|
||||
assert.equal(db.get('SELECT COUNT(*) AS n FROM source_acquisitions')?.n, 2);
|
||||
db.run('UPDATE books SET title = ?, author = ?, source = ? WHERE id = ?', 'Edited title', 'Edited author', 'embedded', first);
|
||||
await acquire('third', 'Different catalog title');
|
||||
assert.deepEqual({ ...db.get('SELECT title, author, source FROM books WHERE id = ?', first) },
|
||||
{ title: 'Edited title', author: 'Edited author', source: 'embedded' });
|
||||
assert.equal((await readdir(join(root, 'acquired'))).length, 1);
|
||||
} finally {
|
||||
db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects HTML pretending to be a PDF without adding a book', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-acquisition-'));
|
||||
const { db, config, context } = fixture(root);
|
||||
try {
|
||||
const importer = new FileAcquisitions(db, config);
|
||||
await assert.rejects(importer.acquire(
|
||||
provider(Buffer.from('<html>login required</html>'), 'application/pdf'), context, { entryRef: 'pdf' },
|
||||
{ kind: 'file', acquisitionRef: 'pdf', mediaType: 'application/pdf' }, { ref: 'pdf', title: 'PDF' },
|
||||
), (error: unknown) => (error as { code?: string }).code === 'INVALID_RESOURCE');
|
||||
assert.equal((await readdir(join(root, 'acquired'))).length, 0);
|
||||
assert.equal(db.get('SELECT COUNT(*) AS n FROM books')?.n, 0);
|
||||
} finally {
|
||||
db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('a database failure rolls back rows and removes downloaded file and extracted cover', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-acquisition-'));
|
||||
const { db, config, context } = fixture(root);
|
||||
try {
|
||||
const zip = new JSZip();
|
||||
zip.file('mimetype', 'application/epub+zip');
|
||||
zip.file('META-INF/container.xml', '<container><rootfiles><rootfile full-path="book.opf"/></rootfiles></container>');
|
||||
zip.file('book.opf', `<package><metadata><dc:identifier>test-book</dc:identifier><dc:title>Embedded</dc:title>
|
||||
<meta name="cover" content="cover"/></metadata><manifest>
|
||||
<item id="c1" href="chapter.xhtml" media-type="application/xhtml+xml"/>
|
||||
<item id="cover" href="cover.png" media-type="image/png" properties="cover-image"/>
|
||||
</manifest><spine><itemref idref="c1"/></spine></package>`);
|
||||
zip.file('chapter.xhtml', '<html><body>Text</body></html>');
|
||||
zip.file('cover.png', Buffer.from([137, 80, 78, 71]));
|
||||
const bytes = await zip.generateAsync({ type: 'nodebuffer' });
|
||||
const importer = new FileAcquisitions(db, config);
|
||||
const missingUser = { ...context, userId: 'missing' };
|
||||
await assert.rejects(importer.acquire(
|
||||
provider(bytes, 'application/epub+zip'), missingUser, { entryRef: 'epub' },
|
||||
{ kind: 'file', acquisitionRef: 'epub', mediaType: 'application/epub+zip' }, { ref: 'epub', title: 'Catalog' },
|
||||
), /FOREIGN KEY constraint failed/);
|
||||
assert.equal(db.get('SELECT COUNT(*) AS n FROM books')?.n, 0);
|
||||
assert.equal(db.get('SELECT COUNT(*) AS n FROM acquired_files')?.n, 0);
|
||||
assert.equal((await readdir(join(root, 'acquired'))).length, 0);
|
||||
assert.equal((await readdir(join(root, 'covers'))).length, 0);
|
||||
} finally {
|
||||
db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -86,6 +86,7 @@ before(async () => {
|
||||
|
||||
after(async () => {
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
@@ -187,10 +188,13 @@ test('progress and notes survive a library re-organisation', async () => {
|
||||
});
|
||||
|
||||
test('duplicate copies collapse into one book', async () => {
|
||||
// Generate once: ZIP headers include timestamps, so re-generating an EPUB
|
||||
// is not a byte-identical copy when this suite crosses a clock boundary.
|
||||
const bytes = await makeEpub({ id: 'urn:test:duplicate-copy', title: '副本测试', creator: '测试作者' });
|
||||
await writeFile(join(booksDir, '副本原书.epub'), bytes);
|
||||
await ctx.scanner.scan();
|
||||
const before = ctx.db.all<{ id: string }>('SELECT id FROM books');
|
||||
await writeFile(join(booksDir, '备份 三体.epub'), await makeEpub({
|
||||
id: 'urn:isbn:9787536692930', title: '三体', creator: '刘慈欣', series: '地球往事', index: '1',
|
||||
}));
|
||||
await writeFile(join(booksDir, '副本备份.epub'), bytes);
|
||||
await ctx.scanner.scan();
|
||||
const after = ctx.db.all<{ id: string }>('SELECT id FROM books');
|
||||
assert.equal(after.length, before.length, 'a byte-identical copy must not create a new book');
|
||||
|
||||
@@ -113,6 +113,7 @@ before(async () => {
|
||||
|
||||
after(async () => {
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
*/
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtemp, mkdir, readFile, chmod, rm } from 'node:fs/promises';
|
||||
import { mkdtemp, mkdir, readFile, chmod, rm, stat } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { loadConfig } from '../src/config/index.ts';
|
||||
@@ -84,6 +84,10 @@ test('READER_TOKEN_SECRET wins and skips the file entirely', async () => {
|
||||
});
|
||||
|
||||
test('an unwritable DATA_DIR fails with the fix, not a bare EACCES', async (t) => {
|
||||
if (process.platform === 'win32') {
|
||||
t.skip('Windows does not enforce Unix directory permission bits');
|
||||
return;
|
||||
}
|
||||
if (process.getuid?.() === 0) {
|
||||
// root bypasses the permission bits, so the failure cannot be reproduced.
|
||||
t.skip('running as root; permission bits are not enforced');
|
||||
@@ -117,14 +121,43 @@ test('an unwritable DATA_DIR fails with the fix, not a bare EACCES', async (t) =
|
||||
}
|
||||
});
|
||||
|
||||
test('DATA_DIR inside BOOKS_DIR is refused at boot', async () => {
|
||||
test('DATA_DIR inside BOOKS_DIR is refused before creating the directory', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-config-'));
|
||||
const booksDir = join(root, 'books');
|
||||
const dataDir = join(booksDir, 'nested', 'data');
|
||||
await mkdir(booksDir, { recursive: true });
|
||||
|
||||
try {
|
||||
await withEnv({ DATA_DIR: join(booksDir, 'data'), BOOKS_DIR: booksDir }, () => {
|
||||
await withEnv({ DATA_DIR: dataDir, BOOKS_DIR: booksDir }, async () => {
|
||||
assert.throws(() => loadConfig(), /must not live inside BOOKS_DIR/);
|
||||
await assert.rejects(stat(join(booksDir, 'nested')), { code: 'ENOENT' });
|
||||
});
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('DATA_DIR equal to BOOKS_DIR is refused before creating either directory', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-config-'));
|
||||
const booksDir = join(root, 'not-created');
|
||||
try {
|
||||
await withEnv({ DATA_DIR: booksDir, BOOKS_DIR: booksDir }, async () => {
|
||||
assert.throws(() => loadConfig(), /must not live inside BOOKS_DIR/);
|
||||
await assert.rejects(stat(booksDir), { code: 'ENOENT' });
|
||||
});
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('a sibling DATA_DIR sharing the BOOKS_DIR prefix is permitted', async () => {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-config-'));
|
||||
const booksDir = join(root, 'books');
|
||||
const dataDir = join(root, 'books-data');
|
||||
try {
|
||||
await withEnv({ DATA_DIR: dataDir, BOOKS_DIR: booksDir }, async () => {
|
||||
assert.equal(loadConfig().dataDir, dataDir);
|
||||
assert.ok((await stat(dataDir)).isDirectory());
|
||||
});
|
||||
} finally {
|
||||
await rm(root, { recursive: true, force: true });
|
||||
|
||||
@@ -226,6 +226,7 @@ before(async () => {
|
||||
|
||||
after(async () => {
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { createOpdsProvider } from '../src/sources/opds.ts';
|
||||
import { AppError } from '../src/lib/errors.ts';
|
||||
import type { Acquisition, SourceContext } from '../src/sources/types.ts';
|
||||
|
||||
const ROOT = 'https://library.example/catalog';
|
||||
const ACQUIRE = 'http://opds-spec.org/acquisition';
|
||||
|
||||
function context(config: Record<string, unknown> = { url: ROOT }): SourceContext {
|
||||
return {
|
||||
instance: { id: 'library', pluginId: 'builtin.opds', sourceType: 'opds', name: 'Library', config, enabled: true },
|
||||
userId: 'reader', signal: new AbortController().signal,
|
||||
};
|
||||
}
|
||||
|
||||
function transport(routes: Record<string, string | object | Response>) {
|
||||
const calls: Array<{ url: string; headers: Headers; redirect?: RequestRedirect }> = [];
|
||||
const fetch: typeof globalThis.fetch = async (input, init) => {
|
||||
const url = String(input);
|
||||
calls.push({ url, headers: new Headers(init?.headers), redirect: init?.redirect });
|
||||
const route = routes[url];
|
||||
assert.notEqual(route, undefined, `Unexpected network request: ${url}`);
|
||||
const response = route instanceof Response
|
||||
? route
|
||||
: new Response(typeof route === 'string' ? route : JSON.stringify(route), { status: 200 });
|
||||
Object.defineProperty(response, 'url', { value: url });
|
||||
return response;
|
||||
};
|
||||
return { fetch, calls };
|
||||
}
|
||||
|
||||
function jsonFeed(extra: Record<string, unknown> = {}) {
|
||||
return {
|
||||
metadata: { title: 'Library' },
|
||||
publications: [{
|
||||
metadata: { identifier: 'urn:book:one', title: 'One', author: [{ name: 'Alice' }, { name: 'Bob' }] },
|
||||
images: [{ href: '/covers/one.jpg', type: 'image/jpeg' }],
|
||||
links: [{ href: '/files/one.epub', type: 'application/epub+zip', rel: ACQUIRE }],
|
||||
}],
|
||||
...extra,
|
||||
};
|
||||
}
|
||||
|
||||
function code(expected: string) {
|
||||
return (error: unknown) => error instanceof AppError && error.code === expected;
|
||||
}
|
||||
|
||||
test('OPDS 1 Atom resolves namespaces, xml:base, navigation, pagination and direct acquisitions', async () => {
|
||||
const xml = `<?xml version="1.0"?>
|
||||
<atom:feed xmlns:atom="http://www.w3.org/2005/Atom" xml:base="/books/">
|
||||
<atom:title>A & B</atom:title>
|
||||
<atom:link rel="next" href="?page=2&count=10" type="application/atom+xml"/>
|
||||
<atom:entry><atom:id>urn:category:fiction</atom:id><atom:title>Fiction</atom:title>
|
||||
<atom:link rel="subsection" href="fiction" type="application/atom+xml;profile=opds-catalog"/>
|
||||
</atom:entry>
|
||||
<atom:entry xml:base="one/"><atom:id>urn:book:one</atom:id><atom:title>书籍一</atom:title>
|
||||
<atom:author><atom:name>作者</atom:name></atom:author><atom:summary>Summary</atom:summary>
|
||||
<atom:link rel="http://opds-spec.org/image/thumbnail" href="cover.jpg"/>
|
||||
<atom:link rel="${ACQUIRE}/open-access" href="book.epub" type="application/epub+zip"/>
|
||||
</atom:entry>
|
||||
</atom:feed>`;
|
||||
const network = transport({ [ROOT]: xml });
|
||||
const provider = createOpdsProvider(network);
|
||||
const page = await provider.browse!(context(), {});
|
||||
assert.equal(page.title, 'A & B');
|
||||
assert.deepEqual(page.navigation, [{ title: 'Fiction', ref: 'https://library.example/books/fiction', kind: 'catalog' }]);
|
||||
assert.equal(page.nextCursor, 'https://library.example/books/?page=2&count=10');
|
||||
assert.equal(page.items[0]?.title, '书籍一');
|
||||
assert.deepEqual(page.items[0]?.authors, ['作者']);
|
||||
assert.equal(page.items[0]?.coverUrl, 'https://library.example/books/one/cover.jpg');
|
||||
assert.equal(page.items[0]?.options?.[0]?.available, true);
|
||||
});
|
||||
|
||||
test('OPDS 2 groups and navigation remain separate from publication entries', async () => {
|
||||
const network = transport({ [ROOT]: jsonFeed({
|
||||
links: [{ rel: 'next', href: '?page=2' }],
|
||||
navigation: [{ title: 'Recent', href: '/recent', type: 'application/opds+json' }],
|
||||
groups: [{ publications: [{ metadata: { identifier: 'urn:book:two', title: 'Two' }, links: [{ rel: ACQUIRE, href: '/two.pdf', type: 'application/pdf' }] }] }],
|
||||
}) });
|
||||
const page = await createOpdsProvider(network).browse!(context(), {});
|
||||
assert.deepEqual(page.items.map((item) => item.title), ['One', 'Two']);
|
||||
assert.deepEqual(page.items[0]?.authors, ['Alice', 'Bob']);
|
||||
assert.equal(page.items[0]?.coverUrl, 'https://library.example/covers/one.jpg');
|
||||
assert.equal(page.navigation?.[0]?.title, 'Recent');
|
||||
assert.equal(page.nextCursor, `${ROOT}?page=2`);
|
||||
});
|
||||
|
||||
test('acquisition revalidates provider references and streams the selected file', async () => {
|
||||
const network = transport({ [ROOT]: jsonFeed(), 'https://library.example/files/one.epub': new Response('epub-bytes', { headers: { 'content-type': 'application/epub+zip', etag: '"one"' } }) });
|
||||
const provider = createOpdsProvider(network);
|
||||
const ctx = context();
|
||||
const entry = (await provider.browse!(ctx, {})).items[0]!;
|
||||
const acquisition = await provider.acquire(ctx, { entryRef: entry.ref });
|
||||
assert.equal(acquisition.kind, 'file');
|
||||
assert.equal('url' in acquisition, false, 'host must resolve a provider reference, not fetch a client-supplied URL');
|
||||
const file = await provider.openFile!(ctx, acquisition as Extract<Acquisition, { kind: 'file' }>);
|
||||
const chunks: Buffer[] = [];
|
||||
for await (const chunk of file.stream!) chunks.push(Buffer.from(chunk));
|
||||
assert.equal(Buffer.concat(chunks).toString(), 'epub-bytes');
|
||||
assert.equal(file.mediaType, 'application/epub+zip');
|
||||
assert.equal(file.etag, '"one"');
|
||||
await assert.rejects(() => provider.acquire(ctx, { entryRef: entry.ref, optionId: 'invented' }), code('RESOURCE_GONE'));
|
||||
await assert.rejects(() => provider.detail(ctx, 'not-a-ref'), code('INVALID_SOURCE_REF'));
|
||||
});
|
||||
|
||||
test('borrowing, purchases and DRM are explicit external actions rather than file downloads', async () => {
|
||||
for (const [rel, mediaType, expected] of [
|
||||
[`${ACQUIRE}/borrow`, 'application/epub+zip', 'borrow'],
|
||||
[`${ACQUIRE}/buy`, 'application/epub+zip', 'external'],
|
||||
[ACQUIRE, 'application/vnd.adobe.adept+xml', 'external'],
|
||||
]) {
|
||||
const network = transport({ [ROOT]: jsonFeed({ publications: [{ metadata: { identifier: 'one', title: 'One' }, links: [{ rel, type: mediaType, href: '/action' }] }] }) });
|
||||
const provider = createOpdsProvider(network);
|
||||
const entry = (await provider.browse!(context(), {})).items[0]!;
|
||||
assert.equal(entry.options?.[0]?.available, false);
|
||||
const acquisition = await provider.acquire(context(), { entryRef: entry.ref });
|
||||
assert.equal(acquisition.kind, 'action-required');
|
||||
if (acquisition.kind === 'action-required') assert.equal(acquisition.action.type, expected);
|
||||
}
|
||||
});
|
||||
|
||||
test('JSON search templates encode user input and pagination follows advertised links', async () => {
|
||||
const searchUrl = 'https://library.example/search?q=%E4%B8%89%E4%BD%93%20%26%20A&limit=20';
|
||||
const network = transport({
|
||||
[ROOT]: jsonFeed({ links: [{ rel: 'search', href: '/search?q={searchTerms}&limit={count}', type: 'application/opds+json', templated: true }] }),
|
||||
[searchUrl]: jsonFeed({ links: [{ rel: 'next', href: '/search?page=2' }] }),
|
||||
'https://library.example/search?page=2': jsonFeed(),
|
||||
});
|
||||
const provider = createOpdsProvider(network);
|
||||
const first = await provider.search!(context(), { query: '三体 & A', limit: 20 });
|
||||
assert.equal(first.items.length, 1);
|
||||
await provider.search!(context(), { query: '三体 & A', cursor: first.nextCursor });
|
||||
assert.equal(network.calls.at(-1)?.url, 'https://library.example/search?page=2');
|
||||
});
|
||||
|
||||
test('OPDS 1 OpenSearch descriptions resolve their relative GET templates', async () => {
|
||||
const network = transport({
|
||||
[ROOT]: '<feed xmlns="http://www.w3.org/2005/Atom"><title>Search</title><link rel="search" href="/search/description.xml" type="application/opensearchdescription+xml"/></feed>',
|
||||
'https://library.example/search/description.xml': '<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/"><Url type="application/atom+xml" template="results?q={searchTerms}&page={startPage}&language={language?}"/></OpenSearchDescription>',
|
||||
'https://library.example/search/results?q=Hello%20World&page=1&language=': '<feed xmlns="http://www.w3.org/2005/Atom"><title>Results</title></feed>',
|
||||
});
|
||||
const result = await createOpdsProvider(network).search!(context(), { query: 'Hello World' });
|
||||
assert.equal(result.title, 'Results');
|
||||
assert.equal(network.calls.length, 3);
|
||||
});
|
||||
|
||||
test('missing search capability produces a stable unsupported error', async () => {
|
||||
const provider = createOpdsProvider(transport({ [ROOT]: jsonFeed() }));
|
||||
await assert.rejects(() => provider.search!(context(), { query: 'book' }), code('SOURCE_UNSUPPORTED'));
|
||||
});
|
||||
|
||||
test('basic credentials stay scoped to the configured origin on allowed CDN redirects', async () => {
|
||||
const network = transport({
|
||||
[ROOT]: jsonFeed(),
|
||||
'https://library.example/files/one.epub': new Response(null, { status: 302, headers: { location: 'https://cdn.example/one.epub' } }),
|
||||
'https://cdn.example/one.epub': new Response('epub', { headers: { 'content-type': 'application/epub+zip' } }),
|
||||
});
|
||||
const ctx: SourceContext = {
|
||||
...context({ url: ROOT, username: 'reader', allowedOrigins: ['https://cdn.example'] }),
|
||||
credentials: { get: async () => 'secret', set: async () => {}, delete: async () => {} },
|
||||
};
|
||||
const provider = createOpdsProvider(network);
|
||||
const entry = (await provider.browse!(ctx, {})).items[0]!;
|
||||
const acquisition = await provider.acquire(ctx, { entryRef: entry.ref });
|
||||
assert.equal(acquisition.kind, 'file');
|
||||
const resource = await provider.openFile!(ctx, acquisition as Extract<Acquisition, { kind: 'file' }>);
|
||||
resource.stream!.resume();
|
||||
assert.equal(network.calls[0]?.headers.get('authorization'), `Basic ${Buffer.from('reader:secret').toString('base64')}`);
|
||||
assert.equal(network.calls.at(-1)?.headers.get('authorization'), null);
|
||||
assert.ok(network.calls.every((call) => call.redirect === 'manual'));
|
||||
});
|
||||
|
||||
test('forged origins and redirects are rejected before reaching an unrelated host', async () => {
|
||||
const network = transport({ [ROOT]: new Response(null, { status: 302, headers: { location: 'http://127.0.0.1/private' } }) });
|
||||
const provider = createOpdsProvider(network);
|
||||
await assert.rejects(() => provider.browse!(context(), { cursor: 'https://other.example/private' }), code('OPDS_UNSAFE_URL'));
|
||||
assert.equal(network.calls.length, 0);
|
||||
await assert.rejects(() => provider.browse!(context(), {}), code('OPDS_UNSAFE_URL'));
|
||||
assert.equal(network.calls.length, 1);
|
||||
assert.throws(() => provider.validateConfig!({ url: 'https://user:password@example.com/opds' }), code('INVALID_SOURCE_CONFIG'));
|
||||
assert.throws(() => provider.validateConfig!({ url: ROOT, password: 'secret' }), code('INVALID_SOURCE_CONFIG'));
|
||||
});
|
||||
|
||||
test('malformed XML, entity declarations, oversized feeds and authentication failures are bounded errors', async () => {
|
||||
for (const xml of ['<feed><title>broken</feed>', '<html><title>Login</title></html>', '{}', '<!DOCTYPE feed [<!ENTITY secret SYSTEM "file:///etc/passwd">]><feed><title>&secret;</title></feed>']) {
|
||||
await assert.rejects(() => createOpdsProvider(transport({ [ROOT]: xml })).browse!(context(), {}), code('OPDS_INVALID_RESPONSE'));
|
||||
}
|
||||
await assert.rejects(() => createOpdsProvider({ ...transport({ [ROOT]: jsonFeed() }), maxFeedBytes: 20 }).browse!(context(), {}), code('RESOURCE_TOO_LARGE'));
|
||||
await assert.rejects(() => createOpdsProvider(transport({ [ROOT]: new Response('', { status: 401 }) })).browse!(context(), {}), code('AUTH_REQUIRED'));
|
||||
await assert.rejects(() => createOpdsProvider(transport({ [ROOT]: jsonFeed() })).browse!(context({ url: ROOT, username: 'reader' }), {}), code('AUTH_REQUIRED'));
|
||||
});
|
||||
|
||||
test('a successful HTML login page cannot masquerade as an EPUB download', async () => {
|
||||
const network = transport({
|
||||
[ROOT]: jsonFeed(),
|
||||
'https://library.example/files/one.epub': new Response('<html>login</html>', { headers: { 'content-type': 'text/html' } }),
|
||||
});
|
||||
const provider = createOpdsProvider(network);
|
||||
const entry = (await provider.browse!(context(), {})).items[0]!;
|
||||
const acquisition = await provider.acquire(context(), { entryRef: entry.ref });
|
||||
await assert.rejects(() => provider.openFile!(context(), acquisition as Extract<Acquisition, { kind: 'file' }>), code('OPDS_INVALID_RESPONSE'));
|
||||
});
|
||||
|
||||
test('download streams enforce the byte budget even without Content-Length', async () => {
|
||||
const network = transport({
|
||||
[ROOT]: jsonFeed(),
|
||||
'https://library.example/files/one.epub': new Response('a'.repeat(64), { headers: { 'content-type': 'application/epub+zip' } }),
|
||||
});
|
||||
const provider = createOpdsProvider({ ...network, maxFileBytes: 32 });
|
||||
const entry = (await provider.browse!(context(), {})).items[0]!;
|
||||
const acquisition = await provider.acquire(context(), { entryRef: entry.ref });
|
||||
const file = await provider.openFile!(context(), acquisition as Extract<Acquisition, { kind: 'file' }>);
|
||||
await assert.rejects(async () => { for await (const _chunk of file.stream!) { /* Drain until the provider rejects the oversized body. */ } }, code('RESOURCE_TOO_LARGE'));
|
||||
});
|
||||
|
||||
test('cancelled operations do not start a network request', async () => {
|
||||
const network = transport({ [ROOT]: jsonFeed() });
|
||||
const ctx = { ...context(), signal: AbortSignal.abort() };
|
||||
await assert.rejects(() => createOpdsProvider(network).browse!(ctx, {}), { name: 'AbortError' });
|
||||
assert.equal(network.calls.length, 0);
|
||||
});
|
||||
@@ -0,0 +1,239 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { access, mkdir, mkdtemp, rm, symlink, writeFile } from 'node:fs/promises';
|
||||
import { join } from 'node:path';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { Db } from '../src/db/index.ts';
|
||||
import { PluginManager } from '../src/sources/plugin-manager.ts';
|
||||
import { SourceRegistry } from '../src/sources/registry.ts';
|
||||
import type { SourceContext, SourceProvider } from '../src/sources/types.ts';
|
||||
|
||||
const WORKER = `import { createInterface } from 'node:readline';
|
||||
createInterface({ input: process.stdin }).on('line', (line) => {
|
||||
const request = JSON.parse(line);
|
||||
if (request.params.entryRef === 'hang') return;
|
||||
if (request.params.entryRef === 'crash') process.exit(1);
|
||||
const result = request.method === 'detail' ? { ref: 'book', title: 'Book' } : null;
|
||||
process.stdout.write(JSON.stringify({ jsonrpc: '2.0', id: request.id, result }) + '\\n');
|
||||
});`;
|
||||
|
||||
const builtin: SourceProvider = {
|
||||
descriptor: { id: 'local', label: 'Local', version: '1', capabilities: ['detail'] },
|
||||
async detail() { return { ref: 'book', title: 'Local book' }; },
|
||||
async acquire() { return { kind: 'ready', publicationId: 'book' }; },
|
||||
};
|
||||
|
||||
function context(): SourceContext {
|
||||
return {
|
||||
userId: 'reader', signal: new AbortController().signal,
|
||||
instance: { id: 'test', pluginId: 'test.plugin', sourceType: 'test', name: 'Test', config: {}, enabled: true },
|
||||
};
|
||||
}
|
||||
|
||||
async function fixture() {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-plugin-manager-'));
|
||||
const dataDir = join(root, 'data');
|
||||
await mkdir(join(dataDir, 'plugins'), { recursive: true });
|
||||
const db = new Db(':memory:');
|
||||
const registry = new SourceRegistry();
|
||||
registry.registerBuiltin('reader.local', builtin);
|
||||
const managers = [new PluginManager(db, dataDir, registry)];
|
||||
return {
|
||||
root, dataDir, db, registry, manager: managers[0]!,
|
||||
async package(folder = 'example', pluginId = 'test.plugin', sourceTypes: unknown[] = [{ id: 'test', label: 'Test', capabilities: ['detail'] }]) {
|
||||
const directory = join(dataDir, 'plugins', folder);
|
||||
await mkdir(directory, { recursive: true });
|
||||
await writeFile(join(directory, 'plugin.json'), JSON.stringify({
|
||||
id: pluginId, name: 'Test plugin', version: '1.0.0', apiVersion: 1, runtime: 'node', entry: 'main.mjs', sourceTypes,
|
||||
}));
|
||||
await writeFile(join(directory, 'main.mjs'), WORKER);
|
||||
return directory;
|
||||
},
|
||||
restart() {
|
||||
const manager = new PluginManager(db, dataDir, registry);
|
||||
managers.push(manager);
|
||||
return manager;
|
||||
},
|
||||
async dispose() {
|
||||
for (const manager of managers) await manager.close();
|
||||
db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
test('installation registers a real provider and lists builtins alongside the persisted package', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
await f.package();
|
||||
const installed = await f.manager.install('example');
|
||||
assert.equal(installed.pluginId, 'test.plugin');
|
||||
assert.equal(installed.folder, 'example');
|
||||
assert.equal(installed.enabled, true);
|
||||
assert.equal(installed.runtime?.state, 'stopped', 'processes start lazily on their first operation');
|
||||
const provider = f.registry.require('test.plugin', 'test').provider;
|
||||
assert.equal((await provider.detail(context(), 'book')).title, 'Book');
|
||||
const plugins = f.manager.list();
|
||||
assert.equal(plugins.find((plugin) => plugin.pluginId === 'reader.local')?.builtin, true);
|
||||
assert.equal(plugins.find((plugin) => plugin.pluginId === 'test.plugin')?.runtime?.state, 'running');
|
||||
assert.equal(f.db.get<{ enabled: number }>('SELECT enabled FROM installed_plugins WHERE plugin_id = ?', 'test.plugin')?.enabled, 1);
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
|
||||
test('disable stops in-flight work, rejects retained providers, and persists across restart', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
await f.package();
|
||||
await f.manager.install('example');
|
||||
const provider = f.registry.require('test.plugin', 'test').provider;
|
||||
const pending = assert.rejects(provider.detail(context(), 'hang'), { code: 'PLUGIN_UNAVAILABLE' });
|
||||
const disabled = await f.manager.setEnabled('test.plugin', false);
|
||||
await pending;
|
||||
assert.equal(disabled.enabled, false);
|
||||
assert.equal(disabled.runtime?.state, 'stopped');
|
||||
assert.equal(f.registry.get('test.plugin', 'test'), undefined);
|
||||
await assert.rejects(provider.detail(context(), 'book'), { code: 'PLUGIN_UNAVAILABLE' });
|
||||
await f.manager.close();
|
||||
const restarted = f.restart();
|
||||
await restarted.loadInstalled();
|
||||
assert.equal(restarted.list().find((plugin) => plugin.pluginId === 'test.plugin')?.enabled, false);
|
||||
assert.equal(f.registry.get('test.plugin', 'test'), undefined);
|
||||
await restarted.setEnabled('test.plugin', true);
|
||||
assert.equal((await f.registry.require('test.plugin', 'test').provider.detail(context(), 'book')).title, 'Book');
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
|
||||
test('enabled packages reload after shutdown and failed package loads do not block healthy packages', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
await f.package();
|
||||
await f.manager.install('example');
|
||||
f.db.run('INSERT INTO installed_plugins (plugin_id, folder, enabled) VALUES (?, ?, 1)', 'broken.plugin', 'missing');
|
||||
await f.manager.close();
|
||||
const restarted = f.restart();
|
||||
await restarted.loadInstalled();
|
||||
await restarted.loadInstalled();
|
||||
assert.ok(f.registry.get('test.plugin', 'test'));
|
||||
assert.equal(restarted.list().find((plugin) => plugin.pluginId === 'broken.plugin')?.error?.code, 'PLUGIN_PACKAGE_NOT_FOUND');
|
||||
assert.equal(f.registry.list().length, 2, 'reloading is idempotent');
|
||||
assert.equal((await f.registry.require('test.plugin', 'test').provider.detail(context(), 'book')).title, 'Book');
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
|
||||
test('a crashed worker remains isolated and can be reloaded through enable', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
await f.package();
|
||||
await f.package('healthy', 'healthy.plugin');
|
||||
await f.manager.install('example');
|
||||
await f.manager.install('healthy');
|
||||
await assert.rejects(f.registry.require('test.plugin', 'test').provider.detail(context(), 'crash'), { code: 'PLUGIN_UNAVAILABLE' });
|
||||
assert.equal(f.manager.list().find((plugin) => plugin.pluginId === 'test.plugin')?.runtime?.state, 'failed');
|
||||
assert.equal((await f.registry.require('healthy.plugin', 'test').provider.detail(context(), 'book')).title, 'Book');
|
||||
await f.manager.setEnabled('test.plugin', true);
|
||||
assert.equal((await f.registry.require('test.plugin', 'test').provider.detail(context(), 'book')).title, 'Book');
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
|
||||
test('uninstall retains package files and source data while removing registrations and install records', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const directory = await f.package();
|
||||
await f.manager.install('example');
|
||||
f.db.run("INSERT INTO source_instances (id, plugin_id, source_type, name, config_json, enabled, created_at) VALUES ('keep', 'test.plugin', 'test', 'Keep', '{}', 1, 0)");
|
||||
f.db.run("INSERT INTO plugin_storage (key, value) VALUES ('keep', 'data')");
|
||||
await f.manager.uninstall('test.plugin');
|
||||
assert.equal(f.registry.get('test.plugin', 'test'), undefined);
|
||||
assert.equal(f.db.get('SELECT plugin_id FROM installed_plugins WHERE plugin_id = ?', 'test.plugin'), undefined);
|
||||
assert.ok(f.db.get("SELECT id FROM source_instances WHERE id = 'keep'"));
|
||||
assert.ok(f.db.get("SELECT key FROM plugin_storage WHERE key = 'keep'"));
|
||||
await access(join(directory, 'main.mjs'));
|
||||
await f.manager.install('example');
|
||||
assert.ok(f.registry.get('test.plugin', 'test'));
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
|
||||
test('builtins, duplicate installations and source conflicts cannot be replaced', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
await f.package();
|
||||
await f.package('builtin', 'reader.local');
|
||||
await assert.rejects(f.manager.install('builtin'), { code: 'BUILTIN_PLUGIN_IMMUTABLE' });
|
||||
await assert.rejects(f.manager.setEnabled('reader.local', false), { code: 'BUILTIN_PLUGIN_IMMUTABLE' });
|
||||
await assert.rejects(f.manager.uninstall('reader.local'), { code: 'BUILTIN_PLUGIN_IMMUTABLE' });
|
||||
f.registry.register({ pluginId: 'test.plugin', provider: builtin });
|
||||
await assert.rejects(f.manager.install('example'), { code: 'PLUGIN_SOURCE_CONFLICT' });
|
||||
assert.equal(f.db.all('SELECT plugin_id FROM installed_plugins').length, 0);
|
||||
f.registry.unregister('test.plugin', 'local');
|
||||
await f.manager.install('example');
|
||||
await assert.rejects(f.manager.install('example'), { code: 'PLUGIN_ALREADY_INSTALLED' });
|
||||
assert.equal(f.registry.require('reader.local', 'local').provider, builtin);
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
|
||||
test('invalid multi-source packages leave no partially registered providers or persisted installation', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
await f.package('invalid', 'invalid.plugin', [
|
||||
{ id: 'valid', label: 'Valid', capabilities: ['detail'] },
|
||||
{ id: 'unsupported-file', label: 'Unsupported file', capabilities: ['detail', 'acquire.file'] },
|
||||
]);
|
||||
await assert.rejects(f.manager.install('invalid'), { code: 'PLUGIN_INCOMPATIBLE', statusCode: 400 });
|
||||
assert.equal(f.registry.list().length, 1);
|
||||
assert.equal(f.db.all('SELECT plugin_id FROM installed_plugins').length, 0);
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
|
||||
test('missing packages, invalid JSON and invalid manifests return stable client errors', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
await assert.rejects(f.manager.install('missing'), { code: 'PLUGIN_PACKAGE_NOT_FOUND', statusCode: 404 });
|
||||
const directory = await f.package();
|
||||
await writeFile(join(directory, 'plugin.json'), '{ invalid JSON');
|
||||
await assert.rejects(f.manager.install('example'), { code: 'PLUGIN_INVALID_MANIFEST', statusCode: 400 });
|
||||
await writeFile(join(directory, 'plugin.json'), JSON.stringify({ id: 'test.plugin' }));
|
||||
await assert.rejects(f.manager.install('example'), { code: 'PLUGIN_INVALID_MANIFEST', statusCode: 400 });
|
||||
await f.package();
|
||||
await rm(join(directory, 'main.mjs'));
|
||||
await assert.rejects(f.manager.install('example'), { code: 'PLUGIN_PACKAGE_NOT_FOUND', statusCode: 404 });
|
||||
assert.equal(f.db.all('SELECT plugin_id FROM installed_plugins').length, 0);
|
||||
assert.equal(f.registry.list().length, 1);
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
|
||||
test('install accepts only one package folder and rejects symlink escapes', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
const directory = await f.package();
|
||||
for (const folder of ['../example', './example', '/example', 'example/child', 'example\\child', '.', '', 'example.']) {
|
||||
await assert.rejects(f.manager.install(folder), { code: 'PLUGIN_INVALID_FOLDER' });
|
||||
}
|
||||
const outside = join(f.root, 'outside');
|
||||
await mkdir(outside);
|
||||
await writeFile(join(outside, 'plugin.json'), '{}');
|
||||
await symlink(outside, join(f.dataDir, 'plugins', 'escape'), process.platform === 'win32' ? 'junction' : 'dir');
|
||||
await assert.rejects(f.manager.install('escape'), { code: 'PLUGIN_PATH_ESCAPE' });
|
||||
await access(join(directory, 'plugin.json'));
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
|
||||
test('changed package identity is refused on restart and lifecycle mutations remain ordered', async () => {
|
||||
const f = await fixture();
|
||||
try {
|
||||
await f.package();
|
||||
await f.manager.install('example');
|
||||
const disabled = f.manager.setEnabled('test.plugin', false);
|
||||
const enabled = f.manager.setEnabled('test.plugin', true);
|
||||
await Promise.all([disabled, enabled]);
|
||||
assert.equal(f.manager.list().find((plugin) => plugin.pluginId === 'test.plugin')?.enabled, true);
|
||||
assert.ok(f.registry.get('test.plugin', 'test'));
|
||||
await f.manager.close();
|
||||
await f.package('example', 'changed.plugin');
|
||||
const restarted = f.restart();
|
||||
await restarted.loadInstalled();
|
||||
assert.equal(restarted.list().find((plugin) => plugin.pluginId === 'test.plugin')?.error?.code, 'PLUGIN_ID_CHANGED');
|
||||
assert.equal(f.registry.get('changed.plugin', 'test'), undefined);
|
||||
await restarted.close();
|
||||
await assert.rejects(restarted.setEnabled('test.plugin', true), { code: 'PLUGIN_MANAGER_CLOSED' });
|
||||
} finally { await f.dispose(); }
|
||||
});
|
||||
@@ -0,0 +1,154 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { mkdtemp, mkdir, rm, writeFile } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, resolve } from 'node:path';
|
||||
import { ProcessPlugin } from '../src/sources/process-plugin.ts';
|
||||
import { SourceRegistry, validatePluginManifest } from '../src/sources/registry.ts';
|
||||
import { decodeChapterAcquisition, decodeManifest } from '../src/sources/protocol.ts';
|
||||
import type { PluginManifest, SourceContext, SourceProvider } from '../src/sources/types.ts';
|
||||
|
||||
function context(signal = new AbortController().signal): SourceContext {
|
||||
return {
|
||||
userId: 'user-one',
|
||||
signal,
|
||||
instance: {
|
||||
id: 'instance-one', pluginId: 'reader.source.demo', sourceType: 'demo-chapters',
|
||||
name: 'Demo', config: {}, enabled: true,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const manifest: PluginManifest = {
|
||||
id: 'test.source', name: 'Test', version: '1.0.0', apiVersion: 1, runtime: 'node', entry: 'main.mjs',
|
||||
sourceTypes: [{ id: 'test', label: 'Test', capabilities: ['detail'] }],
|
||||
};
|
||||
|
||||
async function fixture(code: string): Promise<{ directory: string; dispose(): Promise<void> }> {
|
||||
const directory = await mkdtemp(join(tmpdir(), 'reader-plugin-'));
|
||||
await writeFile(join(directory, 'plugin.json'), JSON.stringify(manifest));
|
||||
await writeFile(join(directory, 'main.mjs'), code);
|
||||
return { directory, dispose: () => rm(directory, { recursive: true, force: true }) };
|
||||
}
|
||||
|
||||
test('registry separates plugin namespaces and rejects unsupported advertised capabilities', () => {
|
||||
const registry = new SourceRegistry();
|
||||
const provider: SourceProvider = {
|
||||
descriptor: { id: 'test', label: 'Test', version: '1', capabilities: ['detail'] },
|
||||
async detail() { return { ref: 'book', title: 'Book' }; },
|
||||
async acquire() { return { kind: 'ready', publicationId: 'book' }; },
|
||||
};
|
||||
registry.registerBuiltin('reader.builtin', provider);
|
||||
registry.register({ pluginId: 'reader.external', provider });
|
||||
assert.equal(registry.list().length, 2);
|
||||
assert.equal(registry.require('reader.builtin', 'test').builtin, true);
|
||||
assert.throws(() => registry.register({ pluginId: 'reader.external', provider }), /already registered/);
|
||||
assert.throws(() => registry.register({
|
||||
pluginId: 'reader.invalid',
|
||||
provider: { ...provider, descriptor: { ...provider.descriptor, capabilities: ['search'] } },
|
||||
}), /does not implement search/);
|
||||
assert.throws(() => registry.register({
|
||||
pluginId: 'reader.invalid',
|
||||
provider: { ...provider, descriptor: { ...provider.descriptor, capabilities: ['acquire.chapters'] } },
|
||||
}), /chapter manifest and resources/);
|
||||
});
|
||||
|
||||
test('plugin manifest rejects API version mismatch and duplicate or incomplete source types', () => {
|
||||
assert.equal(validatePluginManifest(manifest).apiVersion, 1);
|
||||
assert.throws(() => validatePluginManifest({ ...manifest, apiVersion: 2 }), /API version/);
|
||||
assert.throws(() => validatePluginManifest({ ...manifest, apiVersion: undefined }), /API version/);
|
||||
assert.throws(() => validatePluginManifest({ ...manifest, sourceTypes: [manifest.sourceTypes[0], manifest.sourceTypes[0]] }), /duplicate/);
|
||||
assert.throws(() => validatePluginManifest({
|
||||
...manifest, sourceTypes: [{ id: 'test', label: 'Test', capabilities: ['acquire.chapters'] }],
|
||||
}), /must declare manifest and resource/);
|
||||
});
|
||||
|
||||
test('real example process supports discovery, acquisition and stable chapter resources', async () => {
|
||||
const plugin = await ProcessPlugin.load(resolve(import.meta.dirname, '../../examples/plugins/demo-chapters'));
|
||||
try {
|
||||
const registry = new SourceRegistry();
|
||||
const provider = plugin.providers()[0]!;
|
||||
registry.registerPluginManifest(plugin.manifest, provider);
|
||||
await provider.validateConfig?.({});
|
||||
const page = await provider.browse!(context(), {});
|
||||
assert.equal(page.items[0]?.ref, 'demo-book');
|
||||
const acquisition = await provider.acquire(context(), { entryRef: page.items[0]!.ref });
|
||||
assert.deepEqual(acquisition, { kind: 'chapters', publicationRef: 'demo-book' });
|
||||
const snapshot = await provider.getManifest!(context(), 'demo-book');
|
||||
assert.equal(snapshot.items[0]?.id, 'chapter-one');
|
||||
const resource = await provider.readResource!(context(), { publicationRef: 'demo-book', ref: snapshot.items[0]!.ref });
|
||||
assert.match(resource.text!, /独立 Node 进程/);
|
||||
await assert.rejects(provider.detail(context(), 'missing'), { code: 'RESOURCE_GONE' });
|
||||
assert.equal(plugin.status().state, 'running');
|
||||
assert.equal(plugin.status().pendingRequests, 0);
|
||||
} finally {
|
||||
await plugin.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('process timeout terminates pending work and requires a reload', async () => {
|
||||
const source = await fixture('process.stdin.resume();');
|
||||
const plugin = await ProcessPlugin.load(source.directory, { timeoutMs: 100 });
|
||||
try {
|
||||
await assert.rejects(plugin.providers()[0]!.detail(context(), 'book'), { code: 'PLUGIN_TIMEOUT' });
|
||||
assert.equal(plugin.status().state, 'failed');
|
||||
assert.equal(plugin.status().pendingRequests, 0);
|
||||
await assert.rejects(plugin.providers()[0]!.detail(context(), 'book'), { code: 'PLUGIN_UNAVAILABLE' });
|
||||
} finally {
|
||||
await plugin.close();
|
||||
await source.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
test('cancellation kills an uncooperative worker and rejects its concurrent calls', async () => {
|
||||
const source = await fixture('process.stdin.resume();');
|
||||
const plugin = await ProcessPlugin.load(source.directory);
|
||||
try {
|
||||
const controller = new AbortController();
|
||||
const pending = plugin.providers()[0]!.detail(context(controller.signal), 'book');
|
||||
const concurrent = plugin.providers()[0]!.detail(context(), 'another-book');
|
||||
const outcomes = Promise.allSettled([pending, concurrent]);
|
||||
controller.abort();
|
||||
for (const result of await outcomes) {
|
||||
assert.equal(result.status, 'rejected');
|
||||
if (result.status === 'rejected') assert.equal(result.reason.code, 'PLUGIN_CANCELLED');
|
||||
}
|
||||
assert.equal(plugin.status().pendingRequests, 0);
|
||||
assert.equal(plugin.status().state, 'failed');
|
||||
} finally {
|
||||
await plugin.close();
|
||||
await source.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
test('wire validation prevents plugins claiming host books or corrupting chapter identities', () => {
|
||||
assert.throws(() => decodeChapterAcquisition({ kind: 'ready', publicationId: 'other-users-book' }), /must acquire chapter/);
|
||||
assert.throws(() => decodeChapterAcquisition({ kind: 'action-required', action: { type: 'external', label: 'Open', url: 'javascript:alert(1)' } }), /HTTP or HTTPS/);
|
||||
const item = { id: 'stable', seq: 0, title: 'Chapter', kind: 'chapter', mediaType: 'text/plain', ref: 'chapter' };
|
||||
assert.throws(() => decodeManifest({ publicationRef: 'wrong', items: [item] }, 'requested'), /does not match/);
|
||||
assert.throws(() => decodeManifest({ publicationRef: 'book', items: [item, { ...item, seq: 1 }] }, 'book'), /duplicate chapter/);
|
||||
assert.equal(decodeManifest({ publicationRef: 'book', items: [item] }, 'book').items[0]?.id, 'stable');
|
||||
});
|
||||
|
||||
test('oversized responses fail the plugin instead of buffering unbounded output', async () => {
|
||||
const source = await fixture(`process.stdin.on('data', () => process.stdout.write('x'.repeat(4096)));`);
|
||||
const plugin = await ProcessPlugin.load(source.directory, { maxMessageBytes: 1024 });
|
||||
try {
|
||||
await assert.rejects(plugin.providers()[0]!.detail(context(), 'book'), { code: 'PLUGIN_MESSAGE_TOO_LARGE' });
|
||||
assert.equal(plugin.status().state, 'failed');
|
||||
} finally {
|
||||
await plugin.close();
|
||||
await source.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
test('plugin entry cannot escape its package directory', async () => {
|
||||
const source = await fixture('');
|
||||
try {
|
||||
await mkdir(join(source.directory, 'package'));
|
||||
await writeFile(join(source.directory, 'package', 'plugin.json'), JSON.stringify({ ...manifest, entry: '../main.mjs' }));
|
||||
await assert.rejects(ProcessPlugin.load(join(source.directory, 'package')), { code: 'PLUGIN_INVALID' });
|
||||
} finally {
|
||||
await source.dispose();
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,389 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { createServer } from 'node:http';
|
||||
import { cp, mkdir, mkdtemp, readdir, rm, writeFile } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, resolve } from 'node:path';
|
||||
import type { FastifyInstance } from 'fastify';
|
||||
import type { AppConfig } from '../src/config/index.ts';
|
||||
import { openDatabase } from '../src/db/index.ts';
|
||||
import { buildApp } from '../src/http/app.ts';
|
||||
import type { AppContext } from '../src/http/context.ts';
|
||||
import { Scanner } from '../src/indexer/scanner.ts';
|
||||
import { UserService } from '../src/services/users.ts';
|
||||
import { ShelfService } from '../src/services/shelf.ts';
|
||||
import { SyncService } from '../src/services/sync.ts';
|
||||
import { TtsService } from '../src/services/tts.ts';
|
||||
import { BrowseService } from '../src/services/browse.ts';
|
||||
import { UploadService } from '../src/services/uploads.ts';
|
||||
import type { CatalogPage, SourceProvider } from '../src/sources/types.ts';
|
||||
|
||||
interface Session { token: string; id: string }
|
||||
|
||||
function auth(session: Session) { return { authorization: `Bearer ${session.token}` }; }
|
||||
|
||||
async function harness() {
|
||||
const root = await mkdtemp(join(tmpdir(), 'reader-sources-api-'));
|
||||
const booksDir = join(root, 'books');
|
||||
const dataDir = join(root, 'data');
|
||||
await mkdir(booksDir);
|
||||
await mkdir(dataDir);
|
||||
const config: AppConfig = {
|
||||
booksDir, dataDir, host: '127.0.0.1', port: 0, jwtSecret: 'sources-api-test-secret-stable-across-restarts',
|
||||
accessTokenTtl: 86400, refreshTokenTtl: 86400, scanInterval: 0, watchInterval: 0,
|
||||
logLevel: 'silent', publicUrl: '', corsOrigins: [], webDir: join(root, 'no-web'),
|
||||
};
|
||||
let app: FastifyInstance;
|
||||
let ctx: AppContext;
|
||||
let active = false;
|
||||
async function start() {
|
||||
const db = openDatabase(config);
|
||||
ctx = { config, db } as AppContext;
|
||||
app = buildApp(ctx);
|
||||
ctx.log = app.log;
|
||||
ctx.scanner = new Scanner(db, config, { info() {}, warn() {} });
|
||||
ctx.users = new UserService(db, config);
|
||||
ctx.shelf = new ShelfService(db);
|
||||
ctx.sync = new SyncService(db, ctx.shelf);
|
||||
ctx.tts = new TtsService(config);
|
||||
ctx.browse = new BrowseService(db, config, ctx.shelf);
|
||||
ctx.uploads = new UploadService(db, config, ctx.browse, ctx.scanner);
|
||||
active = true;
|
||||
await app.ready();
|
||||
}
|
||||
async function stop() {
|
||||
if (!active) return;
|
||||
active = false;
|
||||
try { await app.close(); } finally { ctx.db.close(); }
|
||||
}
|
||||
await start();
|
||||
const response = await app!.inject({ method: 'POST', url: '/api/v1/auth/register', payload: { username: 'owner', password: 'password123' } });
|
||||
assert.equal(response.statusCode, 201, response.body);
|
||||
const created = response.json();
|
||||
const admin: Session = { token: created.session.accessToken, id: created.user.id };
|
||||
return {
|
||||
root, booksDir, dataDir, admin,
|
||||
get app() { return app; }, get ctx() { return ctx; },
|
||||
async member(): Promise<Session> {
|
||||
const create = await app.inject({ method: 'POST', url: '/api/v1/admin/users', headers: auth(admin), payload: { username: 'reader', password: 'password123' } });
|
||||
assert.equal(create.statusCode, 201, create.body);
|
||||
const login = await app.inject({ method: 'POST', url: '/api/v1/auth/login', payload: { username: 'reader', password: 'password123' } });
|
||||
assert.equal(login.statusCode, 200, login.body);
|
||||
return { token: login.json().accessToken, id: login.json().user.id };
|
||||
},
|
||||
async restart() { await stop(); await start(); },
|
||||
async close() { await stop(); await rm(root, { recursive: true, force: true }); },
|
||||
};
|
||||
}
|
||||
|
||||
async function opdsFixture() {
|
||||
const body = '第一章 远程下载\n\n这是经 OPDS 获取的本地可读正文。\n\n第二章 继续阅读\n\n下载后继续复用现有书架和阅读进度。';
|
||||
let downloads = 0;
|
||||
const server = createServer((request, response) => {
|
||||
if (request.url === '/catalog') {
|
||||
response.setHeader('content-type', 'application/opds+json');
|
||||
response.end(JSON.stringify({
|
||||
metadata: { title: 'Test OPDS' },
|
||||
publications: [{
|
||||
metadata: { identifier: 'urn:reader:test:opds-txt', title: 'OPDS 集成测试书', author: [{ name: '测试作者' }] },
|
||||
links: [{ rel: 'http://opds-spec.org/acquisition/open-access', href: '/download/book.txt', type: 'text/plain' }],
|
||||
}],
|
||||
}));
|
||||
} else if (request.url === '/download/book.txt') {
|
||||
downloads += 1;
|
||||
response.setHeader('content-type', 'text/plain; charset=utf-8');
|
||||
response.end(body);
|
||||
} else {
|
||||
response.statusCode = 404; response.end();
|
||||
}
|
||||
});
|
||||
await new Promise<void>((done) => server.listen(0, '127.0.0.1', done));
|
||||
const address = server.address();
|
||||
assert.ok(address && typeof address !== 'string');
|
||||
return {
|
||||
url: `http://127.0.0.1:${address.port}/catalog`, body,
|
||||
get downloads() { return downloads; },
|
||||
async close() {
|
||||
server.closeAllConnections();
|
||||
await new Promise<void>((done, reject) => server.close((error) => error ? reject(error) : done()));
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
test('built-in source discovery and administrative boundaries preserve configuration and credentials', async (t) => {
|
||||
const h = await harness(); t.after(() => h.close());
|
||||
const member = await h.member();
|
||||
const types = await h.app.inject({ method: 'GET', url: '/api/v1/sources/types', headers: auth(member) });
|
||||
assert.equal(types.statusCode, 200, types.body);
|
||||
assert.deepEqual(types.json().types.map((type: { id: string }) => type.id).sort(), ['local', 'opds']);
|
||||
assert.deepEqual(types.json().types.map((type: { id: string; pluginId: string; builtin: boolean }) =>
|
||||
({ id: type.id, pluginId: type.pluginId, builtin: type.builtin })), [
|
||||
{ id: 'local', pluginId: 'reader.local', builtin: true },
|
||||
{ id: 'opds', pluginId: 'reader.opds', builtin: true },
|
||||
]);
|
||||
assert.ok(h.ctx.sources, 'buildApp initializes the source host');
|
||||
const payload = { id: 'secured-opds', pluginId: 'reader.opds', sourceType: 'opds', name: 'OPDS', config: { url: 'http://127.0.0.1:9999/catalog', username: 'private-config-user' } };
|
||||
const deniedCreate = await h.app.inject({ method: 'POST', url: '/api/v1/sources', headers: auth(member), payload });
|
||||
assert.equal(deniedCreate.statusCode, 403, deniedCreate.body);
|
||||
const deniedInstall = await h.app.inject({ method: 'POST', url: '/api/v1/plugins', headers: auth(member), payload: { folder: 'demo-chapters', trusted: true } });
|
||||
assert.equal(deniedInstall.statusCode, 403, deniedInstall.body);
|
||||
const deniedList = await h.app.inject({ method: 'GET', url: '/api/v1/plugins', headers: auth(member) });
|
||||
assert.equal(deniedList.statusCode, 403, deniedList.body);
|
||||
const noAuth = await h.app.inject({ method: 'GET', url: '/api/v1/sources' });
|
||||
assert.equal(noAuth.statusCode, 401, noAuth.body);
|
||||
|
||||
const created = await h.app.inject({ method: 'POST', url: '/api/v1/sources', headers: auth(h.admin), payload });
|
||||
assert.equal(created.statusCode, 201, created.body);
|
||||
const password = 'source-password-not-in-a-dto';
|
||||
const credential = await h.app.inject({ method: 'PUT', url: '/api/v1/sources/secured-opds/credentials/password', headers: auth(member), payload: { value: password } });
|
||||
assert.equal(credential.statusCode, 200, credential.body);
|
||||
const stored = h.ctx.db.get<{ encrypted_value: string }>('SELECT encrypted_value FROM source_credentials WHERE source_id = ? AND user_id = ?', 'secured-opds', member.id);
|
||||
assert.ok(stored?.encrypted_value);
|
||||
assert.ok(!stored.encrypted_value.includes(password));
|
||||
const publicSources = await h.app.inject({ method: 'GET', url: '/api/v1/sources', headers: auth(member) });
|
||||
assert.equal(publicSources.statusCode, 200, publicSources.body);
|
||||
assert.equal(publicSources.json().sources.find((source: { id: string }) => source.id === 'secured-opds').config, undefined);
|
||||
assert.ok(!publicSources.body.includes('private-config-user'));
|
||||
assert.ok(!publicSources.body.includes(password));
|
||||
const adminSources = await h.app.inject({ method: 'GET', url: '/api/v1/sources', headers: auth(h.admin) });
|
||||
assert.equal(adminSources.statusCode, 200, adminSources.body);
|
||||
assert.equal(adminSources.json().sources.find((source: { id: string }) => source.id === 'secured-opds').config.username, 'private-config-user');
|
||||
assert.ok(!adminSources.body.includes(password));
|
||||
const inlinePassword = await h.app.inject({ method: 'POST', url: '/api/v1/sources', headers: auth(h.admin), payload: { ...payload, id: 'bad-password', config: { ...payload.config, password } } });
|
||||
assert.equal(inlinePassword.statusCode, 400, inlinePassword.body);
|
||||
});
|
||||
|
||||
test('source query validation rejects repeated text parameters and invalid page limits', async (t) => {
|
||||
const h = await harness(); t.after(() => h.close());
|
||||
for (const query of ['limit=0', 'limit=-1', 'limit=201', 'limit=1.5', 'limit=NaN', 'limit=10&limit=20']) {
|
||||
const response = await h.app.inject({ method: 'GET', url: `/api/v1/sources/local/browse?${query}`, headers: auth(h.admin) });
|
||||
assert.equal(response.statusCode, 400, `${query}: ${response.body}`);
|
||||
}
|
||||
for (const query of ['q=first&q=second', 'q=', 'q=valid&cursor=one&cursor=two']) {
|
||||
const response = await h.app.inject({ method: 'GET', url: `/api/v1/sources/local/search?${query}`, headers: auth(h.admin) });
|
||||
assert.equal(response.statusCode, 400, `${query}: ${response.body}`);
|
||||
}
|
||||
});
|
||||
|
||||
test('local source acquisition restores the existing publication to the requesting users shelf', async (t) => {
|
||||
const h = await harness(); t.after(() => h.close());
|
||||
const member = await h.member();
|
||||
await writeFile(join(h.booksDir, '本地书.txt'), '第一章\n\n本地来源沿用既有内容和身份。');
|
||||
await h.ctx.scanner.scan();
|
||||
const browse = await h.app.inject({ method: 'GET', url: '/api/v1/sources/local/browse', headers: auth(member) });
|
||||
assert.equal(browse.statusCode, 200, browse.body);
|
||||
const entry = browse.json().items.find((item: { title: string }) => item.title === '本地书');
|
||||
assert.ok(entry, browse.body);
|
||||
h.ctx.db.run('UPDATE user_books SET hidden = 1 WHERE user_id = ? AND book_id = ?', member.id, entry.ref);
|
||||
const acquired = await h.app.inject({ method: 'POST', url: '/api/v1/sources/local/acquire', headers: auth(member), payload: { entryRef: entry.ref, optionId: 'read' } });
|
||||
assert.equal(acquired.statusCode, 200, acquired.body);
|
||||
assert.deepEqual(acquired.json(), { kind: 'ready', publicationId: entry.ref });
|
||||
const shelf = await h.app.inject({ method: 'GET', url: '/api/v1/books', headers: auth(member) });
|
||||
assert.ok(shelf.json().items.some((book: { id: string }) => book.id === entry.ref), shelf.body);
|
||||
const content = await h.app.inject({ method: 'GET', url: `/api/v1/books/${entry.ref}/content`, headers: auth(member) });
|
||||
assert.equal(content.statusCode, 200, content.body);
|
||||
assert.match(content.body, /既有内容和身份/);
|
||||
});
|
||||
|
||||
test('OPDS acquisition feeds existing reading, shelf and progress APIs and survives source disable and restart', async (t) => {
|
||||
const h = await harness(); t.after(() => h.close());
|
||||
const catalog = await opdsFixture(); t.after(() => catalog.close());
|
||||
const member = await h.member();
|
||||
const source = await h.app.inject({ method: 'POST', url: '/api/v1/sources', headers: auth(h.admin), payload: {
|
||||
id: 'test-opds', pluginId: 'reader.opds', sourceType: 'opds', name: 'Test OPDS', config: { url: catalog.url },
|
||||
} });
|
||||
assert.equal(source.statusCode, 201, source.body);
|
||||
const browse = await h.app.inject({ method: 'GET', url: '/api/v1/sources/test-opds/browse', headers: auth(member) });
|
||||
assert.equal(browse.statusCode, 200, browse.body);
|
||||
const entry = browse.json().items[0];
|
||||
assert.equal(entry.title, 'OPDS 集成测试书');
|
||||
const request = { entryRef: entry.ref, optionId: entry.options[0].id };
|
||||
const acquired = await h.app.inject({ method: 'POST', url: '/api/v1/sources/test-opds/acquire', headers: auth(member), payload: request });
|
||||
assert.equal(acquired.statusCode, 200, acquired.body);
|
||||
const { publicationId: bookId, kind } = acquired.json();
|
||||
assert.equal(kind, 'ready');
|
||||
assert.ok(bookId);
|
||||
assert.equal(catalog.downloads, 1);
|
||||
const originalFiles = (await readdir(join(h.dataDir, 'acquired'))).sort();
|
||||
assert.equal(originalFiles.length, 1);
|
||||
const repeated = await h.app.inject({ method: 'POST', url: '/api/v1/sources/test-opds/acquire', headers: auth(member), payload: request });
|
||||
assert.equal(repeated.statusCode, 200, repeated.body);
|
||||
assert.equal(repeated.json().publicationId, bookId);
|
||||
assert.equal(catalog.downloads, 1, 'repeated acquisition reuses the imported file');
|
||||
assert.deepEqual((await readdir(join(h.dataDir, 'acquired'))).sort(), originalFiles, 'repeated acquisition creates no additional files');
|
||||
|
||||
const otherShelf = await h.app.inject({ method: 'GET', url: '/api/v1/books', headers: auth(h.admin) });
|
||||
assert.ok(!otherShelf.json().items.some((book: { id: string }) => book.id === bookId), otherShelf.body);
|
||||
const otherContent = await h.app.inject({ method: 'GET', url: `/api/v1/books/${bookId}/content`, headers: auth(h.admin) });
|
||||
assert.equal(otherContent.statusCode, 404, otherContent.body);
|
||||
const unauthorizedAdd = await h.app.inject({ method: 'POST', url: '/api/v1/library/browse/shelf', headers: auth(h.admin), payload: { bookIds: [bookId], action: 'add' } });
|
||||
assert.equal(unauthorizedAdd.statusCode, 200, unauthorizedAdd.body);
|
||||
assert.equal(unauthorizedAdd.json().applied, 0);
|
||||
assert.deepEqual(unauthorizedAdd.json().failed, [{ path: bookId, reason: 'NO_LIVE_FILE' }]);
|
||||
const stillForbidden = await h.app.inject({ method: 'GET', url: `/api/v1/books/${bookId}/content`, headers: auth(h.admin) });
|
||||
assert.equal(stillForbidden.statusCode, 404, stillForbidden.body);
|
||||
|
||||
const removedFromShelf = await h.app.inject({ method: 'POST', url: '/api/v1/library/browse/shelf', headers: auth(member), payload: { bookIds: [bookId], action: 'remove' } });
|
||||
assert.equal(removedFromShelf.statusCode, 200, removedFromShelf.body);
|
||||
assert.equal(removedFromShelf.json().applied, 1, removedFromShelf.body);
|
||||
assert.deepEqual(removedFromShelf.json().failed, []);
|
||||
const hiddenShelf = await h.app.inject({ method: 'GET', url: '/api/v1/books', headers: auth(member) });
|
||||
assert.ok(!hiddenShelf.json().items.some((book: { id: string }) => book.id === bookId), hiddenShelf.body);
|
||||
const restored = await h.app.inject({ method: 'POST', url: '/api/v1/sources/test-opds/acquire', headers: auth(member), payload: request });
|
||||
assert.equal(restored.statusCode, 200, restored.body);
|
||||
assert.equal(restored.json().publicationId, bookId);
|
||||
assert.equal(catalog.downloads, 1, 'restoring a book to the shelf reuses the existing download');
|
||||
assert.deepEqual((await readdir(join(h.dataDir, 'acquired'))).sort(), originalFiles);
|
||||
|
||||
const manifest = await h.app.inject({ method: 'GET', url: `/api/v1/books/${bookId}/manifest`, headers: auth(member) });
|
||||
assert.equal(manifest.statusCode, 200, manifest.body);
|
||||
assert.equal(manifest.json().book.title, 'OPDS 集成测试书');
|
||||
const first = manifest.json().items[0]; assert.ok(first);
|
||||
const asset = await h.app.inject({ method: 'GET', url: `/api/v1/books/${bookId}/assets?ref=${encodeURIComponent(first.href)}`, headers: auth(member) });
|
||||
assert.equal(asset.statusCode, 200, asset.body);
|
||||
assert.match(asset.body, /远程下载|OPDS/);
|
||||
const content = await h.app.inject({ method: 'GET', url: `/api/v1/books/${bookId}/content`, headers: auth(member) });
|
||||
assert.equal(content.statusCode, 200, content.body);
|
||||
assert.equal(content.body, catalog.body);
|
||||
const shelf = await h.app.inject({ method: 'GET', url: '/api/v1/books', headers: auth(member) });
|
||||
assert.ok(shelf.json().items.some((book: { id: string }) => book.id === bookId), shelf.body);
|
||||
const progress = await h.app.inject({ method: 'PUT', url: `/api/v1/sync/progress/${bookId}`, headers: auth(member), payload: {
|
||||
locator: first.href, percentage: 0.42, chapterTitle: first.title, device: 'test', updatedAt: Date.now(),
|
||||
} });
|
||||
assert.equal(progress.statusCode, 200, progress.body);
|
||||
assert.equal(progress.json().progress.percentage, 0.42);
|
||||
|
||||
const disabled = await h.app.inject({ method: 'PATCH', url: '/api/v1/sources/test-opds', headers: auth(h.admin), payload: { enabled: false } });
|
||||
assert.equal(disabled.statusCode, 200, disabled.body);
|
||||
const denied = await h.app.inject({ method: 'GET', url: '/api/v1/sources/test-opds/browse', headers: auth(member) });
|
||||
assert.equal(denied.json().error.code, 'SOURCE_DISABLED');
|
||||
await h.ctx.scanner.scan();
|
||||
await h.restart();
|
||||
const retained = await h.app.inject({ method: 'GET', url: `/api/v1/books/${bookId}/content`, headers: auth(member) });
|
||||
assert.equal(retained.statusCode, 200, retained.body);
|
||||
assert.equal(retained.body, catalog.body);
|
||||
const retainedProgress = await h.app.inject({ method: 'GET', url: `/api/v1/sync/progress/${bookId}`, headers: auth(member) });
|
||||
assert.equal(retainedProgress.json().progress.percentage, 0.42);
|
||||
const retainedShelf = await h.app.inject({ method: 'GET', url: '/api/v1/books', headers: auth(member) });
|
||||
assert.ok(retainedShelf.json().items.some((book: { id: string }) => book.id === bookId), retainedShelf.body);
|
||||
});
|
||||
|
||||
test('source host enforces per-source and global concurrency limits and releases slots after completion', { timeout: 10_000 }, async (t) => {
|
||||
const h = await harness(); t.after(() => h.close());
|
||||
const queued: Array<{ sourceId: string; release(): void }> = [];
|
||||
const entered = new Map<string, number>();
|
||||
const watchers: Array<{ sourceId: string; count: number; resolve(): void }> = [];
|
||||
let hold = true;
|
||||
const provider: SourceProvider = {
|
||||
descriptor: { id: 'blocking', label: 'Concurrency test', version: '1', capabilities: ['browse', 'detail'] },
|
||||
async detail(_ctx, ref) { return { ref, title: ref }; },
|
||||
async acquire() { return { kind: 'action-required', action: { type: 'external', label: 'Test' } }; },
|
||||
async browse(ctx) {
|
||||
const count = (entered.get(ctx.instance.id) ?? 0) + 1;
|
||||
entered.set(ctx.instance.id, count);
|
||||
for (const watcher of watchers) {
|
||||
if (watcher.sourceId === ctx.instance.id && count >= watcher.count) watcher.resolve();
|
||||
}
|
||||
if (!hold) return { items: [] };
|
||||
return new Promise<CatalogPage>((resolvePage) => {
|
||||
queued.push({ sourceId: ctx.instance.id, release: () => resolvePage({ items: [] }) });
|
||||
});
|
||||
},
|
||||
};
|
||||
h.ctx.sources!.registry.register({ pluginId: 'reader.test', provider });
|
||||
for (const id of ['limit-one', 'limit-two', 'limit-three', 'limit-four', 'limit-five']) {
|
||||
const created = await h.app.inject({ method: 'POST', url: '/api/v1/sources', headers: auth(h.admin), payload: {
|
||||
id, pluginId: 'reader.test', sourceType: 'blocking', name: id, config: {},
|
||||
} });
|
||||
assert.equal(created.statusCode, 201, created.body);
|
||||
}
|
||||
const waitUntilEntered = (sourceId: string, count: number): Promise<void> => {
|
||||
if ((entered.get(sourceId) ?? 0) >= count) return Promise.resolve();
|
||||
return new Promise((resolveEntered) => watchers.push({ sourceId, count, resolve: resolveEntered }));
|
||||
};
|
||||
const browse = (sourceId: string) => h.app.inject({
|
||||
method: 'GET' as const, url: `/api/v1/sources/${sourceId}/browse`, headers: auth(h.admin),
|
||||
});
|
||||
const pending: ReturnType<typeof browse>[] = [];
|
||||
try {
|
||||
const first = browse('limit-one'); const second = browse('limit-one');
|
||||
pending.push(first, second);
|
||||
await waitUntilEntered('limit-one', 2);
|
||||
const blocked = await browse('limit-one');
|
||||
assert.equal(blocked.statusCode, 429, blocked.body);
|
||||
assert.equal(blocked.json().error.code, 'RATE_LIMITED');
|
||||
assert.equal(entered.get('limit-one'), 2, 'rejected calls never invoke the provider');
|
||||
hold = false;
|
||||
for (const request of queued.splice(0)) request.release();
|
||||
assert.equal((await first).statusCode, 200);
|
||||
assert.equal((await second).statusCode, 200);
|
||||
const recovered = await browse('limit-one');
|
||||
assert.equal(recovered.statusCode, 200, recovered.body);
|
||||
|
||||
hold = true;
|
||||
const sourceIds = ['limit-one', 'limit-two', 'limit-three', 'limit-four'];
|
||||
const globalRequests: ReturnType<typeof browse>[] = [];
|
||||
for (const sourceId of sourceIds) {
|
||||
const count = (entered.get(sourceId) ?? 0) + 2;
|
||||
const pair = [browse(sourceId), browse(sourceId)];
|
||||
globalRequests.push(...pair); pending.push(...pair);
|
||||
await waitUntilEntered(sourceId, count);
|
||||
}
|
||||
const globalBlocked = await browse('limit-five');
|
||||
assert.equal(globalBlocked.statusCode, 429, globalBlocked.body);
|
||||
assert.equal(globalBlocked.json().error.code, 'RATE_LIMITED');
|
||||
assert.equal(entered.get('limit-five'), undefined);
|
||||
hold = false;
|
||||
queued.shift()!.release();
|
||||
assert.equal((await globalRequests[0]!).statusCode, 200);
|
||||
const globalRecovered = await browse('limit-five');
|
||||
assert.equal(globalRecovered.statusCode, 200, globalRecovered.body);
|
||||
} finally {
|
||||
hold = false;
|
||||
for (const request of queued.splice(0)) request.release();
|
||||
await Promise.allSettled(pending);
|
||||
}
|
||||
});
|
||||
|
||||
test('trusted example plugins can be installed, queried, disabled, reloaded after restart and uninstalled', async (t) => {
|
||||
const h = await harness(); t.after(() => h.close());
|
||||
await mkdir(join(h.dataDir, 'plugins'));
|
||||
await cp(resolve(import.meta.dirname, '../../examples/plugins/demo-chapters'), join(h.dataDir, 'plugins', 'demo-chapters'), { recursive: true });
|
||||
const untrusted = await h.app.inject({ method: 'POST', url: '/api/v1/plugins', headers: auth(h.admin), payload: { folder: 'demo-chapters', trusted: false } });
|
||||
assert.ok(untrusted.statusCode >= 400 && untrusted.statusCode < 500, untrusted.body);
|
||||
const installed = await h.app.inject({ method: 'POST', url: '/api/v1/plugins', headers: auth(h.admin), payload: { folder: 'demo-chapters', trusted: true } });
|
||||
assert.equal(installed.statusCode, 201, installed.body);
|
||||
const pluginId = 'reader.source.demo';
|
||||
const created = await h.app.inject({ method: 'POST', url: '/api/v1/sources', headers: auth(h.admin), payload: {
|
||||
id: 'demo-instance', pluginId, sourceType: 'demo-chapters', name: 'Demo', config: {},
|
||||
} });
|
||||
assert.equal(created.statusCode, 201, created.body);
|
||||
const browse = await h.app.inject({ method: 'GET', url: '/api/v1/sources/demo-instance/browse', headers: auth(h.admin) });
|
||||
assert.equal(browse.statusCode, 200, browse.body);
|
||||
assert.equal(browse.json().items[0].ref, 'demo-book');
|
||||
const acquired = await h.app.inject({ method: 'POST', url: '/api/v1/sources/demo-instance/acquire', headers: auth(h.admin), payload: { entryRef: 'demo-book' } });
|
||||
assert.equal(acquired.statusCode, 200, acquired.body);
|
||||
assert.deepEqual(acquired.json(), { kind: 'chapters', publicationRef: 'demo-book' });
|
||||
const manifest = await h.app.inject({ method: 'GET', url: '/api/v1/sources/demo-instance/publications/demo-book/manifest', headers: auth(h.admin) });
|
||||
assert.equal(manifest.statusCode, 200, manifest.body);
|
||||
const resource = await h.app.inject({ method: 'GET', url: '/api/v1/sources/demo-instance/publications/demo-book/resource?ref=chapter-one', headers: auth(h.admin) });
|
||||
assert.equal(resource.statusCode, 200, resource.body);
|
||||
assert.match(resource.body, /独立 Node 进程/);
|
||||
const disabled = await h.app.inject({ method: 'PATCH', url: `/api/v1/plugins/${pluginId}`, headers: auth(h.admin), payload: { enabled: false } });
|
||||
assert.equal(disabled.statusCode, 200, disabled.body);
|
||||
const unavailable = await h.app.inject({ method: 'GET', url: '/api/v1/sources/demo-instance/browse', headers: auth(h.admin) });
|
||||
assert.equal(unavailable.json().error.code, 'PLUGIN_UNAVAILABLE');
|
||||
await h.restart();
|
||||
const stillUnavailable = await h.app.inject({ method: 'GET', url: '/api/v1/sources/demo-instance/browse', headers: auth(h.admin) });
|
||||
assert.equal(stillUnavailable.json().error.code, 'PLUGIN_UNAVAILABLE');
|
||||
const enabled = await h.app.inject({ method: 'PATCH', url: `/api/v1/plugins/${pluginId}`, headers: auth(h.admin), payload: { enabled: true } });
|
||||
assert.equal(enabled.statusCode, 200, enabled.body);
|
||||
await h.restart();
|
||||
const reloaded = await h.app.inject({ method: 'GET', url: '/api/v1/sources/demo-instance/browse', headers: auth(h.admin) });
|
||||
assert.equal(reloaded.statusCode, 200, reloaded.body);
|
||||
assert.equal(reloaded.json().items[0].ref, 'demo-book');
|
||||
const removed = await h.app.inject({ method: 'DELETE', url: `/api/v1/plugins/${pluginId}`, headers: auth(h.admin) });
|
||||
assert.equal(removed.statusCode, 200, removed.body);
|
||||
const instances = await h.app.inject({ method: 'GET', url: '/api/v1/sources', headers: auth(h.admin) });
|
||||
assert.ok(instances.json().sources.some((source: { id: string }) => source.id === 'demo-instance'), 'uninstall keeps instance configuration');
|
||||
});
|
||||
@@ -124,6 +124,7 @@ before(async () => {
|
||||
|
||||
after(async () => {
|
||||
await app?.close();
|
||||
ctx?.db.close();
|
||||
await new Promise<void>((resolve) => upstream?.close(() => resolve()));
|
||||
await rm(root, { recursive: true, force: true });
|
||||
});
|
||||
@@ -131,6 +132,7 @@ after(async () => {
|
||||
/** Rebuilds the instance so the environment (`TTS_URL`) is read fresh. */
|
||||
beforeEach(async () => {
|
||||
await app?.close();
|
||||
ctx?.db.close();
|
||||
await buildInstance();
|
||||
if (!token) {
|
||||
const res = await app.inject({
|
||||
@@ -146,6 +148,7 @@ beforeEach(async () => {
|
||||
test('an instance with no TTS_URL reports http:false rather than 404', async () => {
|
||||
delete process.env.TTS_URL;
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await buildInstance();
|
||||
|
||||
const res = await app.inject({ method: 'GET', url: '/api/v1/tts/voices', headers: auth() });
|
||||
@@ -160,6 +163,7 @@ test('an instance with no TTS_URL reports http:false rather than 404', async ()
|
||||
test('synthesis is refused with a clear error when TTS_URL is unset', async () => {
|
||||
delete process.env.TTS_URL;
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await buildInstance();
|
||||
|
||||
const res = await app.inject({
|
||||
@@ -173,6 +177,7 @@ test('synthesis is refused with a clear error when TTS_URL is unset', async () =
|
||||
test('a sentence is proxied to the upstream service and streamed back as audio', async () => {
|
||||
process.env.TTS_URL = upstreamUrl;
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await buildInstance();
|
||||
upstreamBehaviour = 'audio';
|
||||
|
||||
@@ -192,6 +197,7 @@ test('a sentence is proxied to the upstream service and streamed back as audio',
|
||||
test('an upstream that answers HTML is refused instead of served as silence', async () => {
|
||||
process.env.TTS_URL = upstreamUrl;
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await buildInstance();
|
||||
upstreamBehaviour = 'html';
|
||||
|
||||
@@ -206,6 +212,7 @@ test('an upstream that answers HTML is refused instead of served as silence', as
|
||||
test('an upstream error is reported as such, not as a 200', async () => {
|
||||
process.env.TTS_URL = upstreamUrl;
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await buildInstance();
|
||||
upstreamBehaviour = 'error';
|
||||
|
||||
@@ -220,6 +227,7 @@ test('an upstream error is reported as such, not as a 200', async () => {
|
||||
test('an over-long utterance is refused before it reaches the upstream', async () => {
|
||||
process.env.TTS_URL = upstreamUrl;
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await buildInstance();
|
||||
upstreamBehaviour = 'audio';
|
||||
lastUpstreamQuery = '';
|
||||
@@ -236,6 +244,7 @@ test('an over-long utterance is refused before it reaches the upstream', async (
|
||||
test('the voice list is passed through from the upstream service', async () => {
|
||||
process.env.TTS_URL = upstreamUrl;
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await buildInstance();
|
||||
|
||||
const res = await app.inject({ method: 'GET', url: '/api/v1/tts/voices', headers: auth() });
|
||||
@@ -248,6 +257,7 @@ test('the voice list is passed through from the upstream service', async () => {
|
||||
test('the audio route requires a token, in the header or the query', async () => {
|
||||
process.env.TTS_URL = upstreamUrl;
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await buildInstance();
|
||||
|
||||
const anonymous = await app.inject({ method: 'GET', url: '/api/v1/tts?text=hello' });
|
||||
|
||||
@@ -215,6 +215,7 @@ before(async () => {
|
||||
|
||||
after(async () => {
|
||||
await app.close();
|
||||
ctx.db.close();
|
||||
await rm(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
|
||||
Reference in new issue
Block a user