合并来自 auto/library-manager-eaa9 的合并请求 #27

feat(library): 支持上传书籍与批量管理

PR-URL: #27
This commit is contained in:
cnb.dQC5J6ohAGA authored and cnb committed 2026-09-17 17:21:42 +08:00
commit 61b8668f13
21 files changed
+2654 -31

No files matched your search

+8
View File
@@ -184,6 +184,10 @@ environment:
- **删除要再确认一次**,且会说明「不进回收站、需要重新扫描」。
- **只读挂载上写操作按钮直接不出现**。`:ro` 是文档里推荐的部署方式,所以「这个页面只能看」
是常态;给一排按下去必然 403 的按钮,只会教读者不信任界面。
- **上传**:直接往当前目录传书。`.zip` / `.cbz` 当合集解开(单层包裹目录抹平,更深的树保留,
因为漫画的卷结构就是深层目录)。重名可选保留两份 / 跳过 / 覆盖 / 整批拒绝。
- **批量管理**:长按多选后可以一次改一批的元数据、一起从书架拿掉、一起移动。
改元数据和「从书架拿掉」都不动磁盘——`remove` 只是 `hidden = 1`,书还在索引里,放得回去。
写操作都做两件事:
@@ -592,6 +596,10 @@ android/app/src/main/java/cool/cnb/reader/
- `POST /api/v1/library/browse/rename` — 改名
- `POST /api/v1/library/browse/mkdir` — 新建文件夹
- `POST /api/v1/library/browse/delete` — 删除(递归)
- `POST /api/v1/library/browse/metadata` — 批量改元数据(路径可以是目录,等于目录里每本书)
- `POST /api/v1/library/browse/shelf` — 批量加入书架 / 从书架拿掉(只动 `user_books`,不动磁盘)
- `POST /api/v1/library/upload` — 上传书籍(multipart;`.zip`/`.cbz` 按合集解开)
- `GET /api/v1/library/upload` — 挂载是否可写,客户端在开始传之前先问
**管理员**
- `POST /api/v1/library/scan`
+84
View File
@@ -361,6 +361,90 @@ pdf / 单图没有独立目录,回落到 items。
只读挂载上任何写操作 → `403 READ_ONLY_MOUNT`。
### `POST /library/upload`
`multipart/form-data`。重复的 `file` 字段各带一个文件,字段 `path` 指定目标目录
(书库内相对路径,缺省根目录),字段 `onConflict` 决定重名怎么办。
```json
{ "uploaded": [
{ "path": "科幻/三体.epub", "originalName": "三体.epub", "name": "三体.epub",
"size": 1234567, "kind": "file",
"bookId": "…", "title": "三体" } ],
"skipped": [ { "name": "坏.zip", "reason": "无法解压(METHOD_UNSUPPORTED)" } ],
"scan": { "added": 1, "updated": 0, "scaned": 1, "removed": 0, "failed": 0,
"startedAt": 1789537864820, "finishedAt": 1789537864831 } }
```
**这是唯一会往 `/books` 里写新字节的端点。** 部署文档要求挂载只读,文件管理器也照此
行事,所以在此之前加一本书只能走 SMB 再等扫描——对运维是合理流程,对家里其他人是不可能
的任务,而自部署书库就是给他们用的。
三条规则让它是个例外而不是个洞:
- **字节完整、且已经落在同一个文件系统上,才会进书库。** 请求体先流到
`DATA_DIR/uploads` 的暂存文件(服务端自己的可写空间),只有写完的文件才 `rename`
进 `BOOKS_DIR`。传到 90% 断线、信号没了、密码错了,都不可能留下半本书,
因为书库里从不出现半本书。
- **名字不是路径。** 客户端送的是文件名,不是路径:斜杠、`..`、控制字符、会让扫描器
看不见的前导点,一律改写或拒绝,再过一遍 `resolveInside`。`C:\Users\me\我的书\三体.epub`
只会取最后一段。
- **写完就索引。** 文件在磁盘上、书架上看不到,正是文件管理器要解释的那种状态,
所以上传以一次增量扫描收尾,而不是等调度器。一次请求扫一次,不是每个文件扫一次。
`onConflict`:
- `rename`(默认)— 保留两份,新的是 `三体 (2).epub`
- `skip` — 保留磁盘上已有的那本,新来的记进 `skipped`
- `overwrite` — 同一个路径,索引看到的是「原位编辑」,书保持身份,进度和笔记都在
- `fail` — 整批 `409 DESTINATION_EXISTS`
`.zip` / `.cbz` 会被当**合集**解开(`kind: "archive-entry"`)。这是读者真正下载到的
东西:四十卷装在一个包里。单层包裹目录会被抹平(`系列/第01卷.epub` 落成 `第01卷.epub`),
更深的树保持原样——`系列/第01卷/001.jpg` 是一本**目录书**,抹平它正好毁掉扫描器要读的结构。
压缩包里的名字一个都不信:每个条目按自己的段重建成名字,`../../etc/passwd`、绝对路径、
控制字符全部落在目标目录里。条目解不开(加密、冷门压缩方法)只记 `skipped`,
不会因为一条坏条目丢掉另外三十九卷。
上传进的每一个字节都复制进了 `DATA_DIR`,请求结束(成功或失败)即删——
一个上传不进去第二次的书库比没有这个功能更糟。
挂载只读 → 在写入任何东西之前就 `403 READ_ONLY_MOUNT`。
`GET /library/upload` 返回 `{ "writable": true }`,让客户端在开始传 400MB 之前先问一句。
### `POST /library/browse/metadata`
```json
{ "paths": ["科幻/合集", "科幻/合集/第三卷.epub"],
"fields": { "author": "某某", "series": "合集", "tags": "科幻,合集" } }
```
→ `{ "applied": 3, "books": ["…"], "failed": [ { "path": "空目录", "reason": "NOT_A_BOOK" } ] }`
一次改一批。手动覆盖层是按**书**存的,而书库是按批整理的:四十个「佚名」要的是
一次填同一个作者,不是四十个对话框。
`paths` 里给目录就等于给目录里扫描出来的每一本书:系列文件夹本身没有 `book_files` 行,
问它自己会什么都找不到。`fields` 与单本 `/books/:id/metadata` 同一份白名单——
批量不是写那些接口从不暴露的列的许可证。
不是书的路径记进 `failed` 而不是整批失败:选目录时捎上一个 `.nfo` 很常见,
界面因此可以说「已更新 12 本,跳过 2 项」。
### `POST /library/browse/shelf`
```json
{ "paths": ["旧扫描"], "action": "remove" }
```
→ 同 `metadata` 的结构。`action` 取 `add` / `remove` / `hide` / `unhide`。
**磁盘上什么都不会变**,这正是重点:「把这四十份扫描从书架上拿掉」和
「把这四十份扫描移进某个目录」在一行行列表里只差一个词,后果却完全不同,
所以是两个端点。`remove` 写的是 `hidden = 1` 而不是删行——书还在书库里、还在索引里,
读者放得回去;删行在书架上长得一样,却不可逆。
### `GET /books/:id/file?path=<rel_path>`
流式返回这本书的**某一个**文件(二进制流,不是 JSON)。
+21 -2
View File
@@ -1,4 +1,5 @@
import Fastify, { type FastifyInstance } from 'fastify';
import multipart from '@fastify/multipart';
import type { AppContext } from './context.ts';
import { registerErrorHandler } from './errors.ts';
import { registerAuthRoutes } from './routes/auth.ts';
@@ -11,14 +12,32 @@ import { isOriginAllowed, resolveCorsOrigin } from './cors.ts';
export function buildApp(ctx: AppContext): FastifyInstance {
const app = Fastify({
logger: { level: ctx.config.logLevel },
// Book bodies are streamed straight from disk, so the default 1MB body
// limit is fine and protects the server from oversized sync payloads.
// Book bodies are streamed straight from disk, so a JSON limit this small is
// fine and protects the server from oversized sync payloads. Uploads are the
// exception and are not bounded here: a multipart body is streamed by the
// multipart plugin rather than buffered by Fastify, and its own `fileSize`
// limit is what bounds a request. Raising this number would instead let one
// JSON request (a sync batch, a batch metadata edit) hold 400MB of heap.
bodyLimit: 8 * 1024 * 1024,
trustProxy: true,
});
registerErrorHandler(app);
// Uploads arrive as `multipart/form-data`, one part per file, because that is
// the only encoding a browser and an Android picker both produce without a
// helper library. `attachFieldsToBody` is deliberately off: the handler wants
// the file *stream*, not a buffer, so that a 400MB comic does not have to fit
// in the process's memory before it can be stored.
//
// The per-file limit is generous (a scanned volume is genuinely large) and the
// per-request limit bounds a batch, so a client cannot turn one request into
// an unbounded amount of disk.
app.register(multipart, {
limits: { fileSize: 4 * 1024 * 1024 * 1024, files: 20 },
throwFileSizeLimit: true,
});
app.addHook('onRequest', async (request, reply) => {
if (!isOriginAllowed(ctx.config, request)) {
reply.status(403).send({ error: { code: 'FORBIDDEN', message: 'origin not allowed' } });
+12
View File
@@ -7,6 +7,7 @@ import type { ShelfService } from '../services/shelf.ts';
import type { SyncService } from '../services/sync.ts';
import type { TtsService } from '../services/tts.ts';
import type { BrowseService } from '../services/browse.ts';
import type { UploadService } from '../services/uploads.ts';
export interface AppContext {
config: AppConfig;
@@ -24,6 +25,17 @@ export interface AppContext {
* part of a book's DTO anywhere in the API.
*/
browse: BrowseService;
/**
* Uploads into the library, and the incremental scan that follows.
*
* Held apart from `browse` because the two disagree on the one thing that
* matters: `browse` never writes unless the deployment allows it, and an
* upload *is* the write the reader asked for. Keeping them separate keeps the
* file manager's "this mount is read-only, so here is a screen with no write
* buttons" answer from being weakened by the one feature whose whole purpose
* is to write.
*/
uploads: UploadService;
log: FastifyBaseLogger;
}
+137 -1
View File
@@ -1,7 +1,8 @@
import type { FastifyInstance, FastifyReply } from 'fastify';
import type { FastifyInstance } from 'fastify';
import type { AppContext } from '../context.ts';
import { authenticate, currentUser, requireAdmin } from '../auth.ts';
import { badRequest, notFound } from '../../lib/errors.ts';
import { parseConflictPolicy, type ConflictPolicy, type StagedUpload } from '../../services/uploads.ts';
import { assertSafeRel, normalizeRel, resolveInside } from '../../lib/paths.ts';
import { createHash } from 'node:crypto';
import { createReadStream, existsSync } from 'node:fs';
@@ -550,6 +551,141 @@ export function registerLibraryRoutes(app: FastifyInstance, ctx: AppContext): vo
return ctx.browse.remove(stringList(body.paths, 'paths'));
});
// ---- batch management ----
/**
* The metadata fields a batch may set.
*
* The same allowlist the single-book route uses, and for the same reason: a
* batch is not a licence to write columns the API never exposes. It also
* matches the client's form, so "edit 40 books" cannot offer a field that the
* single-book dialog does not.
*/
const BATCH_FIELDS = [
'title', 'author', 'publisher', 'language', 'isbn', 'description',
'series', 'seriesIndex', 'pubdate', 'tags',
];
app.post('/api/v1/library/browse/metadata', { preHandler: auth }, async (request) => {
const user = currentUser(request);
const body = (request.body ?? {}) as { paths?: unknown; fields?: unknown };
const paths = stringList(body.paths, 'paths');
const fields = (body.fields ?? {}) as Record<string, unknown>;
if (typeof fields !== 'object' || Array.isArray(fields)) {
throw badRequest('fields must be an object');
}
const patch: Record<string, unknown> = {};
for (const key of BATCH_FIELDS) {
if (fields[key] !== undefined) patch[key] = fields[key];
}
if (Object.keys(patch).length === 0) throw badRequest('no editable field supplied');
return ctx.browse.batchMetadata(paths, patch, user.id);
});
/**
* Adds, removes or hides books on the *caller's* shelf.
*
* Nothing on disk changes, and that is the point: "hide these forty scans from
* my shelf" and "move these forty scans into a folder" are one word apart in a
* list of rows and could not be more different in consequence.
*/
app.post('/api/v1/library/browse/shelf', { preHandler: auth }, async (request) => {
const user = currentUser(request);
const body = (request.body ?? {}) as { paths?: unknown; action?: unknown };
const paths = stringList(body.paths, 'paths');
const action = body.action;
if (action !== 'add' && action !== 'remove' && action !== 'hide' && action !== 'unhide') {
throw badRequest('action must be one of add, remove, hide, unhide', 'BAD_ACTION');
}
return ctx.browse.batchShelf(paths, action, user.id);
});
// ---- uploads ----
/**
* Stores uploaded books in the library and indexes them.
*
* `multipart/form-data` with repeatable `file` parts and an optional `path`
* field naming the destination directory. The response reports what landed,
* what was skipped and why, and what the incremental scan saw — the same shape
* a scan reports, so the client has one story to tell about a library change
* rather than two.
*/
app.post('/api/v1/library/upload', { preHandler: auth }, async (request) => {
const user = currentUser(request);
if (!request.isMultipart()) {
throw badRequest('expected multipart/form-data', 'NOT_MULTIPART');
}
const staged: StagedUpload[] = [];
let target = '';
let policy: ConflictPolicy = 'rename';
let targetChecked = false;
// Refused before a single byte is read, rather than reported per file. A
// destination that escapes the library is not a file's problem, and a
// per-file `skipped` note would make a client show "已跳过" for something that
// should never have been attempted.
const ensureTarget = (raw: string): void => {
if (targetChecked) return;
targetChecked = true;
target = ctx.browse.assertDestination(raw);
};
// Every part is drained before any of it is placed, for two reasons that are
// both about correctness rather than tidiness:
//
// - A form built by a browser puts its fields first, but nothing guarantees
// that of a hand-rolled client, and the destination has to be known before
// the first file can be stored.
// - A batch that is refused should be refused whole. Committing file one
// before file two has been received makes "the upload failed" a lie.
//
// Parking is safe because a staged part is a *file* in DATA_DIR, not a
// buffered stream: the multipart plugin will not hand out the next part until
// this one has been consumed, and a NAS upload must not become heap.
try {
for await (const part of request.parts()) {
if (part.type === 'field') {
if (part.fieldname === 'path') ensureTarget(String(part.value ?? ''));
if (part.fieldname === 'onConflict') policy = parseConflictPolicy(String(part.value ?? ''));
continue;
}
// Registered before it is read: a part that fails halfway has still
// created its scratch directory, and not leaking that is the subject of
// the `catch` around this whole loop.
const item = await ctx.uploads.stage(part.file, part.filename ?? '');
staged.push(item);
// Consumed here rather than later: the request body is a stream, and a
// part that is not read blocks every part behind it.
await ctx.uploads.receive(item);
}
} catch (err) {
// Nothing reached the library, so the only cleanup owed is the scratch
// copies — which `commit` would have made, and never got to run.
await ctx.uploads.discard(staged);
throw err;
}
ensureTarget(target);
if (staged.length === 0) throw badRequest('no file was uploaded', 'NO_FILES');
void user;
return ctx.uploads.commit(staged, target, policy);
});
/**
* A one-off write probe for the upload screen.
*
* `browse` already reports whether the mount is writable per directory, and
* this is the same answer for the *root* — asked before a phone starts
* uploading 400MB it cannot store. Not cached: a mount is remounted far more
* often than this is called.
*/
app.get('/api/v1/library/upload', { preHandler: auth }, async () => {
const listing = await ctx.browse.list('');
return { writable: listing.writable };
});
// ---- instance administration ----
app.post('/api/v1/library/scan', { preHandler: auth }, async (request) => {
+3 -1
View File
@@ -6,6 +6,7 @@ import { ShelfService } from './services/shelf.ts';
import { SyncService } from './services/sync.ts';
import { TtsService } from './services/tts.ts';
import { BrowseService } from './services/browse.ts';
import { UploadService } from './services/uploads.ts';
import { buildApp } from './http/app.ts';
import { setLogger } from './lib/log.ts';
import type { AppContext } from './http/context.ts';
@@ -48,7 +49,8 @@ async function main(): Promise<void> {
ctx.shelf = new ShelfService(db);
ctx.sync = new SyncService(db);
ctx.tts = new TtsService(config);
ctx.browse = new BrowseService(db, config);
ctx.browse = new BrowseService(db, config, ctx.shelf);
ctx.uploads = new UploadService(db, config, ctx.browse, ctx.scanner);
app.log.info({ booksDir: config.booksDir, dataDir: config.dataDir }, 'starting reader server');
+153
View File
@@ -87,10 +87,38 @@ export interface BrowseMoveResult {
target: string;
}
export interface BatchResult {
/** Rows changed; a folder counts once per book inside it. */
applied: number;
/** Distinct book ids touched, so the client can refresh exactly those. */
books: string[];
/** Paths that could not be acted on, with the reason. */
failed: Array<{ path: string; reason: string }>;
}
function emptyBatch(): BatchResult {
return { applied: 0, books: [], failed: [] };
}
/** What a batch operation needs from the shelf: the override layer. */
export interface MetadataWriter {
setOverrides(bookId: string, patch: Record<string, unknown>, userId: string): void;
}
export class BrowseService {
constructor(
private readonly db: Db,
private readonly config: AppConfig,
/**
* The manual-override layer, injected rather than imported.
*
* Batch metadata edits are the one thing here that is about *books* rather
* than files, and the override rules (which fields are editable, that a
* manual value is never auto-overwritten) live in the shelf service. Taking
* it as a collaborator keeps one implementation of them; reaching into the
* table directly would be a second.
*/
private readonly metadata: MetadataWriter,
) {}
// ---- reads ----
@@ -302,6 +330,131 @@ export class BrowseService {
return { removed };
}
/**
* Validates a destination directory for an upload, without touching the disk.
*
* Split out from `list`/`move` because an upload's destination arrives as a
* form *field* before any bytes do, and the answer has to be a refusal rather
* than a per-file "skipped" note: a path that escapes the library is a bad
* request, not a bad file.
*/
assertDestination(input: string): string {
if (input === '') return '';
const rel = assertSafeRel(input);
return rel;
}
// ---- batch management ----
/**
* Applies one metadata patch to many paths at once.
*
* Written as a batch because the manual-override layer is per *book*, and a
* library is organised in batches: forty volumes that arrived as `佚名` want
* one author between them, not forty dialogs. The loop lives here rather than
* in the client for the reason every other write does — the mapping from path
* to book id is the server's, and a client that guessed it would be a second
* implementation of the index.
*
* A path that is not a book is reported, not refused: selecting a folder and a
* stray `.nfo` alongside the books is ordinary, and the caller can say "已更新
* 12 本,跳过 2 项" instead of failing the whole operation.
*/
batchMetadata(paths: string[], fields: Record<string, unknown>, userId: string): BatchResult {
if (paths.length === 0) throw badRequest('no paths given', 'NO_PATHS');
const result = emptyBatch();
for (const input of paths) {
const relPath = assertSafeRel(input);
const bookIds = this.booksUnder(relPath);
if (bookIds.length === 0) {
result.failed.push({ path: relPath, reason: 'NOT_A_BOOK' });
continue;
}
for (const bookId of bookIds) {
this.metadata.setOverrides(bookId, fields, userId);
result.applied += 1;
result.books.push(bookId);
}
}
result.books = [...new Set(result.books)];
return result;
}
/**
* Adds or removes many paths on the caller's shelf.
*
* Distinct from the file manager's own operations in the way that matters: it
* changes nothing on disk. "Hide these forty scans from my shelf" and "move
* these forty scans into a folder" look similar in a list of rows and could not
* be more different in consequence, so they are different endpoints.
*/
batchShelf(paths: string[], action: 'add' | 'remove' | 'hide' | 'unhide', userId: string): BatchResult {
if (paths.length === 0) throw badRequest('no paths given', 'NO_PATHS');
const result = emptyBatch();
const now = Date.now();
for (const input of paths) {
const relPath = assertSafeRel(input);
const bookIds = this.booksUnder(relPath);
if (bookIds.length === 0) {
result.failed.push({ path: relPath, reason: 'NOT_A_BOOK' });
continue;
}
for (const bookId of bookIds) {
this.setShelfState(bookId, userId, action, now);
result.applied += 1;
result.books.push(bookId);
}
}
result.books = [...new Set(result.books)];
return result;
}
/**
* The book ids a library path stands for.
*
* A file path is one book. A *folder* stands for everything the scanner
* indexed inside it, which is what makes "fix the metadata of this series"
* one action: the series folder holds forty volume files and no rows of its
* own, so asking for the folder's own row would find nothing.
*/
private booksUnder(relPath: string): string[] {
const own = this.fileRow(relPath);
if (own) return [own.book_id];
const rows = this.db.all<{ book_id: string }>(
'SELECT DISTINCT book_id FROM book_files WHERE rel_path LIKE ?',
`${escapeLike(relPath)}/%`,
);
return rows.map((row) => row.book_id);
}
private setShelfState(bookId: string, userId: string, action: 'add' | 'remove' | 'hide' | 'unhide', now: number): void {
if (action === 'add') {
this.db.run(
'INSERT OR IGNORE INTO user_books (user_id, book_id, added_at) VALUES (?,?,?)',
userId, bookId, now,
);
return;
}
if (action === 'remove') {
// `hidden` rather than a delete: the book stays in the library and stays
// indexed, so the reader can put it back. Deleting the row would look
// identical on the shelf and be irreversible.
this.db.run('UPDATE user_books SET hidden = 1 WHERE user_id = ? AND book_id = ?', userId, bookId);
this.db.run(
'INSERT OR IGNORE INTO user_books (user_id, book_id, added_at, hidden) VALUES (?,?,?,1)',
userId, bookId, now,
);
return;
}
// unhide: an entry that was never added is left alone rather than created —
// "put it back" cannot resurrect a shelf entry that never existed.
this.db.run(
`UPDATE user_books SET hidden = 0 WHERE user_id = ? AND book_id = ?`,
userId, bookId,
);
void action;
}
/**
* Turns an `EPERM`/`EROFS`/`EACCES` from the filesystem into a clean 403.
*
+688
View File
@@ -0,0 +1,688 @@
import { createWriteStream } from 'node:fs';
import { mkdir, mkdtemp, readdir, rename, rm, stat, writeFile } from 'node:fs/promises';
import { dirname, join } from 'node:path';
import { pipeline } from 'node:stream/promises';
import type { Readable } from 'node:stream';
import type { Db } from '../db/index.ts';
import type { AppConfig } from '../config/index.ts';
import type { BrowseService } from './browse.ts';
import { resolveInside } from '../lib/paths.ts';
import { badRequest, conflict, forbidden, notFound } from '../lib/errors.ts';
import { ZipArchive, ZipError } from '../indexer/formats/zip-reader.ts';
import type { Scanner, ScanResult } from '../indexer/scanner.ts';
/**
* Uploading books into the library — the one thing this product's deployment
* model makes awkward.
*
* The design mounts `BOOKS_DIR` **read-only** and the file-manager screen
* follows that rule, so until now the only way to add a book was to reach the
* NAS over SMB and then wait for a scan. That is a reasonable workflow for the
* operator and an impossible one for every other member of the household, which
* is exactly who a self-hosted library is for.
*
* So this is the single deliberate exception to "the server never writes into
* the library", and it is written around three rules that keep it an exception
* rather than a hole:
*
* 1. **Nothing is written to the library until its bytes are complete and
* already on the same filesystem.** The stream goes to a scratch file in
* `DATA_DIR/uploads`, which is the server's own writable space, and only a
* finished file is `rename`d into `BOOKS_DIR`. A request that dies at 90%,
* a client that runs out of signal, a wrong password — none of them can
* leave half a book behind, because a half book never exists inside the
* library at all.
* 2. **The name is not the path.** A client sends a file name, never a path:
* slashes, `..`, control characters and the leading dots that would hide a
* file from the scanner are all rewritten or refused. `resolveInside` is
* still applied, because that is the boundary and it does not get to have
* exceptions.
* 3. **The library is left indexed, not just written.** A file that is on disk
* but not in the index is the exact state the file-manager screen exists to
* explain, so an upload finishes with an incremental scan of the paths it
* touched rather than waiting for the scheduler.
*/
/** How deep an imported archive's directory tree may be. */
const MAX_ARCHIVE_DEPTH = 12;
/** How many entries one imported archive may contribute. */
const MAX_ARCHIVE_ENTRIES = 5000;
/**
* Per-entry ceiling for an imported archive.
*
* A zip carried in over HTTP is untrusted input and the entry sizes in its
* central directory are the *author's* claim, not a fact. Bounding each entry
* keeps the scratch copy from growing past what the disk can take before the
* import notices, and `read()` enforces the same bound while inflating.
*/
const MAX_ARCHIVE_ENTRY_BYTES = 4 * 1024 * 1024 * 1024;
/**
* One part received but not yet placed.
*
* Exists because the destination directory and the conflict policy arrive as
* form fields that a client is free to send after the file, so a part cannot be
* placed while it is still being read. The bytes go to scratch — `DATA_DIR`,
* never the library — and the decisions happen once the whole form is in hand.
*/
export interface StagedUpload {
/** Scratch directory owned by this part, removed when the request ends. */
dir: string;
/** Where the bytes are being written. */
stagedPath: string;
/** Sanitised destination name. */
name: string;
/** Name exactly as the client sent it, so a report can name what the user saw. */
originalName: string;
stream: Readable;
/** Byte count, filled in once the part has been written. */
size?: number;
}
export interface UploadResultItem {
/** Library-relative path the file now lives at. */
path: string;
originalName: string;
/** Name after the conflict policy was applied; differs when a suffix was added. */
name: string;
size: number;
/** `'file'` for the uploaded file itself, `'archive-entry'` for a zip member. */
kind: 'file' | 'archive-entry';
/** Book id once indexed. Absent when the file is not a format the server reads. */
bookId?: string | undefined;
title?: string | undefined;
}
export interface UploadResult {
uploaded: UploadResultItem[];
/** Files that were not stored, with the reason, rather than failing the batch. */
skipped: Array<{ name: string; reason: string }>;
scan: ScanResult;
}
export interface UploadInput {
/** Original filename, exactly as the client sent it. */
name: string;
/** Bytes, streamed. Never buffered whole. */
stream: Readable;
/** Directory the client asked for. Validated here, not by the caller. */
target: string;
/**
* What to do when the destination name is taken.
*
* - `rename` (default): keep both, with `书名 (2).epub`.
* - `skip`: keep the one already on disk and report this one as skipped.
* - `overwrite`: replace it — the same path, so the index sees an in-place
* edit and the book keeps its identity.
* - `fail`: refuse the whole request with `409 DESTINATION_EXISTS`.
*/
onConflict?: ConflictPolicy;
}
export type ConflictPolicy = 'rename' | 'skip' | 'overwrite' | 'fail';
const CONFLICT_POLICIES: readonly ConflictPolicy[] = ['rename', 'skip', 'overwrite', 'fail'];
export function parseConflictPolicy(value: unknown): ConflictPolicy {
if (value === undefined || value === null || value === '') return 'rename';
if (typeof value !== 'string' || !CONFLICT_POLICIES.includes(value as ConflictPolicy)) {
throw badRequest(`onConflict must be one of ${CONFLICT_POLICIES.join(', ')}`, 'BAD_CONFLICT_POLICY');
}
return value as ConflictPolicy;
}
export class UploadService {
constructor(
private readonly db: Db,
private readonly config: AppConfig,
private readonly browse: BrowseService,
private readonly scanner: Scanner,
) {}
/** `DATA_DIR/uploads`, created on demand. Never inside the library. */
private scratchRoot(): string {
return join(this.config.dataDir, 'uploads');
}
/**
* Receives one part and parks it in scratch.
*
* The *only* thing that happens while a request body is being read. Every
* decision — where it lands, what it is called, whether it is a zip — happens
* afterwards, in `commit`, because those decisions depend on form fields that
* may arrive after the file and on names that must all be validated before any
* of them is honoured.
*/
async stage(stream: Readable, filename: string): Promise<StagedUpload> {
const name = sanitizeUploadName(filename);
await mkdir(this.scratchRoot(), { recursive: true });
const dir = await mkdtemp(join(this.scratchRoot(), 'up-'));
return { dir, stagedPath: join(dir, 'part'), name, originalName: filename, stream };
}
/**
* Writes a staged part's bytes, then places and indexes every staged file.
*
* Split from `stage` so the caller can drain the whole request first: a field
* that names the destination directory may arrive after the file, and a client
* that sends two files must not have the first one committed before the second
* one has proved it can be received.
*/
async receive(staged: StagedUpload): Promise<void> {
await this.write(staged.stream, staged.stagedPath);
staged.size = (await stat(staged.stagedPath)).size;
if (staged.size === 0) throw badRequest('the uploaded file is empty', 'EMPTY_UPLOAD');
}
/**
* Places everything staged so far, then indexes once.
*
* One scan for the whole request rather than one per file: a forty-volume
* series is one library change, and running the scanner forty-one times would
* make the last file's response depend on the queue of the first.
*
* A file whose *content* is unusable (an archive entry that cannot be
* inflated, a name that is taken under `skip`) is reported in `skipped`,
* because losing thirty-nine good volumes to one bad entry is the failure this
* feature would be judged on. A request that is wrong — a destination outside
* the library, a read-only mount — throws, and it throws before anything is
* written into the library.
*/
async commit(staged: StagedUpload[], target: string, policy: ConflictPolicy): Promise<UploadResult> {
if (staged.length === 0) throw badRequest('no file was uploaded', 'NO_FILES');
const resolved = this.resolveTarget(target);
await this.guardWritable(resolved);
// Everything that can be known without moving a byte is checked first.
//
// `fail` is a client saying "do not guess", and the only honest answer to a
// batch is to apply none of it: a request that stores three of five files
// and then refuses has left a library the user has to reason about, and no
// response can describe it. Later policies (`rename`, `skip`) resolve
// conflicts rather than refusing, so they never reach this check.
if (policy === 'fail') await this.assertNoConflicts(staged, resolved);
const uploaded: UploadResultItem[] = [];
const skipped: Array<{ name: string; reason: string }> = [];
const written: string[] = [];
try {
for (const item of staged) {
if (isZipContainer(item.name)) {
await this.importArchive(item.stagedPath, item.name, resolved, policy, uploaded, skipped, written);
} else {
const placed = await this.place(item.stagedPath, resolved, item.name, item.originalName, policy);
if (placed) {
uploaded.push(placed);
written.push(placed.path);
} else {
skipped.push({ name: item.originalName, reason: '同名文件已存在' });
}
}
}
const scan = await this.scanner.scan();
this.attachBookIds(uploaded);
return { uploaded, skipped, scan };
} catch (err) {
// A failure partway through placing still has to leave nothing behind.
// The library is the one place in this product where a half-finished
// write is not recoverable by trying again, so the rollback is explicit
// rather than left to the caller.
await this.rollback(written);
throw err;
} finally {
// The scratch copy holds one full copy of every byte received, so it goes
// even when the request failed: a failed upload must cost the user nothing
// but the attempt.
await Promise.all(staged.map((item) => rm(item.dir, { recursive: true, force: true }).catch(() => undefined)));
}
}
/**
* Removes files this request wrote, deepest first.
*
* Only ever called when the request failed, so the cost of getting it slightly
* wrong is confined to a failure path. Directories are left alone: the target
* may have existed before, and a rollback that removed a directory the user
* already had would be worse than the failure it was cleaning up.
*/
private async rollback(paths: string[]): Promise<void> {
for (const relPath of [...paths].reverse()) {
const abs = resolveInside(this.config.booksDir, relPath);
await rm(abs, { force: true }).catch(() => undefined);
}
}
/**
* Whether any staged file would land on a name that is already taken.
*
* Asked per member rather than per archive: an archive is not one destination,
* and a client uploading `系列.zip` on top of an existing `系列/` deserves the
* same answer as one uploading the files directly.
*/
private async assertNoConflicts(staged: StagedUpload[], target: string): Promise<void> {
for (const item of staged) {
if (!isZipContainer(item.name)) {
if (await this.existingName(target, item.name)) {
throw conflict(`${target === '' ? '' : `${target}/`}${item.name} is already taken`, 'DESTINATION_EXISTS');
}
continue;
}
// An archive's destination names are only knowable by opening it, so the
// check happens inside the import rather than here; the archive itself
// being refused whole is `importArchive`'s job.
await this.assertArchiveHasFreeNames(item.stagedPath, item.name, target);
}
}
/** The archive's own names, checked against the library before anything moves. */
private async assertArchiveHasFreeNames(stagedPath: string, archiveName: string, target: string): Promise<void> {
let archive: ZipArchive;
try {
archive = await ZipArchive.open(stagedPath);
} catch (err) {
throw badRequest(
`cannot read ${archiveName} as an archive: ${err instanceof Error ? err.message : String(err)}`,
'INVALID_ARCHIVE',
);
}
const entries = archive.files().filter((entry) => !entry.isDirectory);
const firstSegments = segmentsOf(entries[0]?.name ?? '');
const flattenDepth = firstSegments.length > 1 ? 1 : 0;
for (const entry of entries) {
const segments = sanitizeArchiveSegments(segmentsOf(entry.name));
const relative = segments.slice(flattenDepth);
if (relative.length === 0) continue;
const subdir = relative.slice(0, -1).join('/');
const dir = subdir === '' ? target : target === '' ? subdir : `${target}/${subdir}`;
if (await this.existingName(dir, relative[relative.length - 1]!)) {
throw conflict(`${entry.name} is already taken in the library`, 'DESTINATION_EXISTS');
}
}
}
/** Removes scratch copies without placing anything. Used when a request dies. */
async discard(staged: StagedUpload[]): Promise<void> {
await Promise.all(staged.map((item) => rm(item.dir, { recursive: true, force: true }).catch(() => undefined)));
}
/**
* Store one uploaded file end to end: stage, receive, commit.
*
* Kept as one call for the in-process callers (and the tests) that have no
* form fields to wait for.
*/
async store(input: UploadInput): Promise<UploadResult> {
const staged = await this.stage(input.stream, input.name);
try {
await this.receive(staged);
return await this.commit([staged], input.target, input.onConflict ?? 'rename');
} catch (err) {
// `commit` cleans up once it has been entered; a failure before that
// (an empty body, an unreadable stream) has to clean up here.
await this.discard([staged]);
throw err;
}
}
/** Exposed for tests: no scratch directory may outlive a request. */
scratchDirectory(): string {
return this.scratchRoot();
}
// ---- destination rules ----
private resolveTarget(input: string): string {
if (input === '') return '';
const rel = this.browse.assertDestination(input);
// `..`, an absolute path and a trailing slash all resolve through the same
// check the file-manager endpoints use; there is one boundary and this is
// not a second implementation of it.
return resolveInside(this.config.booksDir, rel) === this.config.booksDir ? '' : rel;
}
private async guardWritable(target: string): Promise<void> {
const abs = resolveInside(this.config.booksDir, target);
const info = await stat(abs).catch(() => null);
if (!info) throw notFound('no such directory', 'DIR_NOT_FOUND');
if (!info.isDirectory()) throw badRequest('target is not a directory', 'NOT_A_DIRECTORY');
// The probe answers "can this mount be written at all", which is the only
// question that produces a clean 403 on a `:ro` bind mount instead of an
// `EROFS` from six layers down.
if (!(await this.browse.writable(abs))) {
throw forbidden(
'the library mount is read-only; remount it without `:ro` to upload into it',
'READ_ONLY_MOUNT',
);
}
}
/**
* Move a finished file into the library under a free name.
*
* `rename` rather than copy, and both paths on the same filesystem, so a file
* appears in the library already complete: a reader whose library is being
* watched by a scanner must never see a partially written book, and a
* same-filesystem rename is atomic where a `write()` into the library is not.
*
* `subdir` is how an archive member keeps the structure it arrived in. A comic
* stored as `系列/第01卷/001.jpg` is a *directory book* to the scanner, and
* flattening it would destroy exactly the layout that makes it readable — so
* the directories are recreated and each member lands where the archive said it
* belongs, bounded by the sanitised segment list the caller already built.
*/
private async place(
stagedPath: string,
target: string,
name: string,
originalName: string,
policy: ConflictPolicy,
kind: 'file' | 'archive-entry' = 'file',
subdir = '',
): Promise<UploadResultItem | null> {
const dir = subdir === '' ? target : target === '' ? subdir : `${target}/${subdir}`;
const existing = await this.existingName(dir, name);
let finalName = name;
if (existing) {
if (policy === 'fail') {
throw conflict(`${dir === '' ? '' : `${dir}/`}${name} is already taken`, 'DESTINATION_EXISTS');
}
if (policy === 'skip') return null;
if (policy === 'rename') finalName = await this.freeName(dir, name);
}
const relPath = dir === '' ? finalName : `${dir}/${finalName}`;
const abs = resolveInside(this.config.booksDir, relPath);
await mkdir(dirname(abs), { recursive: true });
await rename(stagedPath, abs);
const info = await stat(abs);
return { path: relPath, originalName, name: finalName, size: info.size, kind };
}
/** The name already at `dir/name`, or `null`. Compared case-insensitively. */
private async existingName(dir: string, name: string): Promise<string | null> {
const abs = resolveInside(this.config.booksDir, dir === '' ? name : `${dir}/${name}`);
const info = await stat(abs).catch(() => null);
if (info) return name;
// A case-insensitive filesystem (macOS, SMB shares) would make `三体.EPUB`
// and `三体.epub` one file. Listing the directory rather than guessing at the
// filesystem's rules keeps the answer the same as what the user will see.
const entries = await readdir(resolveInside(this.config.booksDir, dir)).catch(() => []);
const lower = name.toLowerCase();
return entries.find((entry) => entry.toLowerCase() === lower) ?? null;
}
/** `三体.epub` -> `三体 (2).epub`, the first suffix that is free. */
private async freeName(dir: string, name: string): Promise<string> {
const dot = name.lastIndexOf('.');
const stem = dot > 0 ? name.slice(0, dot) : name;
const ext = dot > 0 ? name.slice(dot) : '';
for (let index = 2; index < 1000; index += 1) {
const candidate = `${stem} (${index})${ext}`;
if (!(await this.existingName(dir, candidate))) return candidate;
}
throw conflict('too many files with that name', 'DESTINATION_EXISTS');
}
// ---- transport ----
/**
* Stream the request body to a scratch file.
*
* Streamed rather than buffered because the thing being uploaded is routinely
* a scanned PDF or a comic volume of several hundred megabytes, and holding
* one in the server's heap would make "upload a book" an off switch for a
* small NAS.
*/
private async write(stream: Readable, destination: string): Promise<void> {
await mkdir(dirname(destination), { recursive: true });
try {
await pipeline(stream, createWriteStream(destination));
} catch (err) {
// A part that fails to arrive in full is a client-side failure, not a
// server error. The stream's own error is passed through unchanged when it
// is not one of the shapes a broken connection takes, so a genuine server
// fault is still a 500 rather than being dressed up as the client's.
const code = (err as NodeJS.ErrnoException).code;
if (
code === 'ECONNRESET'
|| code === 'ERR_STREAM_PREMATURE_CLOSE'
|| code === 'EPIPE'
|| code === 'ERR_STREAM_DESTROYED'
|| !code
) {
throw badRequest(
`the upload was interrupted before it finished: ${err instanceof Error ? err.message : String(err)}`,
'UPLOAD_INCOMPLETE',
);
}
throw err;
}
}
// ---- archives ----
/**
* Unpack a zip-like container into the target directory.
*
* The whole point of accepting a zip is a series that arrived as one file: a
* folder of forty volumes is what a reader actually downloads, and asking them
* to unpack it first and upload forty files, one at a time, over a phone, is
* not a feature.
*
* What this deliberately does **not** do is trust anything inside the archive.
* Every entry name is rebuilt from its own segments (`sanitizeUploadName` per
* segment, depth bounded), so `../../etc/passwd` and an absolute path and a
* name made of control characters all resolve to something inside the target
* directory. That is a stronger guarantee than checking for `..`: it cannot be
* defeated by an encoding the check did not anticipate.
*
* Only one book-ish level of nesting is flattened. Deeper trees are preserved,
* because a comic stored as `系列/第01卷/001.jpg` is a *directory book* and
* flattening it would destroy the very structure the scanner reads.
*/
private async importArchive(
stagedPath: string,
archiveName: string,
target: string,
policy: ConflictPolicy,
uploaded: UploadResultItem[],
skipped: Array<{ name: string; reason: string }>,
written: string[],
): Promise<void> {
let archive: ZipArchive;
try {
archive = await ZipArchive.open(stagedPath);
} catch (err) {
throw badRequest(
`cannot read ${archiveName} as an archive: ${err instanceof Error ? err.message : String(err)}`,
'INVALID_ARCHIVE',
);
}
const entries = archive.files().filter((entry) => !entry.isDirectory);
if (entries.length === 0) throw badRequest(`${archiveName} contains no files`, 'EMPTY_ARCHIVE');
if (entries.length > MAX_ARCHIVE_ENTRIES) {
throw badRequest(`${archiveName} has too many entries (${entries.length})`, 'ARCHIVE_TOO_LARGE');
}
// The shallowest entry decides the flattening: a zip of one folder is the
// normal shape and that folder is the series the user is uploading, while a
// zip of loose files has nothing to flatten. Anything deeper keeps its
// structure, because a nested tree is what a comic volume layout *is*.
const firstSegments = segmentsOf(entries[0]!.name);
const flattenDepth = firstSegments.length > 1 ? 1 : 0;
const scratch = join(dirname(stagedPath), 'unpacked');
for (const entry of entries) {
if (entry.isDirectory) continue;
if (entry.uncompressedSize > MAX_ARCHIVE_ENTRY_BYTES) {
skipped.push({ name: entry.name, reason: '压缩包内条目过大' });
continue;
}
const segments = sanitizeArchiveSegments(segmentsOf(entry.name));
if (segments.length === 0) continue;
const relative = segments.slice(flattenDepth);
if (relative.length === 0) continue;
if (relative.length > MAX_ARCHIVE_DEPTH) {
skipped.push({ name: entry.name, reason: '目录嵌套过深' });
continue;
}
const destination = join(scratch, ...relative);
try {
await mkdir(dirname(destination), { recursive: true });
await writeFile(destination, await archive.read(entry.name));
} catch (err) {
// One unreadable entry in a real-world archive is common (an encrypted
// file, a method nobody uses). It must not lose the other thirty-nine.
skipped.push({
name: entry.name,
reason: err instanceof ZipError ? `无法解压(${err.code})` : '无法解压',
});
continue;
}
const leaf = relative[relative.length - 1]!;
const subdir = relative.slice(0, -1).join('/');
// The member lands in the directory it arrived in, and `place` reports the
// path it really took — renaming within that directory when the name was
// taken. Reporting `target/leaf` instead is what turned the second volume
// of an archive into a file beside the first one.
const item = await this.place(
destination,
target,
leaf,
entry.name,
policy,
'archive-entry',
subdir,
);
if (!item) {
skipped.push({ name: entry.name, reason: '同名文件已存在' });
continue;
}
uploaded.push(item);
written.push(item.path);
}
}
/** Fills in `bookId`/`title` from the index after the scan has run. */
private attachBookIds(items: UploadResultItem[]): void {
for (const item of items) {
const row = this.db.get<{ book_id: string; title: string }>(
`SELECT f.book_id AS book_id, b.title AS title
FROM book_files f JOIN books b ON b.id = f.book_id
WHERE f.rel_path = ?`,
item.path,
);
if (!row) continue;
item.bookId = row.book_id;
item.title = row.title;
}
// A directory book (a comic stored as a folder of images) is indexed under
// the *folder*, so its members have no row of their own. Reporting the
// nearest enclosing book is what makes "I uploaded 40 pages" answer with
// something the reader can open instead of an empty result.
//
// Walked upwards rather than one level: an archive of `系列/第01卷/001.jpg`
// is indexed under `系列`, and the page itself is two levels below it.
for (const item of items) {
if (item.bookId) continue;
let dir = dirname(item.path);
while (dir !== '.' && dir !== '' && dir !== '/') {
const row = this.db.get<{ book_id: string; title: string }>(
`SELECT f.book_id AS book_id, b.title AS title
FROM book_files f JOIN books b ON b.id = f.book_id
WHERE f.rel_path = ?`,
dir,
);
if (row) {
item.bookId = row.book_id;
item.title = row.title;
break;
}
dir = dirname(dir);
}
}
}
}
// ---- name handling ----
const ZIP_CONTAINER_EXTENSIONS = new Set(['.zip', '.cbz']);
function isZipContainer(name: string): boolean {
const dot = name.lastIndexOf('.');
if (dot <= 0) return false;
return ZIP_CONTAINER_EXTENSIONS.has(name.slice(dot).toLowerCase());
}
function segmentsOf(entryName: string): string[] {
return entryName.replace(/\\/g, '/').split('/').filter((segment) => segment.length > 0);
}
/**
* Reduce an archive entry name to safe segments, dropping rather than fixing.
*
* A segment that sanitizes down to `.` or `..` or to something starting with a
* dot is dropped entirely: it only ever appears in a path that was trying to
* walk out of the target or to hide itself from the scanner, and both of those
* are reasons to leave it out rather than to find a legal spelling for it.
*/
function sanitizeArchiveSegments(segments: string[]): string[] {
const out: string[] = [];
for (const segment of segments) {
// A `.`/`..` segment is only ever an attempt to walk out of the destination,
// so the whole path is dropped rather than spelled differently.
if (segment === '.' || segment === '..') return [];
// A name that was absolute (`/包/书.txt`) or an SMB path (`C:\\x\\书.txt`)
// keeps only its own part: it is a name for a file, and where the client
// thought it lived is not something this side can honour.
if (segment.endsWith(':')) return out.length > 1 ? out.slice(0, 1) : out;
const clean = sanitizeSegment(segment);
if (clean === '') return [];
out.push(clean);
}
return out;
}
/**
* A single file name that is safe to create inside the library.
*
* Not a blacklist of bad characters: the result is a name, not a path, so any
* separator becomes `-` and anything unprintable or leading disappears. `.` and
* `..` are refused outright — they are the two names that mean something to the
* filesystem rather than being a name for a file.
*/
export function sanitizeUploadName(input: string): string {
// Windows and SMB clients send a whole path in the filename field. Only the
// leaf is meaningful, and taking `basename` first is what keeps `C:\x\三体.epub`
// from becoming a name with a colon in it.
const leaf = input.replace(/\\/g, '/').split('/').filter(Boolean).pop() ?? '';
const clean = sanitizeSegment(leaf);
if (clean === '' || clean === '.' || clean === '..') {
throw badRequest('the uploaded file needs a usable name', 'BAD_NAME');
}
return clean;
}
function sanitizeSegment(input: string): string {
const stripped = input
// eslint-disable-next-line no-control-regex
.replace(/[\u0000-\u001f\u007f]/g, '')
.replace(/[/\\]/g, '-')
.trim();
// Trailing dots and spaces are invisible on Windows and get dropped by SMB, so
// leaving them would make the file the user sees differ from the one indexed.
const trimmed = stripped.replace(/[. ]+$/g, '');
// A leading dot is not cosmetic: the scanner skips dot-prefixed entries by
// design, so a book uploaded as `.三体.epub` would be on disk and absent from
// the shelf forever. The dot is dropped rather than the name refused, because
// the user did nothing wrong.
return trimmed.replace(/^\.+/, '').slice(0, 255);
}
/** Exported for tests: the same split the archive import and the scanner use. */
export { segmentsOf };
+1 -1
View File
@@ -79,7 +79,7 @@ before(async () => {
ctx.shelf = new ShelfService(db);
ctx.sync = new SyncService(db);
ctx.tts = new TtsService(config);
ctx.browse = new BrowseService(db, config);
ctx.browse = new BrowseService(db, config, ctx.shelf);
app = buildApp(ctx);
await app.ready();
});
+2 -2
View File
@@ -89,7 +89,7 @@ before(async () => {
ctx.shelf = new ShelfService(db);
ctx.sync = new SyncService(db);
ctx.tts = new TtsService(config);
ctx.browse = new BrowseService(db, config);
ctx.browse = new BrowseService(db, config, ctx.shelf);
app = buildApp(ctx);
await app.ready();
@@ -387,7 +387,7 @@ describe('library file management', () => {
describe('a read-only mount', () => {
test('reports itself as not writable and refuses writes with 403', async () => {
const service = new BrowseService(ctx.db, ctx.config);
const service = new BrowseService(ctx.db, ctx.config, ctx.shelf);
const original = service.writable.bind(service);
// Stands in for a read-only bind mount, which looks writable by mode bits and
// refuses every write. Simulated on the probe because the alternative is
+1 -1
View File
@@ -106,7 +106,7 @@ async function buildInstance(): Promise<void> {
ctx.shelf = new ShelfService(db);
ctx.sync = new SyncService(db);
ctx.tts = new TtsService(config);
ctx.browse = new BrowseService(db, config);
ctx.browse = new BrowseService(db, config, ctx.shelf);
app = buildApp(ctx);
await app.ready();
}
+777
View File
@@ -0,0 +1,777 @@
/**
* Upload tests.
*
* The feature is the only one that writes *new* bytes into the library, so the
* behaviour worth asserting is not "the file arrived" — it is everything around
* that: that a name cannot become a path, that a refused mount is refused before
* anything is written, that a failed upload costs nothing, and that a zip of
* forty volumes lands as forty books rather than as a zip.
*
* Real bytes throughout. The parts that break (an archive's central directory, a
* Chinese filename, a truncated stream) only break against real input.
*/
import { test, before, after, describe } from 'node:test';
import assert from 'node:assert/strict';
import { mkdtemp, mkdir, writeFile, readFile, readdir, rm, stat } from 'node:fs/promises';
import { Readable } from 'node:stream';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { deflateRawSync } from 'node:zlib';
import JSZip from 'jszip';
import Fastify from 'fastify';
import type { FastifyInstance } from 'fastify';
import { loadConfig } from '../src/config/index.ts';
import { openDatabase } from '../src/db/index.ts';
import { Scanner } from '../src/indexer/scanner.ts';
import { UserService } from '../src/services/users.ts';
import { ShelfService } from '../src/services/shelf.ts';
import { SyncService } from '../src/services/sync.ts';
import { TtsService } from '../src/services/tts.ts';
import { BrowseService } from '../src/services/browse.ts';
import { UploadService, sanitizeUploadName } from '../src/services/uploads.ts';
import { buildApp } from '../src/http/app.ts';
import type { AppContext } from '../src/http/context.ts';
let app: FastifyInstance;
let ctx: AppContext;
let root: string;
let booksDir: string;
let token: string;
let userId: string;
const OPF = (title: string, id: string) => `<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://www.idpf.org/2007/opf" version="3.0" unique-identifier="bookid">
<metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
<dc:identifier id="bookid">${id}</dc:identifier>
<dc:title>${title}</dc:title><dc:creator>作者</dc:creator><dc:language>zh</dc:language>
</metadata>
<manifest><item id="c" href="c.xhtml" media-type="application/xhtml+xml"/></manifest>
<spine><itemref idref="c"/></spine></package>`;
async function makeEpub(title: string, id: string): Promise<Buffer> {
const zip = new JSZip();
zip.file('mimetype', 'application/epub+zip');
zip.file(
'META-INF/container.xml',
'<?xml version="1.0"?><container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container"><rootfiles><rootfile full-path="OEBPS/content.opf" media-type="application/oebps-package+xml"/></rootfiles></container>',
);
zip.file('OEBPS/content.opf', OPF(title, id));
zip.file('OEBPS/c.xhtml', `<html><body><p>${title}</p></body></html>`);
return zip.generateAsync({ type: 'nodebuffer' });
}
/** A real zip, built by hand so the test controls entry names and order. */
function makeZip(entries: Array<[string, Buffer | string]>, deflate = true): Buffer {
const CRC_TABLE = (() => {
const table = new Int32Array(256);
for (let i = 0; i < 256; i += 1) {
let c = i;
for (let k = 0; k < 8; k += 1) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;
table[i] = c;
}
return table;
})();
const crc32 = (buf: Buffer): number => {
let crc = -1;
for (const byte of buf) crc = (crc >>> 8) ^ CRC_TABLE[(crc ^ byte) & 0xff]!;
return (crc ^ -1) >>> 0;
};
const locals: Buffer[] = [];
const central: Buffer[] = [];
let offset = 0;
for (const [name, content] of entries) {
const raw = Buffer.isBuffer(content) ? content : Buffer.from(content, 'utf8');
const nameBuf = Buffer.from(name, 'utf8');
const deflated = deflateRawSync(raw);
const useDeflate = deflate && deflated.byteLength < raw.byteLength;
const data = useDeflate ? deflated : raw;
const method = useDeflate ? 8 : 0;
const local = Buffer.alloc(30);
local.writeUInt32LE(0x04034b50, 0);
local.writeUInt16LE(20, 4);
local.writeUInt16LE(0x800, 6);
local.writeUInt16LE(method, 8);
local.writeUInt32LE(crc32(raw), 14);
local.writeUInt32LE(data.byteLength, 18);
local.writeUInt32LE(raw.byteLength, 22);
local.writeUInt16LE(nameBuf.byteLength, 26);
locals.push(local, nameBuf, data);
const cen = Buffer.alloc(46);
cen.writeUInt32LE(0x02014b50, 0);
cen.writeUInt16LE(20, 4);
cen.writeUInt16LE(20, 6);
cen.writeUInt16LE(0x800, 8);
cen.writeUInt16LE(method, 10);
cen.writeUInt32LE(crc32(raw), 16);
cen.writeUInt32LE(data.byteLength, 20);
cen.writeUInt32LE(raw.byteLength, 24);
cen.writeUInt16LE(nameBuf.byteLength, 28);
cen.writeUInt32LE(offset, 42);
central.push(cen, nameBuf);
offset += local.byteLength + nameBuf.byteLength + data.byteLength;
}
const centralBuf = Buffer.concat(central);
const eocd = Buffer.alloc(22);
eocd.writeUInt32LE(0x06054b50, 0);
eocd.writeUInt16LE(entries.length, 8);
eocd.writeUInt16LE(entries.length, 10);
eocd.writeUInt32LE(centralBuf.byteLength, 12);
eocd.writeUInt32LE(offset, 16);
return Buffer.concat([...locals, centralBuf, eocd]);
}
function auth() {
return { authorization: `Bearer ${token}` };
}
/**
* Builds a `multipart/form-data` body. Written by hand rather than with a
* library: the boundary and the exact bytes are part of what is being tested, and
* a helper that quietly normalised a weird filename would hide the case that
* matters most.
*/
function multipart(
files: Array<{ field: string; filename: string; content: Buffer }>,
fields: Record<string, string> = {},
): { body: Buffer; contentType: string } {
const boundary = '----reader-test-boundary';
const chunks: Buffer[] = [];
for (const [name, value] of Object.entries(fields)) {
chunks.push(
Buffer.from(
`--${boundary}\r\ncontent-disposition: form-data; name="${name}"\r\n\r\n${value}\r\n`,
'utf8',
),
);
}
for (const file of files) {
chunks.push(
Buffer.from(
`--${boundary}\r\ncontent-disposition: form-data; name="${file.field}"; filename="${file.filename}"\r\n` +
`content-type: application/octet-stream\r\n\r\n`,
'utf8',
),
file.content,
Buffer.from('\r\n', 'utf8'),
);
}
chunks.push(Buffer.from(`--${boundary}--\r\n`, 'utf8'));
return { body: Buffer.concat(chunks), contentType: `multipart/form-data; boundary=${boundary}` };
}
before(async () => {
root = await mkdtemp(join(tmpdir(), 'reader-uploads-'));
booksDir = join(root, 'books');
await mkdir(booksDir, { recursive: true });
process.env.BOOKS_DIR = booksDir;
process.env.DATA_DIR = join(root, 'data');
process.env.SCAN_INTERVAL = '0';
process.env.WATCH_INTERVAL = '0';
process.env.READER_TOKEN_SECRET = 'test-secret-that-is-long-enough';
const config = loadConfig();
const db = openDatabase(config);
ctx = {
config,
db,
scanner: undefined as never,
users: undefined as never,
shelf: undefined as never,
sync: undefined as never,
tts: undefined as never,
browse: undefined as never,
uploads: undefined as never,
log: undefined as never,
};
const silent = Fastify({ logger: false });
ctx.log = silent.log;
ctx.scanner = new Scanner(db, config, { info: () => {}, warn: () => {} });
ctx.users = new UserService(db, config);
ctx.shelf = new ShelfService(db);
ctx.sync = new SyncService(db);
ctx.tts = new TtsService(config);
ctx.browse = new BrowseService(db, config, ctx.shelf);
ctx.uploads = new UploadService(db, config, ctx.browse, ctx.scanner);
app = buildApp(ctx);
await app.ready();
const created = await app.inject({
method: 'POST',
url: '/api/v1/auth/register',
payload: { username: 'owner', password: 'password123' },
});
assert.equal(created.statusCode, 201, created.body);
const body = created.json() as { session: { accessToken: string }; user: { id: string } };
token = body.session.accessToken;
userId = body.user.id;
await mkdir(join(booksDir, '待整理'), { recursive: true });
});
after(async () => {
await app.close();
await rm(root, { recursive: true, force: true });
});
describe('uploading a book', () => {
/**
* A directory of its own per test.
*
* The prompt's library is shared between tests by design — that is what makes
* the conflict policies testable at all — so the collision cases name their
* own folder instead of relying on a particular file being uploaded first.
*/
let caseIndex = 0;
const freshTarget = async (): Promise<string> => {
caseIndex += 1;
const dir = `用例${caseIndex}`;
await mkdir(join(booksDir, dir), { recursive: true });
return dir;
};
test('stores the file, indexes it, and reports the book it became', async () => {
const target = await freshTarget();
const epub = await makeEpub('三体', 'urn:upload-one');
const { body, contentType } = multipart(
[{ field: 'file', filename: '三体.epub', content: epub }],
{ path: target },
);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
assert.equal(res.statusCode, 200, res.body);
const result = res.json() as {
uploaded: Array<{ path: string; name: string; size: number; bookId?: string; title?: string }>;
skipped: unknown[];
scan: { added: number };
};
assert.equal(result.uploaded.length, 1);
assert.equal(result.uploaded[0]!.path, `${target}/三体.epub`);
assert.equal(result.uploaded[0]!.title, '三体');
// Indexed before the response is written, not at the next scheduled scan:
// a file on disk that the shelf cannot see is the exact state this product
// exists to avoid.
assert.ok(result.uploaded[0]!.bookId, 'the upload should be indexed already');
assert.equal(result.uploaded[0]!.size, epub.byteLength);
const onDisk = await readFile(join(booksDir, target, '三体.epub'));
assert.equal(onDisk.byteLength, epub.byteLength);
});
test('a Chinese name written by a Windows client lands as a name, not a path', async () => {
const epub = await makeEpub('斗破苍穹', 'urn:upload-two');
const { body, contentType } = multipart([
{ field: 'file', filename: 'C:\\Users\\me\\我的书\\斗破苍穹.epub', content: epub },
]);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
assert.equal(res.statusCode, 200, res.body);
const result = res.json() as { uploaded: Array<{ path: string }> };
// Only the leaf is meaningful, and the backslashes must not survive as part
// of a file name.
assert.equal(result.uploaded[0]!.path, '斗破苍穹.epub');
assert.ok((await readdir(booksDir)).includes('斗破苍穹.epub'));
});
test('a name that tries to escape the library is refused rather than sanitised into place', () => {
for (const name of ['..', '../..', '/', '.', '']) {
assert.throws(() => sanitizeUploadName(name));
}
// A traversal inside a *path* collapses to the leaf, which is inside.
assert.equal(sanitizeUploadName('../../etc/passwd.epub'), 'passwd.epub');
assert.equal(sanitizeUploadName('a\u0000b.epub'), 'ab.epub');
// A leading dot is not cosmetic: the scanner skips dot-prefixed entries by
// design, so a book uploaded as `.三体.epub` would be on disk and absent from
// the shelf forever.
assert.equal(sanitizeUploadName('.三体.epub'), '三体.epub');
assert.equal(sanitizeUploadName('..\\.hidden.epub'), 'hidden.epub');
// A trailing dot or space is invisible on Windows and dropped by SMB; the
// name that gets indexed has to be the name the user will see.
assert.equal(sanitizeUploadName('三体.epub.'), '三体.epub');
});
test('traversal in the destination directory is refused', async () => {
const epub = await makeEpub('逃逸', 'urn:upload-three');
for (const target of ['../外面', '..', '/etc', '待整理/../..']) {
const { body, contentType } = multipart(
[{ field: 'file', filename: '逃逸.epub', content: epub }],
{ path: target },
);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
assert.ok(res.statusCode >= 400, `${target} answered ${res.statusCode}`);
}
});
test('a name already taken is renamed, skipped or refused according to the policy', async () => {
const target = await freshTarget();
const first = await makeEpub('重名', 'urn:dupe-1');
const second = await makeEpub('重名', 'urn:dupe-2');
const upload = async (content: Buffer, onConflict?: string) => {
const { body, contentType } = multipart(
[{ field: 'file', filename: '重名.epub', content }],
{ path: target, ...(onConflict ? { onConflict } : {}) },
);
return app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
};
// The first copy takes the name; every upload after it is the conflict the
// policy exists for.
const initial = await upload(first);
assert.equal(initial.statusCode, 200, initial.body);
assert.equal((initial.json() as { uploaded: Array<{ name: string }> }).uploaded[0]!.name, '重名.epub');
const renamed = await upload(second);
assert.equal(renamed.statusCode, 200, renamed.body);
assert.equal((renamed.json() as { uploaded: Array<{ name: string }> }).uploaded[0]!.name, '重名 (2).epub');
const skipped = await upload(first, 'skip');
assert.equal(skipped.statusCode, 200, skipped.body);
const skippedBody = skipped.json() as { uploaded: unknown[]; skipped: Array<{ reason: string }> };
assert.equal(skippedBody.uploaded.length, 0);
assert.equal(skippedBody.skipped.length, 1);
});
test('an empty upload is refused, and a failed upload leaves nothing behind', async () => {
const { body, contentType } = multipart([{ field: 'file', filename: '空.epub', content: Buffer.alloc(0) }]);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
assert.equal(res.statusCode, 400);
assert.equal((res.json() as { error: { code: string } }).error.code, 'EMPTY_UPLOAD');
// Every upload copies its bytes into DATA_DIR/uploads before touching the
// library, so the scratch area is where a failure could accumulate. It must
// not: a library that cannot be uploaded to twice is worse than no feature.
//
// Asked of a *fresh* scratch root, because node:test runs the cases in this
// file concurrently against one library: asserting on the shared directory
// would be asserting on whichever other case happens to be mid-flight.
const isolated = await mkdtemp(join(root, 'scratch-'));
const service = new UploadService(ctx.db, { ...ctx.config, dataDir: isolated }, ctx.browse, ctx.scanner);
const empty = await makeEpub('空', 'urn:empty');
await assert.rejects(
service.store({ name: '空.epub', stream: Readable.from(Buffer.alloc(0)), target: '' }),
(err: unknown) => (err as { code?: string }).code === 'EMPTY_UPLOAD',
);
const left = await readdir(join(isolated, 'uploads')).catch(() => []);
assert.deepEqual(left, [], 'a failed upload must not leave scratch behind');
void empty;
assert.ok(!(await readdir(booksDir)).includes('空.epub'));
});
test('a stream that dies mid-transfer is reported as an incomplete upload', async () => {
const broken = new Readable({
read() {
this.push(Buffer.from('partial'));
this.destroy(new Error('connection reset'));
},
});
await assert.rejects(
ctx.uploads.store({ name: '半本.epub', stream: broken, target: '' }),
(err: unknown) => (err as { code?: string }).code === 'UPLOAD_INCOMPLETE',
);
assert.ok(!(await readdir(booksDir)).includes('半本.epub'));
});
test('a batch is stored together and indexed once', async () => {
const target = await freshTarget();
const { body, contentType } = multipart(
[
{ field: 'file', filename: '合集一.epub', content: await makeEpub('合集一', 'urn:batch-a') },
{ field: 'file', filename: '合集二.epub', content: await makeEpub('合集二', 'urn:batch-b') },
],
{ path: target },
);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
assert.equal(res.statusCode, 200, res.body);
const result = res.json() as {
uploaded: Array<{ path: string; bookId?: string }>;
scan: { startedAt: number; finishedAt: number };
};
assert.equal(result.uploaded.length, 2);
assert.ok(result.uploaded.every((item) => item.bookId), 'both files should be indexed');
// One scan, not one per file: a batch is one library change.
assert.ok(result.scan.finishedAt >= result.scan.startedAt);
});
test('a batch whose later name is taken is still refused whole', async () => {
const target = await freshTarget();
const taken = await makeEpub('占用', 'urn:taken');
const seed = multipart([{ field: 'file', filename: '占用.epub', content: taken }], { path: target });
const seeded = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': seed.contentType },
payload: seed.body,
});
assert.equal(seeded.statusCode, 200, seeded.body);
const { body, contentType } = multipart(
[
{ field: 'file', filename: '新书.epub', content: await makeEpub('新书', 'urn:new') },
{ field: 'file', filename: '占用.epub', content: taken },
],
{ path: target, onConflict: 'fail' },
);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
// `fail` names the one conflict that has no reasonable guess behind it, so
// the whole request is refused. Nothing may have been written for the first
// file either — a half-applied batch is the shape a user cannot reason about.
assert.equal(res.statusCode, 409, res.body);
assert.equal((res.json() as { error: { code: string } }).error.code, 'DESTINATION_EXISTS');
const names = await readdir(join(booksDir, target));
assert.deepEqual(names, ['占用.epub']);
});
test('a JSON body is refused, so the client gets a usable error', async () => {
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': 'application/json' },
payload: { file: 'nope' },
});
assert.equal(res.statusCode, 400);
assert.equal((res.json() as { error: { code: string } }).error.code, 'NOT_MULTIPART');
});
test('the endpoint requires a token', async () => {
const res = await app.inject({ method: 'POST', url: '/api/v1/library/upload' });
assert.equal(res.statusCode, 401);
});
test('the write probe answers before a phone starts sending bytes', async () => {
const res = await app.inject({ method: 'GET', url: '/api/v1/library/upload', headers: auth() });
assert.equal(res.statusCode, 200);
assert.equal((res.json() as { writable: boolean }).writable, true);
});
});
describe('uploading an archive', () => {
let archiveCase = 0;
const freshArchiveTarget = async (): Promise<string> => {
archiveCase += 1;
const dir = `压缩包${archiveCase}`;
await mkdir(join(booksDir, dir), { recursive: true });
return dir;
};
test('a zip of books is unpacked into the library, one book each', async () => {
const target = await freshArchiveTarget();
const archive = makeZip([
['系列/第01卷.epub', await makeEpub('第一卷', 'urn:series-1')],
['系列/第02卷.epub', await makeEpub('第二卷', 'urn:series-2')],
['系列/说明.txt', '第一卷:开始\n第二卷:结束\n'],
]);
const { body, contentType } = multipart(
[{ field: 'file', filename: '系列合集.zip', content: archive }],
{ path: target },
);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
assert.equal(res.statusCode, 200, res.body);
const result = res.json() as { uploaded: Array<{ path: string; title?: string; kind: string }> };
assert.equal(result.uploaded.length, 3);
const paths = result.uploaded.map((item) => item.path).sort();
// The single wrapping folder is flattened, because that is what every
// "download this series" zip looks like.
assert.deepEqual(paths, [`${target}/第01卷.epub`, `${target}/第02卷.epub`, `${target}/说明.txt`]);
assert.ok(result.uploaded.every((item) => item.kind === 'archive-entry'));
assert.ok(result.uploaded.some((item) => item.title === '第一卷'));
});
test('a nested comic archive keeps its structure, because that is what makes it a book', async () => {
const target = await freshArchiveTarget();
const archive = makeZip([
['套图/第01卷/001.jpg', Buffer.from([0xff, 0xd8, 0xff, 0xe0, 0x00, 0x10])],
['套图/第01卷/002.jpg', Buffer.from([0xff, 0xd8, 0xff, 0xe0, 0x00, 0x10])],
['套图/第02卷/001.jpg', Buffer.from([0xff, 0xd8, 0xff, 0xe0, 0x00, 0x10])],
]);
const { body, contentType } = multipart(
[{ field: 'file', filename: '漫画.zip', content: archive }],
{ path: target },
);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
assert.equal(res.statusCode, 200, res.body);
const result = res.json() as { uploaded: Array<{ path: string; bookId?: string; title?: string }> };
// `漫画/` is one folder, so it is flattened; the volumes inside it are not.
// Flattening those would destroy the volume layout the comic handler reads.
assert.deepEqual(
result.uploaded.map((item) => item.path).sort(),
[`${target}/第01卷/001.jpg`, `${target}/第01卷/002.jpg`, `${target}/第02卷/001.jpg`],
);
assert.ok(result.uploaded.every((item) => item.bookId), 'the folder as a whole should be a book');
});
test('an entry name cannot escape the target directory', async () => {
const archive = makeZip([
['../../逃逸.txt', 'nope'],
['/绝对路径.txt', 'nope'],
['ok/正常.txt', '这本书在目录里\n'],
]);
const { body, contentType } = multipart([
{ field: 'file', filename: '恶意.zip', content: archive },
]);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
assert.equal(res.statusCode, 200, res.body);
const result = res.json() as { uploaded: Array<{ path: string }>; skipped: Array<{ name: string }> };
const paths = result.uploaded.map((item) => item.path);
// Nothing outside the library, and nothing that is not simply a name.
for (const path of paths) {
assert.ok(!path.includes('..'), `${path} should not be a traversal`);
assert.ok(!path.startsWith('/'), `${path} should not be absolute`);
}
// The two hostile names are refused rather than rewritten into place.
assert.ok(!paths.some((path) => path.includes('逃逸')));
assert.ok(!paths.some((path) => path.includes('绝对路径')));
await assert.rejects(stat(join(root, '逃逸.txt')));
});
test('a zip that contains nothing is refused', async () => {
const { body, contentType } = multipart([
{ field: 'file', filename: '空.zip', content: makeZip([]) },
]);
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
});
assert.ok(res.statusCode >= 400, res.body);
});
});
describe('batch management', () => {
let batchRoot: string;
before(async () => {
batchRoot = '批量';
await mkdir(join(booksDir, batchRoot), { recursive: true });
await writeFile(join(booksDir, batchRoot, '卷一.epub'), await makeEpub('合集第一卷', 'urn:batch-1'));
await writeFile(join(booksDir, batchRoot, '卷二.epub'), await makeEpub('合集第二卷', 'urn:batch-2'));
await ctx.scanner.scan();
});
test('one patch rewrites the metadata of a whole folder', async () => {
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/browse/metadata',
headers: auth(),
payload: { paths: [batchRoot], fields: { author: '批量作者', series: '合集' } },
});
assert.equal(res.statusCode, 200, res.body);
const result = res.json() as { applied: number; books: string[]; failed: Array<{ path: string }> };
assert.equal(result.applied, 2);
assert.equal(result.books.length, 2);
assert.deepEqual(result.failed, []);
// Read through the shelf, not the base column: a manual override is applied
// when the book is read, so a test that inspected `books.author` would see
// the embedded value and conclude the edit had been lost.
const books = ctx.db.all<{ book_id: string }>(
'SELECT book_id FROM book_files WHERE rel_path LIKE ?',
`${batchRoot}/%`,
);
assert.equal(books.length, 2);
const authors = books.map((row) => ctx.shelf.get(userId, row.book_id).author);
assert.deepEqual(authors, ['批量作者', '批量作者']);
});
test('a path that is not a book is reported rather than failing the batch', async () => {
await mkdir(join(booksDir, '空目录'), { recursive: true });
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/browse/metadata',
headers: auth(),
payload: { paths: [`${batchRoot}/卷一.epub`, '空目录'], fields: { publisher: '某社' } },
});
assert.equal(res.statusCode, 200, res.body);
const result = res.json() as { applied: number; failed: Array<{ path: string; reason: string }> };
assert.equal(result.applied, 1);
assert.deepEqual(result.failed, [{ path: '空目录', reason: 'NOT_A_BOOK' }]);
});
test('books can be taken off the shelf and put back without touching the disk', async () => {
const before = ctx.db.get<{ n: number }>(
'SELECT COUNT(*) AS n FROM user_books WHERE user_id = ? AND hidden = 0',
userId,
);
const removed = await app.inject({
method: 'POST',
url: '/api/v1/library/browse/shelf',
headers: auth(),
payload: { paths: [batchRoot], action: 'remove' },
});
assert.equal(removed.statusCode, 200, removed.body);
assert.equal((removed.json() as { applied: number }).applied, 2);
const after = ctx.db.get<{ n: number }>(
'SELECT COUNT(*) AS n FROM user_books WHERE user_id = ? AND hidden = 0',
userId,
);
assert.equal(after!.n, before!.n - 2);
// The files are still there. This is a shelf operation, not a delete, and
// conflating the two is the mistake this endpoint is shaped to prevent.
await stat(join(booksDir, batchRoot, '卷一.epub'));
const restored = await app.inject({
method: 'POST',
url: '/api/v1/library/browse/shelf',
headers: auth(),
payload: { paths: [batchRoot], action: 'unhide' },
});
assert.equal(restored.statusCode, 200, restored.body);
const back = ctx.db.get<{ n: number }>(
'SELECT COUNT(*) AS n FROM user_books WHERE user_id = ? AND hidden = 0',
userId,
);
assert.equal(back!.n, before!.n);
});
test('an unknown shelf action is refused', async () => {
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/browse/shelf',
headers: auth(),
payload: { paths: [batchRoot], action: 'delete' },
});
assert.equal(res.statusCode, 400);
assert.equal((res.json() as { error: { code: string } }).error.code, 'BAD_ACTION');
});
test('a batch with no editable field is refused', async () => {
const res = await app.inject({
method: 'POST',
url: '/api/v1/library/browse/metadata',
headers: auth(),
payload: { paths: [batchRoot], fields: { id: 'nope' } },
});
assert.equal(res.statusCode, 400);
});
});
describe('a read-only mount', () => {
/**
* Stands in for a read-only bind mount, which looks writable by mode bits and
* refuses every write with `EPERM`.
*
* Simulated on the probe because the alternative is remounting the test's own
* filesystem, which CI runs as root and would let straight through — the whole
* reason the writability check probes by writing rather than reading mode bits.
*/
const withReadOnly = async <T>(fn: () => Promise<T>): Promise<T> => {
const original = ctx.browse.writable.bind(ctx.browse);
ctx.browse.writable = async () => false;
try {
return await fn();
} finally {
ctx.browse.writable = original;
}
};
test('the probes agree that the mount cannot be written to', async () => {
await withReadOnly(async () => {
const probe = await app.inject({ method: 'GET', url: '/api/v1/library/upload', headers: auth() });
assert.equal(probe.statusCode, 200);
assert.equal((probe.json() as { writable: boolean }).writable, false);
});
});
test('refuses an upload with 403 before storing a byte', async () => {
const epub = await makeEpub('只读', 'urn:ro');
const { body, contentType } = multipart([{ field: 'file', filename: '只读.epub', content: epub }], {
path: '待整理',
});
const res = await withReadOnly(() =>
app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
}),
);
// A 403 with a code the client can act on, rather than an `EROFS` stack
// trace: the mount is the deployment's, and the answer has to say so.
assert.equal(res.statusCode, 403, res.body);
assert.equal((res.json() as { error: { code: string } }).error.code, 'READ_ONLY_MOUNT');
await assert.rejects(stat(join(booksDir, '待整理', '只读.epub')));
});
test('leaves no scratch behind when the mount refuses the write', async () => {
const epub = await makeEpub('只读二', 'urn:ro-2');
const { body, contentType } = multipart([{ field: 'file', filename: '只读二.epub', content: epub }]);
const res = await withReadOnly(() =>
app.inject({
method: 'POST',
url: '/api/v1/library/upload',
headers: { ...auth(), 'content-type': contentType },
payload: body,
}),
);
assert.equal(res.statusCode, 403);
// Asked of a *fresh* scratch root, because node:test runs the cases in this
// file concurrently against one library: asserting on the shared directory
// would be asserting on whichever other case happens to be mid-flight.
const isolated = await mkdtemp(join(root, 'ro-scratch-'));
const service = new UploadService(ctx.db, { ...ctx.config, dataDir: isolated }, ctx.browse, ctx.scanner);
await withReadOnly(() =>
assert.rejects(
service.store({ name: '只读三.epub', stream: Readable.from(epub), target: '待整理' }),
(err: unknown) => (err as { code?: string }).code === 'READ_ONLY_MOUNT',
),
);
// The bytes were received before the mount was consulted, so the scratch copy
// is exactly what a refusal has to clean up.
const left = await readdir(join(isolated, 'uploads')).catch(() => []);
assert.deepEqual(left, [], 'a refused upload must not leave scratch behind');
});
});
+87
View File
@@ -1,8 +1,10 @@
import { ApiError, errorForStatus, parseErrorBody } from './errors.ts';
import type {
BatchResult,
Book,
BookContent,
BrowseListing,
ConflictPolicy,
BookListPage,
ContinueReadingItem,
Facets,
@@ -12,9 +14,11 @@ import type {
NoteType,
Progress,
Session,
ShelfAction,
SyncPull,
SyncPushResult,
TocEntry,
UploadResult,
User,
} from './types.ts';
import type { Platform } from '../core/platform.ts';
@@ -312,6 +316,89 @@ export class ReaderApi {
return this.call('/api/v1/library/browse/delete', 'POST', { paths }, options);
}
/**
* Applies one metadata patch to many paths.
*
* A path may name a folder, which the server reads as "every book indexed
* inside it" — a series is organised as a folder, and asking for the folder's
* own row would find nothing. The client never expands a folder itself: that
* mapping is the index's, and a second implementation of it here would be a
* second thing to keep in step.
*/
async browseBatchMetadata(
paths: string[],
fields: Record<string, unknown>,
options: RequestOptions = {},
): Promise<BatchResult> {
return this.call('/api/v1/library/browse/metadata', 'POST', { paths, fields }, options);
}
/**
* Adds or removes books on this account's shelf.
*
* Nothing on disk changes. Kept a separate call from move/delete because the
* two are one word apart in a list of rows and could not be more different in
* consequence.
*/
async browseBatchShelf(
paths: string[],
action: ShelfAction,
options: RequestOptions = {},
): Promise<BatchResult> {
return this.call('/api/v1/library/browse/shelf', 'POST', { paths, action }, options);
}
/** Whether the mount accepts an upload at all; asked before sending bytes. */
async uploadProbe(options: RequestOptions = {}): Promise<boolean> {
const result = await this.get<{ writable: boolean }>('/api/v1/library/upload', options);
return result.writable;
}
/**
* Uploads files into a library directory.
*
* `FormData`, not a hand-built body: a browser sets the multipart boundary and
* streams a `File` off disk, and reimplementing either would mean buffering a
* 400MB comic in memory on a phone.
*
* The progress hook rides on the request rather than replacing the transport.
* The browser itself cannot report upload progress — `fetch` has no such event
* in any shipping engine — so the H5 build simply never calls it and the UI
* shows "上传中…"; a host that can (the Android shell's native path) does, and
* the same screen then shows a real percentage. What must *not* happen is a
* second implementation of the upload that only one host exercises.
*/
async upload(
files: File[],
target: string,
onConflict: ConflictPolicy,
onProgress?: (fraction: number) => void,
options: RequestOptions = {},
): Promise<UploadResult> {
const form = new FormData();
for (const file of files) form.append('file', file, file.name);
if (target) form.append('path', target);
form.append('onConflict', onConflict);
// No `content-type` is set: only the host knows the multipart boundary, and
// a header naming `application/json` over a `FormData` leaves the server
// unable to parse a body that is perfectly well formed.
const response = await this.platform.transport.send({
url: '/api/v1/library/upload',
method: 'POST',
headers: { accept: 'application/json', ...this.authHeader() },
body: form,
...(onProgress ? { onUploadProgress: onProgress } : {}),
...(options.signal ? { signal: options.signal } : {}),
});
return response.json as UploadResult;
}
private authHeader(): Record<string, string> {
return this.session ? { authorization: `Bearer ${this.session.accessToken}` } : {};
}
// ---- content ----
/** Authenticated book body. Not cacheable by the service worker: it needs a token. */
+50
View File
@@ -236,6 +236,56 @@ export interface BrowseListing {
name: string;
}
/** One file that an upload put into the library. */
export interface UploadedItem {
/** Library-relative path it now lives at. */
path: string;
/** The name the client sent, so a report can name what the user chose. */
originalName: string;
/** Name after the conflict policy ran; differs when a suffix was added. */
name: string;
size: number;
/** `archive-entry` for a member of an uploaded zip. */
kind: 'file' | 'archive-entry';
/** Present once the incremental scan has indexed it. */
bookId?: string;
title?: string;
}
export interface UploadResult {
uploaded: UploadedItem[];
/** Files that were not stored, with the reason, rather than failing the batch. */
skipped: Array<{ name: string; reason: string }>;
/**
* Files already on disk are kept rather than replaced.
*
* `rename` keeps both, `skip` keeps the one that was there, `overwrite`
* replaces it — which is the only one of the four that can lose a book, so it
* is not the default and the UI says so.
*/
scan: {
added: number;
updated: number;
removed: number;
failed: number;
startedAt: number | null;
finishedAt: number | null;
} | null;
}
/** Outcome of a batch operation. Mirrors the server's `BatchResult`. */
export interface BatchResult {
/** Rows changed; a folder counts once per book inside it. */
applied: number;
books: string[];
/** Paths that could not be acted on, with the reason. */
failed: Array<{ path: string; reason: string }>;
}
export type ConflictPolicy = 'rename' | 'skip' | 'overwrite' | 'fail';
export type ShelfAction = 'add' | 'remove' | 'hide' | 'unhide';
export interface ApiErrorBody {
error: { code: string; message: string };
}
+3
View File
@@ -19,6 +19,9 @@ export class FetchTransport implements Transport {
response = await fetch(url, {
method: request.method,
headers: request.headers,
// A `FormData` body must be sent without a caller-set `content-type`:
// only the browser knows the multipart boundary, and it can only add one
// it chose itself.
...(request.body !== undefined ? { body: request.body } : {}),
...(request.signal ? { signal: request.signal } : {}),
mode: 'cors',
+20 -2
View File
@@ -12,10 +12,28 @@ export interface HttpRequest {
url: string;
method: string;
headers: Record<string, string>;
/** JSON-encoded request body, when there is one. */
body?: string;
/**
* Request body.
*
* A string for the JSON calls, a `FormData` for an upload. The two are genuinely
* different things and the transport must not guess: a `FormData` carries its
* own `content-type` with the multipart boundary in it, and setting a header
* that says `application/json` over one leaves a body the server cannot parse.
*/
body?: string | FormData;
/** Binary response mode: used for book bodies and covers. */
binary?: boolean;
/**
* Called with `loaded / total` while the body is being sent.
*
* A request *property* rather than a second transport method, because there is
* exactly one kind of request anyone wants progress on — a body the user's own
* connection has to carry — and a host that cannot report it simply never calls
* this. `fetch` is the case that matters: no shipping browser reports upload
* progress, so the browser host reports nothing and the UI falls back to
* "上传中…" rather than to a progress bar that never moves.
*/
onUploadProgress?: (fraction: number) => void;
signal?: AbortSignal;
}
+80 -1
View File
@@ -1432,6 +1432,16 @@ book-content ::selection {
border-top-color: var(--reader-line);
}
/*
* The batch bar scrolls horizontally rather than wrapping or squeezing.
*
* Five actions do not fit across a phone, and neither alternative works: wrapping
* makes the bar two rows tall and pushes the list up under the thumb that is
* mid-scroll, and `flex: 1` per button shrinks 移动到… to an ellipsis that no
* longer says what it does. A scrolling row keeps every label readable and the
* destructive action — deliberately last, so it is never the thing under the
* finger that just opened the bar — stays off screen until it is looked for.
*/
.manager-actions {
display: flex;
gap: 0.5rem;
@@ -1440,6 +1450,13 @@ book-content ::selection {
padding-bottom: calc(0.6rem + env(safe-area-inset-bottom));
border-top: 1px solid var(--reader-line);
background: var(--reader-surface);
overflow-x: auto;
scrollbar-width: none;
-webkit-overflow-scrolling: touch;
}
.manager-actions::-webkit-scrollbar {
display: none;
}
.manager-actions[hidden] {
@@ -1447,7 +1464,9 @@ book-content ::selection {
}
.manager-actions .button {
flex: 1 1 auto;
/* `flex: 0 0 auto`, so a label is never truncated — the bar scrolls instead. */
flex: 0 0 auto;
white-space: nowrap;
}
.manager-selection {
@@ -1537,3 +1556,63 @@ book-content ::selection {
flex-direction: column;
gap: 0.4rem;
}
/* ---- batch metadata ---- */
/*
* The batch dialog has seven optional fields, which do not fit on a phone.
*
* The fields scroll and the buttons do not. Scrolling the whole dialog instead
* puts 保存 below the fold, so the reader fills the form, sees no way to commit
* it, and has to work out that the sheet itself moves — and the on-screen
* keyboard takes another third of the height while they are typing in it.
*
* `max-height` on the *fields* rather than on the dialog also keeps the title
* visible, so it stays obvious what is being edited after a scroll.
*/
.dialog:has(.dialog-field) {
max-height: min(86vh, 40rem);
overflow: hidden;
}
.dialog:has(.dialog-field) .dialog-fields {
display: flex;
flex-direction: column;
gap: 0.6rem;
overflow-y: auto;
min-height: 0;
/* Room for the last field's label, so it is not flush against the buttons. */
padding-bottom: 0.2rem;
}
.dialog-field {
display: flex;
flex-direction: column;
gap: 0.25rem;
font-size: 0.78rem;
color: var(--reader-muted);
}
.dialog-field input {
appearance: none;
border: 1px solid var(--reader-line);
background: var(--reader-paper);
color: var(--reader-ink);
border-radius: var(--reader-radius);
padding: 0.5rem 0.6rem;
/* 16px, so iOS does not zoom the page on focus and push the dialog off-screen. */
font-size: 16px;
font-family: inherit;
width: 100%;
}
/*
* The file input is the trigger for the ⬆ button and is never shown.
*
* `display: none` rather than `position: absolute` off-screen: a control that is
* laid out still takes a tab stop, and this one has no meaning of its own — the
* button beside it is what the user operates.
*/
.manager-upload-input {
display: none;
}
+276 -17
View File
@@ -1,6 +1,6 @@
import { ApiError } from '../api/errors.ts';
import type { ReaderApi } from '../api/client.ts';
import type { BrowseEntry, BrowseListing } from '../api/types.ts';
import type { BrowseEntry, BrowseListing, ConflictPolicy, ShelfAction } from '../api/types.ts';
import { clear, el, formatBytes, formatDate } from './dom.ts';
export interface ManagerScreenOptions {
@@ -45,6 +45,8 @@ export class ManagerScreen {
private readonly actionBar: HTMLDivElement;
private readonly selectionCount: HTMLDivElement;
private readonly mkdirButton: HTMLButtonElement;
private readonly uploadButton: HTMLButtonElement;
private readonly uploadInput: HTMLInputElement;
private readonly listBody: HTMLDivElement;
private current = '';
@@ -52,6 +54,16 @@ export class ManagerScreen {
/** Paths ticked for a batch operation. Empty means "navigate" mode. */
private readonly selected = new Set<string>();
private loading = false;
/** A write is in flight; every other write is refused until it settles. */
private busy = false;
/**
* The message a completed write left for the next render.
*
* Held across the reload a write triggers, because that reload's own status
* line is the directory summary — and the summary is not an answer to "did it
* work".
*/
private outcome: string | null = null;
private longPressTimer: ReturnType<typeof setTimeout> | null = null;
private longPressFired = false;
@@ -67,6 +79,20 @@ export class ManagerScreen {
attrs: { type: 'button', 'aria-label': '新建文件夹' },
on: { click: () => void this.promptMkdir() },
}) as HTMLButtonElement;
// Hidden from the DOM to `display: none`, never hidden by *moving* it: a
// file input removed from the document loses its value in some WebViews, and
// the second upload after a cancel would then send nothing.
this.uploadInput = el('input', {
className: 'manager-upload-input',
attrs: { type: 'file', multiple: true, 'aria-hidden': 'true', tabindex: '-1' },
on: { change: () => void this.handlePicked() },
}) as HTMLInputElement;
this.uploadButton = el('button', {
className: 'icon-button',
text: '⬆',
attrs: { type: 'button', 'aria-label': '上传书籍' },
on: { click: () => this.uploadInput.click() },
}) as HTMLButtonElement;
this.listBody = el('div', {
className: 'manager-body',
@@ -83,13 +109,14 @@ export class ManagerScreen {
on: { click: () => options.onClose() },
}),
this.crumbRow,
this.uploadButton,
this.mkdirButton,
],
});
this.element = el('div', {
className: 'manager-screen',
children: [header, this.listBody, this.actionBar, this.selectionCount],
children: [header, this.listBody, this.actionBar, this.selectionCount, this.uploadInput],
}) as HTMLDivElement;
// Clicking the empty area (not a row) clears a selection, which is the
@@ -106,7 +133,11 @@ export class ManagerScreen {
private async load(path: string): Promise<void> {
if (this.loading) return;
this.loading = true;
this.setStatus('载入中…');
// An outcome message set by a write outlives the reload it triggers.
// Otherwise "已更新 2 本" is replaced by the directory's file counts a tick
// later, and the one thing the user needs to read is the one thing they
// never see.
if (!this.outcome) this.setStatus('载入中…');
try {
const listing = await this.options.api.browse(path);
this.listing = listing;
@@ -119,6 +150,11 @@ export class ManagerScreen {
// say*, and that thing is what `renderEntries` just wrote — how many
// entries, how big, and how many the scanner is skipping. Clearing it left
// the screen with no answer to any of those.
if (this.outcome) {
const message = this.outcome;
this.outcome = null;
this.toast(message);
}
} catch (err) {
this.handleError(err);
} finally {
@@ -135,7 +171,10 @@ export class ManagerScreen {
* and is dimmed by CSS rather than by a second branch in the render path.
*/
private applyWritable(writable: boolean): void {
// The upload control disappears for the same reason the + does: a control
// that can only answer 403 teaches the reader to distrust every other one.
this.mkdirButton.hidden = !writable;
this.uploadButton.hidden = !writable;
this.element.dataset['writable'] = String(writable);
}
@@ -334,7 +373,19 @@ export class ManagerScreen {
this.actionBar.append(
el('button', {
className: 'button',
text: '移动到…',
text: '改资料',
attrs: { type: 'button' },
on: { click: () => void this.promptBatchMetadata() },
}),
el('button', {
className: 'button',
text: '下架',
attrs: { type: 'button' },
on: { click: () => void this.batchShelf('remove') },
}),
el('button', {
className: 'button',
text: '移动…',
attrs: { type: 'button' },
on: { click: () => void this.promptMove() },
}),
@@ -359,6 +410,8 @@ export class ManagerScreen {
}
const action = await this.pick(`「${entry.name}」`, [
{ value: 'select', label: '选择' },
{ value: 'metadata', label: '改资料' },
{ value: 'shelve', label: '从书架拿掉' },
{ value: 'rename', label: '重命名' },
{ value: 'move', label: '移动到…' },
{ value: 'delete', label: '删除' },
@@ -367,21 +420,218 @@ export class ManagerScreen {
this.toggle(entry);
return;
}
// Everything below acts on exactly this row, so the selection is replaced
// rather than merged: an action started from one row's menu must not quietly
// include a batch ticked earlier and now off screen.
this.selectOnly(entry.path);
if (action === 'metadata') return this.promptBatchMetadata();
if (action === 'shelve') return this.batchShelf('remove');
if (action === 'rename') return this.promptRename(entry.path, entry.name);
if (action === 'move') {
this.selected.clear();
this.selected.add(entry.path);
this.renderSelection();
return this.promptMove();
if (action === 'move') return this.promptMove();
if (action === 'delete') return this.confirmDelete();
}
private selectOnly(path: string): void {
this.selected.clear();
this.selected.add(path);
for (const node of this.list.querySelectorAll<HTMLElement>('.manager-row')) {
node.dataset['selected'] = String(node.dataset['path'] === path);
}
if (action === 'delete') {
this.selected.clear();
this.selected.add(entry.path);
this.renderSelection();
return this.confirmDelete();
this.renderSelection();
}
/**
* Uploads the picked files into the directory currently open.
*
* The progress line is the whole point of the display: a book is routinely
* hundreds of megabytes on a phone over Wi-Fi, and a screen that says nothing
* for two minutes reads as broken. It is also the only place in this client
* where a request can outlive a screen change, so the input is cleared *before*
* the upload rather than after — picking the same file twice in a row must
* work.
*/
private async handlePicked(): Promise<void> {
const files = [...(this.uploadInput.files ?? [])];
this.uploadInput.value = '';
if (files.length === 0) return;
const policy = await this.pickConflictPolicy();
if (!policy) return;
const total = files.reduce((sum, file) => sum + file.size, 0);
this.setBusy(true);
try {
const result = await this.options.api.upload(files, this.current, policy, (fraction) => {
this.setStatus(`上传中… ${Math.round(fraction * 100)}%(${formatBytes(total)})`);
});
this.reportUpload(result);
} catch (err) {
this.handleError(err);
} finally {
this.setBusy(false);
}
}
/**
* Asks what should happen to a name that is already taken.
*
* Asked before the bytes move rather than after a 409, because by then the
* user has waited for the whole upload twice. `覆盖` is the only answer that can
* lose a book, so it is described that way instead of being the default.
*/
private async pickConflictPolicy(): Promise<ConflictPolicy | null> {
const answer = await this.pick('同名文件怎么办?', [
{ value: 'rename', label: '两份都留(加 (2))' },
{ value: 'skip', label: '跳过已有的' },
{ value: 'overwrite', label: '覆盖(会替换原文件)' },
{ value: 'fail', label: '有重名就整批不动' },
]);
return answer as ConflictPolicy | null;
}
private reportUpload(result: Awaited<ReturnType<ReaderApi['upload']>>): void {
const parts: string[] = [];
if (result.uploaded.length > 0) parts.push(`已入库 ${result.uploaded.length} 个文件`);
if (result.skipped.length > 0) parts.push(`跳过 ${result.skipped.length} 个`);
const added = result.scan?.added ?? 0;
parts.push(added > 0 ? `新增 ${added} 本` : '没有新书');
// A skipped file is the one thing the user has to act on, so it is named
// rather than counted: "跳过 1 个" without a name is a dead end.
const first = result.skipped[0];
if (first) parts.push(`(${first.name}:${first.reason})`);
this.toast(parts.join(' · '));
}
/**
* Rewrites the metadata of every selected path at once.
*
* The fields are optional on purpose — this is a *patch*, and an empty field
* means "leave it alone", which is the only way one dialog can serve both "fill
* in the missing author on forty files" and "fix the series name".
*/
private async promptBatchMetadata(): Promise<void> {
const paths = [...this.selected];
if (paths.length === 0) return;
const fields = await this.batchMetadataDialog(paths.length);
if (fields === null) return;
if (Object.keys(fields).length === 0) {
this.toast('没有填写任何字段');
return;
}
await this.mutate(
() => this.options.api.browseBatchMetadata(paths, fields),
`改资料(${paths.length} 项)`,
(result) => this.reportBatch(result),
);
}
private async batchShelf(action: ShelfAction): Promise<void> {
const paths = [...this.selected];
if (paths.length === 0) return;
const label = action === 'remove' ? '下架' : '上架';
if (action === 'remove') {
const ok = await this.confirm(
`${label} ${paths.length} 项?`,
'只是从你的书架上拿掉,磁盘上的文件一个都不会动,随时可以放回来。',
);
if (!ok) return;
}
await this.mutate(
() => this.options.api.browseBatchShelf(paths, action),
`${label}(${paths.length} 项)`,
(result) => this.reportBatch(result),
);
}
private reportBatch(result: { applied: number; failed: Array<{ path: string; reason: string }> }): string {
if (result.applied === 0 && result.failed.length > 0) {
return `没有可处理的书籍(${result.failed.length} 项不是书)`;
}
// A path that is not a book is expected — a folder of images, a `.nfo` — and
// saying so is what stops "已更新 12 本" from looking like the other two were
// silently dropped.
const skipped = result.failed.length > 0 ? `,跳过 ${result.failed.length} 项` : '';
return `已更新 ${result.applied} 本${skipped}`;
}
/** A form of optional fields; absent means "do not touch". */
private batchMetadataDialog(count: number): Promise<Record<string, string> | null> {
const FIELDS = [
{ key: 'author', label: '作者' },
{ key: 'publisher', label: '出版社' },
{ key: 'series', label: '系列' },
{ key: 'seriesIndex', label: '系列序号' },
{ key: 'language', label: '语言' },
{ key: 'tags', label: '标签(逗号分隔)' },
{ key: 'pubdate', label: '出版日期' },
];
return new Promise((resolve) => {
const inputs = new Map<string, HTMLInputElement>();
const rows = FIELDS.map((field) => {
const input = el('input', { attrs: { type: 'text', placeholder: '留空则不改' } }) as HTMLInputElement;
inputs.set(field.key, input);
return el('label', { className: 'dialog-field', children: [el('span', { text: field.label }), input] });
});
const close = (result: Record<string, string> | null): void => {
overlay.remove();
resolve(result);
};
const form = el('form', {
className: 'dialog',
on: {
submit: (event) => {
event.preventDefault();
const patch: Record<string, string> = {};
for (const [key, input] of inputs) {
const value = input.value.trim();
if (value !== '') patch[key] = value;
}
close(patch);
},
},
children: [
el('h3', { text: `改资料(${count} 项)` }),
el('p', { className: 'muted', text: '留空的字段保持不变。目录会写成它里面每一本书的资料。' }),
// The fields are their own scroller so the actions below them stay
// reachable: a sheet that scrolls whole puts 保存 past the fold, and a
// form with no visible way to commit it reads as a dead end.
el('div', { className: 'dialog-fields', children: rows }),
el('div', {
className: 'dialog-actions',
children: [
el('button', { className: 'button', text: '取消', attrs: { type: 'button' }, on: { click: () => close(null) } }),
el('button', { className: 'button primary', text: '保存', attrs: { type: 'submit' } }),
],
}),
],
});
const overlay = el('div', {
className: 'dialog-overlay',
on: {
click: (event) => {
if (event.target === overlay) close(null);
},
},
children: [form],
}) as HTMLDivElement;
this.element.append(overlay);
inputs.get('author')?.focus();
});
}
/**
* Blocks a second write while one is in flight.
*
* Uploads are long and the bar stays on screen; a second tap would start a
* second upload into the same directory, and the two would interleave their
* status lines.
*/
private setBusy(busy: boolean): void {
this.busy = busy;
this.uploadButton.disabled = busy;
this.mkdirButton.disabled = busy;
}
private async promptRename(path: string, currentName: string): Promise<void> {
const name = await this.prompt('重命名', currentName);
if (name === null || name.trim() === '' || name === currentName) return;
@@ -428,15 +678,24 @@ export class ManagerScreen {
* directory listing, so the screen would look like the rename simply did not
* happen — the worst possible answer to "why did nothing change".
*/
private async mutate(action: () => Promise<unknown>, label: string): Promise<void> {
private async mutate(
action: () => Promise<unknown>,
label: string,
describe?: (result: never) => string,
): Promise<void> {
if (this.busy) return;
this.toast(`${label}…`);
let result: unknown;
try {
await action();
result = await action();
} catch (err) {
this.handleError(err);
return;
}
this.toast(`${label} 完成`);
// The server's own counts beat a generic "完成": a batch that touched two
// books out of five selected is a different outcome from one that touched
// five, and only the response knows which happened.
this.outcome = describe ? describe(result as never) : `${label} 完成`;
await this.load(this.current);
}
+12 -1
View File
@@ -41,7 +41,18 @@ export interface RecordedRequest {
url: string;
method: string;
headers: Record<string, string>;
body?: string;
/** String for the JSON calls, `FormData` for an upload. */
body?: string | FormData;
}
/** The recorded body as text, for the assertions that read JSON. */
export function bodyText(request: RecordedRequest | undefined): string {
const body = request?.body;
if (body === undefined) return '';
if (typeof body === 'string') return body;
// A `FormData` has no single text form; the upload tests read it field by
// field instead, which is what the server does.
return '';
}
export type Responder = (request: RecordedRequest) => HttpResponse | Promise<HttpResponse>;
+237
View File
@@ -251,6 +251,243 @@ describe('library manager', () => {
});
});
describe('uploading from the manager', () => {
/** A `File` the jsdom environment will accept. */
const file = (name: string, body = 'bytes'): File => new File([body], name, { type: 'application/epub+zip' });
const pick = async (screen: ManagerScreen, files: File[]): Promise<void> => {
const input = screen.element.querySelector<HTMLInputElement>('.manager-upload-input')!;
Object.defineProperty(input, 'files', { value: files, configurable: true });
input.dispatchEvent(new Event('change'));
};
it('asks what to do about a clash before sending any bytes', async () => {
const transport = new FakeTransport();
transport.json(listing({ entries: [entry({ name: '三体.epub', scanned: true })] }));
const { screen } = makeScreen(transport);
await screen.open();
void pick(screen, [file('三体.epub')]);
// The policy dialog appears first, so a user who has waited for a 400MB
// upload is never told afterwards that it was a name clash.
await vi.waitFor(() => expect(screen.element.querySelector('.dialog-list')).not.toBeNull());
const uploads = transport.requests.filter((request) => request.url.includes('upload'));
expect(uploads).toHaveLength(0);
});
it('sends the file, the destination and the policy, and reports what landed', async () => {
const transport = new FakeTransport();
transport.respondWith((request) =>
request.url.includes('upload')
? {
status: 200,
headers: {},
json: {
uploaded: [{ path: '科幻/三体.epub', originalName: '三体.epub', name: '三体.epub', size: 5, kind: 'file', bookId: 'b1' }],
skipped: [{ name: '坏.zip', reason: '无法解压' }],
scan: { added: 1, updated: 0, removed: 0, failed: 0, startedAt: 1, finishedAt: 2 },
},
}
: {
status: 200,
headers: {},
json:
new URL(`http://x${request.url}`).searchParams.get('path') === '科幻'
? listing({
path: '科幻',
name: '科幻',
parent: '',
crumbs: [
{ name: '书库', path: '' },
{ name: '科幻', path: '科幻' },
],
entries: [],
})
: listing({ entries: [entry({ name: '科幻', type: 'dir', path: '科幻' })] }),
},
);
const { screen } = makeScreen(transport);
await screen.open();
// Open the folder first: the upload goes into the directory being viewed,
// which is the whole contract of the feature.
(screen.element.querySelector('.manager-row') as HTMLElement).click();
await vi.waitFor(() => expect(screen.element.querySelector('.manager-crumb[aria-current="true"]')?.textContent).toBe('科幻'));
void pick(screen, [file('三体.epub')]);
await vi.waitFor(() => expect(screen.element.querySelector('.dialog-list')).not.toBeNull());
const rename = [...screen.element.querySelectorAll('.dialog-list .button')].find(
(button) => button.textContent === '两份都留(加 (2))',
) as HTMLElement;
rename.click();
await vi.waitFor(() => expect(screen.element.querySelector('.dialog-list')).toBeNull());
await vi.waitFor(() => {
expect(transport.requests.some((request) => request.url.includes('upload'))).toBe(true);
});
const request = transport.requests.find((r) => r.url.includes('upload'))!;
const form = request.body as FormData;
expect(form.get('path')).toBe('科幻');
expect(form.get('onConflict')).toBe('rename');
expect((form.get('file') as File).name).toBe('三体.epub');
// The status line names the file that was *skipped*: a count alone leaves the
// user with nothing to act on.
await vi.waitFor(() => {
expect(screen.element.querySelector('.manager-status')?.textContent).toContain('坏.zip');
});
});
it('does not offer an upload at all on a read-only mount', async () => {
const transport = new FakeTransport();
transport.json(listing({ writable: false, entries: [] }));
const { screen } = makeScreen(transport);
await screen.open();
const buttons = [...screen.element.querySelectorAll<HTMLButtonElement>('.icon-button')];
expect(buttons.find((button) => button.getAttribute('aria-label') === '上传书籍')?.hidden).toBe(true);
});
});
describe('batch management', () => {
const twoBooks = (): BrowseListing =>
listing({
entries: [
entry({ name: '卷一.epub', path: '卷一.epub', scanned: true, indexed: true }),
entry({ name: '卷二.epub', path: '卷二.epub', scanned: true, indexed: true }),
],
files: 2,
});
/** Selects every row through the action sheet, as a keyboard-less test can. */
const selectAll = async (screen: ManagerScreen): Promise<void> => {
for (const more of [...screen.element.querySelectorAll<HTMLElement>('.manager-more')]) {
more.click();
await vi.waitFor(() => expect(screen.element.querySelector('.dialog-list')).not.toBeNull());
const choose = [...screen.element.querySelectorAll('.dialog-list .button')].find(
(button) => button.textContent === '选择',
) as HTMLElement;
choose.click();
await vi.waitFor(() => expect(screen.element.querySelector('.dialog-list')).toBeNull());
}
};
it('sends one metadata patch for the whole selection', async () => {
const transport = new FakeTransport();
transport.respondWith((request) =>
request.url.includes('/browse/metadata')
? { status: 200, headers: {}, json: { applied: 2, books: ['a', 'b'], failed: [] } }
: { status: 200, headers: {}, json: twoBooks() },
);
const { screen } = makeScreen(transport);
await screen.open();
await selectAll(screen);
const metadata = [...screen.element.querySelectorAll('.manager-actions .button')].find(
(button) => button.textContent === '改资料',
) as HTMLElement;
metadata.click();
await vi.waitFor(() => expect(screen.element.querySelector('[placeholder="留空则不改"]')).not.toBeNull());
const author = screen.element.querySelector<HTMLInputElement>('.dialog input')!;
author.value = '某某';
screen.element.querySelector<HTMLFormElement>('form.dialog')!.dispatchEvent(
new Event('submit', { bubbles: true, cancelable: true }),
);
await vi.waitFor(() => {
expect(transport.requests.some((request) => request.url.endsWith('/browse/metadata'))).toBe(true);
});
const request = transport.requests.find((r) => r.url.endsWith('/browse/metadata'))!;
// Bottom to top, because the *last* row was appended second and the list is
// built in document order — what matters is that both were sent.
expect(JSON.parse(String(request.body))).toEqual({
paths: ['卷一.epub', '卷二.epub'],
fields: { author: '某某' },
});
// The server's count, not a generic "完成".
await vi.waitFor(() => {
expect(screen.element.querySelector('.manager-status')?.textContent).toContain('已更新 2 本');
});
});
it('takes books off the shelf only after saying the files are untouched', async () => {
const transport = new FakeTransport();
transport.respondWith((request) =>
request.url.includes('/browse/shelf')
? { status: 200, headers: {}, json: { applied: 2, books: ['a', 'b'], failed: [] } }
: { status: 200, headers: {}, json: twoBooks() },
);
const { screen } = makeScreen(transport);
await screen.open();
await selectAll(screen);
const shelve = [...screen.element.querySelectorAll('.manager-actions .button')].find(
(button) => button.textContent === '下架',
) as HTMLElement;
shelve.click();
// The confirmation has to say the files survive: this is one word away from a
// delete in the same bar, and the two mean opposite things.
await vi.waitFor(() => expect(screen.element.querySelector('.dialog p')?.textContent).toContain('文件一个都不会动'));
const confirm = [...screen.element.querySelectorAll('.dialog-actions .button')].find(
(button) => button.textContent === '删除',
) as HTMLElement;
confirm.click();
await vi.waitFor(() => {
expect(transport.requests.some((request) => request.url.endsWith('/browse/shelf'))).toBe(true);
});
expect(JSON.parse(String(transport.requests.find((r) => r.url.endsWith('/browse/shelf'))!.body))).toEqual({
paths: ['卷一.epub', '卷二.epub'],
action: 'remove',
});
// And no delete was ever sent.
expect(transport.requests.some((request) => request.url.endsWith('/browse/delete'))).toBe(false);
});
it('keeps the form actions out of the fields-scan scroller', async () => {
const transport = new FakeTransport();
transport.json(twoBooks());
const { screen } = makeScreen(transport);
await screen.open();
await selectAll(screen);
await vi.waitFor(() => expect(screen.element.querySelector('.manager-actions .button')).not.toBeNull());
([...screen.element.querySelectorAll('.manager-actions .button')].find(
(button) => button.textContent === '改资料',
) as HTMLElement).click();
await vi.waitFor(() => expect(screen.element.querySelector('.dialog-fields')).not.toBeNull());
// Seven optional fields do not fit on a phone, so the *fields* scroll and the
// actions do not. A dialog that scrolled whole put 保存 below the fold, which
// reads as a form with no way to commit it.
const fields = screen.element.querySelector('.dialog-fields')!;
const actions = screen.element.querySelector('.dialog-actions')!;
expect(fields.contains(actions)).toBe(false);
expect(fields.querySelectorAll('.dialog-field')).toHaveLength(7);
});
it('would rather say nothing than guess when a path is not a book', async () => {
const transport = new FakeTransport();
transport.respondWith((request) =>
request.url.includes('/browse/shelf')
? { status: 200, headers: {}, json: { applied: 0, books: [], failed: [{ path: '空目录', reason: 'NOT_A_BOOK' }] } }
: { status: 200, headers: {}, json: listing({ entries: [entry({ name: '空目录', type: 'dir', path: '空目录' })], writable: true }) },
);
const { screen } = makeScreen(transport);
await screen.open();
await selectAll(screen);
const shelve = [...screen.element.querySelectorAll('.manager-actions .button')].find(
(button) => button.textContent === '下架',
) as HTMLElement;
shelve.click();
await vi.waitFor(() => expect(screen.element.querySelector('.dialog-actions')).not.toBeNull());
([...screen.element.querySelectorAll('.dialog-actions .button')].find(
(button) => button.textContent === '删除',
) as HTMLElement).click();
await vi.waitFor(() => {
expect(screen.element.querySelector('.manager-status')?.textContent).toContain('不是书');
});
});
});
describe('error classification', () => {
it('does not treat a refused operation as a dead session', () => {
// Both are 403 on this API and only one is about the credentials. Classifying
+2 -2
View File
@@ -3,7 +3,7 @@ import { ReaderApi } from '../src/api/client.ts';
import { ApiError } from '../src/api/errors.ts';
import { SyncEngine } from '../src/core/sync.ts';
import { OfflineStore, isOutboxEmpty } from '../src/store/offline.ts';
import { FakeTransport, MemoryKv, makePlatform } from './helpers/env.ts';
import { FakeTransport, MemoryKv, bodyText, makePlatform } from './helpers/env.ts';
const SESSION = {
user: { id: 'u1', username: 'me', displayName: '我', role: 'member' as const, createdAt: 0 },
@@ -373,7 +373,7 @@ describe('SyncEngine', () => {
const sizes: number[] = [];
transport.respondWith((request) => {
if (request.method === 'POST') {
const body = JSON.parse(request.body ?? '{}') as { progress?: unknown[] };
const body = JSON.parse(bodyText(request)) as { progress?: unknown[] };
sizes.push(body.progress?.length ?? 0);
return { status: 200, headers: {}, json: { accepted: 1, rejected: 0, serverTime: 1, progress: [], notes: [] } };
}