mirror of
https://github.com/crosspoint-reader/crosspoint-reader.git
synced 2026-08-13 14:21:57 -07:00
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Justin Mitchell
|
79f5657fb2 | ||
|
Justin Mitchell
|
48aa3c8e02 |
No files matched your search
@@ -8,8 +8,6 @@ CrossPoint is open-source e-reader firmware - community-built, fully hackable, f
|
||||
|
||||
|
||||
|
||||
> If you're planning to buy an Xteink device, consider purchasing an **X3/X4 Developer Edition** through https://crosspointreader.com. CrossPoint receives a small share of each sale, helping fund development costs.
|
||||
|
||||
## What can CrossPoint do?
|
||||
|
||||
- **Reader engine**: EPUB 2/3 rendering with embedded-style option, image handling, hyphenation, kerning, chapter navigation, footnotes, bookmarks, go-to-percent, auto page turn, orientation control, focus reading, KOReader progress sync and more.
|
||||
@@ -69,6 +67,10 @@ USB port or browser before assuming the device is locked. Only reach for the unl
|
||||
> Flashing any other firmware on a USB-locked device may **permanently brick the device** or leave it **permanently
|
||||
> stuck on that firmware with no recovery path**. Once USB flashing is re-locked, your only way back is via OTA, and if
|
||||
> the firmware you flashed doesn't support OTA, **there is no way out**.
|
||||
>
|
||||
> **The Papyrix fork has removed OTA update support from its code.** If you flash Papyrix onto a
|
||||
> USB-locked unit, you will have **zero update or recovery path** and will be stuck on it forever. **Do not flash
|
||||
> Papyrix (or any other unsupported firmware) on a locked device.**
|
||||
|
||||
## Install firmware
|
||||
|
||||
@@ -129,6 +131,23 @@ Convert your own TTF/OTF files into `.cpfont` files that load from the SD card.
|
||||
|
||||
Conversion runs the firmware repo's `lib/EpdFont/scripts/fontconvert_sdcard.py` script unmodified, so output matches a local host build.
|
||||
|
||||
## Custom SD-card themes
|
||||
|
||||
Downloadable themes are packaged in the tools repo under `../crosspoint-tools/public/themes/<theme-id>/`. Each theme folder must contain a `theme.json`; optional assets such as generated BMP icons live beside it, usually under `icons/`.
|
||||
|
||||
See [SD-card theme creation](./docs/theme-creation.md) for the full JSON format, device-specific overrides, icon generation, CrossInk extension fields, and packaging rules.
|
||||
|
||||
After adding or changing a hosted theme, regenerate the download manifest:
|
||||
|
||||
```bash
|
||||
python3 scripts/generate-theme-manifest.py \
|
||||
--root ../crosspoint-tools/public/themes \
|
||||
--base-url http://crosspointreader.com/themes \
|
||||
--output ../crosspoint-tools/public/themes/themes.json
|
||||
```
|
||||
|
||||
The script scans every theme folder, includes every file in each package, and writes size and CRC32 values used by the device downloader. Commit changed theme package files and the regenerated `themes.json` in `crosspoint-tools`.
|
||||
|
||||
---
|
||||
|
||||
## Documentation
|
||||
@@ -136,6 +155,7 @@ Conversion runs the firmware repo's `lib/EpdFont/scripts/fontconvert_sdcard.py`
|
||||
- [User Guide](./USER_GUIDE.md)
|
||||
- [Web server usage](./docs/webserver.md)
|
||||
- [Web server endpoints](./docs/webserver-endpoints.md)
|
||||
- [SD-card theme creation](./docs/theme-creation.md)
|
||||
- [Project scope](./SCOPE.md)
|
||||
- [Contributing docs](./docs/contributing/README.md)
|
||||
|
||||
@@ -247,7 +267,7 @@ One of the best things about open source is that anyone can take the code in a d
|
||||
|
||||
- [papyrix-reader](https://github.com/bigbag/papyrix-reader) — Adds FB2 and MD format support. Actively maintained with Arabic script support. Custom themes via SD card.
|
||||
|
||||
- ~~[crosspet](https://github.com/trilwu/crosspet) — A Vietnamese fork that adds a Tamagotchi-style virtual chicken that grows based on your reading milestones (pages read, streaks, care). Also: Flashcards, Weather, Pomodoro timer, and mini-games.~~ (Unmaintained)
|
||||
- [crosspet](https://github.com/trilwu/crosspet) — A Vietnamese fork that adds a Tamagotchi-style virtual chicken that grows based on your reading milestones (pages read, streaks, care). Also: Flashcards, Weather, Pomodoro timer, and mini-games.
|
||||
|
||||
- [crosspoint-reader-cjk](https://github.com/aBER0724/crosspoint-reader-cjk) — Purpose-built for Chinese, Japanese, and Korean reading.
|
||||
|
||||
|
||||
+12
-21
@@ -5,7 +5,6 @@ Welcome to the **CrossPoint** firmware. This guide outlines the hardware control
|
||||
- [CrossPoint User Guide](#crosspoint-user-guide)
|
||||
- [1. Hardware Overview](#1-hardware-overview)
|
||||
- [Button Layout](#button-layout)
|
||||
- [Taking a Screenshot](#taking-a-screenshot)
|
||||
- [2. Power \& Startup](#2-power--startup)
|
||||
- [Power On / Off](#power-on--off)
|
||||
- [First Launch](#first-launch)
|
||||
@@ -15,11 +14,7 @@ Welcome to the **CrossPoint** firmware. This guide outlines the hardware control
|
||||
- [3.3 Browse Files Screen](#33-browse-files-screen)
|
||||
- [3.4 Recent Books Screen](#34-recent-books-screen)
|
||||
- [3.5 File Transfer Screen](#35-file-transfer-screen)
|
||||
- [3.5.1 Calibre Wireless Transfers](#351-calibre-wireless-transfers)
|
||||
- [Installing the Plugin in Calibre](#installing-the-plugin-in-calibre)
|
||||
- [Configuring the CrossPoint Plugin in Calibre](#configuring-the-crosspoint-plugin-in-calibre)
|
||||
- [Uploading Books](#uploading-books)
|
||||
- [Removing a Book](#removing-a-book)
|
||||
- [3.5.1 Calibre Wireless Transfers](#351-calibre-wireless-transfers)
|
||||
- [3.6 Settings](#36-settings)
|
||||
- [3.6.1 Display](#361-display)
|
||||
- [3.6.2 Reader](#362-reader)
|
||||
@@ -28,25 +23,21 @@ Welcome to the **CrossPoint** firmware. This guide outlines the hardware control
|
||||
- [3.6.5 OPDS Servers (Multiple Libraries)](#365-opds-servers-multiple-libraries)
|
||||
- [3.6.6 Web Settings (Wi-Fi + OPDS)](#366-web-settings-wi-fi--opds)
|
||||
- [3.6.7 KOReader Sync Quick Setup](#367-koreader-sync-quick-setup)
|
||||
- [Option A: Free Public Server (`sync.koreader.rocks`)](#option-a-free-public-server-synckoreaderrocks)
|
||||
- [Option B: Self-Hosted Server (Docker Compose)](#option-b-self-hosted-server-docker-compose)
|
||||
- [3.7 Sleep Screen](#37-sleep-screen)
|
||||
- [Cover settings](#cover-settings)
|
||||
- [Custom images](#custom-images)
|
||||
- [3.8 Custom Fonts (SD Card)](#38-custom-fonts-sd-card)
|
||||
- [4. Reading Mode](#4-reading-mode)
|
||||
- [Page Turning](#page-turning)
|
||||
- [Chapter Navigation](#chapter-navigation)
|
||||
- [Auto Page Turn](#auto-page-turn)
|
||||
- [Tilt Page Turn (X3 only)](#tilt-page-turn-x3-only)
|
||||
- [Footnote Navigation](#footnote-navigation)
|
||||
- [System Navigation](#system-navigation)
|
||||
- [Supported Languages](#supported-languages)
|
||||
- [Page Turning](#page-turning)
|
||||
- [Chapter Navigation](#chapter-navigation)
|
||||
- [Auto Page Turn](#auto-page-turn)
|
||||
- [Tilt Page Turn (X3 only)](#tilt-page-turn-x3-only)
|
||||
- [Footnote Navigation](#footnote-navigation)
|
||||
- [System Navigation](#system-navigation)
|
||||
- [Supported Languages](#supported-languages)
|
||||
- [5. Reader Menu](#5-reader-menu)
|
||||
- [5.1 Chapter Selection](#51-chapter-selection)
|
||||
- [5.2 Bookmarks](#52-bookmarks)
|
||||
- [6. Current Limitations \& Roadmap](#6-current-limitations--roadmap)
|
||||
- [7. Troubleshooting Issues \& Escaping Bootloop](#7-troubleshooting-issues--escaping-bootloop)
|
||||
- [5.1 Chapter Selection](#51-chapter-selection)
|
||||
- [5.2 Bookmarks](#52-bookmarks)
|
||||
- [6. Current Limitations & Roadmap](#6-current-limitations--roadmap)
|
||||
- [7. Troubleshooting Issues & Escaping Bootloop](#7-troubleshooting-issues--escaping-bootloop)
|
||||
|
||||
## 1. Hardware Overview
|
||||
|
||||
|
||||
+10
-22
@@ -90,13 +90,13 @@ if (parsedSize != fileSize) {
|
||||
|
||||
## `section.bin`
|
||||
|
||||
### Version 29
|
||||
### Version 25
|
||||
|
||||
Each file in `sections/*.bin` stores one laid-out spine section. The header is
|
||||
also the cache-busting key: if any layout-affecting setting differs from the
|
||||
current reader settings, the section is discarded and rebuilt.
|
||||
|
||||
Version 29 includes:
|
||||
Version 25 includes:
|
||||
|
||||
- cache-busting fields for paragraph alignment, hyphenation, embedded CSS,
|
||||
image rendering mode, and Focus Reading
|
||||
@@ -105,12 +105,6 @@ Version 29 includes:
|
||||
- paragraph and list-item LUTs used by KOReader sync page refinement
|
||||
- optional per-word Focus Reading split metadata
|
||||
- per-page footnote entries
|
||||
- serialized word style bits for underline, strikethrough, superscript, and
|
||||
subscript
|
||||
- flat TextBlock word storage (v29): per-word arrays plus one shared
|
||||
NUL-terminated text blob, replacing v28's length-prefixed word strings. The
|
||||
on-disk order mirrors the in-RAM arena so the firmware reads a whole block
|
||||
payload with a single allocation and a single SD read
|
||||
|
||||
ImHex pattern:
|
||||
|
||||
@@ -119,7 +113,7 @@ import std.mem;
|
||||
import std.string;
|
||||
import std.core;
|
||||
|
||||
#define EXPECTED_VERSION 29
|
||||
#define EXPECTED_VERSION 25
|
||||
#define MAX_STRING_LENGTH 65535
|
||||
#define FOOTNOTE_NUMBER_LEN 32
|
||||
#define FOOTNOTE_HREF_LEN 96
|
||||
@@ -180,20 +174,14 @@ struct BlockStyle {
|
||||
|
||||
struct TextBlock {
|
||||
u16 wordCount;
|
||||
u8 hasFocus;
|
||||
u16 textBytes [[comment("Total size of text[], including one NUL per word")]];
|
||||
String words[wordCount];
|
||||
s16 wordXPos[wordCount];
|
||||
WordStyle wordStyle[wordCount];
|
||||
|
||||
if (wordCount > 0) {
|
||||
u16 textOff[wordCount] [[comment("Byte offset of word i's text within text[]")]];
|
||||
s16 wordXPos[wordCount];
|
||||
if (hasFocus != 0) {
|
||||
u16 wordFocusSuffixX[wordCount] [[comment("Suffix x offset from word start")]];
|
||||
}
|
||||
WordStyle wordStyle[wordCount];
|
||||
if (hasFocus != 0) {
|
||||
u8 wordFocusBoundary[wordCount] [[comment("UTF-8 byte boundary between bold prefix and suffix")]];
|
||||
}
|
||||
char text[textBytes] [[comment("All words back to back, each NUL-terminated")]];
|
||||
u8 hasFocus;
|
||||
if (hasFocus != 0) {
|
||||
u8 wordFocusBoundary[wordCount] [[comment("UTF-8 byte boundary between bold prefix and suffix")]];
|
||||
u16 wordFocusSuffixX[wordCount] [[comment("Suffix x offset from word start")]];
|
||||
}
|
||||
|
||||
BlockStyle blockStyle;
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
+1
-1
Submodule freeink-sdk updated: fdd6105eb3...329a4bebef.
@@ -154,21 +154,6 @@ uint32_t EpdFont::applyLigatures(uint32_t cp, const char*& text) const {
|
||||
return cp;
|
||||
}
|
||||
|
||||
bool EpdFont::hasGlyph(const uint32_t cp) const {
|
||||
const int count = data->intervalCount;
|
||||
if (count > 0) {
|
||||
const EpdUnicodeInterval* intervals = data->intervals;
|
||||
const auto it =
|
||||
std::upper_bound(intervals, intervals + count, cp,
|
||||
[](uint32_t value, const EpdUnicodeInterval& interval) { return value < interval.first; });
|
||||
if (it != intervals && cp <= (it - 1)->last) return true;
|
||||
}
|
||||
if (data->glyphMissHandler) {
|
||||
return data->glyphMissHandler(data->glyphMissCtx, cp) != nullptr;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
const EpdGlyph* EpdFont::getGlyph(const uint32_t cp) const {
|
||||
const int count = data->intervalCount;
|
||||
if (count == 0 && !data->glyphMissHandler) return nullptr;
|
||||
|
||||
@@ -12,12 +12,6 @@ class EpdFont {
|
||||
|
||||
const EpdGlyph* getGlyph(uint32_t cp) const;
|
||||
|
||||
/// True when the font actually provides `cp` (interval table or on-demand
|
||||
/// SD load) — unlike getGlyph(), never satisfied by the replacement glyph.
|
||||
/// Used by layout shaping to decide whether a substitution (ligature,
|
||||
/// Arabic presentation form) can really be drawn.
|
||||
bool hasGlyph(uint32_t cp) const;
|
||||
|
||||
/// Returns the kerning adjustment (4.4 fixed-point in pixels) between two codepoints.
|
||||
/// Returns 0 if no kerning data exists for the pair.
|
||||
int8_t getKerning(uint32_t leftCp, uint32_t rightCp) const;
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
#include "EpdFontFamily.h"
|
||||
|
||||
const EpdFont* EpdFontFamily::getFont(const Style style) const {
|
||||
// Extract font style bits; render-time overlay bits do not affect font selection.
|
||||
// Extract font style bits (ignore UNDERLINE bit for font selection)
|
||||
const bool hasBold = (style & BOLD) != 0;
|
||||
const bool hasItalic = (style & ITALIC) != 0;
|
||||
|
||||
@@ -28,16 +28,10 @@ const EpdGlyph* EpdFontFamily::getGlyph(const uint32_t cp, const Style style) co
|
||||
return getFont(style)->getGlyph(cp);
|
||||
}
|
||||
|
||||
bool EpdFontFamily::hasGlyph(const uint32_t cp, const Style style) const { return getFont(style)->hasGlyph(cp); }
|
||||
|
||||
int8_t EpdFontFamily::getKerning(const uint32_t leftCp, const uint32_t rightCp, const Style style) const {
|
||||
return getFont(style)->getKerning(leftCp, rightCp);
|
||||
}
|
||||
|
||||
uint32_t EpdFontFamily::getLigature(const uint32_t leftCp, const uint32_t rightCp, const Style style) const {
|
||||
return getFont(style)->getLigature(leftCp, rightCp);
|
||||
}
|
||||
|
||||
uint32_t EpdFontFamily::applyLigatures(const uint32_t cp, const char*& text, const Style style) const {
|
||||
return getFont(style)->applyLigatures(cp, text);
|
||||
}
|
||||
@@ -17,7 +17,6 @@ class EpdFontFamily {
|
||||
SUP = 16, // superscript: glyph scaled 50%, raised ~40% of ascender
|
||||
SUB = 32, // subscript: glyph scaled 50%, lowered ~25% of ascender
|
||||
};
|
||||
static constexpr uint8_t TEXT_DECORATION_MASK = static_cast<uint8_t>(UNDERLINE | STRIKETHROUGH);
|
||||
|
||||
explicit EpdFontFamily(const EpdFont* regular, const EpdFont* bold = nullptr, const EpdFont* italic = nullptr,
|
||||
const EpdFont* boldItalic = nullptr)
|
||||
@@ -26,13 +25,8 @@ class EpdFontFamily {
|
||||
void getTextDimensions(const char* string, int* w, int* h, Style style = REGULAR) const;
|
||||
const EpdFontData* getData(Style style = REGULAR) const;
|
||||
const EpdGlyph* getGlyph(uint32_t cp, Style style = REGULAR) const;
|
||||
bool hasGlyph(uint32_t cp, Style style = REGULAR) const;
|
||||
int8_t getKerning(uint32_t leftCp, uint32_t rightCp, Style style = REGULAR) const;
|
||||
uint32_t getLigature(uint32_t leftCp, uint32_t rightCp, Style style = REGULAR) const;
|
||||
uint32_t applyLigatures(uint32_t cp, const char*& text, Style style = REGULAR) const;
|
||||
static constexpr bool hasTextDecoration(const Style style) {
|
||||
return (static_cast<uint8_t>(style) & TEXT_DECORATION_MASK) != 0;
|
||||
}
|
||||
|
||||
private:
|
||||
const EpdFont* regular;
|
||||
|
||||
@@ -33,24 +33,12 @@ void FontDecompressor::freePageBuffer() {
|
||||
}
|
||||
|
||||
void FontDecompressor::freeHotGroup() {
|
||||
free(hotGroup);
|
||||
hotGroup = nullptr;
|
||||
hotGroupCapacity = 0;
|
||||
hotGroup.clear();
|
||||
hotGroup.shrink_to_fit();
|
||||
hotGroupFont = nullptr;
|
||||
hotGroupIndex = UINT16_MAX;
|
||||
free(hotGlyphBuf);
|
||||
hotGlyphBuf = nullptr;
|
||||
hotGlyphBufCapacity = 0;
|
||||
}
|
||||
|
||||
bool FontDecompressor::ensureCapacity(uint8_t*& buf, uint32_t& capacity, uint32_t needed) {
|
||||
if (capacity >= needed) return true;
|
||||
// Grow-only, free-then-malloc: every caller fully rewrites the buffer after a grow, so the
|
||||
// old contents are dead -- freeing first gives the allocator its best shot on a tight heap.
|
||||
free(buf);
|
||||
buf = static_cast<uint8_t*>(malloc(needed)); // owned by FontDecompressor, freed in freeHotGroup()
|
||||
capacity = buf ? needed : 0;
|
||||
return buf != nullptr;
|
||||
hotGlyphBuf.clear();
|
||||
hotGlyphBuf.shrink_to_fit();
|
||||
}
|
||||
|
||||
uint16_t FontDecompressor::getGroupIndex(const EpdFontData* fontData, uint32_t glyphIndex) {
|
||||
@@ -182,20 +170,24 @@ const uint8_t* FontDecompressor::getBitmap(const EpdFontData* fontData, const Ep
|
||||
}
|
||||
|
||||
// Check if hot group already has this group decompressed — if not, decompress it
|
||||
if (!(hotGroup != nullptr && hotGroupFont == fontData && hotGroupIndex == groupIndex)) {
|
||||
if (!(!hotGroup.empty() && hotGroupFont == fontData && hotGroupIndex == groupIndex)) {
|
||||
stats.cacheMisses++;
|
||||
const EpdFontGroup& group = fontData->groups[groupIndex];
|
||||
|
||||
// ensureCapacity may free the buffer, so the cached-group identity dies with it either way.
|
||||
hotGroupFont = nullptr;
|
||||
hotGroupIndex = UINT16_MAX;
|
||||
if (!ensureCapacity(hotGroup, hotGroupCapacity, group.uncompressedSize)) {
|
||||
hotGroup.resize(group.uncompressedSize);
|
||||
if (hotGroup.empty()) {
|
||||
LOG_ERR("FDC", "Failed to allocate %u bytes for hot group %u", group.uncompressedSize, groupIndex);
|
||||
hotGroupFont = nullptr;
|
||||
hotGroupIndex = UINT16_MAX;
|
||||
stats.getBitmapTimeUs += micros() - tStart;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
if (!decompressGroup(fontData, groupIndex, hotGroup, group.uncompressedSize)) {
|
||||
if (!decompressGroup(fontData, groupIndex, hotGroup.data(), group.uncompressedSize)) {
|
||||
hotGroup.clear();
|
||||
hotGroup.shrink_to_fit();
|
||||
hotGroupFont = nullptr;
|
||||
hotGroupIndex = UINT16_MAX;
|
||||
stats.getBitmapTimeUs += micros() - tStart;
|
||||
return nullptr;
|
||||
}
|
||||
@@ -208,16 +200,18 @@ const uint8_t* FontDecompressor::getBitmap(const EpdFontData* fontData, const Ep
|
||||
}
|
||||
|
||||
// Compact just the requested glyph from byte-aligned data into scratch buffer
|
||||
if (!ensureCapacity(hotGlyphBuf, hotGlyphBufCapacity, glyph->dataLength)) {
|
||||
LOG_ERR("FDC", "Failed to allocate %u bytes for glyph scratch", (unsigned)glyph->dataLength);
|
||||
if (glyph->dataLength > hotGlyphBuf.size()) {
|
||||
hotGlyphBuf.resize(glyph->dataLength);
|
||||
}
|
||||
if (hotGlyphBuf.empty()) {
|
||||
stats.getBitmapTimeUs += micros() - tStart;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
uint32_t alignedOff = getAlignedOffset(fontData, groupIndex, glyphIndex);
|
||||
compactSingleGlyph(&hotGroup[alignedOff], hotGlyphBuf, glyph->width, glyph->height);
|
||||
compactSingleGlyph(&hotGroup[alignedOff], hotGlyphBuf.data(), glyph->width, glyph->height);
|
||||
stats.getBitmapTimeUs += micros() - tStart;
|
||||
return hotGlyphBuf;
|
||||
return hotGlyphBuf.data();
|
||||
}
|
||||
|
||||
// --- Prewarm: pre-decompress glyph bitmaps for a page of text ---
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
#include <InflateReader.h>
|
||||
|
||||
#include <vector>
|
||||
|
||||
#include "EpdFontData.h"
|
||||
|
||||
class FontDecompressor {
|
||||
@@ -65,22 +67,13 @@ class FontDecompressor {
|
||||
|
||||
// Hot group: last decompressed group (byte-aligned) for non-prewarmed fallback path.
|
||||
// Kept in byte-aligned format; individual glyphs are compacted on demand into hotGlyphBuf.
|
||||
// Nothrow high-water malloc buffers, NOT std::vector: getBitmap() runs on the render path,
|
||||
// and under -fno-exceptions a vector resize that hits OOM abort()s the firmware instead of
|
||||
// failing (field crash: hotGroup.resize() -> std::bad_alloc -> abort with ~11 KB free).
|
||||
// ensureCapacity() returns false on OOM so the caller can skip the glyph gracefully.
|
||||
const EpdFontData* hotGroupFont = nullptr;
|
||||
uint16_t hotGroupIndex = UINT16_MAX;
|
||||
uint8_t* hotGroup = nullptr; // owned; freed in freeHotGroup()/dtor
|
||||
uint32_t hotGroupCapacity = 0;
|
||||
std::vector<uint8_t> hotGroup;
|
||||
|
||||
// Scratch buffer for compacting a single glyph from the hot group.
|
||||
// Valid until the next getBitmap() call. Same ownership/OOM contract as hotGroup.
|
||||
uint8_t* hotGlyphBuf = nullptr;
|
||||
uint32_t hotGlyphBufCapacity = 0;
|
||||
|
||||
// Grow (never shrink) an owned buffer to at least `needed` bytes; false on OOM, buffer freed.
|
||||
static bool ensureCapacity(uint8_t*& buf, uint32_t& capacity, uint32_t needed);
|
||||
// Valid until the next getBitmap() call.
|
||||
std::vector<uint8_t> hotGlyphBuf;
|
||||
|
||||
void freePageBuffer();
|
||||
void freeHotGroup();
|
||||
|
||||
@@ -1061,28 +1061,6 @@ void SdCardFont::mergeIntoAdvanceTable(uint8_t styleIdx, const AdvanceEntry* sor
|
||||
advanceTableSize_[styleIdx] = k;
|
||||
}
|
||||
|
||||
bool SdCardFont::hasGlyphMeta(uint8_t styleIdx, uint32_t codepoint) const {
|
||||
styleIdx &= (MAX_STYLES - 1);
|
||||
const PerStyle& s = styles_[styleIdx];
|
||||
if (!loaded_ || !s.present) return false;
|
||||
return findGlobalGlyphIndex(s, codepoint) >= 0;
|
||||
}
|
||||
|
||||
uint16_t SdCardFont::ensureAdvance(uint8_t styleIdx, uint32_t codepoint) {
|
||||
styleIdx &= (MAX_STYLES - 1);
|
||||
if (!loaded_ || !styles_[styleIdx].present) return 0;
|
||||
const uint16_t cached = getAdvance(codepoint, styleIdx);
|
||||
if (cached != 0) return cached;
|
||||
// 0 is either "not in table" or a genuine zero-width glyph; only pay the
|
||||
// SD read when the resident interval table says the glyph exists.
|
||||
if (findGlobalGlyphIndex(styles_[styleIdx], codepoint) < 0) return 0;
|
||||
uint32_t cp = codepoint;
|
||||
fetchAdvancesForCodepoints(&cp, 1, static_cast<uint8_t>(1u << styleIdx));
|
||||
return getAdvance(codepoint, styleIdx);
|
||||
}
|
||||
|
||||
uint8_t SdCardFont::styleIdxFromMissCtx(void* ctx) { return static_cast<OverflowContext*>(ctx)->styleIdx; }
|
||||
|
||||
bool SdCardFont::hasAdvanceTable() const {
|
||||
for (uint8_t i = 0; i < MAX_STYLES; i++) {
|
||||
if (advanceTable_[i]) return true;
|
||||
|
||||
@@ -55,23 +55,6 @@ class SdCardFont {
|
||||
// Returns the 12.4 fixed-point advance, or 0 if not found.
|
||||
uint16_t getAdvance(uint32_t codepoint, uint8_t style) const;
|
||||
|
||||
// Layout-side metric access — the per-glyph path CpFontAdapter uses while
|
||||
// the FreeInkBook engine measures a chapter. Neither touches bitmap data:
|
||||
// going through getGlyph() would stream every occurrence's bitmap through
|
||||
// the 8-slot overflow ring (one SD read per character — a crawling build).
|
||||
//
|
||||
// hasGlyphMeta answers existence from the resident interval table (no I/O).
|
||||
bool hasGlyphMeta(uint8_t styleIdx, uint32_t codepoint) const;
|
||||
// ensureAdvance returns the 12.4 fixed-point advanceX, fetching the glyph
|
||||
// METADATA from SD once on a miss and merging it into the persistent
|
||||
// advance table (so each unique codepoint costs one small read per
|
||||
// session). Returns 0 when the style lacks the codepoint.
|
||||
uint16_t ensureAdvance(uint8_t styleIdx, uint32_t codepoint);
|
||||
|
||||
// Recover the style index from an EpdFontData::glyphMissCtx (see
|
||||
// fromMissCtx below for recovering the SdCardFont itself).
|
||||
static uint8_t styleIdxFromMissCtx(void* ctx);
|
||||
|
||||
// Returns true if advance table is populated for at least one style.
|
||||
bool hasAdvanceTable() const;
|
||||
|
||||
|
||||
@@ -33,7 +33,6 @@ import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import socket
|
||||
import urllib.request
|
||||
from concurrent.futures import ProcessPoolExecutor, as_completed
|
||||
from pathlib import Path
|
||||
@@ -50,44 +49,17 @@ INSTANCE_DIR = SCRIPT_DIR / "instanced_fonts"
|
||||
DEFAULT_FALLBACK_FONT = EPDFONTS_DIR / "builtinFonts/source/NotoSans/NotoSans-Regular.ttf"
|
||||
|
||||
|
||||
_orig_getaddrinfo = socket.getaddrinfo
|
||||
|
||||
|
||||
def _ipv4_only_getaddrinfo(*args, **kwargs):
|
||||
"""getaddrinfo variant that drops AAAA records (IPv4 only)."""
|
||||
return [ai for ai in _orig_getaddrinfo(*args, **kwargs) if ai[0] == socket.AF_INET]
|
||||
|
||||
|
||||
def download_font(url: str, dest: Path, retries: int = 3) -> Path:
|
||||
"""Download a font file if not already cached. Returns the local path.
|
||||
|
||||
Some sources (e.g. mirrors.ctan.org) are round-robin redirectors that land
|
||||
on a different mirror each request; a mirror may advertise an IPv6 address a
|
||||
host without an IPv6 route cannot reach ([Errno 101] Network is unreachable).
|
||||
Retry on failure, forcing IPv4 resolution after the first attempt.
|
||||
"""
|
||||
def download_font(url: str, dest: Path) -> Path:
|
||||
"""Download a font file if not already cached. Returns the local path."""
|
||||
if dest.exists():
|
||||
return dest
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
print(f" Downloading {dest.name}...")
|
||||
last_err = None
|
||||
for attempt in range(1, retries + 1):
|
||||
force_ipv4 = attempt > 1
|
||||
if force_ipv4:
|
||||
socket.getaddrinfo = _ipv4_only_getaddrinfo
|
||||
try:
|
||||
urllib.request.urlretrieve(url, dest)
|
||||
break
|
||||
except Exception as e: # noqa: BLE001 - reported via RuntimeError below
|
||||
last_err = e
|
||||
dest.unlink(missing_ok=True)
|
||||
if attempt < retries:
|
||||
print(f" Attempt {attempt} failed ({e}); retrying (IPv4-only)...")
|
||||
finally:
|
||||
if force_ipv4:
|
||||
socket.getaddrinfo = _orig_getaddrinfo
|
||||
else:
|
||||
raise RuntimeError(f"Failed to download {url}: {last_err}") from last_err
|
||||
try:
|
||||
urllib.request.urlretrieve(url, dest)
|
||||
except Exception as e:
|
||||
dest.unlink(missing_ok=True)
|
||||
raise RuntimeError(f"Failed to download {url}: {e}") from e
|
||||
size_kb = dest.stat().st_size / 1024
|
||||
print(f" Downloaded {dest.name} ({size_kb:.0f} KB)")
|
||||
return dest
|
||||
|
||||
@@ -132,16 +132,7 @@ families:
|
||||
bold: {url: "https://raw.githubusercontent.com/impallari/Libre-Baskerville/master/fonts/ttf/LibreBaskerville-Bold.ttf"}
|
||||
italic: {url: "https://raw.githubusercontent.com/impallari/Libre-Baskerville/master/fonts/ttf/LibreBaskerville-Italic.ttf"}
|
||||
bolditalic: {url: "https://raw.githubusercontent.com/impallari/Libre-Baskerville/master/fonts/ttf/LibreBaskerville-BoldItalic.ttf"}
|
||||
|
||||
- name: Vollkorn
|
||||
description: "A serif for bread and butter use by Friedrich Althausen (Latin, Greek, Cyrillic)"
|
||||
intervals: latin-ext,greek,cyrillic
|
||||
sizes: [12, 14, 16, 18]
|
||||
styles:
|
||||
regular: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/vollkorn/Vollkorn%5Bwght%5D.ttf", variable: {wght: 400}}
|
||||
bold: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/vollkorn/Vollkorn%5Bwght%5D.ttf", variable: {wght: 700}}
|
||||
italic: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/vollkorn/Vollkorn-Italic%5Bwght%5D.ttf", variable: {wght: 400}}
|
||||
bolditalic: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/vollkorn/Vollkorn-Italic%5Bwght%5D.ttf", variable: {wght: 700}}
|
||||
|
||||
|
||||
# ── Sans-serif ─────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,77 @@
|
||||
#pragma once
|
||||
|
||||
#include <Print.h>
|
||||
|
||||
#include <memory>
|
||||
#include <string>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
|
||||
#include "Epub/BookMetadataCache.h"
|
||||
#include "Epub/css/CssParser.h"
|
||||
|
||||
class ZipFile;
|
||||
|
||||
class Epub {
|
||||
// the ncx file (EPUB 2)
|
||||
std::string tocNcxItem;
|
||||
// the nav file (EPUB 3)
|
||||
std::string tocNavItem;
|
||||
// where is the EPUBfile?
|
||||
std::string filepath;
|
||||
// the base path for items in the EPUB file
|
||||
std::string contentBasePath;
|
||||
// Uniq cache key based on filepath
|
||||
std::string cachePath;
|
||||
// Spine and TOC cache
|
||||
std::unique_ptr<BookMetadataCache> bookMetadataCache;
|
||||
// CSS parser for styling
|
||||
std::unique_ptr<CssParser> cssParser;
|
||||
// CSS files
|
||||
std::vector<std::string> cssFiles;
|
||||
|
||||
bool findContentOpfFile(std::string* contentOpfFile) const;
|
||||
bool parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata, bool writeSpineEntries = true);
|
||||
bool parseTocNcxFile() const;
|
||||
bool parseTocNavFile() const;
|
||||
void discoverCssFilesFromZip();
|
||||
void parseCssFiles() const;
|
||||
|
||||
public:
|
||||
explicit Epub(std::string filepath, const std::string& cacheDir) : filepath(std::move(filepath)) {
|
||||
// create a cache key based on the filepath
|
||||
cachePath = cacheDir + "/epub_" + std::to_string(std::hash<std::string>{}(this->filepath));
|
||||
}
|
||||
~Epub() = default;
|
||||
std::string& getBasePath() { return contentBasePath; }
|
||||
bool load(bool buildIfMissing = true, bool skipLoadingCss = false);
|
||||
bool clearCache() const;
|
||||
void setupCacheDir() const;
|
||||
const std::string& getCachePath() const;
|
||||
const std::string& getPath() const;
|
||||
const std::string& getTitle() const;
|
||||
const std::string& getAuthor() const;
|
||||
const std::string& getLanguage() const;
|
||||
std::string getCoverBmpPath(bool cropped = false) const;
|
||||
bool generateCoverBmp(bool cropped = false) const;
|
||||
std::string getThumbBmpPath() const;
|
||||
std::string getThumbBmpPath(int height) const;
|
||||
bool generateThumbBmp(int height) const;
|
||||
uint8_t* readItemContentsToBytes(const std::string& itemHref, size_t* size = nullptr,
|
||||
bool trailingNullByte = false) const;
|
||||
bool readItemContentsToStream(const std::string& itemHref, Print& out, size_t chunkSize) const;
|
||||
bool getItemSize(const std::string& itemHref, size_t* size) const;
|
||||
BookMetadataCache::SpineEntry getSpineItem(int spineIndex) const;
|
||||
BookMetadataCache::TocEntry getTocItem(int tocIndex) const;
|
||||
int getSpineItemsCount() const;
|
||||
int getTocItemsCount() const;
|
||||
int getSpineIndexForTocIndex(int tocIndex) const;
|
||||
int getTocIndexForSpineIndex(int spineIndex) const;
|
||||
size_t getCumulativeSpineItemSize(int spineIndex) const;
|
||||
int getSpineIndexForTextReference() const;
|
||||
|
||||
size_t getBookSize() const;
|
||||
float calculateProgress(int currentSpineIndex, float currentSpineRead) const;
|
||||
CssParser* getCssParser() const { return cssParser.get(); }
|
||||
int resolveHrefToSpineIndex(const std::string& href) const;
|
||||
};
|
||||
@@ -0,0 +1,460 @@
|
||||
#include "BookMetadataCache.h"
|
||||
|
||||
#include <Logging.h>
|
||||
#include <Serialization.h>
|
||||
#include <Utf8.h>
|
||||
#include <ZipFile.h>
|
||||
|
||||
#include <deque>
|
||||
|
||||
#include "FsHelpers.h"
|
||||
|
||||
namespace {
|
||||
constexpr uint8_t BOOK_CACHE_VERSION = 8; // v8: TOC/book titles stored NFC-composed
|
||||
constexpr char bookBinFile[] = "/book.bin";
|
||||
constexpr char tmpSpineBinFile[] = "/spine.bin.tmp";
|
||||
constexpr char tmpTocBinFile[] = "/toc.bin.tmp";
|
||||
} // namespace
|
||||
|
||||
/* ============= WRITING / BUILDING FUNCTIONS ================ */
|
||||
|
||||
bool BookMetadataCache::beginWrite() {
|
||||
buildMode = true;
|
||||
spineCount = 0;
|
||||
tocCount = 0;
|
||||
LOG_DBG("BMC", "Entering write mode");
|
||||
return true;
|
||||
}
|
||||
|
||||
bool BookMetadataCache::beginContentOpfPass() {
|
||||
LOG_DBG("BMC", "Beginning content opf pass");
|
||||
|
||||
// Open spine file for writing
|
||||
return Storage.openFileForWrite("BMC", cachePath + tmpSpineBinFile, spineFile);
|
||||
}
|
||||
|
||||
bool BookMetadataCache::endContentOpfPass() {
|
||||
// Explicit close() required: member variable persists beyond function scope
|
||||
spineFile.close();
|
||||
return true;
|
||||
}
|
||||
|
||||
bool BookMetadataCache::beginTocPass() {
|
||||
LOG_DBG("BMC", "Beginning toc pass");
|
||||
|
||||
if (!Storage.openFileForRead("BMC", cachePath + tmpSpineBinFile, spineFile)) {
|
||||
return false;
|
||||
}
|
||||
if (!Storage.openFileForWrite("BMC", cachePath + tmpTocBinFile, tocFile)) {
|
||||
// Explicit close() required: member variable persists beyond function scope
|
||||
spineFile.close();
|
||||
return false;
|
||||
}
|
||||
|
||||
if (spineCount >= LARGE_SPINE_THRESHOLD) {
|
||||
spineHrefIndex.clear();
|
||||
spineHrefIndex.resize(spineCount);
|
||||
spineFile.seek(0);
|
||||
for (int i = 0; i < spineCount; i++) {
|
||||
auto entry = readSpineEntry(spineFile);
|
||||
SpineHrefIndexEntry idx;
|
||||
idx.hrefHash = fnvHash64(entry.href);
|
||||
idx.hrefLen = static_cast<uint16_t>(entry.href.size());
|
||||
idx.spineIndex = static_cast<int16_t>(i);
|
||||
spineHrefIndex[i] = idx;
|
||||
}
|
||||
std::sort(spineHrefIndex.begin(), spineHrefIndex.end(),
|
||||
[](const SpineHrefIndexEntry& a, const SpineHrefIndexEntry& b) {
|
||||
return a.hrefHash < b.hrefHash || (a.hrefHash == b.hrefHash && a.hrefLen < b.hrefLen);
|
||||
});
|
||||
spineFile.seek(0);
|
||||
useSpineHrefIndex = true;
|
||||
LOG_DBG("BMC", "Using fast index for %d spine items", spineCount);
|
||||
} else {
|
||||
useSpineHrefIndex = false;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool BookMetadataCache::endTocPass() {
|
||||
// Explicit close() required: member variables persist beyond function scope
|
||||
tocFile.close();
|
||||
spineFile.close();
|
||||
|
||||
spineHrefIndex.clear();
|
||||
spineHrefIndex.shrink_to_fit();
|
||||
useSpineHrefIndex = false;
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
bool BookMetadataCache::endWrite() {
|
||||
if (!buildMode) {
|
||||
LOG_DBG("BMC", "endWrite called but not in build mode");
|
||||
return false;
|
||||
}
|
||||
|
||||
buildMode = false;
|
||||
LOG_DBG("BMC", "Wrote %d spine, %d TOC entries", spineCount, tocCount);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool BookMetadataCache::buildBookBin(const std::string& epubPath, const BookMetadata& metadata) {
|
||||
// Open all three files, writing to meta, reading from spine and toc
|
||||
if (!Storage.openFileForWrite("BMC", cachePath + bookBinFile, bookFile)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!Storage.openFileForRead("BMC", cachePath + tmpSpineBinFile, spineFile)) {
|
||||
// Explicit close() required: member variable persists beyond function scope
|
||||
bookFile.close();
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!Storage.openFileForRead("BMC", cachePath + tmpTocBinFile, tocFile)) {
|
||||
// Explicit close() required: member variables persist beyond function scope
|
||||
bookFile.close();
|
||||
spineFile.close();
|
||||
return false;
|
||||
}
|
||||
|
||||
constexpr uint32_t headerASize =
|
||||
sizeof(BOOK_CACHE_VERSION) + /* LUT Offset */ sizeof(uint32_t) + sizeof(spineCount) + sizeof(tocCount);
|
||||
const uint32_t metadataSize = metadata.title.size() + metadata.author.size() + metadata.language.size() +
|
||||
metadata.coverItemHref.size() + metadata.textReferenceHref.size() +
|
||||
sizeof(uint32_t) * 5;
|
||||
const uint32_t lutSize = sizeof(uint32_t) * spineCount + sizeof(uint32_t) * tocCount;
|
||||
const uint32_t lutOffset = headerASize + metadataSize;
|
||||
|
||||
// Header A
|
||||
serialization::writePod(bookFile, BOOK_CACHE_VERSION);
|
||||
serialization::writePod(bookFile, lutOffset);
|
||||
serialization::writePod(bookFile, spineCount);
|
||||
serialization::writePod(bookFile, tocCount);
|
||||
// Metadata
|
||||
serialization::writeString(bookFile, metadata.title);
|
||||
serialization::writeString(bookFile, metadata.author);
|
||||
serialization::writeString(bookFile, metadata.language);
|
||||
serialization::writeString(bookFile, metadata.coverItemHref);
|
||||
serialization::writeString(bookFile, metadata.textReferenceHref);
|
||||
|
||||
// Loop through spine entries, writing LUT positions
|
||||
spineFile.seek(0);
|
||||
for (int i = 0; i < spineCount; i++) {
|
||||
uint32_t pos = spineFile.position();
|
||||
auto spineEntry = readSpineEntry(spineFile);
|
||||
serialization::writePod(bookFile, pos + lutOffset + lutSize);
|
||||
}
|
||||
|
||||
// Loop through toc entries, writing LUT positions
|
||||
tocFile.seek(0);
|
||||
for (int i = 0; i < tocCount; i++) {
|
||||
uint32_t pos = tocFile.position();
|
||||
auto tocEntry = readTocEntry(tocFile);
|
||||
serialization::writePod(bookFile, pos + lutOffset + lutSize + static_cast<uint32_t>(spineFile.position()));
|
||||
}
|
||||
|
||||
// LUTs complete
|
||||
// Loop through spines from spine file matching up TOC indexes, calculating cumulative size and writing to book.bin
|
||||
|
||||
// Build spineIndex->tocIndex mapping in one pass (O(n) instead of O(n*m))
|
||||
std::deque<int16_t> spineToTocIndex(spineCount, -1);
|
||||
tocFile.seek(0);
|
||||
for (int j = 0; j < tocCount; j++) {
|
||||
auto tocEntry = readTocEntry(tocFile);
|
||||
if (tocEntry.spineIndex >= 0 && tocEntry.spineIndex < spineCount) {
|
||||
if (spineToTocIndex[tocEntry.spineIndex] == -1) {
|
||||
spineToTocIndex[tocEntry.spineIndex] = static_cast<int16_t>(j);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
ZipFile zip(epubPath);
|
||||
// Pre-open zip file to speed up size calculations
|
||||
if (!zip.open()) {
|
||||
LOG_ERR("BMC", "Could not open EPUB zip for size calculations");
|
||||
// Explicit close() required: member variables persist beyond function scope
|
||||
bookFile.close();
|
||||
spineFile.close();
|
||||
tocFile.close();
|
||||
return false;
|
||||
}
|
||||
// NOTE: We intentionally skip calling loadAllFileStatSlims() here.
|
||||
// For large EPUBs (2000+ chapters), pre-loading all ZIP central directory entries
|
||||
// into memory causes OOM crashes on ESP32-C3's limited ~380KB RAM.
|
||||
// Instead, for large books we use a one-pass batch lookup that scans the ZIP
|
||||
// central directory once and matches against spine targets using hash comparison.
|
||||
// This is O(n*log(m)) instead of O(n*m) while avoiding memory exhaustion.
|
||||
// See: https://github.com/crosspoint-reader/crosspoint-reader/issues/134
|
||||
|
||||
std::deque<uint32_t> spineSizes;
|
||||
bool useBatchSizes = false;
|
||||
|
||||
if (spineCount >= LARGE_SPINE_THRESHOLD) {
|
||||
LOG_DBG("BMC", "Using batch size lookup for %d spine items", spineCount);
|
||||
|
||||
std::deque<ZipFile::SizeTarget> targets;
|
||||
targets.resize(spineCount);
|
||||
|
||||
spineFile.seek(0);
|
||||
for (int i = 0; i < spineCount; i++) {
|
||||
auto entry = readSpineEntry(spineFile);
|
||||
std::string path = FsHelpers::normalisePath(entry.href);
|
||||
|
||||
ZipFile::SizeTarget t;
|
||||
t.hash = ZipFile::fnvHash64(path.c_str(), path.size());
|
||||
t.len = static_cast<uint16_t>(path.size());
|
||||
t.index = static_cast<uint16_t>(i);
|
||||
targets[i] = t;
|
||||
}
|
||||
|
||||
std::sort(targets.begin(), targets.end(), [](const ZipFile::SizeTarget& a, const ZipFile::SizeTarget& b) {
|
||||
return a.hash < b.hash || (a.hash == b.hash && a.len < b.len);
|
||||
});
|
||||
|
||||
spineSizes.resize(spineCount, 0);
|
||||
int matched = zip.fillUncompressedSizes(targets, spineSizes);
|
||||
LOG_DBG("BMC", "Batch lookup matched %d/%d spine items", matched, spineCount);
|
||||
|
||||
targets.clear();
|
||||
targets.shrink_to_fit();
|
||||
|
||||
useBatchSizes = true;
|
||||
}
|
||||
|
||||
uint32_t cumSize = 0;
|
||||
spineFile.seek(0);
|
||||
int lastSpineTocIndex = -1;
|
||||
for (int i = 0; i < spineCount; i++) {
|
||||
auto spineEntry = readSpineEntry(spineFile);
|
||||
|
||||
spineEntry.tocIndex = spineToTocIndex[i];
|
||||
|
||||
// Not a huge deal if we don't fine a TOC entry for the spine entry, this is expected behaviour for EPUBs
|
||||
// Logging here is for debugging
|
||||
if (spineEntry.tocIndex == -1) {
|
||||
LOG_DBG("BMC", "Warning: Could not find TOC entry for spine item %d: %s, using title from last section", i,
|
||||
spineEntry.href.c_str());
|
||||
spineEntry.tocIndex = lastSpineTocIndex;
|
||||
}
|
||||
lastSpineTocIndex = spineEntry.tocIndex;
|
||||
|
||||
size_t itemSize = 0;
|
||||
if (useBatchSizes) {
|
||||
itemSize = spineSizes[i];
|
||||
if (itemSize == 0) {
|
||||
const std::string path = FsHelpers::normalisePath(spineEntry.href);
|
||||
if (!zip.getInflatedFileSize(path.c_str(), &itemSize)) {
|
||||
LOG_ERR("BMC", "Warning: Could not get size for spine item: %s", path.c_str());
|
||||
}
|
||||
}
|
||||
} else {
|
||||
const std::string path = FsHelpers::normalisePath(spineEntry.href);
|
||||
if (!zip.getInflatedFileSize(path.c_str(), &itemSize)) {
|
||||
LOG_ERR("BMC", "Warning: Could not get size for spine item: %s", path.c_str());
|
||||
}
|
||||
}
|
||||
|
||||
cumSize += itemSize;
|
||||
spineEntry.cumulativeSize = cumSize;
|
||||
|
||||
// Write out spine data to book.bin
|
||||
writeSpineEntry(bookFile, spineEntry);
|
||||
}
|
||||
// Close opened zip file
|
||||
zip.close();
|
||||
|
||||
// Loop through toc entries from toc file writing to book.bin
|
||||
tocFile.seek(0);
|
||||
for (int i = 0; i < tocCount; i++) {
|
||||
auto tocEntry = readTocEntry(tocFile);
|
||||
writeTocEntry(bookFile, tocEntry);
|
||||
}
|
||||
|
||||
// Explicit close() required: member variables persist beyond function scope
|
||||
bookFile.close();
|
||||
spineFile.close();
|
||||
tocFile.close();
|
||||
|
||||
LOG_DBG("BMC", "Successfully built book.bin");
|
||||
return true;
|
||||
}
|
||||
|
||||
bool BookMetadataCache::cleanupTmpFiles() const {
|
||||
const auto spineBinFile = cachePath + tmpSpineBinFile;
|
||||
if (Storage.exists(spineBinFile.c_str())) {
|
||||
Storage.remove(spineBinFile.c_str());
|
||||
}
|
||||
const auto tocBinFile = cachePath + tmpTocBinFile;
|
||||
if (Storage.exists(tocBinFile.c_str())) {
|
||||
Storage.remove(tocBinFile.c_str());
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
uint32_t BookMetadataCache::writeSpineEntry(HalFile& file, const SpineEntry& entry) const {
|
||||
const uint32_t pos = file.position();
|
||||
serialization::writeString(file, entry.href);
|
||||
serialization::writePod(file, entry.cumulativeSize);
|
||||
serialization::writePod(file, entry.tocIndex);
|
||||
return pos;
|
||||
}
|
||||
|
||||
uint32_t BookMetadataCache::writeTocEntry(HalFile& file, const TocEntry& entry) const {
|
||||
const uint32_t pos = file.position();
|
||||
serialization::writeString(file, entry.title);
|
||||
serialization::writeString(file, entry.href);
|
||||
serialization::writeString(file, entry.anchor);
|
||||
serialization::writePod(file, entry.level);
|
||||
serialization::writePod(file, entry.spineIndex);
|
||||
return pos;
|
||||
}
|
||||
|
||||
// Note: for the LUT to be accurate, this **MUST** be called for all spine items before `addTocEntry` is ever called
|
||||
// this is because in this function we're marking positions of the items
|
||||
void BookMetadataCache::createSpineEntry(const std::string& href) {
|
||||
if (!buildMode || !spineFile) {
|
||||
LOG_DBG("BMC", "createSpineEntry called but not in build mode");
|
||||
return;
|
||||
}
|
||||
|
||||
const SpineEntry entry(href, 0, -1);
|
||||
writeSpineEntry(spineFile, entry);
|
||||
spineCount++;
|
||||
}
|
||||
|
||||
void BookMetadataCache::createTocEntry(const std::string& title, const std::string& href, const std::string& anchor,
|
||||
const uint8_t level) {
|
||||
if (!buildMode || !tocFile || !spineFile) {
|
||||
LOG_DBG("BMC", "createTocEntry called but not in build mode");
|
||||
return;
|
||||
}
|
||||
|
||||
int16_t spineIndex = -1;
|
||||
|
||||
if (useSpineHrefIndex) {
|
||||
uint64_t targetHash = fnvHash64(href);
|
||||
uint16_t targetLen = static_cast<uint16_t>(href.size());
|
||||
|
||||
auto it =
|
||||
std::lower_bound(spineHrefIndex.begin(), spineHrefIndex.end(), SpineHrefIndexEntry{targetHash, targetLen, 0},
|
||||
[](const SpineHrefIndexEntry& a, const SpineHrefIndexEntry& b) {
|
||||
return a.hrefHash < b.hrefHash || (a.hrefHash == b.hrefHash && a.hrefLen < b.hrefLen);
|
||||
});
|
||||
|
||||
while (it != spineHrefIndex.end() && it->hrefHash == targetHash && it->hrefLen == targetLen) {
|
||||
spineIndex = it->spineIndex;
|
||||
break;
|
||||
}
|
||||
|
||||
if (spineIndex == -1) {
|
||||
LOG_DBG("BMC", "createTocEntry: Could not find spine item for TOC href %s", href.c_str());
|
||||
}
|
||||
} else {
|
||||
spineFile.seek(0);
|
||||
for (int i = 0; i < spineCount; i++) {
|
||||
auto spineEntry = readSpineEntry(spineFile);
|
||||
if (spineEntry.href == href) {
|
||||
spineIndex = static_cast<int16_t>(i);
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (spineIndex == -1) {
|
||||
LOG_DBG("BMC", "createTocEntry: Could not find spine item for TOC href %s", href.c_str());
|
||||
}
|
||||
}
|
||||
|
||||
// Compose the title to NFC at index time so the cache stores precomposed glyphs;
|
||||
// device fonts have no combining-mark positioning, so NFD titles render broken.
|
||||
const TocEntry entry(utf8ComposeNfc(title), href, anchor, level, spineIndex);
|
||||
writeTocEntry(tocFile, entry);
|
||||
tocCount++;
|
||||
}
|
||||
|
||||
/* ============= READING / LOADING FUNCTIONS ================ */
|
||||
|
||||
bool BookMetadataCache::load() {
|
||||
if (!Storage.openFileForRead("BMC", cachePath + bookBinFile, bookFile)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
uint8_t version;
|
||||
serialization::readPod(bookFile, version);
|
||||
if (version != BOOK_CACHE_VERSION) {
|
||||
LOG_DBG("BMC", "Cache version mismatch: expected %d, got %d", BOOK_CACHE_VERSION, version);
|
||||
// Explicit close() required: member variable persists beyond function scope
|
||||
bookFile.close();
|
||||
return false;
|
||||
}
|
||||
|
||||
serialization::readPod(bookFile, lutOffset);
|
||||
serialization::readPod(bookFile, spineCount);
|
||||
serialization::readPod(bookFile, tocCount);
|
||||
|
||||
serialization::readString(bookFile, coreMetadata.title);
|
||||
serialization::readString(bookFile, coreMetadata.author);
|
||||
serialization::readString(bookFile, coreMetadata.language);
|
||||
serialization::readString(bookFile, coreMetadata.coverItemHref);
|
||||
serialization::readString(bookFile, coreMetadata.textReferenceHref);
|
||||
|
||||
loaded = true;
|
||||
LOG_DBG("BMC", "Loaded cache data: %d spine, %d TOC entries", spineCount, tocCount);
|
||||
return true;
|
||||
}
|
||||
|
||||
BookMetadataCache::SpineEntry BookMetadataCache::getSpineEntry(const int index) {
|
||||
if (!loaded) {
|
||||
LOG_ERR("BMC", "getSpineEntry called but cache not loaded");
|
||||
return {};
|
||||
}
|
||||
|
||||
if (index < 0 || index >= static_cast<int>(spineCount)) {
|
||||
LOG_ERR("BMC", "getSpineEntry index %d out of range", index);
|
||||
return {};
|
||||
}
|
||||
|
||||
// Seek to spine LUT item, read from LUT and get out data
|
||||
bookFile.seek(lutOffset + sizeof(uint32_t) * index);
|
||||
uint32_t spineEntryPos;
|
||||
serialization::readPod(bookFile, spineEntryPos);
|
||||
bookFile.seek(spineEntryPos);
|
||||
return readSpineEntry(bookFile);
|
||||
}
|
||||
|
||||
BookMetadataCache::TocEntry BookMetadataCache::getTocEntry(const int index) {
|
||||
if (!loaded) {
|
||||
LOG_ERR("BMC", "getTocEntry called but cache not loaded");
|
||||
return {};
|
||||
}
|
||||
|
||||
if (index < 0 || index >= static_cast<int>(tocCount)) {
|
||||
LOG_ERR("BMC", "getTocEntry index %d out of range", index);
|
||||
return {};
|
||||
}
|
||||
|
||||
// Seek to TOC LUT item, read from LUT and get out data
|
||||
bookFile.seek(lutOffset + sizeof(uint32_t) * spineCount + sizeof(uint32_t) * index);
|
||||
uint32_t tocEntryPos;
|
||||
serialization::readPod(bookFile, tocEntryPos);
|
||||
bookFile.seek(tocEntryPos);
|
||||
return readTocEntry(bookFile);
|
||||
}
|
||||
|
||||
BookMetadataCache::SpineEntry BookMetadataCache::readSpineEntry(HalFile& file) const {
|
||||
SpineEntry entry;
|
||||
serialization::readString(file, entry.href);
|
||||
serialization::readPod(file, entry.cumulativeSize);
|
||||
serialization::readPod(file, entry.tocIndex);
|
||||
return entry;
|
||||
}
|
||||
|
||||
BookMetadataCache::TocEntry BookMetadataCache::readTocEntry(HalFile& file) const {
|
||||
TocEntry entry;
|
||||
serialization::readString(file, entry.title);
|
||||
serialization::readString(file, entry.href);
|
||||
serialization::readString(file, entry.anchor);
|
||||
serialization::readPod(file, entry.level);
|
||||
serialization::readPod(file, entry.spineIndex);
|
||||
return entry;
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
#pragma once
|
||||
|
||||
#include <HalStorage.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <deque>
|
||||
#include <string>
|
||||
|
||||
class BookMetadataCache {
|
||||
public:
|
||||
struct BookMetadata {
|
||||
std::string title;
|
||||
std::string author;
|
||||
std::string language;
|
||||
std::string coverItemHref;
|
||||
std::string textReferenceHref;
|
||||
};
|
||||
|
||||
struct SpineEntry {
|
||||
std::string href;
|
||||
uint32_t cumulativeSize;
|
||||
int16_t tocIndex;
|
||||
|
||||
SpineEntry() : cumulativeSize(0), tocIndex(-1) {}
|
||||
SpineEntry(std::string href, const uint32_t cumulativeSize, const int16_t tocIndex)
|
||||
: href(std::move(href)), cumulativeSize(cumulativeSize), tocIndex(tocIndex) {}
|
||||
};
|
||||
|
||||
struct TocEntry {
|
||||
std::string title;
|
||||
std::string href;
|
||||
std::string anchor;
|
||||
uint8_t level;
|
||||
int16_t spineIndex;
|
||||
|
||||
TocEntry() : level(0), spineIndex(-1) {}
|
||||
TocEntry(std::string title, std::string href, std::string anchor, const uint8_t level, const int16_t spineIndex)
|
||||
: title(std::move(title)),
|
||||
href(std::move(href)),
|
||||
anchor(std::move(anchor)),
|
||||
level(level),
|
||||
spineIndex(spineIndex) {}
|
||||
};
|
||||
|
||||
private:
|
||||
std::string cachePath;
|
||||
uint32_t lutOffset;
|
||||
uint16_t spineCount;
|
||||
uint16_t tocCount;
|
||||
bool loaded;
|
||||
bool buildMode;
|
||||
|
||||
HalFile bookFile;
|
||||
// Temp file handles during build
|
||||
HalFile spineFile;
|
||||
HalFile tocFile;
|
||||
|
||||
// Index for fast href→spineIndex lookup (used only for large EPUBs)
|
||||
struct SpineHrefIndexEntry {
|
||||
uint64_t hrefHash; // FNV-1a 64-bit hash
|
||||
uint16_t hrefLen; // length for collision reduction
|
||||
int16_t spineIndex;
|
||||
};
|
||||
std::deque<SpineHrefIndexEntry> spineHrefIndex;
|
||||
bool useSpineHrefIndex = false;
|
||||
|
||||
static constexpr uint16_t LARGE_SPINE_THRESHOLD = 400;
|
||||
|
||||
// FNV-1a 64-bit hash function
|
||||
static uint64_t fnvHash64(const std::string& s) {
|
||||
uint64_t hash = 14695981039346656037ull;
|
||||
for (char c : s) {
|
||||
hash ^= static_cast<uint8_t>(c);
|
||||
hash *= 1099511628211ull;
|
||||
}
|
||||
return hash;
|
||||
}
|
||||
|
||||
uint32_t writeSpineEntry(HalFile& file, const SpineEntry& entry) const;
|
||||
uint32_t writeTocEntry(HalFile& file, const TocEntry& entry) const;
|
||||
SpineEntry readSpineEntry(HalFile& file) const;
|
||||
TocEntry readTocEntry(HalFile& file) const;
|
||||
|
||||
public:
|
||||
BookMetadata coreMetadata;
|
||||
|
||||
explicit BookMetadataCache(std::string cachePath)
|
||||
: cachePath(std::move(cachePath)), lutOffset(0), spineCount(0), tocCount(0), loaded(false), buildMode(false) {}
|
||||
~BookMetadataCache() = default;
|
||||
|
||||
// Building phase (stream to disk immediately)
|
||||
bool beginWrite();
|
||||
bool beginContentOpfPass();
|
||||
void createSpineEntry(const std::string& href);
|
||||
bool endContentOpfPass();
|
||||
bool beginTocPass();
|
||||
void createTocEntry(const std::string& title, const std::string& href, const std::string& anchor, uint8_t level);
|
||||
bool endTocPass();
|
||||
bool endWrite();
|
||||
bool cleanupTmpFiles() const;
|
||||
|
||||
// Post-processing to update mappings and sizes
|
||||
bool buildBookBin(const std::string& epubPath, const BookMetadata& metadata);
|
||||
|
||||
// Reading phase (read mode)
|
||||
bool load();
|
||||
SpineEntry getSpineEntry(int index);
|
||||
TocEntry getTocEntry(int index);
|
||||
int getSpineCount() const { return spineCount; }
|
||||
int getTocCount() const { return tocCount; }
|
||||
bool isLoaded() const { return loaded; }
|
||||
};
|
||||
File renamed without changes.
Loaded 20 of 261 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user