Compare commits

...
Author SHA1 Message Date
Justin Mitchell a328fd912d Add SD catalog as default EPUB container backend
Changes the container backend strategy to use SD catalog by default for all EPUBs. On first open or stale index, builds the catalog once by borrowing the framebuffer (which can be released since e-ink holds the loading popup image). Subsequent opens use the fast path with compact resident tables. Falls back to legacy in-RAM open if catalog build fails due to IoError or OOM.
2026-07-08 14:08:15 -04:00
Justin Mitchell e3f2e27078 Add catalog-backed book cover extraction support
Implement cover extraction for books stored in catalog.fibc omnibus format to avoid loading large (~400KB) book arenas into RAM. Uses catalog's pre-resolved cover entries with minimal memory footprint (~42KB resident). Includes arena management for catalog opening and ZIP entry extraction with configurable scratch buffers.
2026-07-08 13:51:58 -04:00
Justin Mitchell 9bb488dc86 Add catalog mode for large EPUB containers
Implements a two-tier system for handling EPUB books: small books use in-RAM metadata (probe ladder with 48KB/72KB/96KB rungs), while large containers (e.g. webnovel omnibuses with 1500+ spine items) automatically fall back to SD-backed BookCatalog with persistent index. The catalog fast path reuses existing catalog.fibc files to skip probing on subsequent opens. Adds framebuffer borrowing for larger probe rungs and incremental chapter building for giant chapters.
2026-07-08 13:41:16 -04:00
Justin Mitchell c331651e4e Update freeink-sdk submodule 2026-07-08 05:43:38 -04:00
Justin Mitchell bfb8aacdae Add framebuffer release and font metadata access
Introduce methods to temporarily release the 48KB framebuffer during memory-intensive chapter builds, allowing the memory to be reused. Add font metadata query methods (hasGlyphMeta, ensureAdvance) that check glyph existence and fetch advance metrics without loading full bitmap data, avoiding expensive SD reads during layout measurement.
2026-07-08 04:46:25 -04:00
Justin Mitchell e027f60bf0 Optimize memory allocation in BookPaginator
Replace fixed-size arena buffers with two-pass allocation strategy. First pass parses book into temporary full-size arena to determine actual memory footprint (5-40 KB vs 48 KB cap), then reallocates exact-sized buffers. This reduces peak memory usage by avoiding simultaneous allocation of all arenas with chapter-build scratch buffer (~100 KB) on constrained heap (~140 KB free). Stylesheet is also compacted to actual size, and all open-time transients are freed before per-chapter arenas are allocated.
2026-07-08 02:17:29 -04:00
Justin Mitchell 78c71f1c77 Extract book cover utilities into BookCoverUtils
Refactors book cover and metadata handling out of Epub class into a dedicated BookCoverUtils utility. Moves FootnoteEntry struct into reader activity directory. Simplifies cache clearing to use direct storage operations instead of Epub class.
2026-07-08 01:31:24 -04:00
Justin Mitchell d364e54a69 Add KOReader XPath sync support for EPUB navigation
Implements bidirectional conversion between KOReader XPath positions and FreeInkBook character offsets. Uses streaming SAX parsing to match chapter text accounting with ChapterLayout, enabling position sync without requiring DOM or pagination. Handles element ancestry tracking, text offset calculation, and spine index extraction.
2026-07-08 01:11:29 -04:00
Justin Mitchell 3421171670 Add character-offset bookmarks for FreeInkBook
Implements layout-independent bookmarks using character offsets within chapters instead of percentage-based positions. This allows bookmarks to restore to the exact position regardless of font size or orientation changes. Adds TOC navigation helpers, book progress calculation based on spine item sizes, and a NONE BidiBaseDir option for pre-reordered text. Legacy bookmarks without character offsets fall back to percentage-based positioning.
2026-07-08 00:56:53 -04:00
Justin Mitchell 0848649ce2 Add FreeInkBook EPUB engine migration guide
Document migration plan from CrossPoint's lib/Epub to FreeInkBook, a memory-efficient clean-room EPUB engine. Covers advantages (O(paragraph+page) memory via SAX parsing, arena allocation, host-testable, full UAX#14 typography, runtime TTF/OTF support), 9-step migration strategy, ESP32-C3 constraints, and integration approach preserving existing UI layer while replacing parsing/layout stack.
2026-07-08 00:22:21 -04:00
165 changed files with 4464 additions and 94402 deletions

No files matched your search

+15
View File
@@ -154,6 +154,21 @@ uint32_t EpdFont::applyLigatures(uint32_t cp, const char*& text) const {
return cp;
}
bool EpdFont::hasGlyph(const uint32_t cp) const {
const int count = data->intervalCount;
if (count > 0) {
const EpdUnicodeInterval* intervals = data->intervals;
const auto it =
std::upper_bound(intervals, intervals + count, cp,
[](uint32_t value, const EpdUnicodeInterval& interval) { return value < interval.first; });
if (it != intervals && cp <= (it - 1)->last) return true;
}
if (data->glyphMissHandler) {
return data->glyphMissHandler(data->glyphMissCtx, cp) != nullptr;
}
return false;
}
const EpdGlyph* EpdFont::getGlyph(const uint32_t cp) const {
const int count = data->intervalCount;
if (count == 0 && !data->glyphMissHandler) return nullptr;
+6
View File
@@ -12,6 +12,12 @@ class EpdFont {
const EpdGlyph* getGlyph(uint32_t cp) const;
/// True when the font actually provides `cp` (interval table or on-demand
/// SD load) — unlike getGlyph(), never satisfied by the replacement glyph.
/// Used by layout shaping to decide whether a substitution (ligature,
/// Arabic presentation form) can really be drawn.
bool hasGlyph(uint32_t cp) const;
/// Returns the kerning adjustment (4.4 fixed-point in pixels) between two codepoints.
/// Returns 0 if no kerning data exists for the pair.
int8_t getKerning(uint32_t leftCp, uint32_t rightCp) const;
+6
View File
@@ -28,10 +28,16 @@ const EpdGlyph* EpdFontFamily::getGlyph(const uint32_t cp, const Style style) co
return getFont(style)->getGlyph(cp);
}
bool EpdFontFamily::hasGlyph(const uint32_t cp, const Style style) const { return getFont(style)->hasGlyph(cp); }
int8_t EpdFontFamily::getKerning(const uint32_t leftCp, const uint32_t rightCp, const Style style) const {
return getFont(style)->getKerning(leftCp, rightCp);
}
uint32_t EpdFontFamily::getLigature(const uint32_t leftCp, const uint32_t rightCp, const Style style) const {
return getFont(style)->getLigature(leftCp, rightCp);
}
uint32_t EpdFontFamily::applyLigatures(const uint32_t cp, const char*& text, const Style style) const {
return getFont(style)->applyLigatures(cp, text);
}
+2
View File
@@ -26,7 +26,9 @@ class EpdFontFamily {
void getTextDimensions(const char* string, int* w, int* h, Style style = REGULAR) const;
const EpdFontData* getData(Style style = REGULAR) const;
const EpdGlyph* getGlyph(uint32_t cp, Style style = REGULAR) const;
bool hasGlyph(uint32_t cp, Style style = REGULAR) const;
int8_t getKerning(uint32_t leftCp, uint32_t rightCp, Style style = REGULAR) const;
uint32_t getLigature(uint32_t leftCp, uint32_t rightCp, Style style = REGULAR) const;
uint32_t applyLigatures(uint32_t cp, const char*& text, Style style = REGULAR) const;
static constexpr bool hasTextDecoration(const Style style) {
return (static_cast<uint8_t>(style) & TEXT_DECORATION_MASK) != 0;
+22
View File
@@ -1061,6 +1061,28 @@ void SdCardFont::mergeIntoAdvanceTable(uint8_t styleIdx, const AdvanceEntry* sor
advanceTableSize_[styleIdx] = k;
}
bool SdCardFont::hasGlyphMeta(uint8_t styleIdx, uint32_t codepoint) const {
styleIdx &= (MAX_STYLES - 1);
const PerStyle& s = styles_[styleIdx];
if (!loaded_ || !s.present) return false;
return findGlobalGlyphIndex(s, codepoint) >= 0;
}
uint16_t SdCardFont::ensureAdvance(uint8_t styleIdx, uint32_t codepoint) {
styleIdx &= (MAX_STYLES - 1);
if (!loaded_ || !styles_[styleIdx].present) return 0;
const uint16_t cached = getAdvance(codepoint, styleIdx);
if (cached != 0) return cached;
// 0 is either "not in table" or a genuine zero-width glyph; only pay the
// SD read when the resident interval table says the glyph exists.
if (findGlobalGlyphIndex(styles_[styleIdx], codepoint) < 0) return 0;
uint32_t cp = codepoint;
fetchAdvancesForCodepoints(&cp, 1, static_cast<uint8_t>(1u << styleIdx));
return getAdvance(codepoint, styleIdx);
}
uint8_t SdCardFont::styleIdxFromMissCtx(void* ctx) { return static_cast<OverflowContext*>(ctx)->styleIdx; }
bool SdCardFont::hasAdvanceTable() const {
for (uint8_t i = 0; i < MAX_STYLES; i++) {
if (advanceTable_[i]) return true;
+17
View File
@@ -55,6 +55,23 @@ class SdCardFont {
// Returns the 12.4 fixed-point advance, or 0 if not found.
uint16_t getAdvance(uint32_t codepoint, uint8_t style) const;
// Layout-side metric access — the per-glyph path CpFontAdapter uses while
// the FreeInkBook engine measures a chapter. Neither touches bitmap data:
// going through getGlyph() would stream every occurrence's bitmap through
// the 8-slot overflow ring (one SD read per character — a crawling build).
//
// hasGlyphMeta answers existence from the resident interval table (no I/O).
bool hasGlyphMeta(uint8_t styleIdx, uint32_t codepoint) const;
// ensureAdvance returns the 12.4 fixed-point advanceX, fetching the glyph
// METADATA from SD once on a miss and merging it into the persistent
// advance table (so each unique codepoint costs one small read per
// session). Returns 0 when the style lacks the codepoint.
uint16_t ensureAdvance(uint8_t styleIdx, uint32_t codepoint);
// Recover the style index from an EpdFontData::glyphMissCtx (see
// fromMissCtx below for recovering the SdCardFont itself).
static uint8_t styleIdxFromMissCtx(void* ctx);
// Returns true if advance table is populated for at least one style.
bool hasAdvanceTable() const;
-927
View File
File diff suppressed because it is too large. Load diff
-77
View File
@@ -1,77 +0,0 @@
#pragma once
#include <Print.h>
#include <memory>
#include <string>
#include <unordered_map>
#include <vector>
#include "Epub/BookMetadataCache.h"
#include "Epub/css/CssParser.h"
class ZipFile;
class Epub {
// the ncx file (EPUB 2)
std::string tocNcxItem;
// the nav file (EPUB 3)
std::string tocNavItem;
// where is the EPUBfile?
std::string filepath;
// the base path for items in the EPUB file
std::string contentBasePath;
// Uniq cache key based on filepath
std::string cachePath;
// Spine and TOC cache
std::unique_ptr<BookMetadataCache> bookMetadataCache;
// CSS parser for styling
std::unique_ptr<CssParser> cssParser;
// CSS files
std::vector<std::string> cssFiles;
bool findContentOpfFile(std::string* contentOpfFile) const;
bool parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata, bool writeSpineEntries = true);
bool parseTocNcxFile() const;
bool parseTocNavFile() const;
void discoverCssFilesFromZip();
void parseCssFiles() const;
public:
explicit Epub(std::string filepath, const std::string& cacheDir) : filepath(std::move(filepath)) {
// create a cache key based on the filepath
cachePath = cacheDir + "/epub_" + std::to_string(std::hash<std::string>{}(this->filepath));
}
~Epub() = default;
std::string& getBasePath() { return contentBasePath; }
bool load(bool buildIfMissing = true, bool skipLoadingCss = false);
bool clearCache() const;
void setupCacheDir() const;
const std::string& getCachePath() const;
const std::string& getPath() const;
const std::string& getTitle() const;
const std::string& getAuthor() const;
const std::string& getLanguage() const;
std::string getCoverBmpPath(bool cropped = false) const;
bool generateCoverBmp(bool cropped = false) const;
std::string getThumbBmpPath() const;
std::string getThumbBmpPath(int height) const;
bool generateThumbBmp(int height) const;
uint8_t* readItemContentsToBytes(const std::string& itemHref, size_t* size = nullptr,
bool trailingNullByte = false) const;
bool readItemContentsToStream(const std::string& itemHref, Print& out, size_t chunkSize) const;
bool getItemSize(const std::string& itemHref, size_t* size) const;
BookMetadataCache::SpineEntry getSpineItem(int spineIndex) const;
BookMetadataCache::TocEntry getTocItem(int tocIndex) const;
int getSpineItemsCount() const;
int getTocItemsCount() const;
int getSpineIndexForTocIndex(int tocIndex) const;
int getTocIndexForSpineIndex(int spineIndex) const;
size_t getCumulativeSpineItemSize(int spineIndex) const;
int getSpineIndexForTextReference() const;
size_t getBookSize() const;
float calculateProgress(int currentSpineIndex, float currentSpineRead) const;
CssParser* getCssParser() const { return cssParser.get(); }
int resolveHrefToSpineIndex(const std::string& href) const;
};
-460
View File
@@ -1,460 +0,0 @@
#include "BookMetadataCache.h"
#include <Logging.h>
#include <Serialization.h>
#include <Utf8.h>
#include <ZipFile.h>
#include <deque>
#include "FsHelpers.h"
namespace {
constexpr uint8_t BOOK_CACHE_VERSION = 8; // v8: TOC/book titles stored NFC-composed
constexpr char bookBinFile[] = "/book.bin";
constexpr char tmpSpineBinFile[] = "/spine.bin.tmp";
constexpr char tmpTocBinFile[] = "/toc.bin.tmp";
} // namespace
/* ============= WRITING / BUILDING FUNCTIONS ================ */
bool BookMetadataCache::beginWrite() {
buildMode = true;
spineCount = 0;
tocCount = 0;
LOG_DBG("BMC", "Entering write mode");
return true;
}
bool BookMetadataCache::beginContentOpfPass() {
LOG_DBG("BMC", "Beginning content opf pass");
// Open spine file for writing
return Storage.openFileForWrite("BMC", cachePath + tmpSpineBinFile, spineFile);
}
bool BookMetadataCache::endContentOpfPass() {
// Explicit close() required: member variable persists beyond function scope
spineFile.close();
return true;
}
bool BookMetadataCache::beginTocPass() {
LOG_DBG("BMC", "Beginning toc pass");
if (!Storage.openFileForRead("BMC", cachePath + tmpSpineBinFile, spineFile)) {
return false;
}
if (!Storage.openFileForWrite("BMC", cachePath + tmpTocBinFile, tocFile)) {
// Explicit close() required: member variable persists beyond function scope
spineFile.close();
return false;
}
if (spineCount >= LARGE_SPINE_THRESHOLD) {
spineHrefIndex.clear();
spineHrefIndex.resize(spineCount);
spineFile.seek(0);
for (int i = 0; i < spineCount; i++) {
auto entry = readSpineEntry(spineFile);
SpineHrefIndexEntry idx;
idx.hrefHash = fnvHash64(entry.href);
idx.hrefLen = static_cast<uint16_t>(entry.href.size());
idx.spineIndex = static_cast<int16_t>(i);
spineHrefIndex[i] = idx;
}
std::sort(spineHrefIndex.begin(), spineHrefIndex.end(),
[](const SpineHrefIndexEntry& a, const SpineHrefIndexEntry& b) {
return a.hrefHash < b.hrefHash || (a.hrefHash == b.hrefHash && a.hrefLen < b.hrefLen);
});
spineFile.seek(0);
useSpineHrefIndex = true;
LOG_DBG("BMC", "Using fast index for %d spine items", spineCount);
} else {
useSpineHrefIndex = false;
}
return true;
}
bool BookMetadataCache::endTocPass() {
// Explicit close() required: member variables persist beyond function scope
tocFile.close();
spineFile.close();
spineHrefIndex.clear();
spineHrefIndex.shrink_to_fit();
useSpineHrefIndex = false;
return true;
}
bool BookMetadataCache::endWrite() {
if (!buildMode) {
LOG_DBG("BMC", "endWrite called but not in build mode");
return false;
}
buildMode = false;
LOG_DBG("BMC", "Wrote %d spine, %d TOC entries", spineCount, tocCount);
return true;
}
bool BookMetadataCache::buildBookBin(const std::string& epubPath, const BookMetadata& metadata) {
// Open all three files, writing to meta, reading from spine and toc
if (!Storage.openFileForWrite("BMC", cachePath + bookBinFile, bookFile)) {
return false;
}
if (!Storage.openFileForRead("BMC", cachePath + tmpSpineBinFile, spineFile)) {
// Explicit close() required: member variable persists beyond function scope
bookFile.close();
return false;
}
if (!Storage.openFileForRead("BMC", cachePath + tmpTocBinFile, tocFile)) {
// Explicit close() required: member variables persist beyond function scope
bookFile.close();
spineFile.close();
return false;
}
constexpr uint32_t headerASize =
sizeof(BOOK_CACHE_VERSION) + /* LUT Offset */ sizeof(uint32_t) + sizeof(spineCount) + sizeof(tocCount);
const uint32_t metadataSize = metadata.title.size() + metadata.author.size() + metadata.language.size() +
metadata.coverItemHref.size() + metadata.textReferenceHref.size() +
sizeof(uint32_t) * 5;
const uint32_t lutSize = sizeof(uint32_t) * spineCount + sizeof(uint32_t) * tocCount;
const uint32_t lutOffset = headerASize + metadataSize;
// Header A
serialization::writePod(bookFile, BOOK_CACHE_VERSION);
serialization::writePod(bookFile, lutOffset);
serialization::writePod(bookFile, spineCount);
serialization::writePod(bookFile, tocCount);
// Metadata
serialization::writeString(bookFile, metadata.title);
serialization::writeString(bookFile, metadata.author);
serialization::writeString(bookFile, metadata.language);
serialization::writeString(bookFile, metadata.coverItemHref);
serialization::writeString(bookFile, metadata.textReferenceHref);
// Loop through spine entries, writing LUT positions
spineFile.seek(0);
for (int i = 0; i < spineCount; i++) {
uint32_t pos = spineFile.position();
auto spineEntry = readSpineEntry(spineFile);
serialization::writePod(bookFile, pos + lutOffset + lutSize);
}
// Loop through toc entries, writing LUT positions
tocFile.seek(0);
for (int i = 0; i < tocCount; i++) {
uint32_t pos = tocFile.position();
auto tocEntry = readTocEntry(tocFile);
serialization::writePod(bookFile, pos + lutOffset + lutSize + static_cast<uint32_t>(spineFile.position()));
}
// LUTs complete
// Loop through spines from spine file matching up TOC indexes, calculating cumulative size and writing to book.bin
// Build spineIndex->tocIndex mapping in one pass (O(n) instead of O(n*m))
std::deque<int16_t> spineToTocIndex(spineCount, -1);
tocFile.seek(0);
for (int j = 0; j < tocCount; j++) {
auto tocEntry = readTocEntry(tocFile);
if (tocEntry.spineIndex >= 0 && tocEntry.spineIndex < spineCount) {
if (spineToTocIndex[tocEntry.spineIndex] == -1) {
spineToTocIndex[tocEntry.spineIndex] = static_cast<int16_t>(j);
}
}
}
ZipFile zip(epubPath);
// Pre-open zip file to speed up size calculations
if (!zip.open()) {
LOG_ERR("BMC", "Could not open EPUB zip for size calculations");
// Explicit close() required: member variables persist beyond function scope
bookFile.close();
spineFile.close();
tocFile.close();
return false;
}
// NOTE: We intentionally skip calling loadAllFileStatSlims() here.
// For large EPUBs (2000+ chapters), pre-loading all ZIP central directory entries
// into memory causes OOM crashes on ESP32-C3's limited ~380KB RAM.
// Instead, for large books we use a one-pass batch lookup that scans the ZIP
// central directory once and matches against spine targets using hash comparison.
// This is O(n*log(m)) instead of O(n*m) while avoiding memory exhaustion.
// See: https://github.com/crosspoint-reader/crosspoint-reader/issues/134
std::deque<uint32_t> spineSizes;
bool useBatchSizes = false;
if (spineCount >= LARGE_SPINE_THRESHOLD) {
LOG_DBG("BMC", "Using batch size lookup for %d spine items", spineCount);
std::deque<ZipFile::SizeTarget> targets;
targets.resize(spineCount);
spineFile.seek(0);
for (int i = 0; i < spineCount; i++) {
auto entry = readSpineEntry(spineFile);
std::string path = FsHelpers::normalisePath(entry.href);
ZipFile::SizeTarget t;
t.hash = ZipFile::fnvHash64(path.c_str(), path.size());
t.len = static_cast<uint16_t>(path.size());
t.index = static_cast<uint16_t>(i);
targets[i] = t;
}
std::sort(targets.begin(), targets.end(), [](const ZipFile::SizeTarget& a, const ZipFile::SizeTarget& b) {
return a.hash < b.hash || (a.hash == b.hash && a.len < b.len);
});
spineSizes.resize(spineCount, 0);
int matched = zip.fillUncompressedSizes(targets, spineSizes);
LOG_DBG("BMC", "Batch lookup matched %d/%d spine items", matched, spineCount);
targets.clear();
targets.shrink_to_fit();
useBatchSizes = true;
}
uint32_t cumSize = 0;
spineFile.seek(0);
int lastSpineTocIndex = -1;
for (int i = 0; i < spineCount; i++) {
auto spineEntry = readSpineEntry(spineFile);
spineEntry.tocIndex = spineToTocIndex[i];
// Not a huge deal if we don't fine a TOC entry for the spine entry, this is expected behaviour for EPUBs
// Logging here is for debugging
if (spineEntry.tocIndex == -1) {
LOG_DBG("BMC", "Warning: Could not find TOC entry for spine item %d: %s, using title from last section", i,
spineEntry.href.c_str());
spineEntry.tocIndex = lastSpineTocIndex;
}
lastSpineTocIndex = spineEntry.tocIndex;
size_t itemSize = 0;
if (useBatchSizes) {
itemSize = spineSizes[i];
if (itemSize == 0) {
const std::string path = FsHelpers::normalisePath(spineEntry.href);
if (!zip.getInflatedFileSize(path.c_str(), &itemSize)) {
LOG_ERR("BMC", "Warning: Could not get size for spine item: %s", path.c_str());
}
}
} else {
const std::string path = FsHelpers::normalisePath(spineEntry.href);
if (!zip.getInflatedFileSize(path.c_str(), &itemSize)) {
LOG_ERR("BMC", "Warning: Could not get size for spine item: %s", path.c_str());
}
}
cumSize += itemSize;
spineEntry.cumulativeSize = cumSize;
// Write out spine data to book.bin
writeSpineEntry(bookFile, spineEntry);
}
// Close opened zip file
zip.close();
// Loop through toc entries from toc file writing to book.bin
tocFile.seek(0);
for (int i = 0; i < tocCount; i++) {
auto tocEntry = readTocEntry(tocFile);
writeTocEntry(bookFile, tocEntry);
}
// Explicit close() required: member variables persist beyond function scope
bookFile.close();
spineFile.close();
tocFile.close();
LOG_DBG("BMC", "Successfully built book.bin");
return true;
}
bool BookMetadataCache::cleanupTmpFiles() const {
const auto spineBinFile = cachePath + tmpSpineBinFile;
if (Storage.exists(spineBinFile.c_str())) {
Storage.remove(spineBinFile.c_str());
}
const auto tocBinFile = cachePath + tmpTocBinFile;
if (Storage.exists(tocBinFile.c_str())) {
Storage.remove(tocBinFile.c_str());
}
return true;
}
uint32_t BookMetadataCache::writeSpineEntry(HalFile& file, const SpineEntry& entry) const {
const uint32_t pos = file.position();
serialization::writeString(file, entry.href);
serialization::writePod(file, entry.cumulativeSize);
serialization::writePod(file, entry.tocIndex);
return pos;
}
uint32_t BookMetadataCache::writeTocEntry(HalFile& file, const TocEntry& entry) const {
const uint32_t pos = file.position();
serialization::writeString(file, entry.title);
serialization::writeString(file, entry.href);
serialization::writeString(file, entry.anchor);
serialization::writePod(file, entry.level);
serialization::writePod(file, entry.spineIndex);
return pos;
}
// Note: for the LUT to be accurate, this **MUST** be called for all spine items before `addTocEntry` is ever called
// this is because in this function we're marking positions of the items
void BookMetadataCache::createSpineEntry(const std::string& href) {
if (!buildMode || !spineFile) {
LOG_DBG("BMC", "createSpineEntry called but not in build mode");
return;
}
const SpineEntry entry(href, 0, -1);
writeSpineEntry(spineFile, entry);
spineCount++;
}
void BookMetadataCache::createTocEntry(const std::string& title, const std::string& href, const std::string& anchor,
const uint8_t level) {
if (!buildMode || !tocFile || !spineFile) {
LOG_DBG("BMC", "createTocEntry called but not in build mode");
return;
}
int16_t spineIndex = -1;
if (useSpineHrefIndex) {
uint64_t targetHash = fnvHash64(href);
uint16_t targetLen = static_cast<uint16_t>(href.size());
auto it =
std::lower_bound(spineHrefIndex.begin(), spineHrefIndex.end(), SpineHrefIndexEntry{targetHash, targetLen, 0},
[](const SpineHrefIndexEntry& a, const SpineHrefIndexEntry& b) {
return a.hrefHash < b.hrefHash || (a.hrefHash == b.hrefHash && a.hrefLen < b.hrefLen);
});
while (it != spineHrefIndex.end() && it->hrefHash == targetHash && it->hrefLen == targetLen) {
spineIndex = it->spineIndex;
break;
}
if (spineIndex == -1) {
LOG_DBG("BMC", "createTocEntry: Could not find spine item for TOC href %s", href.c_str());
}
} else {
spineFile.seek(0);
for (int i = 0; i < spineCount; i++) {
auto spineEntry = readSpineEntry(spineFile);
if (spineEntry.href == href) {
spineIndex = static_cast<int16_t>(i);
break;
}
}
if (spineIndex == -1) {
LOG_DBG("BMC", "createTocEntry: Could not find spine item for TOC href %s", href.c_str());
}
}
// Compose the title to NFC at index time so the cache stores precomposed glyphs;
// device fonts have no combining-mark positioning, so NFD titles render broken.
const TocEntry entry(utf8ComposeNfc(title), href, anchor, level, spineIndex);
writeTocEntry(tocFile, entry);
tocCount++;
}
/* ============= READING / LOADING FUNCTIONS ================ */
bool BookMetadataCache::load() {
if (!Storage.openFileForRead("BMC", cachePath + bookBinFile, bookFile)) {
return false;
}
uint8_t version;
serialization::readPod(bookFile, version);
if (version != BOOK_CACHE_VERSION) {
LOG_DBG("BMC", "Cache version mismatch: expected %d, got %d", BOOK_CACHE_VERSION, version);
// Explicit close() required: member variable persists beyond function scope
bookFile.close();
return false;
}
serialization::readPod(bookFile, lutOffset);
serialization::readPod(bookFile, spineCount);
serialization::readPod(bookFile, tocCount);
serialization::readString(bookFile, coreMetadata.title);
serialization::readString(bookFile, coreMetadata.author);
serialization::readString(bookFile, coreMetadata.language);
serialization::readString(bookFile, coreMetadata.coverItemHref);
serialization::readString(bookFile, coreMetadata.textReferenceHref);
loaded = true;
LOG_DBG("BMC", "Loaded cache data: %d spine, %d TOC entries", spineCount, tocCount);
return true;
}
BookMetadataCache::SpineEntry BookMetadataCache::getSpineEntry(const int index) {
if (!loaded) {
LOG_ERR("BMC", "getSpineEntry called but cache not loaded");
return {};
}
if (index < 0 || index >= static_cast<int>(spineCount)) {
LOG_ERR("BMC", "getSpineEntry index %d out of range", index);
return {};
}
// Seek to spine LUT item, read from LUT and get out data
bookFile.seek(lutOffset + sizeof(uint32_t) * index);
uint32_t spineEntryPos;
serialization::readPod(bookFile, spineEntryPos);
bookFile.seek(spineEntryPos);
return readSpineEntry(bookFile);
}
BookMetadataCache::TocEntry BookMetadataCache::getTocEntry(const int index) {
if (!loaded) {
LOG_ERR("BMC", "getTocEntry called but cache not loaded");
return {};
}
if (index < 0 || index >= static_cast<int>(tocCount)) {
LOG_ERR("BMC", "getTocEntry index %d out of range", index);
return {};
}
// Seek to TOC LUT item, read from LUT and get out data
bookFile.seek(lutOffset + sizeof(uint32_t) * spineCount + sizeof(uint32_t) * index);
uint32_t tocEntryPos;
serialization::readPod(bookFile, tocEntryPos);
bookFile.seek(tocEntryPos);
return readTocEntry(bookFile);
}
BookMetadataCache::SpineEntry BookMetadataCache::readSpineEntry(HalFile& file) const {
SpineEntry entry;
serialization::readString(file, entry.href);
serialization::readPod(file, entry.cumulativeSize);
serialization::readPod(file, entry.tocIndex);
return entry;
}
BookMetadataCache::TocEntry BookMetadataCache::readTocEntry(HalFile& file) const {
TocEntry entry;
serialization::readString(file, entry.title);
serialization::readString(file, entry.href);
serialization::readString(file, entry.anchor);
serialization::readPod(file, entry.level);
serialization::readPod(file, entry.spineIndex);
return entry;
}
-112
View File
@@ -1,112 +0,0 @@
#pragma once
#include <HalStorage.h>
#include <algorithm>
#include <deque>
#include <string>
class BookMetadataCache {
public:
struct BookMetadata {
std::string title;
std::string author;
std::string language;
std::string coverItemHref;
std::string textReferenceHref;
};
struct SpineEntry {
std::string href;
uint32_t cumulativeSize;
int16_t tocIndex;
SpineEntry() : cumulativeSize(0), tocIndex(-1) {}
SpineEntry(std::string href, const uint32_t cumulativeSize, const int16_t tocIndex)
: href(std::move(href)), cumulativeSize(cumulativeSize), tocIndex(tocIndex) {}
};
struct TocEntry {
std::string title;
std::string href;
std::string anchor;
uint8_t level;
int16_t spineIndex;
TocEntry() : level(0), spineIndex(-1) {}
TocEntry(std::string title, std::string href, std::string anchor, const uint8_t level, const int16_t spineIndex)
: title(std::move(title)),
href(std::move(href)),
anchor(std::move(anchor)),
level(level),
spineIndex(spineIndex) {}
};
private:
std::string cachePath;
uint32_t lutOffset;
uint16_t spineCount;
uint16_t tocCount;
bool loaded;
bool buildMode;
HalFile bookFile;
// Temp file handles during build
HalFile spineFile;
HalFile tocFile;
// Index for fast href→spineIndex lookup (used only for large EPUBs)
struct SpineHrefIndexEntry {
uint64_t hrefHash; // FNV-1a 64-bit hash
uint16_t hrefLen; // length for collision reduction
int16_t spineIndex;
};
std::deque<SpineHrefIndexEntry> spineHrefIndex;
bool useSpineHrefIndex = false;
static constexpr uint16_t LARGE_SPINE_THRESHOLD = 400;
// FNV-1a 64-bit hash function
static uint64_t fnvHash64(const std::string& s) {
uint64_t hash = 14695981039346656037ull;
for (char c : s) {
hash ^= static_cast<uint8_t>(c);
hash *= 1099511628211ull;
}
return hash;
}
uint32_t writeSpineEntry(HalFile& file, const SpineEntry& entry) const;
uint32_t writeTocEntry(HalFile& file, const TocEntry& entry) const;
SpineEntry readSpineEntry(HalFile& file) const;
TocEntry readTocEntry(HalFile& file) const;
public:
BookMetadata coreMetadata;
explicit BookMetadataCache(std::string cachePath)
: cachePath(std::move(cachePath)), lutOffset(0), spineCount(0), tocCount(0), loaded(false), buildMode(false) {}
~BookMetadataCache() = default;
// Building phase (stream to disk immediately)
bool beginWrite();
bool beginContentOpfPass();
void createSpineEntry(const std::string& href);
bool endContentOpfPass();
bool beginTocPass();
void createTocEntry(const std::string& title, const std::string& href, const std::string& anchor, uint8_t level);
bool endTocPass();
bool endWrite();
bool cleanupTmpFiles() const;
// Post-processing to update mappings and sizes
bool buildBookBin(const std::string& epubPath, const BookMetadata& metadata);
// Reading phase (read mode)
bool load();
SpineEntry getSpineEntry(int index);
TocEntry getTocEntry(int index);
int getSpineCount() const { return spineCount; }
int getTocCount() const { return tocCount; }
bool isLoaded() const { return loaded; }
};
-210
View File
@@ -1,210 +0,0 @@
#include "Page.h"
#include <GfxRenderer.h>
#include <Logging.h>
#include <Serialization.h>
#include <new>
namespace {
template <typename Predicate>
void renderFilteredPageElements(const std::vector<std::shared_ptr<PageElement>>& elements, GfxRenderer& renderer,
const int fontId, const int xOffset, const int yOffset, Predicate&& predicate) {
for (const auto& element : elements) {
if (predicate(*element)) {
element->render(renderer, fontId, xOffset, yOffset);
}
}
}
} // namespace
void PageLine::render(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) {
block->render(renderer, fontId, xPos + xOffset, yPos + yOffset);
}
bool PageLine::serialize(HalFile& file) {
serialization::writePod(file, xPos);
serialization::writePod(file, yPos);
// serialize TextBlock pointed to by PageLine
return block->serialize(file);
}
std::unique_ptr<PageLine> PageLine::deserialize(HalFile& file) {
int16_t xPos;
int16_t yPos;
serialization::readPod(file, xPos);
serialization::readPod(file, yPos);
auto tb = TextBlock::deserialize(file);
if (!tb) {
LOG_ERR("PGE", "Deserialization failed: null TextBlock");
return nullptr;
}
auto* line = new (std::nothrow) PageLine(std::move(tb), xPos, yPos);
if (!line) {
LOG_ERR("PGE", "Deserialization failed: could not allocate PageLine");
return nullptr;
}
return std::unique_ptr<PageLine>(line);
}
void PageImage::render(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) {
// Images don't use fontId or text rendering
imageBlock->render(renderer, xPos + xOffset, yPos + yOffset);
}
bool PageImage::serialize(HalFile& file) {
serialization::writePod(file, xPos);
serialization::writePod(file, yPos);
// serialize ImageBlock
return imageBlock->serialize(file);
}
std::unique_ptr<PageImage> PageImage::deserialize(HalFile& file) {
int16_t xPos;
int16_t yPos;
serialization::readPod(file, xPos);
serialization::readPod(file, yPos);
auto ib = ImageBlock::deserialize(file);
return std::unique_ptr<PageImage>(new PageImage(std::move(ib), xPos, yPos));
}
void PageHorizontalRule::render(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) {
(void)fontId;
if (width == 0 || thickness == 0) {
return;
}
renderer.drawLine(xPos + xOffset, yPos + yOffset, xPos + xOffset + width - 1, yPos + yOffset, thickness, true);
}
bool PageHorizontalRule::serialize(HalFile& file) {
serialization::writePod(file, xPos);
serialization::writePod(file, yPos);
serialization::writePod(file, width);
serialization::writePod(file, thickness);
return true;
}
std::unique_ptr<PageHorizontalRule> PageHorizontalRule::deserialize(HalFile& file) {
int16_t xPos = 0;
int16_t yPos = 0;
uint16_t width = 0;
uint8_t thickness = 0;
serialization::readPod(file, xPos);
serialization::readPod(file, yPos);
serialization::readPod(file, width);
serialization::readPod(file, thickness);
if (width == 0 || thickness == 0) {
LOG_ERR("PGE", "Deserialization failed: invalid horizontal rule metadata (width=%u thickness=%u)", width,
thickness);
return nullptr;
}
auto* rule = new (std::nothrow) PageHorizontalRule(width, thickness, xPos, yPos);
if (!rule) {
LOG_ERR("PGE", "Deserialization failed: could not allocate PageHorizontalRule");
return nullptr;
}
return std::unique_ptr<PageHorizontalRule>(rule);
}
void Page::render(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) const {
renderFilteredPageElements(elements, renderer, fontId, xOffset, yOffset, [](const PageElement&) { return true; });
}
void Page::renderImages(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) const {
renderFilteredPageElements(elements, renderer, fontId, xOffset, yOffset,
[](const PageElement& element) { return element.getTag() == TAG_PageImage; });
}
bool Page::serialize(HalFile& file) const {
const uint16_t count = elements.size();
serialization::writePod(file, count);
for (const auto& el : elements) {
// Use getTag() method to determine type
serialization::writePod(file, static_cast<uint8_t>(el->getTag()));
if (!el->serialize(file)) {
return false;
}
}
// Serialize footnotes (clamp to MAX_FOOTNOTES_PER_PAGE to match addFootnote/deserialize limits)
const uint16_t fnCount = std::min<uint16_t>(footnotes.size(), MAX_FOOTNOTES_PER_PAGE);
serialization::writePod(file, fnCount);
for (uint16_t i = 0; i < fnCount; i++) {
const auto& fn = footnotes[i];
if (file.write(fn.number, sizeof(fn.number)) != sizeof(fn.number) ||
file.write(fn.href, sizeof(fn.href)) != sizeof(fn.href)) {
LOG_ERR("PGE", "Failed to write footnote");
return false;
}
}
return true;
}
std::unique_ptr<Page> Page::deserialize(HalFile& file) {
auto page = std::unique_ptr<Page>(new Page());
uint16_t count;
serialization::readPod(file, count);
for (uint16_t i = 0; i < count; i++) {
uint8_t tag;
serialization::readPod(file, tag);
if (tag == TAG_PageLine) {
auto pl = PageLine::deserialize(file);
if (!pl) {
return nullptr;
}
page->elements.push_back(std::move(pl));
} else if (tag == TAG_PageImage) {
auto pi = PageImage::deserialize(file);
if (!pi) {
return nullptr;
}
page->elements.push_back(std::move(pi));
} else if (tag == TAG_PageHorizontalRule) {
auto rule = PageHorizontalRule::deserialize(file);
if (!rule) {
return nullptr;
}
page->elements.push_back(std::move(rule));
} else {
LOG_ERR("PGE", "Deserialization failed: Unknown tag %u", tag);
return nullptr;
}
}
// Deserialize footnotes
uint16_t fnCount;
serialization::readPod(file, fnCount);
if (fnCount > MAX_FOOTNOTES_PER_PAGE) {
LOG_ERR("PGE", "Invalid footnote count %u", fnCount);
return nullptr;
}
page->footnotes.resize(fnCount);
for (uint16_t i = 0; i < fnCount; i++) {
auto& entry = page->footnotes[i];
if (file.read(entry.number, sizeof(entry.number)) != sizeof(entry.number) ||
file.read(entry.href, sizeof(entry.href)) != sizeof(entry.href)) {
LOG_ERR("PGE", "Failed to read footnote %u", i);
return nullptr;
}
entry.number[sizeof(entry.number) - 1] = '\0';
entry.href[sizeof(entry.href) - 1] = '\0';
}
return page;
}
-128
View File
@@ -1,128 +0,0 @@
#pragma once
#include <HalStorage.h>
#include <algorithm>
#include <string>
#include <utility>
#include <vector>
#include "FootnoteEntry.h"
#include "blocks/ImageBlock.h"
#include "blocks/TextBlock.h"
enum PageElementTag : uint8_t {
TAG_PageLine = 1,
TAG_PageImage = 2,
TAG_PageHorizontalRule = 3,
};
// represents something that has been added to a page
class PageElement {
public:
int16_t xPos;
int16_t yPos;
explicit PageElement(const int16_t xPos, const int16_t yPos) : xPos(xPos), yPos(yPos) {}
virtual ~PageElement() = default;
virtual void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) = 0;
virtual bool serialize(HalFile& file) = 0;
virtual PageElementTag getTag() const = 0; // Add type identification
};
// a line from a block element
class PageLine final : public PageElement {
std::shared_ptr<TextBlock> block;
public:
PageLine(std::shared_ptr<TextBlock> block, const int16_t xPos, const int16_t yPos)
: PageElement(xPos, yPos), block(std::move(block)) {}
const std::shared_ptr<TextBlock>& getBlock() const { return block; }
void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) override;
bool serialize(HalFile& file) override;
PageElementTag getTag() const override { return TAG_PageLine; }
static std::unique_ptr<PageLine> deserialize(HalFile& file);
};
// New PageImage class
class PageImage final : public PageElement {
std::shared_ptr<ImageBlock> imageBlock;
public:
PageImage(std::shared_ptr<ImageBlock> block, const int16_t xPos, const int16_t yPos)
: PageElement(xPos, yPos), imageBlock(std::move(block)) {}
void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) override;
bool serialize(HalFile& file) override;
PageElementTag getTag() const override { return TAG_PageImage; }
static std::unique_ptr<PageImage> deserialize(HalFile& file);
const ImageBlock& getImageBlock() const { return *imageBlock; }
};
class PageHorizontalRule final : public PageElement {
uint16_t width;
uint8_t thickness;
public:
PageHorizontalRule(uint16_t width, uint8_t thickness, const int16_t xPos, const int16_t yPos)
: PageElement(xPos, yPos), width(width), thickness(thickness) {}
void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) override;
bool serialize(HalFile& file) override;
PageElementTag getTag() const override { return TAG_PageHorizontalRule; }
static std::unique_ptr<PageHorizontalRule> deserialize(HalFile& file);
};
class Page {
public:
// the list of block index and line numbers on this page
std::vector<std::shared_ptr<PageElement>> elements;
std::vector<FootnoteEntry> footnotes;
static constexpr uint16_t MAX_FOOTNOTES_PER_PAGE = 16;
void addFootnote(const char* number, const char* href) {
if (footnotes.size() >= MAX_FOOTNOTES_PER_PAGE) return; // Cap per-page footnotes
FootnoteEntry entry;
strncpy(entry.number, number, sizeof(entry.number) - 1);
entry.number[sizeof(entry.number) - 1] = '\0';
strncpy(entry.href, href, sizeof(entry.href) - 1);
entry.href[sizeof(entry.href) - 1] = '\0';
footnotes.push_back(entry);
}
void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) const;
void renderImages(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) const;
bool serialize(HalFile& file) const;
static std::unique_ptr<Page> deserialize(HalFile& file);
// Check if page contains any images (used to force full refresh)
bool hasImages() const {
return std::any_of(elements.begin(), elements.end(),
[](const std::shared_ptr<PageElement>& el) { return el->getTag() == TAG_PageImage; });
}
// Get bounding box of all images on the page (union of image rects)
// Returns false if no images. Coordinates are relative to page origin.
bool getImageBoundingBox(int16_t& outX, int16_t& outY, int16_t& outW, int16_t& outH) const {
bool found = false;
int16_t minX = INT16_MAX, minY = INT16_MAX, maxX = INT16_MIN, maxY = INT16_MIN;
for (const auto& el : elements) {
if (el->getTag() == TAG_PageImage) {
const auto& img = static_cast<const PageImage&>(*el);
int16_t x = img.xPos;
int16_t y = img.yPos;
int16_t right = x + img.getImageBlock().getWidth();
int16_t bottom = y + img.getImageBlock().getHeight();
minX = std::min(minX, x);
minY = std::min(minY, y);
maxX = std::max(maxX, right);
maxY = std::max(maxY, bottom);
found = true;
}
}
if (found) {
outX = minX;
outY = minY;
outW = maxX - minX;
outH = maxY - minY;
}
return found;
}
};
File diff suppressed because it is too large. Load diff
-70
View File
@@ -1,70 +0,0 @@
#pragma once
#include <EpdFontFamily.h>
#include <functional>
#include <memory>
#include <string>
#include <vector>
#include "blocks/BlockStyle.h"
#include "blocks/TextBlock.h"
class GfxRenderer;
class ParsedText {
std::vector<std::string> words;
std::vector<EpdFontFamily::Style> wordStyles;
std::vector<bool> wordContinues; // true = word attaches to previous with no break
std::vector<bool> wordNoSpaceBefore; // true = may break before token, but no synthetic space when joined
std::vector<bool> wordIsFocusSuffix; // true = token is the regular tail of a focus bold-prefix split
BlockStyle blockStyle;
bool extraParagraphSpacing;
bool hyphenationEnabled;
bool focusReadingEnabled;
bool isNaturalAlign;
bool hasRtlWord;
std::vector<std::string> reorderedWordsScratch;
std::vector<EpdFontFamily::Style> reorderedStylesScratch;
std::vector<uint16_t> reorderedWidthsScratch;
std::vector<bool> reorderedContinuesScratch;
std::vector<bool> reorderedNoSpaceBeforeScratch;
std::vector<bool> reorderedFocusSuffixScratch;
std::vector<uint16_t> visualOrderScratch;
int resolveFirstLineIndent(bool isFirstLine, const GfxRenderer& renderer, int fontId) const;
std::vector<size_t> computeLineBreaks(const GfxRenderer& renderer, int fontId, int pageWidth,
std::vector<uint16_t>& wordWidths, std::vector<bool>& continuesVec,
std::vector<bool>& noSpaceBeforeVec);
std::vector<size_t> computeHyphenatedLineBreaks(const GfxRenderer& renderer, int fontId, int pageWidth,
std::vector<uint16_t>& wordWidths, std::vector<bool>& continuesVec,
std::vector<bool>& noSpaceBeforeVec);
bool hyphenateWordAtIndex(size_t wordIndex, int availableWidth, const GfxRenderer& renderer, int fontId,
std::vector<uint16_t>& wordWidths, bool allowFallbackBreaks);
void extractLine(size_t breakIndex, int pageWidth, const std::vector<uint16_t>& wordWidths,
const std::vector<bool>& continuesVec, const std::vector<bool>& noSpaceBeforeVec,
const std::vector<size_t>& lineBreakIndices,
const std::function<void(std::shared_ptr<TextBlock>)>& processLine, const GfxRenderer& renderer,
int fontId);
std::vector<uint16_t> calculateWordWidths(const GfxRenderer& renderer, int fontId);
public:
explicit ParsedText(const bool extraParagraphSpacing, const bool hyphenationEnabled = false,
const bool focusReadingEnabled = false, const BlockStyle& blockStyle = BlockStyle())
: blockStyle(blockStyle),
extraParagraphSpacing(extraParagraphSpacing),
hyphenationEnabled(hyphenationEnabled),
focusReadingEnabled(focusReadingEnabled),
isNaturalAlign(false),
hasRtlWord(false) {}
~ParsedText() = default;
void addWord(std::string word, EpdFontFamily::Style fontStyle, bool underline = false, bool attachToPrevious = false);
void setBlockStyle(const BlockStyle& blockStyle) { this->blockStyle = blockStyle; }
BlockStyle& getBlockStyle() { return blockStyle; }
size_t size() const { return words.size(); }
bool isEmpty() const { return words.empty(); }
void layoutAndExtractLines(const GfxRenderer& renderer, int fontId, uint16_t viewportWidth,
const std::function<void(std::shared_ptr<TextBlock>)>& processLine,
bool includeLastLine = true);
};
File diff suppressed because it is too large. Load diff
-157
View File
@@ -1,157 +0,0 @@
#pragma once
#include <functional>
#include <memory>
#include <optional>
#include <string>
#include <vector>
#include "Epub.h"
class Page;
class GfxRenderer;
class ChapterHtmlSlimParser;
class CssParser;
class Section {
std::shared_ptr<Epub> epub;
const int spineIndex;
GfxRenderer& renderer;
std::string filePath;
HalFile file;
void writeSectionFileHeader(int fontId, float lineCompression, bool extraParagraphSpacing, uint8_t paragraphAlignment,
uint16_t viewportWidth, uint16_t viewportHeight, bool hyphenationEnabled,
bool embeddedStyle, uint8_t imageRendering, bool focusReadingEnabled);
uint32_t onPageComplete(std::unique_ptr<Page> page);
// Page-offset table entry, kept in RAM while an incremental build is running so
// already-built pages can be located in the partially-written .bin.
struct PageLutEntry {
uint32_t fileOffset;
uint16_t paragraphIndex;
uint16_t listItemIndex;
};
// Held only while an incremental build is in progress (see startBuild). Carries the
// live parser plus the strings it references (the parser stores them by reference)
// and the in-RAM page-offset table.
struct BuildContext {
std::unique_ptr<ChapterHtmlSlimParser> parser;
std::vector<PageLutEntry> lut;
std::string parsePath;
std::string contentBase;
std::string imageBasePath;
std::string htmlPath;
std::string tmpHtmlPath;
bool reusedHtml = false;
CssParser* cssParser = nullptr;
// HTML byte progress, for estimating the section's total page count while it's still building.
uint32_t bytesConsumed = 0;
uint32_t totalBytes = 0;
// Exponentially-smoothed page-count estimate (0 = not yet seeded) and the bytesConsumed at its
// last update. The raw byte-ratio estimate jitters as the build crosses dense/sparse regions;
// the EMA is stepped once per build advance (not per redraw) to damp that wobble.
float smoothedEstimate = 0;
uint32_t smoothedAtConsumed = 0;
};
std::unique_ptr<BuildContext> build_;
bool buildComplete_ = false;
// Pages laid out by the active build (== build_->lut.size()). Distinct from pageCount,
// which is the pages *available to read* and also counts a loaded partial file's pages.
uint16_t builtPageCount_ = 0;
// A partial section file (suspended build from a previous session) is loaded at filePath.
// Its pages 0..partialPageCount_-1 are readable while a rebuild extends past them.
bool partial_ = false;
uint16_t partialPageCount_ = 0;
// Parse watermark from the partial's trailer, for estimating the total page count.
uint32_t partialBytesConsumed_ = 0;
uint32_t partialTotalBytes_ = 0;
bool finalizeBuild();
// Write the LUTs/anchor map (and, for a partial, the watermark trailer), patch the
// header, stamp the version byte, and swap the tmp .bin over filePath.
bool commitBuildFile(uint8_t version, uint32_t bytesConsumed, uint32_t totalBytes);
// Builds write here and are swapped over filePath only on commit, so a prior
// partial/finalized file stays readable while a rebuild is in progress.
std::string binTmpPath() const { return filePath + ".part"; }
std::unique_ptr<Page> loadPageAt(int page) const;
// Read a page already laid out by the in-progress build (page < build LUT size), from
// the partially-written tmp .bin without disturbing the build's write cursor.
std::unique_ptr<Page> loadPageDuringBuild(int page);
public:
uint16_t pageCount = 0;
int currentPage = 0;
// Constructor and destructor are out-of-line: BuildContext holds a unique_ptr to the
// forward-declared ChapterHtmlSlimParser, whose full definition is only visible in the .cpp.
explicit Section(const std::shared_ptr<Epub>& epub, int spineIndex, GfxRenderer& renderer);
~Section();
bool loadSectionFile(int fontId, float lineCompression, bool extraParagraphSpacing, uint8_t paragraphAlignment,
uint16_t viewportWidth, uint16_t viewportHeight, bool hyphenationEnabled, bool embeddedStyle,
uint8_t imageRendering, bool focusReadingEnabled);
bool clearCache() const;
bool createSectionFile(int fontId, float lineCompression, bool extraParagraphSpacing, uint8_t paragraphAlignment,
uint16_t viewportWidth, uint16_t viewportHeight, bool hyphenationEnabled, bool embeddedStyle,
uint8_t imageRendering, bool focusReadingEnabled,
const std::function<void()>& popupFn = nullptr);
// Incremental build: lay out the section a few pages at a time so a large chapter
// can show its first page immediately and keep the UI responsive while the rest
// builds. createSectionFile() above is the one-shot wrapper over these.
// if (!startBuild(...)) fail;
// each tick: buildSomeMore(N); render up to pageCount; when isBuildComplete() stop.
bool startBuild(int fontId, float lineCompression, bool extraParagraphSpacing, uint8_t paragraphAlignment,
uint16_t viewportWidth, uint16_t viewportHeight, bool hyphenationEnabled, bool embeddedStyle,
uint8_t imageRendering, bool focusReadingEnabled, const std::function<void()>& popupFn = nullptr);
// Lay out up to maxPages more pages (maxPages <= 0 = build to completion). Returns
// false on error (the build is abandoned). Sets isBuildComplete() when finished.
bool buildSomeMore(int maxPages);
bool isBuilding() const { return static_cast<bool>(build_); }
bool isBuildComplete() const { return buildComplete_; }
// Best-known total page count: the exact pageCount once finalized, or a smoothed byte-based
// estimate (pages so far scaled by totalBytes/bytesConsumed, damped by an EMA) while a giant spine
// is still building, so "page X of Y" / progress don't read off the small build watermark.
uint16_t estimatedTotalPages() const;
void abandonBuild();
// Persist an in-progress build as a partial section file (version sentinel + LUTs +
// watermark trailer) instead of discarding it, so the next open of this spine can show
// its pages instantly and only rebuild in the background. Called by the destructor, so
// any teardown path (exit, sleep, navigation) keeps the work already done. Keeps a
// pre-existing partial when it covers more pages than this build reached.
void suspendBuild();
// True when a partial file was loaded: pageCount is a watermark, not the chapter total.
bool isPartial() const { return partial_; }
// Unified page read: from the active build if it has reached the page, otherwise from
// the on-disk file (finalized section, or a partial the rebuild hasn't caught up to).
std::unique_ptr<Page> loadPage(int page);
std::string getTextFromSectionFile();
// Resolve an anchor from the in-progress build first, then the on-disk anchor map
// (covers finalized sections and partials from a previous session).
std::optional<uint16_t> findAnchor(const std::string& anchor) const;
// True if this spine's unzipped HTML is already cached, so a build won't pay the (multi-second on a
// giant spine) zip inflation. Lets the reader skip the indexing popup on a fast reopen/rebuild.
bool hasHtmlCache() const;
// Look up the page number for an anchor id from the section cache file.
std::optional<uint16_t> getPageForAnchor(const std::string& anchor) const;
// Look up an anchor among the pages built so far by the in-progress build, so an anchor jump
// (TOC / chapter select, usually the chapter top = page 0) can resolve without laying out the
// whole chapter. Returns nullopt if the anchor hasn't been reached yet (build more) or no build.
std::optional<uint16_t> findAnchorDuringBuild(const std::string& anchor) const;
// Get the page count from the section cache file without fully loading it.
std::optional<uint16_t> getCachedPageCount() const;
// Look up the page number for a synthetic paragraph index from XPath p[N].
std::optional<uint16_t> getPageForParagraphIndex(uint16_t pIndex) const;
// Look up the page number for a running list-item index from the li LUT.
std::optional<uint16_t> getPageForListItemIndex(uint16_t liIndex) const;
// Look up the synthetic paragraph index for the given rendered page.
std::optional<uint16_t> getParagraphIndexForPage(uint16_t page) const;
};
-15
View File
@@ -1,15 +0,0 @@
#pragma once
class GfxRenderer;
typedef enum { TEXT_BLOCK, IMAGE_BLOCK } BlockType;
// a block of content in the html - either a paragraph or an image
class Block {
public:
virtual ~Block() = default;
virtual BlockType getType() = 0;
virtual bool isEmpty() = 0;
virtual void finish() {}
};
-145
View File
@@ -1,145 +0,0 @@
#pragma once
#include <algorithm>
#include <cstdint>
#include "Epub/css/CssStyle.h"
/**
* BlockStyle - Block-level styling properties
*/
struct BlockStyle {
// Upper bound (in em) for any single side's horizontal margin or padding.
// Some EPUBs apply huge em-based insets to chapter-opener classes; without a
// cap, effectiveWidth collapses to 1-2 words per line and justification dumps
// the remaining space into a single gap.
static constexpr float MAX_HORIZONTAL_INSET_EM = 2.0f;
CssTextAlign alignment = CssTextAlign::Justify;
// Spacing (in pixels)
int16_t marginTop = 0;
int16_t marginBottom = 0;
int16_t marginLeft = 0;
int16_t marginRight = 0;
int16_t paddingTop = 0; // treated same as margin for rendering
int16_t paddingBottom = 0; // treated same as margin for rendering
int16_t paddingLeft = 0; // treated same as margin for rendering
int16_t paddingRight = 0; // treated same as margin for rendering
int16_t textIndent = 0;
bool textIndentDefined = false; // true if text-indent was explicitly set in CSS
bool textAlignDefined = false; // true if text-align was explicitly set in CSS
bool isRtl = false; // true if resolved direction is RTL
bool directionDefined = false; // true if direction was explicitly set in CSS/HTML
// Set when this block was created by a <br> element. Used by startNewTextBlock to inject
// a full line-height gap when the <br> block stays empty (section-break use case).
// NOT propagated through getCombinedBlockStyle so it can't leak into sibling blocks.
bool fromBrElement = false;
// Combined insets (margin + padding)
[[nodiscard]] int16_t leftInset() const { return marginLeft + paddingLeft; }
[[nodiscard]] int16_t rightInset() const { return marginRight + paddingRight; }
[[nodiscard]] int16_t totalHorizontalInset() const { return leftInset() + rightInset(); }
[[nodiscard]] int16_t topInset() const { return marginTop + paddingTop; }
[[nodiscard]] int16_t bottomInset() const { return marginBottom + paddingBottom; }
// Return a copy with bottom margins/padding zeroed out.
[[nodiscard]] BlockStyle withoutBottom() const {
BlockStyle result = *this;
result.marginBottom = 0;
result.paddingBottom = 0;
return result;
}
// Return a copy with bottom margins/padding collapsed (max) with the source's.
// Uses CSS margin collapsing: adjacent parent-child margins resolve to the larger value.
[[nodiscard]] BlockStyle addBottom(const BlockStyle& source) const {
BlockStyle result = *this;
result.marginBottom = std::max(marginBottom, source.marginBottom);
result.paddingBottom = static_cast<int16_t>(paddingBottom + source.paddingBottom);
return result;
}
enum class CombineAxis : uint8_t {
Horizontal = 1, // margins left/right, padding left/right, text-align, text-indent
Vertical = 2, // margins top/bottom, padding top/bottom
};
// Combine this style's properties with a child style along the specified axis.
// Properties on the other axis are kept from the child unchanged.
[[nodiscard]] BlockStyle getCombinedBlockStyle(const BlockStyle& child, CombineAxis axis) const {
BlockStyle result = child;
if (axis == CombineAxis::Horizontal) {
result.marginLeft = static_cast<int16_t>(child.marginLeft + marginLeft);
result.marginRight = static_cast<int16_t>(child.marginRight + marginRight);
result.paddingLeft = static_cast<int16_t>(child.paddingLeft + paddingLeft);
result.paddingRight = static_cast<int16_t>(child.paddingRight + paddingRight);
if (!child.textIndentDefined && textIndentDefined) {
result.textIndent = textIndent;
result.textIndentDefined = true;
}
if (!child.textAlignDefined && textAlignDefined) {
result.alignment = alignment;
result.textAlignDefined = true;
}
} else {
result.marginTop = std::max(child.marginTop, marginTop);
result.marginBottom = std::max(child.marginBottom, marginBottom);
result.paddingTop = static_cast<int16_t>(child.paddingTop + paddingTop);
result.paddingBottom = static_cast<int16_t>(child.paddingBottom + paddingBottom);
}
// Direction is not axis-specific. Inherit from parent when child doesn't define it.
if (!child.directionDefined && directionDefined) {
result.isRtl = isRtl;
result.directionDefined = true;
}
// fromBrElement is consumed by startNewTextBlock when an empty <br> block
// is merged with the following paragraph; never propagate it further.
result.fromBrElement = false;
return result;
}
// Create a BlockStyle from CSS style properties, resolving CssLength values to pixels
// emSize is the current font line height, used for em/rem unit conversion
// paragraphAlignment is the user's paragraphAlignment setting preference
static BlockStyle fromCssStyle(const CssStyle& cssStyle, const float emSize, const CssTextAlign paragraphAlignment,
const uint16_t viewportWidth = 0) {
BlockStyle blockStyle;
const float vw = viewportWidth;
const auto maxHorizontalInsetPx = static_cast<int16_t>(emSize * MAX_HORIZONTAL_INSET_EM);
// Resolve all CssLength values to pixels using the current font's em size and viewport width
blockStyle.marginTop = cssStyle.marginTop.toPixelsInt16(emSize, vw);
blockStyle.marginBottom = cssStyle.marginBottom.toPixelsInt16(emSize, vw);
blockStyle.marginLeft = std::min(cssStyle.marginLeft.toPixelsInt16(emSize, vw), maxHorizontalInsetPx);
blockStyle.marginRight = std::min(cssStyle.marginRight.toPixelsInt16(emSize, vw), maxHorizontalInsetPx);
blockStyle.paddingTop = cssStyle.paddingTop.toPixelsInt16(emSize, vw);
blockStyle.paddingBottom = cssStyle.paddingBottom.toPixelsInt16(emSize, vw);
blockStyle.paddingLeft = std::min(cssStyle.paddingLeft.toPixelsInt16(emSize, vw), maxHorizontalInsetPx);
blockStyle.paddingRight = std::min(cssStyle.paddingRight.toPixelsInt16(emSize, vw), maxHorizontalInsetPx);
// For textIndent: if it's a percentage we can't resolve (no viewport width),
// leave textIndentDefined=false so the space-width fallback in resolveFirstLineIndent() is used
if (cssStyle.hasTextIndent() && cssStyle.textIndent.isResolvable(vw)) {
blockStyle.textIndent = cssStyle.textIndent.toPixelsInt16(emSize, vw);
blockStyle.textIndentDefined = true;
}
blockStyle.textAlignDefined = cssStyle.hasTextAlign();
// User setting overrides CSS, unless "Book's Style" alignment setting is selected
if (paragraphAlignment == CssTextAlign::None) {
blockStyle.alignment = blockStyle.textAlignDefined ? cssStyle.textAlign : CssTextAlign::Justify;
} else {
blockStyle.alignment = paragraphAlignment;
}
// RTL direction from CSS/HTML
if (cssStyle.hasDirection()) {
blockStyle.isRtl = (cssStyle.direction == CssTextDirection::Rtl);
blockStyle.directionDefined = true;
}
return blockStyle;
}
};
-218
View File
@@ -1,218 +0,0 @@
#include "ImageBlock.h"
#include <FontCacheManager.h>
#include <GfxRenderer.h>
#include <Logging.h>
#include <Serialization.h>
#include "Epub/converters/DirectPixelWriter.h"
#include "Epub/converters/ImageDecoderFactory.h"
// Cache file format:
// - uint16_t width
// - uint16_t height
// - uint8_t pixels[...] - 2 bits per pixel, packed (4 pixels per byte), row-major order
ImageBlock::ImageBlock(const std::string& imagePath, int16_t width, int16_t height)
: imagePath(imagePath), width(width), height(height) {}
bool ImageBlock::imageExists() const { return Storage.exists(imagePath.c_str()); }
namespace {
std::string getCachePath(const std::string& imagePath) {
// Replace extension with .pxc (pixel cache)
size_t dotPos = imagePath.rfind('.');
if (dotPos != std::string::npos) {
return imagePath.substr(0, dotPos) + ".pxc";
}
return imagePath + ".pxc";
}
bool renderFromCache(GfxRenderer& renderer, const std::string& cachePath, int x, int y, int expectedWidth,
int expectedHeight) {
HalFile cacheFile;
if (!Storage.openFileForRead("IMG", cachePath, cacheFile)) {
return false;
}
uint16_t cachedWidth, cachedHeight;
if (cacheFile.read(&cachedWidth, 2) != 2 || cacheFile.read(&cachedHeight, 2) != 2) {
return false;
}
// Verify dimensions are close (allow 1 pixel tolerance for rounding differences)
int widthDiff = abs(cachedWidth - expectedWidth);
int heightDiff = abs(cachedHeight - expectedHeight);
if (widthDiff > 1 || heightDiff > 1) {
LOG_ERR("IMG", "Cache dimension mismatch: %dx%d vs %dx%d", cachedWidth, cachedHeight, expectedWidth,
expectedHeight);
return false;
}
// Use cached dimensions for rendering (they're the actual decoded size)
expectedWidth = cachedWidth;
expectedHeight = cachedHeight;
LOG_DBG("IMG", "Loading from cache: %s (%dx%d)", cachePath.c_str(), cachedWidth, cachedHeight);
// Read several rows per SD access. A full-page image is re-rendered on every
// grayscale strip pass (~14x per page), and a one-row-per-read loop here means
// cachedHeight (~728) tiny reads through the storage mutex + SdFat each time —
// the dominant cost of displaying an image page. Batching rows into a ~4KB
// buffer cuts that to ~20 reads per pass without holding the whole image.
const int bytesPerRow = (cachedWidth + 3) / 4; // 2 bits per pixel, 4 pixels per byte
int rowsPerRead = 4096 / bytesPerRow;
if (rowsPerRead < 1) rowsPerRead = 1;
if (rowsPerRead > cachedHeight) rowsPerRead = cachedHeight;
uint8_t* readBuffer = (uint8_t*)malloc((size_t)rowsPerRead * bytesPerRow);
if (!readBuffer) {
// Fall back to a single-row buffer under memory pressure.
rowsPerRead = 1;
readBuffer = (uint8_t*)malloc(bytesPerRow);
}
if (!readBuffer) {
LOG_ERR("IMG", "Failed to allocate row buffer");
return false;
}
DirectPixelWriter pw;
pw.init(renderer);
int rowsInBuffer = 0;
int bufferRow = 0;
for (int row = 0; row < cachedHeight; row++) {
if (bufferRow >= rowsInBuffer) {
const int toRead = (cachedHeight - row < rowsPerRead) ? (cachedHeight - row) : rowsPerRead;
const size_t bytes = (size_t)toRead * bytesPerRow;
if (cacheFile.read(readBuffer, bytes) != static_cast<int>(bytes)) {
LOG_ERR("IMG", "Cache read error at row %d", row);
free(readBuffer);
return false;
}
rowsInBuffer = toRead;
bufferRow = 0;
}
const uint8_t* rowBuffer = readBuffer + (size_t)bufferRow * bytesPerRow;
bufferRow++;
const int destY = y + row;
pw.beginRow(destY);
// On a grayscale strip pass only a narrow column window of the image is in
// the active band; skip the rest instead of unpacking+clipping every pixel.
int colStart, colEnd;
pw.bandColRange(x, cachedWidth, colStart, colEnd);
for (int col = colStart; col < colEnd; col++) {
const int byteIdx = col >> 2; // col / 4
const int bitShift = 6 - (col & 3) * 2; // MSB first within byte
uint8_t pixelValue = (rowBuffer[byteIdx] >> bitShift) & 0x03;
pw.writePixel(x + col, pixelValue);
}
}
free(readBuffer);
LOG_DBG("IMG", "Cache render complete");
return true;
}
} // namespace
void ImageBlock::render(GfxRenderer& renderer, const int x, const int y) {
// The font-prewarm scan pass only accumulates glyphs; an image contributes
// none, and its DirectPixelWriter output bypasses the renderer's scan-mode
// suppression, so it would otherwise do a full (discarded) cache render every
// page view. Skip it here. The image still draws in the real BW/grayscale
// passes; on first view this just moves the one-time decode to the BW pass.
FontCacheManager* fcm = renderer.getFontCacheManager();
if (fcm && fcm->isScanning()) return;
LOG_DBG("IMG", "Rendering image at %d,%d: %s (%dx%d)", x, y, imagePath.c_str(), width, height);
const int screenWidth = renderer.getScreenWidth();
const int screenHeight = renderer.getScreenHeight();
// Bounds check render position using logical screen dimensions
if (x < 0 || y < 0 || x + width > screenWidth || y + height > screenHeight) {
LOG_ERR("IMG", "Invalid render position: (%d,%d) size (%dx%d) screen (%dx%d)", x, y, width, height, screenWidth,
screenHeight);
return;
}
// Tiled grayscale (#2190): skip the whole image when it doesn't touch the
// active band. The per-pixel writer already clips off-band pixels, but without
// this each of the ~7 bands per plane re-ran the full cache load / pixel walk
// and discarded the result — the dominant cost of AA on image pages. The check
// is orientation-aware and returns true when no strip is active, so the BW
// pass and non-tiled controllers render the image exactly as before.
if (!renderer.glyphIntersectsStrip(x, y, x + width - 1, y + height - 1)) {
return;
}
// Try to render from cache first
std::string cachePath = getCachePath(imagePath);
if (renderFromCache(renderer, cachePath, x, y, width, height)) {
return; // Successfully rendered from cache
}
// No cache - need to decode the image
// Check if image file exists
HalFile file;
if (!Storage.openFileForRead("IMG", imagePath, file)) {
LOG_ERR("IMG", "Image file not found: %s", imagePath.c_str());
return;
}
size_t fileSize = file.size();
file.close();
if (fileSize == 0) {
LOG_ERR("IMG", "Image file is empty: %s", imagePath.c_str());
return;
}
LOG_DBG("IMG", "Decoding and caching: %s", imagePath.c_str());
RenderConfig config;
config.x = x;
config.y = y;
config.maxWidth = width;
config.maxHeight = height;
config.useGrayscale = true;
config.useDithering = true;
config.performanceMode = false;
config.useExactDimensions = true; // Use pre-calculated dimensions to avoid rounding mismatches
config.cachePath = cachePath; // Enable caching during decode
ImageToFramebufferDecoder* decoder = ImageDecoderFactory::getDecoder(imagePath);
if (!decoder) {
LOG_ERR("IMG", "No decoder found for image: %s", imagePath.c_str());
return;
}
LOG_DBG("IMG", "Using %s decoder", decoder->getFormatName());
bool success = decoder->decodeToFramebuffer(imagePath, renderer, config);
if (!success) {
LOG_ERR("IMG", "Failed to decode image: %s", imagePath.c_str());
return;
}
LOG_DBG("IMG", "Decode successful");
}
bool ImageBlock::serialize(HalFile& file) {
serialization::writeString(file, imagePath);
serialization::writePod(file, width);
serialization::writePod(file, height);
return true;
}
std::unique_ptr<ImageBlock> ImageBlock::deserialize(HalFile& file) {
std::string path;
serialization::readString(file, path);
int16_t w, h;
serialization::readPod(file, w);
serialization::readPod(file, h);
return std::unique_ptr<ImageBlock>(new ImageBlock(path, w, h));
}
Loaded 20 of 165 files, more files were not shown because too many files have changed in this diff. Show more