## Summary
Extracted some changes from
https://github.com/crosspoint-reader/crosspoint-reader/pull/500 to make
reviewing easier
This PR adds HAL (Hardware Abstraction Layer) for display and GPIO
components, making it easier to write a stub or an emulated
implementation of the hardware.
SD card HAL will be added via another PR, because it's a bit more
tricky.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **NO**
## Summary
* Correctly render italics on image alt placeholders
* Parser incorrectly handled depth of self-closing tags
* Self-closing tags immediately call start and end tag
## Additional Context
* Previously, it would incorrectly make the whole chapter bold/italics,
or not italicised the image alt
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? No
## Summary
Optimizes EPUB metadata indexing for large books (2000+ chapters) from
~30 minutes to ~50 seconds by replacing O(n²) algorithms with O(n log n)
hash-indexed lookups.
Fixes#134
## Problem
Three phases had O(n²) complexity due to nested loops:
| Phase | Operation | Before (2768 chapters) |
|-------|-----------|------------------------|
| OPF Pass | For each spine ref, scan all manifest items | ~25 min |
| TOC Pass | For each TOC entry, scan all spine items | ~5 min |
| buildBookBin | For each spine item, scan ZIP central directory | ~8.4
min |
Total: **~30+ minutes** for first-time indexing of large EPUBs.
## Solution
Replace linear scans with sorted hash indexes + binary search:
- **OPF Pass**: Build `{hash(id), len, offset}` index from manifest,
binary search for each spine ref
- **TOC Pass**: Build `{hash(href), len, spineIndex}` index from spine,
binary search for each TOC entry
- **buildBookBin**: New `ZipFile::fillUncompressedSizes()` API - single
ZIP central directory scan with batch hash matching
All indexes use FNV-1a hashing with length as secondary key to minimize
collisions. Indexes are freed immediately after each phase.
## Results
**Shadow Slave EPUB (2768 chapters):**
| Phase | Before | After | Speedup |
|-------|--------|-------|---------|
| OPF pass | ~25 min | 10.8 sec | ~140x |
| TOC pass | ~5 min | 4.7 sec | ~60x |
| buildBookBin | 506 sec | 34.6 sec | ~15x |
| **Total** | **~30+ min** | **~50 sec** | **~36x** |
**Normal EPUB (87 chapters):** 1.7 sec - no regression.
## Memory
Peak temporary memory during indexing:
- OPF index: ~33KB (2770 items × 12 bytes)
- TOC index: ~33KB (2768 items × 12 bytes)
- ZIP batch: ~44KB (targets + sizes arrays)
All indexes cleared immediately after each phase. No OOM risk on
ESP32-C3.
## Note on Threshold
All optimizations are gated by `LARGE_SPINE_THRESHOLD = 400` to preserve
existing behavior for small books. However, the algorithms work
correctly for any book size and are faster even for small books:
| Book Size | Old O(n²) | New O(n log n) | Improvement |
|-----------|-----------|----------------|-------------|
| 10 ch | 100 ops | 50 ops | 2x |
| 100 ch | 10K ops | 800 ops | 12x |
| 400 ch | 160K ops | 4K ops | 40x |
If preferred, the threshold could be removed to use the optimized path
universally.
## Testing
- [x] Shadow Slave (2768 chapters): 50s first-time indexing, loads and
navigates correctly
- [x] Normal book (87 chapters): 1.7s indexing, no regression
- [x] Build passes
- [x] clang-format passes
## Files Changed
- `lib/Epub/Epub/parsers/ContentOpfParser.h/.cpp` - OPF manifest index
- `lib/Epub/Epub/BookMetadataCache.h/.cpp` - TOC index + batch size
lookup
- `lib/ZipFile/ZipFile.h/.cpp` - New `fillUncompressedSizes()` API
- `lib/Epub/Epub.cpp` - Timing logs
<details>
<summary><b>Algorithm Details</b> (click to expand)</summary>
### Phase 1: OPF Pass - Manifest to Spine Lookup
**Problem**: Each `<itemref idref="ch001">` in spine must find matching
`<item id="ch001" href="...">` in manifest.
```
OLD: For each of 2768 spine refs, scan all 2770 manifest items
= 7.6M string comparisons
NEW: While parsing manifest, build index:
{ hash("ch001"), len=5, file_offset=120 }
Sort index, then binary search for each spine ref:
2768 × log₂(2770) ≈ 2768 × 11 = 30K comparisons
```
### Phase 2: TOC Pass - TOC Entry to Spine Index Lookup
**Problem**: Each TOC entry with `href="chapter0001.xhtml"` must find
its spine index.
```
OLD: For each of 2768 TOC entries, scan all 2768 spine entries
= 7.6M string comparisons
NEW: At beginTocPass(), read spine once and build index:
{ hash("OEBPS/chapter0001.xhtml"), len=25, spineIndex=0 }
Sort index, binary search for each TOC entry:
2768 × log₂(2768) ≈ 30K comparisons
Clear index at endTocPass() to free memory.
```
### Phase 3: buildBookBin - ZIP Size Lookup
**Problem**: Need uncompressed file size for each spine item (for
reading progress). Sizes are in ZIP central directory.
```
OLD: For each of 2768 spine items, scan ZIP central directory (2773 entries)
= 7.6M filename reads + string comparisons
Time: 506 seconds
NEW:
Step 1: Build targets from spine
{ hash("OEBPS/chapter0001.xhtml"), len=25, index=0 }
Sort by (hash, len)
Step 2: Single pass through ZIP central directory
For each entry:
- Compute hash ON THE FLY (no string allocation)
- Binary search targets
- If match: sizes[target.index] = uncompressedSize
Step 3: Use sizes array directly (O(1) per spine item)
Total: 2773 entries × log₂(2768) ≈ 33K comparisons
Time: 35 seconds
```
### Why Hash + Length?
Using 64-bit FNV-1a hash + string length as a composite key:
- Collision probability: ~1 in 2⁶⁴ × typical_path_lengths
- No string storage needed in index (just 12-16 bytes per entry)
- Integer comparisons are faster than string comparisons
- Verification on match handles the rare collision case
</details>
---
_AI-assisted development. All changes tested on hardware._
## Summary
* **What is the goal of this PR?** Add Spanish language hyphenation
support to improve text rendering for Spanish books.
* **What changes are included?**
- Added Spanish hyphenation trie (`hyph-es.trie.h`) generated from
Typst's hypher patterns
- Registered `spanishHyphenator` in `LanguageRegistry.cpp` for language
tag `es`
- Added Spanish to the hyphenation evaluation test suite
- Added Spanish test data file with 5000 test cases
## Additional Context
* **Test Results:** Spanish hyphenation achieves 99.02% F1 Score (97.72%
perfect matches out of 5000 test cases)
* **Compatibility:** Works automatically for EPUBs with
`<dc:language>es</dc:language>` (or es-ES, es-MX, etc.)
<img width="115" height="189" alt="imagen"
src="https://github.com/user-attachments/assets/9b92e7fc-b98d-48af-8d53-dfdc2e68abee"
/>
| Metric | Value |
|--------|-------|
| Perfect matches | 97.72% |
| Overall Precision | 99.33% |
| Overall Recall | 99.42% |
| Overall F1 Score | 99.38% |
---
### AI Usage
Did you use AI tools to help write this code? _**PARTIALLY**_
AI assisted with:
- Guiding and compile
- Preparing the PR description
## Summary
This was originally a comment in #499, but I'm making it its own PR,
because it doesn't depend on anything there and then I can base that PR
on this one.
Currently, `drawBitmap` is used for covers and sleep wallpaper, and
`drawImage` is used for the boot logo. `drawBitmap` goes row by row and
pixel by pixel, so it respects the renderer orientation. `drawImage`
just calls the `EInkDisplay`'s `drawImage`, which works in the eink
panel's native display orientation.
`drawImage` rotates the x,y coordinates where it's going to draw the
image, but doesn't account for the fact that the northwest corner in
portrait orientation becomes, the southwest corner of the image
rectangle in the native orientation. The boot and sleep activities
currently work around this by calculating the north*east* corner of
where the image should go, which becomes the northwest corner after
`rotateCoordinates`.
I think this wasn't really apparent because the CrossPoint logo is
rotationally symmetrical. The `EInkDisplay` `drawImage` always draws the
image in native orientation, but that looks the same for the "X" image.
If we rotate the origin coordinate in `GfxRenderer`'s `drawImage`, we
can use a much clearer northwest corner coordinate in the boot and sleep
activities. (And then, in #499, we can actually rotate the boot screen
to the user's preferred orientation).
This does *not* yet rotate the actual bits in the image; it's still
displayed in native orientation. This doesn't affect the
rotationally-symmetric logo, but if it's ever changed, we will probably
want to allocate a new `u8int[]` and transpose rows and columns if
necessary.
## Additional Context
I've created an additional branch on top of this to demonstrate by
replacing the logo with a non-rotationally-symmetrical image:
<img width="128" height="128" alt="Cat-in-a-pan-128-bw"
src="https://github.com/user-attachments/assets/d0b239bc-fe75-4ec8-bc02-9cf9436ca65f"
/>
https://github.com/crosspoint-reader/crosspoint-reader/compare/master...maeveynot:rotated-cat
(many thanks to https://notisrac.github.io/FileToCArray/)
As you can see, it is always drawn in native orientation, which makes it
sideways (turned clockwise) in portrait.
---
### AI Usage
No
Co-authored-by: Maeve Andrews <maeve@git.mail.maeveandrews.com>
## Summary
* Extract author from XTC/XTCH files
## Additional Context
* Based on updated details in
https://gist.github.com/CrazyCoder/b125f26d6987c0620058249f59f1327d
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? No
## Summary
* **What is the goal of this PR?** Simple quality of life, ease
maintenance
* **What changes are included?**
Update fontconvert.py to include the command used to generate each font
file in the header comment, making it easier to regenerate fonts when
needed.
I plan on adding options to this scripts (kerning, and maybe ligatures),
thus knowing which command was used, even with already existing options
like `--additional-intervals`, is important.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
* **What is the goal of this PR?**
Resolve [KoSync progress does not sync between Crosspoint-reader and
KOReader
(Kindle)](https://github.com/crosspoint-reader/crosspoint-reader/issues/502)
* **What changes are included?**
KOReaderDocumentId::getOffset() - Update the value for the md5 offset
calculation to match KOReader.
## Additional Context
I've tested this with a couple of my ebooks and binary matching with
KOReader sync seems to be working fine now for both pushing and pulling
progress.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
Should address issues discussed in #168 and potentially fix#478.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_
## Summary
* Fixes#519
* Refactors repeated code into new function:
`ChapterHtmlSlimParser::flushPartWordBuffer()`
## Additional Context
* The `<br/>` tag is self closing and _in-line_, so the existing logic
for closing block tags does not get applied to `<br/>` tags.
* This PR adds the _in-line_ logic to:
* Flush the word preceding the `<br/>` tag from `partWordBuffer` to
`currentTextBlock` before calling `startNewTextBlock`
* **New function**: `ChapterHtmlSlimParser::flushPartWordBuffer()`
* **Purpose**: Consolidates the logic for flushing `partWordBuffer` to
`currentTextBlock`
* **Impact**: Simplifies `ChapterHtmlSlimParser::characterData(…)`,
`ChapterHtmlSlimParser::startElement(…)`, and
`ChapterHtmlSlimParser::endElement(…)` by integrating reused code into
single function
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
* Add additional punctuation marks to the list of characters that can be
immediately followed by a line break even where there is no explicit
space
## Additional Context
* Huge appreciation to @osteotek for his amazing work on hyphenation.
Reading on the device is so much better now.
* I am getting bad line breaks when ellipses (…) are between words and
book file does not explicitly include some kind of breaking space.
* Per
[discussion](https://github.com/crosspoint-reader/crosspoint-reader/pull/305#issuecomment-3765411406),
several new characters are added in this PR to the `isExplicitHyphen`
list to allow line breaks immediately after them:
Character | Unicode | Usage | Why include it?
-- | -- | -- | --
Solidus (Slash) | U+002F | / | Essential for breaking URLs and "and/or"
constructs.
Backslash | U+005C | \ | Critical for technical text, file paths, and
coding documentation.
Underscore | U+005F | _ | Prevents "runaway" line lengths in usernames
or code snippets.
Middle Dot | U+00B7 | · | Acts as a semantic separator in dictionaries
or stylistic lists.
Ellipsis | U+2026 | … | Prevents justification failure when dialogue
lacks following spaces.
Midline Horizontal Ellipsis | U+22EF | ⋯ | Useful for mathematical
sequences and technical notation.
### Example:
This shows an example of what line breaking looks like *with* this PR.
Note the line break after "matter…" (which would not previously have
been allowed). It's particularly important here because the book
includes non-breaking spaces in "Mr. Aldrich" and "Mr. Rockefeller."
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **PARTIALLY**
## Summary
- Rewrite OpdsParser to stream parsing instead of full content
- Fix OOM due to big http xml response
Closes#385
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
* Include superscripts and subscripts in fonts
## Additional Context
* Original change came from
https://github.com/crosspoint-reader/crosspoint-reader/pull/248
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? No
---------
Co-authored-by: cor <cor@pruijs.dev>
## Summary
* Adds (optional) Hyphenation for English, French, German, Russian
languages
## Additional Context
* Included hyphenation dictionaries add approximately 280kb to the flash
usage (German alone takes 200kb)
* Trie encoded dictionaries are adopted from hypher project
(https://github.com/typst/hypher)
* Soft hyphens (and other explicit hyphens) take precedence over
dict-based hyphenation. Overall, the hyphenation rules are quite
aggressive, as I believe it makes more sense on our smaller screen.
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
The current behavior of rendering `?` for an unknown Unicode character
can be hard to distinguish from a typo. Use the standard Unicode
"replacement character" instead, that's what it's designed for:
https://en.wikipedia.org/wiki/Specials_(Unicode_block)
I'm making this PR as a draft because I'm not sure I did everything that
was needed to change the character set covered by the fonts. Running
that script is in its own commit. If this is proper, I'll rebase/squash
into one commit and un-draft.
Co-authored-by: Maeve Andrews <maeve@git.mail.maeveandrews.com>
## Summary
- Adds KOReader progress sync integration, allowing CrossPoint to sync
reading positions with other
KOReader-compatible devices
- Stores credentials securely with XOR obfuscation
- Uses KOReader's partial MD5 document hashing for cross-device book
matching
- Syncs position via percentage with estimated XPath for compatibility
# Features
- Settings: KOReader Username, Password, and Authenticate options
- Sync from chapters menu: "Sync Progress" option appears when
credentials are configured
- Bidirectional sync: Can apply remote progress or upload local progress
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
* **What changes are included?**
- Fix inverted colors in Continue Reading cover image for 1-bit XTC
files
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
- Fix `grayValue = pixelBit ? 0 : 255` → `grayValue = pixelBit ? 255 :
0` in `lib/Xtc/Xtc.cpp`
- The thumb BMP generation had inverted polarity compared to cover BMP
generation
- bit=0 should be black, bit=1 should be white (matching the BMP palette
order)
- Update misleading comment about XTC polarity
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_
~~Quick~~ fix for
https://github.com/daveallie/crosspoint-reader/issues/362
(this just applies to the chapter selection menu:)
~~If the orientation is portrait, hints as we know them make sense to
draw. If the orientation is inverted, we'd have to change the order of
the labels (along with everything's position), and if it's one of the
landscape choices, we'd have to render the text and buttons vertically.
All those other cases will be more complicated.~~
~~Punt on this for now by only rendering if portrait.~~
Update: this now draws the hints at the physical button position no
matter what the orientation is, by temporarily changing orientation to
portrait.
---------
Co-authored-by: Maeve Andrews <maeve@git.mail.maeveandrews.com>
Let's start small by showing the ALT text of IMG. This is rudimentary,
but avoids those otherwise completely blank chapters.
I feel we will need this even when we can render images if that
rendering takes >1s - I would then prefer rendering optional and showing
the ALT text first.
## Summary
* **What is the goal of this PR?**: Fix the bug I reported in
https://github.com/daveallie/crosspoint-reader/issues/292
* **What changes are included?**: Instead of silently dropping table
content in EPUBs., replace with an italicized '[Table omitted]' message
where tables appear.
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY **_
---------
Co-authored-by: Evan Fenner <evan@evanfenner.com>
Co-authored-by: Warp <agent@warp.dev>
Currently there is no visual indication whatsoever if something is in a
list. An `<li>` is essentially just another paragraph.
As a partial remedy for this, add a bullet character to the beginning of
`<li>` text blocks so that the user can see that they're list items.
This is incomplete in that an `<ol>` should also have a counter so that
its list items can get numbers instead of bullets (right now I don't
think we track if we're in a `<ul>` or an `<ol>` at all), but it's
strictly better than the current situation.
Co-authored-by: Maeve Andrews <maeve@git.mail.maeveandrews.com>
Currently, when Extra Paragraph Spacing is off, an em-space is added to
the beginning of each ParsedText even for blocks like headers that are
centered. This whitespace makes the centering slightly off. Change the
calculation here to only add the em-space for left/justified text.
Co-authored-by: Maeve Andrews <maeve@git.mail.maeveandrews.com>
Dave Allie
Xuan-Son NguyenandGitHub
Daniel ChellingandGitHub
LaloandGitHub
Boris FaureandGitHub
Carson HicksandGitHub
Jonas DiemerandGitHub
Vincent PolitzerandGitHub
Luke SteinandGitHub
KasyanDiGrisandGitHub
Eunchurn ParkandGitHub
