9 Commits

Include renames

Author	SHA1	Message	Date
3 people	075ad7d021	fix: Use font metrics for combining mark positioning (#1310) ... ## Summary **What is the goal of this PR?** (e.g., Implements the new feature for file uploading.) Combining diacritical marks (U+0300–U+036F) were positioned using a heuristic that centered them at the midpoint of the base glyph's **advance width**. This worked acceptably for Bookerly but produced visibly off-center marks for Noto Sans due to a fundamental difference in how the two fonts design their combining mark metrics. Builds on the work of #1037. ## The problem The two built-in body fonts encode combining mark `left` offsets with very different conventions: | Mark | Bookerly `left` | Noto Sans `left` | |---|---|---| | U+0301 (acute) | -2 | -10 | | U+0300 (grave) | -5 | -15 | | U+0302 (circumflex) | -5 | -5 | | U+0323 (dot below) | -2 | -11 | Noto Sans uses large negative `left` values because its marks are designed for placement at the post-advance cursor position, with `left` pulling the bitmap back over the base glyph. Bookerly uses small offsets because its marks sit closer to the glyph origin. The old `advance/2` centering split the difference poorly — it happened to land close to correct for Bookerly but placed Noto Sans marks roughly 6px left of center on a typical lowercase letter. There was also a bug in the vertical gap heuristic. It unconditionally computed a `raiseBy` value to prevent above-baseline marks from colliding with tall base glyphs, but it applied the same logic to **below-baseline** marks like cedilla (U+0327), dot below (U+0323), and ogonek (U+0328). For those marks, the math produced a large positive raise (e.g., 24px for dot-below on 'a'), launching them above the x-height instead of keeping them below the baseline. ## The fix **Horizontal positioning**: Instead of centering at `advance/2`, align the mark bitmap's visual midpoint directly over the base glyph bitmap's visual midpoint. This uses the base glyph's actual `left` and `width` rather than its advance width, producing correct results regardless of how the font encodes its mark offsets. **Vertical positioning**: The raise heuristic now checks `markTop - markHeight > 0` and skips below-baseline marks entirely, leaving them at their font-designed position. **Consolidation**: The shared math is extracted into two `constexpr` helpers (`combiningMark::centerOver` and `combiningMark::raiseAboveBase`) in `EpdFontData.h`, eliminating the previously triplicated inline calculations across `drawText`, `drawTextRotated90CW`, and `getTextBounds`. The `MIN_COMBINING_GAP_PX` constant is also centralized as `combiningMark::MIN_GAP_PX`. | Before | After | | -- | -- | | <img src="https://github.com/user-attachments/files/25752257/before-noto.bmp" width="250" /> | <img src="https://github.com/user-attachments/files/25752258/after-noto.bmp" width="250" /> | | <img src="https://github.com/user-attachments/files/25752259/before-bookerly.bmp" width="250" /> | <img src="https://github.com/user-attachments/files/25752260/after-bookerly.bmp" width="250" /> | --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES to analyze differences between Noto Sans and Bookerly font metrics**_ --------- Co-authored-by: Uri Tauber <142022451+Uri-Tauber@users.noreply.github.com>	2026-04-14 14:35:50 -05:00
Zach NelsonandGitHub	1398aeb1ed	fix: Use differential rounding for consistent inter-glyph spacing (#1413) ... ## Summary **What is the goal of this PR?** A tweak to the fixed-point x-advance and kerning calculations to ensure that the spacing between any two glyphs is always calculated consistently. I noticed that sometimes I'd see common character pairs like "oo" more than once on a page, and the distance between the two snapped to different pixels depending on the running accumulated error for the line of text. This change uses a differential rounding approach where each glyph's x-advance plus the kerning relative to the next glyph are combined in fixed-point precision, then snapped to a pixel to draw the next glyph. This results in a consistent inter-glyph spacing any time the same two glyphs show up adjacent to each other, regardless of the accumulated error across the line. --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**PARTIALLY**_	2026-04-07 22:20:50 -05:00
Adrian Wilkins-CaruanaandGitHub	f1e9dc7f30	perf: font-compression improvements (#1056) ... ## Purpose This PR includes some preparatory changes that are needed for an upcoming performant CJK font feature. The changes have no impact on render time and heap allocation for latin text. **Despite this, I think these changes stand on their own as a better font compression/decompression implementation.** ## Summary - Font decompressor rewrite: Replaced the 4-slot LRU group cache with a two-tier system — a page buffer (glyphs prewarmed before rendering begins) and a hot-group fallback (last decompressed group retained for non-prewarmed glyphs). - Byte-aligned compressed bitmap format: Glyph bitmaps within compressed groups are now stored row-padded rather than tightly packed before DEFLATE compression, improving compression ratios by making identical pixel rows produce identical byte patterns. Glyphs are compacted back to packed format on demand at render time. Reduces flash size by 155 KB. - Page prewarm system: Added `Page::collectText` and `Page::getDominantStyle` to extract per-style glyph requirements before rendering, and `GfxRenderer::prewarmFontCache` to pre-decompress only the groups needed for the dominant style — eliminating mid-render decompression for the common case. - UTF-8 robustness fixes: `utf8NextCodepoint` now validates continuation bytes and returns a replacement glyph on malformed input; `ChapterHtmlSlimParser` correctly preserves incomplete multi-byte sequences across word-buffer flush boundaries rather than splitting them. --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES**_ Architecture and design was done by me, refined a bit by Claude. Code mostly by Claude, but not entirely.	2026-03-11 21:05:46 +01:00
Zach NelsonandGitHub	7dc518624c	fix: Use fixed-point fractional x-advance and kerning for better text layout (#1168) ... ## Summary **What is the goal of this PR?** Hopefully fixes #1182. _Note: I think letterforms got a "heavier" appearance after #1098, which makes this more noticeable. The current version of this PR reverts the change to add `--force-autohint` for Bookerly, which to me seems to bring the font back to a more aesthetic and consistent weight._ #### Problem Character spacing was uneven in certain words. The word "drew" in Bookerly was the clearest example: a visible gap between `d` and `r`, while `e` and `w` appeared tightly condensed. The root cause was twofold: 1. **Integer-only glyph advances.** `advanceX` was stored as a `uint8_t` of whole pixels, sourced from FreeType's hinted `advance.x` (which grid-fits to integers). A glyph whose true advance is 15.56px was stored as 16px -- an error of +0.44px per character that compounds across a line. 2. **Floor-rounded kerning.** Kern adjustments were converted with `math.floor()`, which systematically over-tightened negative kerns. A kern of -0.3px became -1px -- a 0.7px over-correction that visibly closed gaps. Combined, these produced the classic symptom: some pairs too wide, others too tight, with the imbalance varying per word. #### Solution: fixed-point accumulation with 1/16-pixel resolution, for sub-pixel precision during text layout All font metrics now use a "fixed-point 4" format -- 4 fractional bits giving 1/16-pixel (0.0625px) resolution. This is implemented with plain integer arithmetic (shifts and adds), requiring no floating-point on the ESP32. **How it works:** A value like 15.56px is stored as the integer `249`: ``` 249 = 15 * 16 + 9 (where 9/16 = 0.5625, closest to 0.56) ``` Two storage widths share the same 4 fractional bits: | Field | Type | Format | Range | Use | |-------|------|--------|-------|-----| | `advanceX` | `uint16_t` | 12.4 | 0 -- 4095.9375 px | Glyph advance width | | `kernMatrix` | `int8_t` | 4.4 | -8.0 -- +7.9375 px | Kerning adjustment | Because both have 4 fractional bits, they add directly into a single `int32_t` accumulator during layout. The accumulator is only snapped to the nearest whole pixel at the moment each glyph is rendered: ```cpp int32_t xFP = fp4::fromPixel(startX); // pixel to 12.4: startX << 4 for each character: xFP += kernFP; // add 4.4 kern (sign-extends into int32_t) int xPx = fp4::toPixel(xFP); // snap to nearest pixel: (xFP + 8) >> 4 render glyph at xPx; xFP += glyph->advanceX; // add 12.4 advance ``` Fractional remainders carry forward indefinitely. Rounding errors stay below +/- 0.5px and never compound. #### Concrete example: "drew" in Bookerly **Before** (integer advances, floor-rounded kerning): | Char | Advance | Kern | Cursor | Snap | Gap from prev | |------|---------|------|--------|------|---------------| | d | 16 px | -- | 33 | 33 | -- | | r | 12 px | 0 | 49 | 49 | ~2px | | e | 13 px | -1 | 60 | 60 | ~0px | | w | 22 px | -1 | 72 | 72 | ~0px | The d-to-r gap was visibly wider than the tightly packed `rew`. **After** (12.4 advances, 4.4 kerning, fractional accumulation): | Char | Advance (FP) | Kern (FP) | Accumulator | Snap | Ink start | Gap from prev | |------|-------------|-----------|-------------|------|-----------|---------------| | d | 249 (15.56px) | -- | 528 | 33 | 34 | -- | | r | 184 (11.50px) | 0 | 777 | 49 | 49 | 0px | | e | 208 (13.00px) | -8 (-0.50px) | 953 | 60 | 61 | 1px | | w | 356 (22.25px) | -4 (-0.25px) | 1157 | 72 | 72 | 0px | Spacing is now `0, 1, 0` pixels -- nearly uniform. Verified on-device: all 5 copies of "drew" in the test EPUB produce identical spacing, confirming zero accumulator drift. #### Changes **Font conversion (`fontconvert.py`)** - Use `linearHoriAdvance` (FreeType 16.16, unhinted) instead of `advance.x` (26.6, grid-fitted to integers) for glyph advances - Encode kern values as 4.4 fixed-point with `round()` instead of `floor()` - Add `fp4_from_ft16_16()` and `fp4_from_design_units()` helper functions - Add module-level documentation of fixed-point conventions **Font data structures (`EpdFontData.h`)** - `EpdGlyph::advanceX`: `uint8_t` to `uint16_t` (no memory cost due to existing struct padding) - Add `fp4` namespace with `constexpr` helpers: `fromPixel()`, `toPixel()`, `toFloat()` - Document fixed-point conventions **Font API (`EpdFont.h/cpp`, `EpdFontFamily.h/cpp`)** - `getKerning()` return type: `int8_t` to `int` (to avoid truncation of the 4.4 value) **Rendering (`GfxRenderer.cpp`)** - `drawText()`: replace integer cursor with `int32_t` fixed-point accumulator - `drawTextRotated90CW()`: same accumulator treatment for vertical layout - `getTextAdvanceX()`, `getSpaceWidth()`, `getSpaceKernAdjust()`, `getKerning()`: convert from fixed-point to pixel at API boundary **Regenerated all built-in font headers** with new 12.4 advances and 4.4 kern values. #### Memory impact Zero additional RAM. The `advanceX` field grew from `uint8_t` to `uint16_t`, but the `EpdGlyph` struct already had 1 byte of padding at that position, so the struct size is unchanged. The fixed-point accumulator is a single `int32_t` on the stack. #### Test plan - [ ] Verify "drew" spacing in Bookerly at small, medium, and large sizes - [ ] Verify uppercase kerning pairs: AVERY, WAVE, VALUE - [ ] Verify ligature words: coffee, waffle, office - [ ] Verify all built-in fonts render correctly at each size - [ ] Verify rotated text (progress bar percentage) renders correctly - [ ] Verify combining marks (accented characters) still position correctly - [ ] Spot-check a full-length book for any layout regressions --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES, Claude Opus 4.6 helped figure out a non-floating point approach for sub-pixel error accumulation**_	2026-03-01 10:43:37 -06:00
3 people	0eb8a9346b	feat: Support for kerning and ligatures (#873) ... ## Summary **What is the goal of this PR?** Improved typesetting, including [kerning](https://en.wikipedia.org/wiki/Kerning) and [ligatures](https://en.wikipedia.org/wiki/Ligature_(writing)#Latin_alphabet). **What changes are included?** - The script to convert built-in fonts now adds kerning and ligature information to the generated font headers. - Epub page layout calculates proper kerning spaces and makes ligature substitutions according to the selected font.    ## Additional Context - I am not a typography expert. - The implementation has been reworked from the earlier version, so it is no longer necessary to omit Open Dyslexic, and kerning data now covers all fonts, styles, and codepoints for which we include bitmap data. - Claude Opus 4.6 helped with a lot of this. - There's an included test epub document with lots of kerning and ligature examples, shown in the photos. **_After some time to mature, I think this change is in decent shape to merge and get people testing._** After opening this PR I came across #660, which overlaps in adding ligature support. --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES, Claude Opus 4.6**_ --------- Co-authored-by: Cursor <cursoragent@cursor.com>	2026-02-24 11:31:43 +03:00
3 people	47aa0dda76	perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831) ... ## Summary **What is the goal of this PR?** Compress reader font bitmaps to reduce flash usage by 30.7%. **What changes are included?** - New `EpdFontGroup` struct and extended `EpdFontData` with `groups`/`groupCount` fields - `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group + groups of 8) and compresses each with raw DEFLATE - `FontDecompressor` class with 4-slot LRU cache for on-demand decompression during rendering - `GfxRenderer` transparently routes bitmap access through `getGlyphBitmap()` (compressed or direct flash) - Uses `uzlib` for decompression with minimal heap overhead. - 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated with compression; 5 UI fonts unchanged - Round-trip verification script (`verify_compression.py`) runs as part of font generation ## Additional Context ## Flash & RAM | | baseline | font-compression | Difference | |--|--------|-----------------|------------| | Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B (-30.7%) | | firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) | | RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) | ## Script-Based Grouping (Cold Cache) Comparison of uncompressed baseline vs script-based group compression (4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic, General Punctuation, etc.) instead of sequential groups of 8. ### Render Time | | Baseline | Compressed (cold cache) | Difference | |---|---|---|---| | **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) | | **Pages** | 37 | 37 | | ### Memory Usage | | Baseline | Compressed (cold cache) | Difference | |---|---|---|---| | **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB | | **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB | | **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB | | **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB | ### Cache Effectiveness | | Misses/page | Hit rate | |---|---|---| | **Compressed (cold cache)** | 2.1 | 99.85% | ------ ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES**_ Implementation was done by Claude Code (Opus 4.6) based on a plan developed collaboratively. All generated font headers were verified with an automated round-trip decompression test. The firmware was compiled successfully but has not yet been tested on-device. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>	2026-02-19 20:30:15 +11:00
Dave AllieandGitHub	2ed8017aa2	Move to SDK EInkDisplay and enable anti-aliased 2-bit text (#5) ... * First pass at moving to SDK EInkDisplay library * Add 2-bit grayscale text and anti-aliased rendering of text * Render status bar for empty chapters * Refresh screen every 15 pages to avoid ghosting * Simplify boot and sleep screens * Give FileSelectionScreen task more stack memory * Move text around slightly on Boot and Sleep screens * Re-use existing buffer and write to whole screen for 'partial update'	2025-12-08 19:48:49 +11:00
Dave Allie	79294f6b8f	Build and use 1-bit font, saves a good amount of space	2025-12-07 01:26:49 +11:00
Dave Allie	2ccdbeecc8	Public release	2025-12-03 22:06:45 +11:00