63 Commits
Author SHA1 Message Date
Justin MitchellandClaude Fable 5 460c530888 Bump version to 0.2.7
Fixes #45

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v0.2.7
2026-08-02 23:23:25 -04:00
Justin Mitchell 2bcb355d64 Add debug method for device detection diagnostics
Implements debug_managed_device_detection() to provide verbose output about CrossPoint Reader network detection, including connection status, discovery attempts, and error reporting. Bypasses discovery throttle to force fresh scans during debugging.
2026-08-02 23:22:32 -04:00
Justin Mitchell 3687d5030b Merge pull request #44 from stutrek/entities-numeric
Stop dropping the DTD so entities can resolve
2026-07-28 12:24:24 -04:00
Stu KabakoffandClaude Opus 5 3668729fd9 Keep each document's DOCTYPE through the rewrite
etree.tostring writes out the element tree alone, so the optimizer was
silently dropping the DOCTYPE of every document it re-serialized — and a
document that loses its DTD loses every named entity with it, leaving
&nbsp; and friends as fatal "undefined entity" errors on-device. Hand
the declaration back to the serializer instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 10:41:36 -04:00
Justin MitchellandClaude Fable 5 37519a1deb Fix uploads to existing folders and prune empty folders on delete (#30)
- Check directory existence before /mkdir so re-sending books into
  existing template folders no longer errors or times out; tolerate
  mkdir errors when the folder turns out to exist
- After deleting books, remove any template directories left empty
  (deepest first, never the base upload path or root)
- Stop tracking release zips and __pycache__; ignore them going forward
- Bump version to 0.2.6

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v0.2.6
2026-07-15 11:23:08 -04:00
Justin Mitchell 7354385cb7 Reduce mkdir failures on devices
Some firmware versions fail with non-400 codes or time out when attempting to create an existing directory. Now check if the directory actually exists before raising an error, and proactively check before mkdir to avoid triggering the firmware bug. Also increase timeout from 5s to 10s to reduce spurious failures.
2026-07-15 11:22:28 -04:00
Justin Mitchell 07cf706348 Remove old release archives
Clean up versioned zip files from v0.1.1 through v0.2.5 to reduce repository size and clutter. Release artifacts should be managed through proper release channels rather than committed to the repository.
2026-07-15 11:08:47 -04:00
Justin Mitchell 04ae610391 Merge pull request #41 from thiagokokada/fix-crosspoint-image-sizing
Normalize problematic EPUB image sizing
2026-07-15 11:03:30 -04:00
Justin Mitchell 33ac4dc0ce Merge pull request #39 from ituri/split-text-for-low-ram-firmware
Optimizer: split oversized paragraphs/chapters, strip fonts, extract data-URI images
2026-07-15 11:02:28 -04:00
Justin Mitchell 37233ae3ea Merge pull request #32 from karimknaebel/feat-upload-template
Add configurable upload template
2026-07-15 11:00:51 -04:00
Karim Knaebel 82bb050253 Merge remote-tracking branch 'upstream/master' into feat-upload-template
# Conflicts:
#	crosspoint_reader/README.md
#	crosspoint_reader/config.py
2026-07-15 14:58:36 +02:00
Thiago Kenji Okada 73a703f3b4 Harden inline image style cleanup 2026-07-14 21:54:02 +01:00
Thiago Kenji Okada ef124ca69e Normalize problematic EPUB image sizing 2026-07-14 21:40:21 +01:00
Justin Mitchell 2bd9c7cc67 Merge pull request #29 from 42lizard/master
Fix {id} template field resolving to -1 in upload filenames
2026-07-14 16:11:47 -04:00
Justin Mitchell 3a25234fa0 Merge pull request #40 from thiagokokada/reduce-crosspoint-upload-errors
Improve CrossPoint upload reliability
2026-07-14 16:07:45 -04:00
Thiago Kenji Okada 4d83b73dd9 Share CrossPoint delete request helper 2026-07-13 19:57:44 +01:00
Thiago Kenji Okada 1565fd8fef Wrap upload file IO errors 2026-07-13 19:56:03 +01:00
Thiago Kenji Okada ea168a501e Improve CrossPoint upload reliability 2026-07-13 19:44:58 +01:00
philandClaude Fable 5 a8de40542a Split paragraphs wrapped in a single inline element or containing <img>
Some publishers wrap entire multi-KB paragraphs in one <span>; such a
paragraph survived splitting because tag tokens were atomic. Oversized
inline wrappers (span/em/i/b/strong/a/...) are now split recursively
into several same-tag siblings. <img> no longer blocks paragraph
splitting either — it is inline and stays atomic during grouping.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 12:03:46 +02:00
philandClaude Fable 5 9ec85ca476 Handle processing instructions; make sentence splitting lossless
- parse_nodes treated XML processing instructions (<?dp ...?> page
  markers, common in Random House EPUBs) as unclosed opening tags,
  aborting the text pass for every affected file.
- split_text_sentences dropped the boundary whitespace between pieces;
  when two pieces later landed in the same paragraph group the sentences
  were joined without a space. The visible-text verification caught it
  and reverted those files (no corrupt output), but the split was lost.
  Boundary whitespace now stays with the preceding piece, so
  ''.join(pieces) == text always holds.

Found by batch-converting a 253-book library: 4 books with PI page
markers now split normally, 2 books that previously reverted now pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 11:58:44 +02:00
philandClaude Fable 5 029ee7da11 Address review: remap fragment-only links, tidy guard, document checkbox
- Fragment-only refs (href="#note1") are relative to the file they sit
  in; after a file split the anchor may live in a sibling chunk. Track
  each chunk's original filename and rewrite such refs to the chunk
  that owns the anchor (verified with a synthetic EPUB: 25 local
  footnote links across 3 chunks plus one cross-file ref all resolve).
- Drop the dead `or True` condition; page-list nav removal always runs,
  link remapping only when anchors moved.
- List the new split checkbox in the README's step-by-step options.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 11:47:59 +02:00
philandClaude Fable 5 25370f6e29 Split oversized paragraphs/chapters and strip fonts for low-RAM firmware
The firmware lays out one paragraph at a time, holding every word of the
paragraph in parallel in-RAM vectors (ParsedText.cpp). On ~380KB-RAM
devices a single multi-KB <p> — common in literary fiction with
page-long paragraphs (Fosse, Knausgård, Bernhard, ...) — exhausts the
heap during section indexing and crashes the reader with out-of-memory,
even when the containing file is small. Large spine files and multi-KB
base64 data-URI images cause the same failure mode (see
crosspoint-reader/crosspoint-reader#2163, #1752).

Add an optional text pass (on by default, gated by the existing
optimizer checkbox plus a new "Split large chapters/paragraphs, remove
fonts" option) that runs after the image optimizer:

- split every <p> larger than 1.6 KB into ~1.2 KB siblings at sentence
  boundaries, keeping inline tags atomic and dropping duplicate id
  attributes on continuation paragraphs
- split spine files whose <body> exceeds 9.5 KB into ~7 KB files,
  expanding the OPF manifest/spine and remapping href="...#fragment"
  references onto the chunk that now holds the anchor
- extract base64 data-URI images into real zip entries (re-encoded
  through the existing image pipeline) so neither the attribute value
  nor the device's streaming parser has to swallow them
- remove embedded font files, their manifest items and @font-face rules
- drop page-list navs (print page numbers, dead weight on-device)

Every transformation verifies the visible text is unchanged and reverts
itself on any mismatch; any unexpected error leaves the EPUB as the
image pass produced it, so a transfer is never blocked.

Tested against a library of 250 EPUBs including worst cases: a 4.2 MB
EPUB with 88 KB single-paragraph chapters and an 88 KB data-URI page
(492 KB after, text identical), and a 3.9 MB EPUB with a 1.6 MB single
spine file (240 files after, text identical). Books that already fit
the limits pass through untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 11:36:44 +02:00
Justin Mitchell 6d66f3d325 Add crosspoint_reader v0.2.5 distribution package 2026-06-21 13:04:28 -04:00
Justin Mitchell 6e80153150 Mark on-device books by local library name match instead of downloading
The previous approach read each unknown book's metadata by downloading
it over HTTP. The device's web server streams whole files synchronously
and can't serve a partial/early-aborted read, so streaming just the OPF
stalled the single-threaded server and a connect with side-loaded books
took minutes.

Since a book can only be marked on-device if it exists in the library,
match locally instead: index the current library by normalized title and
resolve each device file by its file name, attaching the library book's
uuid so Calibre marks it. Same-title collisions are disambiguated by an
author appearing in the device path (order-independent, so author-sort
folders match); ambiguous files are left unmarked rather than mismatched.
No network transfer is involved.

The download path is kept only as an opt-in last resort (Fetch metadata,
off by default) for library books whose on-device name doesn't resemble
the title. Also adds per-book progress during books() and removes the
unused streaming reader.
v0.2.5
2026-06-21 13:04:14 -04:00
Justin Mitchell ebd7ce90ff Add library filename matching for device books
Recognize side-loaded books by matching device filenames against library titles, avoiding downloads. The plugin now uses three recognition methods in order: sent-book cache (instant), library name match (new, local), and optional EPUB fetch (slow, opt-in). This makes the fetch metadata setting unnecessary for most users whose device filenames resemble library titles.
2026-06-21 13:03:54 -04:00