feat(sync/koreader): Add KOReader reading progress synchronization (#1560)

* feat(sync/koreader): implement reading progress synchronization

This commit introduces a comprehensive integration with KOReader Sync Server to enable two-way synchronization of reading progress.

The core logic is encapsulated in a new `KoreaderSyncService`, which handles authentication, registration, and progress pushing/pulling based on user-defined strategies (LATEST, KOSYNC, SUWAYOMI).

Key changes include:

- A new GraphQL API is added to manage the KOReader Sync connection:
  - `connectKoSyncAccount` mutation provides a simplified flow that attempts to log in and, if the user doesn't exist, automatically registers them.
  - `logoutKoSyncAccount` mutation to clear credentials.
  - `koSyncStatus` query to check the current connection status.

- Reading progress is now synchronized at key points:
  - The `fetchChapterPages` mutation pulls the latest progress from the sync server before loading the reader. It respects the configured sync strategy and updates the local database if necessary.
  - The `updateChapters` and other progress-updating methods now push changes to the sync server automatically.
- OPDS chapter entries also pull the latest progress, ensuring clients receive up-to-date reading status.

- Supporting backend changes have been made:
  - The `Chapter` table is extended with a `koreader_hash` column to uniquely identify documents. A database migration is included.
  - New configuration options are added to `server.conf` to manage the feature (enable, server URL, credentials, strategy, etc.).

* perf(opds): defer KOReader sync to improve chapter feed performance

Removes the KOReader Sync progress-pulling logic from the `createChapterListEntry` function.

The previous implementation triggered a network request to the sync server for every single chapter when generating a list, leading to severe performance issues and slow load times on feeds with many entries.

This change reverts to the more performant approach of always linking to the chapter's metadata feed. The expensive sync operation will be handled within the metadata entry generation instead, ensuring it's only triggered on-demand for a single chapter. This restores the responsiveness of browsing chapter feeds.

* refactor(koreader): Use enums for sync settings and correct OPDS logic

Refactor Koreader Sync settings to use enums instead of raw strings for `checksumMethod` and `strategy`. This improves type safety, prevents typos, and makes the configuration handling more robust.

The changes include:
- Introducing `KoreaderSyncChecksumMethod` and `KoreaderSyncStrategy` enums.
- Updating `ServerConfig`, GraphQL types, and backup models to use these new enums.
- Refactoring `KoreaderSyncService` to work with the enum types.

Additionally, this commit fixes an issue in `OpdsEntryBuilder` where the logic for determining which sync progress to use (local vs. remote) was duplicated. The builder now correctly delegates this decision to `KoreaderSyncService.pullProgress`, which already contains the necessary strategy logic. This centralizes the logic and ensures consistent behavior.

* refactor(koreader): Improve config handling and remove redundant update

This commit combines several refactoring and cleanup tasks:

- **Koreader Sync:** The sync service is updated to use the modern `serverConfig` provider instead of the legacy `GlobalConfigManager`. This aligns it with the current configuration management approach in the project.

- **Download Provider:** A redundant `pageCount` database update is removed from `ChaptersFilesProvider`. This operation was unnecessary because the `getChapterDownloadReady` function, which is called earlier in the download process, already verifies and corrects the page count. This change eliminates a superfluous database write and fixes a related import issue.

* feat(sync/koreader)!: enhance sync strategy and add progress tolerance

This commit overhauls the KOReader synchronization feature to provide more granular control and robustness. The simple on/off toggle has been replaced with a more flexible strategy-based system.

Key changes include:
- Replaced `koreaderSyncEnabled` with a more powerful `koreaderSyncStrategy` enum.
- Introduced new sync strategies: `PROMPT`, `SILENT`, `SEND`, `RECEIVE`, and `DISABLE`, allowing for fine-grained control over the sync direction and conflict resolution.
- Added a `koreaderSyncPercentageTolerance` setting. This prevents unnecessary sync updates for minor progress differences between Suwayomi and KOReader.
- Refactored the `KoreaderSyncService` to implement the new strategies and use the configurable tolerance.
- Updated GraphQL schemas, mutations, and server configuration to remove the old setting and incorporate the new ones.
- Adjusted the backup and restore process to correctly handle the new configuration parameters.
- Modified API endpoints and internal logic to check and apply remote progress based on the selected strategy.

BREAKING CHANGE: The `koreaderSyncEnabled` setting is removed and replaced by a more granular `koreaderSyncStrategy`. The enum values for the strategy have been completely changed, making previous configurations incompatible.

* fix: remove unused imports

* feat(opds, sync): enhance Koreader sync and OPDS conflict handling

This commit introduces significant improvements to the Koreader synchronization feature, focusing on providing a better user experience for handling progress conflicts in both OPDS and GraphQL clients.

Key changes include:

- **OPDS Conflict Resolution:** When a reading progress conflict is detected, the OPDS feed for a chapter now provides two distinct "Read Online" links: one to continue from the local progress and another to continue from the synced progress from the remote device (e.g., "Continue Reading Online (Synced from KOReader)"). This empowers users to choose which progress to follow.

- **GraphQL Sync Conflict Information:** The `fetchChapterPages` GraphQL mutation now includes a `syncConflict` field in its payload. This field provides the remote device name and page number, allowing GraphQL clients to implement a user-facing prompt to resolve sync conflicts.

- **Improved Sync Strategy Handling:**
  - The `connectKoSyncAccount` mutation no longer unconditionally sets the sync strategy to `PROMPT`. It now respects the user's existing setting, preventing accidental configuration changes upon re-login.
  - The default `koreaderSyncStrategy` in the configuration is changed to `DISABLED`, providing a safer and more intuitive default for new users.

- **Refinements & Fixes:**
  - The fallback for the remote device name is now centralized within the KoreaderSyncService, defaulting to "KOReader" if not provided.
  - Renamed `KoreaderSyncStrategy.DISABLE` to `DISABLED` for consistency.
  - Updated i18n strings for OPDS links to be more descriptive and user-friendly.

* refactor(kosync): rename stream page link titles for consistency

* refactor(kosync): return SettingsType in auth mutation payloads

The `connectKoSyncAccount` and `logoutKoSyncAccount` mutations modify server settings (username and userkey) but did not previously return the updated configuration. This forced client applications to manually refetch settings to avoid a stale cache.

This change modifies the payloads for both mutations to include the full `SettingsType`.

By returning the updated settings directly, GraphQL clients like Apollo Client can automatically update their cache, simplifying client-side state management and ensuring the UI always reflects the current server configuration.

Additionally, `clientMutationId` has been added to `KoSyncConnectPayload` for consistency with GraphQL practices, aligning it with the logout mutation.

Refs: #1560

* refactor(kosync): replace KoSyncConnectPayload with ConnectResult in connect method

* fix(kosync): add koreaderSyncPercentageTolerance default setting
This commit is contained in:
Zeedif
2025-08-19 13:00:19 -06:00
committed by GitHub
parent 16e9c0b19a
commit 590e43c827
23 changed files with 926 additions and 50 deletions

View File

@@ -95,7 +95,12 @@
<string name="opds_linktitle_next_page">Next Page</string>
<string name="opds_linktitle_self_feed">Current Feed</string>
<string name="opds_linktitle_view_on_web">View on Web</string>
<string name="opds_linktitle_stream_pages">View Pages (Streaming)</string>
<string name="opds_linktitle_stream_pages_start">Read Online</string>
<string name="opds_linktitle_stream_pages_continue">Continue Reading Online</string>
<string name="opds_linktitle_stream_pages_start_local">Read Online (Local Progress)</string>
<string name="opds_linktitle_stream_pages_continue_local">Continue Reading Online (Local Progress)</string>
<string name="opds_linktitle_stream_pages_start_remote">Read Online (Synced from %1$s)</string>
<string name="opds_linktitle_stream_pages_continue_remote">Continue Reading Online (Synced from %1$s)</string>
<string name="opds_linktitle_download_cbz">Download CBZ</string>
<string name="opds_linktitle_chapter_cover">Chapter Cover</string>
<string name="opds_linktitle_view_chapter_details">View Chapter Details &amp; Get Pages</string>
@@ -106,7 +111,7 @@
<string name="opds_chapter_status_unread"></string>
<string name="opds_chapter_status_downloaded">⬇️ </string>
<string name="opds_chapter_details_base">%1$s | %2$s</string>
<string name="opds_chapter_details_base">Series: %1$s | %2$s</string>
<string name="opds_chapter_details_scanlator"> | By %1$s</string>
<string name="opds_chapter_details_progress"> | Progress: %1$d of %2$d</string>

View File

@@ -1,6 +1,8 @@
package suwayomi.tachidesk.graphql.mutations
import graphql.execution.DataFetcherResult
import kotlinx.coroutines.GlobalScope
import kotlinx.coroutines.launch
import org.jetbrains.exposed.dao.id.EntityID
import org.jetbrains.exposed.sql.SqlExpressionBuilder.eq
import org.jetbrains.exposed.sql.and
@@ -8,11 +10,14 @@ import org.jetbrains.exposed.sql.deleteWhere
import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.statements.BatchUpdateStatement
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.graphql.asDataFetcherResult
import suwayomi.tachidesk.graphql.types.ChapterMetaType
import suwayomi.tachidesk.graphql.types.ChapterType
import suwayomi.tachidesk.graphql.types.SyncConflictInfoType
import suwayomi.tachidesk.manga.impl.Chapter
import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReadyById
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.manga.model.table.ChapterMetaTable
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.JavalinSetup.future
@@ -68,8 +73,10 @@ class ChapterMutation {
ChapterTable
.select(ChapterTable.id, ChapterTable.pageCount)
.where { ChapterTable.id inList ids }
.groupBy { it[ChapterTable.id].value }
.mapValues { it.value.firstOrNull()?.let { it[ChapterTable.pageCount] } }
.associateBy(
{ it[ChapterTable.id].value },
{ it[ChapterTable.pageCount] },
)
} else {
emptyMap()
}
@@ -94,6 +101,15 @@ class ChapterMutation {
}
}
}
// Sync with KoreaderSync when progress is updated
if (patch.lastPageRead != null || patch.isRead == true) {
GlobalScope.launch {
ids.forEach { chapterId ->
KoreaderSyncService.pushProgress(chapterId)
}
}
}
}
fun updateChapter(input: UpdateChapterInput): DataFetcherResult<UpdateChapterPayload?> =
@@ -241,6 +257,7 @@ class ChapterMutation {
val clientMutationId: String?,
val pages: List<String>,
val chapter: ChapterType,
val syncConflict: SyncConflictInfoType?,
)
fun fetchChapterPages(input: FetchChapterPagesInput): CompletableFuture<DataFetcherResult<FetchChapterPagesPayload?>> {
@@ -249,7 +266,35 @@ class ChapterMutation {
return future {
asDataFetcherResult {
val chapter = getChapterDownloadReadyById(chapterId)
var chapter = getChapterDownloadReadyById(chapterId)
val syncResult = KoreaderSyncService.checkAndPullProgress(chapter.id)
var syncConflictInfo: SyncConflictInfoType? = null
if (syncResult != null) {
if (syncResult.isConflict) {
syncConflictInfo =
SyncConflictInfoType(
deviceName = syncResult.device,
remotePage = syncResult.pageRead,
)
}
if (syncResult.shouldUpdate) {
// Update DB for SILENT and RECEIVE
transaction {
ChapterTable.update({ ChapterTable.id eq chapter.id }) {
it[lastPageRead] = syncResult.pageRead
it[lastReadAt] = syncResult.timestamp
}
}
}
// For PROMPT, SILENT, and RECEIVE, return the remote progress
chapter =
chapter.copy(
lastPageRead = if (syncResult.shouldUpdate) syncResult.pageRead else chapter.lastPageRead,
lastReadAt = if (syncResult.shouldUpdate) syncResult.timestamp else chapter.lastReadAt,
)
}
val params =
buildString {
@@ -273,6 +318,7 @@ class ChapterMutation {
"/api/v1/manga/${chapter.mangaId}/chapter/${chapter.index}/page/${index}$params"
},
chapter = ChapterType(chapter),
syncConflict = syncConflictInfo,
)
}
}

View File

@@ -0,0 +1,43 @@
package suwayomi.tachidesk.graphql.mutations
import suwayomi.tachidesk.graphql.types.KoSyncConnectPayload
import suwayomi.tachidesk.graphql.types.LogoutKoSyncAccountPayload
import suwayomi.tachidesk.graphql.types.SettingsType
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.server.JavalinSetup.future
import java.util.concurrent.CompletableFuture
class KoreaderSyncMutation {
data class ConnectKoSyncAccountInput(
val clientMutationId: String? = null,
val username: String,
val password: String,
)
fun connectKoSyncAccount(input: ConnectKoSyncAccountInput): CompletableFuture<KoSyncConnectPayload> =
future {
val result = KoreaderSyncService.connect(input.username, input.password)
KoSyncConnectPayload(
clientMutationId = input.clientMutationId,
success = result.success,
message = result.message,
username = result.username,
settings = SettingsType(),
)
}
data class LogoutKoSyncAccountInput(
val clientMutationId: String? = null,
)
fun logoutKoSyncAccount(input: LogoutKoSyncAccountInput): CompletableFuture<LogoutKoSyncAccountPayload> =
future {
KoreaderSyncService.logout()
LogoutKoSyncAccountPayload(
clientMutationId = input.clientMutationId,
success = true,
settings = SettingsType(),
)
}
}

View File

@@ -215,6 +215,15 @@ class SettingsMutation {
updateSetting(settings.opdsShowOnlyUnreadChapters, serverConfig.opdsShowOnlyUnreadChapters)
updateSetting(settings.opdsShowOnlyDownloadedChapters, serverConfig.opdsShowOnlyDownloadedChapters)
updateSetting(settings.opdsChapterSortOrder, serverConfig.opdsChapterSortOrder)
// koreader sync
updateSetting(settings.koreaderSyncServerUrl, serverConfig.koreaderSyncServerUrl)
updateSetting(settings.koreaderSyncUsername, serverConfig.koreaderSyncUsername)
updateSetting(settings.koreaderSyncUserkey, serverConfig.koreaderSyncUserkey)
updateSetting(settings.koreaderSyncDeviceId, serverConfig.koreaderSyncDeviceId)
updateSetting(settings.koreaderSyncChecksumMethod, serverConfig.koreaderSyncChecksumMethod)
updateSetting(settings.koreaderSyncStrategy, serverConfig.koreaderSyncStrategy)
updateSetting(settings.koreaderSyncPercentageTolerance, serverConfig.koreaderSyncPercentageTolerance)
}
fun setSettings(input: SetSettingsInput): SetSettingsPayload {

View File

@@ -0,0 +1,13 @@
package suwayomi.tachidesk.graphql.queries
import suwayomi.tachidesk.graphql.types.KoSyncStatusPayload
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.server.JavalinSetup.future
import java.util.concurrent.CompletableFuture
class KoreaderSyncQuery {
fun koSyncStatus(): CompletableFuture<KoSyncStatusPayload> =
future {
KoreaderSyncService.getStatus()
}
}

View File

@@ -20,6 +20,7 @@ import suwayomi.tachidesk.graphql.mutations.DownloadMutation
import suwayomi.tachidesk.graphql.mutations.ExtensionMutation
import suwayomi.tachidesk.graphql.mutations.ImageMutation
import suwayomi.tachidesk.graphql.mutations.InfoMutation
import suwayomi.tachidesk.graphql.mutations.KoreaderSyncMutation
import suwayomi.tachidesk.graphql.mutations.MangaMutation
import suwayomi.tachidesk.graphql.mutations.MetaMutation
import suwayomi.tachidesk.graphql.mutations.SettingsMutation
@@ -32,6 +33,7 @@ import suwayomi.tachidesk.graphql.queries.ChapterQuery
import suwayomi.tachidesk.graphql.queries.DownloadQuery
import suwayomi.tachidesk.graphql.queries.ExtensionQuery
import suwayomi.tachidesk.graphql.queries.InfoQuery
import suwayomi.tachidesk.graphql.queries.KoreaderSyncQuery
import suwayomi.tachidesk.graphql.queries.MangaQuery
import suwayomi.tachidesk.graphql.queries.MetaQuery
import suwayomi.tachidesk.graphql.queries.SettingsQuery
@@ -74,6 +76,7 @@ val schema =
TopLevelObject(DownloadQuery()),
TopLevelObject(ExtensionQuery()),
TopLevelObject(InfoQuery()),
TopLevelObject(KoreaderSyncQuery()),
TopLevelObject(MangaQuery()),
TopLevelObject(MetaQuery()),
TopLevelObject(SettingsQuery()),
@@ -90,6 +93,7 @@ val schema =
TopLevelObject(ExtensionMutation()),
TopLevelObject(ImageMutation()),
TopLevelObject(InfoMutation()),
TopLevelObject(KoreaderSyncMutation()),
TopLevelObject(MangaMutation()),
TopLevelObject(MetaMutation()),
TopLevelObject(SettingsMutation()),

View File

@@ -19,6 +19,11 @@ import suwayomi.tachidesk.manga.model.dataclass.ChapterDataClass
import suwayomi.tachidesk.manga.model.table.ChapterTable
import java.util.concurrent.CompletableFuture
data class SyncConflictInfoType(
val deviceName: String,
val remotePage: Int,
)
class ChapterType(
val id: Int,
val url: String,

View File

@@ -0,0 +1,33 @@
package suwayomi.tachidesk.graphql.types
enum class KoreaderSyncChecksumMethod {
BINARY,
FILENAME,
}
enum class KoreaderSyncStrategy {
PROMPT, // Ask on conflict
SILENT, // Always use latest
SEND, // Send changes only
RECEIVE, // Receive changes only
DISABLED,
}
data class KoSyncStatusPayload(
val isLoggedIn: Boolean,
val username: String?,
)
data class KoSyncConnectPayload(
val clientMutationId: String?,
val success: Boolean,
val message: String?,
val username: String?,
val settings: SettingsType,
)
data class LogoutKoSyncAccountPayload(
val clientMutationId: String?,
val success: Boolean,
val settings: SettingsType,
)

View File

@@ -112,6 +112,15 @@ interface Settings : Node {
val opdsShowOnlyUnreadChapters: Boolean?
val opdsShowOnlyDownloadedChapters: Boolean?
val opdsChapterSortOrder: SortOrder?
// koreader sync
val koreaderSyncServerUrl: String?
val koreaderSyncUsername: String?
val koreaderSyncUserkey: String?
val koreaderSyncDeviceId: String?
val koreaderSyncChecksumMethod: KoreaderSyncChecksumMethod?
val koreaderSyncStrategy: KoreaderSyncStrategy?
val koreaderSyncPercentageTolerance: Double?
}
interface SettingsDownloadConversion {
@@ -206,6 +215,14 @@ data class PartialSettingsType(
override val opdsShowOnlyUnreadChapters: Boolean?,
override val opdsShowOnlyDownloadedChapters: Boolean?,
override val opdsChapterSortOrder: SortOrder?,
// koreader sync
override val koreaderSyncServerUrl: String?,
override val koreaderSyncUsername: String?,
override val koreaderSyncUserkey: String?,
override val koreaderSyncDeviceId: String?,
override val koreaderSyncChecksumMethod: KoreaderSyncChecksumMethod?,
override val koreaderSyncStrategy: KoreaderSyncStrategy?,
override val koreaderSyncPercentageTolerance: Double?,
) : Settings
class SettingsType(
@@ -288,6 +305,14 @@ class SettingsType(
override val opdsShowOnlyUnreadChapters: Boolean,
override val opdsShowOnlyDownloadedChapters: Boolean,
override val opdsChapterSortOrder: SortOrder,
// koreader sync
override val koreaderSyncServerUrl: String,
override val koreaderSyncUsername: String,
override val koreaderSyncUserkey: String,
override val koreaderSyncDeviceId: String,
override val koreaderSyncChecksumMethod: KoreaderSyncChecksumMethod,
override val koreaderSyncStrategy: KoreaderSyncStrategy,
override val koreaderSyncPercentageTolerance: Double,
) : Settings {
constructor(config: ServerConfig = serverConfig) : this(
config.ip.value,
@@ -367,5 +392,13 @@ class SettingsType(
config.opdsShowOnlyUnreadChapters.value,
config.opdsShowOnlyDownloadedChapters.value,
config.opdsChapterSortOrder.value,
// koreader sync
config.koreaderSyncServerUrl.value,
config.koreaderSyncUsername.value,
config.koreaderSyncUserkey.value,
config.koreaderSyncDeviceId.value,
config.koreaderSyncChecksumMethod.value,
config.koreaderSyncStrategy.value,
config.koreaderSyncPercentageTolerance.value,
)
}

View File

@@ -9,7 +9,11 @@ package suwayomi.tachidesk.manga.controller
import io.javalin.http.HandlerType
import io.javalin.http.HttpStatus
import kotlinx.coroutines.GlobalScope
import kotlinx.coroutines.launch
import kotlinx.serialization.json.Json
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.CategoryManga
import suwayomi.tachidesk.manga.impl.Chapter
import suwayomi.tachidesk.manga.impl.ChapterDownloadHelper
@@ -17,9 +21,11 @@ import suwayomi.tachidesk.manga.impl.Library
import suwayomi.tachidesk.manga.impl.Manga
import suwayomi.tachidesk.manga.impl.Page
import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReadyByIndex
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.manga.model.dataclass.CategoryDataClass
import suwayomi.tachidesk.manga.model.dataclass.ChapterDataClass
import suwayomi.tachidesk.manga.model.dataclass.MangaDataClass
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.JavalinSetup.future
import suwayomi.tachidesk.server.util.formParam
import suwayomi.tachidesk.server.util.handler
@@ -315,8 +321,29 @@ object MangaController {
},
behaviorOf = { ctx, mangaId, chapterIndex ->
ctx.future {
future { getChapterDownloadReadyByIndex(chapterIndex, mangaId) }
.thenApply { ctx.json(it) }
future {
var chapter = getChapterDownloadReadyByIndex(chapterIndex, mangaId)
val syncResult = KoreaderSyncService.checkAndPullProgress(chapter.id)
if (syncResult != null) {
if (syncResult.shouldUpdate) {
// Update DB for SILENT and RECEIVE
transaction {
ChapterTable.update({ ChapterTable.id eq chapter.id }) {
it[lastPageRead] = syncResult.pageRead
it[lastReadAt] = syncResult.timestamp
}
}
}
// For PROMPT, SILENT, and RECEIVE, return the remote progress
chapter =
chapter.copy(
lastPageRead = syncResult.pageRead,
lastReadAt = syncResult.timestamp,
)
}
chapter
}.thenApply { ctx.json(it) }
}
},
withResults = {
@@ -341,7 +368,12 @@ object MangaController {
}
},
behaviorOf = { ctx, mangaId, chapterIndex, read, bookmarked, markPrevRead, lastPageRead ->
Chapter.modifyChapter(mangaId, chapterIndex, read, bookmarked, markPrevRead, lastPageRead)
val chapterId = Chapter.modifyChapter(mangaId, chapterIndex, read, bookmarked, markPrevRead, lastPageRead)
// Sync with KoreaderSync when progress is updated
if (lastPageRead != null || read == true) {
GlobalScope.launch { KoreaderSyncService.pushProgress(chapterId) }
}
ctx.status(200)
},
@@ -422,7 +454,11 @@ object MangaController {
ctx.result(it.first)
if (updateProgress == true) {
Chapter.updateChapterProgress(mangaId, chapterIndex, pageNo = index)
val chapterId = Chapter.updateChapterProgress(mangaId, chapterIndex, pageNo = index)
// Sync progress with KoreaderSync if chapter update was successful
if (chapterId != -1) {
GlobalScope.launch { KoreaderSyncService.pushProgress(chapterId) }
}
}
}
}

View File

@@ -431,33 +431,46 @@ object Chapter {
isBookmarked: Boolean?,
markPrevRead: Boolean?,
lastPageRead: Int?,
) {
transaction {
if (listOf(isRead, isBookmarked, lastPageRead).any { it != null }) {
ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder eq chapterIndex) }) { update ->
isRead?.also {
update[ChapterTable.isRead] = it
}
isBookmarked?.also {
update[ChapterTable.isBookmarked] = it
}
lastPageRead?.also {
update[ChapterTable.lastPageRead] = it
update[lastReadAt] = Instant.now().epochSecond
}
}
}
): Int {
val chapterId =
transaction {
val chapter =
ChapterTable
.selectAll()
.where { (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder eq chapterIndex) }
.first()
markPrevRead?.let {
ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder less chapterIndex) }) {
it[ChapterTable.isRead] = markPrevRead
val chapterIdValue = chapter[ChapterTable.id].value
if (listOf(isRead, isBookmarked, lastPageRead).any { it != null }) {
ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder eq chapterIndex) }) { update ->
isRead?.also {
update[ChapterTable.isRead] = it
}
isBookmarked?.also {
update[ChapterTable.isBookmarked] = it
}
lastPageRead?.also {
update[ChapterTable.lastPageRead] = it
update[lastReadAt] = Instant.now().epochSecond
}
}
}
markPrevRead?.let {
ChapterTable.update({ (ChapterTable.manga eq mangaId) and (ChapterTable.sourceOrder less chapterIndex) }) {
it[ChapterTable.isRead] = markPrevRead
}
}
chapterIdValue
}
}
if (isRead == true || markPrevRead == true) {
Track.asyncTrackChapter(setOf(mangaId))
}
return chapterId
}
@Serializable
@@ -733,7 +746,7 @@ object Chapter {
mangaId: Int,
chapterIndex: Int,
pageNo: Int,
) {
): Int {
val chapterData =
transaction {
ChapterTable
@@ -756,5 +769,7 @@ object Chapter {
isBookmarked = null,
markPrevRead = null,
)
return chapterData.id
}
}

View File

@@ -463,6 +463,14 @@ object ProtoBackupExport : ProtoBackupBase() {
opdsShowOnlyUnreadChapters = serverConfig.opdsShowOnlyUnreadChapters.value,
opdsShowOnlyDownloadedChapters = serverConfig.opdsShowOnlyDownloadedChapters.value,
opdsChapterSortOrder = serverConfig.opdsChapterSortOrder.value,
// koreader sync
koreaderSyncServerUrl = serverConfig.koreaderSyncServerUrl.value,
koreaderSyncUsername = serverConfig.koreaderSyncUsername.value,
koreaderSyncUserkey = serverConfig.koreaderSyncUserkey.value,
koreaderSyncDeviceId = serverConfig.koreaderSyncDeviceId.value,
koreaderSyncChecksumMethod = serverConfig.koreaderSyncChecksumMethod.value,
koreaderSyncStrategy = serverConfig.koreaderSyncStrategy.value,
koreaderSyncPercentageTolerance = serverConfig.koreaderSyncPercentageTolerance.value,
)
}
}

View File

@@ -4,6 +4,8 @@ import kotlinx.serialization.Serializable
import kotlinx.serialization.protobuf.ProtoNumber
import org.jetbrains.exposed.sql.SortOrder
import suwayomi.tachidesk.graphql.types.AuthMode
import suwayomi.tachidesk.graphql.types.KoreaderSyncChecksumMethod
import suwayomi.tachidesk.graphql.types.KoreaderSyncStrategy
import suwayomi.tachidesk.graphql.types.Settings
import suwayomi.tachidesk.graphql.types.SettingsDownloadConversion
import suwayomi.tachidesk.graphql.types.WebUIChannel
@@ -84,6 +86,14 @@ data class BackupServerSettings(
@ProtoNumber(53) override var opdsShowOnlyUnreadChapters: Boolean,
@ProtoNumber(54) override var opdsShowOnlyDownloadedChapters: Boolean,
@ProtoNumber(55) override var opdsChapterSortOrder: SortOrder,
// koreader sync
@ProtoNumber(59) override var koreaderSyncServerUrl: String,
@ProtoNumber(60) override var koreaderSyncUsername: String,
@ProtoNumber(61) override var koreaderSyncUserkey: String,
@ProtoNumber(62) override var koreaderSyncDeviceId: String,
@ProtoNumber(63) override var koreaderSyncChecksumMethod: KoreaderSyncChecksumMethod,
@ProtoNumber(64) override var koreaderSyncStrategy: KoreaderSyncStrategy,
@ProtoNumber(65) override var koreaderSyncPercentageTolerance: Double,
) : Settings {
@Serializable
class BackupSettingsDownloadConversionType(

View File

@@ -13,11 +13,14 @@ import libcore.net.MimeUtils
import org.apache.commons.compress.archivers.zip.ZipArchiveEntry
import org.jetbrains.exposed.sql.selectAll
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.Page
import suwayomi.tachidesk.manga.impl.chapter.getChapterDownloadReady
import suwayomi.tachidesk.manga.impl.download.model.DownloadChapter
import suwayomi.tachidesk.manga.impl.util.KoreaderHelper
import suwayomi.tachidesk.manga.impl.util.createComicInfoFile
import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
import suwayomi.tachidesk.manga.impl.util.storage.ImageResponse
import suwayomi.tachidesk.manga.model.table.ChapterTable
@@ -43,20 +46,20 @@ sealed class FileType {
fun getName(): String =
when (this) {
is FileType.RegularFile -> {
is RegularFile -> {
this.file.name
}
is FileType.ZipFile -> {
is ZipFile -> {
this.entry.name
}
}
fun getExtension(): String =
when (this) {
is FileType.RegularFile -> {
is RegularFile -> {
this.file.extension
}
is FileType.ZipFile -> {
is ZipFile -> {
this.entry.name.substringAfterLast(".")
}
}
@@ -189,6 +192,19 @@ abstract class ChaptersFilesProvider<Type : FileType>(
handleSuccessfulDownload()
// Calculate and save Koreader hash for CBZ files
val chapterFile = File(getChapterCbzPath(mangaId, chapterId))
if (chapterFile.exists()) {
val koreaderHash = KoreaderHelper.hashContents(chapterFile)
if (koreaderHash != null) {
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[ChapterTable.koreaderHash] = koreaderHash
}
}
}
}
File(cacheChapterDir).deleteRecursively()
return true

View File

@@ -6,6 +6,8 @@ import org.apache.commons.compress.archivers.zip.ZipArchiveEntry
import org.apache.commons.compress.archivers.zip.ZipArchiveInputStream
import org.apache.commons.compress.archivers.zip.ZipArchiveOutputStream
import org.apache.commons.compress.archivers.zip.ZipFile
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider
import suwayomi.tachidesk.manga.impl.download.fileProvider.FileType
import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
@@ -13,6 +15,7 @@ import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
import suwayomi.tachidesk.manga.impl.util.getMangaDownloadDir
import suwayomi.tachidesk.manga.impl.util.storage.FileDeletionHelper
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.ApplicationDirs
import uy.kohesive.injekt.injectLazy
import java.io.File
@@ -85,6 +88,13 @@ class ArchiveProvider(
}
val cbzDeleted = cbzFile.delete()
if (cbzDeleted) {
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[koreaderHash] = null
}
}
}
FileDeletionHelper.cleanupParentFoldersFor(cbzFile, applicationDirs.mangaDownloadsRoot)
return cbzDeleted
}

View File

@@ -1,10 +1,13 @@
package suwayomi.tachidesk.manga.impl.download.fileProvider.impl
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.manga.impl.download.fileProvider.ChaptersFilesProvider
import suwayomi.tachidesk.manga.impl.download.fileProvider.FileType.RegularFile
import suwayomi.tachidesk.manga.impl.util.getChapterCachePath
import suwayomi.tachidesk.manga.impl.util.getChapterDownloadPath
import suwayomi.tachidesk.manga.impl.util.storage.FileDeletionHelper
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.server.ApplicationDirs
import uy.kohesive.injekt.injectLazy
import java.io.BufferedOutputStream
@@ -61,6 +64,13 @@ class FolderProvider(
}
val chapterDirDeleted = chapterDir.deleteRecursively()
if (chapterDirDeleted) {
transaction {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[koreaderHash] = null
}
}
}
FileDeletionHelper.cleanupParentFoldersFor(chapterDir, applicationDirs.mangaDownloadsRoot)
return chapterDirDeleted
}

View File

@@ -0,0 +1,425 @@
package suwayomi.tachidesk.manga.impl.sync
import eu.kanade.tachiyomi.network.NetworkHelper
import eu.kanade.tachiyomi.network.await
import eu.kanade.tachiyomi.util.lang.Hash
import io.github.oshai.kotlinlogging.KotlinLogging
import io.javalin.json.JsonMapper
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.Request
import okhttp3.RequestBody.Companion.toRequestBody
import org.jetbrains.exposed.sql.transactions.transaction
import org.jetbrains.exposed.sql.update
import suwayomi.tachidesk.graphql.types.KoSyncStatusPayload
import suwayomi.tachidesk.graphql.types.KoreaderSyncChecksumMethod
import suwayomi.tachidesk.graphql.types.KoreaderSyncStrategy
import suwayomi.tachidesk.manga.impl.util.KoreaderHelper
import suwayomi.tachidesk.manga.impl.util.getChapterCbzPath
import suwayomi.tachidesk.manga.model.table.ChapterTable
import suwayomi.tachidesk.manga.model.table.MangaTable
import suwayomi.tachidesk.server.serverConfig
import uy.kohesive.injekt.injectLazy
import java.io.File
import java.util.UUID
import kotlin.math.abs
object KoreaderSyncService {
private val logger = KotlinLogging.logger {}
private val network: NetworkHelper by injectLazy()
private val json: Json by injectLazy()
private val jsonMapper: JsonMapper by injectLazy()
@Serializable
private data class KoreaderProgressPayload(
val document: String,
val progress: String,
val percentage: Float,
val device: String,
val device_id: String,
)
@Serializable
private data class KoreaderProgressResponse(
val document: String? = null,
val progress: String? = null,
val percentage: Float? = null,
val timestamp: Long? = null,
val device: String? = null,
val device_id: String? = null,
)
@Serializable
data class SyncResult(
val pageRead: Int,
val timestamp: Long, // Unix timestamp in seconds
val device: String,
val shouldUpdate: Boolean = false,
val isConflict: Boolean = false,
)
data class ConnectResult(
val success: Boolean,
val message: String? = null,
val username: String? = null,
)
private data class AuthResult(
val success: Boolean,
val message: String? = null,
val isUserNotFoundError: Boolean = false,
)
private fun buildRequest(
url: String,
block: Request.Builder.() -> Unit,
): Request =
Request
.Builder()
.url(url)
.addHeader("Accept", "application/vnd.koreader.v1+json")
.apply(block)
.build()
private suspend fun getOrGenerateDeviceId(): String {
var deviceId = serverConfig.koreaderSyncDeviceId.value
if (deviceId.isBlank()) {
deviceId =
UUID
.randomUUID()
.toString()
.replace("-", "")
.uppercase()
logger.info { "[KOSYNC] Generated new KOSync Device ID: $deviceId" }
serverConfig.koreaderSyncDeviceId.value = deviceId
}
return deviceId
}
private fun getOrGenerateChapterHash(chapterId: Int): String? {
return transaction {
val existingHash =
ChapterTable
.select(ChapterTable.koreaderHash)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.get(ChapterTable.koreaderHash)
if (!existingHash.isNullOrBlank()) {
return@transaction existingHash
}
val checksumMethod = serverConfig.koreaderSyncChecksumMethod.value
val newHash =
when (checksumMethod) {
KoreaderSyncChecksumMethod.BINARY -> {
logger.info { "[KOSYNC HASH] No hash for chapterId=$chapterId. Generating from CBZ content." }
val mangaId =
ChapterTable
.select(ChapterTable.manga)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.get(ChapterTable.manga)
?.value ?: return@transaction null
val cbzFile = File(getChapterCbzPath(mangaId, chapterId))
if (!cbzFile.exists()) {
logger.info { "[KOSYNC HASH] Could not generate hash for chapterId=$chapterId. CBZ not found." }
return@transaction null
}
KoreaderHelper.hashContents(cbzFile)
}
KoreaderSyncChecksumMethod.FILENAME -> {
logger.info { "[KOSYNC HASH] No hash for chapterId=$chapterId. Generating from filename." }
(ChapterTable innerJoin MangaTable)
.select(ChapterTable.name, MangaTable.title)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.let {
val chapterName = it[ChapterTable.name]
val mangaTitle = it[MangaTable.title]
val baseFilename = "$mangaTitle - $chapterName".split('.').dropLast(1).joinToString(".")
Hash.md5(baseFilename)
}
}
}
if (newHash != null) {
ChapterTable.update({ ChapterTable.id eq chapterId }) {
it[koreaderHash] = newHash
}
logger.info { "[KOSYNC HASH] Generated and saved new hash for chapterId=$chapterId" }
} else {
logger.warn { "[KOSYNC HASH] Hashing failed for chapterId=$chapterId." }
}
newHash
}
}
private suspend fun register(
username: String,
userkey: String,
): AuthResult {
val payload =
buildJsonObject {
put("username", username)
put("password", userkey)
}
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/users/create") {
post(payload.toString().toRequestBody("application/json".toMediaType()))
}
return try {
network.client.newCall(request).await().use { response ->
if (response.isSuccessful) {
AuthResult(true, "Registration successful.")
} else {
val errorBody = response.body.string()
val errorMessage =
runCatching {
jsonMapper.fromJsonString<Map<String, String>>(
errorBody,
Map::class.java,
)["message"]
}.getOrNull()
val finalMessage = errorMessage ?: "Registration failed with code ${response.code}"
AuthResult(false, finalMessage)
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC REGISTER] Exception" }
AuthResult(false, e.message)
}
}
private suspend fun authorize(
username: String,
userkey: String,
): AuthResult {
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/users/auth") {
get()
addHeader("x-auth-user", username)
addHeader("x-auth-key", userkey)
}
return try {
network.client.newCall(request).await().use { response ->
if (response.isSuccessful) {
AuthResult(true)
} else {
val isUserNotFound = response.code == 401 // Unauthorized often means user/pass combo is wrong
AuthResult(false, "Authorization failed with code ${response.code}", isUserNotFoundError = isUserNotFound)
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC AUTHORIZE] Exception" }
AuthResult(false, e.message)
}
}
suspend fun connect(
username: String,
password: String,
): ConnectResult {
val userkey = Hash.md5(password)
val authResult = authorize(username, userkey)
if (authResult.success) {
serverConfig.koreaderSyncUsername.value = username
serverConfig.koreaderSyncUserkey.value = userkey
return ConnectResult(true, "Login successful.", username)
}
if (authResult.isUserNotFoundError) {
logger.info { "[KOSYNC CONNECT] Authorization failed, attempting to register new user." }
val registerResult = register(username, userkey)
return if (registerResult.success) {
serverConfig.koreaderSyncUsername.value = username
serverConfig.koreaderSyncUserkey.value = userkey
ConnectResult(true, "Registration successful.", username)
} else {
ConnectResult(false, registerResult.message ?: "Registration failed.", null)
}
}
return ConnectResult(false, authResult.message ?: "Authentication failed.", null)
}
suspend fun logout() {
serverConfig.koreaderSyncUsername.value = ""
serverConfig.koreaderSyncUserkey.value = ""
}
suspend fun getStatus(): KoSyncStatusPayload {
val username = serverConfig.koreaderSyncUsername.value
val userkey = serverConfig.koreaderSyncUserkey.value
if (username.isBlank() || userkey.isBlank()) {
return KoSyncStatusPayload(isLoggedIn = false, username = null)
}
val authResult = authorize(username, userkey)
return KoSyncStatusPayload(isLoggedIn = authResult.success, username = if (authResult.success) username else null)
}
suspend fun pushProgress(chapterId: Int) {
val strategy = serverConfig.koreaderSyncStrategy.value
if (strategy == KoreaderSyncStrategy.DISABLED || strategy == KoreaderSyncStrategy.RECEIVE) return
val username = serverConfig.koreaderSyncUsername.value
val userkey = serverConfig.koreaderSyncUserkey.value
if (username.isBlank() || userkey.isBlank()) return
logger.info { "[KOSYNC PUSH] Init." }
val chapterHash = getOrGenerateChapterHash(chapterId)
if (chapterHash.isNullOrBlank()) {
logger.info { "[KOSYNC PUSH] Aborted for chapterId=$chapterId: No hash." }
return
}
val chapterInfo =
transaction {
ChapterTable
.select(ChapterTable.lastPageRead, ChapterTable.pageCount)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.let {
object {
val lastPageRead = it[ChapterTable.lastPageRead]
val pageCount = it[ChapterTable.pageCount]
}
}
} ?: return
if (chapterInfo.pageCount <= 0) {
logger.warn { "[KOSYNC PUSH] Aborted for chapterId=$chapterId: Invalid pageCount." }
return
}
try {
val deviceId = getOrGenerateDeviceId()
val payload =
KoreaderProgressPayload(
document = chapterHash,
progress = (chapterInfo.lastPageRead + 1).toString(),
percentage = (chapterInfo.lastPageRead + 1).toFloat() / chapterInfo.pageCount.toFloat(),
device = "Suwayomi-Server (${System.getProperty("os.name")})",
device_id = deviceId,
)
val requestBody = json.encodeToString(KoreaderProgressPayload.serializer(), payload)
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/syncs/progress") {
put(requestBody.toRequestBody("application/json".toMediaType()))
addHeader("x-auth-user", username)
addHeader("x-auth-key", userkey)
}
network.client.newCall(request).await().use { response ->
if (!response.isSuccessful) {
logger.warn { "[KOSYNC PUSH] Failed for chapterId=$chapterId: ${response.code}" }
} else {
logger.info { "[KOSYNC PUSH] Success for chapterId=$chapterId" }
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC PUSH] Exception for chapterId=$chapterId" }
}
}
suspend fun checkAndPullProgress(chapterId: Int): SyncResult? {
val strategy = serverConfig.koreaderSyncStrategy.value
if (strategy == KoreaderSyncStrategy.DISABLED || strategy == KoreaderSyncStrategy.SEND) return null
val username = serverConfig.koreaderSyncUsername.value
val userkey = serverConfig.koreaderSyncUserkey.value
if (username.isBlank() || userkey.isBlank()) return null
val chapterHash = getOrGenerateChapterHash(chapterId)
if (chapterHash.isNullOrBlank()) {
logger.info { "[KOSYNC PULL] Aborted for chapterId=$chapterId: No hash." }
return null
}
try {
val request =
buildRequest("${serverConfig.koreaderSyncServerUrl.value.removeSuffix("/")}/syncs/progress/$chapterHash") {
get()
addHeader("x-auth-user", username)
addHeader("x-auth-key", userkey)
}
network.client.newCall(request).await().use { response ->
if (response.isSuccessful) {
val body = response.body.string()
if (body.isBlank() || body == "{}") return null
val progressResponse = json.decodeFromString(KoreaderProgressResponse.serializer(), body)
val pageRead = progressResponse.progress?.toIntOrNull()?.minus(1)
val timestamp = progressResponse.timestamp
val device = progressResponse.device ?: "KOReader"
val localProgress =
transaction {
ChapterTable
.select(ChapterTable.lastReadAt, ChapterTable.lastPageRead, ChapterTable.pageCount)
.where { ChapterTable.id eq chapterId }
.firstOrNull()
?.let {
object {
val lastReadAt = it[ChapterTable.lastReadAt]
val lastPageRead = it[ChapterTable.lastPageRead]
val pageCount = it[ChapterTable.pageCount]
}
}
}
if (pageRead != null && timestamp != null) {
// Ignore XPath progress for now as we only support paginated files
if (progressResponse.progress?.startsWith("/") == true) {
return null
}
val localPercentage =
if (localProgress?.pageCount ?: 0 >
0
) {
(localProgress!!.lastPageRead + 1).toFloat() / localProgress.pageCount
} else {
0f
}
val percentageDifference = abs(localPercentage - (progressResponse.percentage ?: 0f))
// Progress is within tolerance, no sync needed
if (percentageDifference < serverConfig.koreaderSyncPercentageTolerance.value) {
return null
}
when (strategy) {
KoreaderSyncStrategy.RECEIVE -> {
return SyncResult(pageRead, timestamp, device, shouldUpdate = true)
}
KoreaderSyncStrategy.SILENT -> {
if (timestamp > (localProgress?.lastReadAt ?: 0L)) {
return SyncResult(pageRead, timestamp, device, shouldUpdate = true)
}
}
KoreaderSyncStrategy.PROMPT -> {
return SyncResult(pageRead, timestamp, device, isConflict = true)
}
else -> {} // SEND and DISABLED already handled at the start of the function
}
}
} else {
logger.warn { "[KOSYNC PULL] Failed for chapterId=$chapterId: ${response.code}" }
}
}
} catch (e: Exception) {
logger.error(e) { "[KOSYNC PULL] Exception for chapterId=$chapterId" }
}
return null
}
}

View File

@@ -0,0 +1,48 @@
package suwayomi.tachidesk.manga.impl.util
import java.io.File
import java.security.MessageDigest
object KoreaderHelper {
// Helper function to convert ByteArray to Hex String
private fun ByteArray.toHexString() = joinToString("") { "%02x".format(it) }
/**
* Hashes the document according to a custom Koreader hashing algorithm.
* https://github.com/koreader/koreader/blob/master/frontend/util.lua#L1040
* Only applies to epub and cbz files.
* @param file The file object to hash.
* @return The lowercase MD5 hash or null if hashing is not possible.
*/
fun hashContents(file: File): String? {
val extension = file.extension.lowercase()
if (!file.exists() || (extension != "epub" && extension != "cbz")) {
return null
}
try {
file.inputStream().use { fs ->
val step = 1024
val size = 1024
val md5 = MessageDigest.getInstance("MD5")
val buffer = ByteArray(size)
for (i in -1 until 10) {
val position = (step shl (2 * i)).toLong()
if (position >= file.length()) break // Avoid seeking past the end of small files
fs.channel.position(position)
val bytesRead = fs.read(buffer, 0, size)
if (bytesRead > 0) {
md5.update(buffer, 0, bytesRead)
} else {
break
}
}
return md5.digest().toHexString().lowercase()
}
} catch (e: Exception) {
// TODO: Should we log this error?
return null
}
}
}

View File

@@ -39,6 +39,8 @@ object ChapterTable : IntIdTable() {
val pageCount = integer("page_count").default(-1)
val manga = reference("manga", MangaTable, ReferenceOption.CASCADE)
val koreaderHash = varchar("koreader_hash", 32).nullable()
}
fun ChapterTable.toDataClass(

View File

@@ -6,6 +6,7 @@ import kotlinx.coroutines.withContext
import suwayomi.tachidesk.i18n.MR
import suwayomi.tachidesk.manga.impl.ChapterDownloadHelper
import suwayomi.tachidesk.manga.impl.MangaList.proxyThumbnailUrl
import suwayomi.tachidesk.manga.impl.sync.KoreaderSyncService
import suwayomi.tachidesk.manga.model.table.MangaStatus
import suwayomi.tachidesk.opds.constants.OpdsConstants
import suwayomi.tachidesk.opds.dto.OpdsChapterListAcqEntry
@@ -185,7 +186,6 @@ object OpdsEntryBuilder {
append(MR.strings.opds_chapter_details_progress.localized(locale, chapter.lastPageRead, chapter.pageCount))
}
}
return OpdsEntryXml(
id = "urn:suwayomi:chapter:${chapter.id}",
title = entryTitle,
@@ -222,11 +222,23 @@ object OpdsEntryBuilder {
baseUrl: String,
locale: Locale,
): OpdsEntryXml {
// Check remote progress before building the entry
val syncResult = KoreaderSyncService.checkAndPullProgress(chapter.id)
val localLastPageRead = chapter.lastPageRead
val remoteLastPageRead = syncResult?.pageRead
val remoteLastReadAt = syncResult?.timestamp
val showConflict = syncResult?.isConflict == true && remoteLastPageRead != null && localLastPageRead != remoteLastPageRead
val finalLastPageRead = if (syncResult?.shouldUpdate == true) remoteLastPageRead ?: localLastPageRead else localLastPageRead
val finalLastReadAt = if (syncResult?.shouldUpdate == true) remoteLastReadAt ?: chapter.lastReadAt else chapter.lastReadAt
val statusKey =
when {
chapter.downloaded -> MR.strings.opds_chapter_status_downloaded
chapter.read -> MR.strings.opds_chapter_status_read
chapter.lastPageRead > 0 -> MR.strings.opds_chapter_status_in_progress
finalLastPageRead > 0 -> MR.strings.opds_chapter_status_in_progress
else -> MR.strings.opds_chapter_status_unread
}
val titlePrefix = statusKey.localized(locale)
@@ -238,7 +250,7 @@ object OpdsEntryBuilder {
append(MR.strings.opds_chapter_details_scanlator.localized(locale, it))
}
val pageCountDisplay = chapter.pageCount.takeIf { it > 0 } ?: "?"
append(MR.strings.opds_chapter_details_progress.localized(locale, chapter.lastPageRead, pageCountDisplay))
append(MR.strings.opds_chapter_details_progress.localized(locale, finalLastPageRead, pageCountDisplay))
}
val links = mutableListOf<OpdsLinkXml>()
var cbzFileSize: Long? = null
@@ -270,19 +282,73 @@ object OpdsEntryBuilder {
}
}
if (chapter.pageCount > 0) {
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_PSE_STREAM,
href =
"/api/v1/manga/${manga.id}/chapter/${chapter.sourceOrder}/page/{pageNumber}" +
"?updateProgress=${serverConfig.opdsEnablePageReadProgress.value}",
type = OpdsConstants.TYPE_IMAGE_JPEG,
title = MR.strings.opds_linktitle_stream_pages.localized(locale),
pseCount = chapter.pageCount,
pseLastRead = chapter.lastPageRead.takeIf { it > 0 },
pseLastReadDate = chapter.lastReadAt.takeIf { it > 0 }?.let { OpdsDateUtil.formatEpochMillisForOpds(it * 1000) },
),
)
val basePageHref =
"/api/v1/manga/${manga.id}/chapter/${chapter.sourceOrder}/page/{pageNumber}" +
"?updateProgress=${serverConfig.opdsEnablePageReadProgress.value}"
if (showConflict) {
// Option 1: Local progress
val localTitleRes =
if (localLastPageRead >
0
) {
MR.strings.opds_linktitle_stream_pages_continue_local
} else {
MR.strings.opds_linktitle_stream_pages_start_local
}
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_PSE_STREAM,
href = basePageHref,
type = OpdsConstants.TYPE_IMAGE_JPEG,
title = localTitleRes.localized(locale),
pseCount = chapter.pageCount,
pseLastRead = localLastPageRead.takeIf { it >= 0 },
pseLastReadDate = chapter.lastReadAt.takeIf { it > 0 }?.let { OpdsDateUtil.formatEpochMillisForOpds(it * 1000) },
),
)
// Option 2: Remote progress
val remoteTitleRes =
if (remoteLastPageRead >
0
) {
MR.strings.opds_linktitle_stream_pages_continue_remote
} else {
MR.strings.opds_linktitle_stream_pages_start_remote
}
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_PSE_STREAM,
href = basePageHref,
type = OpdsConstants.TYPE_IMAGE_JPEG,
title = remoteTitleRes.localized(locale, syncResult.device),
pseCount = chapter.pageCount,
pseLastRead = remoteLastPageRead,
pseLastReadDate = remoteLastReadAt?.takeIf { it > 0 }?.let { OpdsDateUtil.formatEpochMillisForOpds(it * 1000) },
),
)
} else {
// Normal behavior: single progress link
val titleRes =
if (finalLastPageRead >
0
) {
MR.strings.opds_linktitle_stream_pages_continue
} else {
MR.strings.opds_linktitle_stream_pages_start
}
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_PSE_STREAM,
href = basePageHref,
type = OpdsConstants.TYPE_IMAGE_JPEG,
title = titleRes.localized(locale),
pseCount = chapter.pageCount,
pseLastRead = finalLastPageRead.takeIf { it > 0 },
pseLastReadDate = finalLastReadAt.takeIf { it > 0 }?.let { OpdsDateUtil.formatEpochMillisForOpds(it * 1000) },
),
)
}
links.add(
OpdsLinkXml(
rel = OpdsConstants.LINK_REL_IMAGE,

View File

@@ -26,6 +26,8 @@ import kotlinx.coroutines.flow.mapLatest
import kotlinx.coroutines.flow.onEach
import org.jetbrains.exposed.sql.SortOrder
import suwayomi.tachidesk.graphql.types.AuthMode
import suwayomi.tachidesk.graphql.types.KoreaderSyncChecksumMethod
import suwayomi.tachidesk.graphql.types.KoreaderSyncStrategy
import suwayomi.tachidesk.graphql.types.WebUIChannel
import suwayomi.tachidesk.graphql.types.WebUIFlavor
import suwayomi.tachidesk.graphql.types.WebUIInterface
@@ -195,6 +197,15 @@ class ServerConfig(
val opdsShowOnlyDownloadedChapters: MutableStateFlow<Boolean> by OverrideConfigValue()
val opdsChapterSortOrder: MutableStateFlow<SortOrder> by OverrideConfigValue()
// koreader sync
val koreaderSyncServerUrl: MutableStateFlow<String> by OverrideConfigValue()
val koreaderSyncUsername: MutableStateFlow<String> by OverrideConfigValue()
val koreaderSyncUserkey: MutableStateFlow<String> by OverrideConfigValue()
val koreaderSyncDeviceId: MutableStateFlow<String> by OverrideConfigValue()
val koreaderSyncChecksumMethod: MutableStateFlow<KoreaderSyncChecksumMethod> by OverrideConfigValue()
val koreaderSyncStrategy: MutableStateFlow<KoreaderSyncStrategy> by OverrideConfigValue()
val koreaderSyncPercentageTolerance: MutableStateFlow<Double> by OverrideConfigValue()
@OptIn(ExperimentalCoroutinesApi::class)
fun <T> subscribeTo(
flow: Flow<T>,

View File

@@ -0,0 +1,19 @@
package suwayomi.tachidesk.server.database.migration
/*
* Copyright (C) Contributors to the Suwayomi project
*
* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at https://mozilla.org/MPL/2.0/. */
import de.neonew.exposed.migrations.helpers.AddColumnMigration
@Suppress("ClassName", "unused")
class M0051_AddKoreaderHashToChapterTable :
AddColumnMigration(
"Chapter",
"koreader_hash",
"VARCHAR(32)",
"NULL",
)

View File

@@ -85,3 +85,12 @@ server.opdsMarkAsReadOnDownload = false
server.opdsShowOnlyUnreadChapters = false
server.opdsShowOnlyDownloadedChapters = false
server.opdsChapterSortOrder = "DESC" # "ASC", "DESC"
# Koreader Sync
server.koreaderSyncServerUrl = "http://localhost:17200"
server.koreaderSyncUsername = ""
server.koreaderSyncUserkey = ""
server.koreaderSyncDeviceId = ""
server.koreaderSyncChecksumMethod = "binary" # "binary" or "filename"
server.koreaderSyncStrategy = "disabled" # "prompt", "silent", "send", "receive", "disabled"
server.koreaderSyncPercentageTolerance = 0.00000000000001 # absolute tolerance for progress comparison from 1 (widest) to 1e-15 (strict)