Commit Graph

751 Commits

Author SHA1 Message Date
schroda
3e9d29ea7f Remove username and password from config log (#756) 2023-11-04 18:09:24 -04:00
schroda
4324373e61 Fix/chapter list fetch updating and inserting chapters into database (#749)
* Keep initial fetch date of existing chapters on a list fetch

The fetch at date should not get updated for existing chapters.
Updating this field makes it impossible to detect which chapters were actually newly fetched.
To get the last fetched timestamp of the chapters, the "chaptersLastFetchedAt" field of the manga should be used

got changed in 6d33d72663

* Get real chapter url safely

In case this causes an exception, it should not cause the whole list fetch to fail

was removed in 6d33d72663
2023-11-01 20:24:34 -04:00
schroda
673053d291 Migrate preferences only if necessary (#748)
Currently, the server tries to migrate the preference on every startup, even if the migration was already done.
This can lead to an unhandled exception, if the write permission to the system preference was revoked.
In case the migration has already happened, these permissions should not be required
2023-11-01 09:19:57 -04:00
schroda
5b3975f886 Only batch update in case list is not empty (#747)
Apparently "BatchUpdateStatement" can't handle an empty data set
2023-11-01 09:19:40 -04:00
schroda
7ed8f43859 Fix/backup import failure not resetting status (#746)
* Reset backup status to idle in case of an exception

* Rename "performRestore" function

* Set backup status to failure on exception

Makes it possible to detect if the restore failed or not after the first status was received

* Set backup status to success on completion

Since the status is not provided over a subscription, but over a query that should be pulled, it is not really easily detectable if a restore finished or not, since both states will be indicated by "idle"

* Correctly wait for first new status when triggering backup import

The status is only "Idle" in case no backup import has ever run.
Once the first backup process finished it is either "Failure" or "Success"

* Rename "ProtoBackupImport::restore" function

* Add id to restore process

Makes it possible to differentiate between backup restore processes.
2023-10-31 21:21:11 -04:00
schroda
dcbb1c0dd1 Handle backups with categories having default category name (#745)
We do not allow any category to have the default categories name ("default" case-insensitive).
In case the backup includes a category with this name, it won't be matched to any category in our database and also won't insert a new category.
This will then lead to an exception, causing the backup to fail.
2023-10-30 19:47:33 -04:00
schroda
5d4d417f3e Extract downloaded webUI zip in temp folder for validation (#744)
Could cause issues due to not having permission to create the folder to extract the zip into
2023-10-30 19:47:23 -04:00
schroda
5943c6a2c6 Feature/improve browsing source performance (#743)
* Improve browse source database operations

* Reuse "insertOrGet" for "processEntries"
2023-10-30 19:47:13 -04:00
Alexandre Journet
6d33d72663 #733: Improve perfs on getChapterList with onlineFetch (Less databases calls) (#737)
* improve(#733): less databases calls on getChapterList with onlineFetch

* improve(#733): fixes (delete with ids), tried batch update but not successfull

* improve(#733): fixes (batch update)

* improve(#733): clean imports

* improve(#733): fixes SChapter to ChapterDataClass,

* improve(#733): re-added recognize chap number

---------

Co-authored-by: Alexandre JOURNET <alexandre.journet@axopen.com>
2023-10-30 19:47:03 -04:00
schroda
9d2b098837 Fix/updater update stuck in running status after failure (#731)
* Move running check to update function

* Move updating update status to process function

* Fail all source updates in case of update channel failure

In case the channel failed due to an exception, the update for the source failed completely.
This however was never handled and the pending updates for the source were never set to failed.
Due to this, the global updates running state was always true

* Remove completed update channel from available channels

* Always log specific update job failure
2023-10-30 19:46:54 -04:00
schroda
17bc2d2331 Fetch mangas during the update (#729)
* Optionally fetch mangas during the update

The update only fetched the chapter list of a manga but never the manga itself.
Thus, e.g. unless the manga got online fetched via the ui, it would never get recognized if it is completed or not.
This would e.g. prevent the update setting, to not update completed mangas, from working as intended

* Make settings required
2023-10-30 19:46:43 -04:00
Mitchell Syer
1c192b8db6 Fix Updater (#742) 2023-10-29 12:17:20 -04:00
schroda
76595233fc Prevent mangas from being added to the default category (#741)
Mangas are not supposed to be mapped to the default category in the database.
In case this happens, the category query won't be able to correctly select mangas in the default category
2023-10-29 11:02:31 -04:00
schroda
6531b80998 Delete outdated thumbnails when inserting mangas into database (#739)
In case the database got deleted without deleting cached/downloaded thumbnails, the next time a manga gets inserted, it's possible that a thumbnail was already downloaded for its id.
This then causes mangas to be displayed with incorrect thumbnails due to using the outdated cached/downloaded thumbnails
2023-10-29 11:02:23 -04:00
schroda
05707e29d7 Add missing settings to gql (#738)
* Add missing settings to gql

* Cleanup updating settings
2023-10-29 11:02:13 -04:00
schroda
616ed4637d Handle disabled download ahead limit for new chapters auto download (#734)
In case download ahead is disabled, all new chapters should get downloaded.
Due to incorrectly calculating the index of the first new chapter to download, no new chapter was downloaded at all
2023-10-29 11:02:02 -04:00
schroda
912c340a01 Fix update subscription returning stale data (#727)
In case a manga was already loaded via the data loader, the cached data will get used.
Due to this, the update status did not return the updated manga data, but instead, stale data
2023-10-29 11:01:55 -04:00
Mitchell Syer
583a2f0fad Migrate to XML Settings from Preferences (#722)
* Migrate to XML Settings from Preferences

* Lint
2023-10-29 11:01:46 -04:00
schroda
60015bc041 Return source for preference mutation (#728) 2023-10-26 18:15:29 -04:00
Mitchell Syer
029f445d0a Revert Dex2Jar (#721)
* Revert attempts to fix Dex2Jar

* Revert to v64 of Dex2Jar
2023-10-16 20:13:06 -04:00
schroda
150416b578 Fix/default log level (#719)
* Set default log level to INFO

Default log level was accidentally changed to ERROR for the base logger in 56deea9fb3

* Reduce graphql log level to WARN

Otherwise, thrown exceptions are swallowed by graphql and never logged besides a very brief error in the graphql response
2023-10-16 09:02:56 -04:00
schroda
6684576de1 Fix more missing functions (#718)
2cf9a407e8
2023-10-16 09:02:01 -04:00
Mitchell Syer
289acc9296 Fix Kavita (#716) 2023-10-15 21:58:38 -04:00
Mitchell Syer
2cf9a407e8 Fix MangaDex and Other Sources (#715) 2023-10-15 21:50:30 -04:00
Mitchell Syer
682c364647 Address Build Warnings and Cleanup (#707)
* Address build warnings and cleanup

* Actual name of who defined the protocol

* Remove uneeded detekt supression

* GraphQL Before-After cleanup

* Lint

* Cleanup unused functions

* Fix some discrepancies with the 1.5 source api and fix lang exception
2023-10-15 20:16:30 -04:00
schroda
e70730e9a8 Query for mangas in specific categories (#712) 2023-10-15 20:14:59 -04:00
schroda
0ba6c88d69 Fix/graphql mangas query genre based filtering (#713)
* Filter mangas based on each genre of the genre condition

Genres are stored as a comma separated string in the database.
Thus, unless the correct string was passed in the condition, no manga would match the condition.

* Query mangas based on genre filter
2023-10-15 20:14:20 -04:00
schroda
c56bdea1e2 Do not log ping messages (#709) 2023-10-15 20:14:10 -04:00
schroda
a449a01a24 Fix/web interface manager get latest compatible version (#706)
* Correctly check for none PREVIEW channel latest compatible version

The only working channel was the PREVIEW channel, since any other channel would have fetched the actual version of the preview and used this as the potential latest compatible version.
This was caused due to incorrectly checking if the preview version should be ignored.

* Remove PREVIEW version constant

* Consider versions of different channels

In case the current server version isn't compatible with the latest version of the selected webUI channel, versions of other channel should be considered depending on the selected channel.

E.g. PREVIEW is the latest available version and thus, any version of another channel is also compatible with the PREVIEW channel

* Restrict min compatible version to the bundled version

The oldest compatible version for a server is the bundled version, thus, any version that is older than the bundled one should not be considered compatible
2023-10-07 20:20:09 -04:00
Mitchell Syer
849acfca3d Switch to a new Ktlint Formatter (#705)
* Switch to new Ktlint plugin

* Add ktlintCheck to PR builds

* Run formatter

* Put ktlint version in libs toml

* Fix lint

* Use Zip4Java from libs.toml
2023-10-06 23:38:39 -04:00
schroda
3cd3cb0186 Fix/graphql subscriptions logging (#704)
* Only log operationMessage in case gql logging is enabled

* Always log message type and operation name
2023-10-04 22:02:46 -04:00
Mitchell Syer
a9987e6ab0 Support more image types (#700) 2023-10-04 22:02:20 -04:00
schroda
ef0a6f54b8 Feature/auto download ahead (#681)
* Add "download ahead" mutation

Checks if the specified number of unread chapters, that should be downloaded, are available.
In case not enough chapters are downloaded, the number of missing unread chapters will get downloaded

* Optionally pass the latest read chapter id of a manga

In case a chapter will get marked as read, which also triggered the download ahead call, it's possible, that by the time the download ahead logic gets triggered, the chapter hasn't been marked as read yet.
This could then cause this chapter to be included in the chapters to get downloaded.
By providing the chapter id, this chapter will be used as the latest read chapter instead, and thus, not be included inn the chapters to download.
2023-10-04 22:02:10 -04:00
Mitchell Syer
c8865ad185 Implement Non-Final 1.5 Extensions API (#699)
* Implement non-final 1.5 extensions API

* Bump lib version max

* Add visibility to preferences

* Add preference visibility
2023-10-04 22:01:45 -04:00
schroda
354968fba7 Update version "name" and "code" when installing external extension (#698)
In case a newer version of the extension is installed  and the extension gets manually downgraded, the version in db is still the one of the newer version.
This will prevent detection of available updates, since it won't get recognized, that an older version is currently installed.
2023-10-02 17:46:46 -04:00
schroda
f985ed2131 Order chapters to download by manga and source order (#697)
Chapters were added to the queue by database index order.
In case a chapters of different mangas got added to the queue, downloads got mingled instead of being group inserted per manga.
Also sort manga chapters by source order, to make sure, that, in case chapters of a manga are, for some reason, not in the correct order in the database, they will still get downloaded in the order of the source.
2023-10-02 17:46:38 -04:00
schroda
be2628875f Correctly select results using cursors while sorting (#696)
When using cursors for pagination while sorting, the sort order was inverted (desc -> asc, asc -> desc).
However, this was then not considered when selecting results based on the cursor.
For before/after results where always selected via greater/less.
Due to inverting the sort order, this also needs to be inverted depending on the sort order (desc or asc).
2023-10-02 17:46:26 -04:00
Mitchell Syer
ea2cf5d4ff Fix File Upload (#694)
* Fix File Upload

* Use operations instead of operation

* Fix tests
2023-09-30 21:55:42 -04:00
MangaCrushTeam
41fea1d2a0 remove @Synchronized in CloudflareInterceptor.kt for performance (#688) 2023-09-23 16:26:07 -04:00
schroda
d81fafc9f6 Correctly detect initial fetch of chapters (#689)
Since the number of chapters gets converted to be index based, 1 available chapter would result in 0.

Due to this, in case a manga had exactly one chapter before updating the chapters, it was incorrectly detected as the initial fetch and the new chapters did not get automatically downloaded.
2023-09-23 16:25:59 -04:00
schroda
c9423ef425 Send every download status change to the subscriber (#684)
Flow::stateIn has "Strong equality-based conflation" (see documentation).
Thus, it omits every value in case it's equal to the previous one.
Since the DownloadManger::getStatus function returns a status with a queue, that contains all current "DownloadChapters" by reference, the equality check was always true.
Thus, progress changes of downloads were never sent to subscribers.
Subscriber were only notified about finished downloads (size of queue changed) or downloader status changes
2023-09-16 13:07:43 -04:00
schroda
7086055ec3 Handle finished downloads that weren't removed from the queue (#683)
In case a download was finished, but the downloader got stopped before it was able to remove the finished download from the queue, the downloader got stuck in an endless loop of starting and pausing downloads.

This was caused by selecting the next chapter to download and then recognizing in "Downloader::step", that there is another chapter to download before the current one in the queue.
However, since this recognized chapter is already downloaded, the downloader selected the next queued chapter again.
It was then stuck in this loop until the finished chapter was manually removed from the queue.
2023-09-16 13:07:35 -04:00
schroda
553b35d218 Feature/improve automatic chapter downloads (#680)
* Rename "newChapters" to "updatedChapterList"

* Do not auto download new chapters of entries with unread chapters

Makes it possible to prevent unnecessary chapter downloads in case the entry hasn't yet been caught up

* Optionally limit auto new chapter downloads

* Prevent downloading new chapters for mangas not in the library
2023-09-16 13:07:24 -04:00
schroda
35be9f14e4 Return correct latest compatible webUI version (#677)
The function always returned the PREVIEW version as the latest compatible version.
This was caused by incorrectly selecting the version from the json object, which resulted in the version to be wrapped in '"'.
2023-09-03 18:11:06 -04:00
schroda
abcbec9c2a Fix/downloader not creating folder or cbz file (#676)
* Create manga download dir in case it's missing for cbz downloads

The directory, in which the cbz file should have been saved in, was never created.

* Correctly copy chapter download to final download location

"renameTo" does not include the content of a directory.
Thus, it just created an empty chapter folder int the final download directory
2023-09-03 18:10:54 -04:00
schroda
ff6f5d7e89 Add more fields to the manga graphql type (#675)
These are information that are necessary for nearly all manga requests.
They could be selected via the categories mutation, but this only works for a single manga.
It is not possible to select this information for lists of mangas without having to request all chapters for every manga in the list.
2023-09-03 18:10:49 -04:00
schroda
56deea9fb3 Feature/graphql logging (#674)
* Set graphql logs to error level

Set log level for loggers with names
 - ExecutionStrategy (spams logs with "... completing field ...")
 - notprivacysafe (logs every received request up to 4 times (received, parse, validate, execute))

* Extract logic to get logger for name into function

* Add function to set log level for a logger

* Add settings to enable graphql debug logging
2023-09-03 18:10:43 -04:00
schroda
1c9a139006 Always return "ArchiveProvider" in case "downloadAsCbz" is enabled (#671)
* Move chapter download logic to base class

* Do not reuse "FolderProvider" in "ArchiveProviders" download function

Due to reusing the "FolderProvider" to download a chapter as a cbz file, a normal chapter download folder was created.
In case the download was aborted before the cbz file got created and the folder deleted, the next time the chapter got downloaded, the wrong "FileProvider" was selected, causing the chapter not to get downloaded as a cbz file.
2023-08-28 19:25:50 -04:00
Mitchell Syer
4d89c324b9 Fix Oracle JRE Extension Install (#670)
* Minor cleanup

* Fix Oracle JRE Write issue
2023-08-27 22:39:05 -04:00
schroda
a76ce03911 Throw error instead of returning null (#666)
In case e.g. no manga exists for the passed id, the query returned null.
This makes it harder to have a "streamlined" error handling in the client, since these types of queries need a special handling.
2023-08-27 22:38:52 -04:00