arnekeller/KIT-ILIAS-downloader
1
0
Fork 0
mirror of https://github.com/FliegendeWurst/KIT-ILIAS-downloader.git synced 2024-08-28 04:04:18 +00:00
Code Issues Projects Releases Wiki Activity
231 Commits 4 Branches 34 Tags 1 MiB
603c3eef7c
Commit Graph

130 Commits

Author SHA1 Message Date
FliegendeWurst
dc02ec7a46 Request rate limiting 2021-05-07 09:15:24 +02:00
FliegendeWurst
81543b1d39 Version 0.2.17 2021-05-04 10:47:23 +02:00
FliegendeWurst
3887d6781f Various bugfixes 
course.html: no longer includes right column
course.html/folder.html: only downloaded if useful
correct handling of pseudonymous forums
 2021-04-22 12:17:45 +02:00
FliegendeWurst
338c97821b Various cleanups 2021-04-21 21:38:08 +02:00
FliegendeWurst
e94ce97896 Handle disabled links + save main course page 
+ misc. cleanups
 2021-04-21 21:04:45 +02:00
FliegendeWurst
5c17ec0326 Display a progress bar if possible 
(experimental)
 2021-04-21 20:18:54 +02:00
FliegendeWurst
f291bcbb4b Handle external images in forum posts 2021-04-19 18:47:19 +02:00
FliegendeWurst
3ece654b52 Fix empty forum warning 2021-04-19 18:46:51 +02:00
FliegendeWurst
5ef486244e Fix warning when downloading forum 
'unusual table row (2 cells)' was caused by the thread count above the main table
'unusual table row (0 cells)' was caused by the table header
 2021-04-19 18:27:59 +02:00
FliegendeWurst
bcc16f1ec5 Use async semaphore to limit parallel jobs 2021-04-19 12:32:35 +02:00
FliegendeWurst
dd37cc9d91 Fix colors on windows 2021-04-19 11:15:55 +02:00
FliegendeWurst
a05ea16d83 --sync-url option 2021-04-19 10:33:46 +02:00
FliegendeWurst
d087ea0e92 Keyring authentication + username/password options 2021-04-17 15:38:18 +02:00
FliegendeWurst
381df53264 Code style + formatting + logging macros 2021-04-16 13:24:10 +02:00
lukasprobst
af2c477a04 Add the dependency 'colored' and color relevant messages 2021-04-15 23:37:36 +02:00
Craeckie
75846141f4 socks5h ensures, that DNS requests are also proxied 2021-04-14 12:00:44 +02:00
Craeckie
af094de705 Added proxy support 2021-04-14 11:55:08 +02:00
FliegendeWurst
652fdd91a4 Include forum post header in download 2021-04-10 13:35:13 +02:00
FliegendeWurst
9cba23e999 Download forum post attachments 2021-04-10 12:55:26 +02:00
funnym0nk3y
5e17706f5e Updated to work again 
Don't know why it works
 2021-02-15 23:12:04 +01:00
FliegendeWurst
a8189de518 Use the async fs::canonicalize 2021-01-24 18:11:01 +01:00
FliegendeWurst
77101a7173 Attempt to use long paths on Windows 2021-01-24 17:52:01 +01:00
FliegendeWurst
711bfe8d17 Update tokio to 1.0 2021-01-20 10:26:38 +01:00
Maximilian Bosch
973bb65867
Fix shibboleth login by passing a CSRF token (#4) 
Credits go 2714ac6be6
where I derived this fix from.
 2020-12-31 10:59:28 +01:00
FliegendeWurst
d557efef6b Handle duplicate file names in exercises 2020-12-16 08:32:47 +01:00
FliegendeWurst
6b1e6cb9a6 Replace invalid characters in filenames (Windows only, fixes #3) 2020-12-10 08:39:08 +01:00
FliegendeWurst
7d4b9279ee Terminate once all tasks are complete 2020-11-28 16:42:26 +01:00
FliegendeWurst
09f69b9efd Wait on all spawned tasks 2020-11-28 13:12:42 +01:00
FliegendeWurst
4770bc9beb Version 0.2.10 2020-11-27 08:45:11 +01:00
FliegendeWurst
f195911531 Load username and password from .iliaslogin 2020-11-09 08:58:21 +01:00
FliegendeWurst
97ac79b19f Replace forward and backward slashes in names 2020-11-06 12:52:46 +01:00
FliegendeWurst
84c65b2b99 Update URL parameters to latest OpenCast 2020-11-01 12:05:32 +01:00
FliegendeWurst
b9446c0dfa Extract url out of web link HEAD request errors 2020-07-22 18:02:43 +02:00
FliegendeWurst
449d642bf5 Fix video downloading 
Load the full lecture list differently (should work for every course now)
Actually download new videos
 2020-07-16 11:34:56 +02:00
FliegendeWurst
1610db4c3c Avoid race condition in network request scheduling 
oops
 2020-07-13 09:41:00 +02:00
FliegendeWurst
1aa28f0d00 Only download video metadata if not downloaded yet 
The user can still request metadata checks.
 2020-07-13 09:29:49 +02:00
FliegendeWurst
b3ad9f42c9 Download full OpenCast lecture list 2020-07-02 23:10:51 +02:00
FliegendeWurst
67ba34af85 Make OpenCast lecture update checks optional 2020-06-23 16:42:15 +02:00
FliegendeWurst
7cc780ed18 Create output directory automatically 2020-06-19 16:51:39 +02:00
FliegendeWurst
cb64cdbd9a Don't use raw html content for video file names 2020-05-15 19:38:13 +02:00
FliegendeWurst
d815a6ea1e Simplify code 2020-05-12 15:09:32 +02:00
FliegendeWurst
0fd1e6ce5b Clean up code slightly 2020-05-11 17:35:30 +02:00
FliegendeWurst
dfdd6829fc Utility function to create a directory 2020-05-11 17:17:13 +02:00
FliegendeWurst
9a88edf297 Create utility function to write data to file 2020-05-11 17:02:05 +02:00
FliegendeWurst
728dde9a5d Error handling around URL parsing 
Is currently dead code, but might be useful eventually.
 2020-05-11 16:08:34 +02:00
FliegendeWurst
9f1a2ab2fa Download exercise feedback 2020-05-11 10:06:22 +02:00
FliegendeWurst
9844de220e Download exercise solutions 2020-05-09 13:17:37 +02:00
FliegendeWurst
d1e758e28a Download exercises and weblinks 2020-05-09 09:25:41 +02:00
FliegendeWurst
c4ca0e15b4 Reorder struct definitions 2020-05-09 09:07:13 +02:00
FliegendeWurst
1cc3e608f9 Logging macro 2020-05-09 09:02:33 +02:00
FliegendeWurst
474ad43fc8 Use CARGO_PKG_NAME 2020-05-08 22:03:45 +02:00
FliegendeWurst
d6f635b442 Warn the user if a video has been updated 
This also 'detects' duplicated videos.
 2020-05-08 21:53:32 +02:00
FliegendeWurst
a3f84aa6fb Use anyhow instead of error-chain for errors 2020-05-08 21:25:45 +02:00
FliegendeWurst
a5afd84764 Improve log message 2020-05-03 21:48:13 +02:00
FliegendeWurst
946dae564b Increase download timeout 2020-04-28 12:02:13 +02:00
FliegendeWurst
aa0beae6f3 Update dependencies 2020-04-28 11:48:44 +02:00
FliegendeWurst
e8e81eeb0f Better error handling 2020-04-25 09:33:21 +02:00
FliegendeWurst
137c1ff113 Handle objects in folders that do not have links 2020-04-25 08:42:05 +02:00
FliegendeWurst
2a4e7c7326 Tweak logging 2020-04-24 20:46:05 +02:00
FliegendeWurst
86f24e0f14 Apply .iliasignore settings to all objects 2020-04-24 11:02:12 +02:00
FliegendeWurst
7faf14e37f Ignore courses specified in .iliasignore file 2020-04-24 10:14:53 +02:00
FliegendeWurst
12876a3e8c Update timeout (video CDN is slow) 2020-04-24 09:49:05 +02:00
FliegendeWurst
b1c148f5a9 Make content tree downloading optional 
It is *really* slow.
 2020-04-24 09:46:48 +02:00
FliegendeWurst
97ee02756d Fix content tree extraction 2020-04-23 21:35:11 +02:00
FliegendeWurst
3af01403bd Use content tree sidebar for discovering folders (ILIAS errors always returned) 2020-04-23 15:27:53 +02:00
FliegendeWurst
7f688f33c5 Handle errors properly 2020-04-23 11:21:05 +02:00
FliegendeWurst
f7abcda316 Paginate forum thread posts and warn about ignored forum pages 2020-04-23 10:03:34 +02:00
FliegendeWurst
cf54f9ad04 Always log new files 2020-04-22 14:07:30 +02:00
FliegendeWurst
20fe42d24a Actually wait for queued network jobs 2020-04-22 13:12:48 +02:00
FliegendeWurst
af0743184f Wait for queued network jobs 2020-04-22 12:59:20 +02:00
FliegendeWurst
dc55666255 Tweak network client settings 2020-04-22 12:53:50 +02:00
FliegendeWurst
36a78eb4d9 Always log new videos 2020-04-22 12:37:42 +02:00
FliegendeWurst
6fea6f2d0f Add CLI flag for forum downloading 2020-04-22 12:21:14 +02:00
FliegendeWurst
84eda01c29 Limit parallel network requests to CLI parameter 2020-04-22 12:19:28 +02:00
FliegendeWurst
1aca252f8c Experimental forum download 2020-04-22 12:03:36 +02:00
FliegendeWurst
a1d5ca5650 Change --help wording 2020-04-21 23:35:12 +02:00
FliegendeWurst
753689d0ba Clean up code 2020-04-21 23:20:04 +02:00
FliegendeWurst
e424ff6a72 Prompt user for username and password 2020-04-21 23:12:05 +02:00
FliegendeWurst
f10bba3e3e Parse command-line arguments 2020-04-21 23:03:30 +02:00
FliegendeWurst
1529a678e0
Initial commit 2020-04-21 22:41:28 +02:00