Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
b2cadc0
Drop and recreate migrations
dhvcc Sep 5, 2022
d741dc6
Remove most of the stuff including API, preserve parsing
dhvcc Sep 5, 2022
450e7b9
Update .idea
dhvcc Sep 6, 2022
1848353
Update docker conf
dhvcc Sep 6, 2022
b3bbc04
Add basic ninja API for manga
dhvcc Sep 6, 2022
80f4502
Add async search, add jmeter tests
dhvcc Sep 13, 2022
5be2185
Update a bunch of devops stuff
dhvcc Sep 14, 2022
02af34e
Split parsing app, remove unused apps
dhvcc Sep 17, 2022
02c3d6a
Add jmeter test project
dhvcc Sep 17, 2022
41c6d35
Move readmanga app, clean stuff a bit
dhvcc Sep 18, 2022
1e6567c
Add generated readme
dhvcc Sep 18, 2022
81394f9
Add detail background parsing with scrapyd
dhvcc Sep 18, 2022
933dc02
Update workflows, remove deploy.yaml
dhvcc Sep 18, 2022
8786ce7
Minor updates and fixes
dhvcc Sep 18, 2022
46146eb
Migrate to scrapy background tasks
dhvcc Sep 19, 2022
48a139e
Remove all scrapyd references
dhvcc Sep 19, 2022
19ff5eb
Aggregated commit
dhvcc Oct 1, 2022
7359451
Elasticsearch -> Typesense
dhvcc Oct 1, 2022
0372790
Celery -> RQ, fix task overflowing, docker prod
dhvcc Oct 2, 2022
32c6fa1
Try to cloudbuild
dhvcc Oct 3, 2022
19cf879
Try to use VPC connector ffs
dhvcc Oct 4, 2022
252a3b7
Start using debug
dhvcc Oct 4, 2022
716afc3
Add protocol to typesense settings
dhvcc Oct 4, 2022
0099c57
Add CSRF stub for frontend
dhvcc Oct 4, 2022
9468bdb
Add gcloudignore
dhvcc Oct 4, 2022
2c2cd16
Restore "fast" module. Store all manga details in typesense.
dhvcc Oct 5, 2022
9a83d1f
Add manga identifier, remove alt_title
dhvcc Oct 6, 2022
57a0b95
Building and running rq worker on cloudrun
dhvcc Oct 7, 2022
db9e07f
Add chunks to rebuild_index to avoid timeouts
dhvcc Oct 8, 2022
08c4e38
Update rqworker entrypoint
dhvcc Oct 8, 2022
d3826df
Try to run rqworker with GAE server
dhvcc Oct 8, 2022
fbb20b9
temporary remove log file so it works on GAE
dhvcc Oct 8, 2022
70677bc
Update instance class
dhvcc Oct 8, 2022
3963fd7
Remove custom entrypoint...
dhvcc Oct 8, 2022
8819be4
Remove vpc connector
dhvcc Oct 10, 2022
30ebd28
Return log files
dhvcc Oct 10, 2022
88c17e4
Remove jetbrains redis settings
dhvcc Oct 10, 2022
66d769e
Remove log file for it to work on GAE
dhvcc Oct 10, 2022
3d3cfc1
Refresh parsing, add error handling, make image parsing delayed
dhvcc Oct 17, 2022
c3ff36e
Fix images schema and info script function
dhvcc Oct 20, 2022
283acda
Add auth
dhvcc Oct 30, 2022
5bf7523
Add SaveList and lists api
dhvcc Nov 3, 2022
dc0c83a
Add mangachan
dhvcc Nov 5, 2022
4a689b1
Use scrapyscript fork, drop uvicorn
dhvcc Nov 6, 2022
3096c6f
New Catalogues, Spiders and Enums
dhvcc Nov 6, 2022
1a94754
Fix liniting
dhvcc Nov 6, 2022
72f7f0c
Remove hashes from exported requirements to allow for git repos
dhvcc Nov 6, 2022
a57e7d5
Another build fix
dhvcc Nov 6, 2022
949c948
Make debug_toolbar optional
dhvcc Nov 7, 2022
79081f0
Add tests, fix str enums for 3.10, add pytest
dhvcc Nov 7, 2022
7705d36
Fix unique nulls in postgres and std stubs for cli
dhvcc Nov 8, 2022
617dc52
Add webhook stuff to infra
dhvcc Nov 8, 2022
59813e3
Whoops
dhvcc Nov 8, 2022
a167f20
ping webhook
dhvcc Nov 8, 2022
331af9c
Fix Catalogue parser lookup, add lookup exceptions
dhvcc Nov 8, 2022
d90fd19
Mangachan image spider, readmanga genres
dhvcc Nov 9, 2022
ee387e4
Add CatalogueMap, clean up warnings
dhvcc Nov 14, 2022
6b3044e
Add Elastic APM
dhvcc Nov 14, 2022
fd367d3
Add Elastic APM logging
dhvcc Nov 14, 2022
92dad17
Log elasticapm errors to console
dhvcc Nov 14, 2022
a0c2903
Hardcode agent url
dhvcc Nov 14, 2022
c47329e
Fix transaction name, update apm name
dhvcc Nov 14, 2022
de6e648
Add sentry
dhvcc Nov 19, 2022
a294c85
Fix without_common_prefix
dhvcc Nov 19, 2022
5b3391a
Add bookmarks
dhvcc Nov 19, 2022
afa2dad
Add JWT auth
dhvcc Nov 21, 2022
568d0c8
Remove optional user nonsense
dhvcc Nov 21, 2022
0e66427
I dont remember
dhvcc Dec 7, 2022
fe6a3e4
Add popularity, fix logging in scrapy and management
dhvcc Dec 10, 2022
3b974b0
Remove Elastic APM
dhvcc Dec 10, 2022
0798862
Add notifications, add schedulers (cron for list and detail)
dhvcc Dec 18, 2022
d24efca
Disable rbd snapshots
dhvcc Dec 25, 2022
d6ca3be
Only notify when there were already chapters present
dhvcc Dec 26, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ celerybeat.pid
*.sage.py

# Environments
.envs/.env
.env
.venv
env/
venv/
Expand Down Expand Up @@ -150,3 +150,11 @@ LICENSE
README.md
redis-data
redis.conf
staticfiles/
infra/
docker/
docker-compose.yml
.gcloudignore
.ipython/
.editorconfig
.env.example
2 changes: 2 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[*.py]
max_line_length = 100
30 changes: 30 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Your local config to develop without docker

# These values are already defaults, this is just for preview
# DATABASE_URL=postgres://postgres:postgres@localhost:5432/sora
# REDIS_URL=redis://localhost:8883
# TYPESENSE_HOST=localhost

# Whitespace-separated list
# ALLOWED_HOSTS=*

##########
# DJANGO #
##########

DEBUG=1
SECRET_KEY=DevServer

#########
# Other #
#########

SENTRY_DSN=

GOOGLE_CLIENT=
GOOGLE_SECRET=

TYPESENSE_HOST=localhost
TYPESENSE_API_KEY=tsapikey

PROXY=
16 changes: 0 additions & 16 deletions .envs.example/deployment.env.example

This file was deleted.

38 changes: 0 additions & 38 deletions .envs.example/docker.env.example

This file was deleted.

36 changes: 0 additions & 36 deletions .envs.example/local.env.example

This file was deleted.

221 changes: 221 additions & 0 deletions .gcloudignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,221 @@
### JetBrains template
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio, WebStorm and Rider
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839

# User-specific stuff
.idea/**/workspace.xml
.idea/**/tasks.xml
.idea/**/usage.statistics.xml
.idea/**/dictionaries
.idea/**/shelf

# Generated files
.idea/**/contentModel.xml

# Sensitive or high-churn files
.idea/**/dataSources/
.idea/**/dataSources.ids
.idea/**/dataSources.local.xml
.idea/**/sqlDataSources.xml
.idea/**/dynamic.xml
.idea/**/uiDesigner.xml
.idea/**/dbnavigator.xml

# Gradle
.idea/**/gradle.xml
.idea/**/libraries

# Gradle and Maven with auto-import
# When using Gradle or Maven with auto-import, you should exclude module files,
# since they will be recreated, and may cause churn. Uncomment if using
# auto-import.
# .idea/artifacts
# .idea/compiler.xml
# .idea/jarRepositories.xml
# .idea/modules.xml
# .idea/*.iml
# .idea/modules
# *.iml
# *.ipr

# CMake
cmake-build-*/

# Mongo Explorer plugin
.idea/**/mongoSettings.xml

# File-based project format
*.iws

# IntelliJ
out/

# mpeltonen/sbt-idea plugin
.idea_modules/

# JIRA plugin
atlassian-ide-plugin.xml

# Cursive Clojure plugin
.idea/replstate.xml

# Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml
crashlytics.properties
crashlytics-build.properties
fabric.properties

# Editor-based Rest Client
.idea/httpRequests

# Android studio 3.1+ serialized cache file
.idea/caches/build_file_checksums.ser

### Python template
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

infra/
.ipython/
logs/
.github/
docker/
LICENSE
.flake8
.env.example
.editorconfig
26 changes: 0 additions & 26 deletions .github/workflows/deploy.yml

This file was deleted.

Loading