Decouple Django CSRF, ALLOWED_HOST settings for more complex setups

- Set KHOJ_ALLOWED_DOMAIN to the domain that Khoj is accessible on
  from the host machine. This can be the internal i.p or domain of the
  host machine.

  It can be used by your load balancer/reverse_proxy to access Khoj.
  For example, if the load balancer service is in the khoj docker
  network, KHOJ_DOMAIN will be `server' (i.e service name).

- Set KHOJ_DOMAIN to your externally accessible DOMAIN or I.P to avoid
  CSRF trusted origin or unset cookie issue when trying to access the
  khoj admin panel.

Resolves #1114

Author: debanjum

docker-compose.yml

@@ -81,12 +81,18 @@ services:
       # - OLOSTEP_API_KEY=your_olostep_api_key
       #
       # Uncomment the necessary lines below to make your instance publicly accessible.
-      # Replace the KHOJ_DOMAIN with either your domain or IP address (no http/https prefix).
       # Proceed with caution, especially if you are using anonymous mode.
       # ---
       # - KHOJ_NO_HTTPS=True
+      # Replace the KHOJ_DOMAIN with the server's externally accessible domain or I.P address from a remote machie (no http/https prefix).
+      # Ensure this is set correctly to avoid CSRF trusted origin or unset cookie issue when trying to access the admin panel.
       # - KHOJ_DOMAIN=192.168.0.104
       # - KHOJ_DOMAIN=khoj.example.com
+      # Replace the KHOJ_ALLOWED_DOMAIN with the server's internally accessible domain or I.P address on the host machine (no http/https prefix).
+      # Only set if using a load balancer/reverse_proxy in front of your Khoj server. If unset, it defaults to KHOJ_DOMAIN.
+      # For example, if the load balancer service is added to the khoj docker network, set KHOJ_ALLOWED_DOMAIN to khoj's docker service name: `server'.
+      # - KHOJ_ALLOWED_DOMAIN=server
+      # - KHOJ_ALLOWED_DOMAIN=127.0.0.1
       # Uncomment the line below to disable telemetry.
       # Telemetry helps us prioritize feature development and understand how people are using Khoj
       # Read more at https://docs.khoj.dev/miscellaneous/telemetry

documentation/docs/get-started/setup.mdx

@@ -283,9 +283,14 @@ Go to http://localhost:42110/server/admin and login with the admin credentials y
 Ensure you are using **localhost, not 127.0.0.1**, to access the admin panel to avoid the CSRF error.
 :::
 
+:::info[CSRF Trusted Origin or Unset Cookie Error]
+If using a load balancer/reverse_proxy in front of your Khoj server: Set the environment variable KHOJ_ALLOWED_DOMAIN=your-internal-ip-or-domain to avoid this error.
+If unset, it defaults to KHOJ_DOMAIN.
+:::
+
 :::info[DISALLOWED HOST or Bad Request (400) Error]
 You may hit this if you try access Khoj exposed on a custom domain (e.g. 192.168.12.3 or example.com) or over HTTP.
-Set the environment variables KHOJ_DOMAIN=your-domain and KHOJ_NO_HTTPS=True if required to avoid this error.
+Set the environment variables KHOJ_DOMAIN=your-external-ip-or-domain and KHOJ_NO_HTTPS=True if required to avoid this error.
 :::
 
 :::tip[Note]

src/khoj/app/settings.py

@@ -32,7 +32,8 @@
 
 # All Subdomains of KHOJ_DOMAIN are trusted
 KHOJ_DOMAIN = os.getenv("KHOJ_DOMAIN", "khoj.dev")
-ALLOWED_HOSTS = [f".{KHOJ_DOMAIN}", "localhost", "127.0.0.1", "[::1]", f"{KHOJ_DOMAIN}"]
+KHOJ_ALLOWED_DOMAIN = os.getenv("KHOJ_ALLOWED_DOMAIN", KHOJ_DOMAIN)
+ALLOWED_HOSTS = [f".{KHOJ_ALLOWED_DOMAIN}", "localhost", "127.0.0.1", "[::1]", f"{KHOJ_ALLOWED_DOMAIN}"]
 
 CSRF_TRUSTED_ORIGINS = [
     f"https://*.{KHOJ_DOMAIN}",
@@ -45,7 +46,7 @@
 DISABLE_HTTPS = is_env_var_true("KHOJ_NO_HTTPS")
 
 COOKIE_SAMESITE = "None"
-if DEBUG or os.getenv("KHOJ_DOMAIN") == None:
+if DEBUG and os.getenv("KHOJ_DOMAIN") == None:
     SESSION_COOKIE_DOMAIN = "localhost"
     CSRF_COOKIE_DOMAIN = "localhost"
 else: