feat: module 2 for 2026 cohort (#749)

* feat: pin docker image versions, add 2025 cohort

* feat: cut dbt, add AI Workflows & AI Agents section

* fix: simplify docker set up

* fix: pg db config

* fix: amend pg db name

* feat: reduce the scope

* fix: update structure for 2026

* simplify docker compose set up

* fix: postgres volume

* fix: install instructions

* make module 1 pg work with module 2

* make sure docker compose matches readme

* add more introductory stages

* flesh out readme

* 2.2 details

* fix: headers

* 2.1 complete

* fix heading

* feat: 2.2.1 video

* Add 2.2.2 to 2.3.1

* fix: placeholders for videos

---------

Co-authored-by: Will Russell <[email protected]>
Co-authored-by: Will Russell <[email protected]>

Author: anna-geller

01-docker-terraform/2_docker_sql/README.md

@@ -17,61 +17,21 @@ wget https://github.com/DataTalksClub/nyc-tlc-data/releases/download/yellow/yell
 
 ### Running Postgres with Docker
 
-#### Windows
-
-Running Postgres on Windows (note the full path)
+Running Postgres on Windows, macOS and Linux
 
 ```bash
 docker run -it \
   -e POSTGRES_USER="root" \
   -e POSTGRES_PASSWORD="root" \
   -e POSTGRES_DB="ny_taxi" \
-  -v c:/Users/alexe/git/data-engineering-zoomcamp/week_1_basics_n_setup/2_docker_sql/ny_taxi_postgres_data:/var/lib/postgresql/data \
-  -p 5432:5432 \
-  postgres:13
-```
-
-If you have the following error:
-
-```
-docker run -it \
-  -e POSTGRES_USER="root" \
-  -e POSTGRES_PASSWORD="root" \
-  -e POSTGRES_DB="ny_taxi" \
-  -v e:/zoomcamp/data_engineer/week_1_fundamentals/2_docker_sql/ny_taxi_postgres_data:/var/lib/postgresql/data  \
+  -v ny_taxi_postgres_data:/var/lib/postgresql \
   -p 5432:5432 \
-  postgres:13
-
-docker: Error response from daemon: invalid mode: \Program Files\Git\var\lib\postgresql\data.
-See 'docker run --help'.
-```
-
-Change the mounting path. Replace it with the following:
-
-```
--v /e/zoomcamp/...:/var/lib/postgresql/data
-```
-
-#### Linux and MacOS
-
-
-```bash
-docker run -it \
-  -e POSTGRES_USER="root" \
-  -e POSTGRES_PASSWORD="root" \
-  -e POSTGRES_DB="ny_taxi" \
-  -v $(pwd)/ny_taxi_postgres_data:/var/lib/postgresql/data \
-  -p 5432:5432 \
-  postgres:13
+  postgres:18
 ```
 
 If you see that `ny_taxi_postgres_data` is empty after running
 the container, try these:
 
-* Deleting the folder and running Docker again (Docker will re-create the folder)
-* Adjust the permissions of the folder by running `sudo chmod a+rwx ny_taxi_postgres_data`
-
-
 ### CLI for Postgres
 
 Installing `pgcli`
@@ -143,7 +103,7 @@ Running pgAdmin
 docker run -it \
   -e PGADMIN_DEFAULT_EMAIL="[email protected]" \
   -e PGADMIN_DEFAULT_PASSWORD="root" \
-  -p 8080:80 \
+  -p 8085:80 \
   dpage/pgadmin4
 ```
 
@@ -162,11 +122,11 @@ docker run -it \
   -e POSTGRES_USER="root" \
   -e POSTGRES_PASSWORD="root" \
   -e POSTGRES_DB="ny_taxi" \
-  -v c:/Users/alexe/git/data-engineering-zoomcamp/week_1_basics_n_setup/2_docker_sql/ny_taxi_postgres_data:/var/lib/postgresql/data \
+  -v ny_taxi_postgres_data:/var/lib/postgresql \
   -p 5432:5432 \
   --network=pg-network \
-  --name pg-database \
-  postgres:13
+  --name pgdatabase \
+  postgres:18
 ```
 
 Run pgAdmin
@@ -175,14 +135,14 @@ Run pgAdmin
 docker run -it \
   -e PGADMIN_DEFAULT_EMAIL="[email protected]" \
   -e PGADMIN_DEFAULT_PASSWORD="root" \
-  -p 8080:80 \
+  -p 8085:80 \
   --network=pg-network \
   --name pgadmin-2 \
   dpage/pgadmin4
 ```
 
 
-### Data ingestion
+### Data Ingestion
 
 Running locally
 
@@ -205,22 +165,6 @@ Build the image
 docker build -t taxi_ingest:v001 .
 ```
 
-On Linux you may have a problem building it:
-
-```
-error checking context: 'can't stat '/home/name/data_engineering/ny_taxi_postgres_data''.
-```
-
-You can solve it with `.dockerignore`:
-
-* Create a folder `data`
-* Move `ny_taxi_postgres_data` to `data` (you might need to use `sudo` for that)
-* Map `-v $(pwd)/data/ny_taxi_postgres_data:/var/lib/postgresql/data`
-* Create a file `.dockerignore` and add `data` there
-* Check [this video](https://www.youtube.com/watch?v=tOr4hTsHOzU&list=PL3MmuxUbc_hJed7dXYoJw8DoCuVHhGEQb) (the middle) for more details 
-
-
-
 Run the script with Docker
 
 ```bash
@@ -231,47 +175,41 @@ docker run -it \
   taxi_ingest:v001 \
     --user=root \
     --password=root \
-    --host=pg-database \
+    --host=pgdatabase \
     --port=5432 \
     --db=ny_taxi \
     --table_name=yellow_taxi_trips \
     --url=${URL}
 ```
 
-### Docker-Compose 
+### Docker Compose 
 
 Run it:
 
 ```bash
-docker-compose up
+docker compose up
 ```
 
 Run in detached mode:
 
 ```bash
-docker-compose up -d
+docker compose up -d
 ```
 
 Shutting it down:
 
 ```bash
-docker-compose down
-```
-
-Note: to make pgAdmin configuration persistent, create a folder `data_pgadmin`. Change its permission via
-
-```bash
-sudo chown 5050:5050 data_pgadmin
+docker compose down
 ```
 
-and mount it to the `/var/lib/pgadmin` folder:
+Add a docker volume to the `pgadmin` container:
 
 ```yaml
 services:
   pgadmin:
     image: dpage/pgadmin4
     volumes:
-      - ./data_pgadmin:/var/lib/pgadmin
+      - data_pgadmin:/var/lib/pgadmin
     ...
 ```
 

01-docker-terraform/2_docker_sql/docker-compose.yaml

@@ -1,12 +1,16 @@
+volumes:
+  ny_taxi_postgres_data:
+    driver: local
+
 services:
   pgdatabase:
-    image: postgres:13
+    image: postgres:18
     environment:
       - POSTGRES_USER=root
       - POSTGRES_PASSWORD=root
       - POSTGRES_DB=ny_taxi
     volumes:
-      - "./ny_taxi_postgres_data:/var/lib/postgresql/data:rw"
+      - ny_taxi_postgres_data:/var/lib/postgresql
     ports:
       - "5432:5432"
   pgadmin:
@@ -15,5 +19,5 @@ services:
       - [email protected]
       - PGADMIN_DEFAULT_PASSWORD=root
     ports:
-      - "8080:80"
+      - "8085:80"