Planet does no longer aggregate our blog posts (blog.robur.coop)

[hannesm]

Hi,

I noticed that the last "scrape" commit for data/planet/robur.coop was 4 months ago, and we've published various blog posts since then. Is it possible to see the failures from the "scrape" bot, why newer posts are not detected?

Thanks a lot!

[cuihtlauac]

Hi @hannesm, thanks to let us know about that.

Blog scraping logs are in https://github.com/ocaml/ocaml.org/actions under “Scrape (OCaml Planet)” workflow runs.

Here is the latest log:

https://github.com/ocaml/ocaml.org/actions/runs/17089622150/job/48460540509

I don't see any error on https://blog.robur.coop/feed.xml

[hannesm]

Thanks for your links @cuihtlauac. I cannot see any error either. So, any idea what happens to the entries? Or rather, why they are not processed? Is this an issue with our feed, or an issue on the ocaml.org scraping side?

[cuihtlauac]

Fix PR: #3266

[hannesm]

I'm not sure what motivating switching the default to filter on "ocaml", but I suspect that other feeds are affected as well. To speak for myself, the hannes.robur.coop feed is also all about OCaml and MirageOS, and doesn't always tag articles with "ocaml".

[cuihtlauac]

Yes, other feeds are affected as well, that's why I've opened #3267

[hannesm]

Fascinating, I just looked into the scraper output (https://github.com/ocaml/ocaml.org/actions/runs/17147125377/job/48645431399):

• It managed to scrape and add the missing articles, also pushed a PR [scrape.yml] New OCaml blog posts and videos #3268
• but for reasons unknown to me, it directly force-pushed to that PR branch with the commit of the main branch

the relevant output from the scraper:

2025-08-22T05:45:08.9934724Z [command]/usr/bin/git symbolic-ref HEAD --short
2025-08-22T05:45:08.9962347Z main
2025-08-22T05:45:08.9967208Z Working base is branch 'main'
2025-08-22T05:45:08.9975819Z [command]/usr/bin/git checkout --progress -B 4bd9c71f-41a8-4771-8093-b5f0e5b3a1ef HEAD --
2025-08-22T05:45:09.0103110Z Switched to a new branch '4bd9c71f-41a8-4771-8093-b5f0e5b3a1ef'
2025-08-22T05:45:09.0115708Z [command]/usr/bin/git status --porcelain -unormal -- data/video-watch.yml data/video-youtube.yml data/planet/*/*.md
2025-08-22T05:45:09.0203049Z ?? data/planet/anil/mlgpx-is-the-first-tangled-hosted-package-available-on-opam.md
2025-08-22T05:45:09.0204228Z ?? data/planet/jonludlam/an-ocaml-mcp-server.md
2025-08-22T05:45:09.0204786Z ?? data/planet/robur.coop/a-web-ui-for-dnsvizor.md
2025-08-22T05:45:09.0205263Z ?? data/planet/robur.coop/authentication-and-extended-web-ui-for-dnsvizor.md
2025-08-22T05:45:09.0205933Z ?? data/planet/robur.coop/dnsvizor---run-your-own-dhcp-and-dns-mirageos-unikernel---gets-dnssec-support.md
2025-08-22T05:45:09.0206506Z ?? data/planet/robur.coop/dnsvizor-gets-blocklists.md
2025-08-22T05:45:09.0206820Z ?? data/planet/robur.coop/dnsvizor-manual.md
2025-08-22T05:45:09.0207174Z ?? data/planet/robur.coop/dnsvizor-performance-engineering.md
2025-08-22T05:45:09.0207668Z ?? data/planet/robur.coop/dnsvizor-with-dns-over-tls-and-dns-over-https-support.md
2025-08-22T05:45:09.0208112Z ?? data/planet/robur.coop/privacy-for-dnsvizor.md
2025-08-22T05:45:09.0208465Z ?? data/planet/robur.coop/streaming-binary-data-in-mollymawk.md
2025-08-22T05:45:09.0208896Z ?? data/planet/robur.coop/streaming-console-output-in-mollymawk.md
2025-08-22T05:45:09.0209743Z ?? data/planet/tarides/internship-report-refactoring-tools-coming-to-merlin.md
2025-08-22T05:45:09.0210315Z Uncommitted changes found. Adding a commit.

but the force-push to 1b9034b as mentioned in the opened PR, is nothing I can observe....

The only thing I imagine that may happen is that the other scraper (the "(OCaml changelog)" one) uses the very same branch and force-pushes -- they seem to run at the same time (see https://github.com/ocaml/ocaml.org/actions/runs/17147138761/job/48645467256 for their output). So maybe one of the actions should specify the branch The pull request branch name. create-pull-request/patch (from https://github.com/peter-evans/create-pull-request?tab=readme-ov-file#action-inputs) to have distinct branches for the distinct PRs.

[hannesm]

I added the last suggestion (use distinct branches) in the PR #3269

[cuihtlauac]

I noticed the failure of the PR creation this morning (scraping is set so I can check its results during my breakfast). This morning's failure is something I have never seen. Your idea to use two different branches is excellent. Let's try that.