APE 22 follow-up: Affiliated Packages site updates

[pllim]

https://www.astropy.org/affiliated/ (i.e., the files in https://github.com/astropy/astropy.github.com/tree/main/affiliated ) needs to be updated to reflect the now accepted https://github.com/astropy/astropy-APEs/blob/main/APE22.rst .

Particularly relevant: https://github.com/astropy/astropy-APEs/blob/main/APE22.rst#listing-on-websites

This is a follow up of:

• Affiliated Package: Consolidate with pyOpenSci? astropy-project#334
• APE 24: Astropy Affiliated Packages with pyOpenSci (now APE 22) astropy-APEs#87

cc @dhomeier @WilliamJamieson @hamogu

Blocked by

pyOpenSci feed setup. Need coordination with them to set this up. cc @lwasser @eteq

[lwasser]

hey @pllim @eteq this is the feed that we currently have in place. it is a YAML file that will contain metadata for all packages. it updates automagically via CI every 2 weeks.

altho at some point i suspect we will move to a database, we can always generate a similar yaml file to ensure workflows don't break. Does that yaml file meet astropy's needs?

NOTE: i'm going to add a astropy check box to our review template in the next week or two. and then i'll also ensure that yaml file is populated with affiliated for astropy, sunpy, pangeo, etc. We have a bit of time to develop this this back end workflow given we don't have a completed review, yet.

please just let me know what you need from me here. And please note that i'm creating a master TODO list here to track open issues and pr's. once i have everything in one place i'll try to organize it in a timeline.

[pllim]

Given how unresponsive Erik can get sometimes, I might have to bite the bullet and do the website stuff instead (even though that is not my role).

automagically via CI every 2 week

Would be nice to have a timestamp on the YAML in case automagic is broken but we don't realize?

Does that yaml file meet astropy's needs?

I can make it work. I am not as picky as Erik. I see our friend, SunPy, there. Do you know if SunPy has example code on how they are ingesting this, or any other project, for that matter? Wouldn't want to reinvent the wheel here.

TODO

Very useful. Thanks!

[lwasser]

@pllim how is your site built? what creates those html files? and can you help me understand what the time stamp would provide? i don't think anyone besides me is parsing that file right now. Sunpy is part of the pyhc community work that we're doing and they are i think in a bit earlier stage compared to astropy! ie they haven't fully committed yet but i think there is strong interest (like where we were last spring!).

we use it for our package listing page. it's all liquid / jekyll running that. BUT the partner metadata IS there. sunpy is the first package to have a partner key! AND i will be adding the astropy key for contributors and editors soon as well.

[pllim]

The timestamp is for humans, not the parser. I want to know how old is the listing and whether I should bug someone to update it, things like that. I would probably render that info as a "last updated" field in a footnote.

Currently, our info is stored as JSON and it is ingested using JavaScript that I did not write:

astropy.github.com/js/functions.js

Line 288 in 0ba18d3

populatePackageTable('affiliated', filter_pkg_data(data, "coordinated", false));

I am not looking forward to have to update that code, so if you have a better way, I am all ears!

[lwasser]

@pllim right now we don't have the last updated date in the file. BUT you could easily grab the date from the git history - the last commit. the file is always updated every other week via a ci build. so that is consistent. however sometimes i might push an update sooner especially if i just did some updates to the build and want to test things. so i suppose for now, i'd suggest if it's for humans i'd just look at the file here. and you can see that the file is right was updated 5 days ago. but if you create something automated you could then parse the file itself and grab a date from the commit history.

unfortunately i'm not very proficient with js. id say i'm dangerous at best :( i wonder if hte person who created that js could help?

[lwasser]

alternatively you could ask in our slack for ideas? i just know i specifically won't be super helpful with js based parsing.