New study.yml format #13844

turhantolgaunal · 2025-09-09T17:02:32Z

The study.yml is refactored to a new format to implement SEMVER versioning, variable 'databases' is renamed to 'catalogues' for UI consistency.

In the course of working on this draft pull request the query structure is planned to be enhanced to support multiple query formats including catalog-specific variations.

Steps to test

The study.yml file created when a new slr is started should be in a new format and slr search should function as intended.
New format is:
version: 2.0
authors:

author
title: title
research-questions:
question
queries:
query: (author="author" AND title="title")
description: null
lucene: null
catalogue-specific: null
catalogues:
name: ArXiv
enabled: true
metadata:
notes: null
created-date: null
last-modified: null

Mandatory checks

I own the copyright of the code submitted and I license it under the MIT license
I manually tested my changes in running JabRef (always required)
I added JUnit tests for changes (if applicable)
[/] I added screenshots in the PR description (if change is visible to the user)
[/] I described the change in CHANGELOG.md in a way that is understandable for the average user (if change is visible to the user)
I checked the user documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request updating file(s) in https://github.com/JabRef/user-documentation/tree/main/en.

…ther classes

…ml migrations

…ns from the current study.yml version to the new one

…y.yml file

…n and parsing of study.yml if needed

…r to avoid deadlock

…igrator to JSpecify annotations

…nstead of StudyYamlParser

…ith the rest of the codebase

jablib/src/main/java/org/jabref/logic/crawler/StudyRepository.java

…as not needed

… from getCatalogSpecific form StudyQuery

trag-bot · 2025-09-16T19:47:05Z

jabgui/src/test/java/org/jabref/gui/slr/ManageStudyDefinitionViewModelTest.java

@@ -8,7 +8,7 @@
 import org.jabref.logic.importer.ImportFormatPreferences;
 import org.jabref.logic.importer.ImporterPreferences;
 import org.jabref.model.study.Study;
-import org.jabref.model.study.StudyDatabase;
+import org.jabref.model.study.StudyCatalog;


The import statement for StudyCatalog is correct, but the patch does not address the need to use JavaFX ObservableLists for better practice in managing lists in JavaFX applications.

trag-bot · 2025-09-16T19:47:10Z

jabgui/src/test/java/org/jabref/gui/slr/ManageStudyDefinitionViewModelTest.java

@@ -92,14 +92,14 @@ void studyConstructorFillsDatabasesCorrectly(@TempDir Path tempDir) {
    }

    private ManageStudyDefinitionViewModel getManageStudyDefinitionViewModel(Path tempDir) {
-        List<StudyDatabase> databases = List.of(
-                new StudyDatabase("ACM Portal", true));
+        List<StudyCatalog> catalogs = List.of(


The code uses List.of() which is correct, but it does not utilize JavaFX ObservableLists, which is considered best practice for managing lists in JavaFX applications.

trag-bot · 2025-09-16T19:49:26Z

jabgui/src/main/java/org/jabref/gui/slr/StartNewStudyAction.java

-        StudyYamlParser studyYAMLParser = new StudyYamlParser();
-        studyYAMLParser.writeStudyYamlFile(newStudy, studyRepositoryRoot.resolve(StudyRepository.STUDY_DEFINITION_FILE_NAME));
+        StudyYamlService studyYamlService = new StudyYamlService();
+        studyYamlService.writeStudyYamlFile(newStudy, studyRepositoryRoot.resolve(StudyRepository.STUDY_DEFINITION_FILE_NAME));


The method writeStudyYamlFile should ensure it does not return null. If it does, it should use java.util.Optional to handle potential null values.

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlV1Migrator.java

trag-bot · 2025-09-17T18:58:50Z

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlMigrator.java

+
+        if (CURRENT_VERSION.equals(version)) {
+            // Already current version, read the file normally
+            try (InputStream fileInputStream = Files.newInputStream(studyYamlFile)) {


The try block covers the entire method, which is against best practices. It should cover as few statements as possible to minimize the scope of potential exceptions.

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlV1Migrator.java

trag-bot · 2025-09-24T17:31:34Z

jabgui/src/main/java/org/jabref/gui/slr/EditExistingStudyAction.java

        Study study;
        try {
-            study = new StudyYamlParser().parseStudyYamlFile(studyDirectory.resolve(StudyRepository.STUDY_DEFINITION_FILE_NAME));
+            study = new StudyYamlService().parseStudyYamlFile(studyDirectory.resolve(StudyRepository.STUDY_DEFINITION_FILE_NAME));


The try block covers too many statements, which is against best practices. It should only cover the statement that might throw the exception to improve readability and maintainability.

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlMigrator.java

trag-bot · 2025-09-24T17:40:05Z

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlService.java

+    private Optional<String> getVersionFromFile(Path studyYamlFile) throws IOException {
+        ObjectMapper yamlMapper = new ObjectMapper(new YAMLFactory());
+
+        try (InputStream fileInputStream = Files.newInputStream(studyYamlFile)) {


The try block covers multiple statements, which is not recommended. It should cover as few statements as possible to isolate potential exceptions.

trag-bot · 2025-09-24T17:40:08Z

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlService.java

+     */
+    public Study parseStudyYamlFile(Path studyYamlFile) throws IOException {
+        if (needsMigration(studyYamlFile)) {
+            Study migratedStudy = StudyYamlMigrator.migrateStudyYamlFile(studyYamlFile);


The method migrateStudyYamlFile should not return null. It should return an Optional or throw an exception if migration fails.

jablib/src/main/java/org/jabref/model/study/StudyCatalog.java

trag-bot · 2025-09-24T17:52:25Z

jablib/src/test/java/org/jabref/logic/crawler/StudyYamlParserTest.java

    @Test
    void parseStudyFileSuccessfully() throws IOException {
-        Study study = new StudyYamlParser().parseStudyYamlFile(testDirectory.resolve(StudyRepository.STUDY_DEFINITION_FILE_NAME));
+        Study study = new StudyYamlService().parseStudyYamlFile(testDirectory.resolve(StudyRepository.STUDY_DEFINITION_FILE_NAME));


The method parseStudyYamlFile should throw a generic Exception instead of IOException to adhere to the instruction of using 'throws Exception' in the method declaration.

trag-bot · 2025-09-24T17:52:28Z

jablib/src/test/java/org/jabref/logic/crawler/StudyYamlParserTest.java

-        URL studyDefinition = StudyYamlParser.class.getResource("study-jabref-5.7.yml");
-        Study study = new StudyYamlParser().parseStudyYamlFile(Path.of(studyDefinition.toURI()));
+        URL studyDefinition = StudyYamlService.class.getResource("study-jabref-5.7.yml");
+        Study study = new StudyYamlService().parseStudyYamlFile(Path.of(studyDefinition.toURI()));


The method parseStudyYamlFile should throw a generic Exception instead of IOException to adhere to the instruction of using 'throws Exception' in the method declaration.

calixtus · 2025-09-27T09:14:12Z

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlV1Migrator.java

+    /**
+     * Creates metadata by extracting information from file and existing data
+     */
+    private StudyMetadata createMetadata(Path studyYamlFile, V1StudyFormat oldStudy) throws IOException {


Likewise here: Annotate with @NonNull

calixtus · 2025-09-27T09:16:16Z

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlV1Migrator.java

+        } else if (queryObj instanceof Map) {
+            @SuppressWarnings("unchecked")
+            Map<String, Object> queryMap = (Map<String, Object>) queryObj;


Suggested change

} else if (queryObj instanceof Map) {

@SuppressWarnings("unchecked")

Map<String, Object> queryMap = (Map<String, Object>) queryObj;

} else if (queryObj instanceof Map<String, Object> queryMap) {

I don't know if this works with SuppressWarnings

This gives a unsafe casting error though

Maybe for the whole method this warning can be suppressed, as 4 lines below the warning is suppressed again. Some call it cheating, i call it an elegant way of circumventing a problem. ^^

I think the issue is the runtime erasure of Generics, so this will be always unsafe because type info is lost at runtime. https://www.baeldung.com/java-type-erasure

…tor.java Co-authored-by: Carl Christian Snethlage <[email protected]>

…sion

…ptional

…ator

trag-bot · 2025-09-29T16:46:40Z

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlV1Migrator.java

+     * Generate notes for the migration
+     */
+    private String generateMigrationNotes(V1StudyFormat oldStudy) {
+        StringJoiner notes = new StringJoiner(" ");


StringJoiner is used correctly here, but ensure that it is used consistently throughout the codebase for string concatenation to maintain consistency.

trag-bot · 2025-09-29T16:50:33Z

@trag-bot didn't find any issues in the code! ✅✨

trag-bot · 2025-09-29T16:54:20Z

jablib/src/test/java/org/jabref/logic/crawler/StudyYamlParserTest.java

        assertEquals(expectedStudy, study);
    }

    @Test
    void writeStudyFileSuccessfully() throws IOException {
-        new StudyYamlParser().writeStudyYamlFile(expectedStudy, testDirectory.resolve(StudyRepository.STUDY_DEFINITION_FILE_NAME));
-        Study study = new StudyYamlParser().parseStudyYamlFile(testDirectory.resolve(StudyRepository.STUDY_DEFINITION_FILE_NAME));
+        new StudyYamlService().writeStudyYamlFile(expectedStudy, testDirectory.resolve(StudyRepository.STUDY_DEFINITION_FILE_NAME));


The method writeStudyYamlFile should throw a generic Exception instead of IOException to adhere to the principle of letting JUnit handle exceptions in tests.

trag-bot · 2025-09-29T16:59:27Z

jablib/src/test/java/org/jabref/logic/crawler/StudyYamlParserTest.java

@@ -58,8 +58,8 @@ void readsJabRef57StudySuccessfully() throws URISyntaxException, IOException {
        // If the field is "just" removed from the datamodel, one gets following exception:
        //   com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "last-search-date" (class org.jabref.model.study.Study), not marked as ignorable (5 known properties: "authors", "research-questions", "queries", "title", "databases"])
        // This tests ensures that this exception does not occur
-        URL studyDefinition = StudyYamlParser.class.getResource("study-jabref-5.7.yml");
-        Study study = new StudyYamlParser().parseStudyYamlFile(Path.of(studyDefinition.toURI()));
+        URL studyDefinition = StudyYamlService.class.getResource("study-jabref-5.7.yml");


The method readsJabRef57StudySuccessfully should throw a generic Exception instead of IOException and URISyntaxException to adhere to the principle of letting JUnit handle exceptions in tests.

trag-bot · 2025-09-29T17:05:02Z

jablib/src/test/java/org/jabref/logic/crawler/StudyYamlV1MigratorTest.java

+        assertEquals("Database2", catalogs.get(1).getName());
+
+        // Third database explicitly disabled
+        assertFalse(catalogs.get(2).isEnabled());


The code uses assertFalse to check a boolean condition. It should assert the contents of objects using assertEquals for better clarity and precision.

…yYamlV1Migrator

Siedlerchr · 2025-10-14T17:56:54Z

jablib/build.gradle.kts

    implementation("com.fasterxml.jackson.datatype:jackson-datatype-jsr310")
    // TODO: Somwewhere we get a warning: unknown enum constant Id.CLASS reason: class file for com.fasterxml.jackson.annotation.JsonTypeInfo$Id not found
    // implementation("com.fasterxml.jackson.core:jackson-annotations:2.19.1")
+    implementation("com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.15.2")


Put the version in versions/build.gradle

Siedlerchr

lgtm so far

Version in versions/build.gradle is required.

Reformatted database variable to catalogue

cae829c

turhantolgaunal changed the title ~~Reformatted database variable to catalogue~~ New study.yml format Sep 9, 2025

turhantolgaunal added 12 commits September 12, 2025 20:26

Renamed StudyDatabase.java to StudyCatalog and updated its usage in o…

2921b60

…ther classes

Created an abstract StudyYamlMigrator class as a template for study.y…

1f4a76c

…ml migrations

Created an implementation of the abstract Migrator class for migratio…

4fc5972

…ns from the current study.yml version to the new one

Created a StudyMetadata class to handle the metadata part of the stud…

5c0f359

…y.yml file

Created a new class called StudyYamlService for handling the migratio…

4117dd0

…n and parsing of study.yml if needed

StudyYamlMigrator no longer uses child classes in constructor in orde…

a257409

…r to avoid deadlock

Ran openrewrite and changed the jetbrains annotations in StudyYamlV1M…

6ca6a9c

…igrator to JSpecify annotations

Changed StudyYamlParserTest.java to use the new StudyYamlParserTest i…

ea1df6d

…nstead of StudyYamlParser

Merge branch 'main' into newstudyyaml

4d647b0

Changed the usage of the word catalogue to catalog to be consistent w…

75cd28f

…ith the rest of the codebase

Merge remote-tracking branch 'origin/newstudyyaml' into newstudyyaml

0938051

Added functions for adding a metadata field to the study.yml

89d53d2

trag-bot bot reviewed Sep 14, 2025

View reviewed changes

jablib/src/main/java/org/jabref/logic/crawler/StudyRepository.java Show resolved Hide resolved

turhantolgaunal and others added 3 commits September 16, 2025 22:03

Added unit tests for MigratorV1, removed the study type field as it w…

837a7b5

…as not needed

Changed database variable to catalog and an immutable map is returned…

4167044

… from getCatalogSpecific form StudyQuery

Merge branch 'JabRef:main' into newstudyyaml

be06340

trag-bot bot reviewed Sep 16, 2025

View reviewed changes

turhantolgaunal and others added 2 commits September 17, 2025 20:28

Merge branch 'JabRef:main' into newstudyyaml

b210e1e

Reformatted files

6a1326e

trag-bot bot reviewed Sep 17, 2025

View reviewed changes

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlV1Migrator.java Outdated Show resolved Hide resolved

turhantolgaunal added 2 commits September 17, 2025 21:50

Removed studytype parameter in StudyMetadata

23f3bb6

yaml mapper variable in StudyYamlMigrator is now a static field

950af58

trag-bot bot reviewed Sep 17, 2025

View reviewed changes

turhantolgaunal added 2 commits September 17, 2025 22:04

Changed null handling on some variables to use Optional

9bdc084

Changed the use of StringBuilder to StringJoiner in StudyYamlV1Migrator

e7f35dd

trag-bot bot reviewed Sep 17, 2025

View reviewed changes

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlV1Migrator.java Show resolved Hide resolved

trag-bot bot reviewed Sep 24, 2025

View reviewed changes

jablib/src/main/java/org/jabref/logic/crawler/StudyYamlMigrator.java Show resolved Hide resolved

trag-bot bot reviewed Sep 24, 2025

View reviewed changes

jablib/src/main/java/org/jabref/model/study/StudyCatalog.java Show resolved Hide resolved

trag-bot bot reviewed Sep 24, 2025

View reviewed changes

calixtus reviewed Sep 27, 2025

View reviewed changes

Update jablib/src/main/java/org/jabref/logic/crawler/StudyYamlV1Migra…

eb689a9

…tor.java Co-authored-by: Carl Christian Snethlage <[email protected]>

turhantolgaunal marked this pull request as draft September 27, 2025 11:02

turhantolgaunal added 3 commits September 27, 2025 19:23

Reverted the update to instanceof check because of unsafe type conver…

a7b4e91

…sion

Added nonnull annotations and changed the metadata field to not use O…

da2c6f2

…ptional

Added nonnull annotations to getVersion function from StudyYamlV1Migr…

563110c

…ator

turhantolgaunal marked this pull request as ready for review September 29, 2025 16:31

Merge branch 'main' into newstudyyaml

1670b47

trag-bot bot reviewed Sep 29, 2025

View reviewed changes

turhantolgaunal added 4 commits October 4, 2025 14:53

Merge branch 'main' into newstudyyaml

31debba

Merge branch 'main' into newstudyyaml

3ec41ee

Merge branch 'main' into newstudyyaml

2709a1e

Changed the suppress warning annotation to the whole function in Stud…

2d309fc

…yYamlV1Migrator

Siedlerchr reviewed Oct 14, 2025

View reviewed changes

Siedlerchr previously approved these changes Oct 14, 2025

View reviewed changes

Siedlerchr requested a review from koppor October 14, 2025 17:57

Siedlerchr added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Oct 14, 2025

Uh oh!

New study.yml format #13844

Are you sure you want to change the base?

New study.yml format #13844

Uh oh!

Conversation

turhantolgaunal commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Steps to test

Mandatory checks

Uh oh!

Uh oh!

trag-bot bot Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

trag-bot bot Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

trag-bot bot Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

trag-bot bot Sep 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

trag-bot bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

trag-bot bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

trag-bot bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

trag-bot bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

trag-bot bot Sep 24, 2025

Choose a reason for hiding this comment

Uh oh!

calixtus Sep 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

calixtus Sep 27, 2025

Choose a reason for hiding this comment

Uh oh!

calixtus Sep 27, 2025

Choose a reason for hiding this comment

Uh oh!

turhantolgaunal Sep 27, 2025

Choose a reason for hiding this comment

Uh oh!

calixtus Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

Siedlerchr Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

trag-bot bot Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

trag-bot bot commented Sep 29, 2025

Uh oh!

trag-bot bot Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

trag-bot bot Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

trag-bot bot Sep 29, 2025

Choose a reason for hiding this comment

Uh oh!

Siedlerchr Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

turhantolgaunal commented Sep 9, 2025 •

edited

Loading

calixtus Sep 27, 2025 •

edited

Loading

Siedlerchr Oct 14, 2025 •

edited

Loading