Migrate CLI handling from Apache Commons CLI to PicoCLI (#13012)

* wip

* shorten annotations and fix parsing bug

* Some visual tweaks

* Adapt commands

* Remove code duplication and moved some code to JabRefGUI

* Use common clearOldSearchIndices method

* Reorder

* Prototype KitCommandline in PicoCli

* wip

* Migrate aux

* Migrate search and preferences

* Convert

* Reorder

* Fetch

* Use distinctive vars for files and citekeys and show export formats again

* Remove double annotations

* Cleanups

* Make checkstyle happy again

* Fix compile errors

* Fix checkstyle

* Fix GenerateBibFromAux test

* Add mixins for global options

* Fix tests, refactor

* Fix search

* Make methods static for easier testing

* Cleanup

* Fix banner

* Adapt gui banner

* Fixes to mixins

* CHANGELOG.md

* Added check on bib file to consistency CheckConsistency

* Fit convert for porcelain mode

* JabKit with capital K

* Some orthographic fixes

* Implement ProviderConverter

* Fix fetcher usage message, undo custom converter

* More porcelain

* Porcelain

* Remove doc comment

* Move JABREF_BANNER to jablib

* Change signature

* Reintroduce uri parsing for inputFile

* Rewrite

* Branding for JabKit

* CHANGELOG.md

* DevDocs

* Remove wrong comment

* Remove deprecated comment

* l10n

* Remove obsolete comment

* Modify JABREF_BANNER

* Reword parserResult, replace System.err with LOGGER

* CHANGELOG.md

* Fix StringUtilTest

* Remove obsolete javafx/annotations.xml

* Fix javafx workflow

* l10n

* Remove %n

---------

Co-authored-by: Carl Christian Snethlage <[email protected]>

Author: calixtus

.github/workflows/deployment-ea.yml

@@ -147,6 +147,7 @@ jobs:
         run: |
           set -e
           set -x
+          mkdir javafx
           cd javafx
           curl --no-progress-meter https://jdk.java.net/javafx${{ env.javafx }}/ > javafx.html
 

.gitignore

@@ -14,11 +14,7 @@ jablib/src/main/resources/csl-locales
 
 .kotlin
 
-javafx/javafx-sdk-*
-javafx/javafx-jmods-*
-javafx/javafx.html
-javafx/*.tar.gz
-javafx/*.zip
+javafx/
 
 # generated by https://plugins.jetbrains.com/plugin/15991-plantuml-diagram-generator
 *.puml

CHANGELOG.md

@@ -11,13 +11,15 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv
 
 ### Added
 
+- We introduced a new command line application called `jabkit`. [#13012](https://github.com/JabRef/jabref/pull/13012) [#110](https://github.com/JabRef/jabref/issues/110)
 - We added a new "Add JabRef suggested groups" option in the context menu of "All entries". [#12659](https://github.com/JabRef/jabref/issues/12659)
 - We added an option to create entries directly from Bib(La)TeX sources to the 'Create New Entry' tool. [#8808](https://github.com/JabRef/jabref/issues/8808)
 - We added the provision to choose different CSL bibliography body formats (e.g. First Line Indent, Hanging Indent, Bibliography 1, etc.) in the LibreOffice integration. [#13049](https://github.com/JabRef/jabref/issues/13049)
 - We added "Bibliography Heading" to the available CSL bibliography header formats in the LibreOffice integration. [#13049](https://github.com/JabRef/jabref/issues/13049)
 
 ### Changed
 
+- We moved some functionality from the graphical application `jabref` with new command verbs `generate-citation-keys`, `check-consistency`, `fetch`, `search`, `convert`, `generate-bib-from-aux`, `preferences` and `pdf` to the new toolkit. [#13012](https://github.com/JabRef/jabref/pull/13012) [#110](https://github.com/JabRef/jabref/issues/110)
 - We merged the 'New Entry', 'Import by ID', and 'New Entry from Plain Text' tools into a single 'Create New Entry' tool. [#8808](https://github.com/JabRef/jabref/issues/8808)
 - We renamed the "Body Text" CSL bibliography header format name to "Text body" as per internal LibreOffice conventions. [#13074](https://github.com/JabRef/jabref/pull/13074)
 - We moved the "Modify bibliography title" option from the CSL styles tab of the Select Style dialog to the OpenOffice/LibreOffice side panel and renamed it to "Bibliography properties". [#13074](https://github.com/JabRef/jabref/pull/13074)
@@ -35,6 +37,7 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv
 
 ### Removed
 
+- We removed the ability to change internal preference values. [#13012](https://github.com/JabRef/jabref/pull/13012)
 - We removed support for MySQL/MariaDB and Oracle. [#12990](https://github.com/JabRef/jabref/pull/12990)
 - We removed library migrations (users need to use JabRef 6.0-alpha.1 to perform migrations) [#12990](https://github.com/JabRef/jabref/pull/12990)
 

docs/code-howtos/cli.md

@@ -3,12 +3,16 @@ parent: Code Howtos
 ---
 # Command Line Interface
 
-The package `org.jabref.cli` is responsible for handling the command line options.
+The module `jabkit` is mainly responsible for handling the command line options. The module `jabgui` also provides some rudimentary functionality, mainly responsible for command line interaction with the gui.
+
+In both modules, the package `org.jabref.cli` uses the PicoCli library to process arguments.
 
 During development, one can configure IntelliJ to pass command line parameters:
 
 ![IntelliJ-run-configuration](../images/intellij-run-configuration-command-line.png)
 
-Passing command line arguments using gradle is currently not possible as all arguments (such as `-Dfile.encoding=windows-1252`) are passed to the application.
+Passing command line arguments using gradle is possible by adding the arguments to the run command with the `--args` option.
+
+![IntelliJ-run-configuration](../images/gradle-run-config-with-args.png)
 
 Without [jlink](https://docs.oracle.com/en/java/javase/11/tools/jlink.html), it is not possible to generate a fat jar anymore. During development, the capabilities of the IDE has to be used.

docs/images/gradle-run-config-with-args.png

@@ -1,4 +1,5 @@
 import org.gradle.internal.os.OperatingSystem
+import org.gradle.kotlin.dsl.annotationProcessor
 import org.javamodularity.moduleplugin.extensions.CompileModuleOptions
 import org.javamodularity.moduleplugin.extensions.RunModuleOptions
 
@@ -109,7 +110,8 @@ dependencies {
 
     implementation("com.github.javakeyring:java-keyring:1.0.4")
 
-    implementation("commons-cli:commons-cli:1.9.0")
+    implementation("info.picocli:picocli:4.7.7")
+    annotationProcessor("info.picocli:picocli-codegen:4.7.7")
 
     implementation("de.undercouch:citeproc-java:3.3.0") {
         exclude(group = "org.antlr")

jabgui/build.gradle.kts

@@ -105,7 +105,7 @@
     requires com.google.common;
     requires io.github.javadiffutils;
     // requires java.string.similarity;
-    requires org.apache.commons.cli;
+    requires info.picocli;
     // requires org.apache.commons.compress;
     // requires org.apache.commons.csv;
     requires org.apache.commons.io;

jabgui/src/main/java/module-info.java

@@ -3,43 +3,32 @@
 import java.io.File;
 import java.io.IOException;
 import java.net.Authenticator;
-import java.nio.file.DirectoryStream;
 import java.nio.file.Files;
 import java.nio.file.Path;
-import java.util.ArrayList;
-import java.util.Comparator;
+import java.util.Arrays;
 import java.util.List;
 import java.util.Map;
 
 import org.jabref.cli.ArgumentProcessor;
-import org.jabref.cli.CliOptions;
 import org.jabref.gui.JabRefGUI;
 import org.jabref.gui.preferences.GuiPreferences;
 import org.jabref.gui.preferences.JabRefGuiPreferences;
-import org.jabref.gui.util.DefaultFileUpdateMonitor;
 import org.jabref.logic.UiCommand;
 import org.jabref.logic.citationstyle.CSLStyleLoader;
-import org.jabref.logic.journals.JournalAbbreviationLoader;
-import org.jabref.logic.journals.JournalAbbreviationRepository;
 import org.jabref.logic.net.ProxyAuthenticator;
 import org.jabref.logic.net.ProxyPreferences;
 import org.jabref.logic.net.ProxyRegisterer;
 import org.jabref.logic.net.ssl.SSLPreferences;
 import org.jabref.logic.net.ssl.TrustStoreManager;
 import org.jabref.logic.preferences.CliPreferences;
-import org.jabref.logic.protectedterms.ProtectedTermsLoader;
 import org.jabref.logic.remote.RemotePreferences;
 import org.jabref.logic.remote.client.RemoteClient;
 import org.jabref.logic.search.PostgreServer;
 import org.jabref.logic.util.BuildInfo;
 import org.jabref.logic.util.Directories;
-import org.jabref.logic.util.HeadlessExecutorService;
 import org.jabref.migrations.PreferencesMigrations;
-import org.jabref.model.entry.BibEntryTypesManager;
-import org.jabref.model.util.FileUpdateMonitor;
 
 import com.airhacks.afterburner.injection.Injector;
-import org.apache.commons.cli.ParseException;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.slf4j.bridge.SLF4JBridgeHandler;
@@ -50,23 +39,36 @@
 /// It has two main functions:
 ///
 /// - Handle the command line arguments
-/// - Start the JavaFX application (if not in CLI mode)
+/// - Start the JavaFX application
 public class Launcher {
     private static Logger LOGGER;
 
     public static void main(String[] args) {
         initLogging(args);
 
-        // Initialize preferences
+        Injector.setModelOrService(BuildInfo.class, new BuildInfo());
+
         final JabRefGuiPreferences preferences = JabRefGuiPreferences.getInstance();
         Injector.setModelOrService(CliPreferences.class, preferences);
         Injector.setModelOrService(GuiPreferences.class, preferences);
 
-        DefaultFileUpdateMonitor fileUpdateMonitor = new DefaultFileUpdateMonitor();
-        HeadlessExecutorService.INSTANCE.executeInterruptableTask(fileUpdateMonitor, "FileUpdateMonitor");
+        // Early exit in case another instance is already running
+        if (!handleMultipleAppInstances(args, preferences.getRemotePreferences())) {
+            systemExit();
+        }
 
-        List<UiCommand> uiCommands = processArguments(args, preferences, fileUpdateMonitor);
-        // The method `processArguments` quits the whole JVM if no GUI is needed.
+        configureProxy(preferences.getProxyPreferences());
+        configureSSL(preferences.getSSLPreferences());
+
+        ArgumentProcessor argumentProcessor = new ArgumentProcessor(
+                args,
+                ArgumentProcessor.Mode.INITIAL_START,
+                preferences);
+
+        List<UiCommand> uiCommands = argumentProcessor.processArguments();
+        if (argumentProcessor.shouldShutDown()) {
+            systemExit();
+        }
 
         PreferencesMigrations.runMigrations(preferences);
 
@@ -75,7 +77,7 @@ public static void main(String[] args) {
 
         CSLStyleLoader.loadInternalStyles();
 
-        JabRefGUI.setup(uiCommands, preferences, fileUpdateMonitor);
+        JabRefGUI.setup(uiCommands, preferences);
         JabRefGUI.launch(JabRefGUI.class, args);
     }
 
@@ -89,14 +91,8 @@ public static void initLogging(String[] args) {
         SLF4JBridgeHandler.install();
 
         // We must configure logging as soon as possible, which is why we cannot wait for the usual
-        // argument parsing workflow to parse logging options .e.g. --debug
-        boolean isDebugEnabled;
-        try {
-            CliOptions cliOptions = new CliOptions(args);
-            isDebugEnabled = cliOptions.isDebugLogging();
-        } catch (ParseException e) {
-            isDebugEnabled = false;
-        }
+        // argument parsing workflow to parse logging options e.g. --debug
+        boolean isDebugEnabled = Arrays.stream(args).anyMatch(arg -> "--debug".equalsIgnoreCase(arg));
 
         // addLogToDisk
         // We cannot use `Injector.instantiateModelOrService(BuildInfo.class).version` here, because this initializes logging
@@ -132,60 +128,10 @@ private static void systemExit() {
         System.exit(0);
     }
 
-    public static List<UiCommand> processArguments(String[] args, JabRefGuiPreferences preferences, FileUpdateMonitor fileUpdateMonitor) {
-        try {
-            Injector.setModelOrService(BuildInfo.class, new BuildInfo());
-
-            // Early exit in case another instance is already running
-            if (!handleMultipleAppInstances(args, preferences.getRemotePreferences())) {
-                systemExit();
-            }
-
-            BibEntryTypesManager entryTypesManager = preferences.getCustomEntryTypesRepository();
-            Injector.setModelOrService(BibEntryTypesManager.class, entryTypesManager);
-
-            Injector.setModelOrService(JournalAbbreviationRepository.class, JournalAbbreviationLoader.loadRepository(preferences.getJournalAbbreviationPreferences()));
-            Injector.setModelOrService(ProtectedTermsLoader.class, new ProtectedTermsLoader(preferences.getProtectedTermsPreferences()));
-
-            configureProxy(preferences.getProxyPreferences());
-            configureSSL(preferences.getSSLPreferences());
-
-            clearOldSearchIndices();
-
-            try {
-                Injector.setModelOrService(FileUpdateMonitor.class, fileUpdateMonitor);
-
-                // Process arguments
-                ArgumentProcessor argumentProcessor = new ArgumentProcessor(
-                        args,
-                        ArgumentProcessor.Mode.INITIAL_START,
-                        preferences,
-                        fileUpdateMonitor);
-                argumentProcessor.processArguments();
-                if (argumentProcessor.shouldShutDown()) {
-                    LOGGER.debug("JabRef shut down after processing command line arguments");
-                    systemExit();
-                    return null;
-                }
-
-                return new ArrayList<>(argumentProcessor.getUiCommands());
-            } catch (ParseException e) {
-                LOGGER.error("Problem parsing arguments", e);
-                CliOptions.printUsage(preferences);
-                systemExit();
-                return null;
-            }
-        } catch (Exception ex) {
-            LOGGER.error("Unexpected exception", ex);
-            systemExit();
-            return null;
-        }
-    }
-
     /**
      * @return true if JabRef should continue starting up, false if it should quit.
      */
-    private static boolean handleMultipleAppInstances(String[] args, RemotePreferences remotePreferences) throws InterruptedException {
+    private static boolean handleMultipleAppInstances(String[] args, RemotePreferences remotePreferences) {
         LOGGER.trace("Checking for remote handling...");
         if (remotePreferences.useRemoteServer()) {
             // Try to contact already running JabRef
@@ -225,30 +171,4 @@ private static void configureProxy(ProxyPreferences proxyPreferences) {
     private static void configureSSL(SSLPreferences sslPreferences) {
         TrustStoreManager.createTruststoreFileIfNotExist(Path.of(sslPreferences.getTruststorePath()));
     }
-
-    private static void clearOldSearchIndices() {
-        Path currentIndexPath = Directories.getFulltextIndexBaseDirectory();
-        Path appData = currentIndexPath.getParent();
-
-        try {
-            Files.createDirectories(currentIndexPath);
-        } catch (IOException e) {
-            LOGGER.error("Could not create index directory {}", appData, e);
-        }
-
-        try (DirectoryStream<Path> stream = Files.newDirectoryStream(appData)) {
-            for (Path path : stream) {
-                if (Files.isDirectory(path) && !path.toString().endsWith("ssl") && path.toString().contains("lucene")
-                        && !path.equals(currentIndexPath)) {
-                    LOGGER.info("Deleting out-of-date fulltext search index at {}.", path);
-                    Files.walk(path)
-                         .sorted(Comparator.reverseOrder())
-                         .map(Path::toFile)
-                         .forEach(File::delete);
-                }
-            }
-        } catch (IOException e) {
-            LOGGER.error("Could not access app-directory at {}", appData, e);
-        }
-    }
 }