Issue 432 - Example using Spring Boot

[beargiles]

This is the preliminary implementation of an example implementation using Spring Boot and Test Containers.

It will successfully execute a SpringBootTest using a PostgreSQLContainer and proper initialization of all spring beans.

The current implementation uses a standard PostgreSQL docker image (from
http://hub.docker.com/_/postgres) so it does not support PL/Java-specific tests but
there is a separate effort to produce our own docker images that will have PL/Java pre-installed.

(The key limitation is that the pljava shared
library is not preinstalled.)

At this time there are no tests other than a 'happy path' test that shows that the test's autowired DataSource is pointing to the TestContainer instance.

+==============================================================================+
|  Database Server : Name         : PostgreSQL                                 |
|                  : Version      : 16.3                                       |
|                  : URL          : jdbc:postgresql://localhost:32834/test     |
+------------------+--------------+--------------------------------------------+
|  Driver          : Name         : PostgreSQL JDBC Driver                     |
|                  : Version      : 42.6.0                                     |
|                  : JDBC Version : 4.2                                        |
+------------------+--------------+--------------------------------------------+
|  Client          : User         : test                                       |
|                  : Connection   : com.zaxxer.hikari.pool.HikariProxyConnecti |
+------------------+--------------+--------------------------------------------+
|  Client Host     : User         : bgiles                                     |
|                  : Hostname     : eris.coyotesong.net                        |
|                  : OS Name      : Ubuntu 24.04.2 LTS                         |
|                  : OS Kernel    : 6.8.0-59-generic                           |
+==============================================================================+

[beargiles]

Sorry for the huge PR but it had to cover a lot of territory... and it's still missing some things. (Esp. documentation!)

The README.md file has more details but the gist is that this introduces a new module with three child modules.

Test-framework

We can ignore that. It does lots of spring magic.

Backend

This is what is pushed to the server. It has one responsibility in addition to creating the jar file - it must create a test docker image.

At the moment I'm making some reasonable assumptions (PostgreSQL 17.3, PLJava version 1.6.9) and also reusing the existing 'pljava-examples-1.6.9.jar' instead of copying the locally built jar into the docker working directory. Those are easy to address later.

It would be nice if there was a clean way to reuse the DDR file.

Application

This is a minimal spring boot application that can use the standard tools provided by spring-test. All of the messy stuff is handled by the test-framework and to a lesser extent the docker image that has the pljava jars preinstalled.

A typical test looks like:

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE,
        classes = {
                PersistenceTestConfiguration.class
        }
)
@ContextConfiguration
@ActiveProfiles("test")
@Testcontainers(disabledWithoutDocker = true)
public class HappyPathTest {
    private static final Logger LOG = LoggerFactory.getLogger(HappyPathTest.class);
    protected static final String LOCAL_IMAGE_NAME = "tada/pljava-examples:17.3-1.6.9-bookworm";

    @Container
    @ServiceConnection
    protected static PostgreSQLContainer<?> postgres = new AugmentedPostgreSQLContainer<>(LOCAL_IMAGE_NAME);

    private final DataSource dataSource;

    @Autowired
    public HappyPathTest(DataSource dataSource) {
        this.dataSource = dataSource;
    }

    // ------------------------------
    // actual tests follow
    // ------------------------------

    @Test
    public void happyPathTest() throws SQLException, IOException {
        try (Connection conn = dataSource.getConnection();
             Statement stmt = conn.createStatement()) {

            try (ResultSet rs = stmt.executeQuery("SELECT javatest.randomInts(10)")) {
                while (rs.next()) {
                    LOG.info(rs.getString(1));
                }
            }
        }
    }
}

This looks nasty if you're unfamiliar with the spring annotations but the key thing here is that there's NO references to the actual TestContainer or database server beyond the initial stanza. Everything goes through the DataSource and the test container could be replaced with an external test server with no other changes than commenting out the @Container lines and adding a Spring bean that provides the required connection credentials.

[beargiles]

I've been exploring the cleanest way to handle the 'backend -> docker image' step and a maven plugin looks like the best approach. I know you've mentioned the pgxn plugin in the past but creating a pljava-specific one has some benefits and a minimal implementation should still be good enough for must users.

There's still the issue of deploying pljava + jars to an actual database but I'm not too worried about that since it will be easy to add a flag indicating the plugin should list the actions taken.

[jcflack]

I've been exploring the cleanest way to handle the 'backend -> docker image' step and a maven plugin looks like the best approach. I know you've mentioned the pgxn plugin in the past but creating a pljava-specific one has some benefits

The pljava-pgxs-plugin is already PL/Java-specific and is probably worth your while to get familiar with. I think there is already one scripted-goal in the pljava-packaging POM that could serve as an example.

On the understanding that your earlier-proposed "option 3" (simply writing a dockerfile for the convenience of anyone later building an image) is still what's under discussion, it seems to me a scripted-goal ought to have no trouble doing that.

[beargiles]

I've been working on a separate side project to exercise the generated docker image a bit(*) and discovered that I can simplify the implement. However I also discovered that the flyway scripts only affected the current database. That's not a problem with TestContainers but I discovered problems if I ran docker run ... for a quick sanity check. The solution should be straightforward - the docker iimage will need to modify template1 instead of the default database. Ironically I may need to repeat it for the current connection - I haven't tested that yet.

Re maven plugin - I've missed it FOR YEARS because it doesn't follow the naming conventions. Official maven plugins are named maven-x-plugin and third-party ones are -maven-plugin. I'm sure I'm not the only person who misses anything that doesn't end with -plugin, and of course the -maven- is also important since there can be many different types of plugins. I know you can't remove the existing module but it would probably be a good idea to rename it and add an assembly step to create a second artifact with the old name.

Re goals and scripts - I've been focused on java, not scripts, since I can see potential goals that would be harder to implement via scripts.

What I have today is:

• createServerImage (PostgreSQL artifiacts plus arbitrary content copied from corresponding src/main/docker directory
• createTestImage (same as above but includes the current project's build artifact)
• testImage (launches specified docker image and performs some basic tests)

The first two goals will also call sqlj.install_jar() on everything located in /usr/share/java. Odd location but that's where Debian-based systems install the jars in {package}-java packages. So it can be augmented at build time via maven.

The second should also check the maven module's dependencies (ala mvn dependency:list and add anything not already present in the corresponding docker directory. We'll especially want to ensure anything marked system or runtime is covered.

The goals have a shared implementation - the only difference is a few extra flags for the second goal, and it deliberately creates a 'working directory' at target/docker. The dockerfile itself then pretty straightforward since so much can be handled by copying the entire contents of that working directory into the image - the dockerfile only needs to call sqlj.install_jar() and create a simple script or two. This directory can easily be captured in a maver artifact via the assembly plugin. It could use the same name as the main artifact, but with a docker qualifier and maven type.

(I don't know if that actually exists but the libraries are flexible enough they shouldn't barf on it.)

What I can foresee immediately is:

• listServerImages
• generateCode

The first queries docker hub and lists suitable base images for above. The user doesn't have to use one of these images but it will make it much easier for the user to quickly determine their options. It takes quite a bit of domain knowledge to know where the repos are, how to find specific artifacts, how to list the available versions, and then how to capture all of that into the required docker image name.

I'm sure there's countless ways to do the same thing in a script but to me you want embedded scripts to remain fairly simple. That's also why I prefer using ansible even when a shell script would work - it's just naturally self-organizing into small bits of work. With scripts I often find myself spending more time trying to figure out how the person is doing something well enough for me to make a change than I spend understanding the programs it calls.

(break)

The second may be too much of a stretch but it's worth investigating. There are database -> code generators for all of the major frameworks, and they can include some pretty sophisticated mapping between java classes and database types. For instance jOOQ understands that that postgresql has a native UUID type and while it defaults to a string it's easy to write a bit of code so the java UUID is converted to a database UUID and vice versa.

I've also used this with URLs. At the moment it's just a conversion between the java URL and database string but it's easy to define a 'url' UDT even if it's nothing but a thin wrapper for text. That can improve database reliability, esp. if the UDT actually performs some sanity checks so it's a properly formed URL. (Actually verifying it connects to something is a different story.)

Another very common use is to handle temporal classes. Legacy code has a nasty habit of using different temporal classes but they're all mapped to the same data types in the database. With converters it's easy to have multiple implementations for each temporal type and this eliminates the need to explicit modify the object as it's written to or read from the database. The framework itself will pick the correct implementation.

I haven't used this in JPA but I'm sure it has something similar.

This project doesn't know the user's classes, of course, but if they use a separate maven module for everything that will be pushed to the database it's not hard to see how a plugin could autogen the appropriate conversions and even the corresponding configuration code. I don't know if this will require annotations or if standard java reflection would be enough. There should also be a way to add custom conversions.

One small complication is that this plugin would need to generate a separate code base and artifact. One artifact gets pushed to the server, the other is quietly added to the framework configuration. However I don't think that will be a problem since I've successfully written autogenerated code to src/main/java.jooq. It normally goes to target/generated-sources but putting it into a separate source directory allows me to add it to source control for convenience. In this case the final artifact used both source directories but I could easily the standard build only using java with a plugin only using java.jooq (or whatever.,..)

This may also open the door to something i mentioned earlier - a way to use the same object definition in java and on the server, with either accessor functions or a full UDT. That's far beyond the current work but it's something that I can't see being handled by a script.

[beargiles]

quick note - it looks like the PR may be missing nearly everything - like the PR was pulled from an initial commit. I'll look into that.

(I only noticed since I thought the PR would be the fastest way to find a bit of code in it!)