Unexpected score corruption when adding (but not using) a second declarative shadow variable

[makaimann]

Describe the bug
I'm trying out the latest (extremely cool!) 1.24.0 release features, namely mixing list+basic variables and declarative shadow variables, and hit a strange edge case when adding more than one declarative shadow variable.

This occurred in a simple Job Shop Scheduling encoding where machines have a list of assigned jobs, and jobs have a long delay basic variable that allows them to wait to execute. I could be doing something wrong, but the behavior is very surprising to me.

Expected behavior
I don't expect a score corruption to occur when a declarative shadow variable is added if it is not used.

Actual behavior
When adding (a second) declarative shadow variable without changing the constraint provider at all, I start getting a score corruption when running with TRACKED_FULL_ASSERT.

To Reproduce
I've attached a minimal reproducing example in jssp-repro.zip.

Observe the same behavior by extracting the contents and doing the following:

# Set Up
cd jssp-repro
mvn compile

# Run the broken version and see the score corruption
mvn exec:java

Now open Job.java and comment out the endTime shadow variable at lines 30-31, plus the updateEndTime and getEndTime functions that use it:

    // @ShadowVariable(supplierName = "updateEndTime")
    // public Long endTime;
...
    // @ShadowSources({...})
    // public Long updateEndTime() {
    // ...
    // }

    // public long getEndTime() {

Then rerun with mvn exec:java and see that there's no longer a score corruption.

Environment

Timefold Solver Version or Git ref: 1.24.0

Output of java -version:
Updated: Ran mvn -V on my machine to give:

Java version: 23.0.2, vendor: Homebrew, runtime: /usr/local/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home

Output of uname -a or ver:

Darwin MakMac 24.4.0 Darwin Kernel Version 24.4.0: Fri Apr 11 18:28:23 PDT 2025; root:xnu-11417.101.15~117/RELEASE_X86_64 x86_64

Reproduced on Mac but originally observed on Ubuntu Linux.

Additional information

Let me know if you have any trouble reproducing and thanks for all the great work on Timefold!

[triceo]

Hello @makaimann!

Thanks for reporting - we'll take a look when time permits. Before we do though, are you sure about your Java version?

openjdk version "14.0.1" 2020-04-14
OpenJDK Runtime Environment (build 14.0.1+7)
OpenJDK 64-Bit Server VM (build 14.0.1+7, mixed mode, sharing)

OpenJDK 14 is a very old Java version, and Timefold Solver is compiled to run on 17+. You shouldn't've been able to run the solver on this version at all.

[makaimann]

Awesome, thanks @triceo! Regarding java version, you are absolutely right, mvn -V gives:

Java version: 23.0.2, vendor: Homebrew, runtime: /usr/local/Cellar/openjdk/23.0.2/libexec/openjdk.jdk/Contents/Home

I'll edit the description.

[Christopher-Chianelli]

Greetings @makaimann !

I confirmed the issue and created a PR to fix it: #1697

In summary, when you added the second shadow variable, you changed the graph structure. But the wrong graph structure was used, since "prerequisiteJob.assignedMachine" was incorrectly identified as a NO_PARENT access (accessed directly from the object) instead of an INDIRECT access (accessed from a fact on the object). This caused the graph structure to be SINGLE_DIRECTIONAL_PARENT (which can only be used when there are no indirect accesses) instead of ARBITRARY_SINGLE_ENTITY_AT_MOST_ONE_DIRECTIONAL_PARENT_TYPE.

That being said, you will now get a variable corruption since the initial score calculation for "endTime" changed it from null to a non-null value. I recommend changing "endTime" definition to this:

    @ShadowSources({ "assignedMachine", "delay", "previousJob.endTime" })
    public Long updateEndTime() {
        if (assignedMachine == null) {
            return null;
        }
        var newEndTime = this.getDelay() + this.duration;
        if (this.previousJob != null) {
            newEndTime += this.previousJob.getEndTime();
        }
        return newEndTime;
    }

which will pass TRACKED_FULL_ASSERT once this fix is merged.

[makaimann]

Awesome, thanks so much @Christopher-Chianelli! Thanks for the explanation -- sounds like a tricky bug.

For my understanding, how should I think about the remaining score corruption that you suggested a fix for? Is the issue that endTime is assigned a non-null value before initialization is done? And that's why checking assignedMachine first is the solution?

[Christopher-Chianelli]

Awesome, thanks so much @Christopher-Chianelli! Thanks for the explanation -- sounds like a tricky bug.

For my understanding, how should I think about the remaining score corruption that you suggested a fix for? Is the issue that endTime is assigned a non-null value before initialization is done? And that's why checking assignedMachine first is the solution?

It not a ScoreCorruption; it a VariableCorruption. A VariableCorruption means a shadow variable changed when none of its sources change.

The issue is the input given to the solver must have all shadow variables in a valid state ("valid" meaning if were to force update all the variables, none of them should change). For instance, if you have an @InverseRelationShadowVariable, and have some values assigned, those values' @InverseRelationShadowVariable must point to the entity assigned to those values. So there are ultimately two different solutions:

1. Assign endTime to 0 (the value that would be computed for it) before giving it to the Solver.
2. Assign endTime to null when the value is unassigned.

(1) is confusing; why should something that is unassigned have an endTime of 0? Hence I recommend (2).

Until the bug fix is released, you can use this workaround to continue working:

public class Job {
    // ...
    public final List<Job> prerequisiteJobs;
    // ...

    @ShadowSources({ "assignedMachine", "prerequisiteJobs[].assignedMachine" })
    public Long updateTravelDistance() {
        if (this.prerequisiteJobs == null || this.assignedMachine == null
                || this.prerequisiteJobs.get(0).assignedMachine == null) {
            return null;
        }

        return this.prerequisiteJobs.get(0).assignedMachine.distance(this.assignedMachine);
    }
    // ...
}

prerequisiteJobs is a List that is either null or has exactly one element. This tricks the Solver into using the correct GraphStructure.

[makaimann]

Ahh this makes a lot of sense -- thanks for the rundown on variable corruptions and the workaround. Much appreciated!