Skip to content

Customizable JDK and Maven Versions for CodeQL Agent #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Dec 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ RUN apt-get update && \
file \
dos2unix \
default-jdk \
openjdk-8-jdk \
maven \
gettext && \
apt-get clean && \
Expand Down
12 changes: 12 additions & 0 deletions Dockerfile-java
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# FROM --platform=amd64 maven:3.8-openjdk-17-slim
FROM --platform=amd64 maven:3-jdk-8-slim

RUN ls -lia $JAVA_HOME

RUN mkdir -p /opt/jdk/ /opt/maven/

RUN cp -r $JAVA_HOME/* /opt/jdk/

RUN cp -r $MAVEN_HOME/* /opt/maven/

CMD ["echo"]
42 changes: 32 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ You can set environment variables to use the following supported options:
`SAVE_CACHE_FLAG` | Value `--save-cache`. Aggressively save intermediate results to the disk cache. This may speed up subsequent queries if they are similar. Be aware that using this option will greatly increase disk usage and initial evaluation time.
`ACTION` | Value `create-database-only`. Creating CodeQL database only without executing CodeQL analysis.
`COMMAND` | Value `<command>`. The variable used when you create a CodeQL database for one or more compiled languages, omit if the only languages requested are Python and JavaScript. This specifies the build commands needed to invoke the compiler. If you don't set this variable, CodeQL will attempt to detect the build system automatically, using a built-in autobuilder.
`JAVA_VERSION` | Value `<java_version>`. Set the Java version. The default Java version is Java 11. It must be `8` or `11`.
-----

***Disclaimer:** CodeQL Agent directly forwards these options to the command arguments while running the container. Please take it as your security responsibility.*
Expand Down Expand Up @@ -128,16 +127,39 @@ docker run --rm --name codeql-agent-docker \
<details>
<summary> Specify the Java version and the build database command </summary>

```bash
docker run --rm --name codeql-agent-docker \
-v "$PWD:/opt/src" \
-v "$PWD/codeql-agent-results:/opt/results" \
-e "LANGUAGE=java" \
-e "JAVA_VERSION=8" \
-e "COMMAND=mvn clean install" \
doublevkay/codeql-agent
By default, we use JDK 11 and Maven 3.6.3 for the CodeQL agent image. We can change the versions of Java and Maven by mounting a volume and setting the JAVA_HOME and MAVEN_HOME environment variables in the CodeQL agent container. For example:

```
1. Create a Dockerfile (named Dockerfile-java) for the specific versions of Java and Maven, and place it in the directory that will be used for mounting later:
```Dockerfile
FROM --platform=amd64 maven:3-jdk-8-slim

RUN mkdir -p /opt/jdk/ /opt/maven/

RUN cp -r $JAVA_HOME/* /opt/jdk/

RUN cp -r $MAVEN_HOME/* /opt/maven/

CMD ["echo"]
```
2. Build and run the Docker container, mounting the JDK and Maven directories to the respective volumes:
```bash
docker buildx build -t codeql-java -f Dockerfile-java .
docker run --rm -v "jdkvol:/opt/jdk" -v "mavenvol:/opt/maven" codeql-java
```
3. Finally, run codeql-agent container with mounted volumes and set env variable JAVA_HOME, MAVEN_HOME to the mounted volumes

```bash
docker run --rm --name codeql-agent-docker \
-v "$PWD:/opt/src" \
-v "$PWD/codeql-agent-results:/opt/results" \
-v "jdkvol:/opt/jdk" \
-v "mavenvol:/opt/maven" \
-e "LANGUAGE=java" \
-e "JAVA_HOME=/opt/jdk" \
-e "MAVEN_HOME=/opt/maven" \
-e "COMMAND=mvn clean install" \
doublevkay/codeql-agent
```
</details>

## Build
Expand Down
119 changes: 59 additions & 60 deletions scripts/analyze.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,32 @@ print_red() {
# Set SRC
SRC=/opt/src

# Check if JAVA_HOME is set and not empty
if [ -n "$JAVA_HOME" ]; then
echo "JAVA_HOME is set to $JAVA_HOME"
# Check and add JAVA_HOME/jre/bin to PATH if it exists
if [ -d "$JAVA_HOME/jre/bin" ]; then
export PATH="$JAVA_HOME/jre/bin:$PATH"
fi
# Check and add JAVA_HOME/bin to PATH if it exists
if [ -d "$JAVA_HOME/bin" ]; then
export PATH="$JAVA_HOME/bin:$PATH"
fi
else
echo "JAVA_HOME is not set or empty. Use default."
fi

# Check if MAVEN_HOME is set and not empty
if [ -n "$MAVEN_HOME" ]; then
echo "MAVEN_HOME is set to $MAVEN_HOME"
# Check and add MAVEN_HOME/bin to PATH if it exists
if [ -d "$MAVEN_HOME/bin" ]; then
export PATH="$MAVEN_HOME/bin:$PATH"
fi
else
echo "MAVEN_HOME is not set or empty. Use default."
fi

if [[ -z "${CI_PROJECT_DIR}" ]]; then
SRC=/opt/src
else
Expand All @@ -29,71 +55,60 @@ if [ ! -d "$SRC" ]; then
exit 3
fi

if [ -z $LANGUAGE ]
then
if [ ! -z $CI_PROJECT_REPOSITORY_LANGUAGES ]
then
ListLanguages=(${CI_PROJECT_REPOSITORY_LANGUAGES//,/ })
else
chown -R $(id -u):$(id -g) $SRC
mapfile -t ListLanguages <<< $(github-linguist $SRC)
fi
for val in "${ListLanguages[@]}"; do
lang="$(echo $val | rev | cut -d' ' -f 1 | rev)"
lang=${lang,,}
if [[ "${SupportedLanguage[*]}" =~ "${lang}" ]]; then
LANGUAGE=$lang
break
fi
done
if [[ $LANGUAGE == "" ]]; then
print_red "[!] Can not auto detect language. Please check the source code or specify the LANGUAGE variable."
finalize
exit 4
if [ -z $LANGUAGE ]; then
if [ ! -z $CI_PROJECT_REPOSITORY_LANGUAGES ]; then
ListLanguages=(${CI_PROJECT_REPOSITORY_LANGUAGES//,/ })
else
chown -R $(id -u):$(id -g) $SRC
mapfile -t ListLanguages <<<$(github-linguist $SRC)
fi
for val in "${ListLanguages[@]}"; do
lang="$(echo $val | rev | cut -d' ' -f 1 | rev)"
lang=${lang,,}
if [[ "${SupportedLanguage[*]}" =~ "${lang}" ]]; then
LANGUAGE=$lang
break
fi
done
if [[ $LANGUAGE == "" ]]; then
print_red "[!] Can not auto detect language. Please check the source code or specify the LANGUAGE variable."
finalize
exit 4
fi
fi

# Set options
LANGUAGE=${LANGUAGE,,}
if [[ "$LANGUAGE" == "python" || "$LANGUAGE" == "javascript" || "$LANGUAGE" == "cpp" || "$LANGUAGE" == "csharp" || "$LANGUAGE" == "java" || "$LANGUAGE" == "go" || "$LANGUAGE" == "typescript" || "$LANGUAGE" == "c" ]]
then
if [[ "$LANGUAGE" == "typescript" ]]
then
if [[ "$LANGUAGE" == "python" || "$LANGUAGE" == "javascript" || "$LANGUAGE" == "cpp" || "$LANGUAGE" == "csharp" || "$LANGUAGE" == "java" || "$LANGUAGE" == "go" || "$LANGUAGE" == "typescript" || "$LANGUAGE" == "c" ]]; then
if [[ "$LANGUAGE" == "typescript" ]]; then
LANGUAGE="javascript"
fi
if [[ "$LANGUAGE" == "c" ]]
then
if [[ "$LANGUAGE" == "c" ]]; then
LANGUAGE="cpp"
fi

else
echo "[!] Invalid language: $LANGUAGE"
finalize
exit 5
echo "[!] Invalid language: $LANGUAGE"
finalize
exit 5
fi

if [ -z $FORMAT ]
then
if [ -z $FORMAT ]; then
FORMAT="sarif-latest"
fi

if [ -z $QS ]
then
if [ -z $QS ]; then
QS="$LANGUAGE-security-extended.qls"
fi

if [ -z $OUTPUT ]
then
if [ -z $OUTPUT ]; then
OUTPUT="/opt/results"
fi

if [ -z $THREADS ]
then
if [ -z $THREADS ]; then
THREADS="0"
fi



DB="$OUTPUT/codeql-db"

# Set THREADS
Expand All @@ -108,28 +123,14 @@ print_green " [+] Output: $OUTPUT"
print_green " [+] Format: $FORMAT"
echo "----------------"

# Switch to Java 8
if [[ $JAVA_VERSION ]]
then
if [[ $JAVA_VERSION == "8" ]]; then
update-java-alternatives -s $(update-java-alternatives -l | grep 8 | cut -d " " -f1) || echo '.'
elif [[ $JAVA_VERSION == "11" ]]; then
update-java-alternatives -s $(update-java-alternatives -l | grep 11 | cut -d " " -f1) || echo '.'
else
echo "[Warning] : JAVA_VERSION must be 8 or 11."
fi
fi

# Check action
if [ -z $ACTION ]
then
if [ -z $ACTION ]; then
ACTION='all'
fi

# Functions
create_database() {
if [[ $COMMAND ]]
then
if [[ $COMMAND ]]; then
print_green "[Running] Creating DB: codeql database create --threads=$THREADS --language=$LANGUAGE --command=\"$COMMAND\" $DB -s $SRC $OVERWRITE_FLAG"
codeql database create --threads=$THREADS --language=$LANGUAGE --command="$COMMAND" $DB -s $SRC $OVERWRITE_FLAG
else
Expand All @@ -145,7 +146,7 @@ create_database() {

scan() {
print_green "[Running] Start Scanning: codeql database analyze --format=$FORMAT --threads=$THREADS $SAVE_CACHE_FLAG --output=$OUTPUT/issues.$FORMAT $DB $QS"
codeql database analyze --format=$FORMAT --threads=$THREADS $SAVE_CACHE_FLAG --output=$OUTPUT/issues.$FORMAT $DB $QS
codeql database analyze --off-heap-ram=0 --format=$FORMAT --threads=$THREADS $SAVE_CACHE_FLAG --output=$OUTPUT/issues.$FORMAT $DB $QS
if [ $? -ne 0 ]; then
print_red "[!] CodeQL analyze failed."
finalize
Expand All @@ -163,8 +164,7 @@ convert_sarif_to_sast() {
}

finalize() {
if [[ $USERID && $GROUPID ]]
then
if [[ $USERID && $GROUPID ]]; then
chown -R $USERID:$GROUPID $OUTPUT
chown -R $USERID:$GROUPID $SRC
fi
Expand All @@ -184,4 +184,3 @@ main() {

# Main
main