Building on JH GitLab, Part 1: Java Maven Rules Them All — JIHULAB 101

I’m Xiaobai — the bai of “freeloading”. First, thanks to JH GitLab for this freeloading platform — I mean, SaaS platform — and for the free service it provides; thanks as well to the open source contributors and to every evangelist out there. As a novice, my usage may not follow best practices. Take it as reference only.

This series runs across Part 1: Java Maven Rules Them All, Part 2: Docker Prevails, Part 3: Security Rises Again, and Part 4: APIs, So Enchanting, covering how I use JH GitLab for Java compilation and builds, Docker artifact hosting, security compliance scanning, and a first look at the API.

JH GitLab’s Tianxia Hui

Xiaobai has been wandering the jianghu for years. He heard that the leader of the Tianxia Hui is unmatched in martial arts, his Three-Part Return-to-Origin Qi (DevOps, DevSecOps, GitOps) invincible under heaven. One day he saw that the JH GitLab Tianxia Hui was sending out hero invitations, recruiting Jihu heroes to join the internal testing hall and start training.

Stepping into JH GitLab’s Tianxia Hui was dizzying. Every hall showed off its own specialty.

• Repository Hall holds the Git martial arts manuals: online editing, commit history, branches, tags. Master this discipline and you can walk the jianghu — it’s required training for any hero, so don’t skip it. It’s the prerequisite for the advanced arts, and it’s also the most crowded hall.

• Issues Courtyard is the hall’s main yard, always lively with people coming and going: some post notices to file issues, masters claim them and resolve them. The hall master can review the hall’s activity through Boards, and track progress toward its goals through Milestones — miss the deadline and the leader of Tianxia Hui will have something to say about it.

• Merge Hall gathers masters who spar and discuss techniques; the best moves get merged into the hall’s manual, and from then on your name is written into it.

• CI/CD Arena: however impressive your manual looks, it has to run in the CI/CD Arena to prove it. Pipelines choreograph the sequences you’ll use, and they bear witness to how powerful the manual you wrote really is.

• Security Hall: the heroes here care about whether the manual has holes and weak points. A flaw or weakness appearing is dangerous.

• Package Warehouse is the hall’s artifact repository, where everyone’s manuals are organized and packaged, turning scattered techniques into powerful combinations.

• Analytics Supervisor is the hall’s oversight office; the state of every hall is published there in real time.

• Wiki Commentary is where masters, after reading the hall’s manuals, leave annotations and cautions for newcomers, making sure every rookie learns and uses the manual the right way.

• Snippets Scripture Library collects classic fragments from all kinds of manuals. A novice like me rarely comes here.

• Settings Headquarters governs all the halls. You may not see the entrance, because only hall masters with administrative rights can, and it manages each hall’s affairs — it can even shut a hall down.

Since Xiaobai has wandered the jianghu for years and already has some foundational inner strength, he skipped the earlier halls and headed straight for the CI/CD Arena to spar. That’s where the long-celebrated Three-Part Return-to-Origin Qi (DevOps, DevSecOps, GitOps) is actually used.

The CI/CD Arena

Arriving at the arena — this is the legendary place where the Three-Part Return-to-Origin Qi (DevOps, DevSecOps, GitOps) is unleashed. Everything looked fresh and curious, so let’s visit them one by one.

First stop, Pipelines. The steward asked which discipline I wanted to run, and said he didn’t see me carrying a .gitlab-ci.yml. I was baffled — I had nothing. He said, then learn the rules of the arena first.

First you need a .gitlab-ci.yml file describing which stages your discipline has and which jobs live in each stage. Let me help you write it — what stages are you planning?

Xiaobai thought for a moment. Hmm, my manual needs a specific environment, then it has to be compiled to make sure the syntax is right, then run the unit tests I wrote, and if all’s well it can be packaged!

The steward said, let’s start with four stages: initialize, compile, test, package. Write these first:

stages:
  - initialize
  - compile
  - test
  - package

Then tell me what execution environment each stage needs and which inner-strength commands to run.

Xiaobai said: the initialize stage just needs to download a package from outside and drop it into a specific directory in my manual. I’m most familiar with CentOS, so let’s use that. The download URL and destination are blah blah blah…

The steward said: downloading and extracting needs wget and unzip, so install them first. And whatever you download has to flow to the later stages, so declare artifacts — the following stages will receive it. Then write it like this:

stages:
  - initialize
  - compile
  - test
  - package

initialize:
  stage: initialize
  image: centos:centos7
  script:
    - yum install -y wget unzip
    - wget https://jihulab.com/renfei/ip2location/-/raw/master/IP2LOCATION-LITE-DB11.BIN.ZIP
    - wget https://jihulab.com/renfei/ip2location/-/raw/master/IP2LOCATION-LITE-DB11.IPV6.BIN.ZIP
    - unzip -o IP2LOCATION-LITE-DB11.BIN.ZIP -d renfeid-core/src/main/resources/ip2location/
    - unzip -o IP2LOCATION-LITE-DB11.IPV6.BIN.ZIP -d renfeid-core/src/main/resources/ip2location/
  artifacts:
    expire_in: 1d
    when: on_success
    paths:
      - renfeid-core/src/main/resources/ip2location/

Xiaobai said: oh, I see the pattern now. I can write this myself!

The steward said: nice work, kid. But let me warn you: if your manual is open to the public, then everyone can see your CI/CD Arena, and if you declare artifacts, everyone can download them too. So if you don’t want your artifacts public, go to Settings Headquarters and set the visibility of the CI/CD Arena.

Xiaobai said: got it. Now let me perform on my own.

Next is the compile stage. My skills are Java Maven, so the Maven environment will do. For test I need MySQL and Redis, and I’m still most comfortable with CentOS, so run it there. package can use the Maven environment too.

When the environment needs MySQL and Redis, you can declare additional services with services and set environment variables with variables. So I wrote this:

stages:
  - initialize
  - compile
  - test
  - package

initialize:
  stage: initialize
  image: centos:centos7
  script:
    - yum install -y wget unzip
    - wget https://jihulab.com/renfei/ip2location/-/raw/master/IP2LOCATION-LITE-DB11.BIN.ZIP
    - wget https://jihulab.com/renfei/ip2location/-/raw/master/IP2LOCATION-LITE-DB11.IPV6.BIN.ZIP
    - unzip -o IP2LOCATION-LITE-DB11.BIN.ZIP -d renfeid-core/src/main/resources/ip2location/
    - unzip -o IP2LOCATION-LITE-DB11.IPV6.BIN.ZIP -d renfeid-core/src/main/resources/ip2location/
  artifacts:
    expire_in: 1d
    when: on_success
    paths:
      - renfeid-core/src/main/resources/ip2location/

compile:
  stage: compile
  image: maven:3.8.4-openjdk-8
  script:
    - mvn compile

test:
  stage: test
  image: centos:centos7
  services:
    - name: mariadb:10.6.5
      alias: "mariadb"
    - name: redis:6.0.1
      alias: "redis"
    - name: elasticsearch:7.17.0
      alias: "elasticsearch"
      command: [ "bin/elasticsearch", "-Expack.security.enabled=false", "-Ediscovery.type=single-node" ]
  variables:
    MARIADB_DATABASE: renfeid
    MARIADB_ROOT_PASSWORD: root
  before_script:
    - yum install -y wget java-1.8.0-openjdk* which
    - curl -sS https://downloads.mariadb.com/MariaDB/mariadb_repo_setup | bash
    - rpm --import https://yum.mariadb.org/RPM-GPG-KEY-MariaDB
    - yum install -y MariaDB-server galera-4 MariaDB-client MariaDB-shared MariaDB-backup MariaDB-common
    - mariadb --user=root --password="$MARIADB_ROOT_PASSWORD" --host=mariadb $MARIADB_DATABASE
      < environment/db/renfeid.sql --default-character-set=utf8
  script:
    - ./mvnw test -P gitlab -s environment/maven/settings.xml

Xiaobai said: Steward, how does mine look? Run it, I want to see how powerful my manual really is.

The steward said: your foundational strength is decent, kid, learning this fast. Let’s run it and see. Put this .gitlab-ci.yml in the root directory and commit it.

Running the Manual

The manual was running in Pipelines. Sometimes it failed; retrying worked, but it seemed slow — a long wait each time. The steward said once it runs it’s no longer his concern, go ask the Chief.

Xiaobai found the Chief and complained that the association’s Pipelines were too slow and error-prone. The Chief came over, took a look, and said: Xiaobai, you can cache your Maven dependencies, no need to pull them every single time. Add this config:

cache:
  paths:
    - ".m2/repository/"

That way the dependency jars don’t have to be downloaded over the courier network every run, and everything gets much faster.

Also, Xiaobai, our association was founded domestically, and the environments you depend on are pulled from overseas; the international courier network is busy, so timeouts are unavoidable — caching helps with that too. It’s getting late, I need to head back to headquarters. I’ll tell you about Docker another day.

To be continued — next up: Building on JH GitLab, Part 2: Docker Prevails.