Building a DevOps Pipeline with GitLab CI/CD on AlmaLinux 🚀

GitLab CI/CD represents one of the most comprehensive DevOps platforms available today, offering everything from source control to deployment automation in a single application. This guide walks you through building a production-ready DevOps pipeline on AlmaLinux 9, covering installation, configuration, and implementation of advanced CI/CD workflows.

🎯 Understanding GitLab CI/CD

GitLab CI/CD transforms how teams deliver software by automating the entire development lifecycle. From code commit to production deployment, every step can be automated, tested, and monitored within a unified platform.

Why GitLab for DevOps?

All-in-One Platform - Source control, CI/CD, monitoring, and security 🛠️
Native Kubernetes Integration - Deploy directly to K8s clusters 🐳
Built-in Security Scanning - SAST, DAST, dependency scanning 🔒
Auto DevOps - Pre-configured pipelines for common scenarios 🤖
Extensive API - Integrate with any tool or service 🔌

📋 Prerequisites and Architecture

System Requirements

For production GitLab deployment on AlmaLinux 9:

# Minimum requirements (up to 100 users)
- CPU: 4 cores
- RAM: 8 GB
- Storage: 50 GB SSD

# Recommended for production (500+ users)
- CPU: 8 cores
- RAM: 16 GB
- Storage: 200 GB SSD
- PostgreSQL: Dedicated instance
- Redis: Dedicated instance

Architecture Overview

Developer → Git Push → GitLab → CI/CD Pipeline → Deploy
    ↓          ↓          ↓           ↓             ↓
  Code     Version    Webhook    Build/Test    Production
           Control               Automation

🔧 Installing GitLab on AlmaLinux 9

System Preparation

# Update system
sudo dnf update -y

# Install dependencies
sudo dnf install -y \
  curl \
  policycoreutils \
  openssh-server \
  openssh-clients \
  postfix \
  git

# Enable SSH and Postfix
sudo systemctl enable sshd
sudo systemctl start sshd
sudo systemctl enable postfix
sudo systemctl start postfix

# Configure firewall
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --reload

Install GitLab CE

# Add GitLab repository
curl -sS https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.rpm.sh | sudo bash

# Install GitLab CE
sudo EXTERNAL_URL="https://gitlab.example.com" dnf install -y gitlab-ce

# For HTTP (development only)
sudo EXTERNAL_URL="http://gitlab.example.com" dnf install -y gitlab-ce

Initial Configuration

# Edit GitLab configuration
sudo vim /etc/gitlab/gitlab.rb

# Key configurations to set
external_url 'https://gitlab.example.com'
gitlab_rails['gitlab_email_from'] = '[email protected]'
gitlab_rails['gitlab_email_display_name'] = 'GitLab'
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.gmail.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "[email protected]"
gitlab_rails['smtp_password'] = "your-app-password"
gitlab_rails['smtp_domain'] = "gmail.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_rails['smtp_enable_starttls_auto'] = true

# Reconfigure GitLab
sudo gitlab-ctl reconfigure

Access GitLab

# Get initial root password
sudo cat /etc/gitlab/initial_root_password

# Access GitLab
https://gitlab.example.com

# Login with:
Username: root
Password: [from initial_root_password file]

🏃 Setting Up GitLab Runners

Install GitLab Runner

# Add GitLab Runner repository
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | sudo bash

# Install GitLab Runner
sudo dnf install -y gitlab-runner

# Verify installation
gitlab-runner --version

Register Runner

# Register runner interactively
sudo gitlab-runner register

# Or use one-line registration
sudo gitlab-runner register \
  --non-interactive \
  --url "https://gitlab.example.com/" \
  --registration-token "YOUR_REGISTRATION_TOKEN" \
  --executor "docker" \
  --docker-image alpine:latest \
  --description "docker-runner" \
  --tag-list "docker,aws" \
  --run-untagged="true" \
  --locked="false" \
  --access-level="not_protected"

Configure Docker Executor

# Install Docker
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install -y docker-ce docker-ce-cli containerd.io
sudo systemctl enable docker
sudo systemctl start docker

# Add gitlab-runner to docker group
sudo usermod -aG docker gitlab-runner

# Configure runner
sudo vim /etc/gitlab-runner/config.toml

# /etc/gitlab-runner/config.toml
concurrent = 4
check_interval = 0

[[runners]]
  name = "docker-runner"
  url = "https://gitlab.example.com/"
  token = "YOUR_RUNNER_TOKEN"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "alpine:latest"
    privileged = true
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0
  [runners.cache]
    Type = "s3"
    Shared = true
    [runners.cache.s3]
      ServerAddress = "s3.amazonaws.com"
      AccessKey = "YOUR_ACCESS_KEY"
      SecretKey = "YOUR_SECRET_KEY"
      BucketName = "gitlab-runner-cache"
      BucketLocation = "us-east-1"

📝 Creating Your First Pipeline

Basic .gitlab-ci.yml

# .gitlab-ci.yml
stages:
  - build
  - test
  - deploy

variables:
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

before_script:
  - echo "Starting CI/CD Pipeline"
  - echo "Project: $CI_PROJECT_NAME"
  - echo "Branch: $CI_COMMIT_REF_NAME"

build-job:
  stage: build
  image: node:16-alpine
  script:
    - echo "Building application..."
    - npm install
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 hour
  only:
    - main
    - develop

test-job:
  stage: test
  image: node:16-alpine
  dependencies:
    - build-job
  script:
    - echo "Running tests..."
    - npm test
    - npm run lint
  coverage: '/Coverage: \d+\.\d+%/'
  artifacts:
    reports:
      junit: test-results.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml

deploy-job:
  stage: deploy
  image: alpine:latest
  script:
    - echo "Deploying to production..."
    - apk add --no-cache curl
    - curl -X POST https://deployment-webhook.example.com
  environment:
    name: production
    url: https://app.example.com
  only:
    - main
  when: manual

🐳 Docker Integration

Building Docker Images

# .gitlab-ci.yml for Docker builds
variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "/certs"
  IMAGE_NAME: $CI_REGISTRY_IMAGE
  IMAGE_TAG: $CI_COMMIT_SHORT_SHA

stages:
  - build
  - test
  - push
  - deploy

docker-build:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $IMAGE_NAME:$IMAGE_TAG .
    - docker tag $IMAGE_NAME:$IMAGE_TAG $IMAGE_NAME:latest
  after_script:
    - docker images

docker-test:
  stage: test
  image: docker:latest
  services:
    - docker:dind
  script:
    - docker run --rm $IMAGE_NAME:$IMAGE_TAG npm test
    - docker run --rm $IMAGE_NAME:$IMAGE_TAG npm audit

docker-push:
  stage: push
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker push $IMAGE_NAME:$IMAGE_TAG
    - docker push $IMAGE_NAME:latest
  only:
    - main

Multi-Stage Dockerfile

# Dockerfile
# Build stage
FROM node:16-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build

# Test stage
FROM builder AS tester
RUN npm ci --include=dev
RUN npm test
RUN npm run lint

# Production stage
FROM node:16-alpine
RUN apk add --no-cache dumb-init
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package*.json ./
EXPOSE 3000
USER node
ENTRYPOINT ["dumb-init", "--"]
CMD ["node", "dist/index.js"]

🚀 Advanced Pipeline Features

Parallel Jobs and Matrix Builds

# Parallel testing across multiple versions
test:
  stage: test
  parallel:
    matrix:
      - NODE_VERSION: ["14", "16", "18"]
        OS: ["alpine", "bullseye"]
  image: node:${NODE_VERSION}-${OS}
  script:
    - node --version
    - npm test

# Browser testing matrix
browser-tests:
  stage: test
  parallel:
    matrix:
      - BROWSER: ["chrome", "firefox", "safari"]
  script:
    - npm run test:e2e -- --browser=$BROWSER

Dynamic Environments

# Deploy to dynamic review environments
deploy-review:
  stage: deploy
  script:
    - echo "Deploying to review environment..."
    - helm upgrade --install review-$CI_COMMIT_REF_SLUG ./chart
      --set image.tag=$CI_COMMIT_SHA
      --set ingress.host=$CI_COMMIT_REF_SLUG.review.example.com
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    url: https://$CI_COMMIT_REF_SLUG.review.example.com
    on_stop: stop-review
  only:
    - merge_requests

stop-review:
  stage: deploy
  script:
    - helm delete review-$CI_COMMIT_REF_SLUG
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    action: stop
  when: manual
  only:
    - merge_requests

Pipeline Templates and Includes

# .gitlab-ci.yml
include:
  - project: 'devops/ci-templates'
    ref: main
    file: '/templates/docker.yml'
  - local: '/ci/security.yml'
  - remote: 'https://example.com/ci/quality.yml'
  - template: Security/SAST.gitlab-ci.yml
  - template: Security/Dependency-Scanning.gitlab-ci.yml

stages:
  - build
  - test
  - security
  - deploy

# Extend templates
build-app:
  extends: .docker-build
  variables:
    DOCKERFILE_PATH: "Dockerfile.prod"

🔐 Security Scanning

SAST (Static Application Security Testing)

# Security scanning configuration
include:
  - template: Security/SAST.gitlab-ci.yml
  - template: Security/Secret-Detection.gitlab-ci.yml
  - template: Security/License-Scanning.gitlab-ci.yml

variables:
  SAST_EXCLUDED_PATHS: "vendor/, node_modules/"
  SECRET_DETECTION_EXCLUDED_PATHS: "vendor/, node_modules/"

sast:
  stage: security
  variables:
    SAST_CONFIDENCE_LEVEL: 2
  artifacts:
    reports:
      sast: gl-sast-report.json

Container Scanning

container-scanning:
  stage: security
  image: 
    name: aquasec/trivy:latest
    entrypoint: [""]
  script:
    - trivy image --no-progress --format json --output trivy-report.json $IMAGE_NAME:$IMAGE_TAG
    - trivy image --no-progress --severity HIGH,CRITICAL $IMAGE_NAME:$IMAGE_TAG
  artifacts:
    reports:
      container_scanning: trivy-report.json
  allow_failure: true

Dependency Scanning

dependency-scanning:
  stage: security
  image: node:16
  script:
    - npm audit --json > npm-audit.json || true
    - npm audit --audit-level=high
  artifacts:
    reports:
      dependency_scanning: npm-audit.json

📊 Monitoring and Metrics

Pipeline Metrics

# Collect build metrics
metrics-job:
  stage: .post
  image: alpine:latest
  script:
    - echo "Pipeline duration: $CI_PIPELINE_DURATION seconds"
    - echo "Pipeline status: $CI_PIPELINE_STATUS"
    - |
      curl -X POST https://metrics.example.com/api/v1/pipelines \
        -H "Content-Type: application/json" \
        -d "{
          \"project\": \"$CI_PROJECT_NAME\",
          \"pipeline_id\": \"$CI_PIPELINE_ID\",
          \"duration\": \"$CI_PIPELINE_DURATION\",
          \"status\": \"$CI_PIPELINE_STATUS\",
          \"timestamp\": \"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"
        }"
  when: always

Integration with Prometheus

# Export metrics to Prometheus
prometheus-metrics:
  stage: .post
  image: prom/pushgateway:latest
  script:
    - |
      cat <<EOF | curl --data-binary @- http://prometheus-pushgateway:9091/metrics/job/gitlab-ci/instance/$CI_PROJECT_NAME
      # TYPE ci_pipeline_duration_seconds gauge
      ci_pipeline_duration_seconds{project="$CI_PROJECT_NAME",branch="$CI_COMMIT_REF_NAME"} $CI_PIPELINE_DURATION
      # TYPE ci_pipeline_status gauge
      ci_pipeline_status{project="$CI_PROJECT_NAME",branch="$CI_COMMIT_REF_NAME",status="$CI_PIPELINE_STATUS"} 1
      EOF
  when: always

🔄 Deployment Strategies

Blue-Green Deployment

# Blue-green deployment pipeline
deploy-blue:
  stage: deploy
  script:
    - kubectl set image deployment/app-blue app=$IMAGE_NAME:$IMAGE_TAG
    - kubectl rollout status deployment/app-blue
  environment:
    name: production-blue
    
deploy-green:
  stage: deploy
  script:
    - kubectl set image deployment/app-green app=$IMAGE_NAME:$IMAGE_TAG
    - kubectl rollout status deployment/app-green
  environment:
    name: production-green
    
switch-traffic:
  stage: deploy
  script:
    - kubectl patch service app-service -p '{"spec":{"selector":{"version":"green"}}}'
  when: manual
  environment:
    name: production

Canary Deployment

# Canary deployment with gradual rollout
deploy-canary:
  stage: deploy
  script:
    - |
      # Deploy canary version (10% traffic)
      helm upgrade --install app-canary ./chart \
        --set image.tag=$CI_COMMIT_SHA \
        --set replicaCount=1 \
        --set canary.enabled=true \
        --set canary.weight=10
  environment:
    name: production-canary
    
increase-canary:
  stage: deploy
  script:
    - |
      # Increase canary traffic to 50%
      helm upgrade app-canary ./chart \
        --reuse-values \
        --set canary.weight=50
  when: manual
  
promote-canary:
  stage: deploy
  script:
    - |
      # Promote canary to production
      helm upgrade --install app ./chart \
        --set image.tag=$CI_COMMIT_SHA \
        --set replicaCount=10
      # Remove canary deployment
      helm delete app-canary
  when: manual
  environment:
    name: production

🛠️ Pipeline Optimization

Caching Dependencies

# Efficient caching strategy
variables:
  npm_config_cache: "$CI_PROJECT_DIR/.npm"
  CYPRESS_CACHE_FOLDER: "$CI_PROJECT_DIR/cache/Cypress"

cache:
  key:
    files:
      - package-lock.json
    prefix: ${CI_JOB_NAME}
  paths:
    - .npm
    - cache/Cypress
    - node_modules/

install-dependencies:
  stage: prepare
  script:
    - npm ci --cache .npm --prefer-offline
  artifacts:
    paths:
      - node_modules/
    expire_in: 1 day

Docker Layer Caching

# Use BuildKit for better caching
docker-build-cached:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  variables:
    DOCKER_BUILDKIT: 1
    BUILDKIT_INLINE_CACHE: 1
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - |
      docker build \
        --cache-from $CI_REGISTRY_IMAGE:latest \
        --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA \
        --tag $CI_REGISTRY_IMAGE:latest \
        --build-arg BUILDKIT_INLINE_CACHE=1 \
        .
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

🔧 Troubleshooting Pipelines

Debug Mode

# Enable debug output
debug-job:
  stage: test
  variables:
    CI_DEBUG_TRACE: "true"
  script:
    - env | sort
    - pwd
    - ls -la
    - echo "Debug information collected"
  when: manual

Common Issues and Solutions

# Runner not picking up jobs
sudo gitlab-runner verify
sudo gitlab-runner restart

# Docker permission issues
sudo usermod -aG docker gitlab-runner
sudo systemctl restart docker
sudo systemctl restart gitlab-runner

# Cache not working
# Check cache key and paths
# Ensure artifacts are not overwriting cache

# Pipeline timeout
# Adjust timeout in project settings or job level:
timeout: 3 hours

📈 Best Practices

Pipeline as Code Standards

# Best practices example
workflow:
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_TAG

default:
  image: alpine:latest
  retry:
    max: 2
    when:
      - unknown_failure
      - api_failure
      - runner_system_failure
  interruptible: true

stages:
  - validate
  - build
  - test
  - security
  - deploy
  - monitor

.base-job:
  tags:
    - docker
    - linux
  before_script:
    - echo "Starting job $CI_JOB_NAME"

Security Best Practices

Protect sensitive variables - Use masked and protected variables
Limit runner access - Use tagged runners for production
Review deployment permissions - Implement approval workflows
Scan dependencies regularly - Automate security scanning
Audit pipeline access - Monitor who can modify CI/CD

Performance Optimization

Use specific Docker images - Avoid pulling unnecessary layers
Parallelize tests - Split test suites across jobs
Cache aggressively - Cache dependencies and build artifacts
Optimize images - Use multi-stage builds
Fail fast - Order stages by likelihood of failure

🚀 Advanced GitLab Features

Auto DevOps

# Enable Auto DevOps
# In GitLab UI: Settings → CI/CD → Auto DevOps

# Or via .gitlab-ci.yml
include:
  - template: Auto-DevOps.gitlab-ci.yml

variables:
  POSTGRES_ENABLED: "false"
  PROMETHEUS_ENABLED: "true"
  AUTO_DEVOPS_BUILD_IMAGE_CNB_ENABLED: "true"

GitLab Pages

# Deploy static site to GitLab Pages
pages:
  stage: deploy
  script:
    - npm run build
    - cp -r dist public
  artifacts:
    paths:
      - public
  only:
    - main

Review Apps with Kubernetes

# Deploy review apps to K8s
review:
  stage: deploy
  image: bitnami/kubectl:latest
  script:
    - kubectl create namespace review-$CI_COMMIT_REF_SLUG || true
    - |
      cat <<EOF | kubectl apply -f -
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: app
        namespace: review-$CI_COMMIT_REF_SLUG
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: review-app
        template:
          metadata:
            labels:
              app: review-app
          spec:
            containers:
            - name: app
              image: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
              ports:
              - containerPort: 3000
      EOF
  environment:
    name: review/$CI_COMMIT_REF_SLUG
    url: https://$CI_COMMIT_REF_SLUG.review.example.com
    on_stop: stop_review
  only:
    - merge_requests

📚 Resources and Next Steps

Advanced Topics to Explore

GitLab Kubernetes Agent - Modern cluster integration
Pipeline Triggers - Cross-project pipelines
DAG Pipelines - Complex job dependencies
GitLab Runner Autoscaling - Dynamic runner provisioning
Custom Pipeline Templates - Organization-wide standards

Useful Resources

Building a DevOps pipeline with GitLab CI/CD on AlmaLinux 9 provides a robust foundation for modern software delivery. Start with simple pipelines and gradually add complexity as your team’s needs grow. Remember that the best pipeline is one that your team understands and can maintain. Continuous improvement is key – regularly review and optimize your pipelines based on feedback and metrics. Happy automating! 🚀