💎 Pulp Repository Management on AlmaLinux: Artifact Storage Made Smart

Welcome to the heart of repository management! 🎉 Ready to manage software packages like a pro? Pulp is the powerful platform that makes repository management a breeze! It’s the engine behind many content management systems, including AlmaLinux’s own build system! Think of it as your personal artifact warehouse that can store, organize, and distribute any type of content! 📦✨

🤔 Why is Pulp Important?

Pulp transforms package chaos into organized repositories! 🚀 Here’s why it’s incredible:

• 📦 Multi-Format Support - RPM, Python, Container, Ansible, and more!
• 🔄 Smart Syncing - Mirror repositories efficiently!
• 💾 Artifact Storage - Store packages with deduplication!
• 🌐 Content Distribution - Serve content to thousands!
• 🎯 On-Demand Downloads - Save disk space intelligently!
• 🔌 Plugin Architecture - Extend for any content type!

It’s like having a universal package manager for everything! 🌟

🎯 What You Need

Before building your repository empire, ensure you have:

• ✅ AlmaLinux server (8 or 9)
• ✅ Root or sudo access
• ✅ At least 4GB RAM (8GB recommended)
• ✅ 50GB+ disk space for artifacts
• ✅ PostgreSQL database
• ✅ Redis for caching
• ✅ Love for organized repositories! 💎

📝 Step 1: System Preparation - Setting the Stage!

Let’s prepare AlmaLinux for Pulp! 🏗️

# Update system
sudo dnf update -y

# Install required packages
sudo dnf install -y python3 python3-pip python3-devel
sudo dnf install -y gcc make
sudo dnf install -y postgresql postgresql-server postgresql-contrib
sudo dnf install -y redis nginx

# Initialize PostgreSQL
sudo postgresql-setup --initdb
sudo systemctl enable --now postgresql

# Start Redis
sudo systemctl enable --now redis

# Configure PostgreSQL for Pulp
sudo -u postgres psql << EOF
CREATE USER pulp WITH PASSWORD 'PulpPass123!';
CREATE DATABASE pulp OWNER pulp;
GRANT ALL PRIVILEGES ON DATABASE pulp TO pulp;
EOF

# Edit PostgreSQL config for local connections
sudo sed -i "s/ident/md5/g" /var/lib/pgsql/data/pg_hba.conf
sudo systemctl restart postgresql

Configure storage for artifacts:

# Create Pulp directories
sudo mkdir -p /var/lib/pulp/media
sudo mkdir -p /var/lib/pulp/assets
sudo mkdir -p /var/lib/pulp/tmp

# Create pulp user
sudo useradd --system --shell /bin/bash \
  --home-dir /var/lib/pulp \
  --comment "Pulp Service Account" pulp

# Set ownership
sudo chown -R pulp:pulp /var/lib/pulp

# Configure SELinux contexts (if enabled)
sudo semanage fcontext -a -t httpd_sys_content_t "/var/lib/pulp/media(/.*)?"
sudo restorecon -R /var/lib/pulp/

Perfect! System is ready! 🎯

🔧 Step 2: Installing Pulp - Your Repository Engine!

Time to install Pulp 3! 🚀

# Install Pulp installer
sudo pip3 install pulp-installer

# Create ansible inventory
cat << EOF | sudo tee /etc/pulp/inventory
[all:vars]
pulp_default_admin_password=Admin123!
pulp_settings_databases__default__HOST=localhost
pulp_settings_databases__default__NAME=pulp
pulp_settings_databases__default__USER=pulp
pulp_settings_databases__default__PASSWORD=PulpPass123!
pulp_settings_redis_host=localhost
pulp_settings_redis_port=6379
pulp_content_bind=0.0.0.0:24816
pulp_api_bind=0.0.0.0:24817

[pulp]
localhost ansible_connection=local
EOF

# Run Pulp installer
sudo ansible-playbook -i /etc/pulp/inventory \
  /usr/local/share/pulp_installer/playbooks/install.yml

Manual Installation Alternative:

# Create virtual environment
sudo python3 -m venv /usr/local/lib/pulp

# Activate and install Pulp
source /usr/local/lib/pulp/bin/activate
pip install pulpcore pulp-rpm pulp-container pulp-ansible

# Create settings file
sudo mkdir -p /etc/pulp
cat << EOF | sudo tee /etc/pulp/settings.py
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': 'pulp',
        'USER': 'pulp',
        'PASSWORD': 'PulpPass123!',
        'HOST': 'localhost',
        'PORT': '5432',
    }
}

MEDIA_ROOT = '/var/lib/pulp/media/'
WORKING_DIRECTORY = '/var/lib/pulp/tmp/'
CONTENT_ORIGIN = 'http://$(hostname -f)'
ANSIBLE_API_HOSTNAME = 'http://$(hostname -f)'
ANSIBLE_CONTENT_HOSTNAME = 'http://$(hostname -f)/pulp/content'

REDIS_HOST = 'localhost'
REDIS_PORT = 6379

SECRET_KEY = '$(openssl rand -hex 32)'
EOF

# Run migrations
sudo -u pulp /usr/local/lib/pulp/bin/pulpcore-manager migrate

# Create admin user
sudo -u pulp /usr/local/lib/pulp/bin/pulpcore-manager createsuperuser \
  --username admin --email [email protected]

🌟 Step 3: Configuring Pulp Services - Starting Your Engine!

Let’s set up Pulp services! 🎮

Create systemd services:

# Pulp API service
cat << EOF | sudo tee /etc/systemd/system/pulpcore-api.service
[Unit]
Description=Pulp API Server
After=network-online.target postgresql.service redis.service

[Service]
Type=simple
User=pulp
Group=pulp
WorkingDirectory=/var/lib/pulp
Environment="PULP_SETTINGS=/etc/pulp/settings.py"
ExecStart=/usr/local/lib/pulp/bin/gunicorn pulpcore.app.wsgi:application \
  --bind 0.0.0.0:24817 \
  --workers 4 \
  --access-logfile -

[Install]
WantedBy=multi-user.target
EOF

# Pulp Content service
cat << EOF | sudo tee /etc/systemd/system/pulpcore-content.service
[Unit]
Description=Pulp Content Server
After=network-online.target postgresql.service redis.service

[Service]
Type=simple
User=pulp
Group=pulp
WorkingDirectory=/var/lib/pulp
Environment="PULP_SETTINGS=/etc/pulp/settings.py"
ExecStart=/usr/local/lib/pulp/bin/gunicorn pulpcore.content:server \
  --bind 0.0.0.0:24816 \
  --workers 4 \
  --access-logfile -

[Install]
WantedBy=multi-user.target
EOF

# Pulp Worker service
cat << EOF | sudo tee /etc/systemd/system/[email protected]
[Unit]
Description=Pulp Worker %i
After=network-online.target postgresql.service redis.service

[Service]
Type=simple
User=pulp
Group=pulp
WorkingDirectory=/var/lib/pulp
Environment="PULP_SETTINGS=/etc/pulp/settings.py"
ExecStart=/usr/local/lib/pulp/bin/pulpcore-worker

[Install]
WantedBy=multi-user.target
EOF

Start services:

# Reload systemd
sudo systemctl daemon-reload

# Enable and start services
sudo systemctl enable --now pulpcore-api
sudo systemctl enable --now pulpcore-content
sudo systemctl enable --now pulpcore-worker@1
sudo systemctl enable --now pulpcore-worker@2

# Check status
sudo systemctl status pulpcore-api
sudo systemctl status pulpcore-content

Configure firewall:

# Open Pulp ports
sudo firewall-cmd --permanent --add-port=24816/tcp  # Content
sudo firewall-cmd --permanent --add-port=24817/tcp  # API
sudo firewall-cmd --reload

✅ Step 4: Creating Your First Repository - Let’s Store Content!

Time to create and sync repositories! 📦

Install Pulp CLI:

# Install pulp-cli
pip install pulp-cli[pygments]

# Configure CLI
pulp config create \
  --base-url http://localhost:24817 \
  --username admin \
  --password Admin123!

# Test connection
pulp status
# Should show service status

Create RPM Repository:

# Create remote (upstream source)
pulp rpm remote create \
  --name almalinux-baseos \
  --url https://repo.almalinux.org/almalinux/9/BaseOS/x86_64/os/ \
  --policy on_demand

# Create repository
pulp rpm repository create \
  --name almalinux-9-base \
  --description "AlmaLinux 9 BaseOS Repository"

# Sync repository
pulp rpm repository sync \
  --name almalinux-9-base \
  --remote almalinux-baseos

# Check sync status
pulp task list --limit 1

Create Distribution:

# Create publication
pulp rpm publication create \
  --repository almalinux-9-base

# Create distribution to serve content
pulp rpm distribution create \
  --name almalinux-9-base \
  --base-path almalinux/9/base \
  --repository almalinux-9-base

# Content now available at:
# http://your-server:24816/pulp/content/almalinux/9/base/

🌟 Step 5: Advanced Repository Management - Power Features!

Let’s explore advanced Pulp features! 🎯

Upload Custom Packages:

# Upload RPM to repository
pulp rpm content upload \
  --repository almalinux-9-base \
  --file /path/to/custom-package.rpm

# Create new version with uploaded content
pulp rpm repository version create \
  --repository almalinux-9-base

Content Filtering:

# Sync only specific packages
pulp rpm repository sync \
  --name almalinux-9-base \
  --remote almalinux-baseos \
  --include-packages "kernel,glibc,systemd"

# Exclude packages
pulp rpm repository sync \
  --name almalinux-9-base \
  --remote almalinux-baseos \
  --exclude-packages "*-devel,*-debug"

Mirror Management:

# Create mirror with immediate download
pulp rpm remote create \
  --name epel-mirror \
  --url https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/ \
  --policy immediate

# Sync completely
pulp rpm repository sync \
  --name epel-repo \
  --remote epel-mirror \
  --mirror true

🎮 Quick Examples

Example 1: Python Package Repository

# Create Python repository
pulp python remote create \
  --name pypi \
  --url https://pypi.org/ \
  --includes "django,flask,requests"

pulp python repository create \
  --name python-packages

pulp python repository sync \
  --name python-packages \
  --remote pypi

# Serve Python packages
pulp python distribution create \
  --name python \
  --base-path python \
  --repository python-packages

Example 2: Container Registry

# Create container repository
pulp container remote create \
  --name docker-hub \
  --url https://registry-1.docker.io \
  --upstream-name library/nginx

pulp container repository create \
  --name nginx-images

pulp container repository sync \
  --name nginx-images \
  --remote docker-hub

# Create distribution for containers
pulp container distribution create \
  --name containers \
  --base-path containers \
  --repository nginx-images

Example 3: Ansible Collections

# Create Ansible repository
pulp ansible remote create \
  --name galaxy \
  --url https://galaxy.ansible.com/api/ \
  --requirements "collections:
  - community.general
  - ansible.posix"

pulp ansible repository create \
  --name ansible-collections

pulp ansible repository sync \
  --name ansible-collections \
  --remote galaxy

🚨 Fix Common Problems

Problem 1: Sync Fails

Symptom: Repository sync errors out 😰

Fix:

# Check worker status
sudo systemctl status pulpcore-worker@*

# Check disk space
df -h /var/lib/pulp
# Need sufficient space!

# View task details
pulp task show --href /pulp/api/v3/tasks/[task-id]/

# Retry with verbose output
pulp -v rpm repository sync \
  --name almalinux-9-base \
  --remote almalinux-baseos

# Check logs
sudo journalctl -u pulpcore-worker@1 -f

Problem 2: Cannot Access Content

Symptom: 404 errors when accessing repositories 🌐

Fix:

# Check content service
sudo systemctl status pulpcore-content

# Verify distribution
pulp rpm distribution list

# Check publication
pulp rpm publication list \
  --repository almalinux-9-base

# Recreate distribution
pulp rpm distribution destroy \
  --name almalinux-9-base

pulp rpm distribution create \
  --name almalinux-9-base \
  --base-path almalinux/9/base \
  --repository almalinux-9-base

Problem 3: Database Connection Issues

Symptom: Pulp services won’t start 🔌

Fix:

# Check PostgreSQL
sudo systemctl status postgresql

# Test database connection
sudo -u pulp psql -h localhost -U pulp -d pulp -c "SELECT 1;"

# Check Redis
redis-cli ping
# Should return PONG

# Verify settings
sudo -u pulp /usr/local/lib/pulp/bin/pulpcore-manager shell
>>> from django.db import connection
>>> connection.ensure_connection()

📋 Simple Commands Summary

💡 Tips for Success

🚀 Performance Optimization

Make Pulp blazing fast:

# Increase workers for large deployments
for i in {3..8}; do
  sudo systemctl enable --now pulpcore-worker@$i
done

# Optimize PostgreSQL
echo "shared_buffers = 1GB" | sudo tee -a /var/lib/pgsql/data/postgresql.conf
echo "effective_cache_size = 3GB" | sudo tee -a /var/lib/pgsql/data/postgresql.conf
sudo systemctl restart postgresql

# Use Redis caching effectively
echo "maxmemory 2gb" | sudo tee -a /etc/redis.conf
echo "maxmemory-policy allkeys-lru" | sudo tee -a /etc/redis.conf
sudo systemctl restart redis

# Use on_demand policy for large repos
# Saves disk space and speeds initial sync

🔒 Security Hardening

Keep Pulp secure:

1. Use HTTPS - Always encrypt API traffic! 🔐
2. Strong passwords - Complex credentials! 💪
3. API tokens - Use instead of passwords! 🔑
4. Restrict access - Firewall rules! 🛡️
5. Regular updates - Keep Pulp current! 🔄

# Generate API token
pulp user token create --username admin

# Use token for authentication
export PULP_API_TOKEN="your-token-here"
pulp status

📊 Storage Strategies

Optimize storage usage:

# Use deduplication
# Pulp automatically deduplicates artifacts

# Clean orphaned content regularly
pulp orphan cleanup --protection-time 7

# Monitor storage usage
du -sh /var/lib/pulp/media/artifact/

# Use on_demand for development
# Use immediate for production mirrors

🏆 What You Learned

You’re now a Pulp repository expert! 🎓 You’ve successfully:

• ✅ Installed Pulp on AlmaLinux
• ✅ Configured services and database
• ✅ Created RPM repositories
• ✅ Synced upstream content
• ✅ Set up content distribution
• ✅ Managed artifacts and packages
• ✅ Mastered repository management

Your repository platform is production-ready! 💎

🎯 Why This Matters

Pulp transforms repository management completely! With your artifact platform, you can:

• 📦 Manage everything - Any content type supported!
• 🔄 Mirror efficiently - Smart syncing saves bandwidth!
• 💾 Store intelligently - Deduplication saves space!
• 🌐 Scale massively - Serve thousands of clients!
• 🎯 Control precisely - Filter and version everything!

You’re not just storing packages - you’re building a complete content distribution network! Every artifact is managed, every repository is optimized! 💎

Keep syncing, keep serving, and remember - with Pulp, repository management is powerful yet simple! ⭐

May your repositories sync quickly and your artifacts be always available! 🚀💎🙌