๐ Elasticsearch Complete Search Engine Guide on AlmaLinux
Ready to supercharge your search capabilities? โก Elasticsearch is the worldโs most powerful search engine, running searches for Netflix, GitHub, and Stack Overflow! In this comprehensive guide, weโll install Elasticsearch on AlmaLinux and build lightning-fast search solutions. Letโs master the search revolution! ๐
๐ค Why is Elasticsearch Important?
Elasticsearch is the search engine powering the modern web! ๐ Hereโs why itโs essential:
- ๐ Search Leader: Powers 50% of all enterprise search
- ๐ฐ High Demand: Elasticsearch engineers earn $140k+ annually
- โก Lightning Fast: Millisecond search across billions of documents
- ๐ Real-time Analytics: Process data as it arrives
- ๐ Scalable: Handle petabytes across thousands of nodes
- ๐ฏ Full-text Search: Advanced relevance scoring
- ๐ Big Data: Essential for data science and analytics
- ๐ง Developer Friendly: RESTful API and JSON documents
Companies like Uber process 40 billion searches daily with Elasticsearch! ๐
๐ฏ What You Need
Letโs prepare for Elasticsearch mastery! โ
- โ AlmaLinux 8 or 9 (minimum 2GB RAM)
- โ At least 4GB RAM (8GB+ recommended for production)
- โ 20GB free disk space
- โ Java 11 or higher (weโll install)
- โ Root or sudo access
- โ Network connectivity
- โ 45 minutes for complete setup
- โ Passion for fast search! ๐ฅ
Letโs build your search engine! ๐
๐ Step 1: Install Java and Prepare System
Elasticsearch needs Java to run! โ
# Update system packages
sudo dnf update -y
# Install OpenJDK 11
sudo dnf install -y java-11-openjdk java-11-openjdk-devel
# Verify Java installation
java -version
javac -version
# Set JAVA_HOME environment variable
echo 'export JAVA_HOME=/usr/lib/jvm/java-11-openjdk' >> ~/.bashrc
echo 'export PATH=$JAVA_HOME/bin:$PATH' >> ~/.bashrc
source ~/.bashrc
# Verify JAVA_HOME
echo $JAVA_HOME
# Install additional tools
sudo dnf install -y wget curl vim net-tools
Expected output:
openjdk version "11.0.20" 2023-07-18 LTS
OpenJDK Runtime Environment (Red_Hat-11.0.20.0.8-1.el9_2) (build 11.0.20+8-LTS)
Perfect! โ Java is ready for Elasticsearch!
๐ง Step 2: Add Elasticsearch Repository and Install
Time to install Elasticsearch! ๐
# Import Elasticsearch GPG key
sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
# Create Elasticsearch repository
sudo cat <<EOF > /etc/yum.repos.d/elasticsearch.repo
[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md
EOF
# Install Elasticsearch
sudo dnf --enablerepo=elasticsearch install elasticsearch -y
# Verify installation
elasticsearch --version
# Check installed package
rpm -qi elasticsearch
Example output:
Version: 8.11.0, Build: default/rpm/d9ec3fa628c7b0ba3d25692e277ba26814820b20/2023-11-04T10:04:57.184859352Z
Amazing! ๐ Elasticsearch is installed!
๐ Step 3: Configure Elasticsearch for Production
Letโs optimize Elasticsearch configuration! โ๏ธ
# Backup original config
sudo cp /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.backup
# Create optimized configuration
sudo cat <<EOF > /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
# Cluster name (important for production)
cluster.name: my-application
# Node name (unique for each node)
node.name: node-1
# Path settings
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
# Network settings
network.host: 0.0.0.0
http.port: 9200
# Discovery settings (single node for now)
discovery.type: single-node
# Security settings (disable for initial setup)
xpack.security.enabled: false
xpack.security.enrollment.enabled: false
xpack.security.http.ssl:
enabled: false
keystore.path: certs/http.p12
xpack.security.transport.ssl:
enabled: false
verification_mode: certificate
keystore.path: certs/transport.p12
truststore.path: certs/transport.p12
# Memory settings
bootstrap.memory_lock: true
# HTTP settings
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: "X-Requested-With,Content-Type,Content-Length,Authorization"
EOF
# Configure JVM heap size (50% of available RAM, max 32GB)
sudo nano /etc/elasticsearch/jvm.options.d/heap.options
Add to heap.options:
-Xms2g
-Xms2g
# Set proper permissions
sudo chown -R elasticsearch:elasticsearch /etc/elasticsearch/
sudo chmod 660 /etc/elasticsearch/elasticsearch.yml
# Configure system limits
sudo cat <<EOF >> /etc/security/limits.conf
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
elasticsearch soft nofile 65536
elasticsearch hard nofile 65536
EOF
# Configure systemd for memory locking
sudo mkdir -p /etc/systemd/system/elasticsearch.service.d
sudo cat <<EOF > /etc/systemd/system/elasticsearch.service.d/override.conf
[Service]
LimitMEMLOCK=infinity
EOF
Excellent! โก Elasticsearch is optimally configured!
โ Step 4: Start and Enable Elasticsearch
Letโs get Elasticsearch running! ๐โโ๏ธ
# Reload systemd
sudo systemctl daemon-reload
# Start Elasticsearch
sudo systemctl start elasticsearch
# Enable Elasticsearch to start on boot
sudo systemctl enable elasticsearch
# Check service status
sudo systemctl status elasticsearch
# Verify Elasticsearch is listening
sudo netstat -tlnp | grep :9200
# Check Elasticsearch process
ps aux | grep elasticsearch
Expected output:
โ elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled)
Active: active (running) since Sat 2025-09-14 10:30:45 UTC; 2min ago
# Test Elasticsearch API
curl -X GET "localhost:9200/"
# Check cluster health
curl -X GET "localhost:9200/_cluster/health?pretty"
# List nodes
curl -X GET "localhost:9200/_nodes?pretty"
API Response:
{
"name" : "node-1",
"cluster_name" : "my-application",
"cluster_uuid" : "xyz123",
"version" : {
"number" : "8.11.0"
},
"tagline" : "You Know, for Search"
}
Fantastic! ๐ Elasticsearch is running perfectly!
๐ง Step 5: Create Your First Index and Documents
Time to index some data! ๐
# Create an index
curl -X PUT "localhost:9200/products" -H 'Content-Type: application/json' -d'
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"properties": {
"name": { "type": "text" },
"description": { "type": "text" },
"price": { "type": "float" },
"category": { "type": "keyword" },
"brand": { "type": "keyword" },
"in_stock": { "type": "boolean" },
"created_at": { "type": "date" }
}
}
}
'
# Index some documents
curl -X POST "localhost:9200/products/_doc/1" -H 'Content-Type: application/json' -d'
{
"name": "Gaming Laptop",
"description": "High-performance laptop for gaming and development",
"price": 1299.99,
"category": "Electronics",
"brand": "TechCorp",
"in_stock": true,
"created_at": "2025-09-14T10:00:00"
}
'
curl -X POST "localhost:9200/products/_doc/2" -H 'Content-Type: application/json' -d'
{
"name": "Wireless Mouse",
"description": "Ergonomic wireless mouse with RGB lighting",
"price": 49.99,
"category": "Electronics",
"brand": "MouseTech",
"in_stock": true,
"created_at": "2025-09-14T11:00:00"
}
'
# Verify documents were indexed
curl -X GET "localhost:9200/products/_count?pretty"
# Get document by ID
curl -X GET "localhost:9200/products/_doc/1?pretty"
Amazing! ๐ Your data is indexed and searchable!
๐ Step 6: Master Search Queries
Letโs explore powerful search capabilities! ๐
# Simple match query
curl -X GET "localhost:9200/products/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"match": {
"name": "Gaming"
}
}
}
'
# Multi-field search
curl -X GET "localhost:9200/products/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"multi_match": {
"query": "laptop gaming",
"fields": ["name", "description"]
}
}
}
'
# Range query
curl -X GET "localhost:9200/products/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"range": {
"price": {
"gte": 50,
"lte": 1500
}
}
}
}
'
# Boolean query combining multiple conditions
curl -X GET "localhost:9200/products/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must": [
{ "match": { "category": "Electronics" } },
{ "range": { "price": { "lte": 100 } } }
],
"filter": [
{ "term": { "in_stock": true } }
]
}
},
"sort": [
{ "price": { "order": "asc" } }
],
"size": 10
}
'
# Aggregation query
curl -X GET "localhost:9200/products/_search?pretty" -H 'Content-Type: application/json' -d'
{
"size": 0,
"aggs": {
"categories": {
"terms": {
"field": "category"
}
},
"avg_price": {
"avg": {
"field": "price"
}
},
"price_ranges": {
"range": {
"field": "price",
"ranges": [
{ "to": 50 },
{ "from": 50, "to": 500 },
{ "from": 500 }
]
}
}
}
}
'
Perfect! ๐ฏ Youโre mastering Elasticsearch queries!
โ Step 7: Install Kibana Dashboard (Optional)
Add visual interface for Elasticsearch! ๐
# Install Kibana
sudo dnf --enablerepo=elasticsearch install kibana -y
# Configure Kibana
sudo cat <<EOF > /etc/kibana/kibana.yml
# Basic Kibana Configuration
server.port: 5601
server.host: "0.0.0.0"
server.name: "kibana-server"
elasticsearch.hosts: ["http://localhost:9200"]
logging.appenders.file.type: file
logging.appenders.file.fileName: /var/log/kibana/kibana.log
logging.appenders.file.layout.type: json
logging.root.appenders: [default, file]
pid.file: /run/kibana/kibana.pid
xpack.security.enabled: false
xpack.encryptedSavedObjects.encryptionKey: "something_at_least_32_characters"
EOF
# Start and enable Kibana
sudo systemctl start kibana
sudo systemctl enable kibana
# Check Kibana status
sudo systemctl status kibana
# Open firewall for Kibana (if needed)
sudo firewall-cmd --add-port=5601/tcp --permanent
sudo firewall-cmd --reload
Access Kibana at: http://your-server-ip:5601
Excellent! ๐ Kibana dashboard is ready!
๐ฎ Quick Examples
Practice Elasticsearch with real-world scenarios! ๐ฏ
Example 1: E-commerce Product Search
# Create product catalog index
curl -X PUT "localhost:9200/catalog" -H 'Content-Type: application/json' -d'
{
"settings": {
"number_of_shards": 2,
"number_of_replicas": 0,
"analysis": {
"analyzer": {
"product_analyzer": {
"type": "custom",
"tokenizer": "standard",
"filter": ["lowercase", "stop", "synonym"]
}
},
"filter": {
"synonym": {
"type": "synonym",
"synonyms": [
"laptop,notebook,computer",
"mobile,phone,smartphone"
]
}
}
}
},
"mappings": {
"properties": {
"title": {
"type": "text",
"analyzer": "product_analyzer",
"fields": {
"keyword": { "type": "keyword" }
}
},
"description": { "type": "text", "analyzer": "product_analyzer" },
"price": { "type": "float" },
"category": { "type": "keyword" },
"tags": { "type": "keyword" },
"rating": { "type": "float" },
"reviews_count": { "type": "integer" },
"availability": { "type": "keyword" },
"brand": { "type": "keyword" },
"specs": {
"type": "nested",
"properties": {
"name": { "type": "keyword" },
"value": { "type": "text" }
}
}
}
}
}
'
# Index products with bulk API
curl -X POST "localhost:9200/_bulk" -H 'Content-Type: application/json' -d'
{ "index": { "_index": "catalog", "_id": "1" } }
{ "title": "MacBook Pro 16-inch", "description": "Powerful laptop for professionals", "price": 2499.99, "category": "Laptops", "brand": "Apple", "rating": 4.8, "reviews_count": 1250, "availability": "in_stock", "tags": ["laptop", "apple", "pro", "work"], "specs": [{"name": "RAM", "value": "16GB"}, {"name": "Storage", "value": "512GB SSD"}] }
{ "index": { "_index": "catalog", "_id": "2" } }
{ "title": "iPhone 15 Pro Max", "description": "Latest flagship smartphone", "price": 1199.99, "category": "Smartphones", "brand": "Apple", "rating": 4.9, "reviews_count": 2340, "availability": "in_stock", "tags": ["phone", "apple", "5g", "camera"], "specs": [{"name": "Storage", "value": "256GB"}, {"name": "Camera", "value": "48MP"}] }
{ "index": { "_index": "catalog", "_id": "3" } }
{ "title": "Dell XPS 13", "description": "Ultrabook for business and travel", "price": 999.99, "category": "Laptops", "brand": "Dell", "rating": 4.6, "reviews_count": 890, "availability": "in_stock", "tags": ["laptop", "dell", "ultrabook", "business"], "specs": [{"name": "RAM", "value": "8GB"}, {"name": "Storage", "value": "256GB SSD"}] }
'
# Search with auto-completion and faceted search
curl -X GET "localhost:9200/catalog/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "laptop pro",
"fields": ["title^2", "description", "tags"],
"type": "best_fields",
"fuzziness": "AUTO"
}
}
],
"filter": [
{ "range": { "price": { "lte": 3000 } } },
{ "term": { "availability": "in_stock" } }
]
}
},
"aggs": {
"categories": {
"terms": { "field": "category", "size": 10 }
},
"brands": {
"terms": { "field": "brand", "size": 10 }
},
"price_ranges": {
"range": {
"field": "price",
"ranges": [
{ "key": "Under $500", "to": 500 },
{ "key": "$500-$1000", "from": 500, "to": 1000 },
{ "key": "$1000-$2000", "from": 1000, "to": 2000 },
{ "key": "Over $2000", "from": 2000 }
]
}
},
"top_rated": {
"filter": { "range": { "rating": { "gte": 4.5 } } },
"aggs": {
"products": {
"top_hits": {
"size": 3,
"_source": ["title", "rating", "price"]
}
}
}
}
},
"sort": [
{ "_score": { "order": "desc" } },
{ "rating": { "order": "desc" } }
],
"highlight": {
"fields": {
"title": {},
"description": {}
}
}
}
'
Example 2: Log Analysis System
# Create logs index with time-based pattern
curl -X PUT "localhost:9200/_index_template/logs-template" -H 'Content-Type: application/json' -d'
{
"index_patterns": ["logs-*"],
"template": {
"settings": {
"number_of_shards": 1,
"number_of_replicas": 0,
"index.refresh_interval": "5s"
},
"mappings": {
"properties": {
"@timestamp": { "type": "date" },
"level": { "type": "keyword" },
"message": { "type": "text", "analyzer": "standard" },
"service": { "type": "keyword" },
"host": { "type": "keyword" },
"user_id": { "type": "keyword" },
"ip": { "type": "ip" },
"response_time": { "type": "float" },
"status_code": { "type": "integer" },
"url": { "type": "keyword" },
"method": { "type": "keyword" }
}
}
}
}
'
# Index log entries
curl -X POST "localhost:9200/_bulk" -H 'Content-Type: application/json' -d'
{ "index": { "_index": "logs-2025.09.14" } }
{ "@timestamp": "2025-09-14T10:00:00.000Z", "level": "INFO", "message": "User login successful", "service": "auth-service", "host": "web-01", "user_id": "user123", "ip": "192.168.1.100", "status_code": 200 }
{ "index": { "_index": "logs-2025.09.14" } }
{ "@timestamp": "2025-09-14T10:05:00.000Z", "level": "ERROR", "message": "Database connection timeout", "service": "api-service", "host": "api-01", "response_time": 5000.5, "status_code": 500 }
{ "index": { "_index": "logs-2025.09.14" } }
{ "@timestamp": "2025-09-14T10:10:00.000Z", "level": "WARN", "message": "High memory usage detected", "service": "monitoring", "host": "api-01", "response_time": 1200.3 }
'
# Search for errors in last hour
curl -X GET "localhost:9200/logs-*/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must": [
{ "term": { "level": "ERROR" } },
{
"range": {
"@timestamp": {
"gte": "now-1h",
"lte": "now"
}
}
}
]
}
},
"aggs": {
"errors_by_service": {
"terms": { "field": "service" }
},
"errors_over_time": {
"date_histogram": {
"field": "@timestamp",
"calendar_interval": "5m"
}
}
},
"sort": [
{ "@timestamp": { "order": "desc" } }
]
}
'
Example 3: Geo-location Search
# Create locations index
curl -X PUT "localhost:9200/restaurants" -H 'Content-Type: application/json' -d'
{
"mappings": {
"properties": {
"name": { "type": "text" },
"cuisine": { "type": "keyword" },
"rating": { "type": "float" },
"location": { "type": "geo_point" },
"address": { "type": "text" },
"price_range": { "type": "keyword" },
"features": { "type": "keyword" }
}
}
}
'
# Index restaurants with geo coordinates
curl -X POST "localhost:9200/_bulk" -H 'Content-Type: application/json' -d'
{ "index": { "_index": "restaurants", "_id": "1" } }
{ "name": "Luigi'\''s Pizza", "cuisine": "Italian", "rating": 4.5, "location": { "lat": 40.7128, "lon": -74.0060 }, "address": "123 Main St, New York, NY", "price_range": "$$", "features": ["delivery", "takeout"] }
{ "index": { "_index": "restaurants", "_id": "2" } }
{ "name": "Sakura Sushi", "cuisine": "Japanese", "rating": 4.8, "location": { "lat": 40.7589, "lon": -73.9851 }, "address": "456 Park Ave, New York, NY", "price_range": "$$$", "features": ["dine-in", "fresh-fish"] }
{ "index": { "_index": "restaurants", "_id": "3" } }
{ "name": "Taco Express", "cuisine": "Mexican", "rating": 4.2, "location": { "lat": 40.7505, "lon": -73.9934 }, "address": "789 Broadway, New York, NY", "price_range": "$", "features": ["delivery", "spicy"] }
'
# Find restaurants within 1km of Times Square
curl -X GET "localhost:9200/restaurants/_search?pretty" -H 'Content-Type: application/json' -d'
{
"query": {
"bool": {
"must": [
{ "range": { "rating": { "gte": 4.0 } } }
],
"filter": [
{
"geo_distance": {
"distance": "1km",
"location": {
"lat": 40.7580,
"lon": -73.9855
}
}
}
]
}
},
"sort": [
{
"_geo_distance": {
"location": {
"lat": 40.7580,
"lon": -73.9855
},
"order": "asc",
"unit": "km"
}
}
],
"aggs": {
"cuisines": {
"terms": { "field": "cuisine" }
},
"distance_ranges": {
"geo_distance": {
"field": "location",
"origin": { "lat": 40.7580, "lon": -73.9855 },
"ranges": [
{ "to": 500 },
{ "from": 500, "to": 1000 },
{ "from": 1000, "to": 2000 }
]
}
}
}
}
'
๐จ Fix Common Problems
Elasticsearch troubleshooting made easy! ๐ง
Problem 1: Elasticsearch Wonโt Start
Solution:
# Check service status
sudo systemctl status elasticsearch
# Check Elasticsearch logs
sudo tail -f /var/log/elasticsearch/my-application.log
# Check system resources
free -h
df -h
# Check Java installation
java -version
echo $JAVA_HOME
# Check configuration syntax
elasticsearch-check-config
# Start with verbose output
sudo -u elasticsearch /usr/share/elasticsearch/bin/elasticsearch -v
# Check for port conflicts
sudo netstat -tlnp | grep :9200
Problem 2: Memory Issues and JVM Problems
Solution:
# Check JVM heap settings
cat /etc/elasticsearch/jvm.options.d/heap.options
# Monitor memory usage
sudo -u elasticsearch jstat -gc $(pgrep -f elasticsearch) 5s
# Check memory lock settings
cat /proc/$(pgrep -f elasticsearch)/limits | grep memlock
# Adjust heap size (50% of RAM, max 32GB)
sudo nano /etc/elasticsearch/jvm.options.d/heap.options
Add/modify:
-Xms4g
-Xmx4g
Problem 3: Cluster Health Issues
Solution:
# Check cluster health
curl -X GET "localhost:9200/_cluster/health?pretty"
# Check node status
curl -X GET "localhost:9200/_nodes?pretty"
# Check shard allocation
curl -X GET "localhost:9200/_cat/shards?v"
# Fix unassigned shards
curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
"persistent": {
"cluster.routing.allocation.enable": "all"
}
}
'
# Manually allocate shards if needed
curl -X POST "localhost:9200/_cluster/reroute?retry_failed=true&pretty"
Problem 4: Search Performance Issues
Solution:
# Check index statistics
curl -X GET "localhost:9200/_stats?pretty"
# Analyze slow queries
curl -X GET "localhost:9200/_nodes/stats/indices/search?pretty"
# Check field data usage
curl -X GET "localhost:9200/_nodes/stats/indices/fielddata?pretty"
# Clear field data cache
curl -X POST "localhost:9200/_cache/clear?fielddata=true"
# Check index settings
curl -X GET "localhost:9200/your-index/_settings?pretty"
# Optimize index
curl -X POST "localhost:9200/your-index/_forcemerge?max_num_segments=1"
๐ Simple Commands Summary
Command | Purpose |
---|---|
curl -X GET "localhost:9200/" | Check Elasticsearch status |
curl -X PUT "localhost:9200/index" | Create index |
curl -X POST "localhost:9200/index/_doc" | Index document |
curl -X GET "localhost:9200/index/_search" | Search documents |
curl -X DELETE "localhost:9200/index" | Delete index |
curl -X GET "localhost:9200/_cat/indices" | List all indices |
curl -X GET "localhost:9200/_cluster/health" | Check cluster health |
curl -X GET "localhost:9200/_nodes/stats" | Node statistics |
๐ก Tips for Success
Master Elasticsearch with these pro tips! ๐
- ๐ Index Strategy: Design indices based on query patterns
- ๐พ Memory Management: Set heap to 50% of RAM, max 32GB
- ๐ Query Optimization: Use filters for exact matches
- ๐ Monitoring: Track cluster health and performance
- ๐ฏ Mapping Design: Define mappings before indexing data
- โก Bulk Operations: Use _bulk API for large data loads
- ๐ Security: Enable X-Pack security in production
- ๐ Scaling: Plan for horizontal scaling early
- ๐ ๏ธ Maintenance: Regular index optimization and cleanup
- ๐ค Community: Join Elastic forums and user groups
๐ What You Learned
Congratulations! Youโre now an Elasticsearch expert! ๐
- โ Installed Elasticsearch on AlmaLinux
- โ Configured production-ready settings
- โ Created indices with custom mappings
- โ Mastered search queries and aggregations
- โ Implemented real-world search scenarios
- โ Set up Kibana dashboard interface
- โ Learned performance optimization techniques
- โ Gained troubleshooting expertise
- โ Acquired $140k+ valued search skills
๐ฏ Why This Matters
Your Elasticsearch expertise unlocks incredible opportunities! ๐
- ๐ผ Career Advancement: Search engineers are highly sought after
- ๐ Modern Applications: Power intelligent search features
- ๐ Big Data Analytics: Process and analyze massive datasets
- โก Real-time Insights: Instant data processing and visualization
- ๐ Scalable Solutions: Handle enterprise-level search requirements
- ๐ฏ Industry Standard: Used by major tech companies worldwide
- ๐ฎ Future Ready: Essential for AI and machine learning
Youโve mastered the search engine powering the digital world! ๐
Happy searching! ๐