📘 Benchmark Creation: Performance Testing

🎯 Introduction

Welcome to this exciting tutorial on benchmark creation and performance testing in TypeScript! 🎉 In this guide, we’ll explore how to measure and optimize your code’s performance like a pro.

You’ll discover how creating effective benchmarks can transform your TypeScript applications from slow and sluggish to fast and efficient. Whether you’re building web applications 🌐, server-side code 🖥️, or libraries 📚, understanding performance testing is essential for delivering blazing-fast user experiences.

By the end of this tutorial, you’ll feel confident creating benchmarks that reveal performance bottlenecks and guide your optimization efforts! Let’s dive in! 🏊‍♂️

📚 Understanding Benchmark Creation

🤔 What is Performance Benchmarking?

Performance benchmarking is like a race track for your code 🏁. Think of it as timing different runners (code implementations) to see who crosses the finish line fastest!

In TypeScript terms, benchmarking helps you measure how long operations take, how much memory they use, and how they perform under different conditions. This means you can:

✨ Identify slow code before users complain
🚀 Compare different implementation approaches
🛡️ Prevent performance regressions

💡 Why Use Benchmarks?

Here’s why developers love performance benchmarking:

Data-Driven Decisions 📊: No more guessing which code is faster
Objective Measurements 💻: Real numbers instead of feelings
Performance Tracking 📖: Monitor speed over time
Optimization Confidence 🔧: Know your changes actually help

Real-world example: Imagine building an e-commerce search 🛒. With benchmarks, you can ensure searches stay fast even with millions of products!

🔧 Basic Syntax and Usage

📝 Simple Example

Let’s start with a friendly example:

// 👋 Hello, Performance Testing!
interface BenchmarkResult {
  name: string;      // 🏷️ Test name
  duration: number;  // ⏱️ Time in milliseconds
  ops: number;      // 🚀 Operations per second
}

// 🎨 Creating a simple benchmark
class SimpleBenchmark {
  // ⏱️ Measure execution time
  static measure(name: string, fn: () => void, iterations = 1000): BenchmarkResult {
    const start = performance.now();
    
    // 🔄 Run the function multiple times
    for (let i = 0; i < iterations; i++) {
      fn();
    }
    
    const end = performance.now();
    const duration = end - start;
    const ops = Math.round((iterations / duration) * 1000);
    
    console.log(`✨ ${name}: ${duration.toFixed(2)}ms for ${iterations} runs`);
    console.log(`🚀 ${ops} operations per second`);
    
    return { name, duration, ops };
  }
}

💡 Explanation: Notice how we run the function multiple times to get accurate measurements! Single runs can be misleading due to system variability.

🎯 Common Patterns

Here are patterns you’ll use daily:

// 🏗️ Pattern 1: Comparing implementations
const compareArrayMethods = () => {
  const numbers = Array.from({ length: 10000 }, (_, i) => i);
  
  // 🎨 Test 1: Traditional for loop
  SimpleBenchmark.measure("For Loop", () => {
    let sum = 0;
    for (let i = 0; i < numbers.length; i++) {
      sum += numbers[i];
    }
  });
  
  // 🎨 Test 2: Array reduce
  SimpleBenchmark.measure("Array Reduce", () => {
    numbers.reduce((sum, n) => sum + n, 0);
  });
};

// 🔄 Pattern 2: Memory benchmarking
interface MemorySnapshot {
  heapUsed: number;  // 💾 Memory in bytes
  external: number;  // 📦 External memory
}

// 🎯 Pattern 3: Async benchmarking
async function benchmarkAsync(
  name: string, 
  fn: () => Promise<void>, 
  iterations = 100
): Promise<BenchmarkResult> {
  const start = performance.now();
  
  for (let i = 0; i < iterations; i++) {
    await fn();
  }
  
  const duration = performance.now() - start;
  return { name, duration, ops: (iterations / duration) * 1000 };
}

💡 Practical Examples

🛒 Example 1: E-Commerce Search Optimization

Let’s benchmark a real product search:

// 🛍️ Define our product type
interface Product {
  id: string;
  name: string;
  price: number;
  category: string;
  tags: string[];
  emoji: string; // Every product needs an emoji! 
}

// 🔍 Search implementation to benchmark
class ProductSearch {
  private products: Product[] = [];
  private indexMap: Map<string, Set<Product>> = new Map();
  
  constructor(products: Product[]) {
    this.products = products;
    this.buildIndex();
  }
  
  // 📚 Build search index
  private buildIndex(): void {
    this.products.forEach(product => {
      // Index by name words
      const words = product.name.toLowerCase().split(' ');
      words.forEach(word => {
        if (!this.indexMap.has(word)) {
          this.indexMap.set(word, new Set());
        }
        this.indexMap.get(word)!.add(product);
      });
    });
  }
  
  // 🐌 Slow search (for comparison)
  searchSlow(query: string): Product[] {
    const lowerQuery = query.toLowerCase();
    return this.products.filter(p => 
      p.name.toLowerCase().includes(lowerQuery)
    );
  }
  
  // 🚀 Fast indexed search
  searchFast(query: string): Product[] {
    const lowerQuery = query.toLowerCase();
    const words = lowerQuery.split(' ');
    const results = new Set<Product>();
    
    words.forEach(word => {
      const matches = this.indexMap.get(word) || new Set();
      matches.forEach(product => results.add(product));
    });
    
    return Array.from(results);
  }
}

// 🎮 Let's benchmark it!
const runSearchBenchmark = () => {
  // 🏗️ Create test data
  const products: Product[] = Array.from({ length: 10000 }, (_, i) => ({
    id: `prod-${i}`,
    name: `Product ${i} ${['Awesome', 'Super', 'Ultra'][i % 3]}`,
    price: Math.random() * 100,
    category: ['Electronics', 'Books', 'Clothing'][i % 3],
    tags: ['new', 'sale', 'popular'].slice(0, (i % 3) + 1),
    emoji: ['📱', '📚', '👕'][i % 3]
  }));
  
  const search = new ProductSearch(products);
  
  console.log("🏁 Starting search benchmark...\n");
  
  // 🐌 Benchmark slow search
  SimpleBenchmark.measure("Slow Search", () => {
    search.searchSlow("awesome");
  }, 1000);
  
  console.log("\n");
  
  // 🚀 Benchmark fast search
  SimpleBenchmark.measure("Fast Search", () => {
    search.searchFast("awesome");
  }, 1000);
};

🎯 Try it yourself: Add a fuzzy search method and benchmark it against the others!

🎮 Example 2: Game Engine Performance

Let’s benchmark a game update loop:

// 🏆 Game entity system benchmark
interface GameObject {
  id: string;
  x: number;
  y: number;
  velocity: { x: number; y: number };
  emoji: string;
}

class GameEngine {
  private objects: GameObject[] = [];
  private spatialGrid: Map<string, GameObject[]> = new Map();
  private gridSize = 100;
  
  // 🎮 Add game objects
  addObjects(count: number): void {
    for (let i = 0; i < count; i++) {
      this.objects.push({
        id: `obj-${i}`,
        x: Math.random() * 1000,
        y: Math.random() * 1000,
        velocity: { 
          x: (Math.random() - 0.5) * 10, 
          y: (Math.random() - 0.5) * 10 
        },
        emoji: ['🚀', '🛸', '⭐', '🌟'][i % 4]
      });
    }
    this.updateSpatialGrid();
  }
  
  // 🗺️ Update spatial partitioning
  private updateSpatialGrid(): void {
    this.spatialGrid.clear();
    
    this.objects.forEach(obj => {
      const gridKey = this.getGridKey(obj.x, obj.y);
      if (!this.spatialGrid.has(gridKey)) {
        this.spatialGrid.set(gridKey, []);
      }
      this.spatialGrid.get(gridKey)!.push(obj);
    });
  }
  
  // 🔑 Get grid cell key
  private getGridKey(x: number, y: number): string {
    const gridX = Math.floor(x / this.gridSize);
    const gridY = Math.floor(y / this.gridSize);
    return `${gridX},${gridY}`;
  }
  
  // 🐌 Update without optimization
  updateSlow(deltaTime: number): void {
    // Update positions
    this.objects.forEach(obj => {
      obj.x += obj.velocity.x * deltaTime;
      obj.y += obj.velocity.y * deltaTime;
    });
    
    // Check all collisions (n²)
    for (let i = 0; i < this.objects.length; i++) {
      for (let j = i + 1; j < this.objects.length; j++) {
        const distance = Math.sqrt(
          Math.pow(this.objects[i].x - this.objects[j].x, 2) +
          Math.pow(this.objects[i].y - this.objects[j].y, 2)
        );
        if (distance < 50) {
          // Collision detected! 💥
        }
      }
    }
  }
  
  // 🚀 Update with spatial optimization
  updateFast(deltaTime: number): void {
    // Update positions
    this.objects.forEach(obj => {
      obj.x += obj.velocity.x * deltaTime;
      obj.y += obj.velocity.y * deltaTime;
    });
    
    // Update grid
    this.updateSpatialGrid();
    
    // Check collisions only in nearby cells
    this.spatialGrid.forEach((cellObjects) => {
      for (let i = 0; i < cellObjects.length; i++) {
        for (let j = i + 1; j < cellObjects.length; j++) {
          const distance = Math.sqrt(
            Math.pow(cellObjects[i].x - cellObjects[j].x, 2) +
            Math.pow(cellObjects[i].y - cellObjects[j].y, 2)
          );
          if (distance < 50) {
            // Collision detected! 💥
          }
        }
      }
    });
  }
}

// 🎯 Benchmark the game engine
const benchmarkGameEngine = () => {
  const engine = new GameEngine();
  engine.addObjects(1000); // 1000 game objects!
  
  console.log("🎮 Game Engine Benchmark\n");
  
  SimpleBenchmark.measure("Slow Update", () => {
    engine.updateSlow(0.016); // 60 FPS
  }, 100);
  
  console.log("\n");
  
  SimpleBenchmark.measure("Fast Update", () => {
    engine.updateFast(0.016);
  }, 100);
};

🚀 Advanced Concepts

🧙‍♂️ Advanced Topic 1: Statistical Benchmarking

When you’re ready to level up, try this advanced pattern:

// 🎯 Advanced benchmark with statistics
interface DetailedBenchmarkResult {
  name: string;
  runs: number[];
  mean: number;
  median: number;
  stdDev: number;
  percentile95: number;
  sparkles: "✨" | "🌟" | "💫";
}

class AdvancedBenchmark {
  // 🪄 Run benchmark with statistics
  static async measureDetailed(
    name: string,
    fn: () => void | Promise<void>,
    options = { warmupRuns: 10, testRuns: 100 }
  ): Promise<DetailedBenchmarkResult> {
    const runs: number[] = [];
    
    // 🔥 Warmup runs
    console.log(`🔥 Warming up ${name}...`);
    for (let i = 0; i < options.warmupRuns; i++) {
      await fn();
    }
    
    // 📊 Test runs
    console.log(`📊 Testing ${name}...`);
    for (let i = 0; i < options.testRuns; i++) {
      const start = performance.now();
      await fn();
      const duration = performance.now() - start;
      runs.push(duration);
    }
    
    // 📈 Calculate statistics
    const sorted = [...runs].sort((a, b) => a - b);
    const mean = runs.reduce((a, b) => a + b) / runs.length;
    const median = sorted[Math.floor(sorted.length / 2)];
    const variance = runs.reduce((sum, run) => 
      sum + Math.pow(run - mean, 2), 0) / runs.length;
    const stdDev = Math.sqrt(variance);
    const percentile95 = sorted[Math.floor(sorted.length * 0.95)];
    
    const sparkles = stdDev < mean * 0.1 ? "💫" : 
                     stdDev < mean * 0.2 ? "🌟" : "✨";
    
    return { name, runs, mean, median, stdDev, percentile95, sparkles };
  }
}

🏗️ Advanced Topic 2: Memory Profiling

For the brave developers:

// 🚀 Memory benchmark utilities
class MemoryBenchmark {
  private initialMemory: number = 0;
  
  // 💾 Start memory tracking
  start(): void {
    if (global.gc) {
      global.gc(); // Force garbage collection
    }
    this.initialMemory = process.memoryUsage().heapUsed;
  }
  
  // 📊 Get memory delta
  getMemoryDelta(): number {
    const currentMemory = process.memoryUsage().heapUsed;
    return currentMemory - this.initialMemory;
  }
  
  // 🎯 Measure memory allocation
  static measureMemory<T>(
    name: string,
    fn: () => T
  ): { result: T; memoryUsed: number } {
    const bench = new MemoryBenchmark();
    bench.start();
    
    const result = fn();
    
    const memoryUsed = bench.getMemoryDelta();
    const memoryMB = (memoryUsed / 1024 / 1024).toFixed(2);
    
    console.log(`💾 ${name}: ${memoryMB} MB allocated`);
    
    return { result, memoryUsed };
  }
}

⚠️ Common Pitfalls and Solutions

😱 Pitfall 1: Micro-benchmark Bias

// ❌ Wrong way - too small to measure accurately!
const badBenchmark = () => {
  const start = performance.now();
  const result = 1 + 1;
  const end = performance.now();
  console.log(`Time: ${end - start}ms`); // 💥 Often 0ms!
};

// ✅ Correct way - run multiple iterations!
const goodBenchmark = () => {
  const iterations = 1000000;
  const start = performance.now();
  
  for (let i = 0; i < iterations; i++) {
    const result = 1 + 1;
  }
  
  const end = performance.now();
  const avgTime = (end - start) / iterations;
  console.log(`✅ Average time: ${avgTime * 1000000}ns`);
};

🤯 Pitfall 2: Ignoring JIT Optimization

// ❌ Dangerous - first runs are slow!
function measureOnce(fn: () => void): number {
  const start = performance.now();
  fn();
  return performance.now() - start;
}

// ✅ Safe - warm up the JIT!
function measureWithWarmup(fn: () => void): number {
  // 🔥 Warm up runs
  for (let i = 0; i < 100; i++) {
    fn();
  }
  
  // 📊 Actual measurement
  const start = performance.now();
  for (let i = 0; i < 1000; i++) {
    fn();
  }
  return (performance.now() - start) / 1000;
}

🛠️ Best Practices

🎯 Measure Real Scenarios: Don’t benchmark trivial operations
📝 Use Multiple Runs: Single measurements are unreliable
🛡️ Control Environment: Close other apps during benchmarks
🎨 Compare Apples to Apples: Same data, same conditions
✨ Document Results: Track performance over time

🧪 Hands-On Exercise

🎯 Challenge: Build a String Processing Benchmark Suite

Create a comprehensive benchmark for string operations:

📋 Requirements:

✅ Compare different string concatenation methods
🏷️ Test string search algorithms (indexOf vs regex)
👤 Measure string transformation performance
📅 Include both sync and async operations
🎨 Create visual performance reports!

🚀 Bonus Points:

Add memory usage tracking
Implement result caching benchmarks
Create a CLI tool for running benchmarks

💡 Solution

🔍 Click to see solution

// 🎯 String Processing Benchmark Suite!
interface StringBenchmarkResult {
  method: string;
  duration: number;
  memoryUsed?: number;
  opsPerSecond: number;
  emoji: string;
}

class StringBenchmarkSuite {
  private results: StringBenchmarkResult[] = [];
  
  // 📊 Run all benchmarks
  async runAll(): Promise<void> {
    console.log("🚀 String Processing Benchmark Suite\n");
    
    await this.benchmarkConcatenation();
    await this.benchmarkSearch();
    await this.benchmarkTransformation();
    
    this.printReport();
  }
  
  // 🔗 Benchmark concatenation methods
  private async benchmarkConcatenation(): Promise<void> {
    console.log("🔗 Testing String Concatenation...\n");
    
    const strings = Array.from({ length: 1000 }, (_, i) => `string${i}`);
    
    // Method 1: Plus operator
    const plusResult = await this.measure("Plus Operator", () => {
      let result = "";
      for (const str of strings) {
        result = result + str;
      }
    });
    
    // Method 2: Array join
    const joinResult = await this.measure("Array Join", () => {
      strings.join("");
    });
    
    // Method 3: Template literals
    const templateResult = await this.measure("Template Literals", () => {
      let result = "";
      for (const str of strings) {
        result = `${result}${str}`;
      }
    });
    
    this.results.push(plusResult, joinResult, templateResult);
  }
  
  // 🔍 Benchmark search methods
  private async benchmarkSearch(): Promise<void> {
    console.log("\n🔍 Testing String Search...\n");
    
    const text = "The quick brown fox jumps over the lazy dog".repeat(1000);
    const searchTerm = "fox";
    
    // Method 1: indexOf
    const indexOfResult = await this.measure("indexOf", () => {
      let count = 0;
      let index = 0;
      while ((index = text.indexOf(searchTerm, index)) !== -1) {
        count++;
        index += searchTerm.length;
      }
    });
    
    // Method 2: Regular Expression
    const regexResult = await this.measure("RegExp", () => {
      const matches = text.match(new RegExp(searchTerm, 'g'));
      const count = matches ? matches.length : 0;
    });
    
    this.results.push(indexOfResult, regexResult);
  }
  
  // 🎨 Benchmark transformations
  private async benchmarkTransformation(): Promise<void> {
    console.log("\n🎨 Testing String Transformations...\n");
    
    const text = "Hello World This Is A Test".repeat(100);
    
    // Method 1: toLowerCase
    const lowerResult = await this.measure("toLowerCase", () => {
      text.toLowerCase();
    });
    
    // Method 2: Replace
    const replaceResult = await this.measure("Replace Spaces", () => {
      text.replace(/ /g, '_');
    });
    
    // Method 3: Split and Join
    const splitJoinResult = await this.measure("Split & Join", () => {
      text.split(' ').map(word => word.toLowerCase()).join('-');
    });
    
    this.results.push(lowerResult, replaceResult, splitJoinResult);
  }
  
  // ⏱️ Measure a single operation
  private async measure(
    method: string, 
    fn: () => void
  ): Promise<StringBenchmarkResult> {
    const warmupRuns = 100;
    const testRuns = 1000;
    
    // 🔥 Warmup
    for (let i = 0; i < warmupRuns; i++) {
      fn();
    }
    
    // 📊 Test
    const start = performance.now();
    for (let i = 0; i < testRuns; i++) {
      fn();
    }
    const duration = performance.now() - start;
    const opsPerSecond = Math.round((testRuns / duration) * 1000);
    
    const emoji = opsPerSecond > 10000 ? "🚀" :
                  opsPerSecond > 5000 ? "⚡" :
                  opsPerSecond > 1000 ? "✨" : "🐌";
    
    console.log(`${emoji} ${method}: ${duration.toFixed(2)}ms for ${testRuns} runs`);
    console.log(`   ${opsPerSecond} ops/sec\n`);
    
    return { method, duration, opsPerSecond, emoji };
  }
  
  // 📊 Print final report
  private printReport(): void {
    console.log("\n📊 === BENCHMARK REPORT === 📊\n");
    
    // Sort by performance
    const sorted = [...this.results].sort((a, b) => 
      b.opsPerSecond - a.opsPerSecond
    );
    
    sorted.forEach((result, index) => {
      const medal = index === 0 ? "🥇" : 
                    index === 1 ? "🥈" : 
                    index === 2 ? "🥉" : "  ";
      
      console.log(`${medal} ${result.emoji} ${result.method}`);
      console.log(`   Performance: ${result.opsPerSecond} ops/sec`);
      console.log(`   Duration: ${result.duration.toFixed(2)}ms\n`);
    });
    
    console.log("✨ Benchmark complete! ✨");
  }
}

// 🎮 Run the benchmark!
const runStringBenchmarks = async () => {
  const suite = new StringBenchmarkSuite();
  await suite.runAll();
};

🎓 Key Takeaways

You’ve learned so much! Here’s what you can now do:

✅ Create benchmarks with confidence 💪
✅ Avoid common pitfalls that skew results 🛡️
✅ Apply best practices for accurate measurements 🎯
✅ Debug performance issues like a pro 🐛
✅ Build faster applications with TypeScript! 🚀

Remember: Performance matters, but measure first before optimizing! Don’t guess, test! 🤝

🤝 Next Steps

Congratulations! 🎉 You’ve mastered performance benchmarking!

Here’s what to do next:

💻 Practice with the exercises above
🏗️ Benchmark your own projects
📚 Move on to our next tutorial: Profile-guided Optimization
🌟 Share your benchmark results with the team!

Remember: Every millisecond saved makes users happier. Keep measuring, keep optimizing, and most importantly, have fun! 🚀

Happy benchmarking! 🎉🚀✨

Prerequisites

What you'll learn