195. Python Profiling

Profiling is a way to identify performance bottlenecks in your Python code. By using cProfile or profile, you can measure the time taken by various parts of your code and identify which functions are consuming the most resources. Here are 10 Python code snippets demonstrating how to use profiling with cProfile and profile.

---

1. Basic Profiling with cProfile

Copy

import cProfile

def slow_function():
    total = 0
    for i in range(1000000):
        total += i
    return total

# Profiling the function
cProfile.run('slow_function()')

---

2. Profiling a Function in a Larger Script

Copy

import cProfile

def function_a():
    sum(range(10000))

def function_b():
    for i in range(100):
        sum(range(10000))

def main():
    function_a()
    function_b()

# Profiling the entire main function
cProfile.run('main()')

---

3. Saving Profiling Results to a File

Copy

import cProfile

def example_function():
    return sum(range(1000000))

# Save the profiling results to a file
cProfile.run('example_function()', 'profiling_output.prof')

---

4. Using pstats to Analyze Profiling Data

Copy

import cProfile
import pstats

def function_to_profile():
    total = 0
    for i in range(1000000):
        total += i
    return total

# Run profiling and save to file
cProfile.run('function_to_profile()', 'output.prof')

# Load the profile data and print the stats
p = pstats.Stats('output.prof')
p.sort_stats('time').print_stats()

---

5. Profiling with Different Sorting Options

Copy

import cProfile

def sample_function():
    for i in range(10000):
        sum(range(10000))

# Run and profile the sample_function with sorting by time
cProfile.run('sample_function()', sort_by='time')

---

6. Profiling Multiple Functions

Copy

import cProfile

def function_one():
    for _ in range(5000):
        sum(range(1000))

def function_two():
    for _ in range(10000):
        sum(range(500))

# Profile both functions
cProfile.run('function_one()')
cProfile.run('function_two()')

---

7. Viewing Profiling Data in Graphical Format (using snakeviz)

Copy

pip install snakeviz

After running a script with cProfile, you can visualize the results:

Copy

python -m cProfile -o output.prof my_script.py
snakeviz output.prof

---

8. Profiling a Class Method

Copy

import cProfile

class MyClass:
    def method_one(self):
        return sum(range(10000))

    def method_two(self):
        for _ in range(1000):
            sum(range(1000))

# Profiling class methods
cProfile.run('MyClass().method_one()')
cProfile.run('MyClass().method_two()')

---

9. Using profile for More Granular Control (Profiling a Specific Code Block)

Copy

import profile

def optimized_function():
    total = 0
    for i in range(100000):
        total += i
    return total

# Start profiling a specific code block
profiler = profile.Profile()
profiler.enable()
optimized_function()
profiler.disable()

# Print profiling results
profiler.print_stats()

---

10. Profiling with a Decorator for Easy Reuse

Copy

import cProfile
from functools import wraps

def profile_function(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        profiler = cProfile.Profile()
        profiler.enable()
        result = func(*args, **kwargs)
        profiler.disable()
        profiler.print_stats()
        return result
    return wrapper

@profile_function
def function_to_profile():
    total = 0
    for i in range(100000):
        total += i
    return total

# Call the function and profile it automatically
function_to_profile()

---

Key Concepts:

1. Profiling: cProfile and profile allow you to monitor function call statistics, including execution time.

2. Sorting Stats: You can sort the results based on time, number of calls, or other metrics to identify performance bottlenecks.

3. Saving Output: You can save profiling results to a file and analyze it later, even in graphical formats like snakeviz.

4. Profile Specific Functions: By using decorators or manual profiling commands, you can profile individual parts of your code or entire scripts.

5. Granular Profiling with profile: Provides more detailed control over profiling specific code blocks.

Profiling helps you pinpoint areas in your code that need optimization, making it an essential tool for performance tuning.