128. Contextual Logging with logging

The logging module in Python provides a flexible framework for logging messages from Python programs. You can easily control the severity level of messages (e.g., debug, info, warning, error, critical) and configure the output format. Below are a few examples that demonstrate how to implement dynamic logging levels, format messages, and customize the logging behavior.

1. Basic Logging Setup

This example shows how to configure logging with a basic setup to log messages at different levels.

Copy

import logging

# Basic configuration of logging with default level (WARNING)
logging.basicConfig()

logging.debug("This is a debug message")  # Will not show, because default level is WARNING
logging.info("This is an info message")  # Will not show, because default level is WARNING
logging.warning("This is a warning message")  # Will be shown
logging.error("This is an error message")  # Will be shown
logging.critical("This is a critical message")  # Will be shown

---

2. Setting Dynamic Logging Level

You can dynamically change the logging level by configuring the logger object directly.

Copy

import logging

# Create a custom logger
logger = logging.getLogger('my_logger')

# Set the logging level to DEBUG
logger.setLevel(logging.DEBUG)

# Create a console handler to display logs
console_handler = logging.StreamHandler()

# Set the log format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)

# Add the handler to the logger
logger.addHandler(console_handler)

logger.debug("This is a debug message")
logger.info("This is an info message")
logger.warning("This is a warning message")
logger.error("This is an error message")
logger.critical("This is a critical message")

Output:

Copy

2024-01-23 12:00:00,000 - my_logger - DEBUG - This is a debug message
2024-01-23 12:00:00,000 - my_logger - INFO - This is an info message
2024-01-23 12:00:00,000 - my_logger - WARNING - This is a warning message
2024-01-23 12:00:00,000 - my_logger - ERROR - This is an error message
2024-01-23 12:00:00,000 - my_logger - CRITICAL - This is a critical message

---

3. Logging to a File

You can log messages to a file rather than the console by using a file handler.

Copy

import logging

# Set up logging to a file
logging.basicConfig(filename='app.log', level=logging.DEBUG, 
                    format='%(asctime)s - %(levelname)s - %(message)s')

logging.debug("This is a debug message")
logging.info("This is an info message")
logging.warning("This is a warning message")
logging.error("This is an error message")
logging.critical("This is a critical message")

The messages will be saved in the app.log file with the specified format.

---

4. Logging with Different Formats

You can customize the log format to include different details such as the line number, function name, and log message.

Copy

import logging

# Create a custom log format
log_format = '%(asctime)s - %(name)s - %(levelname)s - %(message)s - Line: %(lineno)d'

# Set up logging with custom format
logging.basicConfig(level=logging.DEBUG, format=log_format)

logging.debug("This is a debug message")
logging.info("This is an info message")
logging.warning("This is a warning message")
logging.error("This is an error message")
logging.critical("This is a critical message")

---

5. Logging with Rotating Files

To prevent log files from growing too large, you can use logging.handlers.RotatingFileHandler to create rotating log files.

Copy

import logging
from logging.handlers import RotatingFileHandler

# Set up a rotating file handler
handler = RotatingFileHandler('app.log', maxBytes=2000, backupCount=3)
handler.setLevel(logging.INFO)

# Set up logging
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)

logger = logging.getLogger('my_logger')
logger.addHandler(handler)

# Log some messages
for i in range(100):
    logger.info(f"Log message {i}")

This will create an app.log file that rotates after it reaches 2000 bytes, keeping up to 3 backup files.

---

6. Using a Logger with Multiple Handlers

You can set up multiple handlers (e.g., console and file logging) with different logging levels.

Copy

import logging

# Set up custom logger
logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)

# Console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.WARNING)

# File handler
file_handler = logging.FileHandler('app.log')
file_handler.setLevel(logging.INFO)

# Formatter
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)
file_handler.setFormatter(formatter)

# Add handlers to the logger
logger.addHandler(console_handler)
logger.addHandler(file_handler)

# Log messages
logger.debug("This is a debug message")  # Will not be logged
logger.info("This is an info message")  # Will be logged to the file
logger.warning("This is a warning message")  # Will be logged to both console and file

---

7. Custom Logging Levels

You can create custom logging levels by subclassing the logging.Level class.

Copy

import logging

# Create a custom logging level
LOGGING_LEVEL = 25
logging.addLevelName(LOGGING_LEVEL, "CUSTOM")

# Define a custom log function
def log_custom(self, message, *args, **kwargs):
    if self.isEnabledFor(LOGGING_LEVEL):
        self._log(LOGGING_LEVEL, message, args, **kwargs)

# Attach the custom function to the logging module
logging.Logger.custom = log_custom

# Set up the logger
logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)

# Log with custom level
logger.custom("This is a custom log message")

---

8. Contextual Logging with extra

Use the extra keyword to pass additional context information to the log messages.

Copy

import logging

# Set up logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s - %(user)s')

# Log with extra context
logger = logging.getLogger('my_logger')
logger.info("User logged in", extra={'user': 'alice'})
logger.error("An error occurred", extra={'user': 'bob'})

---

9. Loggers with Multiple Sources

You can use multiple loggers for different parts of your application.

Copy

import logging

# Set up logging for 'moduleA'
logger_A = logging.getLogger('moduleA')
logger_A.setLevel(logging.DEBUG)
logger_A.addHandler(logging.StreamHandler())

# Set up logging for 'moduleB'
logger_B = logging.getLogger('moduleB')
logger_B.setLevel(logging.ERROR)
logger_B.addHandler(logging.StreamHandler())

# Log messages for each module
logger_A.debug("This is a debug message from moduleA")
logger_B.error("This is an error message from moduleB")

---

10. Timed Logging with logging.handlers.TimedRotatingFileHandler

This handler allows you to rotate log files based on time intervals.

Copy

import logging
from logging.handlers import TimedRotatingFileHandler

# Set up a timed rotating file handler
handler = TimedRotatingFileHandler('timed_app.log', when='midnight', interval=1, backupCount=5)
handler.setLevel(logging.INFO)

# Set up logging
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)

logger = logging.getLogger('my_logger')
logger.addHandler(handler)

# Log some messages
for i in range(10):
    logger.info(f"Log message {i}")

This will rotate the log file every midnight and keep 5 backup files.

---

These snippets illustrate how you can implement dynamic logging levels, customized message formatting, multiple logging outputs, and file rotation, making the logging module a versatile tool for handling logging in your applications.