PyTest

1

PyTest for Flask

To write modern unit test cases for your Flask Inference API, you can use the following libraries:

1. pytest: A powerful testing framework for Python.

2. pytest-flask: A Flask-specific extension for pytest to simplify testing Flask applications.

3. requests-mock: For mocking external API calls.

4. factory-boy or faker: For generating test data.

5. coverage: For measuring test coverage.

Here’s a step-by-step guide to write unit test cases:

1. Set Up a Test Environment

Ensure your Flask app is modular and can be initialized easily for testing. For example:

Copy

# app.py
from flask import Flask, jsonify, request

app = Flask(__name__)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.json
    if not data or 'input' not in data:
        return jsonify({'error': 'Invalid input'}), 400
    # Mock model inference logic
    prediction = len(data['input'])  # Replace with actual model prediction
    return jsonify({'prediction': prediction})

if __name__ == "__main__":
    app.run(debug=True)

2. Install Required Libraries

Install libraries:

Copy

pip install pytest pytest-flask requests-mock faker

3. Create Test Cases

Create a tests/ directory and add test files.

Example: tests/test_app.py

Copy

import pytest
from app import app

@pytest.fixture
def client():
    with app.test_client() as client:
        yield client

def test_predict_success(client):
    response = client.post('/predict', json={'input': 'test data'})
    assert response.status_code == 200
    assert response.json == {'prediction': 9}

def test_predict_invalid_input(client):
    response = client.post('/predict', json={})
    assert response.status_code == 400
    assert response.json == {'error': 'Invalid input'}

def test_predict_no_input(client):
    response = client.post('/predict', json=None)
    assert response.status_code == 400
    assert response.json == {'error': 'Invalid input'}

4. Run the Tests

Run tests using pytest:

Copy

pytest --cov=app tests/

5. Use Mocks for External Calls

If your API calls an external service, mock it using requests-mock.

Example: Mocking an External API

Copy

import requests
from flask import Flask, jsonify

app = Flask(__name__)

@app.route('/external', methods=['GET'])
def external_call():
    response = requests.get('https://api.example.com/data')
    return jsonify(response.json()), response.status_code

Test with mock:

Copy

import pytest
import requests_mock
from app import app

@pytest.fixture
def client():
    with app.test_client() as client:
        yield client

@requests_mock.Mocker()
def test_external_call(mock, client):
    mock.get('https://api.example.com/data', json={'key': 'value'}, status_code=200)
    response = client.get('/external')
    assert response.status_code == 200
    assert response.json == {'key': 'value'}

6. Measure Coverage

To measure test coverage, use:

Copy

pytest --cov=app tests/

Best Practices

1. Use fixtures for setting up the test environment.

2. Use mocks for external dependencies.

3. Keep test cases independent.

4. Aim for high coverage (80%+).

5. Validate both positive and negative scenarios.

---

2

PyTest for FastAPI

To write unit test cases for your FastAPI Inference API using modern testing libraries, you can follow these steps:

---

1. Recommended Libraries

• pytest: A robust testing framework.

• httpx: For making async HTTP requests.

• pytest-asyncio: To test FastAPI’s async endpoints.

• fastapi.testclient: For synchronous testing of FastAPI routes.

• coverage: To measure code coverage.

Install the required libraries:

Copy

pip install pytest pytest-asyncio httpx fastapi[all] coverage

---

2. Example FastAPI Inference API

Here’s a simple FastAPI inference API for demonstration:

Copy

# app.py
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()

class PredictionRequest(BaseModel):
    input: str

@app.post("/predict")
async def predict(request: PredictionRequest):
    if not request.input:
        raise HTTPException(status_code=400, detail="Invalid input")
    # Mock model prediction logic
    prediction = len(request.input)  # Replace with actual ML model inference
    return {"prediction": prediction}

---

3. Writing Unit Tests

Create a tests/ directory and add your test files.

Example: tests/test_app.py

Copy

import pytest
from fastapi.testclient import TestClient
from app import app

client = TestClient(app)

def test_predict_success():
    response = client.post("/predict", json={"input": "test data"})
    assert response.status_code == 200
    assert response.json() == {"prediction": 9}

def test_predict_invalid_input():
    response = client.post("/predict", json={})
    assert response.status_code == 422  # Pydantic validation error

def test_predict_no_input():
    response = client.post("/predict", json=None)
    assert response.status_code == 422  # Pydantic validation error

Using Async Testing with httpx

For async endpoints, use httpx with pytest-asyncio.

Example: tests/test_app_async.py

Copy

import pytest
from httpx import AsyncClient
from app import app

@pytest.mark.asyncio
async def test_predict_success():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/predict", json={"input": "async test"})
    assert response.status_code == 200
    assert response.json() == {"prediction": 11}

@pytest.mark.asyncio
async def test_predict_invalid_input():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/predict", json={})
    assert response.status_code == 422

---

4. Running Tests

Run the tests using pytest:

Copy

pytest --asyncio-mode=auto --cov=app tests/

---

5. Mock External Dependencies

If your FastAPI app calls external APIs, you can mock them with libraries like unittest.mock.

Example: Mocking External API

Copy

# app.py
import httpx
from fastapi import FastAPI

app = FastAPI()

@app.get("/external")
async def call_external_api():
    response = await httpx.get("https://api.example.com/data")
    return response.json()

Test with mock:

Copy

import pytest
from unittest.mock import AsyncMock, patch
from httpx import AsyncClient
from app import app

@pytest.mark.asyncio
async def test_call_external_api():
    mock_response = AsyncMock()
    mock_response.json.return_value = {"key": "value"}
    
    with patch("httpx.get", return_value=mock_response):
        async with AsyncClient(app=app, base_url="http://test") as client:
            response = await client.get("/external")
        assert response.status_code == 200
        assert response.json() == {"key": "value"}

---

6. Coverage

To measure test coverage, run:

Copy

pytest --cov=app tests/

---

Best Practices

1. Use fixtures: Create reusable setup code for tests.

2. Test both positive and negative cases: Validate edge cases, errors, and success paths.

3. Mock dependencies: For external services, databases, or APIs, mock them using patch or unittest.mock.

4. Keep tests fast and isolated: Ensure tests don’t depend on each other.

5. Use async testing: Use pytest-asyncio for testing async routes.

---

3

More Testing Libs

To use modern Python testing libraries for writing unit tests in FastAPI, you can adopt these modern tools beyond the conventional pytest stack:

---

Modern Testing Libraries for FastAPI

1. pytest-httpx: A plugin to simplify mocking HTTPX requests.

2. pytest-mock: Built-in support for mocking with pytest.

3. anyio: An async compatibility layer for FastAPI testing.

4. pytest-randomly: Ensures test order independence by randomizing test execution order.

5. snapshottest: For snapshot testing (e.g., ensuring JSON responses don’t change unexpectedly).

6. schemathesis: Automates testing based on OpenAPI specifications.

7. pytest-docker: Helps test FastAPI services interacting with Docker containers.

Install them:

Copy

pip install pytest-httpx pytest-mock anyio pytest-randomly snapshottest schemathesis pytest-docker

---

Advanced Test Case with Modern Libraries

1. Mock HTTPX Requests with pytest-httpx

If your FastAPI app makes external API calls using HTTPX, mock them like this:

App Code:

Copy

# app.py
from fastapi import FastAPI
import httpx

app = FastAPI()

@app.get("/fetch-data")
async def fetch_data():
    response = await httpx.get("https://api.example.com/data")
    return response.json()

Test Code:

Copy

import pytest
from httpx import AsyncClient
from app import app

@pytest.mark.asyncio
async def test_fetch_data(httpx_mock):
    # Mock the HTTPX response
    httpx_mock.add_response(url="https://api.example.com/data", json={"key": "value"})

    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.get("/fetch-data")
        assert response.status_code == 200
        assert response.json() == {"key": "value"}

---

2. Snapshot Testing with snapshottest

Use snapshot testing to ensure API responses don’t change unintentionally.

Test Code:

Copy

import pytest
from snapshottest.pytest import PyTestSnapshotTest
from fastapi.testclient import TestClient
from app import app

client = TestClient(app)

def test_predict_snapshot(snapshot: PyTestSnapshotTest):
    response = client.post("/predict", json={"input": "test data"})
    assert response.status_code == 200
    # Capture and compare response with the snapshot
    snapshot.assert_match(response.json())

Run the test. On first run, it will save the snapshot. Subsequent runs will compare the response with the saved snapshot.

---

3. Testing OpenAPI with schemathesis

Automate testing based on OpenAPI specifications.

Copy

schemathesis run http://localhost:8000/openapi.json

You can also integrate it with pytest for more control:

Copy

import schemathesis
from app import app

schema = schemathesis.from_asgi(app)

@schema.parametrize()
def test_api(case):
    response = case.call_asgi()
    case.validate_response(response)

---

4. Async Context Management with anyio

Test async endpoints with seamless async context handling.

Test Code:

Copy

import anyio
from httpx import AsyncClient
from app import app

async def test_anyio_example():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/predict", json={"input": "async data"})
        assert response.status_code == 200
        assert response.json() == {"prediction": 10"}

anyio.run(test_anyio_example)

---

5. Randomized Testing with pytest-randomly

Ensure your tests don’t rely on execution order:

Copy

pip install pytest-randomly
pytest --randomly-seed=42

This is particularly useful to catch test order dependencies.

---

6. Test Containers with pytest-docker

If your FastAPI app interacts with databases or services running in Docker containers, test them in isolated containers.

Copy

import pytest
from pytest_docker.plugin import get_docker_services

@pytest.fixture(scope="module")
def api_service(docker_services):
    url = docker_services.port_for("api_container", 8000)
    return f"http://{url}"

def test_container_service(api_service):
    response = httpx.get(f"{api_service}/health")
    assert response.status_code == 200

---

Running Tests

Copy

pytest --asyncio-mode=auto --randomly-seed=42 --cov=app tests/

---

Best Practices with Modern Testing

1. Use snapshottest: To avoid accidental response changes.

2. Adopt pytest-httpx: For reliable external API mocking.

3. Leverage schemathesis: To ensure your FastAPI conforms to OpenAPI specs.

4. Parallelize tests: Use pytest-xdist for faster execution.

5. Randomize order: Avoid hidden dependencies with pytest-randomly.

---

4

Pytest Asyncio

For advanced testing with pytest-asyncio in a FastAPI application, you can incorporate features such as dependency injection, mocking databases, testing middleware, and testing WebSocket endpoints. Below are some advanced examples and patterns you can follow.

---

1. Testing Dependency Overrides

If your FastAPI app uses dependencies (e.g., database sessions, authentication), you can override these dependencies in tests.

App Code:

Copy

from fastapi import FastAPI, Depends, HTTPException
from pydantic import BaseModel

app = FastAPI()

# Dependency
async def get_database():
    return {"db": "connected"}

@app.post("/items/")
async def create_item(item: BaseModel, db=Depends(get_database)):
    if not db.get("db"):
        raise HTTPException(status_code=500, detail="Database not available")
    return {"item": item.dict(), "db_status": db["db"]}

Test Code:

Copy

import pytest
from httpx import AsyncClient
from app import app

# Override the dependency
async def override_get_database():
    return {"db": "mocked"}

app.dependency_overrides[get_database] = override_get_database

@pytest.mark.asyncio
async def test_dependency_override():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/items/", json={"name": "item1", "value": 10})
        assert response.status_code == 200
        assert response.json() == {"item": {"name": "item1", "value": 10}, "db_status": "mocked"}

---

2. Mocking Databases (e.g., SQLAlchemy)

For testing APIs interacting with a database, you can mock the database or use an in-memory SQLite database.

App Code:

Copy

from fastapi import Depends
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.asyncio import create_async_engine

DATABASE_URL = "sqlite+aiosqlite:///:memory:"

engine = create_async_engine(DATABASE_URL, echo=True)
Base = declarative_base()
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine, class_=AsyncSession)

async def get_db():
    async with SessionLocal() as session:
        yield session

Test Code:

Copy

from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.orm import sessionmaker
from app import app, get_db

# Create a test engine and session
test_engine = create_async_engine("sqlite+aiosqlite:///:memory:", echo=True)
TestSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=test_engine, class_=AsyncSession)

# Override the dependency
async def override_get_db():
    async with TestSessionLocal() as session:
        yield session

app.dependency_overrides[get_db] = override_get_db

@pytest.mark.asyncio
async def test_with_mocked_db():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.get("/some-endpoint")
        assert response.status_code == 200

---

3. Testing Middleware

If your app uses middleware (e.g., for authentication, CORS), you can test the middleware logic directly.

App Code:

Copy

from fastapi import FastAPI, Request

app = FastAPI()

@app.middleware("http")
async def custom_middleware(request: Request, call_next):
    response = await call_next(request)
    response.headers["X-Custom-Header"] = "CustomValue"
    return response

@app.get("/test")
async def test_endpoint():
    return {"message": "success"}

Test Code:

Copy

@pytest.mark.asyncio
async def test_custom_middleware():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.get("/test")
        assert response.status_code == 200
        assert response.headers["X-Custom-Header"] == "CustomValue"

---

4. Testing WebSocket Endpoints

FastAPI supports WebSocket connections. Here’s how to test them.

App Code:

Copy

from fastapi import FastAPI, WebSocket

app = FastAPI()

@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
    await websocket.accept()
    data = await websocket.receive_text()
    await websocket.send_text(f"Message received: {data}")
    await websocket.close()

Test Code:

Copy

import pytest
from httpx import AsyncClient

@pytest.mark.asyncio
async def test_websocket():
    async with AsyncClient(app=app, base_url="http://test") as client:
        async with client.websocket_connect("/ws") as websocket:
            await websocket.send_text("Hello")
            data = await websocket.receive_text()
            assert data == "Message received: Hello"

---

5. Testing Exception Handlers

FastAPI provides custom exception handlers. You can test them like this:

App Code:

Copy

from fastapi import FastAPI, HTTPException

app = FastAPI()

@app.exception_handler(HTTPException)
async def custom_exception_handler(request, exc):
    return JSONResponse({"error": exc.detail, "status_code": exc.status_code}, status_code=exc.status_code)

@app.get("/error")
async def error_endpoint():
    raise HTTPException(status_code=404, detail="Item not found")

Test Code:

Copy

@pytest.mark.asyncio
async def test_exception_handler():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.get("/error")
        assert response.status_code == 404
        assert response.json() == {"error": "Item not found", "status_code": 404}

---

6. Parameterized Tests with pytest.mark.parametrize

Run the same test logic with multiple inputs.

Test Code:

Copy

@pytest.mark.parametrize(
    "input_data,expected_status,expected_response",
    [
        ({"input": "test"}, 200, {"prediction": 4}),
        ({"input": ""}, 422, None),
        (None, 422, None),
    ],
)
@pytest.mark.asyncio
async def test_predict_parametrized(input_data, expected_status, expected_response):
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/predict", json=input_data)
        assert response.status_code == expected_status
        if expected_response:
            assert response.json() == expected_response

---

7. Coverage with pytest-cov

To measure test coverage:

Copy

pip install pytest-cov
pytest --asyncio-mode=auto --cov=app tests/

---

Summary

These advanced testing techniques can help ensure robust testing of your FastAPI application:

• Use dependency_overrides to mock dependencies.

• Mock databases or use an in-memory database.

• Test middleware logic and WebSocket connections.

• Test custom exception handlers and edge cases.

• Parameterize tests for multiple input combinations.

• Use tools like pytest-cov for test coverage metrics.

Let me know if you'd like help setting up any specific feature in detail!

---

5

PyTest for vanilla python methods

Writing unit tests for vanilla Python methods involves testing functions or methods that are independent of frameworks like FastAPI or Flask. The process is straightforward and focuses on pure logic, edge cases, and exceptions.

Here’s a guide to writing unit tests for vanilla Python methods using pytest and modern libraries like pytest-mock for mocking, pytest-parametrize for multiple test cases, and faker for generating test data.

---

1. Install Required Libraries

Install the required testing libraries:

Copy

pip install pytest pytest-mock faker

---

2. Sample Vanilla Python Methods

Suppose you have the following utility functions in a file called utils.py:

Copy

# utils.py
def add(a, b):
    """Adds two numbers."""
    return a + b

def is_palindrome(word):
    """Checks if a word is a palindrome."""
    word = word.lower().replace(" ", "")
    return word == word[::-1]

def divide(a, b):
    """Divides two numbers. Raises an exception for division by zero."""
    if b == 0:
        raise ValueError("Cannot divide by zero.")
    return a / b

---

3. Create Unit Tests

Create a tests/ directory and add the test file, e.g., tests/test_utils.py.

Test Cases for utils.py

Copy

import pytest
from utils import add, is_palindrome, divide

# Test add()
def test_add():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0
    assert add(0, 0) == 0

# Test is_palindrome()
@pytest.mark.parametrize(
    "word,expected",
    [
        ("racecar", True),
        ("hello", False),
        ("Madam", True),
        ("Was it a car or a cat I saw", True),
    ],
)
def test_is_palindrome(word, expected):
    assert is_palindrome(word) == expected

# Test divide()
def test_divide_success():
    assert divide(10, 2) == 5
    assert divide(-10, -2) == 5
    assert divide(10, -2) == -5

def test_divide_by_zero():
    with pytest.raises(ValueError, match="Cannot divide by zero."):
        divide(10, 0)

---

4. Using Mocks

If your methods depend on external calls or classes, mock them using pytest-mock.

For example, consider a method using a network call:

Copy

# utils.py
import requests

def fetch_data(url):
    """Fetch data from a URL."""
    response = requests.get(url)
    return response.json()

Test with Mocking:

Copy

from utils import fetch_data

def test_fetch_data(mocker):
    mock_response = mocker.Mock()
    mock_response.json.return_value = {"key": "value"}
    mocker.patch("utils.requests.get", return_value=mock_response)

    result = fetch_data("http://example.com")
    assert result == {"key": "value"}

---

5. Using Faker for Test Data

Generate realistic test data using faker.

Example with Faker:

Copy

from utils import is_palindrome
from faker import Faker

faker = Faker()

def test_random_strings():
    random_string = faker.word()
    assert not is_palindrome(random_string)  # Most random words are not palindromes

def test_random_palindromes():
    word = "radar"
    assert is_palindrome(word) == True

---

6. Edge Case and Boundary Testing

Test edge cases for robustness.

Example for Edge Cases:

Copy

# Edge cases for add()
def test_add_edge_cases():
    assert add(float("inf"), 1) == float("inf")
    assert add(float("-inf"), 1) == float("-inf")
    assert add(float("nan"), 1) != 1  # NaN is not equal to anything, including itself

# Edge cases for divide()
def test_divide_edge_cases():
    assert divide(0, 1) == 0
    with pytest.raises(ValueError):
        divide(1, 0)  # Division by zero

---

7. Run the Tests

Run tests with:

Copy

pytest --cov=utils tests/

---

8. Code Coverage

To measure test coverage, install pytest-cov and run:

Copy

pip install pytest-cov
pytest --cov=utils tests/

---

9. Best Practices for Testing Vanilla Methods

1. Test Logic Thoroughly:

   • Test positive and negative scenarios.

   • Test edge cases (e.g., empty strings, special characters, large numbers).

2. Use Mocking:

   • Mock external dependencies like API calls or file I/O.

3. Parameterize Tests:

   • Use @pytest.mark.parametrize to test multiple input combinations.

4. Readable Assertions:

   • Use pytest.raises for exceptions.

   • Keep assertions simple and meaningful.

5. Measure Coverage:

   • Ensure 80%+ coverage for utility methods.

---

6

Various PyTest Options

pytest offers a wide range of options and configurations to make testing powerful and flexible. Below is a categorized list of various pytest options and their usage:

---

1. Running Specific Tests

• Run a specific test file:

  Copy

  pytest tests/test_utils.py

• Run a specific test within a file:

  Copy

  pytest tests/test_utils.py::test_add

• Run tests matching a keyword:

  Copy

  pytest -k "add"

---

2. Markers

Markers are used to label or group tests:

• Run tests with a specific marker:

  Copy

  pytest -m "slow"

• Define a custom marker in pytest.ini:

  Copy

  [pytest]
  markers =
      slow: marks tests as slow

• Apply a marker to a test:

  Copy

  import pytest
  
  @pytest.mark.slow
  def test_slow_operation():
      assert 1 == 1

---

3. Test Reporting

• Show verbose output:

  Copy

  pytest -v

• Show detailed output for passed tests:

  Copy

  pytest -ra

• Generate an HTML report (requires pytest-html):

  Copy

  pytest --html=report.html

---

4. Test Coverage

• Measure code coverage (requires pytest-cov):

  Copy

  pytest --cov=your_module tests/

• Show uncovered lines:

  Copy

  pytest --cov=your_module --cov-report=term-missing

• Generate an HTML coverage report:

  Copy

  pytest --cov=your_module --cov-report=html

---

5. Running Tests in Parallel

• Install pytest-xdist for parallel testing:

  Copy

  pip install pytest-xdist

• Run tests using 4 parallel processes:

  Copy

  pytest -n 4

---

6. Test Selection

• Fail on the first error:

  Copy

  pytest -x

• Stop after N failures:

  Copy

  pytest --maxfail=3

• Run only failed tests from the last run:

  Copy

  pytest --lf

• Run a specific subset of tests by their node IDs:

  Copy

  pytest tests/test_file.py::test_function

---

7. Logging and Debugging

• Enable logging capture:

  Copy

  pytest --log-cli-level=INFO

• Debug with a specific test:

  Copy

  pytest --pdb tests/test_file.py

• Disable logging capture:

  Copy

  pytest -s

---

8. Test Failures

• Show detailed tracebacks on failure:

  Copy

  pytest --tb=long

• Shorten tracebacks:

  Copy

  pytest --tb=short

---

9. Randomizing Test Order

• Install pytest-randomly for random test order:

  Copy

  pip install pytest-randomly

• Randomize test execution order:

  Copy

  pytest --randomly-seed=42

---

10. Running Tests with Specific Parameters

• Pass custom CLI arguments:

  Copy

  pytest --my_custom_arg=value

• Access CLI arguments in test code:

  Copy

  def pytest_addoption(parser):
      parser.addoption("--my_custom_arg", action="store", default="default_value")
  
  @pytest.fixture
  def custom_arg(request):
      return request.config.getoption("--my_custom_arg")
  
  def test_with_custom_arg(custom_arg):
      assert custom_arg == "expected_value"

---

11. Fixtures

• Use a specific fixture:

  Copy

  pytest --setup-show

• Show all fixtures:

  Copy

  pytest --fixtures

---

12. Skipping and XFail

• Skip specific tests:

  Copy

  import pytest
  
  @pytest.mark.skip(reason="Not implemented yet")
  def test_skip():
      assert False

• Expected failures:

  Copy

  @pytest.mark.xfail(reason="Known bug")
  def test_xfail():
      assert False

---

13. Plugins

Install and use plugins for extended functionality:

• List all installed plugins:

  Copy

  pytest --trace-config

Popular plugins:

• pytest-django: For Django testing.

• pytest-flask: For Flask testing.

• pytest-httpx: For HTTPX mocking.

---

14. Output Control

• Disable warnings:

  Copy

  pytest -p no:warnings

• Only show errors:

  Copy

  pytest -q

---

15. Configuration with pytest.ini

Customize test behavior using a pytest.ini file:

Copy

[pytest]
addopts = -v --maxfail=3
markers =
    slow: marks tests as slow

---

16. Running Specific File Types

• Run doctests in .py files:

  Copy

  pytest --doctest-modules

• Run tests in .ipynb notebooks (requires pytest-ipynb):

  Copy

  pip install pytest-ipynb
  pytest tests/

---

Cheat Sheet Summary

OptionDescription

-v

Verbose output

-q

Quiet output

--maxfail=N

Stop after N failures

--tb=short

Short traceback

--lf

Run only failed tests

--cov=MODULE

Measure coverage

-n 4

Run tests in 4 parallel processes

-k "keyword"

Run tests matching a keyword

-m "marker"

Run tests with a specific marker

--html=report.html

Generate an HTML report

--log-cli-level=INFO

Enable logging

--randomly-seed=42

Randomize test order with a fixed seed

Let me know if you'd like examples for specific use cases!

---

7

How to hide warnings

To hide or suppress warnings during test runs in pytest, you can use the following methods:

---

1. Use pytest Options to Suppress Warnings

Run pytest with the --disable-warnings option:

Copy

pytest --disable-warnings

This suppresses all warnings, including those from external libraries.

---

2. Filter Specific Warnings

Use the -p no:warnings plugin to filter out warnings selectively.

For example:

Copy

pytest -p no:warnings

Or filter only specific warnings using filterwarnings:

Copy

pytest -W ignore::DeprecationWarning

---

3. Configure pytest.ini

Add a pytest.ini file to your project directory and configure it to suppress specific warnings:

Copy

[pytest]
filterwarnings =
    ignore::DeprecationWarning
    ignore:PydanticDeprecatedSince20
    ignore:The 'warn' function is deprecated

This ensures consistent behavior across all test runs.

---

4. Suppress Warnings in Test Code

Suppress warnings programmatically in your test code using the warnings module:

Copy

import warnings

warnings.filterwarnings("ignore", category=DeprecationWarning)

# Example to suppress a specific warning
warnings.filterwarnings("ignore", message="Support for class-based `config` is deprecated")

---

5. Suppress Warnings Globally in conftest.py

Create a conftest.py file in your test directory and configure global warning filters:

Copy

import warnings
import pytest

@pytest.fixture(autouse=True)
def suppress_warnings():
    warnings.filterwarnings("ignore", category=DeprecationWarning)
    warnings.filterwarnings("ignore", message="Support for class-based `config` is deprecated")
    warnings.filterwarnings("ignore", message="The 'warn' function is deprecated")

---

6. Upgrade Libraries

Some warnings, such as the ones related to Pydantic or logging, may be resolved by upgrading the libraries:

• Upgrade Pydantic to the latest version compatible with your project:

  Copy

  pip install --upgrade pydantic

• Update rs_microservice_core:

  Copy

  pip install --upgrade rs-microservice-core

---

7. Suppress Pydantic Warnings

If using Pydantic V2, switch from Config to ConfigDict as suggested in the warning message to eliminate the deprecation notice.

Old:

Copy

class MyModel(BaseModel):
    class Config:
        arbitrary_types_allowed = True

New:

Copy

from pydantic import BaseModel, ConfigDict

class MyModel(BaseModel):
    model_config = ConfigDict(arbitrary_types_allowed=True)

---

Summary of Commands

• Suppress all warnings:

  Copy

  pytest --disable-warnings

• Suppress specific warnings:

  Copy

  pytest -W ignore:PydanticDeprecatedSince20

• Configure warnings globally in pytest.ini:

  Copy

  filterwarnings =
    ignore:PydanticDeprecatedSince20

---

8

Hide warnings 2

To hide or suppress the warnings mentioned during your tests, you can use pytest options, configure filters in a pytest.ini file, or handle them programmatically. Here's how:

---

1. Suppress Warnings via pytest Options

Run pytest with the --disable-warnings option:

Copy

pytest --disable-warnings

This suppresses all warnings during the test run.

---

2. Suppress Specific Warnings Using pytest.ini

Create or modify a pytest.ini file in the root directory of your project and configure filterwarnings to suppress specific warnings:

Copy

[pytest]
filterwarnings =
    ignore::FutureWarning
    ignore::UserWarning

This ensures only the specified warnings are suppressed while allowing others to appear.

---

3. Suppress Specific Warnings Programmatically

You can suppress warnings programmatically in your test file using the warnings module.

Example:

Copy

import warnings

warnings.filterwarnings("ignore", category=FutureWarning)
warnings.filterwarnings("ignore", category=UserWarning)

def test_predict_success():
    response = client.post("/predict", json={"input": "test data"})
    assert response.status_code == 200
    assert response.json() == {"prediction": 9}

---

4. Suppress Warnings in a conftest.py File

If you want to suppress warnings globally for all tests, create or modify a conftest.py file in your test directory:

conftest.py:

Copy

import pytest
import warnings

@pytest.fixture(autouse=True)
def suppress_warnings():
    warnings.filterwarnings("ignore", category=FutureWarning)
    warnings.filterwarnings("ignore", category=UserWarning)

---

5. Fix the Code to Avoid Warnings

Instead of hiding warnings, consider updating the code to remove deprecated or incorrect practices:

Warning 1: DataFrame.applymap Deprecated

The warning suggests replacing applymap with map. Update the code in feature_engineering.py:

Copy

# Current Code
df_feature = df_feature.applymap(lambda value: value.strip() if isinstance(value, str) else str(value))

# Updated Code
df_feature = df_feature.map(lambda value: value.strip() if isinstance(value, str) else str(value))

Warning 2: MLPClassifier with Feature Names

Ensure the input data for the MLPClassifier matches the feature names used during training. If your input X doesn't have feature names, convert it to a DataFrame with appropriate column names before passing it to the model:

Copy

import pandas as pd

# Assuming 'features' is a list of column names used during training
X = pd.DataFrame(data, columns=features)
prediction = model.predict(X)

---

6. Verify Warning Suppression

After applying these fixes or suppressions, run the tests:

Copy

pytest --disable-warnings

Or:

Copy

pytest tests/test_app.py

You should no longer see the warnings.

---

9

Show print on console

In pytest, if print statements don't appear in the output by default, it’s because pytest captures standard output during test execution. You can control this behavior using the -s or --capture options.

---

How to Enable print Output in pytest

1. Use the -s Option

Run your tests with the -s option to disable output capturing:

Copy

pytest -s tests/test_app.py

This ensures that all print statements are displayed in the terminal.

---

2. Use the --capture=no Option

Alternatively, you can use the --capture=no option to achieve the same result:

Copy

pytest --capture=no tests/test_app.py

---

3. Log Instead of Print

If you frequently need to see output, it's better to use Python's logging module instead of print. pytest handles logging well and can show logs even with output capturing.

Example:

Copy

import logging

logging.basicConfig(level=logging.INFO)

def test_example():
    response = {"status_code": 422, "error": "Invalid input"}
    logging.info(f"trap1927: {response}")
    assert response["status_code"] == 422

Run the test with:

Copy

pytest --log-cli-level=INFO

---

4. Combine Print with Debugging (pdb)

You can also enter the debugger if you need to inspect values:

Copy

def test_example():
    response = {"status_code": 422, "error": "Invalid input"}
    assert response["status_code"] == 422

    import pdb; pdb.set_trace()
    print(f"trap1927: {response}")

When pdb is triggered, you can inspect variables and use the print command directly.

---

5. Display Output After Test Completion

You can ensure print outputs appear after the test runs by using the capsys fixture to explicitly access captured output:

Example:

Copy

def test_example(capsys):
    response = {"status_code": 422, "error": "Invalid input"}
    print(f"trap1927: {response}")
    assert response["status_code"] == 422

    # Access captured output
    captured = capsys.readouterr()
    assert "trap1927" in captured.out

---

10

Capsys

capsys is a built-in pytest fixture that captures output to sys.stdout and sys.stderr during test execution. It allows you to test and verify printed output or any text written to the standard output or error streams.

---

How It Works

When you use capsys in a test, it temporarily redirects sys.stdout and sys.stderr to capture anything printed during the test. After capturing, you can access the output and validate it using capsys.readouterr().

---

Example Usage

1. Capturing print Output

Copy

def test_print_output(capsys):
    print("Hello, World!")
    captured = capsys.readouterr()  # Capture output
    assert captured.out == "Hello, World!\n"  # Validate stdout

2. Capturing Both stdout and stderr

Copy

def test_stdout_and_stderr(capsys):
    print("This is standard output")
    import sys
    print("This is standard error", file=sys.stderr)
    
    captured = capsys.readouterr()
    assert captured.out == "This is standard output\n"  # Validate stdout
    assert captured.err == "This is standard error\n"  # Validate stderr

---

Methods Provided by capsys

MethodDescription

capsys.readouterr()

Returns a tuple-like object with out (stdout) and err (stderr) strings.

capsys.disabled()

Temporarily disables capturing (useful for debugging).

---

3. Disabling capsys Temporarily

If you want to disable output capturing for debugging, you can use capsys.disabled():

Copy

def test_disable_capture(capsys):
    with capsys.disabled():
        print("This will appear in the terminal during the test")

---

4. Example: Validating Function Output

Here’s how you can use capsys to test a function that prints output:

Function to Test

Copy

def greet(name):
    print(f"Hello, {name}!")

Test Case

Copy

def test_greet_function(capsys):
    greet("Alice")
    captured = capsys.readouterr()
    assert captured.out == "Hello, Alice!\n"

---

5. Capturing Subprocess Output

You can also use capsys to capture output from subprocesses:

Example

Copy

import subprocess

def test_subprocess_output(capsys):
    subprocess.run(["echo", "Hello from subprocess"])
    captured = capsys.readouterr()
    assert "Hello from subprocess" in captured.out

---

6. Common Use Cases

• Validating print Statements: Ensure the printed output is correct.

• Testing Logging Output: Verify logs written to sys.stdout or sys.stderr.

• Debugging Subprocess Output: Capture and check output from child processes.

---

Integration with Pytest

When you use capsys, it seamlessly integrates into your pytest test functions. There's no need for manual setup or teardown—it’s managed automatically.

---

11

Examples

https://github.com/pluralsight/intro-to-pytest https://gist.github.com/jakelevi1996/2d249adbbd2e13950852b80cca42ed02 https://github.com/pytest-dev/pytest/issues https://github.com/armakuni/pytest-examples https://github.com/hectorcanto/pytest-samples https://github.com/armakuni/pytest-examples https://gist.github.com/hectorcanto/40a7ecbd9e02b0550840f850165bc162 https://gist.github.com/jw-ng/cc07a9e323477bd58c49327107bd8c3b https://gist.github.com/jakelevi1996/2d249adbbd2e13950852b80cca42ed02 https://github.com/htorrence/pytest_examples https://docs.pytest.org/en/stable/example/index.html

---

12

---

13

---

14

---

15

---

16

---

17

---

18

---

19

---

20

---

21

---

22

---

23

---

24

---

25

---

PreviousSimple CRUD

Last updated 5 months ago1

PyTest for Flask

To write modern unit test cases for your Flask Inference API, you can use the following libraries:

1. pytest: A powerful testing framework for Python.

2. pytest-flask: A Flask-specific extension for pytest to simplify testing Flask applications.

3. requests-mock: For mocking external API calls.

4. factory-boy or faker: For generating test data.

5. coverage: For measuring test coverage.

Here’s a step-by-step guide to write unit test cases:

1. Set Up a Test Environment

Ensure your Flask app is modular and can be initialized easily for testing. For example:

Copy

# app.py
from flask import Flask, jsonify, request

app = Flask(__name__)

@app.route('/predict', methods=['POST'])
def predict():
    data = request.json
    if not data or 'input' not in data:
        return jsonify({'error': 'Invalid input'}), 400
    # Mock model inference logic
    prediction = len(data['input'])  # Replace with actual model prediction
    return jsonify({'prediction': prediction})

if __name__ == "__main__":
    app.run(debug=True)

2. Install Required Libraries

Install libraries:

Copy

pip install pytest pytest-flask requests-mock faker

3. Create Test Cases

Create a tests/ directory and add test files.

Example: tests/test_app.py

Copy

import pytest
from app import app

@pytest.fixture
def client():
    with app.test_client() as client:
        yield client

def test_predict_success(client):
    response = client.post('/predict', json={'input': 'test data'})
    assert response.status_code == 200
    assert response.json == {'prediction': 9}

def test_predict_invalid_input(client):
    response = client.post('/predict', json={})
    assert response.status_code == 400
    assert response.json == {'error': 'Invalid input'}

def test_predict_no_input(client):
    response = client.post('/predict', json=None)
    assert response.status_code == 400
    assert response.json == {'error': 'Invalid input'}

4. Run the Tests

Run tests using pytest:

Copy

pytest --cov=app tests/

5. Use Mocks for External Calls

If your API calls an external service, mock it using requests-mock.

Example: Mocking an External API

Copy

import requests
from flask import Flask, jsonify

app = Flask(__name__)

@app.route('/external', methods=['GET'])
def external_call():
    response = requests.get('https://api.example.com/data')
    return jsonify(response.json()), response.status_code

Test with mock:

Copy

import pytest
import requests_mock
from app import app

@pytest.fixture
def client():
    with app.test_client() as client:
        yield client

@requests_mock.Mocker()
def test_external_call(mock, client):
    mock.get('https://api.example.com/data', json={'key': 'value'}, status_code=200)
    response = client.get('/external')
    assert response.status_code == 200
    assert response.json == {'key': 'value'}

6. Measure Coverage

To measure test coverage, use:

Copy

pytest --cov=app tests/

Best Practices

1. Use fixtures for setting up the test environment.

2. Use mocks for external dependencies.

3. Keep test cases independent.

4. Aim for high coverage (80%+).

5. Validate both positive and negative scenarios.

---

2

PyTest for FastAPI

To write unit test cases for your FastAPI Inference API using modern testing libraries, you can follow these steps:

---

1. Recommended Libraries

• pytest: A robust testing framework.

• httpx: For making async HTTP requests.

• pytest-asyncio: To test FastAPI’s async endpoints.

• fastapi.testclient: For synchronous testing of FastAPI routes.

• coverage: To measure code coverage.

Install the required libraries:

Copy

pip install pytest pytest-asyncio httpx fastapi[all] coverage

---

2. Example FastAPI Inference API

Here’s a simple FastAPI inference API for demonstration:

Copy

# app.py
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI()

class PredictionRequest(BaseModel):
    input: str

@app.post("/predict")
async def predict(request: PredictionRequest):
    if not request.input:
        raise HTTPException(status_code=400, detail="Invalid input")
    # Mock model prediction logic
    prediction = len(request.input)  # Replace with actual ML model inference
    return {"prediction": prediction}

---

3. Writing Unit Tests

Create a tests/ directory and add your test files.

Example: tests/test_app.py

Copy

import pytest
from fastapi.testclient import TestClient
from app import app

client = TestClient(app)

def test_predict_success():
    response = client.post("/predict", json={"input": "test data"})
    assert response.status_code == 200
    assert response.json() == {"prediction": 9}

def test_predict_invalid_input():
    response = client.post("/predict", json={})
    assert response.status_code == 422  # Pydantic validation error

def test_predict_no_input():
    response = client.post("/predict", json=None)
    assert response.status_code == 422  # Pydantic validation error

Using Async Testing with httpx

For async endpoints, use httpx with pytest-asyncio.

Example: tests/test_app_async.py

Copy

import pytest
from httpx import AsyncClient
from app import app

@pytest.mark.asyncio
async def test_predict_success():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/predict", json={"input": "async test"})
    assert response.status_code == 200
    assert response.json() == {"prediction": 11}

@pytest.mark.asyncio
async def test_predict_invalid_input():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/predict", json={})
    assert response.status_code == 422

---

4. Running Tests

Run the tests using pytest:

Copy

pytest --asyncio-mode=auto --cov=app tests/

---

5. Mock External Dependencies

If your FastAPI app calls external APIs, you can mock them with libraries like unittest.mock.

Example: Mocking External API

Copy

# app.py
import httpx
from fastapi import FastAPI

app = FastAPI()

@app.get("/external")
async def call_external_api():
    response = await httpx.get("https://api.example.com/data")
    return response.json()

Test with mock:

Copy

import pytest
from unittest.mock import AsyncMock, patch
from httpx import AsyncClient
from app import app

@pytest.mark.asyncio
async def test_call_external_api():
    mock_response = AsyncMock()
    mock_response.json.return_value = {"key": "value"}
    
    with patch("httpx.get", return_value=mock_response):
        async with AsyncClient(app=app, base_url="http://test") as client:
            response = await client.get("/external")
        assert response.status_code == 200
        assert response.json() == {"key": "value"}

---

6. Coverage

To measure test coverage, run:

Copy

pytest --cov=app tests/

---

Best Practices

1. Use fixtures: Create reusable setup code for tests.

2. Test both positive and negative cases: Validate edge cases, errors, and success paths.

3. Mock dependencies: For external services, databases, or APIs, mock them using patch or unittest.mock.

4. Keep tests fast and isolated: Ensure tests don’t depend on each other.

5. Use async testing: Use pytest-asyncio for testing async routes.

---

3

More Testing Libs

To use modern Python testing libraries for writing unit tests in FastAPI, you can adopt these modern tools beyond the conventional pytest stack:

---

Modern Testing Libraries for FastAPI

1. pytest-httpx: A plugin to simplify mocking HTTPX requests.

2. pytest-mock: Built-in support for mocking with pytest.

3. anyio: An async compatibility layer for FastAPI testing.

4. pytest-randomly: Ensures test order independence by randomizing test execution order.

5. snapshottest: For snapshot testing (e.g., ensuring JSON responses don’t change unexpectedly).

6. schemathesis: Automates testing based on OpenAPI specifications.

7. pytest-docker: Helps test FastAPI services interacting with Docker containers.

Install them:

Copy

pip install pytest-httpx pytest-mock anyio pytest-randomly snapshottest schemathesis pytest-docker

---

Advanced Test Case with Modern Libraries

1. Mock HTTPX Requests with pytest-httpx

If your FastAPI app makes external API calls using HTTPX, mock them like this:

App Code:

Copy

# app.py
from fastapi import FastAPI
import httpx

app = FastAPI()

@app.get("/fetch-data")
async def fetch_data():
    response = await httpx.get("https://api.example.com/data")
    return response.json()

Test Code:

Copy

import pytest
from httpx import AsyncClient
from app import app

@pytest.mark.asyncio
async def test_fetch_data(httpx_mock):
    # Mock the HTTPX response
    httpx_mock.add_response(url="https://api.example.com/data", json={"key": "value"})

    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.get("/fetch-data")
        assert response.status_code == 200
        assert response.json() == {"key": "value"}

---

2. Snapshot Testing with snapshottest

Use snapshot testing to ensure API responses don’t change unintentionally.

Test Code:

Copy

import pytest
from snapshottest.pytest import PyTestSnapshotTest
from fastapi.testclient import TestClient
from app import app

client = TestClient(app)

def test_predict_snapshot(snapshot: PyTestSnapshotTest):
    response = client.post("/predict", json={"input": "test data"})
    assert response.status_code == 200
    # Capture and compare response with the snapshot
    snapshot.assert_match(response.json())

Run the test. On first run, it will save the snapshot. Subsequent runs will compare the response with the saved snapshot.

---

3. Testing OpenAPI with schemathesis

Automate testing based on OpenAPI specifications.

Copy

schemathesis run http://localhost:8000/openapi.json

You can also integrate it with pytest for more control:

Copy

import schemathesis
from app import app

schema = schemathesis.from_asgi(app)

@schema.parametrize()
def test_api(case):
    response = case.call_asgi()
    case.validate_response(response)

---

4. Async Context Management with anyio

Test async endpoints with seamless async context handling.

Test Code:

Copy

import anyio
from httpx import AsyncClient
from app import app

async def test_anyio_example():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/predict", json={"input": "async data"})
        assert response.status_code == 200
        assert response.json() == {"prediction": 10"}

anyio.run(test_anyio_example)

---

5. Randomized Testing with pytest-randomly

Ensure your tests don’t rely on execution order:

Copy

pip install pytest-randomly
pytest --randomly-seed=42

This is particularly useful to catch test order dependencies.

---

6. Test Containers with pytest-docker

If your FastAPI app interacts with databases or services running in Docker containers, test them in isolated containers.

Copy

import pytest
from pytest_docker.plugin import get_docker_services

@pytest.fixture(scope="module")
def api_service(docker_services):
    url = docker_services.port_for("api_container", 8000)
    return f"http://{url}"

def test_container_service(api_service):
    response = httpx.get(f"{api_service}/health")
    assert response.status_code == 200

---

Running Tests

Copy

pytest --asyncio-mode=auto --randomly-seed=42 --cov=app tests/

---

Best Practices with Modern Testing

1. Use snapshottest: To avoid accidental response changes.

2. Adopt pytest-httpx: For reliable external API mocking.

3. Leverage schemathesis: To ensure your FastAPI conforms to OpenAPI specs.

4. Parallelize tests: Use pytest-xdist for faster execution.

5. Randomize order: Avoid hidden dependencies with pytest-randomly.

---

4

Pytest Asyncio

For advanced testing with pytest-asyncio in a FastAPI application, you can incorporate features such as dependency injection, mocking databases, testing middleware, and testing WebSocket endpoints. Below are some advanced examples and patterns you can follow.

---

1. Testing Dependency Overrides

If your FastAPI app uses dependencies (e.g., database sessions, authentication), you can override these dependencies in tests.

App Code:

Copy

from fastapi import FastAPI, Depends, HTTPException
from pydantic import BaseModel

app = FastAPI()

# Dependency
async def get_database():
    return {"db": "connected"}

@app.post("/items/")
async def create_item(item: BaseModel, db=Depends(get_database)):
    if not db.get("db"):
        raise HTTPException(status_code=500, detail="Database not available")
    return {"item": item.dict(), "db_status": db["db"]}

Test Code:

Copy

import pytest
from httpx import AsyncClient
from app import app

# Override the dependency
async def override_get_database():
    return {"db": "mocked"}

app.dependency_overrides[get_database] = override_get_database

@pytest.mark.asyncio
async def test_dependency_override():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/items/", json={"name": "item1", "value": 10})
        assert response.status_code == 200
        assert response.json() == {"item": {"name": "item1", "value": 10}, "db_status": "mocked"}

---

2. Mocking Databases (e.g., SQLAlchemy)

For testing APIs interacting with a database, you can mock the database or use an in-memory SQLite database.

App Code:

Copy

from fastapi import Depends
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.asyncio import create_async_engine

DATABASE_URL = "sqlite+aiosqlite:///:memory:"

engine = create_async_engine(DATABASE_URL, echo=True)
Base = declarative_base()
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine, class_=AsyncSession)

async def get_db():
    async with SessionLocal() as session:
        yield session

Test Code:

Copy

from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.orm import sessionmaker
from app import app, get_db

# Create a test engine and session
test_engine = create_async_engine("sqlite+aiosqlite:///:memory:", echo=True)
TestSessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=test_engine, class_=AsyncSession)

# Override the dependency
async def override_get_db():
    async with TestSessionLocal() as session:
        yield session

app.dependency_overrides[get_db] = override_get_db

@pytest.mark.asyncio
async def test_with_mocked_db():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.get("/some-endpoint")
        assert response.status_code == 200

---

3. Testing Middleware

If your app uses middleware (e.g., for authentication, CORS), you can test the middleware logic directly.

App Code:

Copy

from fastapi import FastAPI, Request

app = FastAPI()

@app.middleware("http")
async def custom_middleware(request: Request, call_next):
    response = await call_next(request)
    response.headers["X-Custom-Header"] = "CustomValue"
    return response

@app.get("/test")
async def test_endpoint():
    return {"message": "success"}

Test Code:

Copy

@pytest.mark.asyncio
async def test_custom_middleware():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.get("/test")
        assert response.status_code == 200
        assert response.headers["X-Custom-Header"] == "CustomValue"

---

4. Testing WebSocket Endpoints

FastAPI supports WebSocket connections. Here’s how to test them.

App Code:

Copy

from fastapi import FastAPI, WebSocket

app = FastAPI()

@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
    await websocket.accept()
    data = await websocket.receive_text()
    await websocket.send_text(f"Message received: {data}")
    await websocket.close()

Test Code:

Copy

import pytest
from httpx import AsyncClient

@pytest.mark.asyncio
async def test_websocket():
    async with AsyncClient(app=app, base_url="http://test") as client:
        async with client.websocket_connect("/ws") as websocket:
            await websocket.send_text("Hello")
            data = await websocket.receive_text()
            assert data == "Message received: Hello"

---

5. Testing Exception Handlers

FastAPI provides custom exception handlers. You can test them like this:

App Code:

Copy

from fastapi import FastAPI, HTTPException

app = FastAPI()

@app.exception_handler(HTTPException)
async def custom_exception_handler(request, exc):
    return JSONResponse({"error": exc.detail, "status_code": exc.status_code}, status_code=exc.status_code)

@app.get("/error")
async def error_endpoint():
    raise HTTPException(status_code=404, detail="Item not found")

Test Code:

Copy

@pytest.mark.asyncio
async def test_exception_handler():
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.get("/error")
        assert response.status_code == 404
        assert response.json() == {"error": "Item not found", "status_code": 404}

---

6. Parameterized Tests with pytest.mark.parametrize

Run the same test logic with multiple inputs.

Test Code:

Copy

@pytest.mark.parametrize(
    "input_data,expected_status,expected_response",
    [
        ({"input": "test"}, 200, {"prediction": 4}),
        ({"input": ""}, 422, None),
        (None, 422, None),
    ],
)
@pytest.mark.asyncio
async def test_predict_parametrized(input_data, expected_status, expected_response):
    async with AsyncClient(app=app, base_url="http://test") as client:
        response = await client.post("/predict", json=input_data)
        assert response.status_code == expected_status
        if expected_response:
            assert response.json() == expected_response

---

7. Coverage with pytest-cov

To measure test coverage:

Copy

pip install pytest-cov
pytest --asyncio-mode=auto --cov=app tests/

---

Summary

These advanced testing techniques can help ensure robust testing of your FastAPI application:

• Use dependency_overrides to mock dependencies.

• Mock databases or use an in-memory database.

• Test middleware logic and WebSocket connections.

• Test custom exception handlers and edge cases.

• Parameterize tests for multiple input combinations.

• Use tools like pytest-cov for test coverage metrics.

Let me know if you'd like help setting up any specific feature in detail!

---

5

PyTest for vanilla python methods

Writing unit tests for vanilla Python methods involves testing functions or methods that are independent of frameworks like FastAPI or Flask. The process is straightforward and focuses on pure logic, edge cases, and exceptions.

Here’s a guide to writing unit tests for vanilla Python methods using pytest and modern libraries like pytest-mock for mocking, pytest-parametrize for multiple test cases, and faker for generating test data.

---

1. Install Required Libraries

Install the required testing libraries:

Copy

pip install pytest pytest-mock faker

---

2. Sample Vanilla Python Methods

Suppose you have the following utility functions in a file called utils.py:

Copy

# utils.py
def add(a, b):
    """Adds two numbers."""
    return a + b

def is_palindrome(word):
    """Checks if a word is a palindrome."""
    word = word.lower().replace(" ", "")
    return word == word[::-1]

def divide(a, b):
    """Divides two numbers. Raises an exception for division by zero."""
    if b == 0:
        raise ValueError("Cannot divide by zero.")
    return a / b

---

3. Create Unit Tests

Create a tests/ directory and add the test file, e.g., tests/test_utils.py.

Test Cases for utils.py

Copy

import pytest
from utils import add, is_palindrome, divide

# Test add()
def test_add():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0
    assert add(0, 0) == 0

# Test is_palindrome()
@pytest.mark.parametrize(
    "word,expected",
    [
        ("racecar", True),
        ("hello", False),
        ("Madam", True),
        ("Was it a car or a cat I saw", True),
    ],
)
def test_is_palindrome(word, expected):
    assert is_palindrome(word) == expected

# Test divide()
def test_divide_success():
    assert divide(10, 2) == 5
    assert divide(-10, -2) == 5
    assert divide(10, -2) == -5

def test_divide_by_zero():
    with pytest.raises(ValueError, match="Cannot divide by zero."):
        divide(10, 0)

---

4. Using Mocks

If your methods depend on external calls or classes, mock them using pytest-mock.

For example, consider a method using a network call:

Copy

# utils.py
import requests

def fetch_data(url):
    """Fetch data from a URL."""
    response = requests.get(url)
    return response.json()

Test with Mocking:

Copy

from utils import fetch_data

def test_fetch_data(mocker):
    mock_response = mocker.Mock()
    mock_response.json.return_value = {"key": "value"}
    mocker.patch("utils.requests.get", return_value=mock_response)

    result = fetch_data("http://example.com")
    assert result == {"key": "value"}

---

5. Using Faker for Test Data

Generate realistic test data using faker.

Example with Faker:

Copy

from utils import is_palindrome
from faker import Faker

faker = Faker()

def test_random_strings():
    random_string = faker.word()
    assert not is_palindrome(random_string)  # Most random words are not palindromes

def test_random_palindromes():
    word = "radar"
    assert is_palindrome(word) == True

---

6. Edge Case and Boundary Testing

Test edge cases for robustness.

Example for Edge Cases:

Copy

# Edge cases for add()
def test_add_edge_cases():
    assert add(float("inf"), 1) == float("inf")
    assert add(float("-inf"), 1) == float("-inf")
    assert add(float("nan"), 1) != 1  # NaN is not equal to anything, including itself

# Edge cases for divide()
def test_divide_edge_cases():
    assert divide(0, 1) == 0
    with pytest.raises(ValueError):
        divide(1, 0)  # Division by zero

---

7. Run the Tests

Run tests with:

Copy

pytest --cov=utils tests/

---

8. Code Coverage

To measure test coverage, install pytest-cov and run:

Copy

pip install pytest-cov
pytest --cov=utils tests/

---

9. Best Practices for Testing Vanilla Methods

1. Test Logic Thoroughly:

   • Test positive and negative scenarios.

   • Test edge cases (e.g., empty strings, special characters, large numbers).

2. Use Mocking:

   • Mock external dependencies like API calls or file I/O.

3. Parameterize Tests:

   • Use @pytest.mark.parametrize to test multiple input combinations.

4. Readable Assertions:

   • Use pytest.raises for exceptions.

   • Keep assertions simple and meaningful.

5. Measure Coverage:

   • Ensure 80%+ coverage for utility methods.

---

6

Various PyTest Options

pytest offers a wide range of options and configurations to make testing powerful and flexible. Below is a categorized list of various pytest options and their usage:

---

1. Running Specific Tests

• Run a specific test file:

  Copy

  pytest tests/test_utils.py

• Run a specific test within a file:

  Copy

  pytest tests/test_utils.py::test_add

• Run tests matching a keyword:

  Copy

  pytest -k "add"

---

2. Markers

Markers are used to label or group tests:

• Run tests with a specific marker:

  Copy

  pytest -m "slow"

• Define a custom marker in pytest.ini:

  Copy

  [pytest]
  markers =
      slow: marks tests as slow

• Apply a marker to a test:

  Copy

  import pytest
  
  @pytest.mark.slow
  def test_slow_operation():
      assert 1 == 1

---

3. Test Reporting

• Show verbose output:

  Copy

  pytest -v

• Show detailed output for passed tests:

  Copy

  pytest -ra

• Generate an HTML report (requires pytest-html):

  Copy

  pytest --html=report.html

---

4. Test Coverage

• Measure code coverage (requires pytest-cov):

  Copy

  pytest --cov=your_module tests/

• Show uncovered lines:

  Copy

  pytest --cov=your_module --cov-report=term-missing

• Generate an HTML coverage report:

  Copy

  pytest --cov=your_module --cov-report=html

---

5. Running Tests in Parallel

• Install pytest-xdist for parallel testing:

  Copy

  pip install pytest-xdist

• Run tests using 4 parallel processes:

  Copy

  pytest -n 4

---

6. Test Selection

• Fail on the first error:

  Copy

  pytest -x

• Stop after N failures:

  Copy

  pytest --maxfail=3

• Run only failed tests from the last run:

  Copy

  pytest --lf

• Run a specific subset of tests by their node IDs:

  Copy

  pytest tests/test_file.py::test_function

---

7. Logging and Debugging

• Enable logging capture:

  Copy

  pytest --log-cli-level=INFO

• Debug with a specific test:

  Copy

  pytest --pdb tests/test_file.py

• Disable logging capture:

  Copy

  pytest -s

---

8. Test Failures

• Show detailed tracebacks on failure:

  Copy

  pytest --tb=long

• Shorten tracebacks:

  Copy

  pytest --tb=short

---

9. Randomizing Test Order

• Install pytest-randomly for random test order:

  Copy

  pip install pytest-randomly

• Randomize test execution order:

  Copy

  pytest --randomly-seed=42

---

10. Running Tests with Specific Parameters

• Pass custom CLI arguments:

  Copy

  pytest --my_custom_arg=value

• Access CLI arguments in test code:

  Copy

  def pytest_addoption(parser):
      parser.addoption("--my_custom_arg", action="store", default="default_value")
  
  @pytest.fixture
  def custom_arg(request):
      return request.config.getoption("--my_custom_arg")
  
  def test_with_custom_arg(custom_arg):
      assert custom_arg == "expected_value"

---

11. Fixtures

• Use a specific fixture:

  Copy

  pytest --setup-show

• Show all fixtures:

  Copy

  pytest --fixtures

---

12. Skipping and XFail

• Skip specific tests:

  Copy

  import pytest
  
  @pytest.mark.skip(reason="Not implemented yet")
  def test_skip():
      assert False

• Expected failures:

  Copy

  @pytest.mark.xfail(reason="Known bug")
  def test_xfail():
      assert False

---

13. Plugins

Install and use plugins for extended functionality:

• List all installed plugins:

  Copy

  pytest --trace-config

Popular plugins:

• pytest-django: For Django testing.

• pytest-flask: For Flask testing.

• pytest-httpx: For HTTPX mocking.

---

14. Output Control

• Disable warnings:

  Copy

  pytest -p no:warnings

• Only show errors:

  Copy

  pytest -q

---

15. Configuration with pytest.ini

Customize test behavior using a pytest.ini file:

Copy

[pytest]
addopts = -v --maxfail=3
markers =
    slow: marks tests as slow

---

16. Running Specific File Types

• Run doctests in .py files:

  Copy

  pytest --doctest-modules

• Run tests in .ipynb notebooks (requires pytest-ipynb):

  Copy

  pip install pytest-ipynb
  pytest tests/

---

Cheat Sheet Summary

OptionDescription

-v

Verbose output

-q

Quiet output

--maxfail=N

Stop after N failures

--tb=short

Short traceback

--lf

Run only failed tests

--cov=MODULE

Measure coverage

-n 4

Run tests in 4 parallel processes

-k "keyword"

Run tests matching a keyword

-m "marker"

Run tests with a specific marker

--html=report.html

Generate an HTML report

--log-cli-level=INFO

Enable logging

--randomly-seed=42

Randomize test order with a fixed seed

Let me know if you'd like examples for specific use cases!

---

7

How to hide warnings

To hide or suppress warnings during test runs in pytest, you can use the following methods:

---

1. Use pytest Options to Suppress Warnings

Run pytest with the --disable-warnings option:

Copy

pytest --disable-warnings

This suppresses all warnings, including those from external libraries.

---

2. Filter Specific Warnings

Use the -p no:warnings plugin to filter out warnings selectively.

For example:

Copy

pytest -p no:warnings

Or filter only specific warnings using filterwarnings:

Copy

pytest -W ignore::DeprecationWarning

---

3. Configure pytest.ini

Add a pytest.ini file to your project directory and configure it to suppress specific warnings:

Copy

[pytest]
filterwarnings =
    ignore::DeprecationWarning
    ignore:PydanticDeprecatedSince20
    ignore:The 'warn' function is deprecated

This ensures consistent behavior across all test runs.

---

4. Suppress Warnings in Test Code

Suppress warnings programmatically in your test code using the warnings module:

Copy

import warnings

warnings.filterwarnings("ignore", category=DeprecationWarning)

# Example to suppress a specific warning
warnings.filterwarnings("ignore", message="Support for class-based `config` is deprecated")

---

5. Suppress Warnings Globally in conftest.py

Create a conftest.py file in your test directory and configure global warning filters:

Copy

import warnings
import pytest

@pytest.fixture(autouse=True)
def suppress_warnings():
    warnings.filterwarnings("ignore", category=DeprecationWarning)
    warnings.filterwarnings("ignore", message="Support for class-based `config` is deprecated")
    warnings.filterwarnings("ignore", message="The 'warn' function is deprecated")

---

6. Upgrade Libraries

Some warnings, such as the ones related to Pydantic or logging, may be resolved by upgrading the libraries:

• Upgrade Pydantic to the latest version compatible with your project:

  Copy

  pip install --upgrade pydantic

• Update rs_microservice_core:

  Copy

  pip install --upgrade rs-microservice-core

---

7. Suppress Pydantic Warnings

If using Pydantic V2, switch from Config to ConfigDict as suggested in the warning message to eliminate the deprecation notice.

Old:

Copy

class MyModel(BaseModel):
    class Config:
        arbitrary_types_allowed = True

New:

Copy

from pydantic import BaseModel, ConfigDict

class MyModel(BaseModel):
    model_config = ConfigDict(arbitrary_types_allowed=True)

---

Summary of Commands

• Suppress all warnings:

  Copy

  pytest --disable-warnings

• Suppress specific warnings:

  Copy

  pytest -W ignore:PydanticDeprecatedSince20

• Configure warnings globally in pytest.ini:

  Copy

  filterwarnings =
    ignore:PydanticDeprecatedSince20

---

8

Hide warnings 2

To hide or suppress the warnings mentioned during your tests, you can use pytest options, configure filters in a pytest.ini file, or handle them programmatically. Here's how:

---

1. Suppress Warnings via pytest Options

Run pytest with the --disable-warnings option:

Copy

pytest --disable-warnings

This suppresses all warnings during the test run.

---

2. Suppress Specific Warnings Using pytest.ini

Create or modify a pytest.ini file in the root directory of your project and configure filterwarnings to suppress specific warnings:

Copy

[pytest]
filterwarnings =
    ignore::FutureWarning
    ignore::UserWarning

This ensures only the specified warnings are suppressed while allowing others to appear.

---

3. Suppress Specific Warnings Programmatically

You can suppress warnings programmatically in your test file using the warnings module.

Example:

Copy

import warnings

warnings.filterwarnings("ignore", category=FutureWarning)
warnings.filterwarnings("ignore", category=UserWarning)

def test_predict_success():
    response = client.post("/predict", json={"input": "test data"})
    assert response.status_code == 200
    assert response.json() == {"prediction": 9}

---

4. Suppress Warnings in a conftest.py File

If you want to suppress warnings globally for all tests, create or modify a conftest.py file in your test directory:

conftest.py:

Copy

import pytest
import warnings

@pytest.fixture(autouse=True)
def suppress_warnings():
    warnings.filterwarnings("ignore", category=FutureWarning)
    warnings.filterwarnings("ignore", category=UserWarning)

---

5. Fix the Code to Avoid Warnings

Instead of hiding warnings, consider updating the code to remove deprecated or incorrect practices:

Warning 1: DataFrame.applymap Deprecated

The warning suggests replacing applymap with map. Update the code in feature_engineering.py:

Copy

# Current Code
df_feature = df_feature.applymap(lambda value: value.strip() if isinstance(value, str) else str(value))

# Updated Code
df_feature = df_feature.map(lambda value: value.strip() if isinstance(value, str) else str(value))

Warning 2: MLPClassifier with Feature Names

Ensure the input data for the MLPClassifier matches the feature names used during training. If your input X doesn't have feature names, convert it to a DataFrame with appropriate column names before passing it to the model:

Copy

import pandas as pd

# Assuming 'features' is a list of column names used during training
X = pd.DataFrame(data, columns=features)
prediction = model.predict(X)

---

6. Verify Warning Suppression

After applying these fixes or suppressions, run the tests:

Copy

pytest --disable-warnings

Or:

Copy

pytest tests/test_app.py

You should no longer see the warnings.

---

9

Show print on console

In pytest, if print statements don't appear in the output by default, it’s because pytest captures standard output during test execution. You can control this behavior using the -s or --capture options.

---

How to Enable print Output in pytest

1. Use the -s Option

Run your tests with the -s option to disable output capturing:

Copy

pytest -s tests/test_app.py

This ensures that all print statements are displayed in the terminal.

---

2. Use the --capture=no Option

Alternatively, you can use the --capture=no option to achieve the same result:

Copy

pytest --capture=no tests/test_app.py

---

3. Log Instead of Print

If you frequently need to see output, it's better to use Python's logging module instead of print. pytest handles logging well and can show logs even with output capturing.

Example:

Copy

import logging

logging.basicConfig(level=logging.INFO)

def test_example():
    response = {"status_code": 422, "error": "Invalid input"}
    logging.info(f"trap1927: {response}")
    assert response["status_code"] == 422

Run the test with:

Copy

pytest --log-cli-level=INFO

---

4. Combine Print with Debugging (pdb)

You can also enter the debugger if you need to inspect values:

Copy

def test_example():
    response = {"status_code": 422, "error": "Invalid input"}
    assert response["status_code"] == 422

    import pdb; pdb.set_trace()
    print(f"trap1927: {response}")

When pdb is triggered, you can inspect variables and use the print command directly.

---

5. Display Output After Test Completion

You can ensure print outputs appear after the test runs by using the capsys fixture to explicitly access captured output:

Example:

Copy

def test_example(capsys):
    response = {"status_code": 422, "error": "Invalid input"}
    print(f"trap1927: {response}")
    assert response["status_code"] == 422

    # Access captured output
    captured = capsys.readouterr()
    assert "trap1927" in captured.out

---

10

Capsys

capsys is a built-in pytest fixture that captures output to sys.stdout and sys.stderr during test execution. It allows you to test and verify printed output or any text written to the standard output or error streams.

---

How It Works

When you use capsys in a test, it temporarily redirects sys.stdout and sys.stderr to capture anything printed during the test. After capturing, you can access the output and validate it using capsys.readouterr().

---

Example Usage

1. Capturing print Output

Copy

def test_print_output(capsys):
    print("Hello, World!")
    captured = capsys.readouterr()  # Capture output
    assert captured.out == "Hello, World!\n"  # Validate stdout

2. Capturing Both stdout and stderr

Copy

def test_stdout_and_stderr(capsys):
    print("This is standard output")
    import sys
    print("This is standard error", file=sys.stderr)
    
    captured = capsys.readouterr()
    assert captured.out == "This is standard output\n"  # Validate stdout
    assert captured.err == "This is standard error\n"  # Validate stderr

---

Methods Provided by capsys

MethodDescription

capsys.readouterr()

Returns a tuple-like object with out (stdout) and err (stderr) strings.

capsys.disabled()

Temporarily disables capturing (useful for debugging).

---

3. Disabling capsys Temporarily

If you want to disable output capturing for debugging, you can use capsys.disabled():

Copy

def test_disable_capture(capsys):
    with capsys.disabled():
        print("This will appear in the terminal during the test")

---

4. Example: Validating Function Output

Here’s how you can use capsys to test a function that prints output:

Function to Test

Copy

def greet(name):
    print(f"Hello, {name}!")

Test Case

Copy

def test_greet_function(capsys):
    greet("Alice")
    captured = capsys.readouterr()
    assert captured.out == "Hello, Alice!\n"

---

5. Capturing Subprocess Output

You can also use capsys to capture output from subprocesses:

Example

Copy

import subprocess

def test_subprocess_output(capsys):
    subprocess.run(["echo", "Hello from subprocess"])
    captured = capsys.readouterr()
    assert "Hello from subprocess" in captured.out

---

6. Common Use Cases

• Validating print Statements: Ensure the printed output is correct.

• Testing Logging Output: Verify logs written to sys.stdout or sys.stderr.

• Debugging Subprocess Output: Capture and check output from child processes.

---

Integration with Pytest

When you use capsys, it seamlessly integrates into your pytest test functions. There's no need for manual setup or teardown—it’s managed automatically.

---

11

Examples

https://github.com/pluralsight/intro-to-pytest https://gist.github.com/jakelevi1996/2d249adbbd2e13950852b80cca42ed02 https://github.com/pytest-dev/pytest/issues https://github.com/armakuni/pytest-examples https://github.com/hectorcanto/pytest-samples https://github.com/armakuni/pytest-examples https://gist.github.com/hectorcanto/40a7ecbd9e02b0550840f850165bc162 https://gist.github.com/jw-ng/cc07a9e323477bd58c49327107bd8c3b https://gist.github.com/jakelevi1996/2d249adbbd2e13950852b80cca42ed02 https://github.com/htorrence/pytest_examples https://docs.pytest.org/en/stable/example/index.html

---

12

---

13

---

14

---

15

---

16

---

17

---

18

---

19

---

20

---

21

---

22

---

23

---

24

---

25

---