124. Batch Processing with itertools.islice

Here are 10 Python code snippets demonstrating how to use itertools.islice for batch processing and handling data in chunks efficiently:

1. Basic Usage of itertools.islice

Copy

import itertools

# Sample data
data = range(1, 11)

# Slice the data in chunks
chunk_size = 3
sliced_data = itertools.islice(data, 0, chunk_size)

print(list(sliced_data))  # Output: [1, 2, 3]

---

2. Batch Processing with itertools.islice

Copy

import itertools

# Sample data
data = range(1, 11)

# Batch processing in chunks of 4
chunk_size = 4
for i in range(0, len(data), chunk_size):
    batch = itertools.islice(data, i, i + chunk_size)
    print(list(batch))  # Output: [1, 2, 3, 4], [5, 6, 7, 8], [9, 10]

---

3. Processing Large File in Chunks

Copy

import itertools

# Generator to read large file line by line
def read_file_in_chunks(file_path, chunk_size=5):
    with open(file_path, 'r') as file:
        for chunk in iter(lambda: list(itertools.islice(file, chunk_size)), []):
            yield chunk

# Example usage
# for chunk in read_file_in_chunks('large_file.txt', 10):
#     print(chunk)

---

4. Using islice for Infinite Data Streams

Copy

import itertools

# Infinite data stream (example with range)
data = itertools.count(1)  # Infinite sequence starting from 1

# Take the first 5 numbers in the infinite stream
sliced_data = itertools.islice(data, 0, 5)
print(list(sliced_data))  # Output: [1, 2, 3, 4, 5]

---

5. Iterating Over a List in Chunks Using itertools.islice

Copy

import itertools

# List to process
data = list(range(1, 21))

# Slice data into chunks of 6
chunk_size = 6
for chunk in itertools.zip_longest(*[iter(data)]*chunk_size):
    print(list(chunk))  # Output: [1, 2, 3, 4, 5, 6], [7, 8, 9, 10, 11, 12], etc.

---

6. Skipping Elements with itertools.islice

Copy

import itertools

# Sample data
data = range(1, 11)

# Skip the first 3 elements and process the rest in chunks of 2
chunk_size = 2
sliced_data = itertools.islice(data, 3, None)
for batch in itertools.zip_longest(*[iter(sliced_data)]*chunk_size):
    print(list(batch))  # Output: [4, 5], [6, 7], [8, 9], [10]

---

7. Batch Processing Data with Padding

Copy

import itertools

# Sample data
data = range(1, 10)

# Slice data into chunks of 4 and pad with None if not complete
chunk_size = 4
for chunk in itertools.zip_longest(*[iter(data)]*chunk_size, fillvalue=None):
    print(list(chunk))  # Output: [1, 2, 3, 4], [5, 6, 7, 8], [9, None, None, None]

---

8. Reading Large Data in Fixed-Size Chunks

Copy

import itertools

# Function to read a list in fixed-size chunks
def chunk_data(data, chunk_size):
    for chunk in itertools.zip_longest(*[iter(data)]*chunk_size):
        yield list(chunk)

data = range(1, 16)
for chunk in chunk_data(data, 5):
    print(chunk)  # Output: [1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15]

---

9. Sliding Window Technique with itertools.islice

Copy

import itertools

# Sample data
data = range(1, 6)

# Sliding window of size 3
window_size = 3
windows = itertools.islice(data, 0, window_size)
for window in windows:
    print(window)  # Output: (1, 2, 3), (2, 3, 4), etc.

---

10. Using islice to Limit Data from an Infinite Generator

Copy

import itertools

# Infinite generator
data = itertools.cycle(['A', 'B', 'C'])

# Slice to limit to the first 7 elements
sliced_data = itertools.islice(data, 7)
print(list(sliced_data))  # Output: ['A', 'B', 'C', 'A', 'B', 'C', 'A']

---

These examples show how itertools.islice can be effectively used to process large datasets, files, or streams in chunks, without needing to load everything into memory at once, which helps in batch processing and improving memory efficiency.