130. Serialization with pickle

The pickle module in Python is used to serialize (convert a Python object into a byte stream) and deserialize (reconstruct the Python object from the byte stream) Python objects. It is useful for saving objects to files or sending them over a network. Below are examples that demonstrate how to use pickle for serializing and deserializing Python objects.

1. Basic Serialization (Pickling)

This example shows how to serialize a Python object into a byte stream and store it in a file.

Copy

import pickle

# Sample Python object (dictionary)
data = {'name': 'John', 'age': 30, 'city': 'New York'}

# Serialize the object and save it to a file
with open('data.pickle', 'wb') as file:
    pickle.dump(data, file)

print("Object serialized and saved to data.pickle")

---

2. Basic Deserialization (Unpickling)

This example shows how to read a serialized object from a file and deserialize it back into a Python object.

Copy

import pickle

# Read the serialized object from the file
with open('data.pickle', 'rb') as file:
    loaded_data = pickle.load(file)

print("Object deserialized from data.pickle:")
print(loaded_data)

---

3. Serializing a Custom Class Object

This example demonstrates how to pickle and unpickle an object of a custom class.

Copy

import pickle

# Define a custom class
class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

# Create an instance of the class
person = Person('Alice', 25)

# Serialize the object to a file
with open('person.pickle', 'wb') as file:
    pickle.dump(person, file)

print("Custom class object serialized")

# Deserialize the object
with open('person.pickle', 'rb') as file:
    loaded_person = pickle.load(file)

print("Custom class object deserialized:")
print(f"Name: {loaded_person.name}, Age: {loaded_person.age}")

---

4. Serializing Multiple Objects

You can serialize multiple objects in a single file.

Copy

import pickle

# Objects to serialize
data1 = {'item1': 100, 'item2': 200}
data2 = [1, 2, 3, 4, 5]

# Serialize multiple objects to a file
with open('multiple_objects.pickle', 'wb') as file:
    pickle.dump(data1, file)
    pickle.dump(data2, file)

print("Multiple objects serialized to multiple_objects.pickle")

# Deserialize multiple objects
with open('multiple_objects.pickle', 'rb') as file:
    loaded_data1 = pickle.load(file)
    loaded_data2 = pickle.load(file)

print("Deserialized objects:")
print(loaded_data1)
print(loaded_data2)

---

5. Pickle with File Handling

You can directly use pickle to handle large files or data.

Copy

import pickle

# Sample data
large_data = {'numbers': list(range(1000000))}

# Serialize to a file
with open('large_data.pickle', 'wb') as file:
    pickle.dump(large_data, file)

# Deserialize from the file
with open('large_data.pickle', 'rb') as file:
    deserialized_data = pickle.load(file)

print(f"First 10 numbers from deserialized data: {deserialized_data['numbers'][:10]}")

---

6. Pickle with String (Memory Buffer)

Using pickle to serialize objects into a memory buffer instead of writing to a file.

Copy

import pickle
import io

# Create a memory buffer
buffer = io.BytesIO()

# Serialize object to the memory buffer
data = {'key': 'value'}
pickle.dump(data, buffer)

# Get the byte data
serialized_data = buffer.getvalue()
print(f"Serialized Data (byte format): {serialized_data}")

# Reset the buffer's position to the beginning
buffer.seek(0)

# Deserialize the object
deserialized_data = pickle.load(buffer)
print("Deserialized Data:", deserialized_data)

---

7. Pickle Protocol

The pickle module supports different serialization protocols. The default is protocol 3, but you can use different protocols for compatibility or efficiency.

Copy

import pickle

# Sample data
data = {'name': 'Bob', 'age': 45}

# Serialize using protocol 4 (for Python 3.4 and above)
with open('data_protocol4.pickle', 'wb') as file:
    pickle.dump(data, file, protocol=4)

print("Object serialized using protocol 4")

---

8. Pickle with Try-Except for Error Handling

Error handling when loading a pickled file.

Copy

import pickle

# Deserialize data with error handling
try:
    with open('non_existent.pickle', 'rb') as file:
        data = pickle.load(file)
    print(data)
except FileNotFoundError:
    print("Error: The file does not exist.")
except pickle.UnpicklingError:
    print("Error: There was an issue deserializing the data.")

---

9. Avoiding Pickle Insecurity

Since pickle can execute arbitrary code during deserialization, it’s unsafe to unpickle data from untrusted sources. Always validate the data source.

Copy

import pickle

# Untrusted data source
untrusted_data = b"cos\nsystem\n(S'echo unsafe code executed'\ntR."

# Attempt to load the unsafe pickle data
try:
    result = pickle.loads(untrusted_data)
except pickle.UnpicklingError:
    print("Error: Data is unsafe to unpickle")

---

10. Using pickle with Python's with Statement

Ensure that files are properly closed after pickling by using with statements.

Copy

import pickle

# Object to be serialized
data = {"foo": "bar", "baz": "qux"}

# Serialize the object to a file using the 'with' statement
with open("data_with.pickle", "wb") as file:
    pickle.dump(data, file)

# Deserialize the object from the file using the 'with' statement
with open("data_with.pickle", "rb") as file:
    loaded_data = pickle.load(file)

print("Deserialized data:", loaded_data)

---

Key Points:

• Pickle allows serialization of complex Python objects to a byte stream, which can then be saved or transmitted.

• pickle.dump() serializes and saves an object to a file.

• pickle.load() deserializes and loads an object from a file.

• Pickling custom objects: Custom classes can also be serialized and deserialized.

• Protocols: Pickle supports different serialization protocols for backward compatibility.

• Security: Avoid unpickling data from untrusted sources due to potential code execution risks.