How to define custom serializers / unserializers?

Introduction

All values stored in the cache (with low-level API or with high-level decorators) are serialized into bytes before being stored in redis.

By default, with use the standard pickle module to serialize/unserialize.

But you may want to override this default setting:

• to compress the data inside the cache
• to avoid the serialization/unserialization process overhead (if you store only bytes in the cache)
• ...

How to compress the data inside the cache

import pickle
import zlib
from typing import Any

from rtc import RedisTaggedCache


def custom_serializer(value: Any) -> bytes:
    """Custom serializer is a simple function to convert any value to bytes.

    If an exception is raised here, a warning will be logged
    and the cache will be bypassed.

    """
    serialized = pickle.dumps(value)  # serialize the value
    return zlib.compress(serialized)  # compress the serialized value


def custom_unserializer(value: bytes) -> Any:
    """Custom unserializer is a simple function to convert bytes to a value.

    If an exception is raised here, a warning will be logged
    and the cache will be bypassed.

    """
    serialized = zlib.decompress(value)  # decompress the value
    return pickle.loads(serialized)  # unserialize the value


cache = RedisTaggedCache(
    namespace="foo",
    host="localhost",
    port=6379,
    serializer=custom_serializer,
    unserializer=custom_unserializer,
)

# Use cache normally
value = ["data", "to", "store"]
cache.set("key1", value, tags=["tag1", "tag2"])

How to bypass the serialization

Warning to the high level decorators!

If you are sure to use only bytes as values in the cache low-level API, be careful about the high-level decorators that automatically cache the returned value of functions/methods. In that particular case, they also have to return bytes!

from typing import Any

from rtc import RedisTaggedCache


def custom_serializer(value: Any) -> bytes:
    """Custom serializer is a simple function to convert any value to bytes.

    Here, we consider that value is already bytes.

    If an exception is raised here, a warning will be logged
    and the cache will be bypassed.

    """
    try:
        value.decode()
    except (AttributeError, UnicodeDecodeError):
        raise TypeError("The value must be bytes")
    # value is bytes
    return value  # type: ignore


cache = RedisTaggedCache(
    namespace="foo",
    host="localhost",
    port=6379,
    serializer=custom_serializer,
    unserializer=lambda x: x,  # we don't need any transformation in that particular case
)

# Use cache normally
value_as_bytes = b"foo"
cache.set("key1", value_as_bytes, tags=["tag1", "tag2"])