Introduction

This blog post will explain what a resource cache is, why you might want one, and showcase how I developed one for my renderer.

Why is a resource cache useful?

Inside our renderer, there are various resources (such as textures, models, and Vulkan objects). These can be expensive to create (or acquire), and if they are reused then storing (or caching) them can improve performance which is critical for real-time systems such as a renderer.

They also might need to be used across multiple systems and/or multiple threads within our engine, and having to implement management and thread-safety for each system that uses the resource can quickly turn into a scalability problem.

A resource cache solves this by adding a way where we can manage our resources and keep them loaded in to avoid expensive re-acquisitions.

You should consider implementing a resource cache when:

• Your application has resources that are used by multiple systems
• Your “resources” are costly to build (e.g. loading a texture from a file isn’t instantaneous)
• Your application requires thread-safety over your resources

Requesting a resource asks the resource cache “Does this resource exist?”. If it does it returns the resource to us, and if it does not it creates it. For example, if we wanted to request a texture. The first request would create, load-in, and store the texture object inside the resource cache. Then in each subsequent draw call, or anywhere else in your application for that matter, that requests the same texture it simply returns the texture object that is already stored there.

What is a resource cache?

A resource cache is a type of management system for your application resources. When a resource becomes expensive to acquire (or create) by your system, a resource cache is something that you should look to implement.

It has a pool of memory where it can store specific objects. It is designed so that you cannot directly access the data that is stored inside of it.

The resource cache owns the data that exists inside of it, making it great for objects that need to be managed by multiple systems. This way a resource cache acts as a universal resource manager.

Implementation

The resource cache will make use of polymorphic types and hashing which I will assume you have an understanding of both.

First off we will implement our ResourceCache object:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

class ResourceCache
{
  public:
    ResourceCache() = default;
    ~ResourceCache() = default;

    ResourceCache(const ResourceCache &) = delete;
    ResourceCache &operator=(const ResourceCache &) = delete;

    ResourceCache(ResourceCache &&) = delete;
    ResourceCache &operator=(ResourceCache &&) = delete;

};

We make sure we delete the copy constructor and assignment operator to avoid accidentally copying the cache and duplicating potentially huge areas of memory (incurring a cost).

We also delete the move constructor and assignment operator as the resource cache will exist in one area of the system and never need to be moved.

Next, we need to implement a member variable to own and store our resources:

1
2
3
4
5
6
7
8

class ResourceCache
{
  // ...

  private:
    std::unordered_map<std::type_index, std::unordered_map<size_t, std::unique_ptr<CacheableResource>>> state;

}

We use a hash map (unordered_map) to map typeid()’s of class’s to another nested hash map. The nested hash map then maps resource hashes to a polymorphic base type, which we have called CacheableResource.

Using a polymorphic base type is nice because it means we can mark the class’s of resources that we want to be cached, preventing the developer from accidentally caching anything.

Since the state type is also quite long winded, we can use a typedef to simplify it like so:

1

using CacheState = std::unordered_map<std::type_index, std::unordered_map<size_t, std::unique_ptr<CacheableResource>>>;

The reasons we separate our hash tables by type and not just store all our CacheableResource’s into a global hash map is because:

• We reduce our (already slim) chance of a hash conflict
• Two different types of resources may use identical constructor parameters. For example, imagine a Texture and Model class that takes a filename as its sole constructor parameter.
• Thread-safe access (we will get on to this later).

CacheableResource is our polymorphic base type. We inherit from this in the classes we want to make cacheable by the resource cache.

Our base class:

1
2
3
4
5
6

class CacheableResource
{
  public:
    Cacheable() = default;
    virtual ~Cacheable() = default;
};

And that’s it! We want an empty base class that we can use to mark our resources as cacheable. Just make sure you mark the destructor as virtual, as for a class to be considered as polymorphic (and therefore can be used in inheritance) at least one of its functions needs to be virtual.

Next, we need to write a public function in our resource cache so that the application can request resources.

If we go back to our ResourceCache we can create a templated function that can take in an argument pack and return a reference to the resource, like so:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

class ResourceCache
{
  public:

    // ...

    template <class T, class... A>
    T &request_resource(A &...args)
    {
        // Do stuff
    }
}

Here T is the requested resource type, and A is the argument pack.

Using this would look like so:

1

auto &texture = resource_cache.request_resource<Texture>("textures/image.png");

We should implement a static assertion so that we can check if the developer is trying to request a class type that isn’t derived from CacheableResource to catch any run-time errors at compile-time - I much prefer a compile-time error over a run-time error.

1
2
3
4
5

template <class T, class... A>
T &request_resource(A &...args)
{
    static_assert(std::is_base_of<CacheableResource, T>::value, "T must be derived from CacheableResource");
}

Now our application won’t compile if the function is misused.

So we have checked the type of resource the caller is requesting, but before we can go on to requesting our resources we need to be able to tell our resources apart from one another. To do this we make use of hashing.

Hashing

We check if the resource exists by hashing the argument inputs:

1
2
3
4
5
6
7
8

template <class T, class... Args>
T &request_resource(Args &...args)
{
    // ...

    size_t seed{0};
    hash_args(seed, args...);
}

The seed integer will be a unique identifier that describes the arguments that were passed to the request_resource function. This is our hash, and we will use this as our index into the hash map in the resource cache state.

We make use of C++ argument packing here to account for varying arguments for the different resource class’s. Where you see args..., the compiler will “unpack” the arguments and so hash_parameters(hash, args...) would turn into hash_parameters(hash, arg0, arg1, arg2, arg3).

We can write the following two functions to recursively handle combining the hashes of argument packs.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

template <typename T>
inline void hash_args(size_t &seed, const T &arg)
{
    avn::hash(seed, arg);
}

template <typename T, typename... Args>
inline void hash_args(size_t &seed, const T &first_arg, const Args &...next_args)
{
    hash_args(seed, first_arg);

    hash_args(seed, next_args...);
}

So in the case of hash_args(seed, arg0, arg1, arg2, arg3) this would call the second function in the above snippet. Where arg0 gets passed into first_arg, and then arg1, arg2 and arg3 are repacked into the next_args argument pack.

The first argument is hashed by calling the avn::hash function which calls the actual hashing algorithm and combines it into our seed integer (which we take from the glm maths library) (see next code block).

The second argument then recursively continues this cycle. hash_args(seed, next_args...) gets unpacked into hash_args(seed, arg1, arg2, arg3), where arg1 is hashed and arg2 and arg3 are repacked.

avn::hash definition:

1
2
3
4
5
6

template <class T>
inline void hash(size_t &seed, const T &arg)
{
    std::hash<T> hasher;
    glm::detail::hash_combine(seed, hasher(arg));
}

Note that you will need to make sure that you add a new std::hash<> function definition for each user-defined type in your application so that the hash function understands how to combine the hashes of its member variables.

Now that our resource cache can uniquely identify the resources we want, we can actually request a resource.

Here is the full function, with comments to help explain it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

template <class T, class... Args>
T &request_resource(Args &...args)
{
    // Retrieve a hash ID of the arguments
    size_t seed{0};
    hash_args(seed, args...);

    // Grab a reference to the hash map for the particular type of resource we want
    auto &resource_hash_map = state[typeid(T)];

    // Search the resource hash map for the existance of the requested resource
    auto resource_it = resource_hash_map.find(seed);

    // If resource is found, then we return it as a reference
    if (resource_it != resource_hash_map.end())
    {
        return *dynamic_cast<T *>(resource_it->second.get());
    }

    // Otherwise if the resource was not found, we create it (this can be expensive, but we only need to do it once)
    auto new_resource = std::make_unique<T>(args...);

    // Move it to the hash map so that the cache owns it
    auto &it = resource_hash_map.emplace(seed, std::move(new_resource));

    // We return it as a reference
    return *dynamic_cast<T *>(it->second.get());
}

And there we have it, a resource cache!

Thread saftey

There is one last thing though. What happens when you have multiple threads trying to request resources? You could be potentially writing and reading from the same memory (the state) which can have undefined behavior or memory corruption.

We use a synchronization object called a mutex. This will prevent multiple threads from all writing/reading from the same data.

To do this we can guard our entire request_resource call:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

template <class T, class... Args>
T &request_resource(Args &...args)
{
    auto &mutex = mutexes[typeid(T)];

    std::lock_guard<std::mutex> guard(mutex);

    T &resource = request_resource_implementation<T>(args...);

    return resource;
}

Where mutexes is a new member variable that stores the mutex state for each valid typeid(). We define it as std::unordered_map<std::type_index, std::mutex> mutexes;.

This will lock multiple threads from requesting a resource of the same type.

For example, when a thread first requests a resource it will lock the mutex. If a second thread comes along and tries to request a resource of the same type, it will hang until the original thread releases the mutex (i.e. when the lock_guard is destroyed at the end of the scope).