Reusable Caching Primitives (tendril.utils.www.caching)

TODO Some introduction

class tendril.utils.www.caching.CacheBase(cache_dir='/home/docs/.tendril/cache/soupcache')[source]

Bases: object

This class implements a simple filesystem cache which can be used to create and obtain from various cached requests from internet resources.

The cache is stored in the folder defined by cache_dir, with a filename constructed by the _get_filepath() function.

If the cache’s _accessor() function is called with the getcpath attribute set to True, only the path to a (valid) file in the cache filesystem is returned, and opening and reading the file is left to the caller. This hook is provided to help deal with file encoding on a somewhat case-by-case basis, until the overall encoding problems can be ironed out.

_get_filepath(*args, **kwargs)[source]

Given the parameters necessary to obtain the resource in normal circumstances, return a hash which is usable as the filename for the resource in the cache.

The filename must be unique for every resource, and filename generation must be deterministic and repeatable.

Must be implemented in every subclass.

_get_fresh_content(*args, **kwargs)[source]

Given the parameters necessary to obtain the resource in normal circumstances, obtain the content of the resource from the source.

Must be implemented in every subclass.

static _serialize(response)[source]

Given a response (as returned by _get_fresh_content()), convert it into a string which can be stored in a file. Use this function to serialize structured responses when needed.

Unless overridden by the subclass, this function simply returns the response unaltered.

The actions of this function should be reversed by _deserialize().

static _deserialize(filecontent)[source]

Given the contents of a cache file, reconstruct the original response in the original format (as returned by _get_fresh_content()). Use this function to deserialize cache files for structured responses when needed.

Unless overridden by the subclass, this function simply returns the file content unaltered.

The actions of this function should be reversed by _serialize().

_cached_exists(filepath)[source]
_is_cache_fresh(filepath, max_age)[source]

Given the path to a file in the cache and the maximum age for the cache content to be considered fresh, returns (boolean) whether or not the cache contains a fresh copy of the response.

Parameters
  • filepath – Path to the filename in the cache corresponding to the request, as returned by _get_filepath().

  • max_age – Maximum age of fresh content, in seconds.

_accessor(max_age, getcpath=False, *args, **kwargs)[source]

The primary accessor for the cache instance. Each subclass should provide a function which behaves similarly to that of the original, un-cached version of the resource getter. That function should adapt the parameters provided to it into the form needed for this one, and let this function maintain the cached responses and handle retrieval of the response.

If the module’s _internet_connected is set to False, the cached value is returned regardless.