Package SCons :: Module Memoize
[hide private]
[frames] | no frames]

Module Memoize

source code

Memoizer

A decorator-based implementation to count hits and misses of the computed values that various methods cache in memory.

Use of this modules assumes that wrapped methods be coded to cache their values in a consistent way. In particular, it requires that the class uses a dictionary named "_memo" to store the cached values.

Here is an example of wrapping a method that returns a computed value, with no input parameters:

@SCons.Memoize.CountMethodCall
def foo(self):

    try:                                                    # Memoization
        return self._memo['foo']                            # Memoization
    except KeyError:                                        # Memoization
        pass                                                # Memoization

    result = self.compute_foo_value()

    self._memo['foo'] = result                              # Memoization

    return result

Here is an example of wrapping a method that will return different values based on one or more input arguments:

def _bar_key(self, argument):                               # Memoization
    return argument                                         # Memoization

@SCons.Memoize.CountDictCall(_bar_key)
def bar(self, argument):

    memo_key = argument                                     # Memoization
    try:                                                    # Memoization
        memo_dict = self._memo['bar']                       # Memoization
    except KeyError:                                        # Memoization
        memo_dict = {}                                      # Memoization
        self._memo['dict'] = memo_dict                      # Memoization
    else:                                                   # Memoization
        try:                                                # Memoization
            return memo_dict[memo_key]                      # Memoization
        except KeyError:                                    # Memoization
            pass                                            # Memoization

    result = self.compute_bar_value(argument)

    memo_dict[memo_key] = result                            # Memoization

    return result

Deciding what to cache is tricky, because different configurations can have radically different performance tradeoffs, and because the tradeoffs involved are often so non-obvious. Consequently, deciding whether or not to cache a given method will likely be more of an art than a science, but should still be based on available data from this module. Here are some VERY GENERAL guidelines about deciding whether or not to cache return values from a method that's being called a lot:

-- The first question to ask is, "Can we change the calling code
so this method isn't called so often?" Sometimes this can be done by changing the algorithm. Sometimes the caller should be memoized, not the method you're looking at.

—The memoized function should be timed with multiple configurations to make sure it doesn't inadvertently slow down some other configuration.

-- When memoizing values based on a dictionary key composed of
input arguments, you don't need to use all of the arguments if some of them don't affect the return values.
Classes [hide private]
  Counter
Base class for counting memoization hits and misses.
  CountValue
A counter class for simple, atomic memoized values.
  CountDict
A counter class for memoized values stored in a dictionary, with keys based on the method's input arguments.
Functions [hide private]
 
Dump(title=None)
Dump the hit/miss count for all the counters collected so far.
source code
 
EnableMemoization() source code
 
CountMethodCall(fn)
Decorator for counting memoizer hits/misses while retrieving a simple value in a class method. It wraps the given method fn and uses a CountValue object to keep track of the caching statistics. Wrapping gets enabled by calling EnableMemoization().
source code
 
CountDictCall(keyfunc)
Decorator for counting memoizer hits/misses while accessing dictionary values with a key-generating function. Like CountMethodCall above, it wraps the given method fn and uses a CountDict object to keep track of the caching statistics. The dict-key function keyfunc has to get passed in the decorator call and gets stored in the CountDict instance. Wrapping gets enabled by calling EnableMemoization().
source code
Variables [hide private]
  __revision__ = 'src/engine/SCons/Memoize.py rel_3.0.0:4395:897...
  __doc__ = """Memoi...
  use_memoizer = None
hash(x)
  CounterList = {}
  __package__ = 'SCons'
Variables Details [hide private]

__revision__

Value:
'src/engine/SCons/Memoize.py rel_3.0.0:4395:8972f6a2f699 2017/09/18 12\
:59:24 bdbaddog'

__doc__

Value:
"""Memoizer

A decorator-based implementation to count hits and misses of the compu\
ted
values that various methods cache in memory.

Use of this modules assumes that wrapped methods be coded to cache the\
ir
...