Log in

tipfy.ext.sharded_counter

This extension provides a counter using sharded writes to prevent contention. It is based on Bill Katz' sharded_counter project.

The Counter object should be used for counters that handle a lot of concurrent use. It follows pattern described in a Google I/O talk.

Memcache is used for caching counts and if a cached count is available, it is the most correct. If there are datastore put issues, we store the un-put values into a delayed_incr memcache that will be applied as soon as the next shard put is successful. Changes will only be lost if we lose memcache before a successful datastore shard put or there's a failure/error in memcache.

Example usage

from tipfy.ext.sharded_counter import Counter

# Build a new counter that uses the unique key name 'hits'.
hits = Counter('hits')
# Increment by 1.
hits.increment()
# Increment by 10.
hits.increment(10)
# Decrement by 3.
hits.increment(-3)
# This is the current count.
my_hits = hits.count
# Forces fetching a non-cached count of all shards.
hits.get_count(nocache=True)  
# Set the counter to an arbitrary value.
hits.count = 6

Configuration

By default 10 shards are used by the counter, but you can change this amount in the configuration:

config['tipfy.ext.sharded_counter'] = {
    'shards': 20,
}

To change it in a per counter basis, extend Counter:

from tipfy.ext.sharded_counter import Counter

class MyCounter(Counter):
    shards = 20

hits = MyCounter('hits')

Extension Reference

Powered by Moe. Yeah, the name is Moe. Powered by Google App Engine