10:c104b23c85f0
Anton Shestakov <engored@ya.ru>, Sat, 11 Jan 2014 23:41:32 +0900
qmonitor.

next change 11:251971044ff2

qmonitor/README.md

Permissions: -rw-r--r--

Other formats: Feeds:
One of the new features in SQLAlchemy 0.9 is event removal API. Basically, if
you've added an event listener, this change makes you able to remove it.
Although, to be honest, I suspect there was a way to remove event listeners
before 0.9, but it was probably complicated or undocumented.
With this change you can now do a context manager that can watch queries
executed for an orbitrary (small) block of code without leaving any event
listeners behind or catching unrelated queries by accident.
Anyway, here's some code, and if you need a complete example, see example.py.
:::python
class QMonitor(object):
def __init__(self, max_queries):
self.queries = []
self.max_queries = max_queries
def assist_user(self):
""" User requires assistance in query minimization.
"""
qc = Counter(zip(*self.queries)[0])
if qc.values().count(1) < len(qc):
duplicates = [
'** Repeated {0} times: {1}'.format(c, q)
for q, c in qc.items()
if c > 1
]
print (u'Too many queries, try eliminating duplicates:\n' +
u'\n'.join(duplicates))
else:
print u'Too many queries:\n' + u'\n'.join(zip(*self.queries)[0])
def check(self):
""" Check saved database queries.
"""
if len(self.queries) > self.max_queries:
self.assist_user()
@contextmanager
def qmonitor(max_queries=15):
qm = QMonitor(max_queries)
def before_cursor_execute(conn, cursor, statement, parameters, context, executemany):
context._query_start_time = time.time()
def after_cursor_execute(conn, cursor, statement, parameters, context, executemany):
total = time.time() - context._query_start_time
qm.queries.append((statement, total))
event.listen(engine, 'before_cursor_execute', before_cursor_execute)
event.listen(engine, 'after_cursor_execute', after_cursor_execute)
yield qm
event.remove(engine, 'before_cursor_execute', before_cursor_execute)
event.remove(engine, 'after_cursor_execute', after_cursor_execute)
print '== Naive query'
q = session.query(Address)
with qmonitor(max_queries=1) as qm:
for address in q:
print address.email_address + ' - ' + address.user.fullname
qm.check()
This will check performace (so far just count number of DB queries) and assist
in making the "managed" block of code as fast as possible. It is also quite
easy to use in integration tests for finding database-intensive code.