Anton Shestakov <av6@dwimlabs.net>, Sun, 24 Sep 2017 12:25:05 +0800
viewer: use subqueries to get data on index page
Before, there were bare columns in the aggregate query, their values were
undefined (but it somehow worked), as SQLite docs say. Good news is that now
this bigger query uses (project_id, ordinal) index and is really fast.
.. image:: https://candolint.dwimlabs.net/bitbucket.org/av6/candolint/status.svg?branch=default
:target: https://candolint.dwimlabs.net/bitbucket.org/av6/candolint
- checker.py - command line tool for running a single check
- incoming.py - command line tool for parsing a single check result
- viewer.py - web interface for looking at projects and check results
- hooks-queue.py - web service for handling events from code hosting sites
- poller-queue.py - long-lived HTTP client for polling changes
- worker-queue.py - long-lived worker process for running checks on changes
- incoming-queue.py - long-lived process for receiving and parsing check
- run-tests.py - test runner, see
*bitbucket-pipelines.yml* for some useful
Here's an example similar to
*candolint.yml* of this project itself (plus a
couple more options for demonstration):
# bootstrapping data, optional url: https://bitbucket.org/av6/candolint # linter config, required flags: ['--extract', 'always'] This is the two most basic things needed to check out project source code: its
If you use push hooks or polling, this data is already present in the payload.
You only need to put it in
*projects/<name>.yml* file on the worker if you want
to run checks manually (e.g. for testing).
This is the list of linters to run on project source code. Each item only
- ``name``: determines what configuration file will be used
(
*linters/<name>.yml*, it is often named after the actual executable)
- ``include``: a sequence of file patterns for checking
Optional parameters include:
- ``vars``: variables used for installing and running linters (e.g.
- ``codes``: allowed exit codes (by default
``0`` and
``1`` are allowed); note
that this is only for the actual checking, so if
*linter --version* exits
with code
``1``, it's considered a failure
- ``exclude``: a sequence of file patterns to exclude from checks
- ``flags``: additional command line flags passed to the linter (before passing
- ``post_flags``: like
``flags``, but passed after file path
The exact base command executed for each linter is specified in
*linters/<name>.yml* and cannot be modified in project configuration file, see
Base linter configuration ========================= Every linter needs to be configured via
*linters/<name>.yml* file. Some files
are already there, they are used to check various projects, including this one
itself. Here's what they look like:
- ['virtualenv', '--python=python%(python_version)s', '../venv%(python_version)s'] - ['../venv%(python_version)s/bin/pip', 'install', 'flake8', 'pep8-naming'] exec: ['../venv%(python_version)s/bin/flake8'] - ``exec``: a list of strings that will be fed to Python's
``subprocess.Popen`` - ``version``: a list of strings that will be used as command line flags for
testing that this linter works after installation
- ``setup``: a list of commands (as lists of strings), each will be executed
consecutively to install this linter
- ``vars``: default values for variables, these can be tweaked in project
Individual components talk to each other using a message queue, and that needs
a Redis instance. Here's a sample configuration to use Redis on localhost, it
can be stored in a file and supplied using
``--config`` switch.
poller-queue.py also needs a list of targets to poll and an interval in
seconds. They can be stored together with queue configuration in the same
- url: https://hg.prosody.im/0.10 poll: https://hg.prosody.im/0.10/json-branches - url: https://github.com/kaitai-io/kaitai_struct_python_runtime poll: https://api.github.com/repos/kaitai-io/kaitai_struct_python_runtime/branches