306:edefddec933d
Anton Shestakov <av6@dwimlabs.net>, Sun, 24 Sep 2017 12:25:05 +0800
viewer: use subqueries to get data on index page Before, there were bare columns in the aggregate query, their values were undefined (but it somehow worked), as SQLite docs say. Good news is that now this bigger query uses (project_id, ordinal) index and is really fast.

next change 307:6ce60c358c5c
previous change 287:f6bb16bb874a

README.rst

Permissions: -rw-r--r--

Other formats: Feeds:
.. image:: https://candolint.dwimlabs.net/bitbucket.org/av6/candolint/status.svg?branch=default
:target: https://candolint.dwimlabs.net/bitbucket.org/av6/candolint
:alt: Lint Status
License
=======
MIT.
Entry points
============
- checker.py - command line tool for running a single check
- incoming.py - command line tool for parsing a single check result
- viewer.py - web interface for looking at projects and check results
..
- hooks-queue.py - web service for handling events from code hosting sites
- poller-queue.py - long-lived HTTP client for polling changes
- worker-queue.py - long-lived worker process for running checks on changes
- incoming-queue.py - long-lived process for receiving and parsing check
results from workers
..
- run-tests.py - test runner, see *bitbucket-pipelines.yml* for some useful
arguments
Project configuration
=====================
Here's an example similar to *candolint.yml* of this project itself (plus a
couple more options for demonstration):
.. code:: yaml
# bootstrapping data, optional
url: https://bitbucket.org/av6/candolint
scm: hg
# linter config, required
linters:
- name: flake8
vars:
python_version: 2
include:
- '**.py'
- name: yamllint
include:
- '**.yaml'
- '**.yml'
- name: jshint
flags: ['--extract', 'always']
include:
- '**.html'
Bootstrapping data
------------------
This is the two most basic things needed to check out project source code: its
URL and SCM.
If you use push hooks or polling, this data is already present in the payload.
You only need to put it in *projects/<name>.yml* file on the worker if you want
to run checks manually (e.g. for testing).
Linter configuration
--------------------
This is the list of linters to run on project source code. Each item only
requires two parameters:
- ``name``: determines what configuration file will be used
(*linters/<name>.yml*, it is often named after the actual executable)
- ``include``: a sequence of file patterns for checking
Optional parameters include:
- ``vars``: variables used for installing and running linters (e.g.
``python_version``)
- ``codes``: allowed exit codes (by default ``0`` and ``1`` are allowed); note
that this is only for the actual checking, so if *linter --version* exits
with code ``1``, it's considered a failure
- ``exclude``: a sequence of file patterns to exclude from checks
- ``flags``: additional command line flags passed to the linter (before passing
file path)
- ``post_flags``: like ``flags``, but passed after file path
The exact base command executed for each linter is specified in
*linters/<name>.yml* and cannot be modified in project configuration file, see
the next section.
Base linter configuration
=========================
Every linter needs to be configured via *linters/<name>.yml* file. Some files
are already there, they are used to check various projects, including this one
itself. Here's what they look like:
.. code:: yaml
vars:
python_version: 2
setup:
- ['virtualenv', '--python=python%(python_version)s', '../venv%(python_version)s']
- ['../venv%(python_version)s/bin/pip', 'install', 'flake8', 'pep8-naming']
exec: ['../venv%(python_version)s/bin/flake8']
version: ['--version']
Required parameters:
- ``exec``: a list of strings that will be fed to Python's ``subprocess.Popen``
for running this linter
- ``version``: a list of strings that will be used as command line flags for
testing that this linter works after installation
Optional parameters:
- ``setup``: a list of commands (as lists of strings), each will be executed
consecutively to install this linter
- ``vars``: default values for variables, these can be tweaked in project
configuration file
Queue configuration
===================
Individual components talk to each other using a message queue, and that needs
a Redis instance. Here's a sample configuration to use Redis on localhost, it
can be stored in a file and supplied using ``--config`` switch.
.. code:: yaml
redis-host: 127.0.0.1
redis-port: 6379
redis-password: hunter2
Poller configuration
====================
poller-queue.py also needs a list of targets to poll and an interval in
seconds. They can be stored together with queue configuration in the same
file.
.. code:: yaml
poll-interval: 3600
targets:
- url: https://hg.prosody.im/0.10
poll: https://hg.prosody.im/0.10/json-branches
type: hgweb
- url: https://github.com/kaitai-io/kaitai_struct_python_runtime
poll: https://api.github.com/repos/kaitai-io/kaitai_struct_python_runtime/branches
type: github