Architecture¶
This page describes how the engine is built: the contract every method shares, the data of co-clustering, and the pipeline a block model runs through. It is written for whoever adds a method or reviews one. The mathematical definition of each method lives in its docstring.
Six rules¶
An estimator is a frozen value. Every setting is a named, typed field of a frozen dataclass; nothing is read from the environment.
estimate(data, *, seed=0)is written once, inEstimator, and is never overridden.Every result is a
Result. It carries a value or an explained absence (Unavailable), the provenance, and the uncertainty when the method defines one. NeverNone, never NaN, never a default in place of an absence.Determinism. Same data, same settings, same seed: same result and same fingerprint. Any randomness goes through the seed received.
Dependencies. The package root and
ordinalimport the standard library only.blocksimports NumPy and raisesMissingExtraErrorwhen it is missing. No module imports theciviqhubSDK, a network client or a storage layer.Extension without modification. A new method is a class declared under the entry-point group
civiqhub_selene.estimators, with a keyfamily.choice. It passes the contract kit. No existing class is modified.Three states per cell. In a vote matrix, a cell is voted, shown without a vote, or not shown. An empty cell is never a zero nor a neutral answer.
Modules¶
Module |
Holds |
Imports |
|---|---|---|
|
|
standard library |
|
|
standard library |
|
|
standard library |
|
|
standard library |
|
|
standard library; pytest when a test runs |
|
|
standard library |
|
|
NumPy |
|
|
NumPy |
|
|
NumPy |
|
|
NumPy |
|
|
NumPy |
|
|
NumPy |
|
|
NumPy |
|
|
NumPy |
|
|
NumPy |
|
|
NumPy |
Not written yet: the dissensus index and the collective regime (ordinal), the mixed-membership block model (MixedMembershipBlockModel), the choice of shape (select_shape), the consensus between groups and the cocluster function.
The contract of a method¶
flowchart TD
call["estimator.estimate(data, seed=0)"] --> identity["check the declared identity<br>name, version, input_type"]
identity --> input["check the input type"]
input --> provenance["write the provenance<br>settings, input fingerprint, seed, lineage"]
provenance --> why{"_why_unavailable(data)"}
why -- "an Unavailable" --> absent["Result(Unavailable, provenance)"]
why -- "None" --> compute["_compute(data, seed, computation)"]
compute --> result["Result(value, provenance, uncertainty)"]
A concrete method is a frozen dataclass declared with kw_only=True. It declares name, version, reference and input_type as class attributes and writes _compute. It never builds a Result nor a Provenance itself.
Result.fingerprint() identifies the computation: method name and version, settings, seed, input fingerprint and lineage. The engine version and the versions of dependencies are recorded but left out of the fingerprint.
To prove that an estimator honours the contract, write three methods and inherit seven tests:
from civiqhub_selene.testing import EstimatorContract
class TestMyMethod(EstimatorContract):
def make_estimator(self):
return MyMethod()
def make_data(self):
return ... # an input on which a value exists
def make_empty_data(self):
return ... # an input with no answer, or None if there is none
The data of co-clustering¶
A group gathers voters, a block gathers items, and a co-cluster is the pair of a group and a block.
VoteMatrixholdslevels(0: no vote;k: vote at rankkof the scale) andshown. Models read it throughvotes._states()only: an indicator array of shape (M, P, K + 1), zero everywhere on a cell that is not shown.Partitionarranges identifiers in clusters. It can come from a model or from anywhere else, and carries the fingerprint of the computation it comes from.GroupedVotesreads a matrix through a partition of voters and a partition of items.counts()gives the sufficient statistic of every block model: the number of shown cells of each co-cluster in each state, shape (Q, L, K + 1).CoclusterTablesummarises each co-cluster by a vote rate and a distribution of the answers.
The pipeline of a block model¶
A block model is a latent structure (who belongs to which group, which item to which block) and a response model, which says how the cells of a co-cluster spread between “shown without a vote” and each score. The two are independent: any block model runs with any response model.
flowchart LR
word["plain word<br>'single', 'mixed'"] --> registry["Registry<br>coclustering.word"]
registry --> model["a BlockModel<br>with_shape(Q, L)"]
votes["VoteMatrix"] --> estimate["Estimator.estimate<br>input, provenance, absence"]
model --> estimate
estimate --> loop["IterativeEstimator._compute<br>n_starts seeds, iterate, best start"]
loop --> hooks["_start, _step, _finish<br>written by the model"]
hooks --> response["ResponseModel<br>OrdinalProbit or Categorical"]
hooks --> value["SingleCoclustering<br>or MixedCoclustering"]
value --> result["Result"]
Estimator estimate() written once
|- IterativeEstimator _compute() written once
'- CoclusteringEstimator with_shape(), availability
BlockModel(IterativeEstimator, CoclusteringEstimator)
|- LatentBlockModel "single"
'- MixedMembershipBlockModel "mixed" to come
What is written once:
Estimator.estimatechecks the input, writes the provenance and wraps the outcome.CoclusteringEstimator._why_unavailablereturns an explained absence when the matrix holds no vote, or fewer voters or items with a vote than the shape asks for.IterativeEstimator._computederives one seed per start withnumpy.random.SeedSequence(seed).spawn(n_starts), runs every start withiterate, the only convergence loop of the package, and keeps the start whose final objective is the largest.
What a model writes: its identity, and three hooks.
@dataclass(frozen=True, kw_only=True)
class MyBlockModel(BlockModel[MyState, SingleCoclustering]):
name = "my_block_model"
version = "0.1.0"
reference = "Author, year."
# The plain word of the registry key, and its meaning in plain words.
membership = "mine"
meaning = "each voter belongs to one group"
default_criterion = "integrated_likelihood"
def _start(self, votes, rng):
"""Return a starting state, drawn with rng only. Called once per start."""
def _step(self, votes, state):
"""Return the state after one full iteration. It must not lower state.objective."""
def _finish(self, votes, state, history, computation):
"""Return the public value, from the best final state."""
A state is any frozen object with an objective. A model never writes a loop and never draws a seed. civiqhub_selene/blocks/single.py is the model to read first: LatentBlockModel is these three hooks and nothing else. tests/unit/test_blocks_pipeline.py holds a shorter model written with the public pieces only; it is a test of the architecture, not a delivered method.
A step uses the response model through four calls:
Call |
Returns |
|---|---|
|
starting parameters |
|
the parameters the counts call for; |
|
the log-probability of each state in each co-cluster, shape (Q, L, K + 1) |
|
the |
complexity_penalty((Q, L), votes) gives the penalty of the response model in the integrated likelihood.
Adding a method¶
Write the estimator and its value in their module, with the mathematical definition and the reference in the docstring.
Declare it in
pyproject.toml, then reinstall the package: an editable install reads its entry points when it is installed. For example, the line of the single-membership model:[project.entry-points."civiqhub_selene.estimators"] "coclustering.single" = "civiqhub_selene.blocks.single:LatentBlockModel"
Add a test class that inherits
EstimatorContract, unit tests on known cases, property-based tests for the invariants, and reference vectors undertests/vectorswith their tolerance.Create the page of its module in the reference.
docs/README.md, in the repository, says how; a test fails as long as a public module has no page.
Tests¶
Suite |
Content |
|---|---|
|
One file per module, on small hand-written inputs |
|
Invariants on generated inputs, with Hypothesis; the profile in |
|
Expected values stored as data, with their tolerance and their source |
|
Reserved for the contract between the engine and the service |
Two files of tests/unit check the documentation: test_docs_reference.py (every public module has a page in the reference) and test_docs_examples.py (every Python block of the pages runs and prints the output shown).
A test file of the blocks extra is marked and skips itself when NumPy is missing:
import pytest
pytest.importorskip("numpy")
import numpy as np
from civiqhub_selene.blocks import VoteMatrix
pytestmark = pytest.mark.blocks
The test-core job of the CI installs the package without any extra, checks that NumPy is not installed, and runs pytest -m "not blocks": the package root and ordinal must live without NumPy.
Words¶
Word |
Meaning |
|---|---|
score |
The answer of a vote on the agreement scale: an integer from -2 to +2 |
level |
The rank of a score on the scale, from 1 to K; 0 means “no vote”. Internal to matrices |
voter, item |
The row and the column of a vote matrix, by opaque identifiers |
shown |
The item was shown to the voter |
group, block |
A cluster of voters; a cluster of items |
co-cluster |
The pair of a group and a block |
membership |
The degree to which a voter belongs to each group, or an item to each block; also the choice |
fingerprint |
A stable identifier of a content or of a computation: 16 hexadecimal characters of a SHA-256 |
computation |
The fingerprint of one precise computation: method, settings, seed, input, lineage |