civiqhub_selene.blocks.responses

Response models: how the cells of a co-cluster spread over the states of a cell.

A block model is made of two independent things: a latent structure (who belongs to which group, which item to which block) and a response model, which says how the cells of a co-cluster spread between “shown without a vote” and each score.

A shown cell has K + 1 states: state 0 (shown without a vote) and state k (vote at rank k, from 1 to K). A response model reads counts of cells per state and does not know whether they are integers (single membership) or fractional (mixed membership).

A response model is a frozen dataclass: its settings enter the provenance of the estimator that carries it as they are.

class civiqhub_selene.blocks.responses.Categorical(smoothing=0.5, min_probability=1e-06)[source]

Bases: object

A vote rate per co-cluster, then a free distribution over the K scores.

Definition. For a shown cell of the co-cluster (q, l):

P(no vote) = 1 - vote_rate[q, l]
P(vote at rank k) = vote_rate[q, l] * vote_shares[q, l, k]

with K - 1 free parameters per co-cluster for the distribution and no constraint on its shape. The update is the maximum of the expected likelihood: the vote rate is the share of cells that carry a vote, and the distribution is the share of votes at each rank, after smoothing is added to each count.

It is the simple reference against which the cost and the benefit of the ordinal probit model are judged.

smoothing

Pseudo-count added to the number of votes at each rank before the distribution is normalised, at least 0. Defaults to 0.5.

Type:

float

min_probability

Bound that keeps a vote rate away from 0 and from 1, strictly between 0 and 0.5. Defaults to 1e-6.

Type:

float

Raises:

InvalidParameterError – If a setting is outside its range.

Parameters:
  • smoothing (float)

  • min_probability (float)

initial_parameters(shape, votes, rng)[source]

Return starting parameters: the overall vote rate, and drawn distributions.

The vote rate is the same everywhere: the votes over the shown cells of the matrix, bounded. Each distribution is drawn from a Dirichlet law of concentration 5 on each of the K scores.

Parameters:
  • shape (tuple[int, int])

  • votes (VoteMatrix)

  • rng (Generator)

Return type:

CategoricalParameters

log_probabilities(params)[source]

Return the log-probability of each state in each co-cluster, shape (Q, L, K + 1).

Parameters:

params (CategoricalParameters)

Return type:

NDArray[float64]

update(counts, previous)[source]

Return the vote rates and the distributions that the counts call for.

A co-cluster without any cell keeps its previous vote rate. A co-cluster without any vote keeps its previous distribution when smoothing is 0.

Parameters:
Return type:

CategoricalParameters

complexity_penalty(shape, votes)[source]

Return half a logarithm of the sample size per free parameter.

0.5 * Q * L * (K - 1) * log(max(n_votes, 1)) for the distributions, plus 0.5 * Q * L * log(max(n_shown, 1)) for the vote rates.

Parameters:
Return type:

float

to_table(params, counts, scale)[source]

Return the table of the co-clusters, with the number of votes of the counts.

Parameters:
Return type:

CoclusterTable

class civiqhub_selene.blocks.responses.CategoricalParameters(vote_rate, vote_shares)[source]

Bases: object

Parameters of the Categorical response model.

Parameters:
  • vote_rate (NDArray[float64])

  • vote_shares (NDArray[float64])

vote_rate

Shape (Q, L): probability that a shown cell carries a vote.

Type:

numpy._typing._array_like.NDArray[numpy.float64]

vote_shares

Shape (Q, L, K): distribution of the votes over the scores.

Type:

numpy._typing._array_like.NDArray[numpy.float64]

class civiqhub_selene.blocks.responses.OrdinalProbit(thresholds=None, min_spread=0.3, min_probability=1e-06, max_inner_iterations=10)[source]

Bases: object

A vote rate per co-cluster, then a latent normal opinion cut into K levels.

Definition. In a co-cluster, a shown cell carries a vote with the probability vote_rate. The vote comes from a hidden continuous opinion, normal with mean latent_mean and standard deviation latent_spread, which fixed thresholds t_1 < ... < t_{K-1} cut into K levels, with t_0 = -inf and t_K = +inf:

P(no vote) = 1 - vote_rate
P(vote at rank k) = vote_rate * p_k
p_k = cdf((t_k - latent_mean) / latent_spread)
      - cdf((t_{k-1} - latent_mean) / latent_spread)

where cdf is the standard normal distribution function. Two numbers then sum up how a group answers a block: where the opinion stands and how spread it is.

The thresholds default to k + 1/2 for k from 1 to K - 1. They are fixed, not estimated: latent_mean then reads directly on the axis of levels, from 1 to K. The mean and the spread are estimated by expectation-maximisation on the hidden opinion (see fit_latent_normal).

Reference: Corneli, Bouveyron, Latouche (2020), Co-clustering of ordinal data via latent continuous random variables and not missing at random entries.

thresholds

The K - 1 thresholds, strictly increasing, or None for the default ones.

Type:

tuple[float, …] | None

min_spread

Lower bound of latent_spread, strictly positive. It keeps a unanimous co-cluster from sending its spread to zero. Defaults to 0.3.

Type:

float

min_probability

Bound that keeps a vote rate away from 0 and from 1, strictly between 0 and 0.5. Defaults to 1e-6.

Type:

float

max_inner_iterations

Largest number of expectation-maximisation steps on the hidden opinion at each update, at least 1. Defaults to 10.

Type:

int

Raises:

InvalidParameterError – If a setting is outside its range.

Parameters:
  • thresholds (tuple[float, ...] | None)

  • min_spread (float)

  • min_probability (float)

  • max_inner_iterations (int)

initial_parameters(shape, votes, rng)[source]

Return starting parameters: the overall vote rate and opinions near the middle.

The vote rate is the same everywhere: the votes over the shown cells of the matrix, bounded. Each mean is the middle of the axis of levels, (K + 1) / 2, plus a normal draw of standard deviation 0.5; each spread is 1.

Raises:

InvalidParameterError – If the thresholds given are not K - 1 values.

Parameters:
  • shape (tuple[int, int])

  • votes (VoteMatrix)

  • rng (Generator)

Return type:

OrdinalProbitParameters

log_probabilities(params)[source]

Return the log-probability of each state in each co-cluster, shape (Q, L, K + 1).

Parameters:

params (OrdinalProbitParameters)

Return type:

NDArray[float64]

update(counts, previous)[source]

Return the vote rates and the latent opinions that the counts call for.

The mean and the spread restart from the previous ones and take at most max_inner_iterations steps. A co-cluster without any cell keeps its previous vote rate; a co-cluster without any vote keeps its previous opinion.

Parameters:
Return type:

OrdinalProbitParameters

complexity_penalty(shape, votes)[source]

Return half a logarithm of the sample size per free parameter.

Two parameters of opinion and one vote rate per co-cluster: Q * L * log(max(n_votes, 1)) + 0.5 * Q * L * log(max(n_shown, 1)).

Parameters:
Return type:

float

to_table(params, counts, scale)[source]

Return the table of the co-clusters, with their latent mean and spread.

Parameters:
Return type:

CoclusterTable

_vote_shares(params)[source]

Return the probability of each level, floored at 1e-12.

Parameters:

params (OrdinalProbitParameters)

Return type:

NDArray[float64]

class civiqhub_selene.blocks.responses.OrdinalProbitParameters(vote_rate, latent_mean, latent_spread, thresholds)[source]

Bases: object

Parameters of the OrdinalProbit response model.

Parameters:
  • vote_rate (NDArray[float64])

  • latent_mean (NDArray[float64])

  • latent_spread (NDArray[float64])

  • thresholds (NDArray[float64])

vote_rate

Shape (Q, L): probability that a shown cell carries a vote.

Type:

numpy._typing._array_like.NDArray[numpy.float64]

latent_mean

Shape (Q, L): mean of the latent opinion, on the axis of levels.

Type:

numpy._typing._array_like.NDArray[numpy.float64]

latent_spread

Shape (Q, L): standard deviation of the latent opinion.

Type:

numpy._typing._array_like.NDArray[numpy.float64]

thresholds

The K - 1 thresholds between levels, fixed during the computation.

Type:

numpy._typing._array_like.NDArray[numpy.float64]

class civiqhub_selene.blocks.responses.ResponseModel(*args, **kwargs)[source]

Bases: Protocol[ParametersT]

What a block model asks of the response model of its co-clusters.

ParametersT is the type of the parameters of the model, for Q x L co-clusters.

initial_parameters(shape, votes, rng)[source]

Return starting parameters for shape = (Q, L) co-clusters.

Parameters:
  • shape (tuple[int, int])

  • votes (VoteMatrix)

  • rng (Generator)

Return type:

ParametersT

log_probabilities(params)[source]

Return the log-probability of each state in each co-cluster, shape (Q, L, K + 1).

Parameters:

params (ParametersT)

Return type:

NDArray[float64]

update(counts, previous)[source]

Return the parameters that maximise the expected likelihood.

counts[q, l, s] is the number, integer or fractional, of cells of the co-cluster (q, l) in the state s.

Parameters:
  • counts (NDArray[float64])

  • previous (ParametersT)

Return type:

ParametersT

complexity_penalty(shape, votes)[source]

Return the penalty of the parameters in the integrated likelihood.

Half a logarithm of the sample size per free parameter.

Parameters:
Return type:

float

to_table(params, counts, scale)[source]

Return the readable table of the co-clusters.

Parameters:
  • params (ParametersT)

  • counts (NDArray[float64])

  • scale (OrdinalScale)

Return type:

CoclusterTable