civiqhub_selene.blocks.responses¶
Response models: how the cells of a co-cluster spread over the states of a cell.
A block model is made of two independent things: a latent structure (who belongs to which group, which item to which block) and a response model, which says how the cells of a co-cluster spread between “shown without a vote” and each score.
A shown cell has K + 1 states: state 0 (shown without a vote) and state k (vote at
rank k, from 1 to K). A response model reads counts of cells per state and does
not know whether they are integers (single membership) or fractional (mixed
membership).
A response model is a frozen dataclass: its settings enter the provenance of the estimator that carries it as they are.
- class civiqhub_selene.blocks.responses.Categorical(smoothing=0.5, min_probability=1e-06)[source]¶
Bases:
objectA vote rate per co-cluster, then a free distribution over the K scores.
Definition. For a shown cell of the co-cluster
(q, l):P(no vote) = 1 - vote_rate[q, l] P(vote at rank k) = vote_rate[q, l] * vote_shares[q, l, k]
with K - 1 free parameters per co-cluster for the distribution and no constraint on its shape. The update is the maximum of the expected likelihood: the vote rate is the share of cells that carry a vote, and the distribution is the share of votes at each rank, after
smoothingis added to each count.It is the simple reference against which the cost and the benefit of the ordinal probit model are judged.
- smoothing¶
Pseudo-count added to the number of votes at each rank before the distribution is normalised, at least 0. Defaults to 0.5.
- Type:
float
- min_probability¶
Bound that keeps a vote rate away from 0 and from 1, strictly between 0 and 0.5. Defaults to 1e-6.
- Type:
float
- Raises:
InvalidParameterError – If a setting is outside its range.
- Parameters:
smoothing (float)
min_probability (float)
- initial_parameters(shape, votes, rng)[source]¶
Return starting parameters: the overall vote rate, and drawn distributions.
The vote rate is the same everywhere: the votes over the shown cells of the matrix, bounded. Each distribution is drawn from a Dirichlet law of concentration 5 on each of the K scores.
- Parameters:
shape (tuple[int, int])
votes (VoteMatrix)
rng (Generator)
- Return type:
- log_probabilities(params)[source]¶
Return the log-probability of each state in each co-cluster, shape (Q, L, K + 1).
- Parameters:
params (CategoricalParameters)
- Return type:
NDArray[float64]
- update(counts, previous)[source]¶
Return the vote rates and the distributions that the counts call for.
A co-cluster without any cell keeps its previous vote rate. A co-cluster without any vote keeps its previous distribution when
smoothingis 0.- Parameters:
counts (NDArray[float64])
previous (CategoricalParameters)
- Return type:
- complexity_penalty(shape, votes)[source]¶
Return half a logarithm of the sample size per free parameter.
0.5 * Q * L * (K - 1) * log(max(n_votes, 1))for the distributions, plus0.5 * Q * L * log(max(n_shown, 1))for the vote rates.- Parameters:
shape (tuple[int, int])
votes (VoteMatrix)
- Return type:
float
- to_table(params, counts, scale)[source]¶
Return the table of the co-clusters, with the number of votes of the counts.
- Parameters:
params (CategoricalParameters)
counts (NDArray[float64])
scale (OrdinalScale)
- Return type:
- class civiqhub_selene.blocks.responses.CategoricalParameters(vote_rate, vote_shares)[source]¶
Bases:
objectParameters of the
Categoricalresponse model.- Parameters:
vote_rate (NDArray[float64])
vote_shares (NDArray[float64])
- vote_rate¶
Shape (Q, L): probability that a shown cell carries a vote.
- Type:
numpy._typing._array_like.NDArray[numpy.float64]
Shape (Q, L, K): distribution of the votes over the scores.
- Type:
numpy._typing._array_like.NDArray[numpy.float64]
- class civiqhub_selene.blocks.responses.OrdinalProbit(thresholds=None, min_spread=0.3, min_probability=1e-06, max_inner_iterations=10)[source]¶
Bases:
objectA vote rate per co-cluster, then a latent normal opinion cut into K levels.
Definition. In a co-cluster, a shown cell carries a vote with the probability
vote_rate. The vote comes from a hidden continuous opinion, normal with meanlatent_meanand standard deviationlatent_spread, which fixed thresholdst_1 < ... < t_{K-1}cut into K levels, witht_0 = -infandt_K = +inf:P(no vote) = 1 - vote_rate P(vote at rank k) = vote_rate * p_k p_k = cdf((t_k - latent_mean) / latent_spread) - cdf((t_{k-1} - latent_mean) / latent_spread)
where
cdfis the standard normal distribution function. Two numbers then sum up how a group answers a block: where the opinion stands and how spread it is.The thresholds default to
k + 1/2forkfrom 1 to K - 1. They are fixed, not estimated:latent_meanthen reads directly on the axis of levels, from 1 to K. The mean and the spread are estimated by expectation-maximisation on the hidden opinion (seefit_latent_normal).Reference: Corneli, Bouveyron, Latouche (2020), Co-clustering of ordinal data via latent continuous random variables and not missing at random entries.
- thresholds¶
The K - 1 thresholds, strictly increasing, or
Nonefor the default ones.- Type:
tuple[float, …] | None
- min_spread¶
Lower bound of
latent_spread, strictly positive. It keeps a unanimous co-cluster from sending its spread to zero. Defaults to 0.3.- Type:
float
- min_probability¶
Bound that keeps a vote rate away from 0 and from 1, strictly between 0 and 0.5. Defaults to 1e-6.
- Type:
float
- max_inner_iterations¶
Largest number of expectation-maximisation steps on the hidden opinion at each update, at least 1. Defaults to 10.
- Type:
int
- Raises:
InvalidParameterError – If a setting is outside its range.
- Parameters:
thresholds (tuple[float, ...] | None)
min_spread (float)
min_probability (float)
max_inner_iterations (int)
- initial_parameters(shape, votes, rng)[source]¶
Return starting parameters: the overall vote rate and opinions near the middle.
The vote rate is the same everywhere: the votes over the shown cells of the matrix, bounded. Each mean is the middle of the axis of levels,
(K + 1) / 2, plus a normal draw of standard deviation 0.5; each spread is 1.- Raises:
InvalidParameterError – If the thresholds given are not K - 1 values.
- Parameters:
shape (tuple[int, int])
votes (VoteMatrix)
rng (Generator)
- Return type:
- log_probabilities(params)[source]¶
Return the log-probability of each state in each co-cluster, shape (Q, L, K + 1).
- Parameters:
params (OrdinalProbitParameters)
- Return type:
NDArray[float64]
- update(counts, previous)[source]¶
Return the vote rates and the latent opinions that the counts call for.
The mean and the spread restart from the previous ones and take at most
max_inner_iterationssteps. A co-cluster without any cell keeps its previous vote rate; a co-cluster without any vote keeps its previous opinion.- Parameters:
counts (NDArray[float64])
previous (OrdinalProbitParameters)
- Return type:
- complexity_penalty(shape, votes)[source]¶
Return half a logarithm of the sample size per free parameter.
Two parameters of opinion and one vote rate per co-cluster:
Q * L * log(max(n_votes, 1)) + 0.5 * Q * L * log(max(n_shown, 1)).- Parameters:
shape (tuple[int, int])
votes (VoteMatrix)
- Return type:
float
- to_table(params, counts, scale)[source]¶
Return the table of the co-clusters, with their latent mean and spread.
- Parameters:
params (OrdinalProbitParameters)
counts (NDArray[float64])
scale (OrdinalScale)
- Return type:
Return the probability of each level, floored at 1e-12.
- Parameters:
params (OrdinalProbitParameters)
- Return type:
NDArray[float64]
- class civiqhub_selene.blocks.responses.OrdinalProbitParameters(vote_rate, latent_mean, latent_spread, thresholds)[source]¶
Bases:
objectParameters of the
OrdinalProbitresponse model.- Parameters:
vote_rate (NDArray[float64])
latent_mean (NDArray[float64])
latent_spread (NDArray[float64])
thresholds (NDArray[float64])
- vote_rate¶
Shape (Q, L): probability that a shown cell carries a vote.
- Type:
numpy._typing._array_like.NDArray[numpy.float64]
- latent_mean¶
Shape (Q, L): mean of the latent opinion, on the axis of levels.
- Type:
numpy._typing._array_like.NDArray[numpy.float64]
- latent_spread¶
Shape (Q, L): standard deviation of the latent opinion.
- Type:
numpy._typing._array_like.NDArray[numpy.float64]
- thresholds¶
The K - 1 thresholds between levels, fixed during the computation.
- Type:
numpy._typing._array_like.NDArray[numpy.float64]
- class civiqhub_selene.blocks.responses.ResponseModel(*args, **kwargs)[source]¶
Bases:
Protocol[ParametersT]What a block model asks of the response model of its co-clusters.
ParametersTis the type of the parameters of the model, for Q x L co-clusters.- initial_parameters(shape, votes, rng)[source]¶
Return starting parameters for
shape = (Q, L)co-clusters.- Parameters:
shape (tuple[int, int])
votes (VoteMatrix)
rng (Generator)
- Return type:
ParametersT
- log_probabilities(params)[source]¶
Return the log-probability of each state in each co-cluster, shape (Q, L, K + 1).
- Parameters:
params (ParametersT)
- Return type:
NDArray[float64]
- update(counts, previous)[source]¶
Return the parameters that maximise the expected likelihood.
counts[q, l, s]is the number, integer or fractional, of cells of the co-cluster(q, l)in the states.- Parameters:
counts (NDArray[float64])
previous (ParametersT)
- Return type:
ParametersT
- complexity_penalty(shape, votes)[source]¶
Return the penalty of the parameters in the integrated likelihood.
Half a logarithm of the sample size per free parameter.
- Parameters:
shape (tuple[int, int])
votes (VoteMatrix)
- Return type:
float
- to_table(params, counts, scale)[source]¶
Return the readable table of the co-clusters.
- Parameters:
params (ParametersT)
counts (NDArray[float64])
scale (OrdinalScale)
- Return type: