mantispy.tl.hit_calling

mantispy.tl.hit_calling(adata, groupby='Metadata_Perturbation', reference='negcon', method='mahalanobis', covariance='empirical', use_rep=None, replicate=None, n_permutations=1000, threshold=0.05, seed=0, key_added='hits', copy=False)

Call hits by testing each group’s distance from the controls.

Each group is scored against the half of the reference rows that did not fit the centroid and covariance, and its null is the other ways to draw a group of its size from the group and those controls pooled. Under the null the two are exchangeable, so the null is calibrated; bootstrapping the controls alone is not, because it is centred on that sample’s own median rather than the population’s and leaves the error in that centre out of its spread.

This calibrates each group against control wells; mantispy.tl.empirical_fdr() instead calibrates a per-gene score against control genes that cannot respond, for a genetic screen where non-expressed genes are the better null.

This wants a well-replicated design. The statistic is a group’s median distance, so a group of one or two wells is dominated by whichever wells it holds and no number of permutations recovers that; mantispy.tl.map() with mode="activity" ranks replicate pairs instead and is the usual readout on screens with little replication. JUMP-Target-2 read as a single plate gives every compound one well and is the common way to land in that regime, while the same plate map read across several of them gives one well per plate.

Distance from the controls also rises when a treatment kills cells. Read the calls beside a cell count, or beside mantispy.tl.cytotoxicity(), before taking them for morphology.

Parameters:

adata AnnData

Object to score, at cell or well resolution.

groupby str (default: 'Metadata_Perturbation')

Column defining the groups to test.

reference str | None (default: 'negcon')

Which rows are the controls. They are split in half, one half to fit the covariance and the other to form the null (see Notes).

method str (default: 'mahalanobis')

"mahalanobis" scores the median distance of the group’s rows from the control centroid, measured in the controls’ covariance so that directions the controls already vary in count for less. "ks" scores the Kolmogorov-Smirnov statistic between the group’s and the controls’ distance distributions, which detects a shifted subpopulation that leaves the median unchanged. Use it at cell resolution. Its p-value comes from a permutation null over whole replicates when one is in use (see replicate), and otherwise from scipy.stats.ks_2samp. "hellinger" fits a Gaussian to the group’s rows and one to the controls’, and scores the Hellinger distance between the two, so a change in the group’s own spread or shape counts as well as a shift of its centre; a treatment that sends only some cells into a new state widens the group without moving its median. It needs each side to have at least twice as many rows as features, so use it at cell resolution on a reduced use_rep; a group short of that is left unscored with a warning. It is opt-in; the default stays "mahalanobis". A single Gaussian describes a cloud with several cell states poorly, so on a real screen it can call fewer groups than "mahalanobis" or "ks"; read it beside them rather than instead of them. With covariance="robust" it is the robust Hellinger distance of BioProfiling.jl (see Notes), which resists stray cells but trims a partial responder and runs far slower. "mp_value" scores the squared Mahalanobis distance between the group’s centroid and the controls’, measured in the average of the two sides’ covariances, the multidimensional perturbation value of Hutz et al. (see Notes). Unlike "mahalanobis" it counts the group’s own spread, so a group whose replicates disagree is measured in a wider covariance and moves less. The average is positive definite whenever the controls’ covariance is, so the group needs only two rows and the controls more rows than features; it works on well profiles with a few replicates, where "hellinger" cannot. A group with fewer rows is left unscored with a warning.

covariance str (default: 'empirical')

Scatter the Mahalanobis distance is measured in. "empirical" uses every fitting control row. "robust" uses the minimum covariance determinant subset, so a few stray control wells stop widening the covariance in their own direction and masking real hits there; it needs more control rows than features, so pair it with use_rep. Under method="hellinger" and method="mp_value" it is the location and scatter both sides are fitted with, refitted on every permutation; under method="mp_value" the group then needs more rows than features too.

use_rep str | None (default: None)

Score obsm[use_rep] instead of X. When the covariance-fitting half of the controls has no more rows than there are features, the covariance is singular and a warning suggests a PCA representation.

replicate str | Sequence[str] | None (default: None)

obs column, or sequence of columns, whose groups are the design’s exchangeable unit: the well in an arrayed screen, the guide ("Metadata_sgRNA") in a pooled one. The permutation null then draws whole replicates rather than cells, since cells within a well are not independent replicates (see Notes). Left None, it defaults to the physical well, the (Metadata_Plate, Metadata_Well) pair, on an object explicitly stamped cell resolution that has replicated wells; a same well name on two plates is then two replicates, not one. On such an object where some well holds both reference and tested cells, a pooled design, there is no default and it raises instead. It stays off when the object is not stamped cell resolution and when it has no complete well column, in which case the cell path warns; under method="ks" without a replicate the p-value then falls back to scipy.stats.ks_2samp.

n_permutations int (default: 1000)

Size of the permutation null. Applies to method="mahalanobis", method="hellinger" and method="mp_value", and to method="ks" when a replicate is in use.

threshold float (default: 0.05)

q-value below which a group is called a hit in is_hit.

seed int (default: 0)

Seed for the control split and the permutation null.

key_added str (default: 'hits')

Name for the outputs.

copy bool (default: False)

Return a modified copy instead of mutating in place.

Return type:

AnnData | None

Returns:

None, or the modified copy. Writes uns["mantispy"][key_added] with group, n_obs, distance, pvalue, qvalue and is_hit, where n_obs counts the rows the statistic used rather than the rows the group has, and joins obs[key_added + "_distance"] and obs[key_added + "_qvalue"] back onto the rows. Also writes obs[key_added + "_row_distance"], each row’s own distance from the control centroid rather than its group’s. That is the column a dose-response fit wants: the group statistic is one number repeated over the group’s rows, so the controls show no spread and nothing downstream can read a scale off them. Rows that fitted the centroid sit a little closer to it than the held-out controls do, by the same split the Notes describe. Which rows those are is written to obs[key_added + "_reference_held_out"], true for the controls that did not fit, so that a scale taken from the controls can be taken from the honest half.

Raises:

ValueError – method is not one of METHODS, reference selects fewer than four rows, or replicate is not given on a pooled design at cell resolution.

Notes

The controls are split in half. One half estimates the centroid and the covariance, and the other supplies the null. A null drawn from the rows that defined the centroid would be in-sample while every tested group is out-of-sample, so it would come out too small and call pure noise as hits. Because only half the controls fit the covariance, the singular-covariance warning fires when there are fewer than about twice as many controls as features.

A row that fitted the centroid and the covariance sits closer to the centroid than any other group’s row can, so it stays off the tested side of every group. The controls carry a perturbation label of their own, and that group is therefore left with the held-out half, which is split once more, at random, so that the rows tested and the rows they are tested against are different rows. Its row of the table is a draw from the null rather than a sample compared with part of itself measured against a centroid half of it placed, and n_obs reports about a quarter of the controls for it. Its null is the other ways to halve those rows rather than a bootstrap of them, which would be too wide because the sample is part of what it is drawn from. That row is therefore an honest draw from the null: its p-value is uniform and falls below any cutoff about as often as the cutoff says, so a control group appearing in a hit list is this test working rather than a fault.

The null is drawn from the controls only, and asks whether a group is further out than the same number of control rows would be. Drawing from every row would put real hits into the null, and a screen with many hits would look like one with none.

Calibration degrades with few controls. On pure-noise screens the false positive rate at a nominal 0.05 is inflated with few controls and approaches the nominal rate as controls are added. This is not a bound for another screen, and it was checked with method="mahalanobis". Under method="ks" with a replicate the null draws whole replicates for the pseudo-treated side and recomputes the KS statistic, the same unit replicate fixes for the median; without one each group’s distances are compared with those of the null half by scipy.stats.ks_2samp.

At cell resolution the calibration above assumes a null unit that matches the design. Cells within a well share the well, its plate position, seeding and focus, so they are not independent replicates: the exchangeable unit is the well, and a null that shuffles cells shrinks the statistic’s spread by the cell count rather than the well count and calls pure noise far above nominal. replicate fixes this by drawing whole wells. The controls are then split into whole wells too, one half fitting the centroid and covariance and the other forming the null, so every well in the null pool is complete and the reference group draws whole wells like any other group. replicate defaults to the physical well, the (Metadata_Plate, Metadata_Well) pair, on an object explicitly stamped cell resolution with replicated wells; it warns and leaves the cell-level test in place when no complete well column is present. In a pooled screen every well holds every perturbation, so drawing whole wells cannot separate a group from the reference; the exchangeable unit is the guide, and the call raises until replicate names it. With fewer than about sixteen wells in the null half the well-level null is coarse and runs conservative rather than anti-conservative; for an exact well-level test, aggregate with aggregate() first.

To check the rate on your own screen, diagnose_testing() relabels control wells as pseudo-treatments of your group sizes and reports the fraction called.

method="hellinger" follows the robust morphological perturbation value of BioProfiling.jl (Vulliard et al., Bioinformatics, 2022). It does not split the controls, because both Gaussians are refitted on every draw and no row is spent on a fixed centroid: a group is tested against every control row it does not hold, and its null draws whole replicates, or rows without a replicate, from the two pooled. Rows with a missing value are left out of both fits. With covariance="robust" each fit is the reweighted minimum covariance determinant estimate with the defaults of R robustbase’s covMcd, which BioProfiling.jl calls, so the distance agrees with BioProfiling.jl’s wherever the two searches reach the same subset. That robustness costs a partial responder: the MCD fits the most concentrated half of a side’s rows, so a responding subpopulation smaller than half of the group is trimmed as outlying, where the empirical fit keeps it. The robust fit is also refitted on every permutation, which makes it far slower than the empirical one; the empirical fit is the default for that reason.

method="mp_value" follows the multidimensional perturbation value of Hutz et al. (J Biomol Screen, 2013), with the covariance averaged over the group and the controls as its Methods describe; cytominer-eval’s mp_value uses the controls’ covariance alone. Like method="hellinger" it does not split the controls, and its null draws whole replicates, or rows without a replicate, from the group and the controls pooled. Hutz et al. first project each group and the controls onto the principal components that carry 90% of their variance; that step is left out, and use_rep sets the representation as for the other methods.