Metadata-Version: 2.4
Name: 3SAlgorythm
Version: 0.1.0
Summary: Batch-diversity strategies for Bayesian optimization — when naive top-k selection redundantly clusters a batch, and what to do about it.
Author: Jose Miguel Mangas
License: MIT License
        
        Copyright (c) 2026 Jose Miguel Mangas
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Repository, https://github.com/jmiguelmangas/3SAlgorythm
Project-URL: Findings, https://github.com/jmiguelmangas/3SAlgorythm/blob/main/docs/FINDINGS.md
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: scikit-learn
Provides-Extra: chem
Requires-Dist: rdkit; extra == "chem"
Provides-Extra: torch
Requires-Dist: torch; extra == "torch"
Requires-Dist: gpytorch; extra == "torch"
Requires-Dist: botorch; extra == "torch"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: matplotlib; extra == "dev"
Dynamic: license-file

# 3SAlgorythm

[![tests](https://github.com/jmiguelmangas/3SAlgorythm/actions/workflows/tests.yml/badge.svg)](https://github.com/jmiguelmangas/3SAlgorythm/actions/workflows/tests.yml)
[![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

Bayesian optimization benchmark: is it worth diversifying the batch when picking
which compounds to test in each round of a screen, instead of simply grabbing
whatever the model scores highest?

Executive summary with plots, molecules, and findings:
**[docs/FINDINGS.md](docs/FINDINGS.md)** (readable by anyone with access to this repo,
no claude.ai login needed; there's also an
[interactive version](https://claude.ai/code/artifact/509de377-240e-47d0-b225-dc968e689cfb),
only visible if you're signed into the account that published it).

## What's here

`divbatch/` is an installable package with the reusable core; everything
else at the repo root is the benchmark harness and scripts that use it.

- `divbatch/objectives.py` — objective functions: `CompoundScreen` (synthetic
  screen), `Branin`, `Hartmann6`, and `RealCompoundScreen` (real chemistry —
  any SMILES + continuous-property CSV, MoleculeNet-style; ECFP4 fingerprints
  via RDKit).
- `divbatch/optimizers.py` — `RandomPicker`, `GPExpectedImprovement` with
  pluggable batch selection (`TopKSelector`, `LocalPenalizationSelector`,
  `DPPSelector`, `KrigingBelieverSelector`) and a Tanimoto kernel for real
  chemistry. Core package API — only needs numpy/scipy/scikit-learn.
- `divbatch/recommend.py` — `divbatch-recommend`, a CLI that runs the actual
  TopK/LocalPenalization/DPP comparison on *your own* SMILES+activity CSV and
  reports which one performed best on your data. Exists because Experiment 7
  found no cheap, universal rule for which selector wins on a given
  library — so instead of guessing, this measures it directly, at a modest
  default budget. Installed automatically with the package.
- `divbatch/optimizers_torch.py` — BoTorch/GPyTorch optimizers (optional,
  needs the `torch` extra): `BoTorchQEI` (off-the-shelf
  `qLogExpectedImprovement`, jointly optimized over the batch);
  `TanimotoLocalPenalizedQEI` (hybrid — BoTorch's fitted GP for the quality
  signal, LocalPenalization's quality-agnostic repulsion for diversity,
  repulsion computed from the GP's own Tanimoto kernel instead of an ad-hoc
  Lipschitz radius); `SparseTanimotoLocalPenalizedGP` (same mechanism on a
  stochastic variational GP — scales to training sets an exact GP can't
  afford).
- `scalability_demo.py` — exact vs. sparse GP fit time as the training set
  grows past what this project's real dataset can exercise (synthetic
  fingerprints, since BACE only has 1513 compounds).
- `generalization_test.py` — repeats the TopK/LocalPenalization/DPP
  comparison on real MoleculeNet regression datasets (drug-like and
  quantum-chemistry) to check whether the BACE finding generalizes. It
  mostly doesn't — see `docs/FINDINGS.md`, Experiment 7.
- `isolation_diagnostic.py` — attempts to predict, from fingerprint geometry
  alone (no optimizer runs), which datasets LocalPenalization will win on.
  Negative result, confirmed by extending the dataset pool from 10 to 30 —
  the weak correlations at n=10 got weaker and partly flipped sign at n=30.
  See `docs/FINDINGS.md`.
- `benchmark.py` — command-line runner: sweeps optimizer × batch size × seed,
  saves results to JSON, and produces a comparison plot.
- `test_benchmark.py` — regression tests (pytest), also run on every push via
  GitHub Actions.
- `ALGORITHMS.md` / `SPEC.md` — the original spec and the mathematical detail
  of each selector.

## Installation

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -e ".[dev,chem]"
# optional, only for divbatch/optimizers_torch.py:
pip install -e ".[torch]"
```

`chem` pulls in RDKit (needed for `RealCompoundScreen`); `dev` pulls in
pytest and matplotlib (needed for the benchmark/test scripts, not for the
`divbatch` package itself). Someone who only wants `divbatch`'s core
selectors in their own project can just `pip install 3SAlgorythm` (once
published) or `pip install -e .` here with no extras — numpy/scipy/
scikit-learn are the only hard dependencies. The distribution is named
`3SAlgorythm` (matches this repo), but the importable package is still
`divbatch` — `pip install 3SAlgorythm` then `import divbatch` in your code,
the same split as e.g. `pip install beautifulsoup4` / `import bs4`.

The real-chemistry scenario uses MoleculeNet's BACE dataset:

```bash
mkdir -p data
curl -o data/bace.csv https://deepchemdata.s3-us-west-1.amazonaws.com/datasets/bace.csv
```

`generalization_test.py` (Experiment 7) needs 6 more MoleculeNet CSVs, same
source; `--extra` (the 20-dataset isolation-diagnostic follow-up) needs 2
more on top of those:

```bash
for f in delaney-processed.csv SAMPL.csv Lipophilicity.csv clearance.csv qm7.csv qm8.csv qm9.csv thermosol.csv hppb.csv; do
  curl -o "data/$f" "https://deepchemdata.s3-us-west-1.amazonaws.com/datasets/$f"
done
```

## Usage

```bash
pytest test_benchmark.py -q

python benchmark.py --quick                                    # quick sweep, synthetic
python benchmark.py                                             # full sweep (10 seeds, synthetic)
python benchmark.py --dataset real --seeds 0 1 2 3 4             # real data, PCA+Matern kernel
python benchmark.py --dataset real --optimizers GP-EI+LocalPenalization+Tanimoto \
    --seeds 0 1 2 3 4 --batch-sizes 25 50 --lp-alpha 1.0         # the study's winner
```

`--lp-alpha` controls `LocalPenalizationSelector`'s "risk appetite": `0` ignores
quality and prioritizes pure spatial novelty, `1` is the original behavior, higher
values are more conservative.

### `divbatch-recommend` — which selector should *you* use?

This project's own science (Experiment 7, `docs/FINDINGS.md`) found no
reliable, cheap-to-compute rule for predicting whether
`LocalPenalizationSelector` will beat plain top-EI selection on a given
library — it depends on properties of the data that isolation/clustering
metrics didn't capture, even across 30 independent test datasets. So rather
than ship a rule, `divbatch-recommend` runs the actual comparison on your
data:

```bash
pip install "3SAlgorythm[chem]"
divbatch-recommend my_screen.csv --smiles-col smiles --target-col activity
```

It needs a CSV with a SMILES column and a continuous property to maximize
(any MoleculeNet-style layout works). By default it simulates a 100-compound
budget in batches of 25, repeated over 6 random seeds — fast enough to run
interactively; `--seeds`, `--batch-size`, and `--n-experiments` scale it up
for a more reliable answer at the cost of runtime. It also flags a specific
failure mode this project ran into (Experiment 7's Clearance dataset): a
target capped at an assay ceiling, where many compounds tie for the maximum
and any strategy "wins" trivially.

## Where this fits

A quick survey of what's already out there, and why this project isn't redundant
with it:

- **BoTorch/Ax** (Meta) is the strongest general-purpose framework and what
  `divbatch/optimizers_torch.py` is built on. Its off-the-shelf batch
  acquisition (`qLogExpectedImprovement`) is joint-correct — it avoids picking
  near-duplicates by construction — but it's still quality-weighted, so it
  never bets on a genuinely low-predicted-improvement structural outlier.
  Confirmed twice in this project: with our own Tanimoto kernel, and
  independently with **GAUCHE**'s (see below) — same result both times.
- **GAUCHE** ships Tanimoto/fingerprint kernels for BoTorch out of the box —
  essentially a packaged version of `divbatch.optimizers.TanimotoKernel` /
  `divbatch.optimizers_torch.TanimotoKernelTorch`. It doesn't ship a
  quality-agnostic batch selector, so it inherits the same blind spot as
  plain BoTorch unless you add one.
- **Optuna**, the most popular hyperparameter-tuning library, distributes
  parallel workers close to independently — the same redundancy problem
  `TopKSelector` has, largely unaddressed.
- **EDBO** targets reaction optimization (catalyst × solvent × temperature),
  not library screening — a different problem shape than this project's.
- **Google Vertex AI Vizier**, **AWS SageMaker Automatic Model Tuning** — managed,
  black-box, built for ML hyperparameter tuning rather than experiment design
  with a domain-specific kernel.
- **SigOpt** (Intel) was discontinued as a managed service in September 2023;
  its code lives on as the unmaintained `sigopt-server`.

None of the above ships the mechanism this project's main finding depends on —
diversity that ignores predicted quality once something is spatially unique —
as a reusable component. `TanimotoLocalPenalizedQEI` in
`divbatch/optimizers_torch.py` is the attempt at closing that gap: BoTorch's
own fitted GP for the quality signal, LocalPenalization's repulsion for
diversity, computed from the GP's real Tanimoto kernel instead of a
separately-estimated Lipschitz constant.

Validated over 30 seeds on BACE: it finds the same structural outlier as the
hand-rolled sklearn selector in 27/30 (q=25) and 26/30 (q=50) seeds — against
0/30 for plain `qLogExpectedImprovement` and 0/30 for DPP — while running at
0.02–0.03s/round, faster than the sklearn version (0.2s) and ~250x faster than
off-the-shelf `qLogExpectedImprovement` (5–7s) at this scale.

## Main finding

On real data (BACE, 1513 compounds), with a Tanimoto kernel instead of the usual
PCA projection, `LocalPenalizationSelector` finds the most potent compound in the
entire library across all 30 seeds tested, without exception — neither DPP, nor
KrigingBeliever, nor BoTorch's native acquisition function (`qLogExpectedImprovement`)
manage it. Independently confirmed with GAUCHE's own Tanimoto kernel implementation,
not just ours.

**But it doesn't generalize.** Repeating the same comparison on 9 more real
datasets (`generalization_test.py`) gives 2 clear wins, 2 clear losses, and 6
ties out of 10 total — no domain pattern, roughly a coin flip. The BACE result
depends on a genuine structural outlier existing in the library; when there
isn't one, `LocalPenalization`'s quality-agnostic exploration is a mild
liability, not a free win. This is not a safe default upgrade over `TopK` —
see `docs/FINDINGS.md`, Experiment 7, for the full breakdown and the honest
recommendation table.

## Publishing (for maintainers)

Not yet published to PyPI. `.github/workflows/publish.yml` builds the sdist
and wheel and publishes on every GitHub Release, via
[PyPI Trusted Publishing](https://docs.pypi.org/trusted-publishers/) (OIDC —
no API token stored in the repo). PyPI project name is `3SAlgorythm` (the
distribution name in `pyproject.toml`; the importable package stays
`divbatch` — see Installation above), with a pending publisher already
registered for `jmiguelmangas/3SAlgorythm`, workflow `publish.yml`. After
that, publishing is just cutting a GitHub Release.

`python -m build && twine check dist/*` passes locally as of this package's
current state — the metadata and both artifacts are valid.

## License

[MIT](LICENSE).
