Normalizing Flows
The flows subpackage provides normalizing-flow models for configuration
sampling and physics-guided training.
Flow Networks
- class qvartools.flows.networks.discrete_flow.DiscreteFlowSampler(num_sites, num_coupling_layers=6, hidden_dims=None, prior_std=1.0, n_mc_samples=100)[source]
Bases:
ModuleRealNVP normalizing flow mapping continuous latent to discrete configs.
Uses alternating binary masks to split dimensions across coupling layers. The multi-modal prior ensures uniform coverage of both
{0, 1}values. Continuous outputs are clamped to[-1, 1]and discretised by thresholding at zero.- Parameters:
num_sites (
int) – Number of binary sites in each configuration.num_coupling_layers (
int, optional) – Number of affine coupling layers (default6).hidden_dims (
listofint, optional) – Hidden-layer sizes for the coupling networks (default[128, 128]).prior_std (
float, optional) – Standard deviation of the mixture-of-Gaussians prior components (default1.0).n_mc_samples (
int, optional) – Number of Monte Carlo samples for discrete log-probability estimation (default100).
- prior
The mixture-of-Gaussians prior.
- Type:
MultiModalPrior
- masks
Binary masks for each coupling layer.
- Type:
listoftorch.Tensor
- coupling_nets
Coupling networks for each layer.
- Type:
nn.ModuleList
Examples
>>> flow = DiscreteFlowSampler(num_sites=10, num_coupling_layers=4) >>> configs, unique = flow.sample(batch_size=256) >>> configs.shape torch.Size([256, 10])
- sample_continuous(batch_size)[source]
Sample continuous outputs from the flow, clamped to [-1, 1].
- Parameters:
batch_size (
int) – Number of samples to draw.- Returns:
Continuous samples clamped to
[-1, 1], shape(batch_size, num_sites).- Return type:
- static discretize(y)[source]
Discretise continuous outputs to binary {0, 1} by thresholding.
Values at or above zero are mapped to 1; values below zero are mapped to 0.
- Parameters:
y (
torch.Tensor) – Continuous tensor, shape(..., num_sites).- Returns:
Binary tensor with values in
{0, 1}, same shape as y, dtypetorch.float32.- Return type:
- sample(batch_size)[source]
Sample discrete binary configurations.
Draws continuous samples from the flow, discretises them, and returns both the full batch and the unique configurations.
- Parameters:
batch_size (
int) – Number of samples to draw.- Returns:
configs (
torch.Tensor) – All discrete configurations, shape(batch_size, num_sites).unique_configs (
torch.Tensor) – Unique configurations, shape(n_unique, num_sites)wheren_unique <= batch_size.
- Return type:
- log_prob_continuous(y)[source]
Compute log-probability in continuous space via change of variables.
Uses the inverse flow to map data-space samples back to the prior, then applies the change-of-variables formula:
log p(y) = log p_prior(z) + log |det J^{-1}|.- Parameters:
y (
torch.Tensor) – Continuous data-space samples, shape(batch, num_sites).- Returns:
Log-probabilities, shape
(batch,).- Return type:
- log_prob_discrete(x)[source]
Estimate discrete log-probability via Monte Carlo integration.
For each discrete configuration
xin{0, 1}^n, the probability is estimated by integrating the continuous density over the corresponding Voronoi cell ([-1, 0)for 0 and[0, 1]for 1). This is done by sampling uniform noise within each cell and averaging the continuous density.- Parameters:
x (
torch.Tensor) – Discrete configurations with values in{0, 1}, shape(batch, num_sites).- Returns:
Estimated log-probabilities, shape
(batch,).- Return type:
- forward(batch_size)[source]
Forward pass: sample and compute log-probabilities.
- Parameters:
batch_size (
int) – Number of samples to draw.- Returns:
configs (
torch.Tensor) – Discrete configurations, shape(batch_size, num_sites).unique_configs (
torch.Tensor) – Unique configurations, shape(n_unique, num_sites).log_probs (
torch.Tensor) – Continuous log-probabilities at the pre-discretisation points, shape(batch_size,).
- Return type:
- class qvartools.flows.networks.particle_conserving_flow.ParticleConservingFlowSampler(num_sites, n_alpha, n_beta, hidden_dims=None, temperature=1.0, min_temperature=0.01)[source]
Bases:
ModuleNormalizing flow that exactly conserves alpha and beta particle numbers.
Produces binary configurations of shape
(num_sites,)where the firstnum_sites // 2entries are alpha orbitals and the remaining are beta orbitals. Exactlyn_alphaalpha andn_betabeta orbitals are occupied in every sample.The flow works by:
Scoring alpha orbitals with a learned network.
Selecting the top
n_alphavia differentiable top-k.Scoring beta orbitals conditioned on the alpha configuration.
Selecting the top
n_betavia differentiable top-k.Concatenating
[alpha, beta]to form the full configuration.
- Parameters:
num_sites (
int) – Total number of spin-orbitals (must be even).n_alpha (
int) – Number of alpha electrons.n_beta (
int) – Number of beta electrons.hidden_dims (
listofint, optional) – Hidden-layer sizes for the scoring networks (default[128, 64]).temperature (
float, optional) – Initial temperature for differentiable top-k (default1.0).min_temperature (
float, optional) – Minimum temperature (default0.01).
- alpha_scorer
Scoring network for alpha orbitals.
- Type:
OrbitalScoringNetwork
- beta_scorer
Scoring network for beta orbitals (conditioned on alpha config).
- Type:
OrbitalScoringNetwork
- selector
Differentiable top-k selector.
- Type:
GumbelTopK
Examples
>>> flow = ParticleConservingFlowSampler( ... num_sites=10, n_alpha=2, n_beta=2 ... ) >>> configs, unique = flow.sample(batch_size=100) >>> is_valid, stats = verify_particle_conservation( ... configs, n_orbitals=5, n_alpha=2, n_beta=2 ... ) >>> assert is_valid
- set_temperature(temperature)[source]
Set the temperature for differentiable top-k selection.
- Parameters:
temperature (
float) – New temperature value. Will be clamped to at leastmin_temperature.- Return type:
None
- sample(batch_size, temperature=None)[source]
Sample particle-conserving binary configurations.
Each configuration has exactly
n_alphaoccupied alpha orbitals andn_betaoccupied beta orbitals.- Parameters:
- Returns:
all_configs (
torch.Tensor) – All sampled configurations, shape(batch_size, num_sites). The firstn_orbitalsentries are alpha, the remaining are beta.unique_configs (
torch.Tensor) – Unique configurations, shape(n_unique, num_sites).
- Return type:
- sample_without_replacement(batch_size, temperature=None)[source]
Sample unique configurations using deterministic ordering.
Generates a larger pool of samples and returns the unique configurations sorted by their logit scores (most probable first).
- Parameters:
- Returns:
Unique configurations, shape
(n_unique, num_sites)wheren_unique <= batch_size. Sorted by descending score.- Return type:
- forward(batch_size, temperature=None)[source]
Forward pass — delegates to
sample().- Parameters:
- Returns:
all_configs (
torch.Tensor) – All configurations, shape(batch_size, num_sites).unique_configs (
torch.Tensor) – Unique configurations, shape(n_unique, num_sites).
- Return type:
Training
- class qvartools.flows.training.physics_guided_training.PhysicsGuidedFlowTrainer(flow, nqs, hamiltonian, config, device='cpu')[source]
Bases:
objectMixed-objective trainer for joint flow + NQS optimisation.
Combines three loss terms with configurable weights:
Teacher loss:
-sum_x p_nqs(x) * log p_flow(x)— trains the flow to reproduce the NQS distribution.Physics loss: variational energy
E = sum_x |psi(x)|^2 * E_loc(x) / ZwhereE_loc(x) = sum_{x'} H_{x,x'} * psi(x') / psi(x)— minimises the ground-state energy estimate.Entropy loss:
-H[p_flow](negative entropy) — prevents mode collapse by encouraging distribution spread.
The trainer also:
Accumulates unique configurations into a growing basis set.
Anneals the flow temperature over early epochs.
Optionally injects essential (HF + singles + doubles) configurations into the basis.
Caches Hamiltonian connections for efficiency.
- Parameters:
flow (
nn.Module) – The normalizing flow sampler. Must implementsample(batch_size)returning(all_configs, unique_configs).nqs (
nn.Module) – The neural quantum state. Must implementlog_amplitude(x)returning log-amplitudes of shape(batch,).hamiltonian (
Hamiltonian) – The Hamiltonian operator.config (
PhysicsGuidedConfig) – Training hyperparameters.device (
str, optional) – Torch device override (default usesconfig.device).
- flow
The flow model.
- Type:
nn.Module
- nqs
The NQS model.
- Type:
nn.Module
- hamiltonian
The Hamiltonian.
- Type:
Hamiltonian
- config
Training configuration.
- Type:
- device
Active device.
- Type:
- accumulated_basis
Growing set of unique configurations seen during training.
- Type:
torch.TensororNone
- flow_optimizer
Optimiser for the flow parameters.
- Type:
- nqs_optimizer
Optimiser for the NQS parameters.
- Type:
- train(progress=True)[source]
Run the full training loop.
Trains for up to
config.num_epochsepochs, with early stopping when the unique-configuration ratio converges (change less thanconfig.convergence_thresholdfor two consecutive epochs afterconfig.min_epochs).- Parameters:
progress (
bool, optional) – IfTrue, log epoch-level metrics at INFO level (defaultTrue).- Returns:
Training history with keys matching the epoch metrics, each mapping to a list of per-epoch values:
"teacher_loss": list of float"physics_loss": list of float"entropy_loss": list of float"total_loss": list of float"mean_energy": list of float"unique_ratio": list of float"basis_size": list of int"temperature": list of float
- Return type:
- class qvartools.flows.training.physics_guided_training.PhysicsGuidedConfig(samples_per_batch=500, num_batches=10, num_epochs=200, min_epochs=50, convergence_threshold=0.01, flow_lr=0.001, nqs_lr=0.001, teacher_weight=1.0, physics_weight=0.0, entropy_weight=0.0, use_energy_baseline=True, ema_decay=0.99, use_connection_cache=True, max_cache_size=100000, initial_temperature=2.0, final_temperature=0.1, temperature_decay_epochs=100, inject_essential_configs=True, include_singles_in_basis=True, include_doubles_in_basis=True, device='cpu')[source]
Bases:
objectHyperparameters for
PhysicsGuidedFlowTrainer.All fields have sensible defaults for molecular-scale problems. The class is frozen (immutable) to prevent accidental mutation during training.
- Parameters:
samples_per_batch (
int) – Number of flow samples per mini-batch (default500).num_batches (
int) – Number of mini-batches per epoch (default10).num_epochs (
int) – Maximum number of training epochs (default200).min_epochs (
int) – Minimum epochs before convergence checks activate (default50).convergence_threshold (
float) – Training stops when the unique-configuration ratio changes by less than this amount over consecutive epochs (default0.01).flow_lr (
float) – Learning rate for the flow optimiser (default1e-3).nqs_lr (
float) – Learning rate for the NQS optimiser (default1e-3).teacher_weight (
float) – Weight of the teacher (KL) loss (default1.0).physics_weight (
float) – Weight of the variational energy loss (default0.0).entropy_weight (
float) – Weight of the entropy regularisation loss (default0.0).use_energy_baseline (
bool) – Whether to subtract a running baseline from the energy for variance reduction (defaultTrue).ema_decay (
float) – Exponential moving average decay for the energy baseline (default0.99).use_connection_cache (
bool) – Whether to cache Hamiltonian connections for repeated configs (defaultTrue).max_cache_size (
int) – Maximum number of entries in the connection cache (default100000).initial_temperature (
float) – Starting temperature for flow annealing (default2.0).final_temperature (
float) – Final temperature after annealing (default0.1).temperature_decay_epochs (
int) – Number of epochs over which to anneal temperature (default100).inject_essential_configs (
bool) – Whether to inject Hartree–Fock and nearby configurations into the basis (defaultTrue).include_singles_in_basis (
bool) – Whether to include single excitations in the essential basis (defaultTrue).include_doubles_in_basis (
bool) – Whether to include double excitations in the essential basis (defaultTrue).device (
str) – Torch device for training (default"cpu").
Loss Functions
loss_functions — Loss computation for physics-guided flow training
Standalone loss functions and supporting utilities extracted from the physics-guided training loop:
compute_teacher_loss()— KL-divergence teacher loss.compute_physics_loss()— Variational energy loss with EMA baseline for variance reduction.compute_entropy_loss()— Negative-entropy regularisation.compute_local_energy()— Per-configuration local energy via the Hamiltonian connections.ConnectionCache— LRU-style cache for Hamiltonian connection lookups.
- class qvartools.flows.training.loss_functions.ConnectionCache(max_size=100000)[source]
Bases:
objectHash-based LRU cache for Hamiltonian connections.
Stores
(connected_configs, matrix_elements)tuples keyed by the integer hash of each configuration. Provides O(1) lookup and LRU eviction when the cache exceedsmax_size.- Parameters:
max_size (
int, optional) – Maximum number of entries the cache may hold (default100000). When the cache is full, the least-recently-used entry is evicted on the nextput()orget_or_compute()call.
Examples
>>> cache = ConnectionCache(max_size=1000) >>> config = torch.tensor([1, 0, 1, 0]) >>> cache.put(config, connected, elements) >>> result = cache.get(config)
- hash_batch(configs)[source]
Hash a batch of configurations.
For
n_sites <= 63uses a single matmul and returns an int64 tensor. Forn_sites >= 64falls back to per-config tuple keys and returns a list.- Parameters:
configs (
torch.Tensor) – Binary configurations, shape(n_configs, n_sites).- Returns:
Integer hashes (tensor) or tuple keys (list).
- Return type:
torch.Tensororlist
- get_batch(configs)[source]
Look up cached connections for a batch of configurations.
- Parameters:
configs (
torch.Tensor) – Binary configurations, shape(n_configs, n_sites).- Returns:
One entry per config:
(connected, elements)if cached,Noneotherwise. Cache hits are promoted to most-recently-used.- Return type:
- get(config, hamiltonian=None)[source]
Look up cached connections for a configuration.
On a cache hit the entry is promoted to most-recently-used.
If hamiltonian is provided the call behaves like
get_or_compute()for backward compatibility: a cache miss triggershamiltonian.get_connections(config), caches the result, and returns it (never returnsNone).- Parameters:
config (
torch.Tensor) – Binary configuration vector, shape(n_sites,).hamiltonian (
HamiltonianorNone, optional) – If given, compute and cache on miss instead of returningNone.
- Returns:
(connected_configs, matrix_elements)if found (or computed), otherwiseNonewhen hamiltonian is not provided.- Return type:
- put(config, connections, elements)[source]
Store connections for a configuration in the cache.
If the cache is at capacity, the least-recently-used entry is evicted.
- Parameters:
config (
torch.Tensor) – Binary configuration vector, shape(n_sites,).connections (
torch.Tensor) – Connected configurations, shape(n_conn, n_sites).elements (
torch.Tensor) – Matrix elements, shape(n_conn,).
- Return type:
None
- get_or_compute(config, hamiltonian)[source]
Retrieve connections, computing and caching if absent.
This unifies the lookup-or-compute pattern: if the configuration is already cached its entry is returned (and promoted to most-recently-used); otherwise
hamiltonian.get_connectionsis called, the result is cached, and then returned.- Parameters:
config (
torch.Tensor) – Single configuration, shape(num_sites,).hamiltonian (
Hamiltonian) – The Hamiltonian to query for connections on a cache miss.
- Returns:
connected_configs (
torch.Tensor) – Connected configurations, shape(n_conn, num_sites).matrix_elements (
torch.Tensor) – Corresponding matrix elements, shape(n_conn,).
- Return type:
- stats()[source]
Return cache performance statistics.
- Returns:
Dictionary with keys:
"hits": int — number of successful lookups."misses": int — number of failed lookups."hit_rate": float — fraction of lookups that were hits (0.0 if no lookups have been made)."size": int — current number of cached entries.
- Return type:
- qvartools.flows.training.loss_functions.compute_teacher_loss(configs, log_probs_flow, nqs)[source]
Compute the teacher (KL divergence) loss.
L_teacher = -sum_x p_nqs(x) * log p_flow(x)The NQS probabilities are detached (treated as fixed targets).
- Parameters:
configs (
torch.Tensor) – Sampled configurations, shape(batch, num_sites).log_probs_flow (
torch.Tensor) – Flow log-probabilities, shape(batch,).nqs (
nn.Module) – Neural quantum state with alog_amplitude(x)method.
- Returns:
Scalar teacher loss.
- Return type:
- qvartools.flows.training.loss_functions.compute_physics_loss(configs, nqs, hamiltonian, device, energy_baseline, baseline_initialized, use_energy_baseline, ema_decay, connection_cache=None)[source]
Compute the variational energy (physics) loss.
L_physics = sum_x |psi(x)|^2 * E_loc(x) / ZUses a running EMA baseline for variance reduction when enabled.
- Parameters:
configs (
torch.Tensor) – Sampled configurations, shape(batch, num_sites).nqs (
nn.Module) – Neural quantum state with alog_amplitude(x)method.hamiltonian (
Hamiltonian) – The Hamiltonian operator.device (
torch.device) – Torch device for computation.energy_baseline (
float) – Current EMA energy baseline value.baseline_initialized (
bool) – Whether the baseline has been initialised.use_energy_baseline (
bool) – Whether to apply variance reduction via EMA baseline.ema_decay (
float) – Exponential moving average decay for the baseline.connection_cache (
ConnectionCacheorNone, optional) – Optional cache for Hamiltonian connections.
- Returns:
loss (
torch.Tensor) – Scalar physics loss.mean_energy (
float) – Mean local energy (for logging).updated_baseline (
float) – Updated EMA energy baseline.updated_initialized (
bool) – Whether the baseline is now initialised.
- Return type:
- qvartools.flows.training.loss_functions.compute_entropy_loss(log_probs_flow)[source]
Compute the negative entropy of the flow distribution.
L_entropy = sum_x p_flow(x) * log p_flow(x) = -H[p_flow]Minimising this loss maximises the entropy.
- Parameters:
log_probs_flow (
torch.Tensor) – Flow log-probabilities, shape(batch,).- Returns:
Scalar entropy loss (negative entropy).
- Return type:
- qvartools.flows.training.loss_functions.compute_local_energy(configs, nqs, hamiltonian, device, connection_cache=None)[source]
Compute the local energy E_loc(x) for each configuration.
E_loc(x) = H_{x,x} + sum_{x' != x} H_{x,x'} * psi(x') / psi(x)Optimised to minimise CPU-GPU transfers and batch all NQS evaluations into a single call.
- Parameters:
configs (
torch.Tensor) – Configurations, shape(batch, num_sites).nqs (
nn.Module) – Neural quantum state with alog_amplitude(x)method.hamiltonian (
Hamiltonian) – The Hamiltonian operator.device (
torch.device) – Torch device for computation.connection_cache (
ConnectionCacheorNone, optional) – Optional cache for Hamiltonian connections.
- Returns:
Local energies, shape
(batch,).- Return type:
Gumbel Top-k
gumbel_topk — Differentiable top-k selection mechanisms
Provides differentiable approximations to top-k selection for use in particle-number-conserving normalizing flows:
GumbelTopK— Gumbel-Softmax-based iterative selection.SigmoidTopK— Sigmoid thresholding with implicit binary search.
- class qvartools.flows.training.gumbel_topk.GumbelTopK(temperature=1.0, min_temperature=0.01)[source]
Bases:
ModuleGumbel-Softmax-based differentiable top-k selection.
Adds Gumbel noise to logits and applies a softmax to produce a soft approximation of top-k selection. At low temperatures the selection approaches a hard top-k; at high temperatures it is fully stochastic.
- Parameters:
- forward(logits, k, temperature=None)[source]
Select k elements via Gumbel-Softmax relaxation.
- Parameters:
logits (
torch.Tensor) – Unnormalised scores, shape(batch, n).k (
int) – Number of elements to select.temperature (
floatorNone, optional) – Override temperature for this call. IfNone, uses the instance temperature.
- Returns:
Soft selection mask, shape
(batch, n). Values are in[0, 1]and approximately sum tokper row.- Return type:
- class qvartools.flows.training.gumbel_topk.SigmoidTopK(temperature=1.0, min_temperature=0.01)[source]
Bases:
ModuleSigmoid-based differentiable top-k selection with implicit threshold.
Uses a learned or computed threshold to produce per-element sigmoid activations, then normalises to select exactly k elements in expectation.
- Parameters:
- forward(logits, k, temperature=None)[source]
Select k elements via sigmoid thresholding.
Finds a threshold such that
sum(sigmoid((logits - threshold) / T))is approximatelyk, then returns the sigmoid activations.- Parameters:
logits (
torch.Tensor) – Unnormalised scores, shape(batch, n).k (
int) – Number of elements to select.temperature (
floatorNone, optional) – Override temperature. IfNone, uses instance temperature.
- Returns:
Soft selection mask, shape
(batch, n). Values are in[0, 1]and approximately sum tokper row.- Return type:
Utilities
- qvartools.flows.networks.particle_conserving_flow.verify_particle_conservation(configs, n_orbitals, n_alpha, n_beta)[source]
Validate that all configurations conserve particle numbers.
Checks that each configuration has exactly
n_alphaoccupied alpha orbitals (firstn_orbitalssites) andn_betaoccupied beta orbitals (remainingn_orbitalssites).- Parameters:
configs (
torch.Tensor) – Binary configurations, shape(n_configs, 2 * n_orbitals).n_orbitals (
int) – Number of spatial orbitals (half ofnum_sites).n_alpha (
int) – Expected number of alpha electrons per configuration.n_beta (
int) – Expected number of beta electrons per configuration.
- Returns:
is_valid (
bool) –Trueif every configuration has exactly the correct particle numbers.stats (
dict) – Dictionary with detailed statistics:"n_configs": int — total number of configurations."n_valid": int — number of valid configurations."n_invalid": int — number of invalid configurations."alpha_counts": torch.Tensor — alpha electron count per config."beta_counts": torch.Tensor — beta electron count per config."alpha_violations": int — configs with wrong alpha count."beta_violations": int — configs with wrong beta count.
- Return type:
Examples
>>> configs = torch.tensor([[1, 1, 0, 1, 0, 1]]) # 2 alpha, 2 beta >>> is_valid, stats = verify_particle_conservation(configs, 3, 2, 2) >>> is_valid True