SQwash API summary¶
To go back to the main page, click here.
Module API of reducers:

class
sqwash.
MeanReducer
¶ Reduce a batch of values by their mean.
 Parameters
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.

class
sqwash.
SuperquantileReducer
(superquantile_tail_fraction=0.5)¶ Reduce a batch of values by their superquantile (a.k.a. Conditional Value at Risk, CVaR).
It is given by the value of the linear program
\[\begin{split}\min \quad & \,\, \langle q, \ell\rangle \\ \text{s.t. } \quad & 0 \le q \le \frac{1}{\theta n}, \\ & \sum_{i=1}^n q_i = 1,\end{split}\]where \(\ell\) represents the input batch to the forward method, \(n\) is the dimension of \(\ell\) (and also of \(q\)) and \(\theta\) is superquantile_tail_fraction and \(\nu\) is smoothing_coefficient.
 Parameters
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().
Examples:
criterion = torch.nn.CrossEntropyLoss(reduction='none') # Note: must set `reduction='none'` reducer = SuperquantileReducer(superquantile_tail_fraction=0.6) # define the reducer # Training loop for x, y in dataloader: y_hat = model(x) batch_losses = criterion(y_hat, y) # shape: (batch_size,) loss = reducer(batch_losses) # Additional line to use the superquantile reducer loss.backward() # Proceed as usual from here ...

class
sqwash.
SuperquantileSmoothReducer
(superquantile_tail_fraction, smoothing_coefficient)¶ Reduce a batch of values by a L2 smoothing of their superquantile (a.k.a. Conditional Value at Risk, CVaR).
It is given by the value of the quadratic program
\[\begin{split}\min \quad & \, \, \langle q, \ell \rangle  \frac{\nu}{2n} \big\ q  \frac{\mathbf{1}_n}{n} \big\_2^2 \\ \text{s.t. } \quad & 0 \le q \le \frac{1}{\theta n}, \\ & \sum_{i=1}^n q_i = 1,\end{split}\]where \(\ell\) represents batch, \(n\) is the dimension of \(\ell\) (and also of \(q\)) and \(\theta\) is superquantile_tail_fraction and \(\nu\) is smoothing_coefficient.
 Parameters
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().
smoothing_coefficient (float, default=1.0) – How much to smooth by?
Examples:
criterion = torch.nn.CrossEntropyLoss(reduction='none') # Note: must set `reduction='none'` reducer = SuperquantileSmoothReducer( # define the reducer superquantile_tail_fraction=0.6, smoothing_coefficient=1.0 ) # Training loop for x, y in dataloader: y_hat = model(x) batch_losses = criterion(y_hat, y) # shape: (batch_size,) loss = reducer(batch_losses) # Additional line to use the superquantile reducer loss.backward() # Proceed as usual from here ...
Functional API of reducers:

sqwash.
reduce_mean
(batch: torch.Tensor) → torch.Tensor¶ Reduce a batch of values by their mean.
 Parameters
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.

sqwash.
reduce_superquantile
(batch: torch.Tensor, superquantile_tail_fraction: float = 0.5) → torch.Tensor¶ Reduce a batch of values by their superquantile (a.k.a. Conditional Value at Risk, CVaR).
See the class
SuperquantileReducer
for details. Parameters
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().
Examples:
for x, y in dataloader: y_hat = model(x) batch_loss = torch.nn.functional.cross_entropy(y_hat, y, reduction='none') # must set `reduction='none'` loss = reduce_superquantile(batch_loss, superquantile_tail_fraction=0.6) loss.backward() # Proceed as usual from here ...

sqwash.
reduce_superquantile_smooth
(batch: torch.Tensor, superquantile_tail_fraction: float = 0.5, smoothing_coefficient: float = 1.0) → torch.Tensor¶ Reduce a batch of values by a L2 smoothing of their superquantile (a.k.a. Conditional Value at Risk, CVaR).
See the documentation of the class
SuperquantileSmoothReducer
for details. Parameters
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().
smoothing_coefficient (float, default=1.0) – How much to smooth by?
Examples:
for x, y in dataloader: y_hat = model(x) batch_loss = torch.nn.functional.cross_entropy(y_hat, y, reduction='none') # must set `reduction='none'` loss = reduce_superquantile_smooth( batch_loss, superquantile_tail_fraction=0.6, smoothing_coefficient=1.0 ) loss.backward() # Proceed as usual from here ...
To go back to the main page, click here.