SQwash API summary#
To go back to the main page, click here.
Module API of reducers:
- class sqwash.MeanReducer#
Reduce a batch of values by their mean.
- Parameters:
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
- class sqwash.SuperquantileReducer(superquantile_tail_fraction=0.5)#
Reduce a batch of values by their superquantile (a.k.a. Conditional Value at Risk, CVaR).
It is given by the value of the linear program
\[\begin{split}\min \quad & \,\, \langle q, \ell\rangle \\ \text{s.t. } \quad & 0 \le q \le \frac{1}{\theta n}, \\ & \sum_{i=1}^n q_i = 1,\end{split}\]where \(\ell\) represents the input batch to the forward method, \(n\) is the dimension of \(\ell\) (and also of \(q\)) and \(\theta\) is superquantile_tail_fraction and \(\nu\) is smoothing_coefficient.
- Parameters:
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().
Examples:
criterion = torch.nn.CrossEntropyLoss(reduction='none') # Note: must set `reduction='none'` reducer = SuperquantileReducer(superquantile_tail_fraction=0.6) # define the reducer # Training loop for x, y in dataloader: y_hat = model(x) batch_losses = criterion(y_hat, y) # shape: (batch_size,) loss = reducer(batch_losses) # Additional line to use the superquantile reducer loss.backward() # Proceed as usual from here ...
- class sqwash.SuperquantileSmoothReducer(superquantile_tail_fraction, smoothing_coefficient)#
Reduce a batch of values by a L2 smoothing of their superquantile (a.k.a. Conditional Value at Risk, CVaR).
It is given by the value of the quadratic program
\[\begin{split}\min \quad & \, \, \langle q, \ell \rangle - \frac{\nu}{2n} \big\| q - \frac{\mathbf{1}_n}{n} \big\|_2^2 \\ \text{s.t. } \quad & 0 \le q \le \frac{1}{\theta n}, \\ & \sum_{i=1}^n q_i = 1,\end{split}\]where \(\ell\) represents batch, \(n\) is the dimension of \(\ell\) (and also of \(q\)) and \(\theta\) is superquantile_tail_fraction and \(\nu\) is smoothing_coefficient.
- Parameters:
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().
smoothing_coefficient (float, default=1.0) – How much to smooth by?
Examples:
criterion = torch.nn.CrossEntropyLoss(reduction='none') # Note: must set `reduction='none'` reducer = SuperquantileSmoothReducer( # define the reducer superquantile_tail_fraction=0.6, smoothing_coefficient=1.0 ) # Training loop for x, y in dataloader: y_hat = model(x) batch_losses = criterion(y_hat, y) # shape: (batch_size,) loss = reducer(batch_losses) # Additional line to use the superquantile reducer loss.backward() # Proceed as usual from here ...
Functional API of reducers:
- sqwash.reduce_mean(batch: Tensor) Tensor #
Reduce a batch of values by their mean.
- Parameters:
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
- sqwash.reduce_superquantile(batch: Tensor, superquantile_tail_fraction: float = 0.5) Tensor #
Reduce a batch of values by their superquantile (a.k.a. Conditional Value at Risk, CVaR).
See the class
SuperquantileReducer
for details.- Parameters:
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().
Examples:
for x, y in dataloader: y_hat = model(x) batch_loss = torch.nn.functional.cross_entropy(y_hat, y, reduction='none') # must set `reduction='none'` loss = reduce_superquantile(batch_loss, superquantile_tail_fraction=0.6) loss.backward() # Proceed as usual from here ...
- sqwash.reduce_superquantile_smooth(batch: Tensor, superquantile_tail_fraction: float = 0.5, smoothing_coefficient: float = 1.0) Tensor #
Reduce a batch of values by a L2 smoothing of their superquantile (a.k.a. Conditional Value at Risk, CVaR).
See the documentation of the class
SuperquantileSmoothReducer
for details.- Parameters:
batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.
superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().
smoothing_coefficient (float, default=1.0) – How much to smooth by?
Examples:
for x, y in dataloader: y_hat = model(x) batch_loss = torch.nn.functional.cross_entropy(y_hat, y, reduction='none') # must set `reduction='none'` loss = reduce_superquantile_smooth( batch_loss, superquantile_tail_fraction=0.6, smoothing_coefficient=1.0 ) loss.backward() # Proceed as usual from here ...
To go back to the main page, click here.