# SQwash API summary¶

Module API of reducers:

class sqwash.MeanReducer

Reduce a batch of values by their mean.

Parameters

batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.

class sqwash.SuperquantileReducer(superquantile_tail_fraction=0.5)

Reduce a batch of values by their superquantile (a.k.a. Conditional Value at Risk, CVaR).

It is given by the value of the linear program

$\begin{split}\min \quad & \,\, \langle q, \ell\rangle \\ \text{s.t. } \quad & 0 \le q \le \frac{1}{\theta n}, \\ & \sum_{i=1}^n q_i = 1,\end{split}$

where $$\ell$$ represents the input batch to the forward method, $$n$$ is the dimension of $$\ell$$ (and also of $$q$$) and $$\theta$$ is superquantile_tail_fraction and $$\nu$$ is smoothing_coefficient.

Parameters
• batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.

• superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().

Examples:

criterion = torch.nn.CrossEntropyLoss(reduction='none')  # Note: must set reduction='none'
reducer = SuperquantileReducer(superquantile_tail_fraction=0.6)  # define the reducer

# Training loop
y_hat = model(x)
batch_losses = criterion(y_hat, y)  # shape: (batch_size,)
loss = reducer(batch_losses)  # Additional line to use the superquantile reducer
loss.backward()  # Proceed as usual from here
...

class sqwash.SuperquantileSmoothReducer(superquantile_tail_fraction, smoothing_coefficient)

Reduce a batch of values by a L2 smoothing of their superquantile (a.k.a. Conditional Value at Risk, CVaR).

It is given by the value of the quadratic program

$\begin{split}\min \quad & \, \, \langle q, \ell \rangle - \frac{\nu}{2n} \big\| q - \frac{\mathbf{1}_n}{n} \big\|_2^2 \\ \text{s.t. } \quad & 0 \le q \le \frac{1}{\theta n}, \\ & \sum_{i=1}^n q_i = 1,\end{split}$

where $$\ell$$ represents batch, $$n$$ is the dimension of $$\ell$$ (and also of $$q$$) and $$\theta$$ is superquantile_tail_fraction and $$\nu$$ is smoothing_coefficient.

Parameters
• batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.

• superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().

• smoothing_coefficient (float, default=1.0) – How much to smooth by?

Examples:

criterion = torch.nn.CrossEntropyLoss(reduction='none')  # Note: must set reduction='none'
reducer = SuperquantileSmoothReducer(  # define the reducer
superquantile_tail_fraction=0.6, smoothing_coefficient=1.0
)

# Training loop
y_hat = model(x)
batch_losses = criterion(y_hat, y)  # shape: (batch_size,)
loss = reducer(batch_losses)  # Additional line to use the superquantile reducer
loss.backward()  # Proceed as usual from here
...


Functional API of reducers:

sqwash.reduce_mean(batch: torch.Tensor)torch.Tensor

Reduce a batch of values by their mean.

Parameters

batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.

sqwash.reduce_superquantile(batch: torch.Tensor, superquantile_tail_fraction: float = 0.5)torch.Tensor

Reduce a batch of values by their superquantile (a.k.a. Conditional Value at Risk, CVaR).

See the class SuperquantileReducer for details.

Parameters
• batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.

• superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().

Examples:

for x, y in dataloader:
y_hat = model(x)
batch_loss = torch.nn.functional.cross_entropy(y_hat, y, reduction='none')  # must set reduction='none'
loss = reduce_superquantile(batch_loss, superquantile_tail_fraction=0.6)
loss.backward()  # Proceed as usual from here
...

sqwash.reduce_superquantile_smooth(batch: torch.Tensor, superquantile_tail_fraction: float = 0.5, smoothing_coefficient: float = 1.0)torch.Tensor

Reduce a batch of values by a L2 smoothing of their superquantile (a.k.a. Conditional Value at Risk, CVaR).

See the documentation of the class SuperquantileSmoothReducer for details.

Parameters
• batch (torch.Tensor) – Tensor of values to reduce. Reshaped into a single dimension.

• superquantile_tail_fraction (float, default is 0.5) – What fraction of the tail to average over for the superquantile. If 1.0, the function returns batch.mean() and if 0.0, the function returns batch.max().

• smoothing_coefficient (float, default=1.0) – How much to smooth by?

Examples:

for x, y in dataloader:
y_hat = model(x)
batch_loss = torch.nn.functional.cross_entropy(y_hat, y, reduction='none')  # must set reduction='none'
loss = reduce_superquantile_smooth(
batch_loss, superquantile_tail_fraction=0.6, smoothing_coefficient=1.0
)
loss.backward()  # Proceed as usual from here
...