pandora_llm.features.ZLIB

Module Contents

class pandora_llm.features.ZLIB.ZLIB[source]

Bases: pandora_llm.features.base.FeatureComputer

zlib thresholding attack

compute_features(dataloader)[source]

Compute the negative zlib statistic for a given dataloader.

Parameters:

dataloader (torch.utils.data.DataLoader) – input text data to compute statistic over

Returns:

zlib entropy of input IDs

Return type:

jaxtyping.Num[torch.Tensor, n]

pandora_llm.features.ZLIB.compute_zlib_dl(dataloader)[source]

Compute the zlib cross entropy. Does not batch.

Code taken from https://github.com/ftramer/LM_Memorization/blob/main/extraction.py

Parameters:

dataloader (torch.utils.data.DataLoader) – input text data to compute statistic over

Returns:

zlib entropy of input IDs

Return type:

jaxtyping.Integer[torch.Tensor, n]