pandora_llm.utils.generation_utils

Module Contents

pandora_llm.utils.generation_utils.generate_suffixes(model, dataloader, prefix_length, generation_config, num_generations, accelerate)[source]

Generates from the model using the first prefix_length tokens from each sample in the dataloader.

Returns a numpy array of shape (num_samples,num_generations,prefix_length+suffix_length)

Parameters:
Return type:

numpy.ndarray

pandora_llm.utils.generation_utils.calculate_sentence_probability(logits, input_ids, condition_from_index=0, generation_config=None, logits_processor=LogitsProcessorList(), **kwargs)[source]

Calculates the probability that a sentence is decoded from the logits with given decoding strategy Operates in log space Uses logic of transformers/generation/utils.py

Parameters:

  • logits (torch.FloatTensor)

  • input_ids (torch.LongTensor)

  • condition_from_index (Optional[int])

  • generation_config (Optional[transformers.generation.utils.GenerationConfig])

  • logits_processor (Optional[transformers.generation.logits_process.LogitsProcessorList])

Return type:

torch.FloatTensor

pandora_llm.utils.generation_utils.compute_dataloader_suffix_probability(model, dataloader, prefix_length, generation_config)[source]
Parameters:
Return type:

numpy.ndarray