List of layer names from which to capture and save output tensors.

The Evo 2 model architecture consists of two types of layers:

• HyenaLayer: Uses Hyena mixer for efficient long-range modeling
• TransformerLayer: Uses multi-head self-attention mechanism

Layer distribution by model size:

• 7B models: 32 layers total
  • HyenaLayers: all layers except 3, 10, 17, 24, 31
  • TransformerLayers: layers 3, 10, 17, 24, 31
• 40B models: 50 layers total
  • HyenaLayers: all layers except 3, 10, 17, 24, 31, 35, 42, 49
  • TransformerLayers: layers 3, 10, 17, 24, 31, 35, 42, 49

Model-level layers:

• embedding: Input token embeddings (typically layer 0).
  Note: This refers to the static token embeddings before any
  model computation. This layer is rarely useful for downstream
  analysis. Instead, it is recommended to use the output of an
  intermediate layer or the final MLP layer of a layer (as listed
  below), since these actually capture context-dependent features.
• decoder.final_norm: Final model layer normalization
• output_layer: Final output/logits of shape [seq_len, batch_size,
  512], where 512 is the padded vocabulary size of the tokenizer.

HyenaLayer components (available in HyenaLayers only):

• decoder.layers.[n].mixer: Output of the Hyena mixer submodule in layer [n].
  This captures the short, medium, and long-range sequence modeling
  capabilities of the Hyena mechanism, depending on the layer index.

TransformerLayer components (available in TransformerLayers only):

• decoder.layers.[n].self_attention: Multi-head attention output in layer [n]
• decoder.layers.[n].self_attention.linear_qkv: Query, key, and value projection weights combined
• decoder.layers.[n].self_attention.core_attention: The self-attention mechanism
• decoder.layers.[n].self_attention.linear_proj: Output projection layer

MLP components (available in both layer types):

• decoder.layers.[n].mlp: Complete MLP output in layer [n]
• decoder.layers.[n].mlp.linear_fc1: First MLP layer in layer [n]
• decoder.layers.[n].mlp.linear_fc2: Second MLP layer in layer [n]

Where [n] is the layer index:

• For 7B models: 0 to 31
• For 40B models: 0 to 49

For example: ["output_layer", "decoder.layers.20.mlp.linear_fc2", "decoder.layers.3.self_attention"]

Note: Evo 2's vocabulary size is 512, but for DNA sequence generation,
only 4 tokens (A, C, T, G) are meaningful in the output of the pretrained model.
The remaining tokens are present in the vocabulary for technical reasons
(e.g., input prompts, tokenizer padding, or future fine-tuning), but are not
expected to be generated as output nucleotides.

The mapping from logit indices to DNA bases is as follows:

• A: ASCII 65
• C: ASCII 67
• T: ASCII 84
• G: ASCII 71

The top_k parameter is allowed to be up to 6 for compatibility, but in practice,
only the 4 DNA bases are relevant for sequence generation.