fedem.configurations package

Submodules

fedem.configurations.mamba module

class fedem.configurations.mamba.MambaConfig(vocab_size=50277, d_state=16, d_model=2560, d_conv=4, expand=2, conv_bias=True, bias=False, n_layer=64, dt_rank: int | str = 'auto', pad_vocab_size_multiple=8, initializer_range=0.02, **kwargs)

Bases: PretrainedConfig

model_type: str = 'mamba'

Module contents