PEFT Adapters

Parameter-efficient fine-tuning adapter configurations supported by Takt.

taktiny.takt.BaseAdapter

alias of AdapterBase

class taktiny.takt.LoRAAdapter(targets, rank=8, alpha=16.0, *, dtype=None, rngs)

Bases: AdapterBase

Apply LoRALinear to modules selected by regex patterns.

Parameters:

• targets (str | list[str] | tuple[str, ...])

• rank (int)

• alpha (float)

• dtype (DType | None)

• rngs (Rngs)

class taktiny.takt.DoRAAdapter(targets, rank=8, alpha=16.0, *, dtype=None, rngs, **kwargs)

Bases: AdapterBase

Apply DoRALinear to matching modules.

Parameters:

• targets (str | list[str] | tuple[str, ...])

• rank (int)

• alpha (float)

• dtype (DType | None)

• rngs (Rngs)

• kwargs (Any)

class taktiny.takt.AdaLoRAAdapter(targets, rank=8, alpha=16.0, *, dtype=None, rngs, **kwargs)

Bases: AdapterBase

Apply AdaLoRALinear to matching modules.

Parameters:

• targets (str | list[str] | tuple[str, ...])

• rank (int)

• alpha (float)

• dtype (DType | None)

• rngs (Rngs)

• kwargs (Any)

class taktiny.takt.LoHaAdapter(targets, rank=8, alpha=16.0, *, dtype=None, rngs, **kwargs)

Bases: AdapterBase

Apply LoHaLinear to matching modules.

Parameters:

• targets (str | list[str] | tuple[str, ...])

• rank (int)

• alpha (float)

• dtype (DType | None)

• rngs (Rngs)

• kwargs (Any)

class taktiny.takt.LoKrAdapter(targets, rank=8, alpha=16.0, *, dtype=None, rngs, **kwargs)

Bases: AdapterBase

Apply LoKrLinear to matching modules.

Parameters:

• targets (str | list[str] | tuple[str, ...])

• rank (int)

• alpha (float)

• dtype (DType | None)

• rngs (Rngs)

• kwargs (Any)

class taktiny.takt.VeRAAdapter(targets, rank=8, *, dtype=None, d_initial=0.1, rngs, initializer=<function variance_scaling.<locals>.init>, vera_A=None, vera_B=None, **kwargs)

Bases: AdapterBase

Apply VeRA using frozen random projections shared by all targets.

Parameters:

• targets (str | list[str] | tuple[str, ...])

• rank (int)

• dtype (DType | None)

• d_initial (float)

• rngs (Rngs)

• initializer (Initializer)

• vera_A (Parameter | None)

• vera_B (Parameter | None)

• kwargs (Any)