wx44wx/icassp2027-experiment
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix final-layer LoRA target matching

Browse Source
This commit is contained in:
Xin Wang
2026-08-24 15:38:45 +08:00
parent 7fdda297d3
commit aa4278e13e
2 changed files with 37 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
README.md
View File

@@ -1083,8 +1083,10 @@ The first follow-up to the sparse-LoRA regression should remove sparse
supervision as a variable and constrain adaptation near the Thinker output.
Use dense 5-Hz caches, reserve `Session_12` as development data, and train on
`Session_4` through `Session_11`. `--lora-last-n-layers 8` discovers the
language-layer indices and stores PEFT `layers_to_transform=[28,...,35]` for
the 36-layer Qwen2.5-Omni-3B Thinker. Combining it with
language-layer indices and passes the 24 selected full module paths directly
to PEFT for the 36-layer Qwen2.5-Omni-3B Thinker. Explicit paths are required
because the extracted Thinker exposes names beginning with `layers.0...`,
which some PEFT `layers_pattern` versions fail to recognize. Combining it with
`--lora-target-modules q_proj,v_proj,o_proj` reduces the adapter from all 252
attention/MLP projections to 24 output-side attention projections.