Thanks for the quick response. I added more information to reproduce the error in the PR description and replaced the try-expect block. Let me know about the preferred position for handling the attribute.

Thanks for providing the reproducer. I can confirm that it raises the error and it helped me narrow down why the existing tests didn't catch this. We have a gap where we don't test specifying embed_tokens and lm_head both. To fill the gap, could you please add a unit test? Here is a suggestion:

    def test_trainable_token_indices_targets_head_and_embedding(self):
        # targeting embedding and LM head explicitly, see #2863
        model_id = "hf-internal-testing/tiny-random-OPTForCausalLM"
        with hub_online_once(model_id):
            model = AutoModelForCausalLM.from_pretrained(model_id)
            config = LoraConfig(trainable_token_indices={"lm_head": [0], "embed_tokens": [0]})
            get_peft_model(model, config)  # does not raise

A good spot for this test would be here:

Thanks for providing the right spot for the test. It is added now.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.