enable xpu device for causal_language_modeling example #2680
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
@BenjaminBossan pls help review, thx |
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| "\n", | ||
| "# Hyper-parameters\n", | ||
| "device = \"cuda\"\n", | ||
| "device = \"xpu\" if torch.xpu.is_available() else \"cuda\"\n", |
There was a problem hiding this comment.
how about we use this? device = torch.accelerator.current_accelerator().type if hasattr(torch, "accelerator") else "cuda"
There was a problem hiding this comment.
Oh, I assumed we should install accelerate to use torch.accelerator.current_accelerator() API, it turns out I am wrong...
| "\n", | ||
| "batch = tokenizer(context, return_tensors=\"pt\")\n", | ||
| "batch = {k: v.to(\"cuda\") for k, v in batch.items()}\n", | ||
| "device = \"xpu\" if torch.xpu.is_available() else \"cuda\"\n", |
| "\n", | ||
| "inference_model = PeftModel.from_pretrained(inference_model, \"smangrul/mistral_lora_clm_with_added_tokens\")\n", | ||
| "inference_model.to(\"cuda\")\n", | ||
| "device = \"xpu\" if torch.xpu.is_available() else \"cuda\"\n", |
| "from datasets import load_dataset\n", | ||
| "\n", | ||
| "device = \"cuda\"\n", | ||
| "device = \"xpu\" if torch.xpu.is_available() else \"cuda\"\n", |
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
|
@kaixuanliu Could you please run |
|
@BenjaminBossan Done |
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
|
Thanks, but there still seem to be issues with formatting. Maybe let's check that you have matching ruff version (0.9.10) and that the settings from the |
|
@BenjaminBossan Well my ruff version is old. Have updated to 0.9.10. Can you help trigger the CI check again? |
|
The linter is still complaining. This is the diff I get when running it locally: @@ -247,9 +247,9 @@ def main():
correct = 0
total = 0
- assert len(eval_preds) == len(
- dataset["train"][label_column]
- ), f"{len(eval_preds)} != {len(dataset['train'][label_column])}"
+ assert len(eval_preds) == len(dataset["train"][label_column]), (
+ f"{len(eval_preds)} != {len(dataset['train'][label_column])}"
+ )
for pred, true in zip(eval_preds, dataset["train"][label_column]):
if pred.strip() == true.strip():
correct += 1Update: I see, already fixed. |
|
@kaixuanliu For some reason, GH does not update on your latest commit, could you try pushing an empty commit, hopefully that fixes it. |
|
@BenjaminBossan pls try again. |
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
examples/conditional_generation/peft_lora_seq2seq_accelerate_ds_zero3_offload.py
Outdated
Show resolved
Hide resolved
…s_zero3_offload.py Co-authored-by: Benjamin Bossan <BenjaminBossan@users.noreply.github.com>
BenjaminBossan
left a comment
There was a problem hiding this comment.
Thanks for updating the causal LM examples and for making them work with XPU.
No description provided.