Generated Robustness Evaluation Sample Scripts #117
Conversation
|
|
|
Thanks for raising this @RigvedRocks! I'll review it now. When you get a chance, could you fill out the CLA? |
|
I'm not very familiar with promptbench but it looks like you want to run attacks as experiments and use your eval function to evaluate the experiment outputs. You can check out experiment examples to see how we feed prompts into LLMs for testing, and how we pass in eval functions too. Let me know if you have any specific questions about it. Thanks again for looking into this issue! |
|
Sure Mr. Krawcyzk. I will have a look at the implementation of the library and will make changes according to that. When time permits, I will fill out the CLA. |
|
Hey, I have generated a sample script to evaluate the robustness of the models under the utils folder. Please have a look at it and suggest amends if any. |
Hey, I am working on issue #69 and I have generated 2 sample scripts - one involving prompt substitution and the other for auto evaluation. I am using Promptbench in both scripts. Can you please guide me as to how to integrate the scripts into your experiments?