Area,Question,Client Feedback
Research Contribution Expectations,"What level of research contribution is expected (exploratory implementation, strong evaluation of existing methods, or novel potentially publishable work)?",
Research Contribution Expectations,"Do you expect innovation, or is applying existing methods well enough?",
Project Scope Confirmation,"Can we confirm that the project focuses on building a framework to evaluate prompt injection vulnerabilities in LLM chatbots and test mitigation strategies rather than developing or training new models?",
Project Scope Confirmation,"Please confirm our understanding: This is a security evaluation pipeline for LLM chatbots. It is NOT training LLMs, building ChatGPT, building a new AI model, or doing heavy ML training. It IS designing attacks, measuring vulnerability, adding protection, and measuring improvement.",
Model Prioritization,"Which LLMs should we prioritize testing: API models, open-source models, or both?",
System Complexity in Scope,"Should the project evaluate simple chatbot systems only or more complex architectures such as RAG or enterprise deployments?",
Attacker Capabilities and Scenarios,"Which attacker capabilities and scenarios should be included in scope: user-level interaction, prompt/system instruction extraction, indirect injection via retrieved documents (RAG), adaptive iterative attackers?",
Evaluation Methodology,"What evaluation methodology should we use: ASR only, or also severity metrics, statistical analysis, and defined attack scenarios?",
Evaluation Methodology,"Are we expected to implement one mitigation, or compare multiple?",
Evaluation Methodology,"Is there an expected minimum improvement after mitigation?",
Deliverables,"What format should final results be delivered in: report, GitHub repo, live demo, dashboard, or prototype?",