Last active
December 5, 2024 19:59
-
-
Save onEnterFrame/3db17ec100088bc5b9ccf76a55b9242f to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| let instructions = ` | |
| ### **AI Agent Instructions** | |
| 1. **Understand the Content:** | |
| Use the provided content as the baseline for evaluating the learner's response. Ensure that the response aligns with the principles and safety guidelines in the content. | |
| **Content:** | |
| *"When working in a warehouse environment, always ensure that forklifts and pedestrians are separated by marked walkways. Forklift operators must maintain clear visibility and use warning signals when turning or reversing. Pedestrians should never assume they are seen and should make eye contact with the operator before crossing paths. A common cause of accidents is distraction, such as using mobile phones in high-traffic areas. Proper use of personal protective equipment (PPE), including high-visibility vests and steel-toe boots, is also crucial for minimizing injuries."* | |
| 2. **Understand the Question:** | |
| Evaluate the response in the context of the specific question provided. | |
| **Question:** | |
| *"What should a pedestrian do before crossing paths with a forklift, and why is it important?"* | |
| 3. **Evaluate the Learner’s Response:** | |
| Compare the response to the content and question using the following criteria: | |
| - **Action Taken:** Does the response mention specific actions pedestrians should take (e.g., “make eye contact with the operator”)? | |
| - **Reasoning Provided:** Does the response explain why the action is important (e.g., “to ensure visibility and prevent accidents”)? | |
| - **Alignment with Content:** Does the response align with the content and avoid unsafe or contradictory suggestions? | |
| - **Use of Keywords:** Does the response use appropriate terminology (e.g., "visibility," "communication") and avoid unsafe terms (e.g., "assume the operator sees you")? | |
| 4. **Respond in JSON Format:** | |
| Output the evaluation in JSON format, including: | |
| - 'evaluation': A brief evaluation of the response's accuracy and relevance. | |
| - 'reasoning': A detailed explanation of why the response is correct, partially correct, or incorrect. | |
| - 'score': A numeric score between 0 and 100, based on the accuracy and relevance of the response. | |
| --- | |
| ### **Scoring Criteria** | |
| 1. **Full Credit (100%):** | |
| The response matches the content perfectly, includes required actions, and explains their importance. | |
| 2. **Partial Credit (50–75%):** | |
| The response demonstrates understanding of some key concepts but omits crucial details or lacks full clarity. | |
| 3. **Low Credit (25–50%):** | |
| The response is vague or includes partial inaccuracies but does not contradict the content entirely. | |
| 4. **No Credit (0–25%):** | |
| The response suggests unsafe actions, contradicts the content, or is entirely irrelevant. | |
| --- | |
| ### Example JSON Response Format: | |
| { | |
| "evaluation": "The response is fully correct. It includes the specific action of making eye contact and explains its importance in preventing accidents.", | |
| "reasoning": "The content explicitly states that pedestrians should make eye contact to ensure visibility and avoid assumptions about being seen by the forklift operator.", | |
| "score": 100 | |
| } | |
| `; | |
| //get the player | |
| let player = GetPlayer(); | |
| let learnerResponse = player.GetVar("learnerInput"); | |
| ai.languageModel | |
| .create({ | |
| systemPrompt: instructions, | |
| }) | |
| .then((session) => { | |
| console.log("Session created:", session); | |
| if (!session || !session.prompt) { | |
| throw new Error("Session creation failed or prompt method is missing"); | |
| } | |
| return session.prompt("The learner's response is: "+learnerResponse); // Prompt with the learner's response | |
| }) | |
| .then((result) => { | |
| console.log("Raw result from AI:", result); | |
| // Ensure result is in the expected format | |
| if (result.includes("```")) { | |
| result = result.split("```")[1]; | |
| } | |
| if (result.includes("json")) { | |
| result = result.split("json")[1]; | |
| } | |
| try { | |
| let resultJSON = JSON.parse(result.trim()); | |
| console.log("Parsed JSON result:", resultJSON); | |
| // Set Storyline variables | |
| player.SetVar("ai_score", resultJSON.score || 0); | |
| player.SetVar("ai_evaluation", resultJSON.evaluation || "No evaluation"); | |
| player.SetVar("ai_reasoning", resultJSON.reasoning || "No reasoning"); | |
| } catch (parseError) { | |
| console.error("Error parsing result JSON:", parseError, result); | |
| } | |
| }) | |
| .catch((error) => { | |
| console.error("Error during AI processing:", error); | |
| }); | |
| // License | |
| // This project is licensed under the [MIT License](LICENSE). | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment