link to course

by Isa Fulford - she also wrote OpenAI cookbook and Andrew Ng

Intro

• Base LLM trained on masive and diverse corpus of training text
• Instruction tuned LLM a base LLM that has further been trained on instructions on how to achieve goals (like web searching). trining technique is reinforcement
• course shows examples on how to be 'clear and specific', and give LLM time to think

Guidelines for prompting

• using pip install openai here. (done that before)

Be 'clear and specific'

• use delimiters to indicate distinct parts of the input vs prompt text (Summarize the text delimited by triple backticks. ``` text yo summarize ```) this also helps to avoid prompt injection, as the input is treated differently (? but an attack can put in ``` into his text and attempt to escape right after that ?)
• prompt should ask for structured output, to make it easier to the output in next stage (like putting output between xml tags <open/> ... <close> or 'provide json output')
  • provide an example for he expected output, otherwise the result may differ from what a next stage would expect ('use the following format')
• prompt should ask to . 'check if conditions are satisfied' or 'check assumptions required'
• few shot prompting: provide examples for successful execution of task (your task is to answer in a consistent style: ...in example: dialog that starts with who is speaking ...)

Give the model time to think by breaking the task up into steps

• example: instead of asking 'check if the students solution is correct' ask for
  • first, work out your own solution to the problem
  • then compare your solution to the students solution end evaluate if the students solution is correct'
• being more specific about the process steps also helps to reduce model hallucinations, like 'find relevant information about the text then answer questions based on the relevant information'

iterative prompt development process / how to improve prompts

on the process of: formulating prompt / trying it ot / assessing result + error analysis

example 0f iterative process: task of summarizing a fact sheet. Make first attempt and then add details:

• first attempt "summarize a fact sheet for retail store...." - resulting review is too long
• adds "use at least 50 words". (tries other variants like "use at most 280 characters" - funny, but the result is of same length :-)
• then doesn't like the content of the summary so adds "the description is intended for furniture retailers, so should be technical in nature and focus on the materials the product is constructed from"
• then adds "at the end of the review add the product id in the technical specification.

Important! once it works, use the prompt on larger set of examples (like 50-100) Find average / worst case of performance, etc.

Summarizing text (frequent applications of LLM's)

pretty similar to the previous section.

• Also says: prompt should state the purpose of the review, as clarification (example: 'to give feedback to the XXX department' )
• or tell it to 'extract information relevant to the XXX department' instead of 'summarizing'. Gives a more concise result.

Inferring

Tasks: sentiment analysis, info extractions, etc. Sentiment analysis is one of the strengths of LLMs (as it is much easier to use/requires much less work, compared to ML without LLM !) Andrew NG is going through some example prompts in this area.

• first prompt f"What is the sentiment of the following product review, which is delimited with triple backticks? ```{review}```"
• instead ask for a single word answer: f"""What is the sentiment of the following product review, which is delimited with triple backticks? Give an answer in a single word, "positive" or "negative" ```{review}```""" - this makes it answer to process the the result!
• ... "format your answer as a list of lower case words, separated by comma" - this makes it easier to understand how the customer perceives the product
• ... "is the review expressing anger?" - valuable question for customer support!
• use prompt to extract info from review """ ... Identify the following items from the review text: - item purchased by review - company that made the item. Format your review as a JSON with "Item" and "Brand" keys ... """
• says you can combine different tasks of sentiment analysis & info extraction in a single prompt.

Summarization:

• """determine the five topics that are being discussed in the following text""""
• """Determine whether each item in the following list is a topics is a topic in the text below. Give your answer as a list with 0 or one for each topic.""" this can be used to build a news alert.
• better to ask for json output and specify the keys...

Transforming text

translating/proofreading/conversion between formats.

• f"""Translate the following text to Spanish. Messages included in backticks '''{msg}'''"""
• or identify which language, or to translate into several languages at once.
• Tone transformation: """translate the following from slang into business letter..."
• or translate between json to html
• spellcheck or grammar checking """Proofread and correct the following text. If you find no errors just say 'No error' '''{msg}'''""" (still: sometimes puts quotes around answer, sometimes doesn't - as output format is not specified in the prompt)
  • tip: there is the redline package to show differences between two texts. (redlines.Redlines(in_msg))
• also can be very specific about output: """Proofread and correct this review, Make it more compelling. Ensure it follows APA style guide and targets an advanced reader. Output in markdown format '''{msg}'''""" (I didn't even know that 'APA style guide is developed by the American Psychological Association for clear, precise, and inclusive scholarly communication, commonly used in the social sciences')

Expanding

Take short text and expand on it - write an essay, write reply to an inquiry, etc. (says: don't use it to create spam)

• f""""You are a customer service AI assistant. Your task is to send an email reply to a valued customer. If the sentiment is positive of neutral, thank them for their review. If the sentiment is negative, apologize and suggest that they can reach out to customer service. Make sure to use specific details from the review. Write in a concise and professional tone. Sign the email as `AI customer agent`. Customer review: '''{msg}''' Customer sentiment: '''{sentiment}'''""""""
• use temperature (0 - only use most likely next token prediction, 30 - can use a less likely prediction too)

says use zero for task that require certain outcome (like json creation) and a bit higher temperature for chat agents, for example.

Chat bots

You call a chat bot with the conversation history in json format (the context, in AI speak)

[ { "role": "<conv-role>", "content": "<this turn of the converesation>" } ]

conv-role = assistant|user|system

system - the prompt for chatgtp, that governs the conversation.
assistant - chatgtp said something
user - the user said something

The system prompt for a bot that receives pizza orders:

{ "system": "You are OderBot, an automated service to collect orders for a pizza restaurant. You first greet the customer, then collect the order, then summarize it and check for a final time if the customer wants to add anything else. If it's a delivery, you ask for an address. Finally you collect the payment. Make sure to clarify all options, extras and sizes to uniquely identify the items in the menu. You respond in a short, friendly and conversational style. The menu includes \`\`\`{menu items\`\`\`}"}

• A second prompt for taking this conversation history and to create a json formatted order (Agentic processing!)

---