Workflow Prompting Guide

Choose the Best Block Type

• Most deterministic: Action block
  • single action like click, input text, etc.
• Happy medium: Navigation block
  • single goal, like your goal is to fill out the form
• Most flexible: Navigation V2 block
  • multi-goal, like your goal is to login, then fill the form, etc.

Crafting a Prompt

Anatomy

1. [Required] Main goal
2. Guardrails/details
3. Payload, or information that Skyvern uses in input fields
4. [Required] Completion/Termination criteria

Dos

• Be very clear about your main goal and completion criteria. A good rule of thumb is if someone were to read it with no visual context, would they be able to understand?
  • Use key words like “complete” or “terminate” to let Skyvern know when the task is done or should be ended, respectively
• Start general and get more specific with testing.
  • Making the workflow more general first means there’s a higher chance of it being generalizable across different websites
  • Use parameters in goal fields to add strings of texts that can make the workflow adjustable across different unique sites + their quirks
• Use clear visual indicators and phrases to capture the main goal and completion criteria.
  • For instance, if your goal is to navigate to a certain page, you can say “using the sales dropdown at the top of the page” or “to navigate to invoices, the button has a receipt icon”

Don’ts

• Be unclear. Make sure your goals are clear, that details are rooted in visuals. If you’re confused or a random person would be confused, the LLM will definitely be confused
• Provide too little information. If you’re logging in, you need to provide login credentials, a URL to go to, etc. Think, “what would I need if I were starting this task from scratch?” You can always overprovide information and Skyvern will use what it needs
• Make an action list unless absolutely necessary. If needed as extra details, make sure it’s accompanied by a main goal
• Name the HTML element for items. Some HTML element items are accessible, but not all. Wouldn’t recommend this as a first course of action!

Examples

Example Navigation/Task Block prompt:

Your goal is to fill out the form. Only fill out required fields that you have information for. To add address information, you will need to select the Add button and input the information in a popup modal.

Here is the information you need to get through the form:

{{complete_payload}}

Your goal is complete when you have filled out the form and submitted. You will know your goal is complete when you are viewing a confirmation message that it was submitted.

Example Navigation V2 Block prompt:

Your first goal is to search for an image using Google Lens. To do so, select the Google Lens camera icon. You will know you’re looking at Google Lens when you see the option to drag and drop an image. Then search using the provided image link: {{image_URL}}

Now that you’ve searched google for the image, your goal is to identify the location. If you’re unsure about the location, default to the location that is most represented in the search results.

Now that you have the location, your goal is to go to google maps and extract the address. Provide the address in the following format: {{schema}}

Example Action Block prompt:

Click the continue button