MiniRobotLanguage (MRL)
AIC.Ask Vision URL
Send a Picture via URL to the Open AI "Vision A.I."-Endpoint and receive a description of the picture
Intention
Using the AIC.Ask Vision URL command you can send a picture to the OpenAI Vision endpoint and receive a text-description about this picture.
You can also send a prompt and tell the Vision AI what exactly you want to know about these picture and this way you can influence the result that you get.
The AIC.Ask Vision URL command is developed for complex image processing tasks involving multiple images.
It is particularly useful in scenarios that require batch analysis of images under the guidance of a single prompt.
The current model is best at answering general questions about what is present in the images. While it does understand the relationship between objects in images, it is not yet optimized to answer detailed questions about the location of certain objects in an image. For example, you can ask it what color a car is or what some ideas for dinner might be based on what is in you fridge, but if you show it an image of a room and ask it where the chair is, it may not answer the question correctly.
The latency of the model can also be improved by downsizing your images ahead of time to be less than the maximum size they are expected them to be. For low res mode, we expect a 512px x 512px image. For high rest mode, the short side of the image should be less than 768px and the long side should be less than 2,000px. You can use the AIC.EnsureFormatResize Command to automatically generate resized versions of Pictures.
Actually the OpenAI Vision API will only accept ".jpg"-Files. Therefore you need to provide ".jpg"-Files to use this command.
Technically if the image is not a "jpg" the Smart Package Robot will anyway download it and then convert it to "jpg" before sending it to the Open AI Vision Endpoint.
AIC.Set Key|File
' URL's from "Twitter" (X)
$$URL=https://pbs.twimg.com/media/F-1T8dGXIAASffV?format=png&name=360x360
$$PRO=Explain what you see.
AIC.Ask Vision URL|$$URL|$$PRO|$$RET
'AIC.Show Error
MBX.$$RET
ENR.
Syntax
AIC.Ask Vision URL|P1|P2[|P3]
Parameter Explanation
P1 - URL of the Image, this is the URL of the image to send. Technically if the image is not a "jpg" the Smart Package Robot will anyway download it and then convert it to "jpg" before sending it to the Open AI Vision Endpoint.
P2 - Prompt - Prompt to tell the AI what you want to know from the picture.
P3 -(optional) Variable that will receive the result. If omitted the result is placed on TOS.
Example
'***********************************
' AIC.Ask Multi Vision
'***********************************
AIC.Set Key|File
$$PAT=F:\00_MR\MR_Komponents\U1-DLL
ARR.Set Array|0|0|$$PAT\Testpic_XL.jpg
ARR.Set Array|0|1|$$PAT\X513.jpg
ARR.Set Array|0|2|$$PAT\C.jpg
$$PRO=Tell me the difference between the 3 pictures
AIC.Ask Multi Vision|0|$$PRO|$$RET
AIC.Show Error
MBX.$$RET
ENR.
Remarks
It's crucial to ensure the array number (P1) is within the valid range (0-32). Providing an out-of-range number will result in an error.
Limitations:
Is there a maximum Number of images you can send?
From the SPR side there is no other limitation then Memory limits and bandwidth limits.
From the Server side, pictures should be best not too small (<512 Pixels) and not larger then 2048x1792)
See also:
•
•