SPEAKER 1: Hello and welcome to
this demo video on how to integrate
prompt flow with LLMOps in Azure Machine Learning.
Today we'll be exploring the features of
the flow files and its folder directory,
prompt flow, SDK and CLI,
and how to integrate the flow into a code repository.
First, let's talk about the flow folder.
Each prompt flow is associated with
a flow folder structure that
contains essential files for defining the flow.
You can access the flow folder to
get all files directly in the workspace.
Upon creating a flow,
a corresponding flow folder is automatically
generated and stored in the workspace shared storage.
This folder includes the primary flow definition file,
flow.dag.yaml source code files,
and other included utility files.
The flow.dag.yaml is
the primary flow definition file in yaml format,
includes definition of flow inputs,
outputs, nodes, tools and variants used in the flow.
The flow folder also includes user managed
source code files which are referred
to by the tool/nodes in the flow.
For example, the Python file can be referenced by
the Python tool for defining custom Python logic.
Jinja file can be referenced by the prompt tool
or LLM tool for defining prompt context.
Now let's see how to version
your flow into a code repository.
There is a download button in UI.
You can easily export the flow folder
from the flow UI to your local system.
Once downloaded, you can open in versus code,
then check in this into your code repository.
If you want to test your flow in local,
you can install the prompt flow SDK.
Switch to the prompt flow conda environment,
specify the flow folder path,
data path, then run the flow test function.
Then you can get the single run output result.
Then let's move on to submitting
batch runs to the Cloud from local,
you can use the prompt flow Azure package
to submit a run to Cloud.
You need to connect to the Azure machine
learning work space on the Cloud first,
and use the prompt flow SDK to specify
the run time and connection
you have created in the Cloud.
Then specify the local flow folder path
and the data path for this run.
After successfully submitting to the Cloud,
you will get a link to the run
detail page in your workspace.
Then go to the portal to check the batch run result.
Or you can stay in local run
prompt flow visualize function to view the output.
You will get a local host web page
to visualize the run output.
Similarly, you can submit
another run based on the output of
the previous run for
evaluation and calculation of metrics.
Once the run completed,
you can visualize them locally
as well from the local host web page.
Finally, let's talk about local iterative development.
Beside of the prompt flow SDK,
you can install prompt flow extension in versus code,
which provide a fast and efficient way
to test and refine your flow.
With similar UI experience as workspace portal,
you can easily develop your flow in VS code locally.
You can create the variance of your prompt.
Tune your prompt in different versions.
You can run your flow in
debug mode for better debug ability.
You can run bulk test based on
large dataset or existing runs output.
Then you can see the log of each variant run.
Check the output of the JSON L file.
Click the prompt flow extension in the left navigator,
you quick access your flow in local,
view bulk test history,
manage your connection in local,
view recent edit flows.
Now let's shift our attention to
the Azure machine learning interface.
If you've made changes locally and
want to continue working in the Cloud,
you have the option to import
your local flow directly into the UI.
To do this, simply create
a new flow and select the import from local option.
Navigate to your local flow folder and select it,
and there you have it.
Your local changes are now
reflected in the Cloud and you can
continue refining your flow using
the rich set of tools available here.
That concludes our demo on how to
integrate prompt flow with LLMOps.
This is a perfect example of
the seamless transition between
local and Cloud development that Azure
machine learning prompt flow offers.