In the previous video we learned how to create an automation for Citrix apps.
So far we have covered how to set up reference point using a click event, sending type and
keyboard commands in the remote desktop, and copy and pasting text from and to a Citrix
app.
In this video let's talk about getting data from a Citrix apps in an image format.
We are going to create a workflow that would convert images into text.
Just like the number showing in our sample application right next to the "Total Expense".
This cannot be selected and copied using a cursor.
So what we're going to learn today is scraping data using the OCR technique.
OCR stands for Optical Character Recognition.
What it does is it converts images into text.
In our sample demo app, we will get the Total Expense using an OCR technique.
Let's start by pulling up the recorder.
We will find which image we want to convert into text.
Let's go to Step-by-Step tab and click on Image then select Find.
Just like the click event, we need to highlight a reference point where we want UiPath to
find.
Let's highlight "Total Expense"
Why do we need to do this?
Because we want UiPath to find the specific image region in the screen.
This way, the OCR process will be faster and it make sure that it still work even if the
App window is re-sized or has been moved.
In this case, It will just find the Total Expense image no matter where its location
in the screen.
Next, we will indicate the region relative to the "Total Expense" image.
This is where we highlight the region where the number is placed.
UiPath will only run the OCR in the specified region.
Now that UiPath knows where and what image needs to be scraped, the next activity is
where the actual scraping takes place.
Let's click Text, then choose "Screen Scraping" Let's highlight this number.
This will pull-up the Screen Scraper Wizard.
There are 2 options in scraping the data, Native and OCR.
Native is what UiPath always use when the app is running in the same machine.
Since we use a Citrix app, we need to select OCR.
After the OCR has been performed, it will display the result.
In the right side, we have more options.
The default OCR engine UiPath used is Google's Tesseract.
Microsoft MODI is usually much more reliable and can be installed easily in the Setup Extensions
section of UiPath.
That's it!
Let's press Continue.
Let's go ahead and save the recording.
Let's review the workflow.
First, it will find our reference image, which will be the Total Expense.
The "Translate clipping region" sets the region that needs to be scraped relative to the reference
image.
The next activity is where the OCR process takes place.
The OCR engine is indicated by this activity which can be replaced by Google's OCR engine
if that's what you prefer.
The output text is stored in this variable.
Let's rename it to make it more understandable.
The last activity resets the clipping region.
This way, the recorder will be ready to capture any region on the screen again.
This activity is automatically generated by the Screen Scraping wizard.
Let's add a "Write Line" activity so we can see the result in the Output panel.
Let's put the variable which stores the text output.
Let's run it.
Perfect!
The results showed up in our Output Panel.
Let me add another item here.
This will change the total expense amount.
This is one way for us to test if the OCR is really working.
Let's try it again.
Let's add another expense.
There you go.
So it works!
In the next video we will learn another Citrix specific Technique - Finding the position
of the text using another text as a point of reference.
Thanks for watching!