hello and welcome to get hooked on web hooks with fme server automation tools for your enterprise
my name is carl von stetten i'm a gis analyst with central san and i've been with central san since
about 2000 and have been using fme since about 2002. central sand is a wastewater collection and
treatment agency serving nearly 500 000 people in central contra costa county which is part of the
san francisco bay area in northern california our headquarters are located in martinez
in our complex world there is no single software application or enterprise system that can handle
every business requirement multiple enterprise applications are the norm
these applications and systems frequently need seamless real-time communication and
integration to maximize efficiency and fme server automations can make that easier
so i'm hoping that by the end of this presentation i'll have answered these
two questions for everyone the first is what are web hooks and how do they work
and the second is how can web hooks be used in fme server automations
now this all came about for us at central san because we had a couple of
real world problems to solve with our own enterprise systems we use cityworks asset
management system which is a gis centric asset management and work management system
and one situation we encountered was that we have cctv inspection work orders that are created to
inspect our pipes and they're often assigned to one crew but later reassigned to another crew
and we needed a way to let it pipes another system know that the inspector had changed
a second situation that came up was that for certain work order types we needed to inject
a custom comment into the work order and that comment needed to contain a hyperlink
to yet another application that allowed our field workers to access maintenance history related
documents and manuals and spare parts availability or the asset that they were required to do work on
now this is not something that cityworks can do out of the box without custom programming
so our solution to both of those problems was to use fme server automation specifically web hook
triggers that would respond to city works events so let's get into answering those questions that
i brought up earlier first of all what is a web hook well here's wikipedia's definition
i'll give you a moment to read it but it's probably going to be a bit hard to comprehend
if you're not a programmer or web developer so let's see if we can do that a little bit better
so in this context we're talking about a web hook connecting two business systems so we
need business system a and b to be integrated and so whenever an event on business system a happens
say creating or updating a work order or maybe it's we're working with arcgis
online and we're adding new features to a feature layer or creating new content
we want business system b to be notified so that it can do something uh some take some appropriate
action maybe creating or updating corresponding data so we need to send a message right that's
that's what the web hook does but how does it make the notification it uses an http request
well what's an http request well that's sort of the backbone of how the internet works how
browsers and servers talk to each other when you open your favorite browser and you type google.com
in the url bar and hit enter your browser makes an http request to google server and says i want your
home page give it to me and the server then sends that home page content back to your browser where
you then get to see it now maybe you're on another website and you're filling out a contact form
fill in your name and your email address maybe your telephone number and you hit submit when you
hit that submit button your browser then is making another http request to a server somewhere and not
only is it saying i want this page back to come back from the server but it's sending all that
mess all that data that you entered into the form your name your email address your phone number
so in the cases of business integrations
server a is sending an http request without a browser being involved it
just does it just sends the message using internet protocols to business system b
but where is the request going
well business system b must have an application listening for messages on a web on a standard
web accessible port an accessible url what if it doesn't well we'll get into that in a moment
here's some examples of systems that offer web hooks
you'll notice cityworks is on there and that's what i'll be talking about primarily
but most cloud application systems most large enterprise systems offer web hooks in some form
now web hooks aren't the end all and be all they are they do have issues that have to be dealt with
so let's talk about a few of those um first of all can the two business systems actually talk to each
other directly are there firewalls in the way are they is one in the cloud and one on premise
because if you can't communicate directly web hooks aren't really going to help you
secondly does business system be even listen for http requests
if it doesn't then again web hooks may not help you here
does business system b need to do maybe multiple things
when it gets a request does it need to fire off two or three or four actions on its end
to correspond to the message coming from business system a
or maybe does business system b need more data
from yet another system to combine with the web hook message before it can do something with it
and there are other other issues as well in the real world these problems are often overcome
by writing custom programs which has short and long term cost and schedule implications
so how can fme server make this easier well first of all we start out by not
sending the webhook message to business system b directly
instead we send it to an fme server automation and in that automation we use a webhook trigger
i'll explain that in a minute to listen for those messages now we have a guaranteed message listener
that automation can run an fme workspace or two or three or even more
and those workspaces can do a number of different things they could call apis on business system b
as needed using http caller transformers
i like this or maybe business system b doesn't have any apis so maybe what fme will do is
actually update or insert data directly into business system b's database tables directly
or maybe they pull data in from even additional business systems combine it
all together with the web hook message and then interact with businesses system b regardless
the end result is a no code let me emphasize that no code integration
that you can maintain long term using tools you are already familiar with
so i'm going to do a quick demo and in the interest of time i've already
pre-configured everything but it only took me a few more minutes than explaining it will take
to actually make it happen so let me let me get in here so here we have our fme server
and i'm going to go into the automations section of the server and manage automations
and i've got one already built here so here's our automation we start out with what's called a
a trigger this is a web hook trigger so all i did was
go in here and set the trigger to be web hook trigger and as soon as i save the workspace
the first time it created this url and that url is what we're going to send messages to
next i inserted a log action so that we can log the messages coming in and for testing
and for debugging and then i pass the message on directly to a workspace and that workspace
has some published parameters the last one of which is called has a the label json data
this is where the message is actually going to be sent so i just went in here and pointed it to use
the um the message content coming from the web hook uh trigger so that's that there
all right so i'm gonna go ahead and start this automation so it's actually now running and
waiting and listening for messages all right so now we're over here in cityworks and i'm in the
designer and if i go into action manager which is one of the features of cityworks
i'm going to start by with a template so i want a webhook template action and i've got one down
here i created for this for this demonstration and this url is i copied and pasted directly
from that automation in fme server so that's that's very easy to do um and then i filled in
a header i'm telling it what the content type of the message is this is kind of important
when you're sending data over the internet um i'm telling it that it's application json uh one of
the other options is xml but web hooks typically use json over xml that's why i'm using json here
and the template is just a json object and i can't time doesn't permit me to get into a lot of detail
of what that is but in the resources at the end of the presentation is a link to some documentation
on what json is and how it's structured and basically this is just going to send
what are called key value pairs in the message so this is the key and that will become an fme
attribute when the workspace unpacks the message and these are called tokens in city works and what
happens is when this message is actually created when a work order when an event causes this
template to be fired and the message to be generated it will fill in the actual value
for this token so in this case it'll fill in the work order sid value um at runtime
okay so that's the template the other piece that we have to configure is the event
so we're going to go back again webhook and i've got an fme webhook demo event created
and so this is this is basically configuring when that action template is going to be fired so first
of all um i've linked this to that template i created so this is what it's going to fire
and i've linked it to just a particular type of work orders this one template work order template
and down under conditions i've really just set this to always fire
you could have complex conditions built down here if you only want
certain types of work orders or a certain circumstances if you want this to fire but
this is going to fire for every single new work order of this type okay i'm going to hit the test
button here and what that's going to do is just send a blank or a test message to the web hook
trigger and we'll see that in just a second so i'm gonna hit test
test okay so if i go back to fme server and i look at the log file you can see here's the message
that it just received that's the test message it has a zero for the work order s id and everything
else is blank because it's just a test message but confirming that it did in fact send a message
okay let's do something a little more fun let's actually create a work order so i'm in the create
new work order page in city works and i'm going to send this to a specific work order i've created
that was that gis web hook demonstration that i configured in the web hook
and that's really all i need to do i don't need to add any more information but really quickly just
notice the comments is blank right so there's no comments being created by me if i hit create
city works is doing its thing behind the scenes and it's created a work order
and look at that there's my comment coming from my fme automation i didn't create that myself
the automation did it and just to confirm go back and refresh the log
here's the request from that actual work order and it running the workspace with that message to
to actually create this comment for us
all right so let's finish up the presentation real quickly here
a couple of tips and tricks just keep in mind that web hooks are very versatile
the example i just demonstrated isn't actually connecting to business systems
it's using a web hook to trigger fme to do something that cityworks couldn't do on its own
so this is really cityworks to fme back to cityworks but you can also be using web hooks
that go city works to fme to it pipes or to arcgis or another system that you need to interact with
another tip is when you're starting out you're creating your automation
as i did here just add a logger a logger action and then that way you can test that
it's actually receiving messages that helps you to make sure that your source
application in this case cityworks is actually sending the messages when you think it is
All right to wrap up we've seen that web that fme server automation web hook triggers can
facilitate powerful no code integration solutions using the familiar fme workspace tools publishing
those to fme server and building them into an automation you can accomplish almost anything
as promised there are some resources that will help you to get started with webhooks in cityworks
as well as how the automation triggers all of the triggers including webhooks work in fme server
and that handy json reference if you want to see how json messages are structured
and the next steps are really on on your on to you now so go out there and start
building your own system integrations with fme server automations and make magic happen
thank you for your time if you'd like to reach out to me if you have questions feel free to
send me an email at that address i hope you enjoy the rest of the fme world tour thank you