﻿Trigger-action programming, or TAP for short, is an end-user programming paradigm commonly used
in services like If This Then That. Each trigger-action program contains one
or more event-driven rules, in the form “IF some event happens THEN do something.”
For example, “IF Alice falls asleep THEN lock the front door”.
We can also specify conditions to restrict when the rule applies.
TAP might seem simple! However, when there are many rules,
it is hard to figure out how rules depend on each other and what will actually happen.
One complex situation occurs when users try to compare one
trigger-action program to another. For example, when the user edits a
trigger-action program, comparing the old and new versions will help the user decide whether they
have modified the program as intended. Our work introduces new ways to help users
understand differences between two trigger-action programs, especially
when the two programs are highly similar. To demonstrate, let’s compare Programs #1 and #2.
Don’t worry about the details of each. The most obvious difference between these
two programs is that the text of some rules differs.
To display these differences, traditionally we would show a text diff interface,
just like the one on Github. This interface shows the two programs side-by-side.
The red and green colors indicate rules that differ between the two programs.
However, it is difficult to understand the true impact of these differences on
the system using just text diff. In this work, we propose, implement, and
evaluate two new ways of displaying differences between related trigger-action programs.
Back to our example, in some situations, Program #1 results in a different outcome than Program #2.
For instance, maybe in one program the door will be locked in a specific situation,
whereas in the other it will be unlocked. We call these: outcome differences.
We first designed a flowchart interface showing outcome differences.
This situation starts off with Alice asleep during the day while the front door is locked.
Then, someone tries to unlock the door. The flowchart forks to show that the two
programs achieve different outcomes: The top branch shows that with Program #1,
the door unlocks. The bottom branch shows that Program #2 keeps the door locked.
But maybe Alice already knows that she wants the door locked in this situation,
and doesn’t really care what other possible outcomes exist.
Thus, we designed an alternate interface that asks the user which outcomes they want
in specific circumstances. Based on the answers, the interface shows how well each program matches
these outcomes. Here, Alice wants the door to be locked whenever there is a difference.
The interface shows that Program #2 is more suitable than Program #1.
To take a completely different approach, Alice might want to know differences in the general
properties of the programs. For example, our Program #2 has the property that the
door is always locked when Alice is asleep. Program #1 does not have this property.
Our property-diff interface highlights these differences. Here, we see that Program #2
does have the property I mentioned before. In addition to designing interfaces to display
differences in outcomes and properties, we also introduced algorithms for efficiently
identifying these differences. We model TAP programs as transition systems. The state space
of a trigger-action program is relatively small, making it tractable to model all possible states.
Our algorithms convert differences in the edges and nodes of the transition systems
into differences in outcomes or properties. We conducted an online user study to evaluate
these new ways of displaying program differences.
Each participant was randomly assigned to one of the four interfaces we showed before,
or an interface where we only showed the programs themselves.
We asked them to complete 6 tasks comparing related trigger-action programs.
Tasks #1 and #2 required comparison between a few simple programs. The other tasks involved complex
programs, or involved many programs. Most participants completed the simple
tasks correctly regardless of the interface they used.
For most of the complex tasks, however, our novel interfaces that show differences in
outcomes and properties significantly outperformed the traditional interfaces that show differences
in text or the programs themselves. Based on these results, we believe our
interfaces that rethink how we communicate differences between related TAP programs
can help non-technical users better reason about automation systems.