Our work is on Wikum+, a System for Interleaving Discussion
and Summarization in Online Collaboration.
When collaborating, groups typically
discuss in a tool like Slack and then move
to a separate document-writing software like Google Docs.
But this means that the document loses its link
from the discussion that generated it and can result
in scattered and lost ideas.
The upfront work for the initial writer can be significant,
and the writer themselves may be biased, reducing inclusiveness.
Instead, Wikum+ Plus interleads the discussion in synthesis
phases to allow for continuous discussion and feedback loops,
creating living summaries, which grow with new discussion
and shrink after deliberation comes to a consensus and is
merged.
This process allows for users to chase how the document evolved.
A user starts off on an empty Wikum+ page and can begin
by creating their first comment.
Other users can then join in on the discussion page
and add their own thoughts.
Notice that the comments shown here
are in blue, representing comments
in an unsummarized state.
Once users are done discussing an idea,
they can summarize that thread and cite anything
that they are summarizing.
This creates a summary, shown in orange.
Now, the originally blue unsummarized comments
became collapsed underneath and turned yellow,
representing summarized comments.
Users can then recursively summarize
the entire conversation.
This pulls in subsummaries and unsummarized content
into a single orange summary note.
This top-level summary can be seen
as a collaborative document built up from the discussion.
Now, when a new reader comes in, they see the summary first.
But it can be expanded to see the original comments
underneath.
If discussion gives them a new thought,
they can leave a new comment under the summary.
The Summary icon updates into a partial summary state
to show that the summary now needs
to be updated to incorporate the new comments.
Another user might deem the summary incomplete,
biased, or poorly written and can flag the summary as so.
Finally, someone can update the summary
to include that new thought.
Then, if they believe that the summary reflects
the new comment, they can evaluate the summary
and mark that the comment was included.
Since all of the comments are now included,
the summary icon returns back to orange,
representing a fully summarized state.
We first conducted a Wikum subject study
consisting of smaller groups.
Each group worked closely to create two proposals,
one in Wikum+, and one using both Slack and Google Docs.
This graph shows the default number
of words shown on each interface over time.
In Wikum+, this value includes the words and summaries
and unsummarized comments because summarized comments are
collapsed.
In the control group, this is just the total word count.
We can see that after users ideated for a while in Wikum+
Plus, they interleave discussion and summarization,
causing the conversation to grow and shrink.
On the other hand, in the control,
users worked mostly in two stages,
one involving discussion, followed by edits in the doc.
The dark blue here represents post-summary comments in Wikum+
that reopened existing summaries for more discussion and allowed
for iterative improvements, the phenomenon that was not seen
in the control.
We also found that users bring new strategies from Wikum+ when
working in Google Docs afterwards.
However, unlike in Wikum+, the control proposals lacked direct
references back to the discussion.
Users reported that it was easier to locate both new ideas
and topics that still needed work in Wikum+ allowing
for more summary improvements.
Users also found that the writing process in Wikum+ was
significantly more organized than in the control.
When evaluating the final proposal,
we found that Wikum+ users wrote more concisely,
while users introduced greater narration in the control,
resulting in what users deemed a higher quality of writing.
This was likely a result of the bullet-point structure from
the subsummaries created in Wikum+.
Overall, users found Wikum+ to be a more cooperative
and inclusive experience than in the control,
with the majority of users preferring Wikum+ over
the control as a system.
We then conducted a between subject study
with two larger groups over a longer period of time,
with staggered entry times for participants.
While both conditions had comments disagreeing with
summary content, these post-summary comments in Wikum+
resulted in updated summaries to include more possible solutions
and opinions, while in the control,
these comments were left unresolved since the control
users went with a majority vote as the deciding factor
for which opinions to include.
As a result, Wikum+ users expressed significantly greater
feeling of inclusion in the post-study survey.
Since more opinions were included in Wikum+,
users felt that the proposal was also more comprehensive.
Independent raters scored the Wikum+ final proposal higher
in quality, comprehensiveness and success.
Overall, Wikum+ allowed for greater organization
and lightweight coordination in smaller groups and more
inclusion and comprehensiveness in the resulting document
in larger groups.
You can check out Wikum at wikum.org.
Thanks for listening.