7th FrameworkEFMHuman Computer InteractionProjectsSOCIETIESUser Trials

SOCIETIES Workshop and User Evaluation @ Intel

By 24th April 2013 No Comments

 by Edel Jennings and Mark Roddy (EFM/TSSG)


Last week the SOCIETIES consortium held a Workshop at the Intel Innovation Open Lab in Leixlip, Ireland.

A significant number of Intel employees were invited to attend and were presented with some of the value propositions from the project’s Innovation Tree.


David McKitterick from Intel presents the SOCIETIES Innovation Tree


The Workshop started with an architectural overview of the SOCIETIES platform from the project’s Technical Co-ordinator, David McKitterick from Intel, and was followed by Dr. Ioanna Roussaki from the National Technical University of Athens who provided attendees with an insight into the project’s Context Management & Inference engine.

Then Prof. Nick Taylor from Heriot-Watt University in Edinburgh explained how the platform uses the Personalisation & Learning enablers to tailor the SOCIETIES experience for the project’s targeted user communities.

Bruno Jean-Bart & Olivier Maridat from TRIALOG presenting the Privacy by Design techniques

Tailoring this user pervasive experience relies heavily on the platform’s ability to have a fine-grained understanding of the user’s context, which could be regarded suspiciously by our end users. These suspicions were taken on board in our final presentation by Bruno Jean-Bart & Olivier Maridat from TRIALOG based in Paris, who jointly presented the project’s ‘Privacy by Design’ techniques. If the Q&A session at the end of the presentations was anything to go by then the workshop was judged to be a resounding success.

The Workshop was followed by an evaluation of the first SOCIETIES prototype, and six people from Intel elected to participate in the user evaluation exercise as part of the SOCIETIES Enterprise Community, which was based around a ‘Networking at a Conference’ scenario. The user evaluation featured two of the project’s current third party services: the Networking Zones application, and the Context Aware Wall application; along with the main SOCIETIES platform application. The technical set-up, support and running of the trial system was successfully staged by Intel and IBM researchers, who provided essential expertise and support through the setup, including the management of WIFI sniffers, which successfully demonstrated indoor location discovery.

Since the prototype system software is still very much in development, and there are many known usability and integration issues; it was decided to begin with a participative walkthrough demo, to introduce participants to the software. Each of the six Enterprise Group participants was provided with a smart phone device, on which the required system software had been installed. They were asked to follow along a set of predefined tasks, as they were simultaneously being demonstrated and explained on a large monitor by the SOCIETIES presenter.  A task sheet with a checklist of the basic tasks and steps involved was provided, which also included a section for comments, to invite feedback from each individual participant as they were guided through the platform and Context-Aware Wall applications. Seven SOCIETIES researchers were also present to support, observe and record the exercise. Post-its and pens were provided and a large whiteboard in the room had two sections organised to capture comments and questions, for each of the applications.

A ‘chase the clues’ treasure hunt game, designed to illustrate the potential of location and context aware communication within different communities, was played by all of the participants. This brought the first part of the trial to a pause when all game participants discovered the final destination specific message and shared the final prize – lunch! All six participants completed the System Usability Scale (SUS) Survey at this point.

‘Guardian Angels’ Alec and Guy helping some of the DragonSoc contenders

Lunch was followed by an introduction to SOCIETIES’ Networking Zones application. Then, four participants were engaged in a scenario role play exercise, called DragonSoc.   Scripts were provided for four characters, or personas, to illustrate a ‘dragon’s den’/’secret millionaire’ game scenario, designed to play out how users selective sharing impacts on relevance, and implicit/explicit learning. Participants were required to use and switch between all three of the applications, and make decisions regarding how much information to share to follow the scripts. Societies researchers provided ‘guardian angel’ and ‘tech support’ services to assist users as required, and a conceptual ‘large prize’ was provided to the eventual winner of the game.


Fabio the Investor presents a delighted Kieran with his €1,000,000 post-it prize


Feedback flowed freely between participants and researchers during this exercise, as questions, comments, discussions and suggestions were captured in post-its and recorded observations. A closing participative discussion engaged both the enterprise trial participants and some SOCIETIES researchers on the projects’ value proposition and key innovations. This brought the trial to an end.

The Intel participants were actively and critically engaged throughout the trial. They provided intelligent insight from the user perspective to the proposed scenarios. The data from the SOCIETIES’ Enterprise user trials will now be analysed and reported on in detail in a forthcoming project deliverable, and the results will be harnessed to assist the development team in focusing their future efforts.

SOCIETIES User Trials provide an opportunity to see how the work done by all the SOCIETIES partners, can be integrated and utilised to create a rich social and pervasive experience, which can be appreciated by real people. When real users begin to glimpse the potential of the project’s proposed innovations, despite the limitations and constraints of early prototypes, it does indicate and strengthen the project’s underlying achievements.