Uploading Media from a Device

The transfer of personal content from devices to the cloud is an important requirement of our system. Utilizing the Microsoft Surface as a central hub for personal content, a user is able to upload personal videos to the tabletop. A user places the iPhone or iPod onto the Microsoft Surface which triggers a display of thumbnails to appear, each corresponding to a video on the device. A user can then select and drag a thumbnail from the custom display (located on the tabletop) onto an open area of the table, which causes the video content to be transferred to the tabletop for viewing, organization and tagging. A user is also able to send this newly organized and tagged content to the cloud, allowing them to access it from other devices and locations.

Displaying Media on a Device

Viewing personal content on devices that are different from the content source is another requirement of our system. As mentioned previously, the Microsoft Surface serves as the primary personal content hub, and users are able to send video from either the cloud or tabletop to a variety of devices. This allows users to view content on the iPhone, iPod, iPad and even a TV that is linked to a set-top box, as indicated by Figure 1. Additionally, as this content is stored on a particular device, users are able to view content at a later time or in different environments.


In order to implement the features previously described, deep integration between a tabletop and the connected devices is required. Concretely, we needed to link iPhones, iPads and the Microsoft Surface. This is difficult because the Apple iOS devices are an entirely different platform than the Windows-based Microsoft Surface. Traditional methods of inter-device communication such as SOAP are not supported on iOS devices and existing remote procedural call frameworks (RPC’s), which can also facilitate this communication, are also not available on iOS devices. We developed a layer on top of sockets for remote calls and for file exchange capabilities.

Sending Method Calls
Method Calls, or MC’s, are the messages passed between the iOS device and the tabletop. These encompass return information, processing instructions and function parameters required for the execution of code on the iOS device \cite{AppleSDK}. They are packaged into plain XML (POX), where each parameter is represented as a node and the processing instructions and message ID are standard parameters to each appropriate function. The MC is then parsed by the device and the particular method is triggered. On the Media Surface, this feature is implemented by two distinct interactions. The interaction involving placing a device onto the tabletop triggers a MC that instructs the device to upload thumbnails to a custom display on tabletop. The second interaction involves a user selecting and dragging a thumbnail from the custom display, as shown in Figure 2. By dragging and dropping a thumbnail onto the tabletop, a MC is triggered, sending instructions to the device to upload the video content to the tabletop.

Receiving Method Calls
MC’s can also travel from the device to the tabletop to trigger predefined functionality. This is implemented by hosting a small web server and exposing a small RESTful API, that is associated with the desired methods and can be called from the iOS device. REST was primarily chosen because SOAP is not available on iOS devices. Additionally, it was chosen in order to make the API accessible easily given the networking frameworks available on the iOS device.


After stabilizing our prototype, we plan to conduct a feasibility study for using touch based devices to organize personal content. Additionally, we plan to extend compatibility by utilizing Android and Windows Mobile touch-based devices as well as other digital tabletops. Extending compatibility also involves stabilizing the iOS Remote Connector API.

Last edited Sep 6, 2010 at 6:52 PM by zabedakbar, version 5


No comments yet.