MMS tests on multimedia mobile phones are part of the process of verifying interoperability between handsets, as Markus Hendeli of Rohde & Schwarz explains
GPRS, EGPRS and UMTS are powerful data transmission networks. To exploit available features to the full, network operators offer mobile phone users attractive applications such as the simultaneous transmission of pictures, video clips or music files in the form of multimedia messages (MMS) to one or more users. Network operators must ensure smooth multimedia file exchange. However, as a prerequisite for an efficient network, all mobile phones have to meet specific minimum requirements for handling the file formats used.
A multimedia message consists of pictures and various other multimedia files. An SMS (push notification) informs recipients that they have new messages which they can download from a server in the network. In some cases problems arise due to the different file formats that mobile phones support. For example if a mobile phone user sends a picture in GIF format to a recipient whose mobile phone can only display JPEG format, the picture would first have to be converted from GIF to JPEG in the network.
Multitude of phone types
Given the multitude of available mobile phone types, it would be virtually impossible for network operators to accomplish this task. All mobile phones must therefore be able to correctly replay pictures and other multimedia content in various sizes and file formats. This problem is solved by a uniform standard for the transmission of multimedia messages. It is also important that the MMS functionality of mobile phones be tested for message creation, replay and delivery options, in order to ensure smooth operation in the network.
To meet this minimum standard and verify interoperability between different mobile phones, the Open Mobile Alliance (OMA) defined test cases on behalf of the Global Certification Forum (GCF). These test cases make it possible to assess the creation and replay of various kinds of multimedia content. Some of these test cases were adopted by the GCF and the PCS Type Certification Review Board (PTCRB). They have thus become part of the certification for all mobile phones supporting MMS.
Unlike signaling test cases, these test cases verify application layers instead of lower protocol layers. Since the protocol layers tested are higher than the layers that are specific to mobile radio, the tests are basically independent of the mobile radio technology. They mainly test the correct creation, transmission, receipt and display of multimedia content. Signaling tests, on the other hand, verify, for example, whether a mobile phone correctly establishes a GPRS connection for data transmission.
Independent purpose
In MMS testing the purpose of the test is independent of the underlying signaling protocols and the mobile radio technology used. For this reason the expression “bearer agnostic testing” is also used in this context. Nevertheless a system simulator is needed, and not every system simulator can be used with every MMS test system. The push notification to a recipient of an MMS is transmitted in the form of an SMS. Though the MMS transmission takes place via IP, SMS transmission is system-specific. For SMS transmission, the test system has to control the mobile radio protocols and protocol stacks of the mobile radio network concerned. A driver specifically tailored to the system simulator is therefore required.
Divided tests
The test cases developed by the OMA can be divided into transmission tests and reception tests. These can be further subdivided into tests which check for correct replay or creation of an MMS, or the MMS-specific settings, or the transmission of status information. Each of these test classes has specific test evaluation characteristics and test setup requirements.
Every test setup requires a system simulator to establish the RF connection to the mobile phone and to handle the necessary protocol for setting up an IP connection and for transmitting a push SMS. An MMS server or an emulation of such a server is needed for transmitting the MMS. Since messages between the MMS server and a mobile phone (MM1 protocol) are based on HTTP, but not all mobile phones can process HTTP, a WAP gateway may have to be used. This gateway converts HTTP into wireless session protocol (WSP) and vice versa.
Verification
Reception tests verify the entire system from receiving an MMS and decoding the SMIL file, right through to the screen display. A SMIL file is a structure specification containing among other things information on the presentation and arrangement of the multimedia content. For test purposes, a multimedia message is sent to the mobile phone. A pass or fail result is obtained by manually comparing the multimedia message displayed by the mobile phone with the reference content provided by the OMA.
In the case of transmission tests, however, the verdict can be reached automatically. The mobile phone sends multimedia messages to the tester. Using a parser, the tester splits the MMS into its individual media files and the structure specification (SMIL), then checks whether they comply with the reference content. These tests are therefore not intended to verify the display, but rather to check that an MMS consisting of pictures, text and other content is created in compliance with the specification.
The comparison is made at bit level, since the individual components of the MMS can be extracted from the MMS. A mobile phone user creates an MMS using only the menu on the phone. This makes it possible to create a presentation of the various items of content without having to create a SMIL structure specification manually. The phone must also complete this task in line with the specification.
Besides replaying multimedia content, a mobile phone also has a number of settings which need to be verified. For example the mobile phone can be ordered to download a new message immediately or simply to leave a message informing the user that there are new messages available. It is then up to the user to decide whether and when to download the message. It is also possible to test message forwarding as well as the transmission of information about whether the recipient has opened the message.
Summary
Certification of the MMS functionality introduced a new class of GCF and PTCRB tests alongside RF and protocol tests. These tests significantly differ from the previous tests and therefore require a special test setup. However, since the transmission of a multimedia message is to be verified from the RF right through to display of the content, a system simulator is still needed. It establishes the IP connection between the MMS server and the mobile phone, and allows the push SMS to be sent.
The test itself is analyzed in the same way for all the mobile radio technologies used (e.g. GPRS, EGPRS or WCDMA). The MMS test cases belong to the same class of application enabler tests as push-to-talk, instant messaging and video telephony. Certification tests are also being considered for these applications. This area of testing will therefore expand further and become an attractive field of activity for network operators, phone manufacturers and the test industry alike.
The operator viewpoint: testing essential for targeted services
Keith Dyer spoke to Francois Mahiue, head of business devices marketing, and Cynthia Gordon, international marketing manager, Orange Business Solutions, at Orange, on how testing enables Orange to provide carefully targeted devices in a segmented way to its business customers.
FM: “The way we do it is to operate on two principles. We test phones at a number of laboratories in Europe and in France within the France Telecom Group. Then we have customer trials, in which there are a number of different users and scenarios. Whenever a device comes to Orange for testing, testing takes place in up to eight labs, mostly in Paris but also in Bristol.
There are five to six key priorities for the business market, The radio side, device functionality, connectivity features, battery life, the way the device can be customised for use and its ability to connect to the network in the way we want it to.
The testing period takes about six weeks on average, with about 10-12 people being involved in the testing of each device.
Then there is a trial phase with users, in which the focus is much more on the usage of the phone. For example with the SPV M2000, which has a QWERTY key pad we were looking at the way email was used within the business. The configuration of the keyboard, connectivity to laptop and headset, and the stability of the software for intensive email use all came under scrutiny.
CG: It’s very important in the business market in terms of the selection of devices, as part of the proposition we offer. Whether it is our Mobile Office data card or mobile email with a smartphone, we test the entire experience. Extensive user testing looks at how it works with the network and how the software on the PC works with the card.
FM: We find in some way suppliers value our feedback. They are not necessarily getting the same from other operators.Every single device is like a mini-project. Part of our strategy is to make sure we are first to market on a number of key products. Usually the first to market is the one that will experience the pain on products.
Our Orange Group Device Requirements is a set of specifications provided twice a year to our suppliers. It defines the marketing requirements within Orange and is very clear on the product features we require, as well as application support. It’s a very complicated technical approach specifying the challenges new services would create. By doing this we are continuously drawing up a bible where suppliers can get all our technical requirements for launch.
CG: Part of the OGDR is the requirement to define the Orange home screen. On some devices we set requirements for one click access to specific apps. We have started to implement homescreens on smartphones and PDAs. The K7001 would be an example of a phone with a proprietary OS and proprietary homescreen that has been specifically designed for us and guarantees a similar experience across different types of handset.
Another example where the business market has different requirements is in something like ringtones.
So for us it’s not just about the stop/go test of a device but about providing an overall customer experience.
