Transcoding Module Evolution
From Morfeo Wiki
Contents |
Current approach
Introduction
Current state of MyMobileWeb transcoder module allows dynamic adaptation of images, including image resizing respect device’s display size and image conversion to fit the format needed by target terminal.
To carry out this issue MyMobileWeb uses the following libraries:
- size transformations: JAI
- format transformations: WBMPMaster, Gif89Encoder, Batik.
- weight transformations: JAI
Architecture
The current architecture is compound of five packages which are shown below. Each package links to the corresponding class diagram.
- org.morfeo.tidmobile.transcoder (Class Diagram)
- org.morfeo.tidmobile.transcoder.server (Class Diagram)
- org.morfeo.tidmobile.transcoder.image (Class Diagram)
- org.morfeo.tidmobile.transcoder.image.gif (Class Diagram)
- org.morfeo.tidmobile.transcoder.image.svg (Class Diagram)
CSS properties
MyMobileWeb users can guide the transcoding process by means of the CSS properties listing below:
- Transcode: boolean variable that stays whether transcoding should be applied to specified image or not.
- Weight-width: weight of the width of the image respect display's width
- Weight-height: weight of the height of the image respect display's height
- Aspect-ratio: maintaining aspect ratio, yes or no?
- Width: width of the image
- Height: height of the image
OMA STI
The Open Mobile Alliance (OMA) has defined a Standard Transcoding Interface (STI v1.0) between the application servers and the media adaptation engine in order to enable a new architecture where the transcoding engine is shared between various services deployed at the operator's site. The use of STI in the transcoding module evolution will be considered since MyMobileWeb has a commitment with open standards, so it is reasonable to follow the approach of the greatest organization in the world standardizing the mobile technologies.
Architecture
The scope of the OMA Standard Transcoding Interface (STI) sub-working group is to provide a standardized interface between Multimedia Application Platforms and a Transcoding Platform to allow transcoding of media Content files based on Application specified transcoding parameters and/or User Equipment capabilities. In spite of the fact that the STI goal is to be generic enough to allow different applications (such as browsing, media download services, push services, etc) to make use of such an interface, those aspects closer to MyMobileWeb platform use case will be considered with special attention in this chapter. The Standard Transcoding Interface defines three reference points:
- TI-1: Transcoding interface
- TI-2 and TI-3: Access to remote content stored in databases or servers
- TI-4 and TI-5: Access to reference databases or servers
The picture below shows a generic view of the OMA generic architecture (grey arrows are out of the scope of STI), including the possible transference protocols. Particularly, the access to a Device Description Repository (the entity described as “UAProf DB or Server”) is being taken into account by the MyMobileWeb project through its participation in the W3C Device Description working group, as partners of this project are editors of the document Device Description Repository API, which is going to be proposed as a W3C Recommendation.
Taking into account this general view of what STI offers, it's possible to go further and analyse the MyMobileWeb use case:
- MyMobileWeb Application would represent the “Application Platform” in the picture.
- The Transcoding Platform would be the combination of hardware and software that provide transcoding functionality.
- The Application Platform and the Transcoding Platform can share the same physical machine, but are still different logical entities using STI as the interface.
Other interesting issues are commented below:
- The Application Platform may include the output formats and characteristics (i.e. the detailed transcoding parameters) in the transcoding request and/or include a reference to User Equipment information (e.g. handset profile reference). The Transcoding Platform may use the handset profile reference to retrieve the detailed transcoding profile from the reference database/server or from an internal database, and/or retrieve the basic User Equipment capabilities from a UAProf database/server (or, in general, a Device Description Repository).
- The resulting User Equipment capabilities and/or transcoding parameters are then used by the Transcoding Platform to perform the content adaptation that is best suited for the specific User Equipment and the Application requirements.
- In the response, in addition to the transcoded content, the Transcoding Platform will return information such as a return code, total duration (for transcoding), and file size. How the Application Platform uses this additional information (if at all) is out of the scope of the STI specification.
- The Transcoding Platform is also responsible for reporting (or returning) errors and warnings to the requesting Application Platform. The errors can be attributed to an erroneous transcoding request or the inability for the Transcoding Platform to perform the requested transcoding.
How to perform a TranscodingRequest
The next picture shows the overall UML Diagram to compose a transcoding Request following STI standard.
The figure below represents a detailed transcodingParams UML Diagram, very useful to see how to specify the transoding parameters:
Implementations
Alembik (http://alembik.sourceforge.net/)
Alembik Media Transcoding Server is a Java application providing transcoding services for variety of clients. Sharing the name with a device for carrying out alchemical transmutations it reshapes and reformats media files according to needs and capabilities of any environment. Any Alembik client needs only to pass to the server the URL of a media file with a set of transcoding parameters (like desired dimensions, size, mime-type, etc.). The latters may even remain unspecified - Alembik will evaluate them automatically according to the User-Agent string passed by a client.
Alembik is developed as an open-source project, whose goal is to provide an OMA-STI fully-compliant media transcoding server based on J2EE technology. OMA-STI stands for Open Mobile Alliance's Standard Transcoding Interface, which precisely defines a contract between a client (requesting some media files to be transcoded) and a server (which carries out the transformation process according to the passed parameters set). Alembik is the first open-source attempt to meet that standard.
There are four communication channels at present that Alembik server exposes:
- SOAP service for all types of clients,
- JSP tag library for development in Java-based web containers,
- EJB facade for J2EE clients,
- HTTP service for web pages development in any environment.
The application architecture is composed in a modular fashion, where five main modules are: Java transcoding API, SOAP service implementation, EJB-based facade (including JMS-based asynchronous processing support), JSP tag library and core libraries. Further on the core part may be logically divided into several blocks:
- Pluggable media processors, each one serving a particular media type (image, audio or video),
- Storage providing media loading features and quick access cache for original and transcoded media files,
- WURFL library-based User-Agent resolver,
- Mime-type detector using media's file extension and actual content analysis (Mime-Util library),
- Pre-defined transcoding profiles repository,
- Synchronizer component securing concurrent access to resources.
Alembik may be deployed into any Java-based web/application server, although it has been so far tested and released for three platforms only: Glassfish, JBoss and Apache-Tomcat. Each release bundle contains a relevant archive file (EAR or WAR) as well as client JAR files (both SOAP- and EJB-based) and an example web application (WAR) showing the JSP transcoding tag library in action. Although still in the beta stage, Alembik has been already deployed in production for Spanish telecom operators (e.g. Telefonica and Vodafone).
Gaia Transcoder Server (http://gaia-git.sourceforge.net/roadmap.htm)
Gaia Reply has announced, as the second step of their roadmap, the implementation of a Transcoder Server (TS) compliant with OMA STI standard, supporting transcoding for image, video and audio too. The server will response to SOAP-RPC calls. As far as transcoding server architecture is concerned they have announced their intention to implement the following components:
- a STI compliant API interface for managing images, audio or video
- a STI parser to enable STI messages based RPC
The transcoding process will be characterized by four different steps:
- the transcoding server receives an STI request
- the parser interprets the message and translate it into a sequence of API interface calls
- the API interface implementation calls "plugged" transcoding library native methods
- the plugged transcoding library (audio,image or video) performs transcoding stuff
Volantis (http://opensource.volantis.com/)
On the 19th of March Volantis has released its Mobility Server to the open source community under the GNU General Public License (GPL), version three. As they had announced in the overview (http://www.volantis.com/docs/Volantis-Mobility-Server-1.0-Overview.pdf) they offer STI support, but it basically consists on mapping the ResourceDescriptor they use to describe a transcoding operation into a TranscodingRequest with an unique TranscodingJob (according to OMA interfaces) in order to make easy for the user to invoke an external STI transcoding server. However they haven`t released any STI-compliant transcoder.
Proofs of Concept
Images & audio transcoding using STI
The intention of these proofs is to evaluate the integration of MyMobileWeb with a transcoding platform by using the OMA transcoding standards. As it was mentioned above, MyMobileWeb use case could fit right in with the architecture specified by OMA standards. If we think in MyMobileWeb as the application platform, all we need to carry out a simple proof is to get an OMA-compliant Transcoding Platform. The Transcoding platform chosen to do this task is the Alembik project, as it fulfils our pre-requisites and it uses LGPL. To perform this transcoding operation Alembik has been deployed under the same tomcat application server that a MyMobileWeb project does, so MyMobileWeb will invoke Alembik (SOAP over HTTP) each time a transcoding operation is needed. What respects to Device Description Repositories or Server, it has been left out for now. Alembik offers the possibility to specify a profileID (user-agent String) and it would be in charge of discovering the device capabilities by looking up on WURFL (also an UAPROF xml could be specify), but in this basic example it is not really necessary to do that search since MyMobileWeb platform had already gotten the device capabilities from WURFL.
Next picture shows the architecture for that examples:
Note that the current MyMobileWeb cache system is kept, so each time a transcoding operation is done, the image result will be save under the temporal folder specified in the Cache_Work_Directory of MyMobilWeb, and if the same transcoding process is required for a second time, the previously saved resource is immediately returned.
How to configure Alembik and MyMobileWeb to run the proofs
In the Installation prerequisites section of Alembik web some information needed to deploy the Alembik project could be found. We have decided to deploy it into an Apache Tomcat Server as it is the server used to deploy MyMobileWeb applications. The only restriction which comes with using Tomcat is that we are forced to do transcodifications without asynchronous processing and EJB-based connectivity, since Tomcat is not an EJB3 container.
The basic steps to get the examples running are listed below:
- Install ImageMagick
- Install Ffmpeg Currently Alembik supports two versions of libavcodec: 51.40.4 and 51.48.0. To learn which version is installed, please execute the following command line: ffmpeg -version
- Install M4Box [Optional by the moment]
- Install Darwin streaming server [Optional by the moment]
- Download the last version of JAXWS 2.x (Java Api for XML Web Services) All its library jars have to be copied into /%tomcat_install_dir%/shared/lib/. The Alembik administration console requires also Java Standard Tag Library jars, which should be placed in the same folder. The latest JSTL release (1.1.2) can be obtained from Jakarta site.
- To avoid the next error message: "You are running on JDK6 which comes with JAX-WS 2.0 API, but this tool requires JAX-WS 2.1 API. Use the endorsed standards override mechanism http://java.sun.com/javase/6/docs/technotes/guides/standards/), or set xendorsed="true" on <wsimport>":
- Copy the jars jaxb-api.jar, jaxb-impl.jar and jaxws-api.jar in /%tomcat_install_dir%/common/endorsed/ and in <Java\jdk1.6.x>\jre\lib\endorsed.
- Download Alembik for Tomcat which includes alembik.war and alembik-soap-client.jar
- Copy Alembik war in your tomcat webapps folder
- Start you server and go to http://localhost:8080/Alembik to configure the installation paths. The Storage directory has to be chosen carefully, since it is directly related with a URL path to transcoded files, which is sent back to clients. At present, the root directory of Storage has to be equal to the real path, into which the appointed URL gets resolved by Alembik's web server.
- Deploy the MyMobileWeb application (alembik-soap-client.jar is included in the lib folder)
alembikProof
That simple project performs a transcodingRequest compound by four basic transcoding jobs in order to show how a Transcoding Request should be prepared to invoke a STI Transcoding Platform (Alembik in this case) from MyMobileWeb. Some transcodifications parameters are commented so as it can be seen different possibilities to prepare the transcoding process. The corresponding code is available in https://svn.forge.morfeo-project.org/svn/mymobileweb/trunk/MyMobileWebProofsConcept/alembikProof/
AdvancedSTI
That example (available in https://svn.forge.morfeo-project.org/svn/mymobileweb/trunk/MyMobileWebProofsConcept/AdvancedSTI/) goes further and tries to perform the Advanced project included into the example package of MyMobileWeb using the STI standards by calling Alembik. To carry out this issue the project includes (under WEB-INF/src folder) the MyMW Transcoder with basic modifications so as it maps the custom transformations into transcoding parameters to compose STI TranscodingJobs, which will group in a TranscodingRequest sent to Alembik. The final target of the example is only to get an easy-to-get starting point to see both the performance, and the possibilities that an OMA-STI based project offers, so the changes done in the architecture can not be considered as part of the new transcoder design.
The main changes are done in ImgTranscoder.java. Instead of invoking the doIt() method in the different transformation objects ,a new STItranscode() method has been implemented in order to map a set of transformations into an unique STI TranscodingJob.
New approach
The intention is to improve the functionality of the current transcoder by designing a new module which:
- covers different content types (audio, video, etc) and formats.
- has to be open enough to make easy the incorporation of new features.
- complies the existing standards regarding transcoding for mobile devices.
In order to follow, as far as possible, the OMA Standard Transcoding Interface:
- MyMobileWeb will provide a generic STITranscoder interface which has to be implemented in order to invoke a possible remote transcoding server. The main method of this interface is called transcode and it receives a TranscodingRequest, compound internally by MyMobileWeb, and it returns a TranscodingResponse, made up by the Transcoding Server.
- By default MyMobileWeb will include a concrete implementation of the STITranscoder interface called MymwSTITranscoder, which will perform transcoding operations in a local manner, without those features which could decrease the transcoding performance.
CSS Properties
A first approach for the transcoding module CSS properties has been designed based on the STI specification, so users could define most of the transformations specified in STI, by means of CSS.
A basic CSS properties definition could be specified as follows:
link.image.transcode {
brightness-level:15;
codec:image/jpeg;
codec-params: mode lossless quality-factor 90;
color-level: 15;
color-scheme-depth: 10;
color-scheme-scheme:paletteColor;
content-type:image/jpeg;
content-type-params: type-jfif;
contrast-level:10;
height:80%;
level-correction:true;
mirror-axis:UD;
noise-reduction:false;
resize-directive:aspectRatio;
rotation:180;
sharpen:false;
size-limit:454900;
transcode:true;
upsize-allowed:false;
width:80%;
}
Cache System
The Standard Transcoding Interface Specification V1.0 doesn't make reference to any issue regarding cache systems, thus in spite of the fact that most STI transcoding servers could implement its own cache, the existence of such system cannot be assumed by the Application Platform (MyMobileWeb in our case), which should implement it.
Current state of MyMobileWeb Cache System
Up to now, MyMobileWeb offered its own cache implementation. It has been implemented as a set of HashMaps available to images storage inside the org.morfeo.tidmobile.server package. The class ContentMapper stores the correspondence between a logical reource and the path to the physical file used to realize it according to the device that performs the request. The ResourceId, the ImageHint and the Device are used to compose the specific key to keep the resource.
New approach
The intention is to offer an open cache interface, so different cache systems could be implemented by following it. By default, Mymobileweb will provide some implementations of this interface and users, by means of configuration files, could chose what cache system to use for each kind of media type (audio,video, ...), according to their needs.
Note that the cache system will be embeded inside the transcoding module, although other modules of MyMobileWeb can use it. One of the controls of MyMobileWeb which will need to make use of the cache system is the map control, since its performance could be improved by caching some specific files.
Other features about the cache system:
- The cache system is independent of the cache mechanism that external transcoding servers could offer. On one hand a user of MyMobileWeb could use different servers, on the other they could use the default MymwSTITranscoder, which performs trancoding processes in a local way.
- The cache system will provide both memory and disk stores
- It will support persistent disk store:
Alternatives
http://www.javaworld.com/javaworld/jw-05-2004/jw-0531-cache.html
WhirlyCache
https://whirlycache.dev.java.net/
OpenSymphonyCache
http://www.opensymphony.com/oscache/
JavaCaching System
http://jakarta.apache.org/jcs/
EHCache
A concrete implementation of the cache interface based on EHCache will be offered by default. http://ehcache.sourceforge.net/
Features
- Fast, simple and easy to use
- Standards Based
- Full implementation of JSR107 JCACHE API
- Application Persistence
- Persistent disk store which stores data between VM restarts
- Flush to disk on demand
- Fully documented
- Open Source Licensing
- Apache 2.0 license
Configuration
A specific MyMobileWeb.Transcoder.xml should be configured as follows:
<!-- Transcoder configuration file -->
<configuration name = "root">
<module name="ImageTranscoder">
<property name="Cache_System" value="EHCache" />
<property name="Max_Elements_In_Memory" value="300" />
<property name="Time_To_Idle_Seconds" value="1500" />
<property name="Time_To_Live_Seconds" value="1500" />
<property name="Overflow_To_Disk" value="true" />
<property name="Disk_Persistence" value="true" />
</module>
<module name="AudioTranscoder">
<property name="Cache_System" value="OSCache" />
...
</module>
<module name="VideoTranscoder">
<property name="Cache_System" value="EHCache" />
...
</module>
</configuration>
How to compose the cache entry key
Both the Source and the Target have to be considered to compose the cache entry key, so if two transcoding process have been done for the same source, but any of the transcoding parameters inside the Target is different, the result must be cached.
Other transcoding issues
Images Transcoding
A wide list of Java libraries to read and write pixel image files can be found here, and other to do it with vector image ones here Up to now, MyMobileWeb has been using JAI as the chosen library to carry out size and weight transformations. Some of the previously mentioned libraries have been tested in different projects involving content adaptation for mobile devices and JMagick turned out to be a better option. JMagick (which has a LGPL license) is an open source Java interface of ImageMagick, implemented in the form of Java Native Interface (JNI), although currently only implements a subset of ImageMagick APIs.
However, as it was shown above, a great variety of libraries is nowadays available and probably some of them could be grade as the best option depending on each particular case, so offering a STI support users could choose the STI-compliant transcoding server which best fits in their purpose. Note that Alembik allows you to configure the transcoding server, so as you can choose different MediaProccesors (each one delegating on different libraries) to perform the transcoding process.
