Measuring Mobile Broadband Networks in Europe

OPEN CALL 1

Call objectives
Building and operating a mobile broadband (MBB) testbed is the key goal of the MONROE project. Such a testbed, however, cannot be built without the feedback of the experimenters who will be the main users of it. Therefore, external users are in the core of the MONROE project. Through the first competitive open call, MONROE aims to involve the external users in the design process of the testbed to create a platform that meets the expectations of its users. The experimenters selected through the first open call will access to the prototype platform. They will be allowed to run active/passive measurements as well as propose protocol experiments, which require active traffic generation and SW support from the MONROE consortium. They can further propose SW extensions to the platform as well as HW extensions to the infrastructure. Since the experimenters will have access to the prototype system, they are expected, as their contribution, to provide SW/HW extensions and/or feedback that will be used to refine the platform to its final version.

Call information
Project full name: MONROE–Measuring Mobile Broadband Networks in Europe
Project number: 644399
Call identifier: MONROE-OC1
Call name: First MONROE Open Call for Experiments and Extensions
Total budget: 1,800,000 Euros
Number of proposals to be funded: up to 12
Maximum funding per proposal: 150,000 Euros
Number of partners per proposal: The target number of participants per proposal is maximum 2.
Type of participants: The profile of participants is academics, industry or SMEs active in research and/or development of mobile broadband technologies, protocols and/or applications. The rules of participation are the same as for any H2020 proposal.
Duration of the experiment: The maximum allowed duration of each experiment is 18 Months
Language of the proposal: English
Proposal submission: via email to info@monroe-project.eu
Call deadline: Tuesday, March 15, 2016 at 17:00h CET (Brussels time)
Notification of acceptance: Monday, May 2, 2016
Starting date: June, 2016

Proposal Template: MONROE First Open Call applicants must use this Proposal Template (in Word format)

Detailed Call Information: Call Announcement

Contact: info@monroe-project.eu

 

UPDATE: OPEN CALL RESULTS

We have funded the following 12 proposal from a total number of 40 proposals received during the first open call:

  • Performance of Web RTC-enabled services on the MONROE platform Acronym: PoWeR, Industrial Innovation, N. Amram Technologies Ltd, Israel.

WebRTC is an emerging technology enabling real-time voice, video and data sharing in a web browser without the need for browser plugins. Providing a comprehensive set of standards, protocols and APIs; WebRTC’s potential in transforming communications is huge. We are involved in the development of an application enabling real time collaboration and interaction of geographically distributed users connecting through a multitude of internet-connected devices. Using the WebRTC technology to develop this application is an attractive option. However, the novelty of the technology and the lack of systematic studies raise some concerns whether and how the required high level of user experience can be achieved consistently in a heterogeneous environment. In this context, to carry out experiments on a large scale measurement platform is a very attractive opportunity to mitigate technological and business risks related to our development. In the proposed experiment, we aim to measure and evaluate key performance indicators of providing WebRTC-enabled applications through MONROE; in particular, the relationship between network metrics and metrics that are directly observed by the end user. In line with our experiment, we aim to implement software extension to MONROE in order to integrate it with the NuboMedia platform. The aim of the NuboMedia is to enable the development of rich and fully-featured WebRTC-enabled applications by providing an elastic scalable cloud platform specifically designed for real-time interactive multimedia services. We believe that both our experiment and our extension will result in benefits beyond our business interests, positively impacting various stakeholders of the platform. The results of our experiment are highly relevant to network operators, regulators and other application developers; while the integration of MONROE and NuboMedia has considerable synergies that can contribute to sustaining the MONROE platform by engaging future experimenters.

 

  • Mobile Network Analytics for Apps Performance Design: Napplytics, Scientific Excellence, EUROB CREATIVE and UNIVERSIDAD POLITECNICA DE VALENCIA, Spain.

Mobile devices and notably smartphones have become cornerstones in our daily lives. Being able to access to real time information is taken for granted in our modern societies as a key enabler for economic and social development. From a pool of 55% of randomly picked Android apps, 84% require Internet access1. Therefore, understanding mobile network performance is critical for providing a superior user experience and might in ultimate way determine the business success for an app, specially if it heavily depends on Internet access. FOR users, as already pointed out by MONROE call, is becoming popular using measurements network tools (like Netalyzr of MobiPerf) to determine problems in their connectivity. However, for Apps SW developers this approach has not yet become a standard procedure when debugging apps performance. While developers already analyse the user behaviour when using an app: usage frequency, time spent, used functionalities, etc and there are several well-known tools for this (like Yahoo Mobile, Mixpanel or Countly), these tools specifically focus in the user interaction with the app and do not provide data for an effective app tuning in terms of protocols, security, network usage, connectivity strategies, etc from a pure mobile network approach. There is thus an evident room and need for such dedicated tools and related knowledge. By NApplytics project EUROB and the Universidad Politecnica de Valencia will determine which are the KPI that govern an app User Experience considering both protocol and radio issues by means of an android library that could be easily incorporated to any Android app. In order to properly design the library and correctly identify all relevant parameters that may impact user experience the HW, SW and tools provided by MONROE will be essential in the project execution and results.

 

  • Quality-of-Experience of adaptive video streaming in mobile broadband networks: Nestor, Industrial Innovation, StreamOwl (SO) and Athens University of Economics and Business (AUEB), Greece.

The wide deployment of multimedia services over packet networks has highlighted that the original design of the Internet as a best-effort network makes it problematic for bandwidth-intensive and delay-sensitive applications, like video streaming. Additionally, network operators typically employ centralized delivery architectures that lead to long paths between end-users and content servers, waste of network resources and increased delays. This issue becomes more important in mobile broadband networks, which have stringent resource limitations and fast- changing conditions. The NESTOR project aims at conducting an experiment campaign based on the MONROE platform to evaluate the Quality-of-Experience (QoE) of popular video streaming services (e.g. YouTube and Netflix) with active and passive measurements. Special emphasis will be given to adaptive video streaming (especially MPEG-DASH) which enables the seamless adaptation of the video client to the specific network conditions of each user and is more relevant to mobile broadband networks. The understanding of the impact of the network parameters and the media content on the human perception are key factors in optimizing the end-to-end delivery chain. More importantly, the NESTOR project will help create a sustainable experimental platform which will be able to be used after the end of the measurement campaign by third-parties (e.g., operators, video platforms, and network engineers) to test and evaluate the performance of adaptive video streaming services in a seamless, intuitive and unobtrusive manner. Thus, the experiments will assist in benchmarking mobile broadband operators in terms of reliability and in the identification of key performance indicators that could be used to improve their networks.

 

  • Software defined network based available bandwidth measurement in MONROE: Sometime, Scientific Excellence, University of Napoli Federico II, Italy.

In SOMETIME, we plan to add Available Bandwidth (ABw) estimation to the set of metrics collected by MONROE and to characterize in terms of ABw the broadband mobile networks considered in MONROE. We will implement the estimation by active measurements leveraging Software Defined Networks (SDN) paradigm, both to tune the technique considering interference with node-local processes (a more realistic scenario compared with mutually exclusive measurements), and to mitigate such interference. ABw can be considered as the spare capacity on a network path, and is an important metric with many applications; optimized routing, adaptive encoding, just to name the most common. It is related to other path capacity metrics, such as TCP bulk transfer capacity, but is different from them, with different measurement procedures and practical applications.Specifically, ABw provides an upper bound to available network capacity, dependent on network infrastructure and usage, but independent from the transport protocol and application. This makes ABw an highly valued metric to characterize operational conditions of a network path, useful to inform at the same time different kinds of applications without per-application tests. As a counterbalance of its importance, ABw estimation is not trivial, especially in wired-cum-wireless scenarios (like monroe testbed), that also pose constraints related to volume based billing and capping for cellular data connections. Possible applications for experiments in MONROE (e.g. multimedia streaming to smartphones, tablets, in-vehicle-infotainment systems) present the additional challenge of possible sharing of computing and communication resources during the measurements: to emulate, assess and mitigate this interference we will adopt an SDN-based approach to manage traffic in mobile terminal (MONROE node). This approach will have the additional outcome of evaluating the possibility of running multiple active experiments concurrently on the MONROE node, guaranteeing isolation of traffic engineering or routing setups required by the different experimenters.

 

  • Movement: Extending and Experimenting upon the MONROE platform towards Voice and VidEo StreaMing AssEssmeNT in Mobile Networks, Industrial Innovation, COSMOTE Mobile Telecommunications S.A. and FERON TECHNOLOGIES P.C. Greece.

The installation and maintenance of sustainable, large-scale platforms for monitoring commercial mobile broadband networks performance is considered a key theme within all mobile technology stakeholders, including operators, service providers, telecom SMEs, and regulatory authorities. Such platforms obtain objective information on the experience levels perceived by the end-users and correlate these levels with network performance indicators. MONROE is a promising approach towards this direction, and MOVEMENT aims to add significant value to the existing platform. This will be achieved through the provision of an open access “toolbox” of innovative components developed upon the MONROE open platform, that will support the experimentation and analysis of voice and video streaming services over 4G/Wi-Fi mobile data networks. MOVEMENT plans to: a) geographically expand the existing monitoring network with the deployment, integration, and operation of a set of new stationary and mobile nodes; b) enable the automated active testing of voice and video streaming services, including voice-over-IP (VoIP), VoLTE and on-demand video streaming (VoD) over 4G/Wi-Fi data networks, by developing a set of client and server agents for performing the tests, along with the front-end and back-end infrastructure for collecting the measurements and visualizing the QoS and QoE metrics; c) run various VoIP and VoD quality assessment campaigns over a live commercial network, as well as deploy an experimental platform based on open server tools (IMS, SIP) and live/test 4G eNBs for testing VoLTE services. The proposed extension and experiment activities are expected to enhance the MONROE project footprint, contribute towards the long-term project sustainability, and have a direct impact beyond the MONROE community, including the broader industrial and scientific communities, by providing open measurement and assessment tools and methodologies, as well as measurement data for prevailing (VoIP, VoD) and emerging VoLTE services.

 

  • AFFORDABLE LTE NETWORK BENCHMARKING BASED ON MONROE Acronym: Monroe-LTE, Industrial Innovation, Allbesmart LDA, Portugal.

To gain competitive advantage in today’s mobile market, cellular network testing, monitoring and improving customer experience is crucial. Today independent benchmarking companies are hired by mobile operators to run drive tests in a certain geographical areas. The high cost for running these tests results in a low frequency of execution, typically this benchmarking is executed no more than 2- 3 times per year, which is not sufficient to follow the dynamics of an LTE network in a dense urban area. The majority of the drive testing costs come from the car, driver, and the in-car technician. ALLBESMART, a SME specialized in smart city solutions, has already developed a Bus Tracker solution for several transportation companies in Portugal. The business idea behind this proposal is to take advantage of the existing partnerships with these companies to carry on and sell network benchmarking services to Mobile Network Operators (MNO). With this approach, unattended measurement nodes can be deployed in existing transportation fleets without the need for dedicated field personnel, reducing the cost of testing up to 70%. This project is both an experiment and an HW/SW extension of MONROE. The experiment will use mobile nodes available in Turin (Italy) to create and validate the Allbesmart automatic LTE network benchmarking tool using the MONROE nodes placed in buses. After that, 5 additional MONROE mobile nodes will be deployed in vans circulating in Lisbon (Portugal) city centre. The “benchmarking tool” (software extension) will be developed allowing an easy comparative analyses of mobile network QoS and QoE KPIs based on MONROE raw data. To showcase the MONROE extension in Lisbon, three LTE MNOs will be considered and the results discussed with representatives of the targeted MNOs and the Portuguese regulator. Furthermore, a viable sustainability model for the proposed MONROE extension will be developed.

 

  • Software Radio for Measuring Mobile Broadband Networks: Sophia, Scientific Excellence, Software Radio Systems, Ireland.

A software radio combines a generic radio frequency (RF) front-end with a general-purpose processor to provide a powerful platform for wireless systems testing, measurement and experimentation. The proliferation of commodity RF front-end devices and the increasing capabilities of general-purpose processors have enabled a thriving open-source software radio ecosystem with projects focusing on every imaginable wireless technology and application. Software Radio Systems (SRS) is an Irish SME which builds on over 15 years of software radio research and development. Through the open-source srsLTE and srsUE projects, SRS provides high-performance tools for analysis and implementation of 4G LTE broadband networks and applications. Under SOPHIA, SRS will extend the MONROE platform to support software radio applications. SOPHIA will use the existing SRS open-source libraries and applications to demonstrate the benefits of software radio and to carry out experiments to perform detailed measurements of mobile broadband networks. The project will further examine the capabilities and limitations of the MONROE node as a software radio platform and provide comprehensive feedback to the MONROE consortium on performance, user experience and recommended extensions and improvements. The impact of SOPHIA will be to significantly expand the range of experiments which can be supported by the MONROE testbed, to enhance the value of existing experiments and to greatly increase the pool of potential testbed users and experimenters. SOPHIA will vastly enhance the detail with which mobile broadband networks can be measured and analyzed. It will increase the useful lifetime of the testbed by enabling over-the-air software upgrades to deploy new measurement tools and support newly deployed technologies. Finally, SOPHIA will open the MONROE testbed up to the full open-source software radio ecosystem, the huge array of existing projects, tools and applications and the vibrant community of designers, developers and users.

 

  • Maril-in-Monroe Measurement Adaptation and Reporting in LTE, Scientific Excellence, University of the Basque Country, Spain.

The rise of Mobile Broadband (MBB) usage comes together with an increasing awareness in quality of service as perceived by customers. Whilst many approaches from both the standardization bodies and the scientific environment have tried to define tools to measure and represent objective and subjective network performance, no consensus has been reached regarding the most feasible and accurate measurement methodology. Taking advantage of MONROE MBB networks measurement deployment, MARiL proposes to develop and contribute as open source multi-layer measurement tools that implement the most significative standardized Internet access measurement methods. In this regard, MARiL targets both main TCP-based and IP-layer model-based approaches. Regarding the validation of this proposal and seeking for a deep study among different contexts of use, several tests will be designed and carried out in the MONROE’s Pan-European deployment. Firstly, static nodes will be used in order to establish a fair comparison between both measurement philosophies. Secondly, a large-scale measurement phase will be held under different mobility patterns, constraining developed tools in such challenging conditions. The results comparison amongst these alternatives will help determining the most adequate measurement philosophy under each scenario. Finally, helped by the metadata obtainable from MONROE node, MARiL will propose a cross-layer mapping of objective performance metrics aiming to define a more accurate relationship between performance at different layers under the aforementioned conditions. Hence, MONROE will benefit from MARiL’s for its validation and addressing of MONROE as the main large-scale platform in Europe, as well as for its developments and studies for future open-calls and the further research advance. The outcome of MARiL will also input to different standardization bodies, including ITU-T, IETF and ETSI, while targeting liaison with RIPE and M-LAB initiatives. Thus, MARiL leverages the impact of the whole MONROE project in terms of transfer-to-standards and open data/open source communities.

 

  • Rapid Interpretation and Cross-Experiment Root-Cause Analysis in Network Data with Orange: Ricercando, Scientific Excellence, University of Ljubljana, Slovenia.

The goal of the RICERCANDO project is to develop an advanced toolbox for mining MONROE data to support integrative exploration, visualization and interpretation of data and meta-data across multiple experiments. The main use of the envisioned toolbox is to facilitate the identification and interpretation of anomalies and problems within the data (e.g., clusters of measurements reporting particularly poor performances). Our aim is to (1) ease the problem discovery and troubleshooting of the MONROE monitoring system, (2) avoid erroneous accounting to the monitored broadband mobile network(s) of data anomalies caused by glitches within the monitoring system, and (3) speed up the correct identification of the source of the problem (Root-Cause Analysis). RICERCANDO does not propose new additional experiments. Instead, it will consider data and meta- data from the whole range of experiments supported by the MONROE platform, and combine them with context data about the status of the MONROE system itself (e.g., software and hardware configuration of every system component). The integration of these data with advanced data mining, visualization and interactive data exploration features will support the human expert(s) in the process of detecting and understanding the problem, narrowing down to a short list of candidate critical dimensions. A distinguishing feature of RICERCANDO is the interdisciplinary composition of the project team that includes established data mining experts working together with networking experts. The envisioned toolbox will be developed as an extension of Orange (orange.biolab.si), an established open-source data mining tool widely adopted by different scientific communities, including but not limited to bioinformatics and teaching of data science. The new toolbox, specifically tailored for MONROE data and use-cases, will be released as open-source.

 

  • Prioritisation and Resilience for Emergency Communications: PREC, Scientific Excellence, University of Aberdeen Court, Scotland.

This proposal has two components: It extends the MONROE testbed to include new types of infrastructure, and it proposes new experiments to explore survivable networking that rely upon both this new infrastructure and on the existing testbed nodes. Four new permanent nodes will be added to the MONROE platform to increase the diversity of link characteristics. They will support combinations of mobile, Ethernet connections, fixed point-to-point wireless links, low-band radio interfaces and satellite broadband. All these classes of link are used routinely in emergency preparedness exercises and have been used in disaster situations to provide communications. The use-case brings these together in a unified IP network. The performance of a radio link may be affected by environmental factors. The new nodes therefore provide an interface to query physical layer performance data. We will also deploy weather monitoring equipment to allow experimenters to determine why a link has degraded quality. The project proposes an application use-case for emergency communications in disaster situations. This is based on IP multi-homed support that enables resilient differentiated services, enabling an application to select the best available transport path. Resource-limited paths will ensure traffic is critical traffic prioritised over other traffic, so that network degradation does not halt operation of the application, although performance may be degraded. These techniques will be deployed across the distributed test infrastructure provided by the MONROE platform, allowing experiments to take advantage of the diverse capabilities and connectivity of the nodes. A distributed measurement campaign will measure performance of the developed techniques combining passive link measurements with active traffic generators to measure key performance parameters (e.g., packet loss, latency and jitter). We expect the results of this measurement campaign to inform research into new methods and their applicability across a heterogeneous network.

 

  • Monitoring and Analysis of Quality of Experience in Mobile Broadband Networks: Mobi-QoE, Scientific Excellence, AIT Austrian Institute of Technology GmbH Tran-Gia Informationstechnik GmbH, Austria.

Quality of Experience (QoE) is a well-known concept in the networking research community, but its development has been traditionally limited to laboratory studies. We are witnessing a growing demand from mobile operators for more user-centric approaches to manage their networks in an increasingly competitive scenario. This need has boosted the research interest in scaling QoE out of the lab, bringing it into the management of operational networks. This combined industry/research growing interest in QoE-based solutions for network management motivates the Mobi-QoE proposal. Mobi-QoE proposes both to extend MONROE’s testbed with new software (SW) tools as well as conducting innovative experiments. The first objective of Mobi-QoE is to extend MONROE’s testbed to the QoE domain, by integrating novel SW-based QoE-capable measurement tools and QoE subjective-models to assess the performance of MBB networks for popular end-user services (e.g., YouTube, Facebook, Spotify, etc.) from a user- centric perspective. Such tools and models provide a multi-layer monitoring perspective, measuring QoE- relevant features at the network and application layers, and forecasting end-user experience (e.g., MOS scores). The second objective of Mobi-QoE is to evaluate MONROE’s testbed visibility in terms of QoE metrics, relying on crowdsourced field trials. Mobi-QoE plans to evaluate the MONROE QoE capabilities introduced by the Mobi- QoE tools and models, comparing the MONROE-Mobi-QoE forecasted end-user experience to that reported by real MBB users through field trials, conducted with Mobi-QoE tools directly at end-user mobile devices (i.e., smartphones) in areas covered by the MONROE platform. Crowdsourced measurements would allow to further refine the integrated QoE models. To further show the benefits of the Mobi-QoE extensions to MONROE, the proposal foresees to apply the tools and models to study different MBB measurement problems from a QoE perspective, conducting several experiments. These include: benchmarking ISPs from a QoE perspective, analyzing the impact of mobility on QoE, and detecting and diagnosing QoE-relevant degradation in MBB networks.

 

  • Utility-based Networking experiments for Improving QUality of Experience in mobile broadband environments: Unique, Scientific Excellence, Institute of Communications & Computer Systems (ICCS) and Incelligent (Incel), Greece.

In UNIQUE, we aspire to evolve, implement and experiment with a unified utility-based framework based on both QoS (network-centric) and QoE (user-centric) features and parameters, which allows to enhance, formulate and improve various advanced network operations in mobile broadband (MBB) infrastructures (network selection, offloading, resource allocation, etc.). The proposed methodology and intended MONROE software extensions are expected to be provided as services to future experimenters that desire to investigate their own mechanisms and protocols. UNIQUE targets at two key runs of experiments and respective software extensions, depicting the operation and benefit of the proposed utility-based framework, i.e., a) utility-based access network selection, and b) utility-based concurrent multipath routing and traffic control. They aim at demonstrating the power of the proposed framework for key user-desired and bandwidth intensive network services and applications, such as multimedia downloading. In order to enable these scenarios and enhance MONROE capabilities for advanced experimentation involving end-to- end performance evaluation and measurements, various software extensions enabling utility-based network study and optimization will be developed and integrated in MONROE platform. Thus overall, UNIQUE, in addition to advancing the state-of-the-art in the research area of network utility performance optimization in wireless mobile networks, suggests long-term software extensions of the platform, and experiments that will leverage the available infrastructure and provide feedback to MONROE for improving its operation and extending the offered services. The expertise of the two participating groups (i.e., ICCS with long research experience in wireless networks and Incelligent a company offering software products & services for management of wireless networks) is complementary, ensuring a full-circle approach from scientific excellence to final integration of the proposed methodology and experiments, to software and service delivery.