An Adaptive Application Framework with Customizable Quality Metrics

NSF The research activities in the lab have been partially funded by
NSF CNS award No. 1617551 "CSR:EDS:Small:Energy-Aware Redundancy Management" (PI: Ulrich Kremer) and NSF II-EN award No. 1730043 "Collaborative Research: Enhancing the Parasol Experimental Testbed for Sustainable Computing" (PI: Thu Nguyen).

Any opinions, findings, and conclusions or recommendations expressed in material related to this project do not necessarily reflect the views of the National Science Foundation.


Lead researchers

Ulrich Kremer (PI), Department of Computer Science, Rutgers University

Liu Liu (PhD student), Department of Computer Science, Rutgers University

Sibren Isaacman (Collaborator), Department of Computer Science, Loyola University Maryland

Research assistants

Rajanya Dhar (graduate student), Mahdi Emad (undergrad student), Connor Franc (undergrad student), Abdall Ahmed (REU undergrad student), Joseph Bolye (REU undergrad student), Ashley Dunn (REU undergrad student), Kaivalya Mishra (REU undergrad student), and Thomas Murrills (REU undergrad student)

Project Description and Outcomes

Many resource-constrained devices such as battery-operated embedded systems or smartphones require applications to produce outcomes under different, potentially changing, resource constraints which include energy and time. Relaxing application semantics through approximations enables trading off resource usage for outcome quality. Approximation is a form of redundancy where acceptable outcomes differ in their quality and resource usage.

Although quality is a highly subjective notion, previous work assumes given, fixed low- level quality metrics that often lack a strong correlation to a user's higher-level quality experience. Users may also change their minds with respect to their quality expectations depending on the resource budgets they are willing to dedicate to an execution. This motivates the need for an adaptive application framework where application developers provide quality tuning knobs for different high-level quality aspects of the application. Just before application execution, users can customize their quality expectation through different quality knob settings and a user provided resource budget. The execution framework then monitors the resource usage of the running application and adapts an application's behavior in order to produce the highest customized quality outcome while respecting and utilizing the provided resource budget. The adaptation may be necessary due to uncertainties in the predicted resource usage, addressing the situation where the predicted and actual observed resource usage differ.

Such uncertainties can occur when multiple applications share common resources, for example when running on the same smartphone. Individual applications may perceive the resource usage of other applications as resource uncertainty, prompting adaptive behavior changes across all active application, leading to suboptimal, overall quality outcomes. The adaptive framework has been extended to summarize the resource impacts of different active applications under different individual adaptation strategies. This allows adaptations to be coordinated across all active applications, resulting in improved overall quality outcomes of the individual applications.

Customized quality notions and execution resource budgets

A key outcome of this project is the design and implementation of a novel framework (RAPIDS) where application developers, application users, and automatic techniques work together to achieve approximate application executions where subjective quality is optimized while the provided budget is respected and utilized as much as possible. The KDG (knob dependence graph) is a new representation to express the configuration space, construct performance and quality models, and determine reconfigurations at application execution time (Figure 1).

RAPIDS workflow

Approximation can be an effective technique to allow tradeoffs between quality of outcomes and resource usage. Since quality is subjective, i.e., depends on user perceptions, it must be considered a first-order concept together with a resource budget. Application users have to be able to make informed decisions about quality vs. resource tradeoffs, which means that both entities have to be described at abstraction levels that make sense to users. For example, it is not clear how low-level quality metrics (e.g.: bit errors) relate to high-level quality notions, for instance for a video player where brightness of image, smoothness of image transitions (frame rate), and image size are tangible user experiences.

To make quality a first-order user-level concept, the application developer has to be involved. Application developers have not only to define levels of approximations of the program, i.e., program configurations, but also outcome quality metrics that they want to expose to the user. Users are then able to specify a customized quality metric by defining their overall quality experience as a weighted, linear combination of the developer provided basic quality metrics. For example, the developer may expose image metrics as discussed above to the users. Before a program execution, one user may prefer a brighter but smaller image at a medium frame rate, while another wants a high color fidelity image, at the cost of a lower frame rate and less image brightness.

Dynamic reconfiguration to address uncertainties

Dynamic remapping of configurations, i.e., changes to the approximation level can be used to deal with uncertainties of resource availabilities or environmental conditions. We have shown that reconfiguration can be an effective technique to adapt to these uncertainties, allowing program executions to successfully terminate where static configuration choices would have led to significant underutilization of resources or execution failures, i.e., depletion of all resources before reaching the end of the execution.

Cross-application configuration management

RAPIDM dendogram Since approximations are driven by resource usage and resource availability, finding the best approximations across a set of concurrently executing programs on a shared computing platform is another challenging problem. Treating the set of applications as a "single" application, i.e., optimizing the cartesian product of all individual configuration spaces is not feasible due to the size of the resulting combined configuration space. The RAPIDM framework shows that individual configuration spaces can be effectively clustered into "buckets", i.e., equivalence classes, that have the same resource footprint and behavior under reduced resource availabilities (Figure 2). This summarization technique makes the exact solution of global configuration selection feasible since only the cartesian product of the different buckets need to be considered. This approach enables more successful program executions with overall higher quality outcomes.


An Adaptive Application Framework with Customizable Quality Metrics
L. Liu, S. Isaacman, and U.Kremer.
ACM Transactions on Design Automation of Electronic Systems (TODAES), Special Issue on Approximate Systems, Vol.27, No.2, Article 13, November 2021.

Global Cost/Quality Management across Multiple Applications
L. Liu, S. Isaacman, and U.Kremer.
ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE'20), Sacramento, CA, November 2020.

Trading Quality for Resource Consumption Through Approximation Management
Liu Liu
PhD Dissertation, Rutgers University, Department of Computer Science, January 2020.

Leveraging Structural Information to Ease Approximation Management
L. Liu, S. Isaacman, and U.Kremer.
Workshop on Approximate Computing Across the Stack (WAX), Williamsburg, VA, March 2018.

Exploiting Approximations for Energy/Quality Tradeoffs in Service-Based Applicaions
L. Liu, S. Isaacman, A. Bhattacharjee, and U.Kremer.
International Conference on Parallel Architectures and Compilation Techniques (PACT), poster paper, Portland, OR, September 2017.

Software Distributions and Resources

The software distribution and support consist of a tutorial to use the RAPID system, the RAPID library, and the RAPIDS and RAPIDM prototype implementations. Please note that the current RAPIDS implementation uses gurobi, a commercially available mixed integer programming tool. Most of the system is written in Python, with some parts in C++.

Tutorial and RAPIDS/M library

RAPIDS code base

RAPIDM code base

Last updated by Uli Kremer at 1:50pm on February 8, 2022