13 research outputs found
Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques
No full text
Deformity Removal from Handwritten Text Documents using Variable CycleGAN
Full text linkText recognition systems typically work well for printed documents but struggle with handwritten documents due to different writing styles, background complexities, added noise of image acquisition methods, and deformed text images such as strikeoffs and underlines. These deformities change the structural information, making it difficult to restore the deformed images while maintaining the structural information and preserving the semantic dependencies of the local pixels. Current adversarial networks are unable to preserve the structural and semantic dependencies as they focus on individual pixel-to-pixel variation and encourage non-meaningful aspects of the images. To address this, we propose a Variable Cycle Generative Adversarial Network (VCGAN) that considers the perceptual quality of the images. By using a variable Content Loss (Top-k Variable Loss (TVk) ), VCGAN preserves the inter-dependence of spatially close pixels while removing the strike-off strokes. The similarity of the images is computed with TVk considering the intensity variations that do not interfere with the semantic structures of the image. Our results show that VCGAN can remove most deformities with an elevated F1 score of 97.40% and outperforms current state-of-the-art algorithms with a character error rate of 7.64% and word accuracy of 81.53% when tested on the handwritten text recognition system.TRUEpu
Improved Decision Module Selection for Hierarchical Inference in Resource-Constrained Edge Devices
Full text linkThe Hierarchical Inference (HI) paradigm employs a tiered processing: the
inference from simple data samples are accepted at the end device, while
complex data samples are offloaded to the central servers. HI has recently
emerged as an effective method for balancing inference accuracy, data
processing, transmission throughput, and offloading cost. This approach proves
particularly efficient in scenarios involving resource-constrained edge
devices, such as IoT sensors and micro controller units (MCUs), tasked with
executing tinyML inference. Notably, it outperforms strategies such as local
inference execution, inference offloading to edge servers or cloud facilities,
and split inference (i.e., inference execution distributed between two
endpoints). Building upon the HI paradigm, this work explores different
techniques aimed at further optimizing inference task execution. We propose and
discuss three distinct HI approaches and evaluate their utility for image
classification
Improved Decision Module Selection for Hierarchical Inference in Resource-Constrained Edge Devices
Full text linkThe Hierarchical Inference (HI) paradigm has recently emerged as an effective method for balancing inference accuracy, data processing, transmission throughput, and offloading cost. This approach proves particularly efficient in scenarios involving resource-constrained edge devices like micro controller units (MCUs), tasked with executing tinyML inference. Notably, it outperforms strategies such as local inference execution, inference offloading, and split inference (i.e., inference execution distributed between two endpoints). Building upon the HI paradigm, this work explores different techniques aimed at further optimizing inference task execution. We propose three distinct HI approaches and evaluate their utility for image classification.Ministry of Economic Affairs and Digital Transformation, European Union Next Generation-EU, project TSI-063000- 2021-59, and through MSCA-PF projectTRUEpu
Utility of the revised Edmonton Symptom Assessment System (ESAS-r) and the Patient-Reported Functional Status (PRFS) in lung cancer patients
No full textCNN based Metrics for Performance Evaluation of Generative Adversarial Networks
Full text linkIn this work, we propose two Convolutional Neural Network (CNN) based metrics, Classification Score (CS) and Distribution Score (DS), for performance evaluation of Generative Adversarial Networks (GANs). Though GAN-generated images can be evaluated through manual assessment of visual fidelity, it is prolonged, subjective, challenging, tiresome, and can be misleading. Existing quantitative methods are biased towards memory GAN and fail to detect over-fitting. CS and DS allow us to experimentally prove that training of GANs is actually guided by the data set, that it improves with every epoch and gets closer to following the distribution of the data set. Both methods are based on GAN-generated image classification by CNN. CS is the root mean square (RMS) value of three different classification techniques, Direct Classification (DC), Indirect Classification (IC), and Blind Classification (BC). It exhibits the degree to which GAN can learn the features and generate fake images similar to real data sets. DS shows the contrast between the mean distribution of GAN-generated data and the real data. It indicates the extent to which GANs can create synthetic images with similar distribution to real data sets. We evaluated CS and DS metrics for different variants of GANs and compared their performances with existing metrics. Results show that CS and DS can evaluate the different variants of GANs quantitatively and qualitatively while detecting over-fitting and mode collapse.TRUEpu
Hierarchical Inference at the Edge: A Batch Processing Approach
No full textDeep learning (DL) applications have rapidly evolved to address increasingly complex tasks by leveraging large-scale, resource-intensive models. However, deploying such models on low-power devices is not practical or economically scalable. While cloud-centric solutions satisfy these computational demands, they present challenges in terms of communication costs and latencies for real-Time applications when every computation task is offloaded. To mitigate these concerns, hierarchical inference (HI) frameworks have been proposed, enabling edge devices equipped with small ML models to collaborate with edge servers by selectively offloading complex tasks. Existing HI approaches depend on immediate offloading of data upon selection, which can lead to inefficiencies due to frequent communication, especially in time-varying wireless environments. In this work, we introduce Batch HI, an approach that offloads samples in batches, thereby reducing communication overhead and improving system efficiency while achieving similar performance as existing HI methods. Additionally, we find the optimal batch size that attains a crucial balance between responsiveness and system time, tailored to specific user requirements. Numerical results confirm the effectiveness of our approach, highlighting the scenarios where batching is particularly beneficial.</p
The Case for Hierarchical Deep Learning Inference at the Network Edge
No full textResource-constrained Edge Devices (EDs), e.g., IoT sensors and microcontroller units, are expected to make intelligent decisions using Deep Learning (DL) inference at the edge of the network. Toward this end, developing tinyML models is an area of active research - DL models with reduced computation and memory storage requirements - that can be embedded on these devices. However, tinyML models have lower inference accuracy. On a different front, DNN partitioning and inference offloading techniques were studied for distributed DL inference between EDs and Edge Servers (ESs). In this paper, we explore Hierarchical Inference (HI), a novel approach proposed in [19] for performing distributed DL inference at the edge. Under HI, for each data sample, an ED first uses a local algorithm (e.g., a tinyML model) for inference. Depending on the application, if the inference provided by the local algorithm is incorrect or further assistance is required from large DL models on edge or cloud, only then the ED offloads the data sample. At the outset, HI seems infeasible as the ED, in general, cannot know if the local inference is sufficient or not. Nevertheless, we present the feasibility of implementing HI for image classification applications. We demonstrate its benefits using quantitative analysis and show that HI provides a better trade-off between offloading cost, throughput, and inference accuracy compared to alternate approaches.</p
