1,721,163 research outputs found
GreenNAS: A Green Approach to the Hyperparameters Tuning in Deep Learning
Full text linkThis paper discusses the challenges of the hyperparameter tuning in deep learning models and proposes a green approach to the neural architecture search process that minimizes its environmental impact. The traditional approach of neural architecture search involves sweeping the entire space of possible architectures, which is computationally expensive and time-consuming. Recently, to address this issue, performance predictors have been proposed to estimate the performance of different architectures, thereby reducing the search space and speeding up the exploration process. The proposed approach aims to develop a performance predictor by training only a small percentage of the possible hyperparameter configurations. The suggested predictor can be queried to find the best configurations without training them on the dataset. Numerical examples of image denoising and classification enable us to evaluate the performance of the proposed approach in terms of performance and time complexity
Automatic Setting of Learning Rate and Mini-batch Size in Momentum and AdaM Stochastic Gradient Methods
No full textThe effectiveness of stochastic gradient methods strongly depends on a suitable selection of the hyperparameters which define them. Particularly, in the context of large-scale optimization problems often arising in machine learning applications, to properly fix both the learning rate and the mini-batch size in the definition of the stochastic directions is crucial for obtaining fast and efficient learning procedures. In a recent paper [1], the authors propose to define these hyperparameters by combining an adaptive subsampling strategy and a line search scheme. The aim of this work is to adapt this idea to both the stochastic gradient algorithm with momentum and the AdaM method in order to exploit the good numerical behaviour of the momentum-like stochastic gradient methods and the automatic technique to select the hyperparameters discussed in [1]. An extensive numerical experimentation carried out on convex functions, with different data sets, highlights that such combined hyperparameters technique makes the tuning of the hyperparameters computationally less expensive than the selection of suitable constant learning rate and mini-batch size and this is significant from the perspective of GreenAI. Furthermore, the proposed versions of the stochastic gradient method with momentum and AdaM have promising convergence behaviour compared to the original counterparts
Going Beyond Counting First Authors in Author Co-citation Analysis
Full text linkThe present study examines one of the fundamental aspects of author co-citation analysis (ACA) - the way co-citation
counts are defined. Co-citation counting provides the data on which all subsequent statistical analyses and mappings
are based, and we compare ACA results based on two different types of co-citation counting - the traditional type that
only counts the first one among a cited work's authors on the one hand and a non-traditional type that takes into
account the first 5 authors of a cited work on the other hand. Results indicate that the picture produced through this non-traditional author co-citation counting contains more coherent author groups and is therefore considerably clearer. However, this picture represents fewer specialties in the research field being studied than that produced through the traditional first-author co-citation counting when the same number of top-ranked authors is selected and analyzed. Reasons for these effects are discussed
Viscoelasticity of human descending thoracic aorta in a mock circulatory loop
No full textHealthy human descending thoracic aortas, obtained during organ donation for transplant and research, were tested in a mock circulatory loop to measure the mechanical response to physiological pulsatile pressure and flow. The viscoelastic properties of the aortic segments were investigated at three different pulse rates. The same aortic segments were also subjected to quasi-static pressure tests in order to identify the aortic dynamic stiffness ratio, which is defined as the ratio between the stiffness in case of pulsatile pressure and the stiffness measured for static pressurization, both at the same value of pressure. The loss factor was also identified. The shape of the deformed aorta under static and dynamic pressure was measured by image processing to verify the compatibility of the end supports with the natural deformation of the aorta in the human body. In addition, layer-specific experiments on 10 human descending thoracic aortas allowed to precisely identify the mass density of the aortic tissue, which is an important parameter in cardiovascular dynamic models
Organic additives in ceramic production : study of the thermal behaviour and identification of the pyrolysis products
No full textFor the production of ceramic tiles many additives are used.During the subsequent firing process these additives undergo to a pyrolysis whose products were studied by means of thermal analyses, GC_M
Ritz-like values in steplength selections for stochastic gradient methods
Full text linkThe steplength selection is a crucial issue for the effectiveness of the stochastic gradient methods for large-scale optimization problems arising in machine learning. In a recent paper, Bollapragada et al. (SIAM J Optim 28(4):3312–3343, 2018) propose to include an adaptive subsampling strategy into a stochastic gradient scheme, with the aim to assure the descent feature in expectation of the stochastic gradient directions. In this approach, theoretical convergence properties are preserved under the assumption that the positive steplength satisfies at any iteration a suitable bound depending on the inverse of the Lipschitz constant of the objective function gradient. In this paper, we propose to tailor for the stochastic gradient scheme the steplength selection adopted in the full-gradient method knows as limited memory steepest descent method. This strategy, based on the Ritz-like values of a suitable matrix, enables to give a local estimate of the inverse of the local Lipschitz parameter, without introducing line search techniques, while the possible increase in the size of the subsample used to compute the stochastic gradient enables to control the variance of this direction. An extensive numerical experimentation highlights that the new rule makes the tuning of the parameters less expensive than the trial procedure for the efficient selection of a constant step in standard and mini-batch stochastic gradient methods
Steplength and Mini-batch Size Selection in Stochastic Gradient Methods
No full textThe steplength selection is a crucial issue for the effectiveness of the stochastic gradient methods for large-scale optimization problems arising in machine learning. In a recent paper, Bollapragada et al. [1] propose to include an adaptive subsampling strategy into a stochastic gradient scheme. We propose to combine this approach with a selection rule for the steplength, borrowed from the full-gradient scheme known as Limited Memory Steepest Descent (LMSD) method [4] and suitably tailored to the stochastic framework. This strategy, based on the Ritz-like values of a suitable matrix, enables to give a local estimate of the local Lipschitz constant of the gradient of the objective function, without introducing line-search techniques, while the possible increase of the subsample size used to compute the stochastic gradient enables to control the variance of this direction. An extensive numerical experimentation for convex and non-convex loss functions highlights that the new rule makes the tuning of the parameters less expensive than the selection of a suitable constant steplength in standard and mini-batch stochastic gradient methods. The proposed procedure has also been compared with the Momentum and ADAM methods
Stochastic Floyd-Steinberg dithering on GPU: image quality and processing time improved
No full textError diffusion dithering is a technique that is used to represent a grey-scale image in a format usable by a printer. At every step, an algorithm converts the grey-scale value of a pixel to a new value within the allowed ones, generating a conversion error. To achieve the effect of continuous-tone illusion, the error is distributed to the neighboring pixels. Among the existent algorithms, the most commonly used is Floyd-Steinberg. However, this algorithm suffers two issues: artifacts and slowness. Regarding artifacts, those are textures that can appear after the image elaboration, making it visually different from the original one. In order to avoid this effect, we will use a stochastic version of Floyd-Steinberg algorithm. To evaluate the results, we will apply the Weighted Signal to Noise Ratio (WSNR), a visual-based model to account for perceptivity of dithered textures. This filter has a low-pass characteristic and, in particular, it uses a Contrast Sensitivity Function to evaluate the similarity between the original image and the final image. Our claim is that the new stochastic algorithm is better suited for both the WSNR measure and the visual analysis. Secondly, we will face slowness: we will describe a parallel version of Floyd-Steinberg algorithm that will exploit GPU (Graphics Processing Unit), drastically reducing the spent time. Specifically, we noticed that the serial version computational time increases quadratically with the input size, while the parallel version one increases linearly. Both the image quality and the computational performance of the parallel algorithm are evaluated on several large-scale images
Automatic stochastic dithering techniques on GPU: Image quality and processing time improved
Full text linkDithering or error diffusion is a technique used to obtain a binary image, suitable for printing, from a grayscale one. At each step, the algorithm computes an allowed value of a pixel from a grayscale one, applying a threshold and, therefore, causing a conversion error. To obtain the optical illusion of a continuous tone, the obtained error is distributed to adjacent pixels. In literature there are many algorithms of this type, to cite some Jarvis, Judice and Ninke (JJN), Stucki, Atkinson, Burkes, Sierra but the most known and used is the Floyd-Steinberg. We compared various types of dithering, which differ from each other for the weights and number of pixels involved in the error diffusion scheme. All these algorithms suffer from two problems: artifacts and slowness. First, we address the artifacts, which are undesired texture patterns generated by the dithering algorithm, leading to a less appealing visual results. To address this problem, we developed a stochastic version of Floyd-Steinberg's algorithm. The Weighted Signal to Noise Ratio (WSNR) is adopted to evaluate the outcome of the procedure, an error measure based on human visual perception that also takes into account artifacts. This measure behaves similarly to a low-pass filter and, in particular, exploits a contrast sensitivity function to compare the algorithm's result and the original image in terms of similarity. We will show that the new stochastic algorithm is better in terms of both WSNR measurement and visual analysis. Secondly, we address the method's inherent computational slowness: We implemented a parallel version of the Floyd-Steinberg algorithm that takes advantage of GPGPU (General Purtose Graphics Processing Unit) computing, drastically reducing the execution time. Specifically, we observed a quadratic time complexity with respect to the input size for the serial case, whereas the computational time required for our parallel implementation increased linearly. We then evaluated both image quality and the performance of the parallel algorithm on a exhaustive image database. Finally, to make the method fully automatic, an empirical technique is presented to choose the best degree of stochasticity
- …
