9,186 research outputs found
Truth After cinema: The explosion of facts in the documentary films of Jia Zhangke
No full textThis is the author's accepted manuscript. The final published article is available from the link below. Copyright @ 2013 Intellect Books.This article identifies and elaborates on two models of resistance evident in JiaZhangke’s film corpus. The deployment of different cinematic strategies produces an experimental calling into question of the value of truth and of truth as value. In the films here analysed Jia moves from resistance through organic observation to a model of resistance structured around a series of fabulations. If the first regime addresses the truth of ideology, then the target of the second is the ideology of truth. It is in this passage that Jia enters political cinema, collapsing the distinction between factual and fictional and opening up a space that belongs to no collectivity
Improving training and inference for embedded machine learning
No full textMany emerging applications are driving the development of Artificial Intelligence (AI) for embedded systems that require AI models to operate in resource constrained environments. Desirable characteristics of these models are reduced memory, computation and power requirements, that still deliver powerful performance. Deep learning has evolved as the state-of-the-art machine learning paradigm becoming more widespread due to its power in exploiting large datasets for inference. However, deep learning techniques are computationally and memory intensive, which may prevent them from being deployed effectively on embedded platforms with limited resources and power budgets. To address this problem, I focus on improving the efficiency of these algorithms. I show that improved compression and optimization algorithms can be applied to the deep learning framework from training through inference to meet this goal. This thesis introduces a new compression method that significantly reduces the number of parameters requirements of deep learning models by first-order optimization and sparsity-inducing regularization. This compression method can reduce model size by up to 300× without sacrificing prediction accuracy. To improve the performance of deep learning models, optimization techniques become more important, especially in large-scale applications. As a result, I develop two new first-order optimization algorithms that improve over existing methods by controlling the variance of the gradients, determining optimal batch sizes, scheduling adaptive learning rates, and balancing biased/unbiased estimations of the gradients, which can improve the convergence rate to provide a lower computational complexit
A variance controlled stochastic method with biased estimation for faster non-convex optimization
No full textThis paper proposed a new technique Variance Controlled Stochastic Gradient (VCSG) to improve the performance of the stochastic variance reduced gradient (SVRG) algorithm. To avoid over-reducing the variance of gradient by SVRG, a hyper-parameter λ is introduced in VCSG that is able to control the reduced variance of SVRG. Theory shows that the optimization method can converge by using an unbiased gradient estimator, but in practice, biased gradient estimation can allow more efficient convergence to the vicinity since an unbiased approach is computationally more expensive. λ also has the effect of balancing the trade-off between unbiased and biased estimations. Secondly, to minimize the number of full gradient calculations in SVRG, a variance-bounded batch is introduced to reduce the number of gradient calculations required in each iteration. For smooth non-convex functions, the proposed algorithm converges to an approximate first-order stationary point (i.e. E‖ ∇ f(x) ‖
2≤ ϵ ) within O(min{ 1 / ϵ
3 / 2, n
1 / 4/ ϵ} ) number of stochastic gradient evaluations, which improves the leading gradient complexity of stochastic gradient-based method SCSG (O(min{ 1 / ϵ
5 / 3, n
2 / 3/ ϵ} ) [19]. It is shown theoretically and experimentally that VCSG can be deployed to improve convergence.
</p
Sparse deep neural networks for embedded intelligence
No full textDeep learning is becoming more widespread due to its power in solving complex classification problems. However, deep learning models often require large memory and energy consumption, which may prevent them from being deployed effectively on embedded platforms, limiting their application. This work addresses the problem of memory requirements by proposing a regularization approach to compress the memory footprint of the models. It is shown that the sparsity-inducing regularization problem can be solved effectively using an enhanced stochastic variance reduced gradient optimization approach. Experimental evaluation of our approach shows that it can reduce the memory requirements both in the convolutional and fully connected layers by up to 300 without affecting overall test accuracy
Chen Bi-sheng & Yang Guo-zhen, Chen Jia-geng zhuan
No full textChing-Fatt Yong. Chen Bi-sheng & Yang Guo-zhen, Chen Jia-geng zhuan. In: Archipel, volume 27, 1984. pp. 201-202
A stochastic gradient method with biased estimation for faster nonconvex optimization
No full textA number of optimization approaches have been proposed for optimizing nonconvex objectives (e.g. deep learning models), such as batch gradient descent, stochastic gradient descent and stochastic variance reduced gradient descent. Theory shows these optimization methods can converge by using an unbiased gradient estimator. However, in practice biased gradient estimation can allow more efficient convergence to the vicinity since an unbiased approach is computationally more expensive. To produce fast convergence there are two trade-offs of these optimization strategies which are between stochastic/batch, and between biased/unbiased. This paper proposes an integrated approach which can control the nature of the stochastic element in the optimizer and can balance the trade-off of estimator between the biased and unbiased by using a hyper-parameter. It is shown theoretically and experimentally that this hyper-parameter can be configured to provide an effective balance to improve the convergence rate
Chen Bi-sheng & Yang Guo-zhen, Chen Jia-geng zhuan
No full textChing-Fatt Yong. Chen Bi-sheng & Yang Guo-zhen, Chen Jia-geng zhuan. In: Archipel, volume 27, 1984. pp. 201-202
Dataset for "GhostShiftAddNet: More Features from Energy-Efficient Operations"
No full textThis dataset supports the publication: GhostShiftAddNet: More Features from Energy-Efficient Operations.' in 'British Machine Vision Conference 2021'.</span
GhostShiftAddNet: More Features from Energy-Efficient Operations
No full textDeep convolutional neural networks (CNNs) are computationally and memory intensive. In CNNs, intensive multiplication can have resource implications that may challenge the ability for effective deployment of inference on resource-constrained edge devices. This paper proposes GhostShiftAddNet, where the motivation is to implement a hardware-efficient deep network: a multiplication-free CNN with less redundant features. We introduce a new bottleneck block, GhostSA, that converts all multiplications in the block to cheap operations. The bottleneck uses an appropriate number of bit-shift filters to process intrinsic feature maps, then applies a series of transformations that consist of bit-shifts with addition operations to generate more feature maps that fully learn information underlying intrinsic features. We schedule the number of bit-shift and addition operations for different hardware platforms. We conduct extensive experiments and ablation studies with desktop and embedded (Jetson Nano) devices for implementation and measurements. We demonstrate the proposed GhostSA block can replace bottleneck blocks in the backbone of state-of-the-art networks architectures and gives improved performance on image classification benchmarks. Further, our GhostShiftAddNet can achieve higher classification accuracy by using fewer FLOPs and parameters (reduced by up to 3x) than GhostNet. When compared to GhostNet, inference latency on the Jetson Nano is improved by about 1.3x and 2x on GPU and CPU respectively
Bioanalysis Young Investigator: Jia Li
No full text Supervisor’s supporting comments I supervised Jia Li during her PhD research on the metabolic characterization of host–parasite interactions. Jia excelled at the analytical chemistry components of her thesis and specialized in the integration of multiplatform data (NMR, UPLC–MS, CE–MS and 454 data). Since obtaining her PhD (2009), she has achieved a prestigious Imperial College Junior Research Fellowship and has focused on applying data integration techniques to investigate mechanisms of bariatric surgery. To date she has 16 publications in journals including Proceedings of the National Academy of Sciences, Molecular Systems Biology and Gut, many as first author. Her achievements include the characterization of multicompartmental changes in Schistosoma mansoni infected mice and the metabolic characterization of bariatric surgery in rats. She used NMR spectroscopic analysis, hierarchical principal component analysis and partial least square discriminant analysis to create systems level models of parasitic infection, and subsequently developed a CE method to validate the biomarkers. For the bariatric surgery model she applied a combination of UPLC–MS and NMR methods for profiling biofluids and related them to changes in the fecal microbiome profiled using 454 sequencing. She has presented her work at international conferences and has developed a network of external collaborations. Jia is an excellent role model; she is genuinely innovative and motivated, and has a flair for analytical technology. </jats:p
- …
