41 research outputs found
High-efficiency Coding for Shaking Surveillance Videos Based on Global Motion Compensation
No full textDue to the complex environment conditions, many surveillance videos are captured from cameras which are influenced by shaking more or less. This presents a significant challenge for background-modeling-based video coding since it is difficult to generate good background frames from such shaking videos. To solve this problem, this paper proposes a global motion compensation method using motion vectors (MV-GMC) for shaking surveillance video coding. In the proposed MV-GMC method, more accurate motion vectors (MVs) are extracted from HEVC encoder to estimate the global motion model in an efficient way, and we compensate each frame before background modeling. Then the compensated frames are used to model a good background frame for surveillance video coding. Compared with the optical-flow-based GMC (OPT-GMC) method which can be used to obtain more precise motion compensation, the proposed MV-GMC method has a comparable coding performance but a much lower computational complexity. Experiments on our surveillance video sequences show that the proposed MV-GMC method has significantly improved the coding performance by decreasing BD rate 49.83% over HM 12.0 on average while OPT-GMC can save 49.84% BD rate. The MVGMC method also saves 92.71% background modeling time compared with the OPT-GMC method.EICPCI-S(ISTP)[email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cellphone remote intelligent neuroregulation with self-powered wireless brain probe
No full textThe neural regulation that is tele-controlled remotely by medical professionals or artificial intelligence (AI) agents can meet the requirement of rapid, precise, personalized intervention and optimize the allocation of medical resources. Here, we develop a new self-powered wireless mini-invasive brain probe for remote intelligent neuroregulation. The probe can be wirelessly powered and controlled by cellphone audio tones (through piezoelectric effect) with the assistance of AI cellphone video analysis. The probe consists of a biocompatible pedestal integrated with implanting stimulation electrodes connected to embedded magnets and a magnetically coupled custom-designed signal-intensifying resonator integrated with a piezoelectric powered signal modulation circuit. A programmed audio tone functions as the wireless power source, and it can be tele-transmitted remotely from other cellphones with desired neural stimulation protocols. Combined with AI-enabled video monitoring of the epileptic tremor, we show that the probe can relieve the seizure events in the epileptic mice, and the therapeutic effect is confirmed by in-vivo electroencephalography and free-moving scenario. The multi-functionality of versatility, AI-assistance, wireless power transfer, and tele-transmission of the cellphone-interacted brain probe opens the possibility for remote precision neural modulation
Rate-Performance-Loss Optimization for Inter-Frame Deep Feature Coding From Videos
No full textWith the explosion in the use of cameras in mobile phones or video surveillance systems, it is impossible to transmit a large amount of videos captured from a wide area into a cloud for big data analysis and retrieval. Instead, a feasible solution is to extract and compress features from videos and then transmit the compact features to the cloud. Meanwhile, many recent studies also indicate that the features extracted from the deep convolutional neural networks will lead to high performance for various analysis and recognition tasks. However, how to compress video deep features meanwhile maintaining the analysis or retrieval performance still remains open. To address this problem, we propose a high-efficiency deep feature coding (DFC) framework in this paper. In the DFC framework, we define three types of features in a group-of-features (GOFs) according to their coding modes (i.e., I-feature, P-feature, and S-feature). We then design two prediction structures for these features in a GOF, including a sequential prediction structure and an adaptive prediction structure. Similar to video coding, it is important for P-feature residual coding optimization to make a tradeoff between feature bitrate and analysis/retrieval performance when encoding residuals. To do so, we propose a rate-performance-loss optimization model. To evaluate various feature coding methods for large-scale video retrieval, we construct a video feature coding data set, called VFC-1M, which consists of uncompressed videos from different scenarios captured from real-world surveillance cameras, with totally 1M visual objects. Extensive experiments show that the proposed DFC can significantly reduce the bitrate of deep features in the videos while maintaining the retrieval accuracy.National Basic Research Program of China [2015CB351806]; National Natural Science Foundation of China [U1611461, 61390515, 61425025]SCI(E)ARTICLE125743-57572
CNUSVM: Hybrid CNN-Uneven SVM Model for Imbalanced Visual Learning
No full textRecently, deep Convolutional Neural Networks (CNNs) have been used to achieve state-of-the-art performance on a wide range of visual learning tasks. However, when facing some imbalanced learning tasks where the training samples are unevenly distributed among different classes, CNNs tend to produce performance bias toward the majority class, making them not suitable for applications in which the recognition ability on the minority class is highly valued. To address the problem, this paper proposes a hybrid classification model by combining CNN with Support Vector Machine (SVM) that has uneven margins. In this model, CNN works as a feature extractor and the extracted features are then sent into a L2-SVM with linear uneven margins. We also develop a gradient-descent learning approach for this hybrid CNN-uneven SVM (CNUSVM) model by minimizing an uneven margin based L2-hinge loss. Our experiments on two benchmark datasets show that the CNUSVM model can make more favorable decisions for imbalanced visual learning tasks in comparison with the standard CNN and the hybrid CNN-SVM model.EICPCI-S(ISTP)[email protected]; [email protected]; [email protected]; [email protected]
Joint Learning of Semantic and Latent Attributes
No full textAs mid-level semantic properties shared across object categories, attributes have been studied extensively. Recent approaches have attempted joint modelling of multiple attributes together with class labels so as to exploit their correlations for better attribute prediction and object recognition. However, they often ignore the fact that there exist some shared properties other than nameable/semantic attributes, which we call latent attributes. Basically, they can be further divided into discriminative and non-discriminative parts depending on whether they can contribute to an object recognition task. We argue that learning the latent attributes jointly with user-defined semantic attributes not only leads to better representation for object recognition but also helps with semantic attribute prediction. A novel dictionary learning model is proposed which decomposes the dictionary space into three parts corresponding to semantic, latent discriminative and latent background attributes respectively. An efficient algorithm is then formulated to solve the resultant optimization problem. Extensive experiments show that the proposed attribute learning method produces state-of-the-art results on both attribute prediction and attribute-based person re-identification.CPCI-S(ISTP)[email protected]; [email protected]; [email protected]; [email protected]; [email protected]
CNN vs. SIFT for Image Retrieval: Alternative or Complementary?
No full textIn the past decade, SIFT is widely used in most vision tasks such as image retrieval. While in recent several years, deep convolutional neural networks (CNN) features achieve the state-of-the-art performance in several tasks such as image classification and object detection. Thus a natural question arises: for the image retrieval task, can CNN features substitute for SIFT? In this paper, we experimentally demonstrate that the two kinds of features are highly complementary. Following this fact, we propose an image representation model, complementary CNN and SIFT (CCS), to fuse CNN and SIFT in a multi-level and complementary way. In particular, it can be used to simultaneously describe scene level, object-level and point-level contents in images. Extensive experiments are conducted on four image retrieval benchmarks, and the experimental results show that our CCS achieves state-of-the-art retrieval results.CPCI-S(ISTP)[email protected]; [email protected]; [email protected]; [email protected]; [email protected]
MULTI-VIEW GAIT RECOGNITION WITH INCOMPLETE TRAINING DATA
No full textChanges in the viewing angles pose a major challenge for gait recognition because the human gait silhouettes can be different under the various viewing angles. Recently, View Transformation Model (VTM) was proposed to tackle this problem by transforming gait features from across views to a common viewing angle. However, VTM must use the data of subjects crossing all views to train the pre-constructed model, which might be unsuitable for the real applications. To address this problem, this paper proposes a View Feature Recovering Model (VFRM) to generate the VTM with incomplete training data. In our algorithm, if the gait signature of a pedestrian is missing under a view, it can be recovered from the K-nearest pedestrians whose gait features are available in the same view. Moreover, the Geodesic distance based K-Nearest Neighbor (GKNN) algorithm is adopted in our algorithm to better measure the neighborhood between two pedestrians. Experimental results on a benchmark database has demonstrated the effectiveness of our method.EICPCI-S(ISTP)[email protected]
CNN Based Vehicle Counting with Virtual Coil in Traffic Surveillance Video
No full textThis paper presents an efficient method of vehicle counting based on convolutional neural network (CNN) with virtual coils. Within virtual coils, foreground is obtained by background substraction. Vehicle is then detected by voting of virtual coil sub-regions. To deal with vehicle cross-lane cases, a cascade classifier combining connected component analysis (CCA) and CNN is adopted. Experiments are carried out on seven real traffic videos. The proposed approach works well on recognizing cross-lane vehicles, achieving above 90% accuracy with real-time processing speed.EICPCI-S(ISTP)[email protected]; [email protected]; [email protected]
Multi-camera Pedestrian Detection with a Multi-view Bayesian Network Model
No full textIn this paper, we propose a novel method with the multi-view Bayesian network (MBN) model to detect pedestrians from multi-camera surveillance videos. In our method, the ground plane is discretized in a predefined set of locations and our aim is to estimate the occupancy probability of each location that can be then used to predict the occurrence of pedestrians. To reduce the possible phantoms, we use MBN to model the potential occlusion relationship of all locations in all views, and the "subjective supposing" node states (SSNS) as a set of Boolean parameters of MBN to denote whether a pedestrian occurs at the corresponding location. Thus a learning algorithm is proposed to estimate the SSNS parameters, by finding such a configuration that the final occupancy possibility can best explain the image observations (i.e., foreground masks) from different views. The experimental results on the APIDIS and PETS09 S2L1 benchmark datasets show that our method can obtain at least 10% performance gain compared with several state-of-the-art algorithms.Computer Science, Artificial IntelligenceEICPCI-S(ISTP)
Sequential Deep Trajectory Descriptor for Action Recognition With Three-Stream CNN
No full textLearning the spatial-temporal representation of motion information is crucial to human action recognition. Nevertheless, most of the existing features or descriptors cannot capture motion information effectively, especially for long-term motion. To address this problem, this paper proposes a long-term motion descriptor called sequential deep trajectory descriptor (sDTD). Specifically, we project dense trajectories into two-dimensional planes, and subsequently a CNN-RNN network is employed to learn an effective representation for long-term motion. Unlike the popular two-stream ConvNets, the sDTD stream is introduced into a three-stream framework so as to identify actions from a video sequence. Consequently, this three-stream framework can simultaneously capture static spatial features, short-term motion, and long-term motion in the video. Extensive experiments were conducted on three challenging datasets: KTH, HMDB51, and UCF101. Experimental results show that our method achieves state-of-the-art performance on the KTH and UCF101 datasets, and is comparable to the state-of-the-art methods on the HMDB51 dataset.National Basic Research Program of China [2015CB351806]; National Natural Science Foundation of China [61390515, U1611461, 61425025, 61471042]; Beijing Municipal Commission of Science and Technology [Z151100000915070]; Shenzhen Peacock PlanSCI(E)ARTICLE71510-15201
