10 research outputs found
Kinect controlled chess playing robot
No full textTanberk, Senem (Dogus Author) -- Tükel, Dilek (Dogus Author) -- Conference full title: 17th International Conference on Smart Technologies; Ohrid; Macedonia; 6 July 2017 through 8 July 2017.In this project, we developed a chess play system based on Kinect vision sensor, which recognizes different hand gestures as a command and sends these commands to V-REP robotic simulation tool. Delta type robot will play chess according to the commands sent via vision software. In our system, we have tested for 2 different chess opening scenarios and DOF checking mode scenario, then we obtained results. Hence we propose an integration between Kinect hand gestures and V-REP simulation
İnsan aktivitesi tanımadan Robot Kavrama'ya derin öğrenme yöntemlerini kullanarak yeni bir yaklaşım
No full textFen Bilimleri Enstitüsü, Bilgisayar Mühendisliği Ana Bilim Dalı, Bilgisayar Mühendisliği Bilim DalıBu çalışmanın araştırma amacı, videolardaki insan hareketlerini otomatik olarak analiz etmek için ileri bir derin öğrenme modeli geliştirmek ve derin öğrenme destekli temel bir robotik taklit sisteminin prototipini sunmaktır. Bu amaçla, insan aktivite tanıma için hibrit bir derin model, insan taklidi için robotik simülatör altyapısı ve entegrasyon için basit, akıllı, genişletilebilir ve pluggable bir video analitik çerçevesi öneriyoruz. İlk olarak, videolardaki insan aktivitelerini tanıyacak yeni bir hibrit derin model sunuyoruz. Yoğun optik akış ve yardimci bilgilerin kaynastirildigi, 3D-CNN ve LSTM kombinasyonu ile gerçeklenen 3-akışlı yeni bir hibrit derin model oluşturduk. Hibrit derin modeldeki 3D-CNN'ler çoklu çerçeve ve yoğun optik akışla beslenir, LSTM ise yardımcı bilgilerle beslenir. Sınıflandırıcı olarak SVM kullandık. Manyetik duvar satranç tahtası video veri seti (MCDS) ve standart satranç tahtası video veri seti (CDS) olmak üzere 2 farklı veri seti oluşturduk. Bu veri setleri, satranç oyuncusu tarafından yapılan anlamlı hareketleri içeren 5-6 saniyelik mikrovideolardan oluşurlar. İki yeni veri seti ile hibrit derin modeli denedik, deneysel sonuçlar, teknoloji harikası diğer çalışmalara kıyasla dikkate değer bir performans gösterdi. Geliştirdiğimiz hibrit derin model, hareket ile görüntü, ses ve text gibi farklı özellikteki bilgileri aynı derin modelde kaynaştırabilme yeteneği sayesinde kompleks hareket tanımada kullanılabilir. İkinci olarak, satranç oynayan delta robot için robotik bir simülasyon altyapısı geliştirdik. Coppelia Robotics tarafından geliştirilen V-REP sanal robot deney platformunu kullandık. Geliştirilen robot simülatörü hem bağımsız olarak hem de dış bir sistem tarafından kontrol edilerek çalışabilir. Son olarak, insan hareket taklit sistemi için basit bir video analitik çerçevesi tasarladık ve hibrit derin öğrenme modelimizi bu çerçevede kullandık. Çevrimdışı modda belirli bir senaryo ile uçtan uca sistemi test ettik. Bu şekilde, satranç oyuncusunun robot simulatörü ile taklit edilmesi problemi özelinde kullanılacak bir yapay zeka destekli insan taklit sistemi prototipi geliştirmiş olduk. Önerilen sistemdeki robot simülatörü, satranç oyuncusunu hareket ilkel yaklaşımı ile taklit edebilir. Sonuç olarak, yapay zeka destekli, akıllı, genişletilebilir ve pluggable bir insan taklit sistem prototipi elde ettik.The research goal of this work is to develop an advanced deep learning model to analyse automatically human motion on videos and to present the prototype of a basic robotic imitation system that mimics the human movement supported by deep learning. For this purpose, we propose a hybrid deep model for human activity recognition, robotic simulator infrastructure for human imitation, and simple, intelligent, extensible and pluggable video analytic framework for integration. First, we present a new hybrid deep learning model for human activity recognition in videos. We proposed a new 3-stream hybrid deep model with data fusion of 3D-CNNs fed by dense optical flow and LSTM fed by auxiliary information. We used SVM as a classifier. We generated 2 different datasets, namely the magnetic wall chess board video dataset (MCDS), and standard chess board video dataset (CDS). They consist of microvideos with duration of 5-6 second that contain meaningful movements by the chess player. We experimented hybrid deep model with two new datasets, the experimental results show remarkable performance compared to the state-of-the-art studies. The proposed hybrid deep model can be used in complex motion recognition tasks, thanks to its ability to fuse information with different characteristics such as motion, image, sound and text. Second, we developed a robotic simulation infrastructure for chess-playing delta robot. We used V-REP virtual robot experiment platform by Coppelia Robotics. Generated robotic simulator can operate both standalone and by being controlled by an external system. Finally, we designed a simple video analytic framework for human motion imitation system and used our hybrid deep learning model in this framework. We tested end to end system with a specific scenario in offline mode. In this way, we have developed an artificial intelligence supported human imitation system prototype to be used in the specific problem of imitating chess player by robot simulator. The robot simulator in the proposed system can imitate the chess player with motion primitive approach. As a result, we achieved an AI-powered, intelligent, extensible, and pluggable human imitation system prototype
Key Frame Extraction with Attention Based Deep Neural Networks
No full textAutomatic keyframe detection from videos is an exercise in selecting scenes
that can best summarize the content for long videos. Providing a summary of the
video is an important task to facilitate quick browsing and content
summarization. The resulting photos are used for automated works (e.g.
summarizing security footage, detecting different scenes used in music clips)
in different industries. In addition, processing high-volume videos in advanced
machine learning methods also creates resource costs. Keyframes obtained; It
can be used as an input feature to the methods and models to be used. In this
study; We propose a deep learning-based approach for keyframe detection using a
deep auto-encoder model with an attention layer. The proposed method first
extracts the features from the video frames using the encoder part of the
autoencoder and applies segmentation using the k-means clustering algorithm to
group these features and similar frames together. Then, keyframes are selected
from each cluster by selecting the frames closest to the center of the
clusters. The method was evaluated on the TVSUM video dataset and achieved a
classification accuracy of 0.77, indicating a higher success rate than many
existing methods. The proposed method offers a promising solution for key frame
extraction in video analysis and can be applied to various applications such as
video summarization and video retrieval.Comment: in Turkish languag
Ensemble Learning with CNN-LSTM Combination for Speech Emotion Recognition
No full textInternational Conference on Computing and Communication Networks (ICCCN) -- NOV 19-20, 2021 -- Manchester Metropolitan Univ, Manchester, ENGLANDSpeech plays the most significant role in communication between people. The voice enables a speaker's unique characteristics to be mapped with biometric properties as well as carrying emotions. Emotion contains many non-linguistic signals to express ourselves as humans. Emotion recognition in human speech is a challenging task in different applications in fields such as healthcare, services, telecommunications, video conferencing, and human-computer interaction (HCI). Deep learning techniques are becoming a significant focus in recent research in the speech emotion recognition (SER) domain. In this paper, we present an ensemble learning approach based on various combinations of CNN and LSTM networks to address the limitations of the existing SER models. The proposed system is evaluated using the RAVDESS dataset. More specifically, the LSTM, CNN, and CNN and LSTM models achieved an accuracy rate of 0.64, 0.73, and 0.71, respectively. The simulation outcomes confirm that ensemble learning of the three deep model combinations contributes to the effectiveness of SER
The Design of a 3D Character Animation System for Digital Twins in the Metaverse
No full textIn the context of Industry 4.0, digital twin technology has emerged with rapid advancements as a powerful tool for visualizing and analyzing industrial assets. This technology has attracted considerable interest from researchers across diverse domains such as manufacturing, security, transportation, and gaming. The metaverse has emerged as a significant enabler in these domains, facilitating the integration of various technologies to create virtual replicas of physical assets. The utilization of 3D character animation, often referred to as avatars, is crucial for implementing the metaverse. Traditionally, costly motion capture technologies are employed for creating a realistic avatar system. To meet the needs of this evolving landscape, we have developed a modular framework tailored for asset digital twins as a more affordable alternative. This framework offers flexibility for the independent customization of individual system components. To validate our approach, we employ the English peg solitaire game as a use case, generating a solution tree using the breadth-first search algorithm. The results encompass both qualitative and quantitative findings of a data-driven 3D animation system utilizing motion primitives. The presented methodologies and infrastructure are adaptable and modular, making them applicable to asset digital twins across diverse business contexts. This case study lays the groundwork for pilot applications and can be tailored for education, health, or Industry 4.0 material development
Abstractive Text Summarization for Resumes With Cutting Edge NLP Transformers and LSTM
No full textText summarization is a fundamental task in natural language processing that
aims to condense large amounts of textual information into concise and coherent
summaries. With the exponential growth of content and the need to extract key
information efficiently, text summarization has gained significant attention in
recent years. In this study, LSTM and pre-trained T5, Pegasus, BART and
BART-Large model performances were evaluated on the open source dataset (Xsum,
CNN/Daily Mail, Amazon Fine Food Review and News Summary) and the prepared
resume dataset. This resume dataset consists of many information such as
language, education, experience, personal information, skills, and this data
includes 75 resumes. The primary objective of this research was to classify
resume text. Various techniques such as LSTM, pre-trained models, and
fine-tuned models were assessed using a dataset of resumes. The BART-Large
model fine-tuned with the resume dataset gave the best performance
Software Log Classification in Telecommunication Industry
No full textSoftware system admins depend on log data for understanding system beliavior, monitoring anomalies, tracking software bugs, and malfunctioning detection. Log analysis based on machine learning techniques enables to transform of raw logs into meaningful information that helps the DevOps team and administrators to solve problems. AI ensures to group similar logs together and keeps periodic logs more organized and sorted, allowing us to get to where we need to look faster. In this paper, we present a log classification system on log data generated by VoIP (Voice over Internet Protocol) soft-switch product. In this way, we targeted to detect the problem, direct it to the relevant department, allocate resources, and solve software bugs faster and more efficiently. Machine learning algorithms such as Linear Classifiers, Support Vector Machines, Decision Tree, Random Forest, Boosting, K-Nearest Neighbors, and Multilayer Perceptron are used for log classification
A hybrid deep model using deep learning and dense optical flow approaches for human activity recognition
No full textHuman activity recognition is a challenging problem with many applications including visual surveillance, human-computer interactions, autonomous driving and entertainment. In this study, we propose a hybrid deep model to understand and interpret videos focusing on human activity recognition. The proposed architecture is constructed combining dense optical flow approach and auxiliary movement information in video datasets using deep learning methodologies. To the best of our knowledge, this is the first study based on a novel combination of 3D-convolutional neural networks (3D-CNNs) fed by optical flow and long short-term memory networks (LSTM) fed by auxiliary information over video frames for the purpose of human activity recognition. The contributions of this paper are sixfold. First, a 3D-CNN, also called multiple frames is employed to determine the motion vectors. With the same purpose, the 3D-CNN is secondly used for dense optical flow, which is the distribution of apparent velocities of movement in captured imagery data in video frames. Third, the LSTM is employed as auxiliary information in video to recognize hand-tracking and objects. Fourth, the support vector machine algorithm is utilized for the task of classification of videos. Fifth, a wide range of comparative experiments are conducted on two newly generated chess datasets, namely the magnetic wall chess board video dataset (MCDS), and standard chess board video dataset (CDS) to demonstrate the contributions of the proposed study. Finally, the experimental results reveal that the proposed hybrid deep model exhibits remarkable performance compared to the state-of-the-art studies
