22 research outputs found
Study on Human Motion Forecasting using Self-Attention based Approach
No full text三重大学博士(工学)application/pdfHuman motion forecasting is a necessary variable to analyze human motion concerning the safety system of the autonomous system that could be used in many applications, such as in auto-driving vehicles, auto-pilot logistics delivery, and gait analysis in the medical field. At the same time, many types of research have been conducted on 2D and 3D human motion prediction for short and long-term goals. In this dissertation, human motion forecasting in the 2D plane has been conducted as a reliable alternative in motion capture of the RGB camera attached to the devices. While for a more precise location in the real-world automation application, 3D human motion forecasting is also necessary since the device could detect the exact location in the 3D plane. The unannotated dataset is used as the samples to conduct the works on 2D human motion forecasting to realize the usability of the task in real-world applications. On the unannotated dataset prediction task, the author proposed the feature extraction by OpenPose as the commonly used pose estimator and then obtained the future prediction movement by the RNN-LSTM or Kalman Filter. As a result, the usability of human motion prediction by applying the RGB camera is confirmed. The prediction results obtained by the Kalman Filter show better performance than the RNN-LSTM based on the correct prediction result within the correct location range.
In contrast, the annotated dataset is used to improve the quality and performance of the prediction results obtained by the models. The author proposed a method, the time series self-attention approach to generate the next future human motion in the short-term of 400 milliseconds and longterm of 1000 milliseconds, resulting that the model could predict human motion with a slight error of 23.51 pixels for short-term prediction and 10.3 pixels for longterm prediction on average compared to the ground truth in the quantitative and qualitative evaluation. Our method outperformed the LSTM and GRU models on the Human3.6M dataset based on the MPJPE and MPJVE metrics. The average loss of correct key points varied based on the tolerance value. Our method performed better within the 50 pixels tolerance. In addition, our method is tested by images without key point annotations using OpenPose as the pose estimation method. As a result, our method could predict well the position of the human but could not predict well for the human body pose. This research is a new baseline for the 2D human motion prediction using the Human3.6M dataset.
Subsequently, studies were carried out to predict human motion in 3D, aiming to improve various applications. Building upon the groundwork established by previous studies, the time series self-attention method was utilized as the model with modifications to accommodate 3D input data. As a result, our approach showed good performance in both short and longterm prediction tasks. It had an average error of 36.4mm between the prediction and ground truth in short-term predictions and 73.2mm in longterm predictions.
Overall, the studies of human motion forecasting have been conducted based on 2D and 3D input. In this study, we confirmed the realization of our method to predict human motion in the short and long term.本文/Graduate School of Engineering Mie University105pdoctoral thesi
Study on Human Motion Forecasting using Self-Attention based Approach
No full textapplication/pdfHuman motion forecasting is a necessary variable to analyze human motion concerning the safety system of the autonomous system that could be used in many applications, such as in auto-driving vehicles, auto-pilot logistics delivery, and gait analysis in the medical field. At the same time, many types of research have been conducted on 2D and 3D human motion prediction for short and long-term goals. In this dissertation, human motion forecasting in the 2D plane has been conducted as a reliable alternative in motion capture of the RGB camera attached to the devices. While for a more precise location in the real-world automation application, 3D human motion forecasting is also necessary since the device could detect the exact location in the 3D plane. The unannotated dataset is used as the samples to conduct the works on 2D human motion forecasting to realize the usability of the task in real-world applications. On the unannotated dataset prediction task, the author proposed the feature extraction by OpenPose as the commonly used pose estimator and then obtained the future prediction movement by the RNN-LSTM or Kalman Filter. As a result, the usability of human motion prediction by applying the RGB camera is confirmed. The prediction results obtained by the Kalman Filter show better performance than the RNN-LSTM based on the correct prediction result within the correct location range.
In contrast, the annotated dataset is used to improve the quality and performance of the prediction results obtained by the models. The author proposed a method, the time series self-attention approach to generate the next future human motion in the short-term of 400 milliseconds and longterm of 1000 milliseconds, resulting that the model could predict human motion with a slight error of 23.51 pixels for short-term prediction and 10.3 pixels for longterm prediction on average compared to the ground truth in the quantitative and qualitative evaluation. Our method outperformed the LSTM and GRU models on the Human3.6M dataset based on the MPJPE and MPJVE metrics. The average loss of correct key points varied based on the tolerance value. Our method performed better within the 50 pixels tolerance. In addition, our method is tested by images without key point annotations using OpenPose as the pose estimation method. As a result, our method could predict well the position of the human but could not predict well for the human body pose. This research is a new baseline for the 2D human motion prediction using the Human3.6M dataset.
Subsequently, studies were carried out to predict human motion in 3D, aiming to improve various applications. Building upon the groundwork established by previous studies, the time series self-attention method was utilized as the model with modifications to accommodate 3D input data. As a result, our approach showed good performance in both short and longterm prediction tasks. It had an average error of 36.4mm between the prediction and ground truth in short-term predictions and 73.2mm in longterm predictions.
Overall, the studies of human motion forecasting have been conducted based on 2D and 3D input. In this study, we confirmed the realization of our method to predict human motion in the short and long term.本文/Graduate School of Engineering Mie University105
Study on Human Motion Forecasting using Self-Attention based Approach
No full textapplication/pdf内容の要旨・審査結果の要旨/システム工学専
Human Motion Prediction Using 2D Person Pose Estimation on Unstable Data with RGB Camera
No full textapplication/pdfThe development of the autonomous system in the world is growing rapidly, it is becoming important to solve many problems. In case of human-machine interaction, machines such as an autonomous car or a robot that works in human living environment need to know human’s future motion for its moving trajectories. An autonomous system needs to know all possible human behaviors to estimate the future motion of human. However, our technology is still too far to remember all human behavior of movement which is unique by personality. Even though, in order to advance these trajectories, a research needs to be performed as soon as the technologies is growing itself.
Some of the previous works were using the Kinect RGB-D camera which has the depth sensor that could be used to provide the pose of human body.
This research uses the RGB camera as the other option that we can rely on. Currently, RGB-D camera is not widely available in many devices. We realize that the pose estimation which has been obtained from the RGB camera is not as precise as RGB-D camera yet. It is still reliable enough when we can optimize the data as an obstacle we need to get through. We propose the system to obtain the human body motion prediction by using regular digital RGB camera including a smartphone camera or even a surveillance camera. We set a goal to predict 1 second ahead of the motion, and 30 fps videos have been prepared which include simple motions such as hand gesture and walking movement.
We used OpenPose library from OpenCV to extract features of a human body pose including 14 points. Since OpenPose estimation is not always precise as we expected and to minimize the estimation error of the OpenPose we restricted the image area to perform human pose estimation using YOLOv3. We input distance and direction which are calculated from the features by comparing two consecutive frames into Recurrent Neural Network Long Short-Term Memory (RNN-LSTM) model and Kalman Filter. For the evaluation, we com-pare the result by the distance from the prediction result to the ground truth which is the position of the node after 1 second in the video and group the distance with value that are lower than 1.8% of the diagonal frame size, we called it the successful percentage of prediction. As the results, Kalman Filter reached 93% in average, and RNN-LSTM reached 75% in average on our dataset. While Kalman Filter reached 77% in average, and RNN-LSTM reached 52% in average on CMU dataset. Mostly, Kalman Filter show better estimate accuracy than RNN-LSTM and based on the human motions, motion such as hand gesture and moving to the right side are easier than more complex motion like hand gesture and moving to the left side. We confirmed the validity of RGB-camera based method in the simple human motion case from the result, and we conclude that this is an important step to realize the prediction of more complex human motion.Human Interface Laboratory Division of Information Engineering Graduate School of Emgineering Mie University42pthesi
Human Motion Prediction Using 2D Person Pose Estimation on Unstable Data with RGB Camera
No full textapplication/pdfThe development of the autonomous system in the world is growing rapidly, it is becoming important to solve many problems. In case of human-machine interaction, machines such as an autonomous car or a robot that works in human living environment need to know human’s future motion for its moving trajectories. An autonomous system needs to know all possible human behaviors to estimate the future motion of human. However, our technology is still too far to remember all human behavior of movement which is unique by personality. Even though, in order to advance these trajectories, a research needs to be performed as soon as the technologies is growing itself.
Some of the previous works were using the Kinect RGB-D camera which has the depth sensor that could be used to provide the pose of human body.
This research uses the RGB camera as the other option that we can rely on. Currently, RGB-D camera is not widely available in many devices. We realize that the pose estimation which has been obtained from the RGB camera is not as precise as RGB-D camera yet. It is still reliable enough when we can optimize the data as an obstacle we need to get through. We propose the system to obtain the human body motion prediction by using regular digital RGB camera including a smartphone camera or even a surveillance camera. We set a goal to predict 1 second ahead of the motion, and 30 fps videos have been prepared which include simple motions such as hand gesture and walking movement.
We used OpenPose library from OpenCV to extract features of a human body pose including 14 points. Since OpenPose estimation is not always precise as we expected and to minimize the estimation error of the OpenPose we restricted the image area to perform human pose estimation using YOLOv3. We input distance and direction which are calculated from the features by comparing two consecutive frames into Recurrent Neural Network Long Short-Term Memory (RNN-LSTM) model and Kalman Filter. For the evaluation, we com-pare the result by the distance from the prediction result to the ground truth which is the position of the node after 1 second in the video and group the distance with value that are lower than 1.8% of the diagonal frame size, we called it the successful percentage of prediction. As the results, Kalman Filter reached 93% in average, and RNN-LSTM reached 75% in average on our dataset. While Kalman Filter reached 77% in average, and RNN-LSTM reached 52% in average on CMU dataset. Mostly, Kalman Filter show better estimate accuracy than RNN-LSTM and based on the human motions, motion such as hand gesture and moving to the right side are easier than more complex motion like hand gesture and moving to the left side. We confirmed the validity of RGB-camera based method in the simple human motion case from the result, and we conclude that this is an important step to realize the prediction of more complex human motion.Human Interface Laboratory Division of Information Engineering Graduate School of Emgineering Mie University42
Study on Human Motion Forecasting using Self-Attention based Approach
No full text三重大学博士(工学)application/pdf内容の要旨・審査結果の要旨/システム工学専攻thesi
Improve Exercise Movement: Detecting Mistakes on Yoga with Mediapipe and MLP
No full textYoga is known as a comprehensive practice for maintaining physical and mental health. However, improper execution of yoga postures can cause injury, hinder progress, and potentially damage health. To overcome this problem, this research utilizes Mediapipe as a data preprocessing tool to identify yoga poses, which are then classified using the Multi-Layer Perceptron (MLP) algorithm. In the process, data normalization is carried out to increase prediction accuracy. This research uses a dataset consisting of six classes of yoga poses, namely tree, downdog, goddess, warrior, and plank. Experimental results show that the model achieved 98% accuracy during training, but accuracy during testing decreased to 95%. This shows an indication of overfitting, where the model adapts too much to the training data and is less able to generalize to the test data. This study makes an important contribution to the development of a safer and more accurate yoga pose classification system, which can be applied to practice yoga properly and prevent injuries.
Manuscript received: 15 Oct 2024 | Revised: 30 Jan 2025 | Accepted: 11 Feb 2025 | Published: 31 Mar 202
Human Fall Motion Prediction – A Review
No full textAbstract – In predicting human fall motion, focused on enhancing safety and quality of life for the elderly and individuals at risk of falls. By highlighting the critical role of Human Pose Estimation, advancements in human motion forecasting, and fall prediction. It explores the continuous efforts to improve fall detection systems using innovative technologies, such as wearable sensors and IoT devices to implement deep learning models and analyze human poses and gestures. Various methods show promise in accurately predicting human fall motion by capturing complex patterns and relationships in the data. For instance, self-attention mechanisms can revolutionize human motion prediction by effectively capturing these intricate patterns, leading to more accurate predictions. Future research directions should focus on enhancing model accuracy, exploring new techniques for capturing complex patterns, and enabling real-time implementation in wearable devices or smart environments. By addressing these areas, fall detection systems can be significantly improved, benefiting individuals and healthcare systems worldwide.
Manuscript received: 15 Apr 2024 | Revised: 20 June 2024 | Accepted: 12 July 2024 | Published: : 30 Sep 202
Rupiah Banknotes Detection Comparison of The Faster R-CNN Algorithm and YOLOv5
No full textMoney is an essential part of human life. Humans are never separated from activities related to money. As time goes by, money is not only a means of transactions between humans but also between humans and machines. Machines can recognize money in various ways, including object detection. Object detection is one of the most popular branches of computer vision. There are many methods for carrying out object detection, such as Faster R-CNN and YOLO. Faster R-CNN has been widely used in various fields to perform object detection tasks. Faster R-CNN has advantages over its predecessor because it uses a Region Proposal Network (RPN) as a substitute for selective search, which requires less compilation time. YOLO (You Only Look Once) is the most frequently used object detection method. This method divides the image into grids; each part of the grid predicts objects and their probabilities. The main advantages of YOLO are its high speed and ability to recognize objects in various conditions and positions with reasonably high accuracy. This research compares the Faster R-CNN algorithm model using the ResNet-50 architecture with YOLOv5 to recognize rupiah banknotes. The dataset used is 1120 images consisting of 8 classes. The YOLOv5 model trained on RGB data had the best results, with calculation accuracy reaching 1. Test results on three images also showed suitable results. The hope is that this research can be applied in other research to build a system for recognizing rupiah banknotes
Enhancing LLM Efficiency: A Literature Review of Emerging Prompt Optimization Strategies
No full textThis study focuses on enhancing the performance of Large Language Models (LLMs) through innovative prompt engineering techniques aimed at optimizing outputs without the high computational costs of model fine-tuning or retraining. The primary objective is to investigate efficient alternatives, such as black-box prompt optimization and ontology-based prompt refinement, which improve LLM performance by refining prompts externally while maintaining the model's internal parameters. The study explores various prompt optimization techniques, including instruction-based, role-based, question-answering, and contextual prompting, alongside advanced methods like CoT and ToT prompting. Methodologically, the research involves a comprehensive literature review, benchmarking prompt optimization techniques against existing models using standard datasets such as Big-Bench Hard and GSM8K. The study evaluates the performance of approaches like APE, PromptAgent, self-consistency prompting, and many more. The results demonstrate that these techniques significantly enhance LLM performance, particularly in tasks requiring complex reasoning, multi-step problem-solving, and domain-specific knowledge integration. The findings suggest that prompt engineering is crucial for improving LLM efficiency without excessive resource demands. However, challenges remain in ensuring prompt scalability, transferability, and generalization across different models and tasks. The study highlights the need for further research on integrating ontologies and automated prompt generation to refine LLM precision and adaptability, particularly in low-resource settings. These advancements will be vital for maximizing the utility of LLMs in increasingly complex and diverse applications.
Manuscript received: 3 Oct 2024 | Revised: 13 Dec 2024 | Accepted: 25 Dec 2024 | Published: 31 Mar 202
