DR-NTU (Data) (Nanyang Technological University)
DR-NTU (Data) (Nanyang Technological University)Not a member yet
1955 research outputs found
Sort by
Generalizable Implicit Motion Modeling for Video Frame Interpolation
No full textMotion modeling is critical in flow-based Video Frame Interpolation (VFI). Existing paradigms either consider linear combinations of bidirectional flows or directly predict bilateral flows for given timestamps without exploring favorable motion priors, thus lacking the capability of effectively modeling spatiotemporal dynamics in real-world videos. To address this limitation, in this study, we introduce Generalizable Implicit Motion Modeling (GIMM), a novel and effective approach to motion modeling for VFI. Specifically, to enable GIMM as an effective motion modeling paradigm, we design a motion encoding pipeline to model spatiotemporal motion latent from bidirectional flows extracted from pre-trained flow estimators, effectively representing input-specific motion priors. Then, we implicitly predict arbitrary-timestep optical flows within two adjacent input frames via an adaptive coordinate-based neural network, with spatiotemporal coordinates and motion latent as inputs. Our GIMM can be smoothly integrated with existing flow-based VFI works without further modifications. We show that GIMM performs better than the current state of the art on the VFI benchmarks
Octopus: Embodied Vision-Language Programmer from Environmental Feedback
No full textLarge vision-language models (VLMs) have achieved substantial progress in multimodal perception and reasoning. Furthermore, when seamlessly integrated into an embodied agent, it signifies a crucial stride towards the creation of autonomous and context-aware systems capable of formulating plans and executing commands with precision. In this paper, we introduce Octopus, an embodied VLM designed to 1) proficiently decipher an agent's visual and textual task objectives, 2) formulate intricate action sequences, and 3) generate executable code. Our design allows the agent to adeptly handle a wide spectrum of tasks, ranging from mundane daily chores in simulators to sophisticated interactions in complex video games. Octopus is trained by leveraging GPT-4 to control an explorative agent to generate training data, i.e., action blueprints and the corresponding executable code, within our experimental environment called OctoVerse. We also collect the feedback that allows the enhanced training scheme of Reinforcement Learning with Environmental Feedback (RLEF). Through a series of experiments, we illuminate Octopus's functionality and present compelling results, and the proposed RLEF turns out to refine the agent's decision-making. By open-sourcing our model architecture, simulator, and dataset, we aspire to ignite further innovation and foster collaborative applications within the broader embodied AI community
Parameterization-driven Neural Surface Reconstruction for Object-oriented Editing in Neural Rendering
No full textThe advancements in neural rendering have increased the need for techniques that enable intuitive editing of 3D objects represented as neural implicit surfaces. This paper introduces a novel neural algorithm for parameterizing neural implicit surfaces to simple parametric domains like spheres and polycubes. Our method allows users to specify the number of cubes in the parametric domain, learning a configuration that closely resembles the target 3D object's geometry. It computes bi-directional deformation between the object and the domain using a forward mapping from the object's zero level set and an inverse deformation for backward mapping. We ensure nearly bijective mapping with a cycle loss and optimize deformation smoothness. The parameterization quality, assessed by angle and area distortions, is guaranteed using a Laplacian regularizer and an optimized learned parametric domain. Our framework integrates with existing neural rendering pipelines, using multi-view images of a single object or multiple objects of similar geometries to reconstruct 3D geometry and compute texture maps automatically, eliminating the need for any prior information. We demonstrate the method's effectiveness on images of human heads and man-made objects
Appendix G: Exposure data
No full textExposure data (excel file) and metadata description for all hazards, thresholds, categories, and volcanoe
Replication Data for: Collisions of vortex rings with hemispheres
No full textThe data support the findings in the paper of "Collisions of vortex rings with hemispheres"
Replication Data for: Organic and inorganic sublattice coupling in two-dimensional lead halide perovskites
No full textReplication Data for paper entitled "Organic and inorganic sublattice coupling in two-dimensional lead halide perovskites" that will be published in Nature Communication
Replication Data for: AptaFluorescence: An aptamer-based fluorescent imaging protocol for biomolecule visualization
No full textAptaFluorescence and Immunofluorescence Image
Related Data for: Secondary spin current driven efficient THz spintronic emitters
No full textFemtosecond laser-induced photoexcitation of ferromagnet (FM)/heavy metal (HM) heterostructures has attracted attention by emitting broadband terahertz frequencies. The phenomenon relies on the formation of an ultrafast spin current, which is primarily attributed to the direct photoexcitation of the FM layer. However, during the process, the FM layer also experiences a secondary excitation led by the hot electrons from the HM layer that travel across the FM/HM interface and transfer additional energy in the FM. Thus, the generated secondary spins enhance the total spin current formation and lead to amplified spintronic terahertz emission. These results emphasize the significance of the secondary spin current, which even exceeds the primary spin currents when FM/HM heterostructures with thicker HM are used. An analytical model is developed to provide deeper insights into the microscopic processes within the individual layers, underlining the generalized ultrafast superdiffusive spin-transport mechanism
Related Data for: Effects of bevelled nozzles on standoff shocks in supersonic impinging jets
No full textSchlieren images of supersonic impinging jet
Replication Data for: Enhancing FAPbI3 perovskite solar cell performance with a methanesulfonate-based additive
No full textThis dataset contains the experimental data for the paper: "Enhancing FAPbI3 perovskite solar cell performance with a methanesulfonate-based additive