TU Wien Research Data
Not a member yet
512 research outputs found
Sort by
Proof of Concept: JUNON Digital Twin (v1.3)
No full text<h2>Proof of Concept: JUNON Digital Twin (v1.3)</h2><h3>Context and methodology</h3><p><a href="https://www.lestudium-ias.com/programmes/ambition-research-development-programmes-ard-cvl/ard-cvl-junon-programme">JUNON</a> is an ambitious research programme to develop a digital research cluster on the continental environment (agricultural, urban, forestry and river) in the Centre-Val de Loire region, France. This cluster aims to design digital services to improve the monitoring and understanding of the environment, for better management of natural resources.</p><p>This repository contains a proof of concept (PoC) of the JUNON Digital Twin. It tests some planned parts of DT, data exchange protocols and communication processes. This PoC is implemented in a single machine that emulates a DATA network. External repositories (<a href="https://confluence.ecmwf.int/display/CKB/ERA5">ERA5</a>, <a href="https://hubeau.eaufrance.fr/page/api-piezometrie">Hubeau-ADEs</a> and <a href="https://python-visualization.github.io/folium/latest/#">Folium</a>) are connected through official APIs. The developed PoC serves a use case in which the DT collects and manipulates data related to 255 piezometric sensors located in different regions of France. They collect data of aquifer water levels (<a href="https://hubeau.eaufrance.fr/page/api-piezometrie">ADEs</a>). Additionally, this information is combined with measurements and estimations concerning potential evaporation, total precipitation, and temperature at 2m for the same geographic locations (<a href="https://confluence.ecmwf.int/display/CKB/ERA5">ERA5</a>). <a href="https://python-visualization.github.io/folium/latest/#">Folium</a> repositories are accessed to obtain interactive geographic maps.</p><p>The DT offers the visualization of monthly averaged time series from 2000 to 2020 and, additionally, it is able to run clustering and anomaly detection analysis over the 255 mesurement points (each of them represented by a multi-variate time series composed by the four collected variables). Applied advanced machine learning methids include: <a href="https://classix.readthedocs.io/en/stable/index.html">CLASSIX</a>, <a href="https://pypi.org/project/sdoclust/0.2/">SDOclust</a>, and <a href="https://tslearn.readthedocs.io/en/stable/user_guide/clustering.html">Time Series kMeans</a> with Soft and Barycenter Averaging Dynamic Time Warping metrics. Users access the PoC through a Dashboard.</p><h3>Technical details</h3><ul><li>[assets], folder to save downloaded resources used by the dashboard.</li><li>[database], stores the SQlite database.</li><li>[misc] contains metadata related to geolocated points of measurement (BSS codes).</li><li>[screenshots] contains some screenshots of the dashbooard after clustering data with the different implemented methods.</li><li>[utils] contains additional scripts related to the database, API access and clustering libraries.</li><li>"app.py" runs the dashboard.</li><li>"download_background_data.py" runs the process that downloads raw data from official servers.</li><li>"README.md" contains the documentation.</li><li>"setup.py" sets up the environment for building the database.</li></ul><p>All data files are licensed under CC BY 4.0, all software is licensed under MIT License.</p>
Behavioral Correlates of Environmental Familiarity (Metadata)
No full text<h2>Behavioral Correlates of Environmental Familiarity in Pedestrian Wayfinders</h2><p>This repository contains all the data collected during my MSCA Global Fellowship (duration: 3/2022-8/2024).</p><h3>Context and methodology</h3><ul><li>This research belongs to the domain of GIScience.</li><li>The dataset will contain online study, in-situ study, and systematic review data.</li></ul><h3>Technical details</h3><ul><li>The dataset is structured in a way such that the current record is used for metadata only (in Computer Science terms one can think of it as a root node in a graph).</li></ul><h3> </h3>
Numerical Constraints for the Wigner and the Sigma Equation (FWF Project P 33151)
No full text<h2>Wigner equation in quantum transport</h2>
<p>This dataset was produced at the Institute for Microelectronics, TU Wien, with funding from the Austrian Science Fund FWF. It contains the research data and meta data used to generate the figures in two journal articles and three conference papers.</p>
<h3>Context and methodology</h3>
<ul>
<li>The dataset was created in the course of FWF project P33151.</li>
<li>The dataset contains supplementary data to the publications listed below.</li>
<li>Data are the result of numerical simulations based on the Wigner equation, the von Neumann equation, and the Schroedinger equation.</li>
</ul>
<h3>Technical details</h3>
<ul>
<li>A folder exists for each publication</li>
<li>Each folder contains human-readable data in CSV format and a README file containing information about the actual publication and how to interpret the data.</li>
</ul>
Indoor MIMO Channel Measurements of the Near-Field to Far-Field Transition in FR3
No full text<h1>Overview</h1>
<p>This dataset provides detailed measurements of wireless channel sounding in the 6 GHz to 24 GHz frequency range, focused on the near-field to far-field transition in MIMO systems. the measurements were performed using virtual uniform linear arrays (ULAs) for Tx and Rx. All channel traces, including the ones for the noise estimation, are S21 parameters over frequency and were recorded using a vector network analyzer.</p>
<p>The dataset includes the following components:</p>
<ol>
<li><strong>channel.json.gz</strong>: The captured channel data in JSON format and compressed using GZIP.</li>
<li><strong>noise.json.gz</strong>: A noise meaurement to judge system performance in the same format as the channel data, also a GZIP compressed JSON file.</li>
<li><strong>demo.py</strong>: A Python script demonstrating how to interact with the dataset for analysis.</li>
<li><strong>README.md</strong>: The readme file.</li>
</ol>
<h1>Dataset Components</h1>
<p>The two main files of the dataset are the channel data file and the noise data file.<br>Both are GZIP compressed, UTF-8/ASCII encoded, JSON files that are structured as key/value pairs.<br>The values are always arrays with at least one dimension.</p>
<p>The channel traces in both files are complex numbers, split into two real-valued arrays in the JSON file, where the last two dimensions are MxN channel matrices.</p>
<h2>Channel Data File: channel.json.gz</h2>
<h3>Dimensions</h3>
<p>A list of shorthands for the dimensions of the arrays appearing in the channel data file.</p>
<ul>
<li><strong>P / 6</strong>: Rx positions, higher values are closer to the Tx</li>
<li><strong>A / 9</strong>: Far-field factors, 0.1 - 10, higher values are more in the far-field</li>
<li><strong>L / 6001</strong>: Frequency points</li>
<li><strong>M / 9</strong>: MIMO channel Rx antennas/ports</li>
<li><strong>N / 9</strong>: MIMO channel Tx antennas/ports</li>
<li><strong>3</strong>: The cartesian coordinates of 3D space</li>
</ul>
<h3>JSON - Key/Value pairs</h3>
<p>A list of <strong>key [value dimensions]</strong> for the entries in the channel data file.</p>
<ul>
<li><strong>frequency [L]</strong>: The frequency points in Hz</li>
<li><strong>far_field_factor [A]</strong>: The far-field factors</li>
<li><strong>room_size [3]</strong>: The bounding box size of the measurement environment in m</li>
<li><strong>position_rx [P, A, M, 3]</strong>: The Rx antenna positions in m and room coordinates</li>
<li><strong>rotation_rx [3, 3]</strong>: The rotation matrix for the transformation from room coordinates to Rx antenna local coordinates</li>
<li><strong>position_tx [P, A, N, 3]</strong>: The Tx antenna positions in m and room coordinates</li>
<li><strong>rotation_tx [3, 3]</strong>: The rotation matrix for the transformation from room coordinates to Tx antenna local coordinates</li>
<li><strong>channel_real [P, A, L, M, N]</strong>: The real part of the channel transfer matrices</li>
<li><strong>channel_imag [P, A, L, M, N]</strong>: The imaginary part of the channel transfer matrices.</li>
</ul>
<h2>Noise Data File: noise.json.gz</h2>
<h3>Dimensions</h3>
<p>A list of shorthands for the dimensions of the arrays appearing in the noise data file.</p>
<ul>
<li><strong>R / 200</strong>: Measurement repetitions for statistical evaluation</li>
<li><strong>L / 6001</strong>: Frequency points</li>
<li><strong>M / 3</strong>: MIMO channel Rx antennas/ports</li>
<li><strong>N / 3</strong>: MIMO channel Tx antennas/ports</li>
<li><strong>3</strong>: The cartesian coordinates of 3D space</li>
</ul>
<h3>JSON - Key/Value pairs</h3>
<p>A list of <strong>key [value dimensions]</strong> for the entries in the noise data file.</p>
<ul>
<li><strong>frequency [L]</strong>: The frequency points in Hz</li>
<li><strong>room_size [3]</strong>: The bounding box size of the measurement environment in m</li>
<li><strong>position_rx [M, 3]</strong>: The Rx antenna positions in m and room coordinates</li>
<li><strong>rotation_rx [3, 3]</strong>: The rotation matrix for the transformation from room coordinates to Rx antenna local coordinates</li>
<li><strong>position_tx [N, 3]</strong>: The Tx antenna positions in m and room coordinates</li>
<li><strong>rotation_tx [3, 3]</strong>: The rotation matrix for the transformation from room coordinates to Tx antenna local coordinates</li>
<li><strong>channel_real [R, L, M, N]</strong>: The real part of the channel transfer matrices</li>
<li><strong>channel_imag [R, L, M, N]</strong>: The imaginary part of the channel transfer matrices.</li>
</ul>
<h2>Demo Code: demo.py</h2>
<p>The demo code is a Python script designed to demonstrate how to load, process, and analyze the channel data.</p>
<h3>Capabilities</h3>
<ul>
<li>Loading channel data and noise data from the GZIP compressed JSON files</li>
<li>Plot frequency- and delay-domain channel traces</li>
<li>Plot an analysis of the near-field to far-field transition</li>
<li>Plot the positions and orientations of Tx and Rx antennas</li>
<li>Plot the SNR over frequency, estimated from the noise data<br><br></li>
</ul>
<h3>Requirements</h3>
<p>To run the demo code, the following Python setup is required:</p>
<p><code>python >= 3.12</code><br><code>numpy >= 2.0.0</code><br><code>matplotlib >= 3.9</code></p>
<p>The code was tested with these packages, however, it likely also works with previous versions.</p>
<h1>Acknowledgment</h1>
<p>This work has been funded by the <strong>Christian Doppler Laboratory for Digital Twin assisted AI for sustainable Radio Access Networks</strong>.<br>The financial support by the Austrian Federal Ministry for Digital and Economic Affairs, the National Foundation for Research, Technology and Development and the Christian Doppler Research Association is gratefully acknowledged.</p>
<h1>License</h1>
<p>The data files <strong>channel.json.gz</strong>, <strong>noise.json.gz</strong> and <strong>README.md</strong> are licensed under CC BY 4.0. The code file <strong>demo.py</strong> is licensed under the MIT License.</p>
The Impact of Traffic Lights on Modal Split and Route Choice: A use-case in Vienna
No full text<p>The data and code scripts used for the analysis in the paper entitled "<strong>The Impact of Traffic Lights on Modal Split and Route Choice: A use-case in Vienna</strong>", submitted to AGILE (Association of Geographic Information Laboratories in Europe) 2024 Conference.</p><p>It comprises three folders within the zip file:</p><ol><li><strong>Data</strong>: Contains the datasets for the analysis.</li><li><strong>Code</strong>: Includes script files essential for conducting the analysis. The scripts are written in Python.</li><li><strong>Results</strong>: Includes the outcomes showcased in the associated paper.</li><li><strong>Visualizations</strong> : Includes a jupyter notebook for the generated plots in the the associated paper.</li></ol><p>Programming Language: Python </p><p>For reproducibility read the README.txt file included in the zip folder.</p><p>All data files are licensed under CC BY 4.0, all software is licensed under MIT License.</p><p>The transportation dynamics within a European city, Vienna, are examined using a multi-graph representation of the city's network. The focus is on time-optimized routing algorithms and the effects of altering the average waiting penalty at traffic lights. The impact of these modifications, whether an increase to 60, 90, or even 150 seconds or a decrease to 10 seconds, is observed in the selection of transportation modes and routes for identical origin and destination pairs. The investigation also extends to whether routes shift towards secondary street networks to avoid traffic lights as the waiting penalty increases. Experimental variations in average waiting time for cars aim to uncover detailed effects on transportation mode choices, route length and time changes, and variations in human energy expenditure. These findings could provide valuable insights into the transportation network and its possibilities and help in urban planning and policy development. The results indicate a shift in transportation mode as the waiting penalty for cars at traffic lights increases, and in some instances, routes are redirected to roads of lower importance such as residential or service roads.</p>
ESA CCI SM MODELFREE Long-term Climate Data Record of Surface Soil Moisture from merged multi-satellite observations
No full text<p>This dataset was produced with funding from the European Space Agency (ESA) Climate Change Initiative (CCI) Plus Soil Moisture Project (CCN 3 to ESRIN Contract No: 4000126684/19/I-NB "ESA CCI+ Phase 1 New R&D on CCI ECVS Soil Moisture"). Project website:<a href="https://climate.esa.int/en/projects/soil-moisture/"> https://climate.esa.int/en/projects/soil-moisture/</a></p>
<p>This dataset contains information on the Surface Soil Moisture (SM) content derived from satellite observations in the microwave domain.</p>
<h2>Abstract</h2>
<p>The MODELFREE product of the ESA CCI SM v9.1 science data suite provides - similar to the COMBINED product - global, harmonized daily satellite soil moisture measurements from both radar and radiometer observations. This product contains soil moisture estimates at 0.25-degree spatial resolution, and covers the period from 2002-2023. Soil moisture is derived from observations of 13 different active and passive satellites operating across various frequency bands (K, C, X, and L-band). Unlike the COMBINED product, for which soil moisture fields from the GLDAS Noah model dataset are used to harmonize individual satellite sensor measurements, the MODELFREE product utilizes a satellite-only scaling reference dataset (Madelon et al., 2022). This reference incorporates gap-filled soil moisture derived from AMSR-E (2002-2010) and from intercalibrated SMAP/SMOS brightness temperature data (2010-2023). The merging algorithm employed is consistent with that of the v9.1 COMBINED product. The new scaling reference leads to significantly different absolute soil moisture values, especially in latitudes above 60 °N. Data from the SMMR, SSMI and ERS missions are not included in this product.</p>
<p><strong>This product is in its early development stage and should be used with caution, as it may contain incomplete or unvalidated data.</strong></p>
<h3>Summary</h3>
<ul>
<li>
<p>First version of a model-independent version of the ESA CCI SM COMBINED product</p>
</li>
<li>
<p>2002-2023, global, 0.25 deg. resolution</p>
</li>
<li>
<p>GLDAS Noah (model) is replaced with a purely satellite-based scaling reference</p>
</li>
<li>
<p>Different absolute value range compared to the COMBINED product is expected due to the different scaling reference used</p>
</li>
</ul>
<h3>Known issues</h3>
<ul>
<li>
<p>A temporal inconsistency is observed between the AMSR-E and SMOS period (at 01-2010). This can affect long-term trends in the data</p>
</li>
<li>
<p>In the period from 01-2002 to 06-2002 no data are available above 37 °N and below 37 °S respectively (all measurements in this period are from the TRMM Microwave Imager)</p>
</li>
</ul>
<h2>Technical Details</h2>
<p>The dataset provides global daily estimates for the 2002-2023 period at 0.25° (~25 km) horizontal grid resolution. Daily images are grouped by year (YYYY), each subdirectory containing one netCDF image file for a specific day (DD), month (MM) in a 2-dimensional (longitude, latitude) grid system (CRS: WGS84). The file name has the following convention:</p>
<blockquote>
<p>ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED_MODELFREE-YYYYMMDD000000-fv09.1.nc</p>
</blockquote>
<p>Each netCDF file contains 3 coordinate variables (WGS84 longitude, latitude and time stamp), as well as the following data variables:</p>
<ul>
<li><strong>sm</strong>: (float) The Soil Moisture variable reflects estimates of daily average volumetric soil moisture content (m3/m3) in the soil surface layer (~0-5 cm) over a whole grid cell (0.25 degree).</li>
<li><strong>sm_uncertainty</strong>: (float) The Soil Moisture Uncertainty variable reflects the uncertainty (random error) of satellite observations. Derived using triple collocation analysis.</li>
<li><strong>dn_flag</strong>: (int) Indicator for satellite orbit(s) used in the retrieval (day/nighttime). 1=day, 2=night, 3=both</li>
<li><strong>flag</strong>: (int) Indicator for data quality / missing data indicator. For more details, see netcdf attributes.</li>
<li><strong>freqbandID</strong>: (int) Indicator for frequency band(s) used in the retrieval. For more details, see netcdf attributes.</li>
<li><strong>mode</strong>: (int) Indicator for satellite orbit(s) used in the retrieval (ascending, descending)</li>
<li><strong>sensor</strong>: (int) Indicator for satellite sensor(s) used in the retrieval. For more details, see netcdf attributes.</li>
<li><strong>t0</strong>: (float) Representative time stamp, based on overpass times of all merged satellites.</li>
</ul>
<p>Additional information for each variable is given in the netCDF attributes.</p>
<h3>Software to open netCDF files</h3>
<p>These data can be read by any software that supports Climate and Forecast (CF) conform metadata standards for netCDF files, such as:</p>
<ul>
<li><a title="xarray" href="https://github.com/pydata/xarray" target="_blank" rel="noopener">Xarray </a>(python)</li>
<li><a title="netCDF4" href="https://unidata.github.io/netcdf4-python/" target="_blank" rel="noopener">netCDF4 </a>(python)</li>
<li><a title="esa_cci_sm" href="https://github.com/TUW-GEO/esa_cci_sm">esa_cci_sm </a>(python)</li>
<li>Similar tools exists for other programming languages (Matlab, R, etc.)</li>
<li>Software packages and GIS tools can open netCDF files, e.g. <a href="https://code.mpimet.mpg.de/projects/cdo" target="_blank" rel="noopener">CDO</a>, <a href="http://nco.sourceforge.net/" target="_blank" rel="noopener">NCO</a>, <a href="https://www.qgis.org/" target="_blank" rel="noopener">QGIS</a>, ArCGIS</li>
<li>You can also use the GUI software <a href="https://www.giss.nasa.gov/tools/panoply/" target="_blank" rel="noopener">Panoply</a> to view the contents of each file</li>
</ul>
<h2>References</h2>
<p>R. Madelon et al., “Toward the Removal of Model Dependency in Soil Moisture Climate Data Records by Using an L-Band Scaling Reference," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 15, pp. 831-848, 2022, doi: <a href="https://ieeexplore.ieee.org/document/9657199">10.1109/JSTARS.2021.3137008</a>.</p>
<h2>Related Records</h2>
<p>The following records are all part of the <a href="../communities/soilmoisture-climaterecords/records?q=&l=list&p=1&s=10&sort=newest">Soil Moisture Climate Data Records from satellites</a> community</p>
<table>
<tbody>
<tr>
<td>1</td>
<td>
<p>ESA CCI SM RZSM Root-Zone Soil Moisture Record </p>
</td>
<td><a href="../doi/10.48436/v8cwj-jk556">10.48436/v8cwj-jk556</a></td>
</tr>
<tr>
<td>2</td>
<td>
<p>ESA CCI SM GAPFILLED Surface Soil Moisture Record </p>
</td>
<td><a href="../doi/10.48436/hcm6n-t4m35">10.48436/hcm6n-t4m35</a></td>
</tr>
</tbody>
</table>
<p> </p>
<p> </p>
openBIM permit process of the Vienna building authority
No full text<h2>Description</h2><p>The published data represents a BPMN model of the openBIM building permit process of the Vienna building authority.</p><h3>Context and methodology</h3><ul><li>The process was developed as part of the research project BRISE-Vienna.</li><li>The process was determined based on workshops and interviews with building authority officers and BIM experts followed by a pilot phase.</li></ul><h3>Technical details</h3><ul><li>BPMN 2.0 </li></ul>
Key Characteristics of Algorithms' Dynamics Beyond Accuracy - Evaluation Tests (v2)
No full text<h2>Key Characteristics of Algorithms' Dynamics Beyond Accuracy - Evaluation Tests (v2)</h2>
<p>conducted for the paper: <strong>What do anomaly scores actually mean? Key characteristics of algorithms' dynamics beyond accuracy</strong> by F. Iglesias, H. O. Marques, A. Zimek, T. Zseby</p>
<h3>Context and methodology</h3>
<p>Anomaly detection is intrinsic to a large number of data analysis applications today. Most of the algorithms used assign an outlierness score to each instance prior to establishing anomalies in a binary form. The experiments in this repository study how different algorithms generate different dynamics in the outlierness scores and react in very different ways to possible model perturbations that affect data.</p>
<p>The study elaborated in the referred paper presents new indices and coefficients to assess the dynamics and explores the responses of the algorithms as a function of variations in these indices, revealing key aspects of the interdependence between algorithms, data geometries and the ability to discriminate anomalies. Therefeore, this repository reproduces the conducted experiments, which study eight algorithms (ABOD, HBOS, iForest, K-NN, LOF, OCSVM, SDO and GLOSH), submitted to seven perturbations related to: cardinality, dimensionality, outlier proportion, inlier-outlier density ratio, density layers, clusters and local outliers, and collects behavioural profiles with eleven measurements (Adjusted Average Precission, ROC-AUC, Perini's Confidence [1], Perini's Stability [2], S-curves, Discriminant Power, Robust Coefficients of Variations for Inliers and Outliers, Coherence, Bias and Robustness) under two types of normalization: linear and Gaussian, the latter aiming to standardize the outlierness scores issued by different algorithms [3].</p>
<p>This repository is framed within the research on the following domains: <em>algorithm evaluation, outlier detection, anomaly detection, unsupervised learning, machine learning, data mining, data analysis. </em>Datasets and algorithms can be used for experiment replication and for further evaluation and comparison.</p>
<h3>References</h3>
<p>[1] Perini, L., Vercruyssen, V., Davis, J.: <em>Quantifying the confidence of anomaly detectors in their example-wise predictions</em>. In: The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases. Springer Verlag (2020).</p>
<p>[2] Perini, L., Galvin, C., Vercruyssen, V.: <em>A Ranking Stability Measure for Quantifying the Robustness of Anomaly Detection Methods</em>. In: 2nd Workshop on Evaluation and Experimental Design in Data Mining and Machine Learning @ ECML/PKDD (2020).</p>
<p>[3] Kriegel, H.-P., Kröger, P., Schubert, E., Zimek, A.: Interpreting and unifying outlier scores. In: Proceedings of the 2011 SIAM International Conference on Data Mining (SDM), pp. 13–24 (2011)</p>
<h3>Technical details</h3>
<p>Experiments are tested Python 3.9.6. Provided scripts generate all synthetic data and results. We keep them in the repo for the sake of comparability and replicability ("outputs.zip" file). The file and folder structure is as follows:</p>
<ul>
<li>"compare_scores_group.py" is a Python script to extract new dynamic indices proposed in the paper.</li>
<li>"generate_data.py" is a Python script to generate datasets used for evaluation.</li>
<li>"latex_table.py" is a Python script to show results in a latex-table format. </li>
<li>"merge_indices.py" is a Python script to merge accuracy and dynamic indices in the same table-structured summary.</li>
<li>"metric_corr.py" is a Python script to calculate correlation estimations between indices.</li>
<li>"outdet.py" is a Python script that runs outlier detection with different algorithms on diverse datasets.</li>
<li>"perini_tests.py" is a Python script to run Perini's confidence and stability on all datasets and algorithms' performances.</li>
<li>"scatterplots.py" is a Python script that generates scatter plots for comparing accuracy and dynamic performances.</li>
<li>"README.md" provides explanations and step by step instructions for replication.</li>
<li>"requirements.txt" contains references to required Python libraries and versions.</li>
<li>"outputs.zip" contains all result tables, plots and synthetic data generated with the scripts.</li>
<li>[data/real_data] contain CSV versions of the Wilt, Shuttle, Waveform and Cardiotocography datasets (inherited and adapted from the <a href="https://www.dbs.ifi.lmu.de/research/outlier-evaluation/DAMI/">LMU repository</a>)</li>
</ul>
<p> </p>
<h3>License</h3>
<p>The CC-BY license applies to all data generated with the "generated_data.py" script. All distributed code is under the GNU GPL license. For the "ExCeeD.py" and "stability.py" scripts, please consult and refer to the original sources provided above. </p>
Supplementary Material for "On Different Symbolic Music Representations for Algorithmic Composition Approaches based on Neural Sequence Models"
No full text<h2>Overview</h2>
<p>This repository contains additional resources for our paper</p>
<p>Felix Schön, Hans Tompits: <strong>On Different Symbolic Music Representations for Algorithmic Composition Approaches based on Neural Sequence Models.</strong> AIxIA 2024.</p>
<blockquote>
<p>Among the different approaches for automated music composition, those based on neural sequence models like the transformer show particular promise. A critical aspect for such approaches is how given music data sets are represented, or <em>tokenised</em>, for serving as suitable inputs for such models, as the choice of representation influences the quality of the produced output. In this paper, we introduce seven novel tokenisation techniques for converting MIDI data into numeric sequences. We compare characteristics of our tokenisers based on sets of musical data translated using our approaches. Our results show that some of our techniques greatly outperform the approaches found in the literature with respect to different metrics such as sequence length, information density, or memory requirements. Moreover, to evaluate the influence of our tokenisation approaches on the quality of the output of a model, we trained an ensemble of transformer models on the sets of tokenised musical data and performed a user study to assess the quality of the generated music pieces. The result of the study shows that the quality of pieces produced using our most promising techniques is equal to or outperforms state-of-the-art approaches.</p>
</blockquote>
<p>In this paper, we introduce and compare seven tokenisation techniques for symbolic music generation. Symbolic music generation refers to the process of composing music via some formal process. Here, music is represented using "tokens" rather than audio waveforms. The exact way we represent music using these tokens (referred to as a "tokenisation" process) can have a significant impact on a model's ability to learn and the quality of its output. To evaluate our tokenisation approaches, we trained an ensemble of seven Transformer models on musical data represented using our techniques. We then conducted a user study on the output of the models to compare the different approaches.</p>
<p>This repository contains both the code for the tokenisation approaches and the code for the models and their training loop. It furthermore contains the weights of the seven models trained on our tokenisation techniques, as well as two comparison models trained on approaches commonly used in the field. Lastly, we include the raw output of the user study we conducted and the samples used for it, both in the MIDI and .mp3 format.</p>
<h2>Structure</h2>
<p>This repository is organised as follows:</p>
<ul>
<li><code>./code.zip</code> contains the code used to train the Transformer models. Note that this directory functions as a snapshot of the code at the moment of submission of the paper and thus contains some aspects aimed at tasks not related to this publication that were still work-in-progress at the time of publishing. It serves for reproducibility and educational purposes.</li>
<li><code>./samples.zip</code> contains 64 pieces generated by each of the different models in the .mp3 and .midi format as well as the token representation of the pieces.</li>
<li><code>./survey.zip</code> contains the raw answers of all survey participants</li>
<li><code>./weights.zip</code> contains the compressed checkpoints of the trained models. Also contains logs of the training process.</li>
<li><code>./scoda.zip</code> contains the code for the different tokenisation approaches in the form of our MIDI processing library S-Coda v2.1. Note that S-Coda is still in active development and may change in the future. Up-to-date versions can be found at its <a href="https://github.com/FelixSchoen/S-Coda" target="_blank" rel="noopener">git repository</a>.</li>
<li><code>./requirements.txt</code> contains the dependencies and their versions compatible with this project.</li>
</ul>
<h2>Dependencies</h2>
<p>We used Python v3.11, PyTorch v2.5, and S-Coda v2.1 for this project. For more information, see both S-Coda's and the training code's <code>pyproject.toml</code>, which specify additional (minor) dependencies.</p>
<h2>License</h2>
<p>The code of both S-Coda and the training scripts is licensed under the MIT license, while the data is licensed under the Creative Commons Attribution 4.0 International license.</p>
<h2>Notes</h2>
<p>In order to run the project a <code>[code_root]/cfg/environment_config.json</code> file (where <code>[code_root]</code> marks the root folder of the code contained in <code>./code.zip</code>) is needed. This file points to the root folders used to load and store the datasets and models. An example file could look like this:</p>
<p><code>{</code><br><code>"paths": {</code><br><code> "datasets": "/path/to/datasets/",</code><br><code> "models": "/path/to/models/",</code><br><code> "wd_datasets": "/path/to/working/directory/of/datasets/"</code><br><code> }</code><br><code>}</code></p>
High-Energy Ball Milling Enables an Ultra-fast Wittig Olefination Under Ambient and Solvent-free Conditions
No full text<p><strong>Analytical Data and Compound Numbering (in paper numbering vs. ELN entries) for the Publication entitled:<br><em>"High-Energy Ball Milling Enables an Ultra-fast Wittig Olefination Under Ambient and Solvent-free Conditions"</em></strong></p>
<p>The paper was published on 2024-08-29 in Angewandte Chemie International Edition</p>
<p>Angew. Chem. Int. Ed. 2024, e202411536.</p>
<p>DOI: <a href="https://doi.org/10.1002/anie.202411536">10.1002/anie.202411536</a></p>
<p>Authors: Johanna Templ and Michael Schnürch</p>
<p>Funded by the Austrian Science Fund (FWF, project number P33064-N)</p>
<p><strong>Context and methodology</strong></p>
<p>30 Seconds to success! – The Wittig reaction, a fundamental and extensively utilized reaction in organic chemistry, enables the efficient conversion of carbonyl compounds to olefins using phosphonium salts. Traditionally, meticulous reaction setup, including the pre-formation of a reactive ylide species via deprotonation of a phosphonium salt, is crucial for achieving high-yielding reactions under classical solution-based conditions. In this report, we present an unprecedented protocol for an ultra-fast mechanically induced Wittig reaction under solvent-free and ambient conditions, often eliminating the need for tedious ylide pre-formation under strict air and moisture exclusion. A range of aldehydes and ketones were reacted with diverse phosphonium salts under high-energy ball milling conditions, frequently giving access to the respective olefins in only 30 seconds.</p>
<p>The publication and its Supporting Information can be found as open-access files on the publisher's website (see DOI above).</p>
<p>All detailed files containing the analytical raw data, for all compounds given in the Supporting Information of the manuscript are uploaded. An additional Word file named<strong><em> Angew. Chem. Int. Ed. 2024, e202411536_compound number list.pdf </em></strong>is uploaded, that should clearly link the compound number given in the paper to the respective entry in the ELN (jotempl) and the respective analytical data files. </p>
<p><strong>Technical details</strong></p>
<p>The files uploded contain the FIDs of NMR spectra recorded by an in-house Bruker Spectrometer. A software to display NMR-spectra is needed, such as <a href="https://mestrelab.com/download/mnova/">MestreNova</a> or <a href="https://www.bruker.com/en/products-and-solutions/mr/nmr-software/topspin.html">Topspin</a>).</p>