1,718 research outputs found
Data for: Scalability Model for the LOFAR Direction Independent Pipeline
No full textRun times of the LOFAR prefactor pipeline obtained by scaling the Number of CPUs, Data size and skymodel size. The prefactor version used for this data was https://github.com/apmechev/prefactor/commit/da4ac885bce9b24e604c9aac6bf649992065326
Orchestration of Distributed LOFAR Workflows
No full textThe LOFAR radio telescope produces petabytes of data every year. Radio Astronomers use complex multi-step pipelines to process this data and produce scientific images. In this thesis, we describe methods for astronomers to efficiently process massive amounts of data on high-throughput clusters. We gain insights into efficient acceleration of complex scientific pipelines, as well as understanding of the scalability of the underlying software. These insights ensure the scientific output of LOFAR keeps up with the pace of the data the telescope produces. Moreover, our results can be applied to arbitrary scientific studies, greatly increasing the potential of the LOFAR telescope. Finally, our discoveries can be applied to current and future telescopes. These lessons will help astronomers efficiently producing science in the big data era.Instrumentatio
LOFAR discovery of an ultra-steep radio halo and giant head-tail radio galaxy in Abell 1132
No full textLow-Frequency Array (LOFAR) observations at 144 MHz have revealed large-scale radio sources in the unrelaxed galaxy cluster Abell 1132. The cluster hosts diffuse radio emission on scales of similar to 650 kpc near the cluster centre and a head-tail (HT) radio galaxy, extending up to 1 Mpc, south of the cluster centre. The central diffuse radio emission is not seen in NRAO VLA FIRST Survey, Westerbork Northern Sky Survey, nor in C & D array VLA observations at 1.4 GHz, but is detected in our follow-up Giant Meterwave Radio Telescope (GMRT) observations at 325 MHz. Using LOFAR and GMRT data, we determine the spectral index of the central diffuse emission to be alpha = -1.75 +/- 0.19 (S alpha nu(alpha) ). We classify this emission as an ultra-steep spectrum radio halo and discuss the possible implications for the physical origin of radio haloes. The HT radio galaxy shows narrow, collimated emission extending up to 1 Mpc and another 300 kpc of more diffuse, disturbed emission, giving a full projected linear size of 1.3 Mpc - classifying it as a giant radio galaxy (GRG) and making it the longest HT found to date. The head of the GRG coincides with an elliptical galaxy (SDSS J105851.01+564308.5) belonging to Abell 1132. In our LOFAR image, there appears to be a connection between the radio halo and the GRG. The turbulence that may have produced the halo may have also affected the tail of the GRG. In turn, the GRG may have provided seed electrons for the radio halo
Building LOFAR as a Service
No full textThe LOFAR radio telescope is a low-frequency aperture synthesis radio telescope with headquarters in the Netherlands and stations across Europe. As a general purpose telescope, LOFAR produces petabytes of data each year serving a wide range of science cases. The data volumes produced are difficult or impossible to process on a single machine or even a small cluster at a scientific institute. We provide a layout for serving LOFAR processing to the astronomical community by providing access to LOFAR pipelines accelerated on a high throughput platform. We build this on our previous success with parallelizing the LOFAR Surveys pipeline and with creating automated LOFAR workflows on a distributed architecture. The LOFAR As A Service platform will serve the LOFAR Key Science Projects (KSPs), specifically the LOFAR Surveys KSP, which aims to provide science ready products to the scientific community. Additionally, this system will provide a robust method to re-process LOFAR data with a single click.Instrumentatio
Data for: Scalability Model for the LOFAR Direction Independent Pipeline
No full textRun times of the LOFAR prefactor pipeline obtained by scaling the Number of CPUs, Data size and skymodel size. The prefactor version used for this data was https://github.com/apmechev/prefactor/commit/da4ac885bce9b24e604c9aac6bf649992065326fTHIS DATASET IS ARCHIVED AT DANS/EASY, BUT NOT ACCESSIBLE HERE. TO VIEW A LIST OF FILES AND ACCESS THE FILES IN THIS DATASET CLICK ON THE DOI-LINK ABOV
Pipeline Collector: Gathering performance data for distributed astronomical pipelines
No full textModern astronomical data processing requires complex software pipelines to process ever growing datasets. For radio astronomy, these pipelines have become so large that they need to be distributed across a computational cluster. This makes it difficult to monitor the performance of each pipeline step. To gain insight into the performance of each step, a performance monitoring utility needs to be integrated with the pipeline execution. In this work we have developed such a utility and integrated it with the calibration pipeline of the Low Frequency Array, LOFAR, a leading radio telescope. We tested the tool by running the pipeline on several different compute platforms and collected the performance data. Based on this data, we make well informed recommendations on future hardware and software upgrades. The aim of these upgrades is to accelerate the slowest processing steps for this LOFAR pipeline. The pipeline_collector suite is open source and will be incorporated in future LOFAR pipelines to create a performance database for all LOFAR processing
LOFAR SKSP Software Image
No full text This Singularity image contains an install of LOFAR INSTALLDIR/aoflagger
* Casacore: INSTALLDIR/dysco
* LOFAR: INSTALLDIR/losoto
* PyBDSF: INSTALLDIR/RMextract
To execute a command, use
singularity exec -B
for example:
#singularity exec lofar.simg genericpipeline.py -d -c pipeline.cfg pipeline.parset
All packages located in /opt/lofar:
Available packages:
DPPP 4.1
LOFARBeam 2.8.12.2
RMextract 0.4
aoflagger 2.14.1 (2019-05-21)
armadillo 8.600.0
boost 1_6_30
casacore 2.2.1
cfitsio 3.41
cmake 2.8.12.2
ds9 8.0.1
dysco
fftw 3.3.8
hdf5 1.8.21
idg
lofar
losoto 2.0
lsmtool 1.4.1
make 3.82
openblas
pybdsf 1.8.15
python-casacore
superlu
wcslib
wsclean 2.7.2
Python Packages:
APLpy==1.1.1
astropy==2.0.14
atomicwrites==1.3.0
attrs==19.1.0
backports.functools-lru-cache==1.5
backports.ssl-match-hostname==3.5.0.1
boost==0.1
certifi==2019.6.16
chardet==3.0.4
cycler==0.10.0
decorator==4.4.0
funcsigs==1.0.2
h5py==2.9.0
idna==2.8
iniparse==0.4
ipaddress==1.0.16
IPy==0.75
kitchen==1.1.1
kiwisolver==1.1.0
linecache2==1.0.0
Mastodon.py==1.4.6
matplotlib==2.2.4
mimeparse==0.1.3
mock==3.0.5
more-itertools==5.0.0
nose==1.3.7
numexpr==2.7.0
numpy==1.16.4
pluggy==0.7.1
policycoreutils-default-encoding==0.1
pp==1.6.5
progressbar==2.5
py==1.8.0
pycurl==7.19.0
pygobject==3.22.0
pygpgme==0.3
pyliblzma==0.5.3
pyparsing==2.4.2
pytest==3.6.4
python-dateutil==2.8.0
python-magic==0.4.15
python-monetdb==11.19.3.2
pytz==2019.2
pyvo==0.9.3
pyxattr==0.5.1
requests==2.22.0
scipy==1.2.2
seobject==0.1
sepolicy==1.1
six==1.12.0
SQLAlchemy==1.3.7
subprocess32==3.5.4
tables==3.5.2
traceback2==1.4.0
unittest2==1.1.0
urlgrabber==3.10
urllib3==1.25.3
xmlrunner==1.7.7
yum-metadata-parser==1.1.4
</p
Ecological pyro-hydrometallurgical technology of nickel pyrrhotite concentrate treatment with non-traditional and reagentless recovery of sulphur dioxide from low concentrated gases
No full textLOFAR discovery of a radio halo in the high-redshift galaxy cluster PSZ2 G099.86+58.45
No full textInternational audienceIn this Letter, we report the discovery of a radio halo in the high-redshift galaxy cluster PSZ2 G099.86+58.45 (z = 0.616) with the LOw Frequency ARray (LOFAR) at 120–168 MHz. This is one of the most distant radio halos discovered so far. The diffuse emission extends over ∼1 Mpc and has a morphology similar to that of the X-ray emission as revealed by XMM-Newton data. The halo is very faint at higher frequencies and is barely detected by follow-up 1–2 GHz Karl G. Jansky Very Large Array observations, which enable us to constrain the radio spectral index to be α ≲ 1.5–1.6, i.e., with properties between canonical and ultra-steep spectrum radio halos. Radio halos are currently explained as synchrotron radiation from relativistic electrons that are re-accelerated in the intracluster medium by turbulence driven by energetic mergers. We show that in such a framework radio halos are expected to be relatively common at ∼150 MHz (∼30%–60%) in clusters with mass and redshift similar to PSZ2 G099.86+58.45; however, at least two-thirds of these radio halos should have a steep spectrum and thus be very faint above ∼1 GHz frequencies. Furthermore, because the luminosity of radio halos at high redshift depends strongly on the magnetic field strength in the hosting clusters, future LOFAR observations will also provide vital information on the origin and amplification of magnetic fields in galaxy clusters
Requirements of a beam loss monitoring system for the CLIC two beam modules
No full textThe Compact Linear Collider (CLIC) study investigates the feasibility of a high-energy electron-positron linear collider optimized for a centre of mass energy of 3 TeV. To achieve the high accelerating gradients, the RF power is produced by a novel two-beam acceleration method in which a decelerating drive beam supplies energy to the main accelerating beam. The linacs are arranged in modular structures referred to as the two beam modules which cover 42 km of beamline. Beam losses from either beam can have severe consequences due to the high intensity drive beam and the high energy, small emittance main beam. This paper presents recent developments towards the design of a Cherenkov fiber BLM system and discusses its ability to distinguish losses originating from either beam
- …
