328 research outputs found
Message Passing Computational Methods with Pharmacometrics Applications
No full textA pharmaceutical company needs to invest in the costly and tightly regulated
multi-year drug development process early on. While many compounds are
considered initially, only a few make it to the final phase where the newly
developed drug is made available to the wide population. Time and effort is lost
on the majority of candidate compounds as these turn out to not be efficacious
and no return on investment is made. For this reason, pharmaceutical companies
are interested in methodologies and approaches to better detect the ones that have
more promise as soon as possible. In addition, the exclusivity granted by a patent
is limited in time, and speeding up the process translates into material financial
gains. Model Based Drug Development, a quantitative approach where gathered
data is leveraged in an online fashion to improve decision making, has been
suggested as one way to optimize the process. The domain of Pharmacometrics,
a key component of Model Based Drug Development, is concerned with modeling
human-compound interactions. One of the challenges in Pharmacometrics is that
the computational requirements of the models preclude agility and timeliness.
Currently, modelers switch between different projects to avoid stalls as typical
computational runs can take up to weeks, but arguably this hampers swift
modeling due to the long feedback cycle.
Recent computing systems have seen a surge in the number of explicitly
exposed parallel resources due to the limits being reached in single processor
systems. Leveraging the computational resources of these systems is far from
trivial. Pharmacometrics, like other branches of computational science, is a multidisciplinary field. Scientists that specialize in the models are rarely equipped
with the right Computer Science background to write efficient computational
codes. Therefore, the common approach is to rely on software packages that provide a toolbox of mathematical and statistical methods. However, contemporary
packages lack in efficiency when deployed on a parallel system. This motivates
the need for new approaches like those explored in this thesis. In the context of computational modeling, there are two prominent strategies
for parallelization. First, computations on models are fit using an iterative
optimization routine where multiple processors can be kept busy within each
iteration by evaluating multiple candidate parameters concurrently. This part
of the computation is referred to as the back-end. While this approach can
hide the parallel constructs within the routine improving the usability of these
routines for scientists from other domains, it requires the optimization routine
to be designed to run in parallel. However, this might not always be feasible.
Second, in the front-end a single candidate parameter can be evaluated in
parallel if permitted by the dependency structure of the model, a strategy suitable
both for more sequential optimization routines as well as parallel optimization
routines where it further improves performance. Even if a task can be decomposed
into smaller concurrently executable tasks, doing so manually is tedious, errorprone and requires the right parallel computing background. Arguably, the
scientist concerned with building these models is in an even worse position; their
expertise is probably not in parallel computing and more automated approaches
are preferable.
This thesis proposes novel ways to leverage parallelism in both the front-end
and the back-end. In the former, two approaches are presented to parallelize
evaluations without any input from the user. In the latter, changes to two
existing state-of-the-art Markov Chain Monte Carlo samplers are presented that
allow to better deal with large parallel systems in the message passing paradigm.
Improvements for samplers running with data-bound models are explored as
well.
One of the main properties of Pharmacometrics models is that computation
time required for parts of the model depends highly on the choice of model
parameters. For this reason, common approaches fail to perform well in this
regard as they assume a more uniform execution time across evaluations. By
neglecting this property, idle times are introduced resulting in poor use of
available resources.
Performance gains observed by the presented techniques vary greatly from 10%
all the way to many hundred fold reductions in execution time. It is important
to note that these improvements depend not only on the targeted algorithms,
but also on the computation models and on the platform. Nevertheless, the
ideas are presented at a reasonably abstract level to support generalizations to
other domains and computational models
Message Passing Computational Methods with Pharmacometrics Applications
No full textA pharmaceutical company needs to invest in the costly and tightly regulated
multi-year drug development process early on. While many compounds are
considered initially, only a few make it to the final phase where the newly
developed drug is made available to the wide population. Time and effort is lost
on the majority of candidate compounds as these turn out to not be efficacious
and no return on investment is made. For this reason, pharmaceutical companies
are interested in methodologies and approaches to better detect the ones that have
more promise as soon as possible. In addition, the exclusivity granted by a patent
is limited in time, and speeding up the process translates into material financial
gains. Model Based Drug Development, a quantitative approach where gathered
data is leveraged in an online fashion to improve decision making, has been
suggested as one way to optimize the process. The domain of Pharmacometrics,
a key component of Model Based Drug Development, is concerned with modeling
human-compound interactions. One of the challenges in Pharmacometrics is that
the computational requirements of the models preclude agility and timeliness.
Currently, modelers switch between different projects to avoid stalls as typical
computational runs can take up to weeks, but arguably this hampers swift
modeling due to the long feedback cycle.
Recent computing systems have seen a surge in the number of explicitly
exposed parallel resources due to the limits being reached in single processor
systems. Leveraging the computational resources of these systems is far from
trivial. Pharmacometrics, like other branches of computational science, is a multidisciplinary field. Scientists that specialize in the models are rarely equipped
with the right Computer Science background to write efficient computational
codes. Therefore, the common approach is to rely on software packages that provide a toolbox of mathematical and statistical methods. However, contemporary
packages lack in efficiency when deployed on a parallel system. This motivates
the need for new approaches like those explored in this thesis. In the context of computational modeling, there are two prominent strategies
for parallelization. First, computations on models are fit using an iterative
optimization routine where multiple processors can be kept busy within each
iteration by evaluating multiple candidate parameters concurrently. This part
of the computation is referred to as the back-end. While this approach can
hide the parallel constructs within the routine improving the usability of these
routines for scientists from other domains, it requires the optimization routine
to be designed to run in parallel. However, this might not always be feasible.
Second, in the front-end a single candidate parameter can be evaluated in
parallel if permitted by the dependency structure of the model, a strategy suitable
both for more sequential optimization routines as well as parallel optimization
routines where it further improves performance. Even if a task can be decomposed
into smaller concurrently executable tasks, doing so manually is tedious, errorprone and requires the right parallel computing background. Arguably, the
scientist concerned with building these models is in an even worse position; their
expertise is probably not in parallel computing and more automated approaches
are preferable.
This thesis proposes novel ways to leverage parallelism in both the front-end
and the back-end. In the former, two approaches are presented to parallelize
evaluations without any input from the user. In the latter, changes to two
existing state-of-the-art Markov Chain Monte Carlo samplers are presented that
allow to better deal with large parallel systems in the message passing paradigm.
Improvements for samplers running with data-bound models are explored as
well.
One of the main properties of Pharmacometrics models is that computation
time required for parts of the model depends highly on the choice of model
parameters. For this reason, common approaches fail to perform well in this
regard as they assume a more uniform execution time across evaluations. By
neglecting this property, idle times are introduced resulting in poor use of
available resources.
Performance gains observed by the presented techniques vary greatly from 10%
all the way to many hundred fold reductions in execution time. It is important
to note that these improvements depend not only on the targeted algorithms,
but also on the computation models and on the platform. Nevertheless, the
ideas are presented at a reasonably abstract level to support generalizations to
other domains and computational models
Improving Operational Intensity in Data Bound Markov Chain Monte Carlo
No full textTypically, parallel algorithms are developed to leverage the processing power of multiple processors simultaneously speeding up overall execution. At the same time, discrepancy between \{DRAM\} bandwidth and microprocessor speed hinders reaching peak performance. This paper explores how operational intensity improves by performing useful computation during otherwise stalled cycles. While the proposed methodology is applicable to a wide variety of parallel algorithms, and at different scales, the concepts are demonstrated in the machine learning context. Performance improvements are shown for Bayesian logistic regression with a Markov chain Monte Carlo sampler, either with multiple chains or with multiple proposals, on a dense data set two orders of magnitude larger than the last level cache on contemporary systems.Part of the work presented in this paper was funded by Johnson & Johnson. This project has received funding from the European Union’s Horizon 2020 Research and Innovation programme under Grant Agreement no. 671555
From Conditional Independence to Parallel Execution in Hierarchical Models
No full textHierarchical models describe phenomena by grouping data into multiple levels. Due to the size of these models, parallel execution is required to avoid prohibitively long computing time. While it is occasionally possible to specify some of these models using parallel building blocks, this limits expressivity. Therefore, a more general generative specification is preferred. To leverage parallel computing capacity, these specifications can be annotated, but doing so effectively assumes that the modeler has expertise from computer science. This paper outlines how to identify parallel parts automatically by leveraging the conditional independence property in the graphical model extracted from the dataflow graph of model specifications. Computation related to random variables with the same depth in the graphical model are identified as candidates for parallel execution. Since subsequent proposals in the parameter space exploration of the model are clustered together, the results show that the well known longest processing time scheduling heuristic deals adequately with load imbalance. The proposed parallelization is evaluated on two pharmacometrics models, a domain where hierarchical models with load imbalance are common due to the numeric simulation of pharmacokinet-ics and pharmacodynamics of human subjects. The varying number of measurements taken per subject further exacerbates load imbalance.Acknowledgments
Part of the work presented in this paper was funded by Johnson & Johnson
Automatic Parallelization of Probabilistic Models with Varying Load Imbalance
No full textAs scientists are designing increasingly complex and intricate models, the prominent way today to achieve acceptable execution time without sacrificing accuracy is through parallel computing. These techniques can improve execution time either on the level of the optimization methods or on the level of the model evaluations. This paper outlines an automatic par-allelization approach for the latter. Processor specific procedures with embedded communication primitives are generated from static schedules produced by an evolutionary algorithm. These are passed to an optimizing compiler to avoid the overhead of typical task runtime systems. The two key insights are that the parallel structure of probabilistic models is revealed when the data is combined with the model and that static schedules can be combined into more robust schedules that can deal with varying load imbalance. For this, LogP model parameters and execution time of each computational task are measured and fed into a discrete event simulator to estimate the running time on the target parallel system. Performance is evaluated with three pharmacological models with different characteristics. The first model lacks enough exploitable parallelism while up to approximately 6x and 8x improvements are achieved for the other models. Compared to a theoretical system with infinite processors and no communication delay, this equates to exploiting 66% and 99% of the available parallelism. Performance improves even when load imbalance varies
Relaxing Scalability Limits with Speculative Parallelism in Sequential Monte Carlo
No full textSequential Monte Carlo methods are a useful tool to tackle non-linear problems in a Bayesian setting. A target posterior distribution is approximated by moving a set of weighted particles through a sequence of distributions. To counteract degeneracy caused by sequentially changing the underlying distribution, particles occasionally need to be resampled. Deciding if this is necessary requires a reduction operation on the weights after each update. Hence, scalability on a cluster is not only determined by the number of particles used, but also by how well load is balanced. This paper shows how speculative execution in Sequential Monte Carlo with Markov Chain Monte Carlo steps can improve parallel scalability. The key insight is that decisions taken based on the reduction result in each step can be accurately predicted. Consequently, synchronization inherent in the reduction can, in most cases, be avoided, relaxing the limit imposed by load imbalance. Particles are renumbered during resampling to further improve accuracy. Multiple test scenarios, each with different load balance characteristics, are studied empirically on a compute cluster. Tests show that when decisions are predicted correctly, execution time is reduced drastically for use cases with high load imbalance. Furthermore, the maximum theoretical gain, derived from execution characteristics, is compared with the measured improvement to verify that most speculative evaluations are actually useful. If predictions are incorrect, or load is balanced, speculation has no measurable negative impact. Performance is also evaluated in a weak scaling setting on cluster with 36 cores in each system.Part of the work presented in this paper was funded by Johnson & Johnson. Part of the computational resources and services used in this work were provided by the VSC (Flemish SupercomputerCenter), funded by the Research Foundation - Flanders (FWO) and the Flemish Government - department EWI
Reproducible Roulette Wheel Sampling for Message Passing Environments
No full textRoulette Wheel Sampling, sometimes referred to as Fitness Proportionate Selection, is a method to sample from a set of objects each with an associated weight. This paper introduces a distributed version of the method designed for message passing environments. Theoretical bounds are derived to show that the presented method has better scalability than naive approaches. This is verified empirically on a test cluster, where improved speedup is measured. In all tested configurations, the presented method performs better than naive approaches. Through a renumbering step, communication volume is minimized. This step also ensures reproducibility regardless of the underlying architecture.The work has been partially supported by the Ministerio de
Economia y Competitividad under projects ENE2017-89029-P and MTM2014-58159-P, the Generalitat Valenciana under PROMETEO II/2014/008 and the Universitat Politècnica de València under FPI-2013
Structure of the lithosphere within the Trans-Hudson Orogen (results of the 1993 LITHOPROBE Trans-Hudson refraction experiment)
No full textData from three refraction profiles of the 1993 LITHOPROBE refraction experiment were used to investigate the structure of the lithosphere of the Trans-Hudson Orogen. Novel digital processing of the wide-angle reflections arrivals with standard type reflection processing techniques revealed significant crustal thickness and Moho reflectivity variations within the orogenic belt. The obtained information was than complemented with the results of near-vertical incidence seismic sections to estimate the crustal thickness variations over the entire study area. The detected crustal thickness and Moho reflectivity changes could not be correlated to the location and extent of geological domains; they appear to reflect the complex deformation and metamorphic history of the orogen. The P-wave velocity image of the crust and upper mantle was established through ray-tracing and inverse modelling of the primary and secondary crustal and mantle arrivals. Successful modelling of the observation required the incorporation of non-standard inversion techniques into the processing sequence. Although the detected crustal velocity variations appear to correlate well with the changes of Moho reflectivity. These variations in the property of the crust are interpreted to be a consequence of differences between the tectonic evolution of orogenic units in the north-western and south-eastern parts of the study area. The transition belt separating these two areas appear to coincide with an anomalous zone located in the upper mantle. This mantle region exhibits strong P-wave velocity anisotropy, determined primarily from modelling of the mantle refraction arrivals. I interpret this anomalous mantle region as a highly deformed zone, a possibly suture, between the two collided Archean plates of the orogen. Additional information on the structure of the mantle was obtained by analyzing the secondary mantle phases, observed at offsets larger than 400 km on the shot records. The broad structure of this deeper mantle region was established by incorporating the results of regional teleseismic studies into the velocity models and refining the models with raytracing. Detailed acoustic properties of the mantle were investigated after introducing random perturbations into the models and comparing the computed finite-difference synthetic-seismic responses to the observations
Network infrastructure optimization
No full textInternetaanbieders gebruiken netwerkapparatuur die ontwikkeld is voor slechts één functie. De apparatuur wordt geïmplementeerd met ASICs (hardware ontwikkeld voor slechts één doel). De gespecialiseerde apparatuur is nodig om alle data tijdig te verwerken. Om bijvoorbeeld firewall functionaliteit te voorzien, moet er een nieuwe toestel geïnstalleerd worden dat deze functionaliteit aanbiedt. Tegelijkertijd wordt er momenteel ook aan een versneld tempo geïnnoveerd op gebied van netwerk protocollen. In sommige gevallen zorgt de beperking van de aangeboden functionaliteit van de apparatuur ervoor dat de apparatuur vervangen moet worden. Een oplossing voor de beperking in functionaliteit is het gebruik van programmeerbare ASICs. Op het eerste zicht lijkt dit de ideale oplossing te zijn, maar de kosten zijn te hoog. Daarom wordt er verder gezocht naar goedkopere alternatieven en overwegen internetaanbieders om over te stappen naar software oplossingen die geïmplementeerd worden op standaard servers. Deze thesis is een haalbaarheidsstudie die gericht is op de volgende vraag: 'Kan netwerkapparatuur, die gebruikt wordt door internetaanbieders, vervangen worden door software implementaties op processoren die dienen voor algemene doeleinden, zonder de kwaliteit van de aangeboden diensten negatief te beïnvloeden?'. Hierbij richten we ons op één netwerkfunctie (Quality of Service in combinatie met een Broadband Remote Access Server)
- …
