This is an archival copy of the Visualization Group's web page 1998 to 2017. For current information, please vist our group's new web page.

Visgroup Home
About
Events
Presentations
Publications
Research
Search
Site Map
Software
Staff
Tutorials and HowTos
Vignettes   Cover Art   Other Galleries
What's New?
Vis@NERSC
Museum of Old Stuff
Internal Only

High Performance Interactive Visualization of Laser Wakefield Particle Accelerator simulations

a) Parallel co-ordinates plot of a LWFA simulation dataset. Particles satisfying multiple interesting criteria are highlighted in red. b) Volume rendering of plasma density and the selected particles. c) Tracing of selected particles across multiple timesteps.


Analysis of laser wakefield particle acceleration data is a challenging task. Our approach combines and extends techniques from high performance scientific data management and visualization, enabling researchers to gain insight from extremely large, complex, time-varying laser wakefield accelerator simulation data. Histogram-based parallel coordinates are used as a visual information display as well as an interface for guiding and performing data mining operations. Multi-dimensional thresholding is used as vehicle for selecting particles of interest at a particular timepoint. We use FastBit, a state-of-the-art index/query system for data extraction and subsetting. All parts of our approach are integrated into the visualization system VisIt providing a product-quality visual data analysis infrastructure. Using our system, scientists are now able to track interesting particles in a fraction of a second, a task which used to take them hours using a naive approach.

LWFA Simulations are performed over 2D and 3D domains using the VORPAL code. Due to the large amount of particles required to achieve accurate simulation results, it is not possible to simulate the entire plasma at once. Simulations are therefore restricted to a window that covers only a subset of the plasma in x direction in the vicinity of the beam. The simulation code moves the window along the local x axis over the course of the simulation. Each simulation produces a set of files for the particle and field data (at typically 40-100 timesteps) with the following main characteristics:

System diagram
To accelerate data mining operations, we use FastBit, a state-of-the-art data management technology for indexing and searching. We use FastBit to perform data subsetting/selection and to compute conditional histograms. We implemented these operations using FastBit directly in the file-reader stage of the processing pipeline in VisIt, a production-quality, parallel capable visual analysis system. The conditional histograms serve as basis for the visual presentation of data vis-a-vis histogram-based parallel coordinates. In contrast to earlier work, we employ a histogram-based parallel coordinates rendering for both context and focus views of large, complex data. Via FastBit, we can recompute conditional histograms efficiently, thus enabling support for fast data selection and smooth drill-down into finer level of detail in very large datasets. As a further improvement, we also support adaptively binned (equal-weight) histograms.
Our system uses two main types of data selection that are implemented by FastBit: i) multi-variate thresholding, and ii) identifier based selection. Multivariate thresholding is used for defining "interesting" data subsets. ID-based selection is the basis for tracing of particles over time. We parallelize computations over the temporal domain to accelerate operations such as particle tracking.

In order to gain a deeper understanding of the acceleration process, we need to address complex questions such as: i) Which particles become accelerated? ii) How are particles accelerated? and iii) How is the beam of highly accelerated particles formed and how does it evolve? To identify those particles that were accelerated, we first perform a selection of particles at a given timestep when the beam has already formed. By tracing the selected particles over time (using ID-queries) we can effectively analyze the temporal behavior of the beam. By refining selections based on information from different timesteps, we are then able to identify characteristic substructures of a beam. An example analysis of a 3D LWFA dataset is shown in the top figure.

Plot of beam evolution using temporal parallel coordinates.
FastBit enables us to quickly specify interesting particles through the parallel co-ordinates interface. These particles can then be efficiently tracked in earlier or later timesteps. For example, we can analyze beam formation by manually selecting beam particles towards the end of the simulation and then track the beam particles back to the time when particles enter the simulation and are injected into the beam. Using temporal parallel co-ordinates, we can analyze the general evolution of the beam in multiple dimensions.

Serial times for computing histograms Parallel times for computing histograms Scalability of parallel histogram computations

Serial times for ID queries Parallel times for particle tracking Scalability of parallel particle tracking

We conducted a large number of tests to examine the performance of FastBit enabled computations for LWFA datasets. In all cases, we compare performance to a custom code which does not have access to index information, and hence loads up the entire dataset. But thereafter, the custom code efficiently computes histograms or does particle tracking. Serial tests are conducted on a standalone dual-opteron workstation. Parallel tests are conducted on upto 100 nodes of franklin.nersc.gov. In parallel mode, each node processes unique timesteps distributed in a strided fashion.

The first panel of plots indicates times taken for computing unconditional and conditional histgrams. We observe that FastBit performance is dramatically better when the number of selected particles forms a small fraction of the entire dataset (as is usually the case for LWFA simulations). As the number of selections approaches a significant fraction of the dataset, the overhead of using FastBit increases. We observe that this particular type of computation scales well across 100 nodes.

The second set of plots indicates performance for running ID queries. Again, there is a dramatic difference between FastBit and the custom code. Similar to the histogram case, if the number of hits approaches a significant fraction of the entire dataset, it might be useful to just read the entire data rather than reading the indices. We should note however that our typical use cases involve tracking upto thousands of particles, which is an ideal performance regime for FastBit. Parallel particle tracking scales very well on upto 100 nodes.

Media

Publications

More Information