# 2.1.5. EPICS Area Detector Examples¶

Two examples in this section show how to write NeXus HDF5 data files with EPICS Area Detector images. The first shows how to configure the HDF5 File Writing Plugin of the EPICS Area Detector software. The second example shows how to write an EPICS Area Detector image using Python.

## 2.1.5.1. HDF5 File Writing Plugin¶

This example describes how to write a NeXus HDF5 data file using the EPICS [1] Area Detector [2] HDF5 file writing plugin [3]. We will use the EPICS SimDetector [4] as an example. (PV prefix: 13SIM1:) Remember to replace that with the prefix for your detector’s IOC.

One data file will be produced for each image generated by EPICS.

You’ll need AreaDetector version 2.5 or higher to use this as the procedures for using the HDF5 file writing plugin changed with this release.

### configuration files¶

There are two configuration files we must edit to configure an EPICS AreaDetector to write NeXus files using the HDF5 File Writer plugin:

file description
attributes.xml what information to know about from EPICS and other sources
layout.xml where to write that information in the HDF5 file

Put these files into a known directory where your EPICS IOC can find them.

## 2.1.5.2. attributes.xml¶

The attributes file is easy to edit. Any text editor will do. A wide screen will be helpful.

Each <Attribute /> element declares a single ndattribute which is associated with an area detector image. These ndattribute items can be written to specific locations in the HDF5 file or placed by default in a default location.

Note

The attributes file shown here has been reformatted for display in this manual. The downloads section below provides an attributes file with the same content using its wide formatting (one complete Attribute per line). Either version of this file is acceptable.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48  

If you want to add additional EPICS process variables (PVs) to be written in the HDF5 file, create additional <Attribute /> elements (such as the calc1_val) and modify the name, source, and description values. Be sure to use a unique name for each ndattribute in the attributes file.

Note

ndattribute : item specified by an <Attribute /> element in the attributes file.

## 2.1.5.3. layout.xml¶

You might not need to edit the layout file. It will be fine (at least a good starting point) as it is, even if you add PVs (a.k.a. ndattribute) to the attributes.xml file.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33  

If you do not specify where in the file to write an ndattribute from the attributes file, it will be written within the group that has ndattr_default="true". This identifies the group to the HDF5 file writing plugin as the default location to store content from the attributes file. In the example layout file, that default location is the /entry/instrument/NDAttributes group:

<group
name="NDAttributes"
ndattr_default="true">
<attribute
name="NX_class"
source="constant"
value="NXcollection"
type="string"/>
</group>


To specify where PVs are written in the HDF5 file, you must create <dataset /> (or <attribute />) elements at the appropriate place in the NeXus HDF5 file layout. See the NeXus manual [5] for placement advice if you are unsure.

You reference each ndattribute by its name value from the attributes file and use it as the value of the ndattribute in the layout file. In this example, ndattribute="calc1_val" in the layout file references name="calc1_val" in the attributes file and will be identified in the HDF5 file by the name userCalc1:

<dataset
name="userCalc1"
source="ndattribute"
ndattribute="calc1_val"/>


Note

A value from the attributes file is only written either in the default location or in the location named by a <dataset/> or <attribute/> entry in the layout file. Expect problems if you define the same ndattribute in more than one place in the layout file.

You can control when a value is written to the file, using when="" in the layout file. This can be set to one of these values: OnFileOpen, OnFileClose

Such as:

<dataset
name="userCalc1"
source="ndattribute"
ndattribute="calc1_val"
when="OnFileOpen"/>


or:

<attribute
name="exposure_s"
source="ndattribute"
ndattribute="AcquireTime"
when="OnFileClose"/>


Additional configurations of the EPICS Area Detector and the HDF5 File Plugin are done using the EPICS screens (shown here using caQtDM [6]):

• Set Image mode to “Single”
• Set Exposure time as you wish
• Set # Images to 1
• for testing, it is common to bin the data to reduce the image size
• The full path to the attributes.xml file goes in the bottom/left File box

Additional configuration on the NDFileHDF5 screen:

• Set the File path and “File name” to your choice.
• Set Auto save to “Yes”.
• Set Compression to “zlib” if you wish (optional)
• Set Enable to “Enable” or the HDF5 plugin won’t get images to write!
• Set Callbacks block to “Yes” if you want to wait for HDF5 files to finish writing before collecting the next image
• The full path to the layout.xml file goes into the bottom/right XML File name box
• Leave the Attributes file box empty in this screen.

When you enter the names of these files in the configuration screen boxes, AreaDetector will check the files for errors and let you know.

### Example view¶

We collected data for one image, /tmp/mrinal_001.h5, in the HDF5 file provided in the downloads section. You may notice that the values for calc1_val and calc2_val were arrays rather than single values. That was due to an error in the original attributes.xml file, which had type="PARAM" instead of type="EPICS_PV". This has been fixed in the attributes.xml file presented here.

## 2.1.5.4. Python code to store an image in a NeXus file¶

Suppose you want to write area detector images into NeXus HDF5 files python code. Let’s assume you have the image already in memory in a numpy array, perhaps from reading a TIFF file or from an EPICS PV using PyEpics. The file write_nexus_file.py (provided below) reads an image from the sim detector and writes it to a NeXus HDF5 data file, along with some additional metadata.

### using the h5py package¶

This example uses the h5py [7] package to write the HDF5 file.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 import numpy as np import h5py import datetime def write_nexus_file(fname, image, md={}): """ write the image to a NeXus HDF5 data file Parameters ---------- fname : str name of the file (relative or absolute) to be written image : numpy array the image data md : dictionary key: value where value is something that can be written by h5py (such as str, int, float, numpy array, ...) """ nexus = h5py.File(fname, "w") nexus.attrs["filename"] = fname nexus.attrs["file_time"] = str(datetime.datetime.now()) nexus.attrs["creator"] = "write_nexus_file()" nexus.attrs["H5PY_VERSION"] = h5py.__version__ # /entry nxentry = nexus.create_group("entry") nxentry.attrs["NX_class"] = "NXentry" nexus.attrs["default"] = nxentry.name # /entry/instrument nxinstrument = nxentry.create_group("instrument") nxinstrument.attrs["NX_class"] = "NXinstrument" # /entry/instrument/detector nxdetector = nxinstrument.create_group("detector") nxdetector.attrs["NX_class"] = "NXdetector" # /entry/instrument/detector/image ds = nxdetector.create_dataset("image", data=image, compression="gzip") ds.attrs["units"] = "counts" ds.attrs["target"] = "/entry/instrument/detector/image" # /entry/data nxdata = nxentry.create_group("data") nxdata.attrs["NX_class"] = "NXdata" nxentry.attrs["default"] = nxdata.name # /entry/data/data --> /entry/instrument/detector/image nxdata["data"] = nexus["/entry/instrument/detector/image"] nxdata.attrs["signal"] = "data" if len(md) > 0: # /entry/instrument/metadata (optional, for metadata) metadata = nxinstrument.create_group("metadata") metadata.attrs["NX_class"] = "NXcollection" for k, v in md.items(): try: metadata.create_dataset(k, data=v) except Exception: metadata.create_dataset(k, data=str(v)) nexus.close() if __name__ == "__main__": """demonstrate how to use this code""" import epics prefix = "13SIM1:" img = epics.caget(prefix+"image1:ArrayData") size_x = epics.caget(prefix+"cam1:ArraySizeX_RBV") size_y = epics.caget(prefix+"cam1:ArraySizeY_RBV") # edit the full image for just the binned data img = img[:size_x*size_y].reshape((size_x, size_y)) extra_information = dict( unique_id = epics.caget(prefix+"image1:UniqueId_RBV"), size_x = size_x, size_y = size_y, detector_state = epics.caget(prefix+"cam1:DetectorState_RBV"), bitcoin_value="15000", ) write_nexus_file("example.h5", img, md=extra_information) 

The output from that code is given in the example.h5 file. It has this tree structure:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 example.h5 : NeXus data file @filename = example.h5 @file_time = 2017-12-07 18:22:14.582105 @creator = write_nexus_file() @H5PY_VERSION = 2.5.0 @default = /entry entry:NXentry @NX_class = NXentry @default = /entry/data data:NXdata @NX_class = NXdata @signal = data data --> /entry/instrument/detector/image instrument:NXinstrument @NX_class = NXinstrument detector:NXdetector @NX_class = NXdetector image:NX_UINT8[40,30] = __array __array = [ [48, 80, 112, '...', 208] [240, 16, 48, '...', 176] [208, 240, 16, '...', 144] ... [16, 48, 80, '...', 176] ] @units = counts @target = /entry/instrument/detector/image metadata:NXcollection @NX_class = NXcollection bitcoin_value:NX_CHAR = 15000 detector_state:NX_INT64[] = size_x:NX_INT64[] = size_y:NX_INT64[] = unique_id:NX_INT64[] = 

Note

Alternatively, the metadata shown in this example might be placed in the /entry/instrument/detector (NXdetector) group along with the image data since it provides image-related information such as size.

In the interest of keeping this example simpler and similar to the one above using the HDF5 File Writing Plugin, the metadata has been written into a NXcollection group at /entry/instrument/metadata location. (Compare with the NXcollection group /entry/instrument/NDAttributes above.)

### using the nexusformat package¶

The nexusformat [8] package for python simplifies the work to create a NeXus file. Rewriting the above code using nexusformat:

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 import numpy as np import nexusformat def write_nexus_file(fname, image, md={}): """ write the image to a NeXus HDF5 data file Parameters ---------- fname : str name of the file (relative or absolute) to be written image : numpy array the image data md : dictionary key: value where value is something that can be written by h5py (such as str, int, float, numpy array, ...) """ nx = NXroot() nx['/entry'] = NXentry(NXinstrument(NXdetector())) nx['entry/instrument/detector/image'] = NXfield(image, units='counts', compression='gzip') nx['entry/data'] = NXdata() nx['entry/data'].makelink(nx['entry/instrument/detector/image']) nx['entry/data'].nxsignal = nx['entry/data/image'] if len(md) > 0: # /entry/instrument/metadata (optional, for metadata) metadata = nx['/entry/instrument/metadata'] = NXcollection() for k, v in md.items(): metadata[k] = v nx.save(fname, 'w') if __name__ == "__main__": """demonstrate how to use this code""" import epics prefix = "13SIM1:" img = epics.caget(prefix+"image1:ArrayData") size_x = epics.caget(prefix+"cam1:ArraySizeX_RBV") size_y = epics.caget(prefix+"cam1:ArraySizeY_RBV") # edit the full image for just the binned data img = img[:size_x*size_y].reshape((size_x, size_y)) extra_information = dict( unique_id = epics.caget(prefix+"image1:UniqueId_RBV"), size_x = size_x, size_y = size_y, detector_state = epics.caget(prefix+"cam1:DetectorState_RBV"), bitcoin_value="15000", ) write_nexus_file("example.h5", img, md=extra_information) 

## 2.1.5.5. Visualization¶

You can visualize the HDF5 files with several programs, such as: hdfview [9], nexpy [10], or pymca [11]. Views of the test image shown using NeXPy (from the HDF5 file) and caQtDM (the image from EPICS) are shown.

Get the installation instructions for any of these programs from a web search. Other data analysis programs such as MatLab, IgorPro, and IDL can also read HDF5 files but you might have to work a bit more to get the data to a plot.

file description
attributes.xml The attributes file
layout.xml The layout file
mrinal_001.h5 example NeXus HDF5 file written from EPICS
write_nexus_file.py Python code to get images from EPICS and write a NeXus file
write_nexus_file2.py write_nexus_file.py rewritten with nexusformat package
example.h5 example NeXus HDF5 file written from Python