With all this introductory stuff out of the way, let us look at the process required to define an application definition:

This is actually the hard bit. There are two things to consider:

For the first part, one of the NeXus guiding principles gives us - Guidance! “A NeXus file must contain all the data necessary for standard data analysis.”

Not more and not less for an application definition. Of course the definition of standard data for analysis or a standard plot depends on the science and the type of data being described. Consult senior scientists in the field about this is if you are unsure. Perhaps you must call an international meeting with domain experts to haggle that out. When considering this, people tend to put in everything which might come up. This is not the way to go.

A key test question is: Is this data item necessary for common data analysis? Only these necessary data items belong in an application definition.

The purpose of an application definition is that an author of upstream software who consumes the file can expect certain data items to be there at well defined places. On the other hand if there is a development in your field which analyzes data in a novel way and requires more data to do it, then it is better to err towards the side of more data.

Now for the case of WONI, the standard data analysis is either Rietveld refinement or profile analysis. For both purposes, the kind of radiation used to probe the sample (for WONI, neutrons), the wavelength of the radiation, the monitor (which tells us how long we counted) used to normalize the data, the counts and the two theta angle of each detector element are all required. Usually, it is desirable to know what is being analyzed, so some metadata would be nice: a title, the sample name and the sample temperature. The data typically being plotted is two theta against counts, as shown in Figure 3.2, “Example Powder Diffraction Plot from (fictional) WONI at HYNES” above. Summarizing, the basic information required from WONI is given in Table 3.1, “basic information required from WONI”.

If you start to worry that this is too little information, hold on, the section on Using an Application Definition (the section called “Using an Application Definition”) will reveal the secret how to go from an application definition to a practical file.

This step is actually easier then the first one. We need to map the data items which were collected in Step 1 into the NeXus hierarchy. A NeXus file hierarchy starts with an NXentry group. At this stage it is advisable to pull up the base class definition for NXentry and study it. The first thing you might notice is that NXentry contains a field named title. Reading the documentation, you quickly realize that this is a good place to store our title. So the first mapping has been found.

title = /NXentry/title

Another thing to notice in the NXentry base class is the existence of a group of class NXsample. This looks like a great place to store information about the sample. Studying the NXsample base class confirms this view and there are two new mappings:

sample name = /NXentry/NXsample/name
sample temperature = /NXentry/NXsample/temperature

Scanning the NXentry base class further reveals there can be a NXmonitor group at this level. Looking up the base class for NXmonitor reveals that this is the place to store our monitor information.

monitor = /NXentry/NXmonitor/data

For the other data items, there seem to be no solutions in NXentry. But each of these data items describe the instrument in more detail. NeXus stores instrument descriptions in the /NXentry/NXinstrument branch of the hierarchy. Thus, we continue by looking at the definition of the NXinstrument base class. In there we find further groups for all possible instrument components. Looking at the schematic of WONI (Figure 3.1, “The (fictional) WONI example powder diffractometer”), we realize that there is a source, a monochromator and a detector. Suitable groups can be found for these components in NXinstrument and further inspection of the appropriate base classes reveals the following further mappings:

probe = /NXentry/NXinstrument/NXsource/probe
wavelength = /NXentry/NXinstrument/NXcrystal/wavelength
two theta of detector elements = /NXentry/NXinstrument/NXdetector/polar angle
counts for each detector element = /NXentry/NXinstrument/NXdetector/data

Thus we mapped all our data items into the NeXus hierarchy! What still needs to be done is to decide upon the content of the NXdata group in NXentry. This group describes the data necessary to make a quick plot of the data. For WONI this is counts versus two theta. Thus we add this mapping:

two theta of detector elements = /NXentry/NXdata/polar angle
counts for each detector element = /NXentry/NXdata/data

The full mapping of WONI data into NeXus is documented in Table 3.2, “Full mapping of WONI data into NeXus”.

Looking at this one might get concerned that the two theta and counts data is stored in two places and thus duplicated. Stop worrying, this problem is solved at the NeXus API level. Typically NXdata will only hold links to the corresponding data items in /NXentry/NXinstrument/NXdetector.

In this step problems might occur. The first is that the base class definitions contain a bewildering number of parameters. This is on purpose: the base classes serve as dictionaries which define names for everything which possibly can occur. You do not have to give all that information. The key question is, as already said, What is required for typical data analysis for this type of application? You might also be unsure how to correctly store a particular data item. In such a case, contact the NIAC for help. Another problem which can occur is that you require to store information for which there is no name in one of the existing base classes or you have a new instrument component for which there is no base class alltogether. In such a case, please feel free to contact the NIAC with a suggestion for an extension of the base classes in question.

This is even easier. Some XML editing is necessary. Fire up your XML editor of choice and open a file. If your XML editor supports XML schema while editing XML, it is worth to load nxdl.xsd. Now your XML editor can help you to create a proper NXDL file. As always, the start is an empty template file. This looks like Example 3.1, “NXDL template file”. This is just the basic XML for a NXDL definition. It is advisable to change some of the documentation strings.

<?xml version="1.0" encoding="UTF-8"?>
<!--
# NeXus - Neutron and X-ray Common Data Format
# 
# Copyright (C) 2008-2011 NeXus International Advisory Committee (NIAC)
# 
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 3 of the License, or (at your option) any later version.
#
# This library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
#
# For further information, see http://www.nexusformat.org

########### SVN repository information ###################
# $Date: 2011-01-26 23:49:00 +0000 (Wed, 26 Jan 2011) $
# $Author: Pete Jemian $
# $Revision: 808 $
# $HeadURL: file:///isis/svn/nexus/definitions/trunk/manual/examples/NX__template__.nxdl.xml $
# $Id: NX__template__.nxdl.xml 808 2011-01-26 23:49:00Z Pete Jemian $
########### SVN repository information ###################
-->
<definition name="NX__template__" extends="NXobject" type="group"
    category="application"
    xmlns="http://definition.nexusformat.org/nxdl/3.1"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd"
    version="1.0b"
    >
    <doc>template for a NXDL application definition</doc>
</definition>

For example, copy and rename the file to NXwoni.nxdl.xml. Then, locate the XML root element definition and change the name attribute (the XML shorthand for this attribute is /definition/@name) to NXwoni. Change the doc as well. Also consider keeping track of /definition/@version as suits your development of this NXDL file.

The next thing which needs to be done is adding groups into the definition. A group is defined by some XML like this:

<group type="NXdata">

</group>

The type is the actual NeXus base class this group belongs to. Optionally a name attribute may be given (default is data).

Next, one needs to include data items too. The XML for such a data item looks like this:

<field name="polar_angle" type="NX_FLOAT units="NX_ANGLE">
  <doc>Link to polar angle in /NXentry/NXinstrument/NXdetector</doc>
  <dimensions rank="1">
    <dim index="1" value="ndet"/>
  </dimensions>
</field>

The meaning of the name attribute is intuitive, the type can be looked up in the relevant base class definition. A field definition can optionally contain a doc element which contains a description of the data item. The dimensions entry specifies the dimensions of the data set. The size attribute in the dimensions tag sets the rank of the data, in this example: rank="1". In the dimensions group there must be rank dim fields. Each dim tag holds two attributes: index determines to which dimension this tag belongs, the 1 means the first dimension. The value attribute then describes the size of the dimension. These can be plain integers, variables, such as in the example ndet or even expressions like tof+1.

Thus a NXDL file can be constructed. The full NXDL file for the WONI example is given in the section called “Full listing of the WONI Application Definition”. Clever readers may have noticed the strong similarity between our working example NXwoni and NXmonopd since they are essentially identical. Give yourselves a cookie if you spotted this.

Basically you are done. Your first application definition for NeXus is constructed. In order to make your work a standard for that particular application type, some more steps are required:

The NIAC must review an application definition before it is accepted as a standard. The one year curation period is in place in order to gain practical experience with the definition and to sort out bugs from Step 1. In this period, data shall be written and analyzed using the new application definition.

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="nxdlformat.xsl" ?>
<!--
# NeXus - Neutron and X-ray Common Data Format
# 
# Copyright (C) 2008-2011 NeXus International Advisory Committee (NIAC)
# 
# This library is free software; you can redistribute it and/or
# modify it under the terms of the GNU Lesser General Public
# License as published by the Free Software Foundation; either
# version 3 of the License, or (at your option) any later version.
#
# This library is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
# Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this library; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
#
# For further information, see http://www.nexusformat.org

########### SVN repository information ###################
# $Date: 2011-02-04 16:02:51 +0000 (Fri, 04 Feb 2011) $
# $Author: Pete Jemian $
# $Revision: 811 $
# $HeadURL: file:///isis/svn/nexus/definitions/trunk/applications/NXmonopd.nxdl.xml $
# $Id: NXmonopd.nxdl.xml 811 2011-02-04 16:02:51Z Pete Jemian $
########### SVN repository information ###################
-->
<definition name="NXmonopd" extends="NXobject" type="group"
    category="application"
    xmlns="http://definition.nexusformat.org/nxdl/3.1"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://definition.nexusformat.org/nxdl/3.1 ../nxdl.xsd"
    version="1.0b"
    svnid="$Id: NXmonopd.nxdl.xml 811 2011-02-04 16:02:51Z Pete Jemian $">
    <doc> Monochromatic Neutron and X-Ray Powder Diffraction. Instrument definition for a powder
        diffractometer at a monochromatic neutron or X-ray beam. This is both suited for a powder
        diffractometer with a single detector or a powder diffractometer with a position sensitive
        detector. </doc>
    <group type="NXentry" name="entry">
        <field name="title"/>
        <field name="start_time" type="NX_DATE_TIME"/>
        <field name="definition">
            <doc> Official NeXus NXDL schema to which this file conforms </doc>
            <enumeration>
                <item value="NXmonopd"/>
            </enumeration>
        </field>
        <group type="NXinstrument">
            <group type="NXsource">
                <field name="type"/>
                <field name="name"/>
                <field name="probe">
                    <enumeration>
                        <item value="neutron"/>
                        <item value="x-ray"/>
                        <item value="electron"/>
                    </enumeration>
                </field>
            </group>
            <group type="NXcrystal">
                <field name="wavelength" type="NX_FLOAT" units="NX_WAVELENGTH">
                    <doc>Optimum diffracted wavelength</doc>
                    <dimensions rank="1">
                        <dim index="1" value="i"/>
                    </dimensions>
                </field>
            </group>
            <group type="NXdetector">
                <field name="polar_angle" type="NX_FLOAT" axis="1">
                  <doc>where ndet = number of detectors</doc>
                  <dimensions rank="1">
                    <dim index="1" value="ndet" />
                  </dimensions>
                </field>
                <field name="data" type="NX_INT" signal="1">
                  <doc>
                    detector signal (usually counts) are already
                    corrected for detector efficiency
                  </doc>
                  <dimensions rank="1">
                    <dim index="1" value="ndet" />
                  </dimensions>
                </field>
            </group>
        </group>
        <group type="NXsample">
            <field name="name">
                <doc>Descriptive name of sample</doc>
            </field>
            <field name="rotation_angle" type="NX_FLOAT" units="NX_ANGLE">
                <doc> Optional rotation angle for the case when the powder diagram has been obtained
                    through an omega-2theta scan like from a traditional single detector powder
                    diffractometer </doc>
            </field>
        </group>
        <group type="NXmonitor">
            <field name="mode">
                <doc>Count to a preset value based on either clock time (timer) or received monitor
                    counts (monitor).</doc>
                <enumeration>
                    <item value="monitor"/>
                    <item value="timer"/>
                </enumeration>
            </field>
            <field name="preset" type="NX_FLOAT">
                <doc>preset value for time or monitor</doc>
            </field>
            <field name="integral" type="NX_FLOAT" units="NX_ANY">
                <doc>Total integral monitor counts</doc>
            </field>
        </group>
        <group type="NXdata">
            <link name="polar_angle" target="/NXentry/NXinstrument/NXdetector/polar_angle">
                <doc>Link to polar angle in /NXentry/NXinstrument/NXdetector</doc>
            </link>
            <link name="data" target="/NXentry/NXinstrument/NXdetector/data">
                <doc>Link to data in /NXentry/NXinstrument/NXdetector</doc>
            </link>
        </group>
    </group>
</definition>

The application definition is like an interface for your data file. In practice files will contain far more information. For this, the extendable capability of NeXus comes in handy. More data can be added, and upstream software relying on the interface defined by the application definition can still retrieve the necessary information without any changes to their code.

NeXus application definitions only standardize classes. You are free to decide upon names of groups, subject to them matching regular expression for NeXus name attributes (see the regular expression pattern for NXDL group and field names in #RegExpName). Note the length limit of 63 characters imposed by HDF5. Please use sensible, descriptive names and separate multi worded names with underscores.

Something most people wish to add is more metadata, for example in order to index files into a database of some sort. Go ahead, do so, if applicable, scan the NeXus base classes for standardized names. For metadata, consider to use the NXarchive definition. In this context, it is worth to mention that a practical NeXus file might adhere to more then one application definition. For example, WONI data files may adhere to both the NXmonopd and NXarchive definitions. The first for data analysis, the second for indexing into the database.

Often, instrument scientists want to store the complete state of their instrument in data files in order to be able to find out what went wrong if the data is unsatisfactory. Go ahead, do so, please use names from the NeXus base classes.

Site policy might require you to store the names of all your bosses up to the current head of state in data files. Go ahead, add as many NXuser classes as required to store that information. Knock yourselves silly over this.

Your Scientific Accounting Department (SAD) may ask of you the preposterous; to store billing information into data files. Go ahead, do so if your judgment allows. Just do not expect the NIAC to provide base classes for this and do not use the prefix NX for your classes.

In most cases, NeXus files will just have one NXentry class group. But it may be required to store multiple related data sets of the results of data analysis into the same data file. In this case create more entries. Each entry should be interpretable standalone, i.e. contain all the information of a complete NXentry class. Please keep in mind that groups or data items which stay constant across entries can always be linked in.