spec2nexus.plugin

An extensible plug-in architecture is used to handle the different possible control line control lines (such as #F, #E, #S, …) in a SPEC data file.

A SPEC control line provides metadata about the SPEC scan or SPEC data file.

Plugins can be used to parse or ignore certain control lines in SPEC data files. Through this architecture, it is possible to support custom control lines, such as #U (SPEC standard control line for any user data). One example is support for the UNICAT-style of metadata provided in the scan header.

Plugins are now used to handle all control lines in spec2nexus.spec. Any control line encountered but not recognized will be placed as text in a NeXus NXnote group named unrecognized_NNN (where NNN is from 1 to the maximum number of unrecognized control lines).

Supplied spec plugin modules

These plugin modules are supplied:

spec_common	SPEC data file standard control lines
fallback	Fallback handling for any SPEC data file control lines not recognized by other handlers
apstools_specwriter	#MD : Bluesky metadata from apstools SpecWriterCallback.
unicat	#H & #V - Metadata in SPEC data files as defined by APS UNICAT
uim	#UIM : Image header information from EPICS areaDetector
uxml	#UXML: UXML structured metadata
XPCS	SPEC data file control lines unique to the APS XPCS instrument

Writing a custom plugin

While spec2nexus provides a comprehensive set of plugins to handle the common SPEC control line control lines, custom control lines are used at many facilities to write additional scan data and scan metadata into the SPEC data file. Custom plugins are written to process these additions.

• How to write a custom plugin module
  • Load a plugin module
  • Write a plugin module
  • Full Example: #PV control line
  • Example to ignore a #Y control line
  • Postprocessing
  • Custom HDF5 writer
  • Custom key match function
  • Summary Requirements for custom plugin
  • Changes in plugin format with release 2021.0.0
  • Footnotes

Overview of the supplied spec plugins

Plugins for these control lines [1] are provided in spec2nexus:

SPEC_File	#F – original data file name (starts a file header block)
SPEC_Epoch	#E – the UNIX epoch (seconds from 00:00 GMT 1/1/70)
SPEC_Date	#D – date/time stamp
SPEC_Comment	#C – any comment either in the scan header or somewhere in the scan
SPEC_Geometry	#G – diffractometer geometry (numbered rows: #G0, #G1, …)
SPEC_NormalizingFactor	#I – intensity normalizing factor
SPEC_CounterNames	#J – names of counters (each separated by two spaces) (new with SPEC v6)
SPEC_CounterMnemonics	#j – mnemonics of counter (new with SPEC v6)
SPEC_Labels	#L – data column labels
SPEC_Monitor	#M – counting against this constant monitor count (see #T)
SPEC_NumColumns	#N – number of columns of data [ num2 sets per row ]
SPEC_PositionerNames	#O – positioner names (numbered rows: #O0, #O1, …)
SPEC_PositionerMnemonics	#o – positioner mnemonics (new with SPEC v6)
SPEC_Positioners	#P – positioner values at start of scan (numbered rows: #P0, #P1, …)
SPEC_HKL	#Q – \(Q\) (\(hkl\)) at start of scan
SPEC_Scan	#S – SPEC scan
SPEC_CountTime	#T – counting against this constant number of seconds (see #M)
SPEC_UserReserved	#U – Reserved for user
SPEC_TemperatureSetPoint	#X – Temperature Set Point (desired temperature)
SPEC_DataLine	(scan_data) – scan data line
SPEC_MCA	#@MCA – MCA data formatting declaration (ignored for now)
SPEC_MCA_Array	@A – MCA Array data
SPEC_MCA_Calibration	#@CALIB – coefficients to compute a scale based on the MCA channel number
SPEC_MCA_ChannelInformation	#@CHANN – MCA channel information
SPEC_MCA_CountTime	#@CTIME – MCA count times
SPEC_MCA_RegionOfInterest	#@ROI – MCA ROI (Region Of Interest) channel information
UnrecognizedControlLine	unrecognized control line
UNICAT_MetadataMnemonics	#H – UNICAT metadata names (numbered rows: #H0, #H1, …)
UNICAT_MetadataValues	#V – UNICAT metadata values (numbered rows: #V0, #V1, …)
UIM_generic	#UIM – various image header information
XPCS_VA	#VA
XPCS_VD	#VD
XPCS_VE	#VE

[1]	Compare this list with Control lines (keys) defined by SPEC

source code documentation

define the plug-in architecture

Use spec2nexus.plugin.ControlLineHandler as a metaclass to create a plugin handler class for each SPEC control line. In each such class, it is necessary to:

• define a string value for the key (class attribute)
• override the definition of process()

It is optional to:

• define postprocess()
• define writer()
• define match_key()

Classes

ControlLineHandler	base class for SPEC data file control line handler plugins
PluginManager()	Manage the set of SPEC data file control line plugins

Exceptions

DuplicateControlLineKey	This control line key regular expression has been used more than once.
DuplicateControlLinePlugin	This control line handler has been used more than once.
DuplicatePlugin	This plugin file name has been used more than once.
PluginBadKeyError	The plugin ‘key’ value is not acceptable.
PluginDuplicateKeyError	This plugin key has been used before.
PluginKeyNotDefined	Must define ‘key’ in class declaration.
PluginProcessMethodNotDefined	Must define ‘process()’ method in class declaration.

class spec2nexus.plugin.AutoRegister(*args)

plugin to handle a single control line in a SPEC data file

This class is a metaclass to auto-register plugins to handle various parts of a SPEC data file. See spec_common for many examples.

Parameters:key (str) – regular expression to match a control line key, up to the first space Returns:None

class spec2nexus.plugin.ControlLineHandler

base class for SPEC data file control line handler plugins

define one ControlLineHandler class for each different type of control line

Parameters:

• key (str) – regular expression to match a control line key, up to the first space
• scan_attributes_defined ([str]) – list of scan attributes defined in this class

Returns:

None

EXAMPLE of match_key method:

Declaration of the match_key method is optional in a subclass. This is used to test a given line from a SPEC data file against the key of each ControlLineHandler.

If this method is defined in the subclass, it will be called instead of match_key(). This is the example used by SPEC_DataLine:

def match_key(self, text):
    try:
        float( text.strip().split()[0] )
        return True
    except ValueError:
        return False

postprocess(header, *args, **kws)

optional: additional processing deferred until after data file has been read

process(text, spec_file_obj, *args, **kws)

required: handle this line from a SPEC data file

writer(h5parent, writer, scan, nxclass=None, *args, **kws)

optional: Describe how to store this data in an HDF5 NeXus file

exception spec2nexus.plugin.DuplicateControlLineKey

This control line key regular expression has been used more than once.

exception spec2nexus.plugin.DuplicateControlLinePlugin

This control line handler has been used more than once.

exception spec2nexus.plugin.DuplicatePlugin

This plugin file name has been used more than once.

exception spec2nexus.plugin.PluginBadKeyError

The plugin ‘key’ value is not acceptable.

exception spec2nexus.plugin.PluginDuplicateKeyError

This plugin key has been used before.

exception spec2nexus.plugin.PluginException

parent exception for this module

exception spec2nexus.plugin.PluginKeyNotDefined

Must define ‘key’ in class declaration.

class spec2nexus.plugin.PluginManager

Manage the set of SPEC data file control line plugins

Class Methods

get(key)	return the handler identified by key or None
getKey(spec_data_file_line)	Find the key that matches this line in a SPEC data file.
load_plugins()	load all spec2nexus plugin modules
match_key(text)	test if any handler’s key matches text
process(key, *args, **kw)	pick the control line handler by key and call its process() method
register_control_line_handler(handler)	auto-registry of all AutoRegister plugins

get(key)

return the handler identified by key or None

getKey(spec_data_file_line)

Find the key that matches this line in a SPEC data file. Return None if not found.

Parameters:spec_data_file_line (str) – one line from a SPEC data file

load_plugins()

load all spec2nexus plugin modules

called from spec2nexus.plugin.get_plugin_manager()

match_key(text)

test if any handler’s key matches text

Parameters:text (str) – first word on the line, up to but not including the first whitespace Returns:key or None

Applies a regular expression match using each handler’s key as the regular expression to match with text.

process(key, *args, **kw)

pick the control line handler by key and call its process() method

register_control_line_handler(handler)

auto-registry of all AutoRegister plugins

Called from AutoRegister.__init__

exception spec2nexus.plugin.PluginProcessMethodNotDefined

Must define ‘process()’ method in class declaration.

spec2nexus.plugin.get_plugin_manager()

get the instance of the plugin_manager (a singleton)

Create instance of PluginManager() if necessary. Also,