JEOL EDS data cube

[Ben Buse]

Hi

Has anyone managed to read the JEOL EDS data cube for 8530/8230 machines - I've found this on github https://github.com/ialxn/JEOL_eds and this relating to hyperspy https://hyperspy.org/rosettasciio/supported_formats/jeol.html both refer to 'jeol analysis station' whatever that is, but both include .pts files which JEOL map generates

Thanks

Ben

[Ben Buse]

Or has anyone used this https://hyperspy.org/hyperspy-doc/v1.7/api/hyperspy.io_plugins.jeol.html with hyperspy bundle install?

[sem-geologist]

I will reply as old and practically inactive hyperspy dev. pts are old format as far I remember used with jeol analysis station - the old nice, clean C written, logic and tidy software used with older generation JEOL SEM's (like from around 2000's). I remember I started to reverse engineer that long ago (its the most human readable binary format I had ever seen), but as we switched to ZEISS SEM's and even replaced old JEOL EDS with Bruker SDD EDS on our Jeol SEM, I lost motivation and someone else took my code and implemented nice reader for these analysis station data structure.

Hyperspy is now a much bigger package, which got split into multiple parts. The part which reads and parses the data into python structures is "rosettasciio" python lib.
Here you can see what formats are supported:
https://hyperspy.org/rosettasciio/supported_formats/index.html
and if you will click on "Jeol", you will see that it supports only old type data from analysis station, which is folder based structure (very clean) with master .aws file, and samples/ROIs/points mappings and etc stored in sub (sub-)folders.

If you need tools to read the modern vomited spaghetti mess of modern Jeol EDS hyper-cubes – then its a very bad luck. I doubt it ever will be implemented in hyperspy, even if Jeol would officially openly share the specification (I doubt there could be any specification for such kind of a mess) nobody sane enough would try implementing it to hyperspy... The only thing, which kinda works exported from modern Jeol GUI's are Tiffs, which also often caused kind of pain...

[sem-geologist]

BTW, I just seen that EDAX shared their format spec! 
That makes absolutely worth to invest in EDS from EDAX (Although last time I had tried their acquisition software GUI, I found it too different (not intuitive) from others and getting in the way)! Had I knew this, our last SEM specification would had included that. This is for me so surprising, EDAX is under Ametek, and they tend to prefer closed-source. Is it because the HyperSpy grownup so much, and so it started to be a deciding factor?

[Ben Buse]

Thanks, Ok so rosetta no good as wants aws files.

How about this for pts files https://hyperspy.org/hyperspy-doc/v1.7/api/hyperspy.io_plugins.jeol.html

I'm struggling to work out how to use it hyperspy console

Code Select Expand

from hyperspy.io_plugins import jeol

ModuleNotFoundError: No module named 'hyperspy.io_plugins'

[yuji]

How about to simply try this?

Code Select Expand

import hyperspy.api as hs
s = hs.load("test.pts")
I think hyperspy or rosettasciio is tested with SEM/STEM data, not EPMA.
I'm not sure if the output is correct for stage-scan data, especially for spatial scale.
But reading pts file would be possible.

[Ben Buse]

Hi Yuji

Thanks for this,

Yes I get the following error

Code Select Expand

TypeError                                Traceback (most recent call last)
Cell In[2], line 1
----> 1 s = hs.load(r"C:\Users\glxbb\OneDrive - University of Bristol\JEOL8530F\EpmaData\Ben\eds wds sjio pfe\eds wds sjio pfe_0002_MAP\Pos_0001\EDS\2406PTTD.PTS")

File ~\AppData\Local\hyperspy-bundle\Lib\site-packages\hyperspy\io.py:667, in load(filenames, signal_type, stack, stack_axis, new_axis_name, lazy, convert_units, escape_square_brackets, stack_metadata, load_original_metadata, show_progressbar, file_format, **kwds)
    663        objects.append(signal)
    664 else:
    665    # No stack, so simply we load all signals in all files separately
    666    objects = [
--> 667        load_single_file(filename, lazy=lazy, **kwds) for filename in filenames
    668    ]
    670 if len(objects) == 1:
    671    objects = objects[0]

File ~\AppData\Local\hyperspy-bundle\Lib\site-packages\hyperspy\io.py:745, in load_single_file(filename, **kwds)
    739    raise ValueError(
    740        "`file_format` should be one of None, str, or a custom file reader object"
    741    )
    743 try:
    744    # Try and load the file
--> 745    return load_with_reader(filename=filename, reader=reader, **kwds)
    747 except BaseException:
    748    _logger.error(
    749        "If this file format is supported, please "
    750        "report this error to the RosettaSciIO developers at "
    751        "https://github.com/hyperspy/rosettasciio/issues"
    752    )

File ~\AppData\Local\hyperspy-bundle\Lib\site-packages\hyperspy\io.py:767, in load_with_reader(filename, reader, signal_type, convert_units, load_original_metadata, **kwds)
    765 lazy = kwds.get("lazy", False)
    766 if isinstance(reader, dict):
--> 767    file_data_list = importlib.import_module(reader["api"]).file_reader(
    768        filename, **kwds
    769    )
    770 else:
    771    # We assume it is a module
    772    file_data_list = reader.file_reader(filename, **kwds)

File ~\AppData\Local\hyperspy-bundle\Lib\site-packages\rsciio\jeol\_api.py:134, in file_reader(filename, lazy, rebin_energy, sum_frames, SI_dtype, cutoff_at_kV, downsample, only_valid_data, read_em_image, frame_list, frame_shifts, frame_start_index)
    132 file_ext = os.path.splitext(filename)[-1][1:].lower()
    133 if file_ext in extension_to_reader_mapping:
--> 134    return extension_to_reader_mapping[file_ext](filename, **kwargs)
    135 else:
    136    _logger.info(f"{filename} : File type {file_ext} is not supported. Skipping")

File ~\AppData\Local\hyperspy-bundle\Lib\site-packages\rsciio\jeol\_api.py:362, in _read_pts(filename, rebin_energy, sum_frames, SI_dtype, cutoff_at_kV, downsample, only_valid_data, read_em_image, frame_list, frame_start_index, frame_shifts, lazy, **kwargs)
    360 datefile = datetime(1899, 12, 30) + timedelta(days=np.fromfile(fd, "d", 1)[0])
    361 fd.seek(head_pos + 12)
--> 362 header = _parsejeol(fd)
    363 meas_data_header = header["PTTD Data"]["AnalyzableMap MeasData"]
    364 width, height = meas_data_header["Meas Cond"]["Pixels"].split("x")

File ~\AppData\Local\hyperspy-bundle\Lib\site-packages\rsciio\jeol\_api.py:702, in _parsejeol(fd)
    700        tmp_dict[kwrd]["index"] = value
    701    else:
--> 702        tmp_dict[kwrd] = value
    703 if kwrd == "Limits":
    704    pass

TypeError: 'numpy.uint16' object does not support item assignment

[Ben Buse]

And if I try the python version https://github.com/ialxn/JEOL_eds and suppress errors using try and except.

It partially reads the file. Extracting some parameters correctly, like name of SEM/EPMA, name of detector, takeoff angle etc... but ultimately fails and reads some parameters wrong...

[yuji]

Hi Ben

https://github.com/Yuji-Tan/jeol_pts_reader
I found your question on hyperspy community and got your pts file.
This is a simple script to create spectrum cube.
All of the metadata are ignored.
please type map size manually.
I tested this script on Gatan DigitalMicrograph (GMS3) for visualization.
Please comment out "import DigitalMicrograph as DM" and related function (DM.CreateImage(), DM.ShowImage())