Quote from: Ben Buse on May 11, 2026, 11:59:35 AMNo I'm not using PFE net intensities but PFE spectra from which NeXL calculates kratios which are converted into unknown net intensities for use with PFE standards

Ok, cool. So you have to export the standard spectra from PFE and extract the net intensities, but then how do you run the quant in CalcImage?  Do you edit the PFE MDB EDS net intensity table with the new net intensities from NeXL?

Quote from: Ben Buse on May 11, 2026, 11:59:35 AMI think as I showed elsewhere I get very similar results, but not same as NeXL and JEOL different algorithms

So whilst loaded as unknown intensity maps I am essentially using NeXL calculated k-ratios rather than JEOL calculated k-ratios

It seems with EDS we should think about the k-ratio algorithm as a option as well as the matrix correction option. Just like if had Bruker or Thermo eds would give slightly different k-ratios using their algorithms.

I would be interested to see a table of how close the net intensities are for JEOL and NeXL.

Hi John

Yes WDS and EDS via PC-EPMA

No I'm not using PFE net intensities but PFE spectra from which NeXL calculates kratios which are converted into unknown net intensities for use with PFE standards

I think as I showed elsewhere I get very similar results, but not same as NeXL and JEOL different algorithms

So whilst loaded as unknown intensity maps I am essentially using NeXL calculated k-ratios rather than JEOL calculated k-ratios

It seems with EDS we should think about the k-ratio algorithm as a option as well as the matrix correction option. Just like if had Bruker or Thermo eds would give slightly different k-ratios using their algorithms.

Quote from: Ben Buse on May 11, 2026, 10:43:27 AMFinally done it, thank you John for explaining this file, and it's great it's codable, now I just need to open the CIP file in CalcImage.

I ask the user for location of grd files, for live time map, for jeol wds cnd file, and assume only one set of maps in the folder, and only correct mdb is present in folder, then read from the mdb the last sample as the setup for the map quantification, and read the element order from the mdb.

And surfer results attached... for maps generated via NeXL

This is very impressive, congrats!

To confirm, you are acquiring the WDS and EDS maps together using the JEOL PC-EPMA software (WDS scan generator), so you can quant stage or beam scans?

And you are using the EDS standard intensities from the PFE MDB file?  And you get good quant results on your maps? 

It would be cool to do a map on a standard and then see if you get the same results using point analyses on the same standard in PFE. In other words, do you get the same net cps live time intensities for both point standards in PFE and map pixel intensities?

Finally done it, thank you John for explaining this file, and it's great it's codable, now I just need to open the CIP file in CalcImage.

I ask the user for location of grd files, for live time map, for jeol wds cnd file, and assume only one set of maps in the folder, and only correct mdb is present in folder, then read from the mdb the last sample as the setup for the map quantification, and read the element order from the mdb.

Like the PTS to rpl and rpl via NeXL to grd, I will make this as plugin into hyperspy

Here's the python code, not yet arranged for hyperspy

#where only one set of maps in folder
#where uses last sample in mdb
#where mdb in same folder and only one mdb present
import tkinter
from tkinter import filedialog
import numpy as np
from numpy import genfromtxt
import csv
import datetime
import pyodbc
import glob
import re
import os
cndfile = tkinter.filedialog.askopenfile(title="Select WDS element .cnd file")
reader = csv.reader(open(cndfile.name), delimiter=" ")
d = list(reader)
ProjectName = ' '.join(d[29][1:len(d[29])])
path = filedialog.askdirectory(title="Select location of grd files")
fc = open(path + '/' + 'test cip creation.cip','w')
fc.write(' 12'+'          '+'"'+'CalcImage Quantitative Image Project'+' '+datetime.datetime.now().strftime("%x")+' '+datetime.datetime.now().strftime("%X")+'"'+'\n')
for fileMDB in glob.glob(os.path.join(path,'*.MDB')):
    print(fileMDB)
MDBname = re.sub("[]'[]",'',fileMDB.split('\\')[1]).split('.MDB')[0]
MDBfile = re.sub("[]'[]",'',fileMDB.split('\\')[1])
spacer = '            '
fc.write('"'+MDBname+'_'+ProjectName+'00001_'+'"'+'\n')
fc.write('"'+MDBfile+'"'+'\n')
MDB = fileMDB
DRV = 'Microsoft Access Driver (*.mdb, *.accdb)'
PWD = 'pw'
con = pyodbc.connect('DRIVER={};DBQ={};PWD={}'.format(DRV,MDB,PWD))
cur = con.cursor()
SQL = 'SELECT * FROM Sample;'
rows = cur.execute(SQL).fetchall()
LastSample = len(rows)
SQL = 'SELECT * FROM Element;'
rows = cur.execute(SQL).fetchall()
elementtable = np.array(rows)
eletable3=elementtable[np.isin(elementtable[:,0], str(LastSample))]
elementorder=[]
elementlist=[]
elementmotor=[]
for x in range(len(eletable3[:,1])):
    elementorder.append(eletable3[x,1])
    elementlist.append(eletable3[x,4])
    elementmotor.append(eletable3[x,2])
cur.close()
con.close()
MapCounter = len(glob.glob1(path,'*_.grd'))
MapCounterEDS = len(glob.glob1(path,'*EDS.grd'))
#MinusOxygen = 1
MapCounterPOFF = len(glob.glob1(path,'*POFF.grd'))
MapCounterNOFF = len(glob.glob1(path,'*NOFF.grd'))
#NumMaps = MapCounter+MapCounterEDS-MinusOxygen
NumMaps = MapCounter+MapCounterEDS+MapCounterPOFF+MapCounterNOFF
TotalsImageFlag = -1
StoicOxFlag = -1
ExcessOxFlag = 0
NetIntenFlag = 0
BgdIntenFlag = 0
KratioIntenFlag = 0
QuantPercFlag = -1
AtomPercFlag = 0
OxidePercFlag = -1
FormulaBasisFlag = 0
EleDiffFlag = 0
DetLimFlag = 0
ErrorFlag = 0
LogPctFlag = 0
DLBlankFlag = 0
ErrBlankFlag = 0
Final = 0
sub = '$XM_AP_SA_DWELL_TIME%0'
WDSDwell = [s for s in d if sub in s][0][1]
sub2 = '$XM_DATA_SAVE_DATE'
offset = 693594
#exceldate = datetime.datetime.date(datetime.datetime.strptime([s for s in d if sub2 in s][0][1],"%Y/%m/%d")).toordinal()-offset
exceltim = datetime.datetime(1899,12,30)
cnddatetime=[s for s in d if sub2 in s][0][1]+' '+[s for s in d if sub2 in s][0][2]
delta=datetime.datetime.strptime(cnddatetime,'%Y/%m/%d %H:%M:%S')-exceltim
excelstore = float(delta.days)+(float(delta.seconds)/86400)
sub3 = '$XM_CP_DATE'
cnddatetime3=[s for s in d if sub3 in s][0][1]+' '+[s for s in d if sub3 in s][0][2]
delta3=datetime.datetime.strptime(cnddatetime3,'%Y/%m/%d %H:%M:%S')-exceltim
excelstore3 = float(delta3.days)+(float(delta3.seconds)/86400)
fc.write(' '+str(LastSample)+spacer+str(NumMaps)+spacer+str(TotalsImageFlag)+spacer+str(StoicOxFlag)+spacer+str(ExcessOxFlag)+spacer+str(NetIntenFlag)+spacer+str(BgdIntenFlag)+spacer+str(KratioIntenFlag)+spacer+str(QuantPercFlag)+spacer+str(AtomPercFlag)+spacer+str(OxidePercFlag)+spacer+str(FormulaBasisFlag)+spacer+str(EleDiffFlag)+spacer+str(DetLimFlag)+spacer+str(ErrorFlag)+spacer+str(LogPctFlag)+spacer+str(DLBlankFlag)+spacer+str(ErrBlankFlag)+spacer+str(Final)+'\n')
PreBeam = "{:.3f}".format(float([s for s in d if '$XM_DATA_PROBE_CURRENT_PRE' in s][0][1])*1e9)
PostBeam = "{:.3f}".format(float([s for s in d if '$XM_DATA_PROBE_CURRENT_POST' in s][0][1])*1e9)
for fileGRD in glob.glob(os.path.join(path,'*_.grd')):
    print(fileGRD)
    wdsloc = re.sub("[]'[]",'',fileGRD.split('\\')[1]).find('WDS')
    ChanNum = re.sub("[]'[]",'',fileGRD.split('\\')[1])[wdsloc+3]
    mapelement = re.sub("[]'[]",'',fileGRD.split('\\')[1])[wdsloc+5:wdsloc+7].split('_')[0]
    fileGRDname = re.sub("[]'[]",'',fileGRD.split('\\')[1])
    for x in [i for i,val in enumerate(elementlist) if val==mapelement]:
        if elementmotor[x] == ChanNum:
            ElePos = elementorder[x]
    fc.write(' 0'+spacer+str(ElePos)+spacer+str(float(WDSDwell)/1000)+spacer+str(PreBeam)+spacer+str(PostBeam)+spacer+str(excelstore)+spacer+str(excelstore3)+spacer+'"'+fileGRDname+'"'+spacer+'0'+'\n')
livefile = tkinter.filedialog.askopenfile(title="Select live time .csv file")
livedata = genfromtxt(livefile, delimiter=',')
EDSLive = np.median(livedata)
for fileGRD in glob.glob(os.path.join(path,'*EDS.grd')):
    print(fileGRD)
    wdsloc = re.sub("[]'[]",'',fileGRD.split('\\')[1]).find('WDS')
    ChanNum = re.sub("[]'[]",'',fileGRD.split('\\')[1])[wdsloc+3]
    mapelement = re.sub("[]'[]",'',fileGRD.split('\\')[1])[wdsloc+5:wdsloc+7].split('_')[0]
    fileGRDname = re.sub("[]'[]",'',fileGRD.split('\\')[1])
    for x in [i for i,val in enumerate(elementlist) if val==mapelement]:
        if elementmotor[x] == ChanNum:
            ElePos = elementorder[x]
    fileGRDname = re.sub("[]'[]",'',fileGRD.split('\\')[1])
    fc.write(' 0'+spacer+str(ElePos)+spacer+str(EDSLive)+spacer+str(PreBeam)+spacer+str(PostBeam)+spacer+str(excelstore)+spacer+str(excelstore3)+spacer+'"'+fileGRDname+'"'+spacer+'0'+'\n')
for fileGRD in glob.glob(os.path.join(path,'*POFF.grd')):
    print(fileGRD)
    wdsloc = re.sub("[]'[]",'',fileGRD.split('\\')[1]).find('WDS')
    ChanNum = re.sub("[]'[]",'',fileGRD.split('\\')[1])[wdsloc+3]
    mapelement = re.sub("[]'[]",'',fileGRD.split('\\')[1])[wdsloc+5:wdsloc+7].split('_')[0]
    fileGRDname = re.sub("[]'[]",'',fileGRD.split('\\')[1])
    for x in [i for i,val in enumerate(elementlist) if val==mapelement]:
        if elementmotor[x] == ChanNum:
            ElePos = elementorder[x]
    fc.write(' 1'+spacer+str(ElePos)+spacer+str(float(WDSDwell)/1000)+spacer+str(PreBeam)+spacer+str(PostBeam)+spacer+str(excelstore)+spacer+str(excelstore3)+spacer+'"'+fileGRDname+'"'+spacer+'0'+'\n')
for fileGRD in glob.glob(os.path.join(path,'*NOFF.grd')):
    print(fileGRD)
    wdsloc = re.sub("[]'[]",'',fileGRD.split('\\')[1]).find('WDS')
    ChanNum = re.sub("[]'[]",'',fileGRD.split('\\')[1])[wdsloc+3]
    mapelement = re.sub("[]'[]",'',fileGRD.split('\\')[1])[wdsloc+5:wdsloc+7].split('_')[0]
    fileGRDname = re.sub("[]'[]",'',fileGRD.split('\\')[1])
    for x in [i for i,val in enumerate(elementlist) if val==mapelement]:
        if elementmotor[x] == ChanNum:
            ElePos = elementorder[x]
    fc.write(' 2'+spacer+str(ElePos)+spacer+str(float(WDSDwell)/1000)+spacer+str(PreBeam)+spacer+str(PostBeam)+spacer+str(excelstore)+spacer+str(excelstore3)+spacer+'"'+fileGRDname+'"'+spacer+'0'+'\n')
fc.write(' 10'+'\n')
fc.write(' 0  0  0  0  0  0  0  0  0  0 '+'\n')
fc.write(' 0'+'\n')
TitleAppendFlag = -1
SurferTemplateFlag = 0
SurferSliceTemplateFlag = 0
SurferPolyTemplateFlag = 0
SurferStripTemplateFlag = 0
SpareFlag1 = 0
SpareFlag2 = 0
fc.write(str(TitleAppendFlag)+spacer+str(SurferTemplateFlag)+spacer+str(SurferSliceTemplateFlag)+spacer+str(SurferPolyTemplateFlag)+spacer+str(SurferStripTemplateFlag)+spacer+str(SpareFlag1)+spacer+str(SpareFlag2)+'\n')
#EDS spectrum image not utilized
SpecImageFlag = 0
fc.write(str(SpecImageFlag)+spacer+'"'+'"'+'\n')
#Could read scan type from cnd file. Are these flags used?
ScanType = 0 #0=beam 1=stage
StagePolarity = 0 #0=cameca 1=jeol
fc.write(str(ScanType)+spacer+str(StagePolarity)+'\n')
SEIflag = 0
COMPflag = 0
for fileGRD in glob.glob(os.path.join(path,'*SEI.grd')):
    print(fileGRD)
    SEIname = re.sub("[]'[]",'',fileGRD.split('\\')[1])
    SEIflag = 1
for fileGRD in glob.glob(os.path.join(path,'*COMP.grd')):
    print(fileGRD)
    COMPname = re.sub("[]'[]",'',fileGRD.split('\\')[1])
    COMPflag = 1
fc.write(str(SEIflag)+spacer+'"'+'SE'+'"'+spacer+str(COMPflag)+spacer+'"'+'BSE'+'"'+spacer+'0'+spacer+'"'+'CL'+'"'+'\n')
fc.write('"'+SEIname+'"'+spacer+'"'+COMPname+'"'+spacer+'"'+'"'+'\n')
#tdi corrections
tdi_num = 0 #number of tdi maps
tdi_intervals = 0 #number of tdi intervals
fc.write(' '+str(tdi_num)+spacer+str(tdi_intervals)+'\n')
#placeholder flags
fc.write(' 0            0            0            0            0            0 '+'\n')
fc.write(' 0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 0            0 '+'\n'+' 0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 0            0 '+'\n'+' 0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 0            0 '+'\n'+' 0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 0            0 '+'\n'+' 0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 0            0 '+'\n'+' 0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 1            0            0 '+'\n'+' 0            0 ')
fc.close()
And surfer results attached... for maps generated via NeXL

Great thing is julia can be called from python, so hoping to run entire process as one python script which can be loaded as plugin into hyperspy, giving .grd files and .CIP file for opening in calcimage. And also add spatial and energy calibration to plots shown above

That's great, I'hadn't checked it was there, I'd just assumed it wasn't, I will give it a go.

Quote from: Nicholas Ritchie on May 09, 2026, 04:28:08 PMAnother hack is use to collect EDS standard spectra:

Rather than collecting a single spectrum for 300 s, collect 5 spectra from 5 distinct points (usually selected at random (shotgun-style)) for 60 s each.  Then compare them analytically (https://search.proquest.com/openview/1ac95d227ead32a092137a9e61c71e20/1?pq-origsite=gscholar&cbl=33692) and/or visually to ensure that they agree. Remove any that don't and sum the remainders into a single spectrum to use as the standard.

* Ritchie, Nicholas WM. "Strategies for Standardizing EDS Measurements." Microscopy and Microanalysis 28.S1 (2022): 496-498.

I completely agree that all analyses should be based on the average of replicate measurements and then evaluated statistically, including standards!

That is why the default number of automated point acquisitions for standard samples in Probe for EPMA is 5, though of course one can acquire more or fewer points:



I am reminded of a geology student I worked with long ago who had gotten his undergraduate degree in math, and who always insisted on measuring 25 points for each material. Because, you know, statistics:

Student's "t":

   # pts    d.f.    0.60    0.80    0.90    0.95    0.99
       2       1  1.3764  3.0777  6.3138 12.7062 63.6567
       3       2  1.0607  1.8856  2.9200  4.3027  9.9248
       4       3   .9785  1.6377  2.3534  3.1824  5.8409
       5       4   .9410  1.5332  2.1318  2.7764  4.6041
       6       5   .9195  1.4759  2.0151  2.5706  4.0321
       7       6   .9057  1.4398  1.9432  2.4469  3.7074
       8       7   .8961  1.4149  1.8946  2.3646  3.4995
       9       8   .8889  1.3968  1.8595  2.3061  3.3554
      10       9   .8834  1.3831  1.8331  2.2622  3.2498
      11      10   .8791  1.3722  1.8125  2.2281  3.1693
      12      11   .8755  1.3634  1.7959  2.2010  3.1058
      13      12   .8726  1.3562  1.7823  2.1788  3.0545
      14      13   .8702  1.3502  1.7709  2.1604  3.0123
      15      14   .8681  1.3451  1.7613  2.1448  2.9768
      16      15   .8662  1.3406  1.7531  2.1315  2.9467
      17      16   .8647  1.3368  1.7459  2.1199  2.9208
      18      17   .8633  1.3334  1.7396  2.1098  2.8982
      19      18   .8621  1.3304  1.7341  2.1009  2.8784
      20      19   .8610  1.3277  1.7291  2.0931  2.8609
      21      20   .8600  1.3253  1.7247  2.0860  2.8453
      22      21   .8591  1.3232  1.7207  2.0796  2.8314
      23      22   .8583  1.3212  1.7171  2.0739  2.8188
      24      23   .8575  1.3195  1.7139  2.0687  2.8073
      25      24   .8569  1.3178  1.7109  2.0639  2.7969
      26      25   .8562  1.3163  1.7081  2.0595  2.7874
      27      26   .8557  1.3150  1.7056  2.0555  2.7787
      28      27   .8551  1.3137  1.7033  2.0518  2.7707
      29      28   .8546  1.3125  1.7011  2.0484  2.7633
      30      29   .8542  1.3114  1.6991  2.0452  2.7564

https://skeptric.com/two-point-eight/

Quote from: Nicholas Ritchie on May 09, 2026, 04:16:11 PMRight John, the tag is ##D2STDCMP.  It is a tag I created for use in DTSA-II and also implemented in NeXL. You've probably already figured this out from the name but I'll state it explicitly anyway:  It is intended as a convenient way to record the composition of spectra collected from standards (from materials of known composition.) It isn't intended to record the measured composition.

OK, thanks for confirming the spelling of this tag.  And yes, the values output to this tag are the "published" values from the standard composition database. But only when the spectrum in question is from a standard sample.

Another hack is use to collect EDS standard spectra:

Rather than collecting a single spectrum for 300 s, collect 5 spectra from 5 distinct points (usually selected at random (shotgun-style)) for 60 s each.  Then compare them analytically (https://search.proquest.com/openview/1ac95d227ead32a092137a9e61c71e20/1?pq-origsite=gscholar&cbl=33692) and/or visually to ensure that they agree. Remove any that don't and sum the remainders into a single spectrum to use as the standard.

* Ritchie, Nicholas WM. "Strategies for Standardizing EDS Measurements." Microscopy and Microanalysis 28.S1 (2022): 496-498.

Right John, the tag is ##D2STDCMP.  It is a tag I created for use in DTSA-II and also implemented in NeXL. You've probably already figured this out from the name but I'll state it explicitly anyway:   It is intended as a convenient way to record the composition of spectra collected from standards (from materials of known composition.) It isn't intended to record the measured composition.