Ok I'm starting to get a hang of Julia and NeXL, but can't manage to get the quant results, it says no take of angle, despite adding it
I'm working through https://pages.nist.gov/NeXLSpectrum.jl/k412refs/
So far this is what I've done
using NeXLSpectrum
using NeXLMatrixCorrection
refs = references([reference(n"Mn",joinpath(path12,"2406SPC.emsa"),mat"Mn"),reference(n"Al",joinpath(path13,"2406SPC.emsa"),mat"Al2O3"),reference(n"Mg",joinpath(path14,"2406SPC.emsa"),mat"MgO"),reference(n"Fe",joinpath(path15,"2406SPC.emsa"),mat"Fe2O3"),reference(n"Si",joinpath(path16,"2406SPC.emsa"),mat"Si59Mg107Fe11O240"),reference(n"O",joinpath(path13,"2406SPC.emsa"),mat"Al2O3"),],130)
res=fit_spectrum(s1,refs)
but couldn't figure out how to quantify single result - alternative command to
quant = quantify.(res)
So did
unks = [s1,s1]
res2= [fit_spectrum(u,refs) for u in unks]
quant=quantify.(res2)
Then no take off angle so tried
unks[1][:TaKeOffAngle]=40
unks[2][:TaKeOffAngle]=40
Still no take off angle when try quant.
Should take off angle be in degrees or radians?
Maybe I need to add take of angle to standards? - How do I do this?
But got this nice fit. What do the numbers of the plot represent?
(https://smf.probesoftware.com/gallery/453_29_04_26_4_24_08.png)
And how do I get kratio and net intensity?
Is there a nicer application to run Julia in then it's console?
Thanks for sharing, NeXL isn't something I was familiar with and from what you've already shown, it seems to have potential. It would be great if we could move away from OEM-based EDS interpretation and just take the raw spectrum (regardless of manufacturer) and quant it ourselves.
I'll admit to being a bit naive on this front, but if we're looking at collecting standards and unknowns on the same EDS system (e.g. the one bolted to our probes) - as we do with WDS - would all the detector-specific bits (window compositions, fano factors etc etc) get cancelled out as we create K ratios via LLSQ fitting?
As both Julia and NeXL are open source (MIT and Unilicense, respectively), is this the kind of thing we could build in to PfE in the future?
Quote from: Ben Buse on April 29, 2026, 04:25:34 AMIs there a nicer application to run Julia in then it's console?
There's a Julia plugin for VSCode: Julia VSCode (https://www.julia-vscode.org/)
Yes part of consistency between instruments is being able to apply same procedures to the raw data.
My current fudge until I can work out how to add take off angle to standards - or someone can tell me, is I can add the take off angle to the input files emsa/msa by adding
#ELEVANGLE : 40.0The file keywords being https://microscopy.org/files/galleries/MSA-MAS_FileFormatSpecification_1991.pdf but unfortunately it seems take off angle isn't mandatory or hasn't been adopted.
Giving me a quant of
Converged to [O=0.3782,Mg=0.0000,Al=0.1036,Si=0.1705,Mn=0.3207,Fe=0.0037] in 6 steps.But still question how do I get the raw kratios and the net intensities?
Ben, You've made great progress. I started to answer your first iteration of this question but I had guests... Julia and NeXL aren't the easiest to get going with. Julia is "theoretically" a wonderful language (it does some of the most amazing things) but in practice it can be a bit of a pain.
```
s[:TakeOffAngle] = deg2rad(40.0) # The take-off angle is in radians (why? just because...)
quant = quantify.(specs) # Is a vector operation, the dot implies apply the function quantify to all elements of the container 's'. The function of a single argument equivalent is:
quant = quantify(spec)
```
The numbers on the fit plot are the k-ratios for each fitted ROI.
I'll provide a couple more answers after I've had a little time to work up some examples.
Take care, Nicholas
Let me help you out by going line-by-line through Fitting K412 (simple API) (https://pages.nist.gov/NeXLSpectrum.jl/k412refs/) I'll not replicate the commentary on the page but add context.
First load the necessary `packages` as Julia calls libraries. Then we are going to add references. References provide the spectral data and meta-data necessary to fit the characteristic peaks in the measurand spectra.
The `references(...)` function takes a vector of FilteredReference instances created using the reference(...)
function like this:
reference(n"Fe", joinpath(path, "Fe std.msa"), mat"Fe")
Vectors are constructed with matching `[` and `]` brackets.
The first argument to
references(...) uses the
n"Xx" syntax which can be used to construct Element, AtomicSubShell and CharXRay instances depending upon the format of the string within the quotes.
n"Fe" produces an Element representing Fe.
n"Fe L3" produces a AtomicSubShell representing the iron L3 sub-shell.
Fe K-L3 produces a CharXRay.
reference(...) requires an Element so...
refs = references(
[
# Specify a reference for iron (arg 1), in "Fe std.msa" (arg2) that is pure iron (arg3).
reference(n"Fe", joinpath(path, "Fe std.msa"), mat"Fe"),
# specify a conductive surface coating layer using the `coating` named argument
reference(n"Si", joinpath(path, "SiO2 std.msa"), mat"SiO2", coating = Film(pure(n"C"), 10.0e-7)),
reference(n"O", joinpath(path, "SiO2 std.msa"), mat"SiO2", coating = Film(pure(n"C"), 10.0e-7)),
reference(n"Ca", joinpath(path, "CaF2 std.msa"), mat"CaF2", coating = Film(pure(n"C"), 10.0e-7)),
# Read the composition from the spectrum file's ##D2STDCMP tag
reference(n"Mg", joinpath(path, "MgO std.msa"), coating = Film(pure(n"C"), 10.0e-7)),
# Read the conductive coating from the spectrum file's ##CONDCOATING tag
reference(n"Al", joinpath(path, "Al2O3 std.msa"), mat"Al2O3"),
],
132.0 # Detector resolution at Mn Kα (eV)
)produces references for the elements O, Mg, Al, Si, Ca and Fe.
The
mat"..." syntax produces Composition instances. It too is flexible.
mat"0.3*Fe+0.6*Cr" produces 30% Fe by mass, 60% Cr by mass
mat"Fe2O3" produces 2 atoms of Fe and 3 atoms of O.
mat"Ca5(PO4)3OH" produces 5 atoms Ca, 3 atom of P, 13 atoms of O and 1 atom of H.
The
132.0 specifies the FWHM which is the only extra piece of information needed that can't be pulled from the spectra. (See
BasicEDS[..] below which shows the other contextual information used to describe the EDS detector.
# Note: This is output - not code
References[
BasicEDS[4096 chs, 1.63032 + 9.99856⋅ch eV, 132.0 eV @ Mn K-L3, 1 ch LLD,
[Be,Sc,Ba,Pu]],
k[Fe L3-M5 + 13 others, Fe],
k[Fe K-L3 + 1 other, Fe],
k[Fe K-M3 + 3 others, Fe],
k[Si K-L3 + 3 others, SiO2],
k[O K-L3 + 1 other, SiO2],
k[Ca K-L3 + 3 others, CaF2],
k[Mg K-L3 + 1 other, MgO],
k[Al K-L3 + 3 others, Al2O3],
]
The we load the measurand
Spectrum object instances using
loadspectra(...). We assign an additional piece of contextual information about the sample coating using the
:Coating Symbol. (Note: If you provide coating meta-data for the reference also provide it for the unknown. Otherwise, your results may vary... If the coating is the same for both, you are probably better off not defining :Coating for either.)
Spectrum objects can be indexed using integers to access the channel data or by "Symbols" to access property data. If `s` is a Spectrum then:
s[:ProbeCurrent] # returns the probe current in nA
s[120] # returns the 120th channel in the spectrum (remember [1] is the first channel)
s[120:130] # returns channels 120 to 130 inclusive.
sum(s[120:130]) # returns the sum over channels 120 to 130.
s[1234.0] # Returns counts in the channel at 1234.0 eV
s[n"Fe K-L3"] # returns the channel at the Fe Kalpha energy
# Now load all the unknown spectra and assign a carbon coating`
unks = map(0:4) do i
s = loadspectrum(joinpath(path, "III-E K412[$i][4].msa"))
# assign a carbon coating
s[:Coating] = Film(pure(n"C"), 30.0e-7)
s
end
Other important properties are
:LiveTime, :ProbeCurrent, :TakeOffAngle, :BeamEnergy which are in seconds, nano-amps, radians (not degrees) and electron-volts (not keV). Note the `:` character means that these labels called Symbols not variable names.
The
plot(...) function is defined in Gadfly (https://gadflyjl.org/stable/) but extended in NeXL to understand spectra and other NeXL specific data types. (In Julia, functions can have many different implementation depending upon the argument types). So my implementation of plot uses Gadfly as a basis but implements EPMA related items like "klms".
In NeXLSpectrum, the
fit_spectrum(...) function fits the references to the measurands and returns a FilterFitResult containing the resulting k-ratios. Thus
res= [ fit_spectrum(u,refs) for u in unks ]
produces a Vector of FilterFitResult instances one for each Spectrum.
The
quantify function takes
FilterFitResult results and performs matrix correction.
quant = quantify.(res) # The dot maps the function quantify across all FilterFitResult instances in res
q = quantify(res[1]) # Applies the function quantify to `res[1]` - the first FilterFitResult instance
The subsequent
plot(...) functions plot the results using specializations of Gadfly functions for Compositions.
There is a lot more functionality to each of these objects that is detailed here. (https://pages.nist.gov/NeXLSpectrum.jl/methods/)
A method you will often find useful is the
NeXLUncertainties.asa(...) function. To use it, load the DataFrames and NeXLUncertainties packages.
using DataFrames, NeXLUncertaintiesThen using the syntax:
asa(DataFrame, res)for various different kinds of `res` will produce a tabulation of the data items (k-ratios, compositions, etc.) DataFrames (https://dataframes.juliadata.org/stable/) is Julia's equivalent to Python's Pandas. It can slice and dice data in wonderful ways.
using CSV will load the CSV package which can read and write DataFrames to CSV files. There are alternatives for other file types like HDF5 or Excel or ...
CSV.write("data.csv", asa(DataFrame, cs))
HyperSpectra work similar to Spectrum objects but with 2D indexes.
using Unitful # For the units in the fov
lt = 0.72*4.0*18.0*3600.0/(1024*1024) # 18.0 hours on 4 detectors (1.0-0.72 dead time)
hs = NeXLSpectrum.compress(HyperSpectrum(
LinearEnergyScale(0.0,10.0),
Dict{Symbol,Any}(
:TakeOffAngle => deg2rad(35.0),
:ProbeCurrent => 1.0,
:LiveTime => lt,
:BeamEnergy => 10.0e3,
:Name => "Mn Nodule"
),
readrplraw(raw"/home/nwmr/Documents/Spectra/Fe3C_10keV_map/map[15]"),
fov = [ 4.096u"mm", 4.096u"mm"], offset= [ 0.0u"mm", 0.0u"mm" ]
))
This code loads the data from a RPL/RAW file (`readrplraw(...)`) and creates a HyperSpectrum from the resulting data then compresses it into the smallest representation that won't lose any channel data (smallest data type and number of channels.) It applies the specified :TakeOffAngle, :ProbeCurrent, :LiveTime, :BeamEnergy since this isn't typically in the RPL header. It also specifies the dimensions of the images via the `fov` property. The detector has a `LinearEnergyScale(...)` with 10.0 eV/ch and 0.0 eV offset.
hs[1,1] # returns the Spectrum at row 1, column 1
hs[2,3][120] # Returns the 120th channel in the Spectrum at row 2, column 3
hs[2,3][120:130] # Returns the 120 to 130 channel (inclusive) in ...
sum(float.(hs[2,3][120:130])) # This converts channels 120 to 130 to a float-type and then sums the result.
# The conversion to float ensures that the sum doesn't overflow as frequently HyperSpectrum object's data
# is represented in short unsigned integer types u8 or u16.
sum_of_120_to_130 = [ sum(float.(hs[ci][120:130])) for ci in CartesianIndices(hs) ] # Creates a 2D matrix of sums of the channels 120 to 130
And there is more...
plane(hs, n"Fe K-L3")
will integrate one FWHM channel range centered on Fe Kalpha producing a matrix of values.
colorize(hs, [ n"Fe K-L3", n"Fe L3-M5", n"C K-L2"])
produces a RGB image with intensities around these characteristic X-ray lines. See Images.jl (https://github.com/JuliaImages/Images.jl) for details on working with images.
To quantify HyperSpectrum objects, use the same functions to create a references (as above) which you apply in the same way:
res=quantify(hs, refs)
except it produces a Materials instance that is a more efficient way of expressing a large matrix of compositions. Index it like:
res[2,3]
to extract Material instances or
res[2,3][n"Fe"]
for individual mass fractions. (See also NeXLCore documentation (https://pages.nist.gov/NeXLCore.jl)) and NeXLMatrixCorrection documentation. (https://pages.nist.gov/NeXLMatrixCorrection.jl)
Thanks Nicholas for all this detailed information, I'm just working through it. I might have missed it but how do I add take off angle to references if it's not contained in the msa, can you address references individually after they are created. Dataframes works a treat. I'm now going to start to look at quantifying the JEOL spectrum map as extracted via Yuji code into hyperspy as discussed here https://smf.probesoftware.com/index.php?topic=1836.0.
Looking at this
LiveTime => lt,Seems live time is fixed for every pixel in the map is it possible to use an array of live time
Using Nicholas example above, I loaded the map save from hyperspy as rpl,
hs = NeXLSpectrum.compress(HyperSpectrum(
LinearEnergyScale(0.0,40.0),
Dict{Symbol,Any}(
:TakeOffAngle => deg2rad(35.0),
:ProbeCurrent => 1.0,
:LiveTime => lt,
:BeamEnergy => 30.0e3,
:Name => "8 sp1 spess"
),
readrplraw(raw"C:\\Users\\glxbb\\julia\\edstry2"),
fov = [ 60u"mm", 60u"mm"], offset= [ 0.0u"mm", 0.0u"mm" ]
))
60 × 60 HyperSpectrum{UInt16,(:Y, :X)}[8 sp1 spess, 0.0 + 40.0⋅ch eV, 512 ch]
And plotted the spectrum for a pixel
plot(hs[1,1])
plot(hs[60,60])
(https://smf.probesoftware.com/gallery/453_30_04_26_1_58_55.png)
Ok by plotting element labels, I realise the hyperspy energy offset wasn't carried through
So here's with energy offset
hs2 = NeXLSpectrum.compress(HyperSpectrum(
LinearEnergyScale(-1538.3,40.0),
Dict{Symbol,Any}(
:TakeOffAngle => deg2rad(35.0),
:ProbeCurrent => 1.0,
:LiveTime => lt,
:BeamEnergy => 30.0e3,
:Name => "8 sp1 spess"
),
readrplraw(raw"C:\\Users\\glxbb\\julia\\edstry2"),
fov = [ 60u"mm", 60u"mm"], offset= [ 0.0u"mm", 0.0u"mm" ]
))
60 × 60 HyperSpectrum{UInt16,(:Y, :X)}[8 sp1 spess, -1538.3 + 40.0⋅ch eV, 512 ch]And then max pixel plot
mp2 = maxpixel(hs2)
plot(mp2, klms=[n"C",n"O",n"Al",n"Si",n"Ca",n"Mn",n"Fe"])
To give
(https://smf.probesoftware.com/gallery/453_30_04_26_2_33_53.png)
And looking at that using the plus key to zoom in on gadfly plot realised I needed to shift it a further 100eV
And there's also plot sum spectra
plot(sum(hs3), klms=[n"C",n"O",n"Al",n"Si",n"Ca",n"Mn",n"Fe"])I reading Nicholas paper https://pmc.ncbi.nlm.nih.gov/articles/PMC9437143/ which also explains creation of ROI maps
(https://smf.probesoftware.com/gallery/453_30_04_26_2_48_33.png)
Now need to figure out how to quantify the map
Can I make a map sum a standard?
MapSum = sum(hs3)
MapSum[:LiveTime]=hs3[1,1][:LiveTime]*60*60
maprefs = references([reference(n"Al",MapSum,mat"Ca3Mn57Al40Si60O240")],128)
ERROR: DomainError with -2082.385000000002:
sqrt was called with a negative real argument but will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
Quote from: Ben Buse on April 30, 2026, 01:11:21 AMLooking at this
LiveTime => lt,Seems live time is fixed for every pixel in the map is it possible to use an array of live time
Using Nicholas example above, I loaded the map save from hyperspy as rpl,
hs = NeXLSpectrum.compress(HyperSpectrum(
LinearEnergyScale(0.0,40.0),
Dict{Symbol,Any}(
:TakeOffAngle => deg2rad(35.0),
:ProbeCurrent => 1.0,
:LiveTime => lt,
:BeamEnergy => 30.0e3,
:Name => "8 sp1 spess"
),
readrplraw(raw"C:\\Users\\glxbb\\julia\\edstry2"),
fov = [ 60u"mm", 60u"mm"], offset= [ 0.0u"mm", 0.0u"mm" ]
))
60 × 60 HyperSpectrum{UInt16,(:Y, :X)}[8 sp1 spess, 0.0 + 40.0⋅ch eV, 512 ch]
You can specify per pixel live-times by adding the `livetime` argument to the HyperSpectrum constructor assigning it a matrix the same dimensions as the RPL/RAW data. (Remove the `:LiveTime => lt,`)
readrplraw(....),
livetime=fill(get(props, :LiveTime, 1.0), (512, 512))
)
You can also construct references in two steps. 1) Load the spectrum and set properties. 2) Replace the reference spectrum path with the loaded spectrum.
spec=pathspectrum("path\to\references\SiO2")
spec[:LiveTime] = 60.0
reference(n"Si", spec, mat"SiO2")
Quote from: Ben Buse on April 30, 2026, 03:48:51 AMCan I make a map sum a standard?
MapSum = sum(hs3)
MapSum[:LiveTime]=hs3[1,1][:LiveTime]*60*60
maprefs = references([reference(n"Al",MapSum,mat"Ca3Mn57Al40Si60O240")],128)
ERROR: DomainError with -2082.385000000002:
sqrt was called with a negative real argument but will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
I'm not exactly sure where the problem lies. It may be a overflow in
hs3[1,1][:LiveTime]*60*60
Try:
float(hs3[1,1][:LiveTime])*60*60 # or
hs3[1,1][:LiveTime]*60.0*60.0
Thanks Nicholas, I will give it a go.
Now I'm trying to figure out how to make my map and standards have the same detector, when the map has a large energy offset, compared to the emsa standards which have no energy offset. I've reloaded map with 2048 channels
60 × 60 HyperSpectrum{UInt16,(:Y, :X)}[8 sp1 spess, -1638.3 + 10.0⋅ch eV, 2048 ch]
refs4
References[
BasicEDS[2048 chs, 0.0 + 10.0⋅ch eV, 130.0 eV @ Mn K-L3, 2 ch LLD, [Be,Sc,Ba,Pu]]
so
resmap4 = fit_spectrum(hs4,refs4)
ERROR: AssertionError: The detector for the hyper-spectrum must match the detector for the filtered references.
Stacktrace:
Any suggestions
Quote from: Nicholas Ritchie on April 30, 2026, 06:00:34 AMQuote from: Ben Buse on April 30, 2026, 03:48:51 AMCan I make a map sum a standard?
MapSum = sum(hs3)
MapSum[:LiveTime]=hs3[1,1][:LiveTime]*60*60
maprefs = references([reference(n"Al",MapSum,mat"Ca3Mn57Al40Si60O240")],128)
ERROR: DomainError with -2082.385000000002:
sqrt was called with a negative real argument but will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
I'm not exactly sure where the problem lies. It may be a overflow in
hs3[1,1][:LiveTime]*60*60
Try:
float(hs3[1,1][:LiveTime])*60*60 # or
hs3[1,1][:LiveTime]*60.0*60.0
Here the problem seems to be using the map spectra as a reference
maprefs = references([reference(n"Al",hs3[1,1],mat"Al")],128)
ERROR: DomainError with -2082.385000000002:
sqrt was called with a negative real argument but will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
Stacktrace:
[1] throw_complex_domainerror(f::Symbol, x::Float64)
@ Base.Math .\math.jl:33
[2] sqrt
@ .\math.jl:627 [inlined]
[3] resolution_to_fwhm
@ C:\Users\glxbb\.julia\packages\NeXLSpectrum\eVZfj\src\detector.jl:155 [inlined]
[4] resolution
@ C:\Users\glxbb\.julia\packages\NeXLSpectrum\eVZfj\src\detector.jl:165 [inlined]
[5] resolution(eV::Float64, det::BasicEDS)
@ NeXLSpectrum C:\Users\glxbb\.julia\packages\NeXLSpectrum\eVZfj\src\detector.jl:309
[6] buildfilter(::Type{Float64}, ty::Type{G2Filter}, det::BasicEDS, a::Float64, b::Float64)
@ NeXLSpectrum C:\Users\glxbb\.julia\packages\NeXLSpectrum\eVZfj\src\filter.jl:320
[7] buildfilter
@ C:\Users\glxbb\.julia\packages\NeXLSpectrum\eVZfj\src\filter.jl:312 [inlined]
[8] references(refs::Vector{NeXLSpectrum.ReferencePacket}, det::BasicEDS; ftype::Type{Float64}, filter::Type)
@ NeXLSpectrum C:\Users\glxbb\.julia\packages\NeXLSpectrum\eVZfj\src\reference.jl:156
[9] references
@ C:\Users\glxbb\.julia\packages\NeXLSpectrum\eVZfj\src\reference.jl:146 [inlined]
[10] references(refs::Vector{NeXLSpectrum.ReferencePacket}, fwhm::Int64; kwargs::@Kwargs{})
@ NeXLSpectrum C:\Users\glxbb\.julia\packages\NeXLSpectrum\eVZfj\src\reference.jl:166
[11] top-level scope
@ REPL[255]:1
As I post just above I'm wondering how to use the emsa standards when map has energy offset
There is a recalibrate function that might work to change the energy offset.
"""
recalibrate(s::Spectrum{T}, es::LinearEnergyScale)
Allows changing the energy scale on a spectrum from one LinearEnergyScale to another as though the spectrum were
measured on a different detector. The algorithm uses a FFT-base scheme to rescale and shift the spectral data.
This scheme allows for fractional shifts of offset and fractional changes in the width. It is limited in that
the change in width must produce an integral number of channels in the resulting spectrum. The algorithm
maintains the total spectrum integral so the new spectrum can be used for quantitative purposes.
Plotting one spectrum over the other should maintain peak position but is likely to change the channel counts.
"""
function recalibrate(s::Spectrum{T}, es::LinearEnergyScale)
Example:
les = LinearEnergyScale(3.0, 10.1)
spec = loadspectrum("\path\to\spectrum\spec.msa")
new_spec = recalibrate(spec, les)
It could be the 128 should be 128.0 (float instead of int) in
references([reference(n"Al",MapSum,mat"Ca3Mn57Al40Si60O240")],128)
Hi Nicholas, thanks I'll give that a go. I've also removed the energy shift from the map dataset at source https://smf.probesoftware.com/index.php?msg=14193.
And now have a quant map
refs3 = references([reference(n"Mn",joinpath(path12,"2406SPCtakeoff.emsa"),mat"Mn"),reference(n"Al",joinpath(path13,"2406SPCtakeoff.emsa"),mat"Al2O3"),reference(n"Mg",joinpath(path14,"2406SPCtakeoff.emsa"),mat"MgO"),reference(n"Fe",joinpath(path15,"2406SPCtakeoff.emsa"),mat"Fe2O3"),reference(n"Si",joinpath(path16,"2406SPCtakeoff.emsa"),mat"Si59Mg107Fe11O240"),reference(n"O",joinpath(path13,"2406SPCtakeoff.emsa"),mat"Al2O3"),],130)
hs7 = NeXLSpectrum.compress(HyperSpectrum(
LinearEnergyScale(0.0,10.0),
Dict{Symbol,Any}(
:TakeOffAngle => deg2rad(40.0),
:ProbeCurrent => 9.993,
:LiveTime => 0.372575,
:BeamEnergy => 15.0e3,
:Name => "spess"
),
readrplraw(raw"C:\\Users\\glxbb\\julia\\spess"),
fov = [ 60u"mm", 60u"mm"], offset= [ 0.0u"mm", 0.0u"mm" ]
))
resmap7 = fit_spectrum(hs7,refs3)
bestresmap7 = optimizeks(SimpleKRatioOptimizer(2.0),resmap7)
resmap7q = quantify(resmap7)
where quant for first pixel gives
resmap7q[1,1]
Map[1,1][O=0.4264,Al=0.1058,Si=0.1672,Mn=0.3032,Fe=0.0024]
I just now can't figure out how to export the quant map as csv for each element
resmap7q
Materials[Map, 60 × 60 × (O, Mg, Al, Si, Mn, Fe)]
asa(DataFrame,resmap7q)
ERROR: MethodError: no method matching asa(::Type{DataFrame}, ::Materials{Float32, Float64, 2})
The function `asa` exists, but no method is defined for this combination of argument types.
So the array of data for 1st element is
resmap7q.massfractions[:,:,1]But I can't see to use this with asa dataframe or CSV write
Quote from: Nicholas Ritchie on April 30, 2026, 05:48:25 AMYou can specify per pixel live-times by adding the `livetime` argument to the HyperSpectrum constructor assigning it a matrix the same dimensions as the RPL/RAW data. (Remove the `:LiveTime => lt,`)
readrplraw(....),
livetime=fill(get(props, :LiveTime, 1.0), (512, 512))
)
Ok I've saved Live time data in hyperspy as csv text image
And loaded the data in Julia
testprop3 = CSV.read("LiveSpess.csv",Tables.matrix,header=0)
livems=testprop2*10
livesec=livems/1000
Array is
60×60 Matrix{Float64}But trying to load array into live time gives an error
fill(get(livesecs, :LiveTime, 1), (60, 60))
ERROR: MethodError: no method matching get(::Matrix{Float64}, ::Symbol, ::Int64)
The function `get` exists, but no method is defined for this combination of argument types.
Closest candidates are:
get(::Base.TTY, ::Symbol, ::Any)
@ Base terminfo.jl:394
get(::Base.TermInfo, ::Symbol, ::Int64)
@ Base terminfo.jl:218
get(::Pkg.Types.Manifest, ::Any, ::Any)
This should work to load the live-times.
hs7 = NeXLSpectrum.compress(HyperSpectrum(
LinearEnergyScale(0.0,10.0),
Dict{Symbol,Any}(
:TakeOffAngle => deg2rad(40.0),
:ProbeCurrent => 9.993,
:LiveTime => 0.372575,
:BeamEnergy => 15.0e3,
:Name => "spess"
),
readrplraw(raw"C:\\Users\\glxbb\\julia\\spess"),
fov = [ 60u"mm", 60u"mm"], offset= [ 0.0u"mm", 0.0u"mm" ],
livetime = CSV.read("LiveSpess.csv",Tables.matrix,header=0)
))
I'll have to do a little work to figure out how to export the quant maps as a CSV. It shouldn't be hard. Usually, I export the data in a flat form since this is frequently easier to work with.
Headers: "Row, Col, O, Mg, Al, Si, Mn, Fe"
It can be shaped using DataFrames Split-Apply-Combine (https://dataframes.juliadata.org/stable/man/split_apply_combine)
Thanks will give it go, I've just figured out export to csv
using DelimitedFiles
writedlm("TestResult.csv", resmap7q.massfractions[:,:,1],',')
How do you produce quant images?
Hi Nicholas, great thanks live time corrected now good
hs8 = NeXLSpectrum.compress(HyperSpectrum(
LinearEnergyScale(0.0,10.0),
Dict{Symbol,Any}(
:TakeOffAngle => deg2rad(40.0),
:ProbeCurrent => 9.993,
:BeamEnergy => 15.0e3,
:Name => "spess"
),
readrplraw(raw"C:\\Users\\glxbb\\julia\\spess"),
livetime = livesec,fov = [ 60u"mm", 60u"mm"], offset= [ 0.0u"mm", 0.0u"mm" ]
))
resmap8 = fit_spectrum(hs8,refs3)
bestresmap8 = optimizeks(SimpleKRatioOptimizer(2.0),resmap8)
resmap8q = quantify(resmap8)
And here's a couple of pixel comparisons using the average live time for the whole map and the pixel per pixel live time, it's a homogeneous area.
resmap7q[1,1]
Map[1,1][O=0.4264,Al=0.1058,Si=0.1672,Mn=0.3032,Fe=0.0024]
julia> resmap8q[1,1]
Map[1,1][O=0.4294,Al=0.1065,Si=0.1684,Mn=0.3053,Fe=0.0024]
julia> resmap7q[1,2]
Map[1,2][O=0.3456,Mg=0.0026,Al=0.1120,Si=0.1671,Mn=0.3031]
julia> resmap8q[1,2]
Map[1,2][O=0.3389,Mg=0.0026,Al=0.1098,Si=0.1639,Mn=0.2972]
This should do what you want...
function NeXLUncertainties.asa(::Type{DataFrame}, mat::Materials; uncertainties = false)
function wu(v)
isa(v, UncertainValue) ? (uncertainties ? uncertainty(v) : value(v)) : (uncertainties ? NaN64 : v)
end
ci = CartesianIndices(mat)
elms = sort(collect(keys(mat)))
res = DataFrame(
"Row" => reshape(collect(map(ci->ci[1], ci)),(:)),
"Col" => reshape(collect(map(ci->ci[2],ci)),(:)),
map(elm->symbol(elm)=>collect(reshape(map(ci->wu(mat[ci][elm]),ci),(:))),elms)...
)
end
using Images
function grayscale(mats::Materials, elm::Element)
vof(v::UncertainValue) = value(v)
vof(v) = v
Gray.(N0f8.(map(CartesianIndices(mats)) do ci
clamp(vof(mats[ci][elm]),0.0,1.0)
end))
end
function colorize(mats::Materials, elms::AbstractArray{Element})
colori(i) = length(elms)>=i && haskey(mats.planes, elms[i]) ? grayscale(mats, elms[i]) : zeros(N0f8, size(mats))
colorview(RGB, colori(1), colori(2), colori(3))
end
This will display in a Jupyter notebook as an image.
Thanks Nicholas will give a try.
With refs I have error
[ Info: A material containing Fe and O is not a suitable reference for "Fe L3-M5 + 13 others" due to a peak interference.
Is it possible to have a separate reference for Fe L from Fe K, like in DTSA2 for shape only?
To summarise where at: I've quantified maps using either emsa from tiny maps of each standard as refs, or using the spectra within PFE and it's emsa export function. For PFE spectra once loaded into julia I multiplied by live time to give counts rather than counts per sec.
sPFE_Al2O3 = sPFE*sPFE[:LiveTime]PFE emsa already contain take off angle. I also have the option of saving the kratio maps and loading into calcimage to combine with wds signal, provided I recast the kratio maps as unknown intensities. Conversely could PFE kratios maps for WDS, by combined with kratio EDS maps in NeXL for quantification.
And to check/compare map quantification and kratios median seems a good function where map a single phase
using Statistics
median(bestresmap8[1].kratios)
median(resmap8q.massfractions[:,:,3])
And to output kratio and quant maps as csv
for i in 1:length(bestresmapPFE2)
writedlm(string(bestresmapPFE2[i].xrays)*"_kratios.csv",bestresmapPFE2[i].kratios,',')
end
for i in 1:length(bestresmapPFE2)
writedlm(string(bestresmapPFE2[i].xrays)*"_quant.csv",resmapPFE2q.massfractions[:,:,i],',')
end
Do I understand you correctly, that PFE exports EMSA files with the values in "Counts/s" rather than "Counts"? Reading through the standard, I didn't see anything that forbids this but it would be a novel interpretation of the standard. (In which case, I should add support for it to NeXL and DTSA-II.) Would you post an example PFE EMSA file to the forum for me to reverse engineer?
If this is the case, it would be acceptable to say:
sPFE[:LiveTime] = 1.0
which would be equivalent to
sPFE[:LiveTime]=measured_live_time
sPFE_Al2o3=sPFE*sPFE[:LiveTime]
The Delimited files package solution to writing CSV files seems like a good one.
Quote from: Nicholas Ritchie on May 04, 2026, 04:40:58 AMDo I understand you correctly, that PFE exports EMSA files with the values in "Counts/s" rather than "Counts"? Reading through the standard, I didn't see anything that forbids this but it would be a novel interpretation of the standard. (In which case, I should add support for it to NeXL and DTSA-II.) Would you post an example PFE EMSA file to the forum for me to reverse engineer?
Yes I've attached a file, and you see it has
#YUNITS : cps
So here's the NeXL maps quantified in CalcImage (with exception of Ca which is from hyperspy because I forgot to include it). Also attached square analyses
(https://smf.probesoftware.com/gallery/453_04_05_26_8_39_09.png)
Interestingly NeXL appears to do a better job of Fe-Mn interference than the JEOL net intensity, JEOL net intensity giving ca. 1 wt % Fe, whereas NeXL has ca. 0.247 wt %. The PFE spot analyses using the JEOL net intensity are, with comparison WDS analysis in Standard.
(https://smf.probesoftware.com/gallery/453_04_05_26_8_43_06.png)
Here because of the collimator on the JEOL EDS (with varying size apertures) we can analysis WDS trace elements at 1uA, whilst the concurrent EDS is running at 7% dead time for major elements
To achieve this the kratios where exported from NeXL to csv as shown above. Then in python recast to unknown intensity and grid files created.
import csv
import tkinter
from tkinter import filedialog
import numpy as np
import struct
import os
import glob
import re
import pandas as pd
mapbeamcurrent = 997.9
#maplivetime = 1319.49
#livetimepixel = 0.366525
#livetimepixel = maplivetime/(60*60)
#load info from wds cnd file
cndfile = tkinter.filedialog.askopenfile(title="Select WDS element .cnd file")
reader = csv.reader(open(cndfile.name), delimiter=" ")
d = list(reader)
Pos2_X1 = d[113][1]
Pos2_Y1 = d[113][2]
Pos2_Z1 = d[113][3]
Pos3_X2 = d[115][1]
Pos3_Y2 = d[115][2]
Pos3_Z2 = d[115][3]
ProjectName = ' '.join(d[29][1:len(d[29])])
#load each of the net intensity csv maps produced by hyperspy
stdintfile = tkinter.filedialog.askopenfile(title="Select exported PFE unknown .dat file containing net inten and kraw")
df = pd.read_csv(stdintfile,sep='\t')
path = filedialog.askdirectory(title="Select location of csv kratio map files")
directories = path.split('/')
for filename in glob.glob(os.path.join(path,'*kratios.csv')):
with open(filename) as f:
reader = csv.reader(f)
#csvfile = tkinter.filedialog.askopenfile(title="Select map .csv file")
#reader = csv.reader(csvfile)
ImageList = list(reader)
NumList = np.array(ImageList)
NumFloat=NumList.astype(float)
#NumFloat=NumFloat/livetimepixel/beamcurrent
#NumFloat=NumFloat/livetimepixel
#NumFloat from kratio to cps - using standard cps
elementfile=re.sub("[]'[]",'',filename.split('\\')[1]).split('.csv')[0]
print(filename)
Element = elementfile.split(' ')[0]
ElLine = elementfile.split(' ')[1]
try:
#df.filter(regex='EDS').filter(regex='NECNT').filter(regex='Al ka').mean()*100
NECNT = df.filter(regex='EDS').filter(regex='NECNT').filter(regex=Element+' '+ElLine[:1].lower()+'a').mean()
Kraw = df.filter(regex='EDS').filter(regex='K-RAW').filter(regex=Element+' '+ElLine[:1].lower()+'a').mean()
StdInt = (1/Kraw.iloc[0])*NECNT.iloc[0]*mapbeamcurrent
NumFloat = NumFloat*StdInt
thead_tag = 1112691524
thead_size = 4
tGrid_size =72
tGrid_ncol=NumFloat.shape[1]
tGrid_nrow=NumFloat.shape[0]
tGrid_xLL=Pos3_X2
tGrid_yLL=Pos2_Y1
tGrid_zmin=NumFloat.min()
tGrid_zmax=NumFloat.max()
tGrid_rotation=0
tGrid_BlankValue=1.71041e38
tData_tag=1096040772
tData_size=tGrid_nrow * tGrid_ncol * 8
tData_hData=NumFloat
thead_version=1
tGrid_tag=1145655879
part1=(int(thead_tag),int(thead_size),int(thead_version),int(tGrid_tag),int(tGrid_size),int(tGrid_ncol),int(tGrid_nrow))
tGrid_xsize=1/1000
tGrid_ysize=1/1000
tGrid_rotation=0
tGrid_blank=1.71041e38
part2=(float(tGrid_xLL),float(tGrid_yLL),float(tGrid_xsize),float(tGrid_ysize),float(tGrid_zmin),float(tGrid_zmax),float(tGrid_rotation),float(tGrid_blank))
part3=(int(tData_tag),int(tData_size))
part4=NumFloat
fp = open(path + '/' +ProjectName+'_00001_WDS0_'+Element+'_'+'EDS'+'.grd','wb')
fp.write(struct.pack('<iiiiiii',part1[0],part1[1],part1[2],part1[3],part1[4],part1[5],part1[6]))
fp.write(struct.pack('<dddddddd',part2[0],part2[1],part2[2],part2[3],part2[4],part2[5],part2[6],part2[7]))
fp.write(struct.pack('<ii',part3[0],part3[1]))
fp.write(NumFloat.astype('<d').tobytes())
fp.close()
except:
print("An error occured, element may not be present in PFE file")
Quote from: Ben Buse on May 04, 2026, 08:45:00 AMInterestingly NeXL appears to do a better job of Fe-Mn interference than the JEOL net intensity, JEOL net intensity giving ca. 1 wt % Fe, whereas NeXL has ca. 0.247 wt %. The PFE spot analyses using the JEOL net intensity are, with comparison WDS analysis in Standard.
Note that PFE allows one to correct for spectral interferences between WDS and EDS elements.
Quote from: John Donovan on May 04, 2026, 08:51:38 AMNote that PFE allows one to correct for spectral interferences between WDS and EDS elements.
Nice, I'll have to give it a go.
Quote from: Ben Buse on May 04, 2026, 08:37:33 AMQuote from: Nicholas Ritchie on May 04, 2026, 04:40:58 AMDo I understand you correctly, that PFE exports EMSA files with the values in "Counts/s" rather than "Counts"? Reading through the standard, I didn't see anything that forbids this but it would be a novel interpretation of the standard. (In which case, I should add support for it to NeXL and DTSA-II.) Would you post an example PFE EMSA file to the forum for me to reverse engineer?
Yes I've attached a file, and you see it has
#YUNITS : cps
Or maybe John would consider changing it if counts is standard?
Quote from: Ben Buse on May 04, 2026, 09:08:00 AMQuote from: Ben Buse on May 04, 2026, 08:37:33 AMQuote from: Nicholas Ritchie on May 04, 2026, 04:40:58 AMDo I understand you correctly, that PFE exports EMSA files with the values in "Counts/s" rather than "Counts"? Reading through the standard, I didn't see anything that forbids this but it would be a novel interpretation of the standard. (In which case, I should add support for it to NeXL and DTSA-II.) Would you post an example PFE EMSA file to the forum for me to reverse engineer?
Yes I've attached a file, and you see it has
#YUNITS : cps
Or maybe John would consider changing it if counts is standard?
We would be happy to change it but then it would mess up everyone else that's already used this format. I guess we could pop up a question as to whether cps or count units should be used. One merely has to multiply by the live time.
Agreed easy to convert. And it's great PFE has take off angle included unlike JEOL emsa
Quote from: John Donovan on May 04, 2026, 09:30:36 AMQuote from: Ben Buse on May 04, 2026, 09:08:00 AMQuote from: Ben Buse on May 04, 2026, 08:37:33 AMQuote from: Nicholas Ritchie on May 04, 2026, 04:40:58 AMDo I understand you correctly, that PFE exports EMSA files with the values in "Counts/s" rather than "Counts"? Reading through the standard, I didn't see anything that forbids this but it would be a novel interpretation of the standard. (In which case, I should add support for it to NeXL and DTSA-II.) Would you post an example PFE EMSA file to the forum for me to reverse engineer?
Yes I've attached a file, and you see it has
#YUNITS : cps
Or maybe John would consider changing it if counts is standard?
We would be happy to change it but then it would mess up everyone else that's already used this format. I guess we could pop up a question as to whether cps or count units should be used. One merely has to multiply by the live time.
Actually I misspoke in a previous post, assuming a live time of 1 s will mess with the count statistical contributions to the uncertainty.
PfE is the only application I know to use "cps" rather than "counts." Now that I know, I can add a special case for you. [Link to XKCD comic](https://xkcd.com/927/)
Quote from: Nicholas Ritchie on May 04, 2026, 01:54:40 PMPfE is the only application I know to use "cps" rather than "counts." Now that I know, I can add a special case for you. [Link to XKCD comic](https://xkcd.com/927/)
Welcome to my world! :)
JEOL uses L-units for spectrometer positions and Cameca uses sin theta * 10^5, JEOL used mm for stage positions and Cameca uses microns., JEOL scans the stage from top to bottom and Cameca scans from left to right, etc., etc.
Based on Ben's feedback, I've updated NeXLCore and NeXLSpectrum to:
- Read PfE-style EMSA files. (NeXLCore v0.3.17)
- Output `Materials` (the data structure to efficiently hold multi-dimensional (0D, 1D, 2D, 3D etc) compositional data) as a table or as a elemental map.(NeXLSpectrum v0.3.13)
Quote from: John Donovan on May 04, 2026, 02:13:45 PMWelcome to my world! :)
JEOL uses L-units for spectrometer positions and Cameca uses sin theta * 10^5, JEOL used mm for stage positions and Cameca uses microns., JEOL scans the stage from top to bottom and Cameca scans from left to right, etc., etc.
The problem really is that it is really hard to write an unambiguous standard. The EMSA standard, as generally observed, is pretty good but there are a handful of ambiguous places - like the #YUNITS tag. In theory, I could write a standard-observing EMSA file that contained "counts/coulomb" or "nanocounts/electron" or ... However, it would be practically impossible to write an EMSA reader that could predict all units a future vendor might decide to use. Over flexible standards become "write only" standards - easy enought to write but almost impossible to write a functionally complete reader. There is an ISO/EMSA 2.0. It almost became a "write-only" standard when the committee almost decided to permit the file use any of the hundreds of different character encodings (https://en.wikipedia.org/wiki/Character_encoding). Instead they chose to permit only UTF8 which most modern programming languages can handle.
Quote from: Nicholas Ritchie on May 05, 2026, 11:31:50 AMQuote from: John Donovan on May 04, 2026, 02:13:45 PMWelcome to my world! :)
JEOL uses L-units for spectrometer positions and Cameca uses sin theta * 10^5, JEOL used mm for stage positions and Cameca uses microns., JEOL scans the stage from top to bottom and Cameca scans from left to right, etc., etc.
The problem really is that it is really hard to write an unambiguous standard. The EMSA standard, as generally observed, is pretty good but there are a handful of ambiguous places - like the #YUNITS tag. In theory, I could write a standard-observing EMSA file that contained "counts/coulomb" or "nanocounts/electron" or ... However, it would be practically impossible to write an EMSA reader that could predict all units a future vendor might decide to use. Over flexible standards become "write only" standards - easy enought to write but almost impossible to write a functionally complete reader. There is an ISO/EMSA 2.0. It almost became a "write-only" standard when the committee almost decided to permit the file use any of the hundreds of different character encodings (https://en.wikipedia.org/wiki/Character_encoding). Instead they chose to permit only UTF8 which most modern programming languages can handle.
I totally get what you are saying, but I figured that "cps" was even more *standard* (that is, more commonly utilized in microanalysis) than "counts"! :)
Quote from: Nicholas Ritchie on April 29, 2026, 11:20:03 AMrefs = references(
[
# Specify a reference for iron (arg 1), in "Fe std.msa" (arg2) that is pure iron (arg3).
reference(n"Fe", joinpath(path, "Fe std.msa"), mat"Fe"),
# specify a conductive surface coating layer using the `coating` named argument
reference(n"Si", joinpath(path, "SiO2 std.msa"), mat"SiO2", coating = Film(pure(n"C"), 10.0e-7)),
reference(n"O", joinpath(path, "SiO2 std.msa"), mat"SiO2", coating = Film(pure(n"C"), 10.0e-7)),
reference(n"Ca", joinpath(path, "CaF2 std.msa"), mat"CaF2", coating = Film(pure(n"C"), 10.0e-7)),
# Read the composition from the spectrum file's ##D2STDCMP tag
reference(n"Mg", joinpath(path, "MgO std.msa"), coating = Film(pure(n"C"), 10.0e-7)),
# Read the conductive coating from the spectrum file's ##CONDCOATING tag
reference(n"Al", joinpath(path, "Al2O3 std.msa"), mat"Al2O3"),
],
132.0 # Detector resolution at Mn Kα (eV)
)
Looking at
# Read the composition from the spectrum file's ##D2STDCMP tagIs this something PFE could do? I understand ## is optional tag in EMSA/MSA and would it affect anything else reading EMSA. If PFE did add this tag what format would the composition be in?
function parsedtsa2comp(value::AbstractString)::Material
try
sp = split(value, ",")
name = sp[1]
mf, density = Dict{Element,Float64}(), missing
for item in sp[2:end]
if item[1] == '(' && item[end] == ')'
sp2 = split(item[2:end-1], ":")
mf[parse(Element, sp2[1])] = 0.01 * parse(Float64, sp2[2])
else
density = parse(Float64, item)
end
end
return material(name, mf, density=density)
catch err
@warn "Error parsing composition $(value) - $(err)"
end
end
This code reads the contents of the ##D2STDCOMP tab. It splits the value on commas.
The first item in the resulting list is material name. Subsequent (except maybe the last) are like "(Fe:43.2)" or "(Al:12.3)", an element abbreviation followed by a mass percentage. The last item could be a number representing the optional density in g/cm^3.
It be great if PFE could for standard spectra read standard composition from standard database and write to emsa file using this tag.
Does DTSA-2 use this tag or just NeXL?
Quote from: Ben Buse on May 07, 2026, 11:42:01 PMIt be great if PFE could for standard spectra read standard composition from standard database and write to emsa file using this tag.
Does DTSA-2 use this tag or just NeXL?
It appears to already be implemented, but using a slightly different tag spelling:
##D2STDCMP : NBS K-411 mineral glass,(Si:25.382),(Fe:11.209),(Mg:8.847),(Ca:11.057),(Al:0.053),(Mn:0.077),(O:43.55805)
This code was implemented a while back I think at Nicholas' request... if my comments in the code are correct.
Right John, the tag is ##D2STDCMP. It is a tag I created for use in DTSA-II and also implemented in NeXL. You've probably already figured this out from the name but I'll state it explicitly anyway: It is intended as a convenient way to record the composition of spectra collected from standards (from materials of known composition.) It isn't intended to record the measured composition.
Quote from: Nicholas Ritchie on May 09, 2026, 04:16:11 PMRight John, the tag is ##D2STDCMP. It is a tag I created for use in DTSA-II and also implemented in NeXL. You've probably already figured this out from the name but I'll state it explicitly anyway: It is intended as a convenient way to record the composition of spectra collected from standards (from materials of known composition.) It isn't intended to record the measured composition.
OK, thanks for confirming the spelling of this tag. And yes, the values output to this tag are the "published" values from the standard composition database. But only when the spectrum in question is from a standard sample.
That's great, I'hadn't checked it was there, I'd just assumed it wasn't, I will give it a go.
Hi Nicholas, I think it's great NeXL screens out unsuitable standards which contain overlaps.
I'm concerned as to whether it should be screening out due to the presence of a minor element
In the emsa file for Sanidine I have 0.76 Ba
B4 Sanidine,(Na:1.27),(Al:9.8),(Si:30.27),(K:11.8),(O:46.03),(Ba:0.76)
This results in Sanidine being rejected
Info: A material containing Al, Ba, Na, Si, O and K is not a suitable reference for "K K-L3 + 3 others" due to a peak interference.
┌ Warning: A material containing Al, Ba, Na, Si, O and K provides no references for K.
└ @ NeXLSpectrum
Whereas if I delete the Ba concentration from the emsa it is no longer rejected.
Is sanidine with 0.76 Ba a suitable EDS standard.
Thanks
Similarly with Durango Apatite
emsa
##D2STDCMP : B4 Durango Apatite,(P:17.79694),(Ca:38.60773),(Si:0.1589262),(Na:0.170626),(S:0.1481775),(F:3.53),(Cl:0.41),(Ce:0.4268828),(La:0.3410718),(Nd:0.128602),(O:37.432)
Rejected on basis of small amount of Si and S
[ Info: A material containing S, Ca, Ce, Nd, O, Cl, Na, F, Si, La and P is not a suitable reference for "P K-L3 + 3 others" due to 2 peak interferences.
┌ Warning: A material containing S, Ca, Ce, Nd, O, Cl, Na, F, Si, La and P provides no references for P.
If remove Si and S it is no longer rejected.
As you observe, there is a limit below which a trace element no longer matters - at least in practical terms. However, as implemented, NeXL doesn't even consider the mass fraction when determining whether a material is suitable. Even 1 part in 10^6 of an element will trigger a rejection. Clearly, this doesn't always make sense. But nor is there a clear cut-off between matters/doesn't matter.
As an alternative, I usually suggest to use pure elements or simple stoichiometric compounds as simple references. If you really want to use a similar standard, fit the unknown with the simple reference, fit the similar standard with the simple reference (and the other elements as necessary) and compute the ratio of the two to get the k-ratio of the unknown relative to the similar standard. This ratio of k-ratios idea is the key notion behind the k-ratio database.
Yes I agree it's complicated to define an almost limit depending on distance away, x-ray yield and concentration.
I do suspect though it's exacerbated in EPMA as we want to list the trace elements in the standards, which may be at EDS detection limit or negligible contribution compared to the major element being used for the standard.
For EDS I've been trying to use simple oxides Mgo, Al2O3, NaCl, TiO2, but also finding I running out of simple compound standards, and quartz is unstable under the electron beam, at high beam currents, which I can use with small aperture selected on collimator. I'd be interested in simple synthetic standards if anyone is sharing any material
I guess one way would be to read the emsa files exported from PFE remove trace elements and write the files. Recognizing this could cause an error where the interference is a problem
Do you find matrix matching important, or do you find pure metals working for silicates. In terms of your suggestion do I fit the unk and the complex standard as unk, using fitted_spectrum, to give the kratios. Then I take the kratios and divide them by each other.
Personally, I find pure elements and simple stoichiometric compounds sufficient for almost everything. As JD points out, matrix correction has improved to the point where similar standards are rarely necessary. He suggests to use secondary standards to validate the measurement rather than similar standards to make the measurement.
My typical probe currents are on the order of 1 nA so material instability under the beam is not an issue. However, I understand that for the probe currents necessary for WDS there can be a problem.
Reading and rewriting the composition makes sense to me. It is a little tedious but it keeps you in control when deciding whether a trace element can be ignored. I'd be much less concerned about omitting an element for hyperspectral maps where the data spectra are low count. I would be much more careful when ignoring elements on references to be used for point spectra.
I've implemented a mechanism to standardize using similar standards in NeXLSpectrum/standardize.jl (https://github.com/usnistgov/NeXLSpectrum.jl/blob/master/src/standardize.jl) THere is an example here: https://github.com/usnistgov/NeXLSpectrum.jl/blob/master/test/standardize.jl
Quote from: Ben Buse on May 12, 2026, 10:21:32 AMInfo: A material containing Al, Ba, Na, Si, O and K is not a suitable reference for "K K-L3 + 3 others" due to a peak interference.
┌ Warning: A material containing Al, Ba, Na, Si, O and K provides no references for K.
└ @ NeXLSpectrum
This feels a really basic question, but is there any way of writing these warning to file.
Yes. One popular way is to use Jupyter notebooks which record all the inputs and outputs. I really like Weave.jl (https://weavejl.mpastell.com/stable/). Weave is more script-like and generates HTML, PDF, Markdown, ... documents from a simple text format file.
Quote from: Nicholas Ritchie on May 28, 2026, 09:30:40 AMYes. One popular way is to use Jupyter notebooks which record all the inputs and outputs. I really like Weave.jl (https://weavejl.mpastell.com/stable/). Weave is more script-like and generates HTML, PDF, Markdown, ... documents from a simple text format file.
Weave looks nice, look forward to giving it a go next week
Ok I gave weave a go, and it's nice, and whilst shows if error in weave report, it doesn't show the info and warning in weave report to say which standards where not suitable. Any ideas how to get the info and warnings
(https://smf.probesoftware.com/gallery/453_02_06_26_6_12_55.png)
Ok I can log the warnings to the file using the julia logger
jl.seval('using Logging')
jl.seval('io = open("log.txt", "w+")')
jl.seval('logger = SimpleLogger(io)')
jl.seval('global_logger(logger)')
jl.seval(''+referencestring+referencemiddle+referenceend)
jl.seval('weave(filename; doctype = "md2html", out_path = :pwd)')
jl.seval('close(io)')
Ok I've attached a python script that quantifies all the spectrum collected by PFE using NeXL, using the PFE assigned standards. Spectrum exported to emsa first using option within PFE. Standard interference problems written to log file and result quant and kratios written to csv files
I'll have to try using the logger in Weave. Seems like a good solution.
Also I couldn't get gadfly plots to show in the weave html report, but found gadfly can also write interactive svg plots
eg for fitted spectra
res9= [fit_spectrum(u,refs5) for u in unks]
for x in 1:length(res9)
myplot = plot(res9[x])
filetest = string(res9[x].label) * string(x) * ".svg"
Gadfly.draw(SVGJS(filetest),myplot)
end
And for standards
myplot = plot(spectra(refs5)..., klms=collect(elms(refs5)), xmax=8.0e3)
Gadfly.draw(SVGJS("refs5.svg"),myplot)
Weave doc attached
So if combine with above python script for processing PFE spectra get additionally the svg plots and the weave report - in python script attached.
??? I have had good success in the past getting Gadfly plots to work in Weave.
Maybe this will help: https://discourse.julialang.org/t/im-not-able-to-use-gadfly-weave/47810/5
It claims that Weave embeds PNG files which require Cairo and Fontconfig to generate.
using Cairo, Fontconfig
Yes I tried that, not sure what my problem is...
Separately is it possible to quant k-ratios in NeXL where oxygen is calculated by stoichometry rather than using a oxygen standard.
I see you have functions
NeXLCore.obystoichiometry
NeXLCore.asoxide
Can I apply them to the quantify command?
Here's the results of NeXL quantifying the PFE spectra, you can see for Wollasonite (the standard for Ca), the O is high, but other values pretty good.
(https://smf.probesoftware.com/gallery/453_03_06_26_5_24_29.png)
Figured out gadfly in weave, it likes breaks in the code for the plot to appear
other plots
```julia
Gadfly.plot(res9[1])
```
Seems if not last line in block of code, plot doesn't show
which means can't run a for loop of plots to html report, but can when saving as interactive svg
Quote from: Ben Buse on June 03, 2026, 05:18:26 AMHere's the results of NeXL quantifying the PFE spectra, you can see for Wollasonite (the standard for Ca), the O is high, but other values pretty good.
(https://smf.probesoftware.com/gallery/453_03_06_26_5_24_29.png)
Very impressive!
The key to O-by-stoic is within NeXLMatrixCorrection.
`quantify` takes an optional argument `iteration` that allows you to configure the iteration algorithm.
NeXLMatrixCorrection.quantify(
ffr::FitResult;
strip::AbstractVector{Element} = Element[],
iteration::Iteration = Iteration(mc=XPP, fc=ReedFluorescence, cc=Coating),
kro::KRatioOptimizer = SimpleKRatioOptimizer(1.5),
coating::Union{Nothing, Pair{CharXRay, <:Material}}=nothing
)::IterationResult
https://github.com/usnistgov/NeXLMatrixCorrection.jl/blob/master/src/iterate.jl
You can configure the iteration procedure using the
Iteration(;
mc::Type{<:MatrixCorrection}=XPP,
fc::Type{<:FluorescenceCorrection}=ReedFluorescence,
cc::Type{<:CoatingCorrection}=Coating,
updater=WegsteinUpdateRule(),
converged=RMSBelowTolerance(0.00001),
unmeasured=NullUnmeasuredRule()
)
structure. The key being the unmeasured argument which defaults to NullUnmeasuredRule() but can be replaced with a rule from https://github.com/usnistgov/NeXLMatrixCorrection.jl/blob/master/src/unmeasuredelement.jl.
The rule you want is
OByStoichiometry(valences = NeXLCore.defaultValences)
where default valances is defined in https://github.com/usnistgov/NeXLCore.jl/blob/master/src/stoichiometry.jl.