Probe Software Users Forum

Software => DTSA II => Topic started by: sem-geologist on March 30, 2021, 06:01:57 AM

Title: How to sum up few spectra
Post by: sem-geologist on March 30, 2021, 06:01:57 AM
Hello,
Recently I got into trying to use DTSA-II, I watched the videos and had read few papers about DTSA-II. It is mentioned that it is good practice to get few measurements of standards instead of single but very long measurement of the standard. The problem is that I can't find how to sum such separate spectra into a single entity.

For standards it is not such a huge problem as those are pretty stable, but I need that for my unknown mineral from some experiments. It is losing sodium like crazy, so I made 30 separate measurements of 2 seconds length and want to sum this batch into a single spectra. My files are bruker spx.

I appreciate the working examples how to do Quanti alien, but I feel I am missing the step-by-step understanding how to build my standard database. Maybe I am understanding the workflow wrongly?

On EPMA (I am used to Cameca microprobes) we can have these huge standard databases built, as temperature stability in the room and WDS position repeatability and forced geometry, also gas-flow proportional counters (constant gas properties) allows to get the same measurements of standards in few months, and even years. We don't need to do every-day re-calibration, just QC checks to be sure that there is no changes.

I don't see this in DTSA-II, we need to keep standards as separate files? and with every quantification attempt we need those to be imported, and missing fields filled in (spx, or Bruker system has no idea about probe current, coating, and DTSA-II is not reading some fileds which are in spx like WD). This looks a bit inefficient and I hate doing the same things again and again (who does not hate it), I probably am missing something very obvious. Or this is just forced feature as I know lots of laboratories don't run stable air conditioning and does not keep track of other microscope parameters, so is it to force to do new standard analyses per sample?

Title: Re: How to sum up few spectra
Post by: Nicholas Ritchie on March 30, 2021, 06:28:16 AM
I think I can help you.
The tool you want is the "Standard Bundler."  The "Standard Bundler" performs a number of tasks and I've been working on enhancing it recently to make it much more efficient to use.
I'll describe the process in the most recent "pre-release" version of DTSA-II available here: DTSA-II Pre-Release (https://drive.google.com/file/d/17YAXRYkQ9vAQ509AwP5eZagdGfShX-dE/view?usp=sharing)
First, select all the spectra you wish to combine in the "Spectrum list" on the "Spectrum" tab
1. Right click the spectrum list and select "Make "Spectrum Bundle"" menu item to initiate the "Spectrum Bundler" dialog
2. You can now enter probe current and live time data here in the first panel of the "Spectrum Bundler".  Select the spectra to update and then enter the "probe current" and "live time" as necessary.
3. In the next panel, enter the material and other meta data
4. In the next panel, the spectra are compared one to another.  The score tells you how similar the spectra are.  ~1 is excellent but is realized only for the most homogeneous samples and careful measurements.  Spectra that are more than a factor of two above the mean score should probably be removed.
5. In the "Special Options" panel, specify special options.  (Probably none to start...)
6. In the "References" panel, any required references are listed and can be provided.
When the "Finish" button is selected, you will be asked for a directory into which to place the "Standard Bundle" with a "zstd" extension containing the standard and any provided references and an "msa" file with the sum standard spectrum.   You may use either as a standard although if references are required, the "zstd" will be simpler and quicker to use.

This should help you to construct a standard database which you can reuse over time.
Title: Re: How to sum up few spectra
Post by: Nicholas Ritchie on March 30, 2021, 06:38:45 AM
Advanced users:
There is also another method to sum spectra from the command line.
Part 1: Select some spectra
Mode A: Select the currently displayed spectra
1> ss = selected()
Mode B: Select from all loaded spectra.  First list spectra and then use the alias to create a list.
1> ls()
Name   Spectrum
s21   Mn 1
s22   Mn 2
s23   Mn 3
s24   Mn 4
s25   Mn 5
s26   Mn std
2> ss = [ s21, s22, s24 ]
Part 2:  Sum the spectra and display it
3> s = sum(ss)
4> display(s)

or in short:
1> display(sum(selected()))
or
1> display(sum( [ s21, s22, s24 ]))
Title: Re: How to sum up few spectra
Post by: sem-geologist on March 30, 2021, 08:03:09 AM
Thanks a lot,
Simple summing of spectras through jython (no comparison) is much more practical for my 2 second unknown spectras (due to very short time it is natural that they will be significantly different), and bundle button looks more practical for standards. When I sum the spectra through command line it will sum live time automatically?

But I had bumped into another problem (See attachement):
Why Quanti alien is asking me for those lines, It is unobtainable at 20kV, even in pure form of those elements the lines would be poor (and getting new metal REE standards just for this would be quite an overkill). I probably again had missed something. I am stuck at that point (well the final mineral which I need to determine composition is not chevkinite, but gagarinite - that is Na(Ca,REE)F6, but to convince colleagues that EDS result of gagarinite is good I need to show-off that chevkinite can be done with exactly same method (which has near 40 elements). Well, this is quite complicated...
Title: Re: How to sum up few spectra
Post by: sem-geologist on March 30, 2021, 08:33:58 AM
I start to fear that DTSA-II will not cut through my case at the moment, as that would require investment in additional standards. I.e. on EPMA I use two Yttrium standards YPO4 and YAG (Y-alumina garnet), both of them looks poor for DTSA-II, while are perfectly OK on EPMA; I probably should use YF3 to succeed, but what about LREE.... This shape reference thing makes me scratch my head – is not EDS spectrum broadening a result of pure natural distribution/or Gaussian which is just gain noise from multi-channel processing unit? I mean why we need the peak shapes if it is just sum of Gaussian convoluted spectra. Why not just do deconvolution on spectra with dynamic sigma Gaussian? On the other hand, doing similar method on WDS scans - this real shape approach would be the only working approach as on WDS shape is riddled with artifacts and is nearly impossible to correctly be modeled. I am probably missing something from big picture again.
Title: Re: How to sum up few spectra
Post by: Nicholas Ritchie on March 30, 2021, 08:53:38 AM
You are right that it is possible to fit Gaussians (and a continuum model) to the X-ray peaks.  In fact, the "Spectrum Bundler" can do this for you when a reference simply isn't available.  (Use the "Auto" button on the "References" tab) However, the true peak shape is only approximately Gaussian.  Additional subtleties like incomplete charge collection introduce non-Gaussian characteristics to the shape.  The results is that modeled peaks shape references won't work as well as measured ones.  This becomes a particular issue when trying to resolve peak interferences.

Unfortunately, both YPO4 and YAG require a Y reference.  Al is too close to the Y L2-M3 and P covers the Y L3-M5.  It is true that some materials that are suitable for WDS are not suitable for references for EDS.  In the meanwhile, I'd be tempted to use the "Auto" reference button on the YAG spectrum.  It might work reasonably well.
Title: Re: How to sum up few spectra
Post by: sem-geologist on March 30, 2021, 09:21:00 AM
Oh I see, So I need to go through "Bundle" option and at tab "references" click Auto for REE's and Y files when making standard bundles. so basically there are few ways, and they differ a bit. I was importing spectras as standards at Quanti alien wizard, thats probably a problem. If I will prepare bundles for every element I need, maybe this will progress.
Title: Re: How to sum up few spectra
Post by: Nicholas Ritchie on March 30, 2021, 09:22:40 AM
With respect to the low energy lines, you can specify which elements are visible on a detector using the "Calibration Alien".  This creates a new calibration that can then be applied to the unknown spectrum using the "Spectrum Properties" dialog.  I often set the first visible K line to Sc, the first L to La, first M to La and first N to Am.  This is ok at high beam energies but isn't suitable at low beam energies at which the relative visibility of these lines increases.

What you have is a very challenging problem.  To fit O, you need to account for the interfering peaks due to Ti L and the REE Ms.  When peaks interfere, you can't just fit some of the peaks.  You have to fit them all.
Title: Re: How to sum up few spectra
Post by: sem-geologist on March 30, 2021, 10:04:17 AM
actually O is not a problem, I calculate it stochiometrically.
So That was that: I had to setup standard bundle files prior trying doing quanti alien. And it went all well. I am impressed – The result is quite remarkable (there is some minor deficiencies, but that probably can be solved with better calibrations, i.e. Lanthanium and Si got quite significantly (La2O3 8 % instead of 10%; and SiO2 17.3 instead of 18%) undercalculated; in case of La i can't see there problem lies, but for Si I think SiO2 is not good standard, I normally use diopside on EPMA. Anyway, other elements are just close to what I get on EPMA for same mineral. It is even more mind-blowing if comparing the conditions: on EPMA that takes (well, measured more trace elements) tens of minutes, while this is just 1minute EDS spectra. on EPMA I had to resolve more than 30 peak-overlaps calibration of those all stuff took few weeks to get close to what it is supposed to be from structural analysis. And these EDS calibrations are just 1 minute calibrations. I just wonder how far this can go.

Now some things which gets in the way and is annoying at DTSA: wizard window is not resizable. When final result for quantification is displayed in the wizard I can't see a thing as there is so many elements and that automatically narrows columns in the result table so I can see only zeroes. The only way to see the result is to finish the wizard and look into Report, and then If I want to change something in quantification process I need to do all things from the beginning in quantification alien, instead stepping back in the wizard. I get that my application is a bit niche (it is no secret, that I am getting into similar issues (which are alien for most of normal people) with OEM software on EPMA), but if window can't be resized, it would be nice if result table could be scroll-able.
Title: Re: How to sum up few spectra
Post by: orlandin on March 30, 2021, 03:08:21 PM
Hi sem-geologist - your last point about the size of the results window and how often one might want to go back in the workflow to tinker with a setting/standard (cough carbon thickness) is something I have solved by cutting and pasting the whole results output into a pre-formatted Excel spreadsheet that extracts the wt% and uncertainty into different rows. Then moving back and forth through the steps and seeing if you made a difference is very quick and easy without committing to the 'finish' - although I am learning that the Report file saved automatically after hitting 'finish' is VERY USEFUL for remembering what the heck you did.
Title: Re: How to sum up few spectra
Post by: sem-geologist on March 31, 2021, 02:13:51 AM
Gagarinite by EDS looks nearly impossible, that is due to La M lines overlapping on the F (major peak), and I can't find any metal La standards to buy (available LaF3 and La2O3, or what we have in lab: La-glass, LaPO4, or LaP4O12 (or something like that)) - none of those has La M lines without nearby other element peaks or overlaps. I am not giving up, I think I could try removing Oxygen peak from my LaPO4 spectra and feed that artificially Oxygen-less spectra as a reference, as it needs only the shape of those lines. I probably should do the same for all LREE references as I see the DTSA-II auto reference for those lines does not work satisfactory (see attachement), getting those right is important also for correct overlap correction of Na. I should say that on WDS (PC0 XTAL for F, and TAP for Na) the situation is similarly over complicated and background is impossible to be measured accurately for this set of elements on WDS (maybe MAN of ProbeSoftware would cut through this problem, but I have none to try it out). Oh, but that is not the worst case - we also have some phases with this set of elements + Ba (with its M lines... and oxygen).

I am wondering why is this like that:

I understand that this is really hard, and now I very clearly see why we need to rely on the reference peaks.
Title: Re: How to sum up few spectra
Post by: sem-geologist on March 31, 2021, 03:17:16 AM
sorry for post after post...
But maybe this could benefit from my ideas. I see that calibration alien uses only single element for calibration. How then DTSA-II would know what peak broadening factor at given energy is? it is not only characteristic peaks which are broadened by pulse counter gain noise, but all continuum (that includes absorption edges). And most EDS by different vendors have non-linear broadening (as other noise gets more important at low energies). This could be measured and calibrated (as vendors does not give the factor how broadening changes) from 3 peaks instead of single (Mn Ka) - in example C, F (from CaF2) and Mn. This could then help in better auto reference fitting. The same is applicable for absorption edges, with better assessed broadening factor for given energy there should be less of artifacts at residual spectra at lower energies. It looks for me that they are resulting not only from peak shifts, but by too sharp absorption edge models applied.

P.S. I would contribute some code if this would be in python (maybe You are aware - I was doing some work on HyperSpy), unfortunately my brain is not Java friendly  ;) .

Getting back to my complaint for not reading/loading working distance from spx - that is rather correct behaviour. SEM often uses beam focus with physical working distance interchangeably. There is no problem if beam is fully focused, but in case if dealing with the beam-sensitive material, with defocused beam WD is reported incorrectly. It is better to fill this by hand.
Title: Re: How to sum up few spectra
Post by: Nicholas Ritchie on March 31, 2021, 04:03:16 AM
DTSA-II uses an extension of the "Fiori model" to estimate the resolution of an EDS detector as a function of energy. (See Goldstein) The "Calibration Alien" fits a high energy peak and a low energy peak and uses this to estimate the "noise" term in the resolution model. The standard is to report the resolution at Mn Ka but this doesn't mean I can't compute it at other energies.

The artifacts at low energies result from two classes of effects - first, the continuum has real edge structure at low energies.  Second, there are numerous small "chemical" or "solid state" effects that lead to changes in X-ray energy, X-ray weights-of-lines, mass absorption coefficient, fluorescence yield, Coster-Kronig and other factors.  These factors lead to the structure seen in the residual under low energy peaks.  It is physics not a shortcoming of the fitting process.

Finally, I assume that the image is in focus and the focal distance is the working distance.  I suggest setting the focal distance once (I always use 17 mm in my instrument) and then using the Z-axis of the stage to keep the sample in focus.  This maintains the constant working distance as the optical microscope does for WDS.  If you don't maintain the same working distance, your quant will not work out correctly just as in WDS.

I'd love help.  If you contribute a useful, novel algorithm in Python, I'd be happy to rewrite it in Java.
Title: Re: How to sum up few spectra
Post by: sem-geologist on March 31, 2021, 06:10:44 AM
As experienced WDS guy I had seen lots of shifts at low energy, however they are problematic on high spectral resolution portions of WDS position range (higher sin theta), where it is not so important on low sin theta portions of WDS spectra. Another thing is that most shifted are the element lines where oxidation state changes from one sign to the other (in.example Si in SiO2 bearing minerals (Si as positive) will have different position than NiSi (where Si is negative ion)), this is all those elements which can change its oxidation state. However I had never saw shifts for elements which do not change its state (i.e. Fluorine). Albeit, They do some funky thing like spectral broadening, depending from mineral composition and structure, but peak center is still on the same position. To overcome such funkiness for F I use PC0 XTAL where spectral resolution is worse (thus these funky effects are nearly completely canceled, and have no effect to peak amplitude in contrast to the TAP XTAL). And EDS broadening is nearly twice more than that of PC0 for F Ka so funky effects of energy shifts/blurring should be completely overcome by worse spectral resolution. Yes, this is what I like about EDS - the worse resolution is a blessing not a curse in many cases!
Title: Re: How to sum up few spectra
Post by: sem-geologist on March 31, 2021, 07:41:12 PM
Quote(See Goldstein)
is it that one?:
J. I. Goldstein et al., Scanning Electron Microscopy and X-ray Microanalysis, 2003

The EDS part is very informative, I had revisited that at the moment and learned something new. I came across that publication a while ago while looking for more detailed information about WDS, and just tossed that book away... due to WDS part... which is incomplete, many wrong/conflicting numbers, outdated (at least few decades as for 2003), lacks critical view, states few misconceptions... That publication presents WDS as superior to EDS  – which is not (If it would be, I probably would be not looking to DTSA-II to solve some complex issues, which is hardly solvable on most "modern" WDS). Anyway, It is good I had revisited it for EDS part, looks that authors knew that subject very well, and clearly had to get hands dirty with a hardware at some point (contrary to WDS).
Title: Re: How to sum up few spectra
Post by: Nicholas Ritchie on April 01, 2021, 05:18:49 AM
Or Goldstein 2017 aka SEM-XM 4 :)

I remain somewhat surprised by how much structure comes through in the residual despite the (relatively) poor resolution of the detector.
Title: Re: How to sum up few spectra
Post by: sem-geologist on April 01, 2021, 07:41:30 AM
Oh Goldsetein 2017 – an excellent book without WDS (just proves my point :) about WDS in Goldstein 2003).

Now I also can't understand why auto reference does so bad. If I use simulation  alien for same detector and simulate LaPO4, it does much better, albeit it simulates it with wider peaks than measured. (same goes for CePO4 and so on). I am a bit mixed up with window of detector, it shows in GUI it as  MOXTEK AP 5, while in file spx at coresponging XML field it is "SLEW AP 3.3" (guess I could use Moxtek AP 3.3 is it the same?). I think I had missed something important like calibration of detector? When I defined detector I had imported spectra, should I do calibration alien on Mn sample probably? Probably it is that as spx does contain no information about real width at Mn DTSA-II used some default values(?), that needs to be measured on real Mn. I am wondering how much mistakes I had conducted, and still I could get reasonable chevkinite composition. I am going to delete all resulting bundled standards and detectors and start a fresh.
Title: Re: How to sum up few spectra
Post by: Probeman on April 01, 2021, 11:27:04 AM
I don't want to get into an argument about WDS vs EDS because I think most reasonable people would agree that both methods have advantages and disadvantages with respect to each other.

Personally I think the use of integrated WDS and EDS provides the most flexible and efficacious approach depending on the particular analytical problem, e.g.,

https://smf.probesoftware.com/index.php?topic=79.msg7818#msg7818

In terms of mere popularity, yes, there will always be more SEM EDS labs compared with WDS EPMA labs. On the other hand, if one compares the number of SEM EDS labs that actually utilize standards with the number of WDS EPMA labs (who *must* use standards), the numbers are much closer I think.

:)
Title: Re: How to sum up few spectra
Post by: sem-geologist on April 01, 2021, 04:07:05 PM
Well I have such cases where WDS simple can't cut through .These Gagarinite i.e. that is Na(Ca,REE)2F6 (according to some oldy published data Na is 0.9 apfu, but I guess they just had overlooked the loss of Na). 5µm grains, not much space to put a point with reasonable defocus. Na just gets halved in a few seconds if beam touches that thing, even at 1.5nA (!). With every second the Na is being depleted. TDI? only the first elements, what to do with subsequent ones, and there is plenty to measure? Even for TDI to work reasonably it requires some sufficient statistics (enough counts). If I want better statistics, I need more (nA * s) of beam, but then I loose more of Na, so I would trade accuracy for precision. Accurate Fluorine?, while there is REE and some Ba present in the mineral, it is impossible to intercept the background under the F Ka in a traditional (two point) ways (Maybe MAN would cut through this, I don't know, this is low energy region). Until there are some spots where background can be measured the WDS is nice, but when there is too many elements in the mineral and there is not a single interference-free background spot – WDS is no more fun. Use PHA? Well, if it would be implemented in not half-assed way it could be helpful, but circuits for PHA (AFAIK) on Cameca EPMA's introduces enormous unnecessary energy broadening (yeah I know Goldstein et al times ago explained that broadening originates by processes in proportional... blah blah... what so ever... my physically connected to the raw signal oscilloscope is telling completely something else). How on the earth PHA in 5 volt range can't cleanly separate energy levels which differs by factor of 2 (lets say second order), while cheapo bargain low quality rubbish oscilloscope can distinguish effortlessly few levels of raw signal (in mV) captured in-between pre-amplifier circuitry and spectrometer VME board (before PHA). Then I mean "distinguish" I mean You could easily smuggle elephant in between those clearly different energy levels. I understand that this PHA circuits were "wow" in 80'ies (or as vendors loves this term "a state of art", I would insert "antique") as it had a clear edge over EDS counting, it could be even acceptable in SX100's at end of millennium and had a bit of faster counting than available EDS. But in 21st century EDS SDD's just had took this crown from WDS as EPMA vendors were just putting the same circuits into newest generation e-microprobes. What is maximum pulse throughtput on current WDS on cameca? Goldstein et al 2003 lied – contrary to stated 50kcps, WDS can go actually to a bit over 200kcps (You could do that in 2003 either with SX50 or SX100), but actually it should better not go much over 10kcps, even on a new SXFives, as above that counting rate unaccounted pulse-pileups kicks in, and there is no pileup correction. But what counting speed have to do with PHA? - it is the same bottleneck. The pulses are artificially shaped to live long enough so the "State of antique art" 8bit ADC can digitize amplitude of pulse (actually it is a bit more complicated). For comparison of speeds modern pre-amplifier is able to handle 8 million of pulses per second and such throughput is achievable with most modern EDS while not sacrificing precission

In my opinion to compete with SDD throughput vendors of EPMA should change current counting electronics and replace that with newest generation of preamplifiers which would feed signal directly to fast ADC (40MHz 12bit would be enough), which would feed that digitized signal to modern FPGAs (FPGAs have plenty of DSP cores, so could compute and output amplitude in near real-time). That would bring many improvement for WDS: practically no deadtime (unless using 10µA beam), really functional PHA witch would effortlessly distinguish second order peaks. For spectrometer control there would be left only bias settings – no more gain setting for spectrometer - it would lead to simplified proportional range optimization. No more PHA shifting from intensity level (As DSP's in FPGA would measure not absolute amplitude, but relative to the base level).

But I guess that won't happen, this requires some investments in R&D and we and our pockets are too little to matter. SEM and EDS is more used, so more developed, and used more because being actively developed.

... Oh damn this PHA, I was writing about Gagarinite analytical difficulties.
So on SEM and EDS there is huge difference. On SEM the EDS detectors are close to the sample and so even with lower current (~0.5nA) I get more counts per element than with a bit higher current on EPMA WDS. (at 0.5nA I am getting ~32kcps, so spectral lines are getting around 1k-5kcps).

I agree that combined WDS EDS is good way to go, but that is not a panacea for all problems. There are situations where EDS alone will perform better than WDS, or combining WDS with EDS does not brings anything additional to EDS-alone technique.
Title: Re: How to sum up few spectra
Post by: Probeman on April 01, 2021, 04:44:48 PM
As I stated in my post: "depending on the particular analytical problem".   :)

Also I hope you realize that if one has a WDS system and an EDS system on their EPMA (or SEM), they can utilize both in an integrated manner, or utilize only WDS or only EDS.  I hope that is that flexible enough for you?   ;D
Title: Re: How to sum up few spectra
Post by: sem-geologist on April 06, 2021, 02:06:00 AM
QuoteAlso I hope you realize that if one has a WDS system and an EDS system on their EPMA (or SEM)...

As I started to play with DTSA-II I realize this very much. Using both EDS and WDS could improve throughput. Now question is how those well integrate. I had not used that on Cameca Peaksight, as that has very marginal integration, can be used with very simple phases and predefined set of elements. The problem is it have hard-written positions for background (looked so last time I inspected how does it works), there is no documentation how those are integrated (peaksight have two options "fullanalysis" and "deconvolution", but no explanation what it does). I think the way how DTSA-II handles the EDS spectra is the right way (no need to point where is background). And as far as I understand ProbeSoftware has its own implementation. Does ProbeSoftware deconvolution? does it needs to define background regions?

Now I am interested if DTSA-II calculations from k-ratios. I wish there would be possibility to make mixed analyses in DTSA-II. The last question is automation. It is ok to spend lots of time to do methodologically correctly bells and whistles for few (first) analyses of given mineral. But the beauty of EPMA is that when You setup how to measure and do recalculations for defined species/groups of minerals/materials, it can be highly automated for high throughput. I wish epq in DTSA-II would have better documentation. I think as DTSA-II have jython, it should be possible to make some server script (Need to test that there is no timeout for python scripts, as that should run basically forever in opened DTSA-II instance) which would accept k-ratios, spectras and instructions, then do calculations using epq and would return the results (two way communication through platform independent network sockets). I feel so spoiled with modern python environments, that this plain (no tab-completion, no hints) jython environment is so cumbersome to explore, and need to depend on fundamental python functions (like 'dir()', 'help()').