We are doing rapid screening and need to process lots of maps collected in probe image. Is there a way that we can read the raw intensities directly from the PRBIMG file. I can open one in a text editor and see the header and footer information, but the row data look like nonsense. Is there anyway we can convert this to something meaningful?
Thanks,
-Dan
Example:
Row605=AECXRACgB0UAIBlFAAAgRQDgJkUAQDBFAIA7RQDAKEUAACVFAGA4RQDAQUUAwC1FAKA+RQCgSEUAQCZFAKAWRQAAOUUA4CFFAGAaRQBANUUAIBlFACAZRQBAU0UAoCpFAMAoRQAAJUUAAC9FACAeRQBgM0UAIEtFAKAqRQAgS0UAwEFFAEBTRQBgM0UA4DpFAGA9RQDAPEUA4FhFAMAyRQAgGUUAoDRFAEArRQCgQ0UAQDpFAIATRQBAP0UAQDVFAMBLRQBgM0UAwF9FACBuRQBAXUUA4F1FAGBbRQBAWEUAIHhFAOB7RQDAUEUA4GJFAKBDRQAgMkUA4DVFAMA8RQBgGkUAwGRFAGBHRQDgJkUA4BxFAKBDRQDAGUUAQBxFAMAtRQDgMEUAIB5FAGAkRQCgIEUAoBtFAMAURQAAG0UA4CZFAOAmRQDALUUAIDJFAGAuRQBAJkUAACBFAGAuRQDgHEUAYEJFAEAXRQCgJUUAwChFAEA/RQCgOUUAABtFACAtRQCgQ0UAIDJFAKA0RQDgSUUAAGFFAKBSRQBAXUUA4F1FAEBnRQCgUkUAAEhFAGAzRQAAPkUAoEhFAGApRQBAF0UA4ElFAIBARQDARkUA4CZFAGAuRQBgLkUAYDhFAOA1RQDAKEUAgDFFAKAlRQCAMUUAAD5FAAAqRQDgP0UAoEhFAABrRQCAaEUAYEJFAIBURQCgV0UAQERFAMAURQBAF0UA4BxFAKA0RQAgLUUAoBZFAGAVRQBAF0UAQDBFAAAbRQBgBkUAYClFAKAqRQCgG0UAQDVFAGA4RQCAQEUAIBRFAGAQRQAAEUUAABZFAEAXRQAgFEUAwEFFAGBMRQBgQkUAYFtFAGBvRQDgYkUA4ERFAEBYRQDAVUUAIEZFAOBTRQBAU0UAoGFFAGAuRQBAREUAAFdFAGBMRQBgOEUAYEdFAEA/RQCgSEUAgCxFAGApRQBAMEUAIC1FACAyRQCgUkUAgEpFAGAkRQAANEUAQDpFAEA/RQDAN0UAoEhFACBkRQCgcEUAQF1FAABIRQAgeEUAAJFFAGCLRQAgfUUA4IhFAICERQBgakUAwFBFAIBPRQCgSEUAgFlFAOBERQCgTUUAIEFFAABNRQDgK0UA4CtFAKA+RQDAQUUAQCtFACBBRQDAPEUA4FhFAEBTRQBgTEUAAHBFAPCHRQBgb0UAAFJFAABNRQCgPkUAwF9FAAA5RQAANEUAIFVFACBLRQBgPUUAwDJFAEA1RQCALEUAIDxFAEBERQCgUkUAAEhFAIBKRQAgbkUAQDBFAOBTRQDgK0UA4ERFAIBoRQAAUkUA8IdFAEBTRQAAPkUAoGZFAEBsRQCQiEUAAGFFAGB5RQCAfEUA8IJFAKB6RQDwh0UA0J1FAJCXRQBQpUUAoK9FAKClRQBAukUAEKlFAAC5RQCQoUUAoKpFAICnRQCws0UAAL5FAECmRQBgmkUAkJJFADCJRQCwhkUA0IlFAICdRQCAp0UAwKhFANCxRQAAuUUAoK9FAJCcRQDgpkUAgKJFAACMRQDAikUAkJJFABCGRQDgkkUAAH9FAGCBRQAgj0UAMIlFAGCfRQDQjkUAEJpFAHCtRQAAr0UAoMhFAJCwRQBwrUUAcJ5FABCQRQBQjEUAQHZFACB9RQDwgkUAgHxFAECSRQBAg0UAwI9FABCVRQBQkUUAIJ5FAODERQCA3kUAEOVFAFDrRQCw20UAINBFABCzRQAAm0UAIJRFAMCFRQCgf0UAgHJFAOCDRQBgmkUAEKRFAECrRQBApkUAgKJFAFCWRQDQk0UAwH1FACB9RQBgeUUAAHpFAICJRQDgkkUA8JtFADCxRQAAr0UA0MBFAMCyRQBQw0UAEMxFAODiRQDw+kUA8OZFAMDkRQCA1EUAIO5FABDlRQAA10UAwMZFANDZRQCgzUUAkNhFAJDERQAwykUAUMNFAJDORQDAvEUAIMZFACDBRQDwvkUAUNdFAGDMRQBwy0UA4LVFAKDIRQBgzEUAAMhFALDHRQAgy0UAgMVFAAC0RQCAsUUAUKVFALCfRQAwokUAoJFFAMCPRQAQpEUAYLhFACDLRQCw4EUAwORFAPD6RQCg/0UA0OhFAMDGRQCQv0UAMMBFAFDDRQBAzkUAcLJFAIDARQAA0kUAcMtFAJC6RQCQ00UAEMxFAIDARQDAxkUAYLhFACDBRQDQsUUA4L9FACDQRQDgsEUAsMJFACDQRQBQvkUAENZFAEDERQAwz0UAILxFAIC7RQDg2EUAoMNFAFDSRQAA0kUAMMBFAHDLRQDgxEUAwMtFAGDWRQBQ3EUAQM5FAJDORQCgw0UAENtFAFDIRQAwxUUA4L9FANCxRQDAxkUAcLxFAFDDRQDQ1EUA0MBFABDHRQCAz0UAELNFAMCoRQBwnkUAMJ1FAJCcRQAwykUA0PJFADD3RQBQ60UA4PZFADD3RQCg5kUA0LtFAHCyRQBQyEUAwO5FADDoRQDwuUUAgMVFAOC6RQAwykUAQMRFABDMRQCwx0UA0M9FABDMRQBgx0UAAM1FAPDIRQAQzEUA0MBFAFDSRQDgyUUAoNJFAMDkRQDQ40UA0ONFABDMRQCAykUAAMNFANCxRQBwskUAgMBFAGDWRQCAz0UA4OdFANDZRQDQykUA8MNFACCyRQAQuEUA0NRFAJDERQBwwUUAIMFFANDURQAAw0UAILxFAEC6RQAA10UAAM1FAJDJRQAA0kUA0MBFAIDARQDAy0UAENZFALDMRQAAw0UAIMFFAAC0RQBAukUAsMxFABC9RQDw10UAcMtFAFDDRQDAt0UAILxFANCxRQBQpUUAkKFFAIDARQDgzkUAoNJFAIDKRQAgy0UAEMxFADDjRQDQ2UUAIMFFADCORQAQn0UAAHpFAECIRQAApUUAQKZFANCTRQAQs0UAwMtFABDlRQCQ50UAcNVFAGDMRQCQxEUAEMJFADDKRQBwy0UAEMdFACDLRQBgzEUAUMNFAGC4RQAgskUAcMFFACDBRQCgw0UAgMpFADDPRQAQzEUA8M1FAEDORQCg0kUA8NJFAMC8RQBAyUUA4M5FALDRRQCwwkUAULlFAOCmRQBApkUA8JFFAPCRRQCAsUUAQL9FAECwRQDAskUA4LpFAEC6RQDg00UAsMJFAGDHRQBAyUUAUK9FAPC+RQDgxEUAYMdFACC8RQAAvkUAgLtFAJC/RQBAv0UAULRFAPDIRQBQuUUAcMFF
Hi Dan,
The pixel values in the raw x-ray intensity PrbImg files are encoded in Base64:
https://en.wikipedia.org/wiki/Base64
The values are in cps (not normalized to beam current).
There are several possibilities for you:
1. Write your own Base64 decoder using a library such as MatLab.
2. Use my VB code to decode the data. See files Base64Coder.bas and Base64Reader.bas on the Openmicroanalysis site:
https://github.com/openmicroanalysis/calczaf
3. Use the menu in CalcImage to load a PrbImg file, and the app will automatically create a .GRD file which is much easier to read.
Does any of this help?
john
That definitely helps! Thanks!
Here is what I found for MATLAB.
The base64decode.m file can be found here:
https://www.mathworks.com/matlabcentral/fileexchange/39526-byte-encoding-utilities?focused=3773211&tab=function
Once this code is run, you will need to run one additional function to convert the data, so the script looks like this:
packed_data=base64decode(raw_data);
unpacked_data = typecast(packed_data,'single');
Dan
Hi,
We also like to process directly in MATLAB data raw cps data for mapping. The ASCII format given by PeakSight was very handy: one could quickly check in the text editor some of the main acquisition parameters and the matrix of raw counts was directly visible and easily uploadable in MATLAB. Now we moved to probesoftware and I have difficulties with the loading of PRBIMG in MATLAB. I quickly tried that the base64decode according to the instructions of Dan Ruscitto but at first sight, it seems that a bit more scripting will be necessary to handle the header and extract an actual matrix of cps. If anybody is willing to share a working Matlab script (using the base64decode) to extract the matrix of cps, I am very much interested.
Rémi
You might find it easier to use CalcImage to convert the PrbImg files into GRD files. Once they are in GRD files, you can get the intensities (in cps) exported in Surfer to ASCII.
I could also make an ASCII export option menu in CalcImage. Would you like a "Convert all PrbImg Files To ASCII" menu? What ASCII format would be best for your work? Would you like other information in the ASCII file, or just the intensities?
I'm sure you have your own (nuclear specific!) reasons for performing the quant in MatLab as opposed to CalcImage, but I'm curious as to exactly why. What exactly are you doing in MatLab that CalcImage isn't doing?
Hi Remi,
I don't know if this will do what you need, but I just added a new menu to CalcImage to convert all PrbImg files in a selected folder to ASCII (tab delimited) files. These ASCII files can be easily loaded into, say Excel or MatLab, to obtain the x-ray map intensities.
To find this feature, open CalcImage (no need to load a project), and click on the Window | Log Window menu. Then select the new menu as shown here:
(https://smf.probesoftware.com/gallery/1_14_02_18_8_46_56.png)
It will ask you to select one PrbImg file and then when you click Open it will read and convert all the map intensities for all .PrbImg files in that folder into ASCII files with the same name, but with a .DAT extension.
Right now I'm just putting the file name of the original PrbImg file in the top and then the x by y pixel intensities as seen here:
"C:\UserImages\Roberts\Pos_0001\PrbImg\171117 cassiterites_00001_WDS1_Sn_PETJ_.PrbImg"
46.666668 43.333332 43.333332 30.000000 26.666666 23.333334 36.666668
56.666668 33.333332 43.333332 16.666666 40.000000 30.000000 40.000000
43.333332 20.000000 40.000000 33.333332 46.666668 33.333332 36.666668
40.000000 40.000000 36.666668 23.333334 63.333332 46.666668 23.333334
46.666668 30.000000 56.666668 33.333332 40.000000 56.666668 36.666668
33.333332 23.333334 36.666668 46.666668 50.000000 60.000000 33.333332
30.000000 36.666668 40.000000 50.000000 26.666666 16.666666 60.000000
16.666666 30.000000 33.333332 46.666668 23.333334 26.666666 63.333332
46.666668 33.333332 30.000000 26.666666 50.000000 30.000000 40.000000
etc.
I can re-format this output to whatever works best for you, but it's a start at least.
Hi John,
"Convert All PrbImg Files in Folder to ASCII" is excellent.
I can now open them in ImageJ using open TextImage.
Two things
(1) Any chance you can remove the first line [the file name] for I have to delete it before opening it in ImageJ?
(2) It be really good if you could do this for grd files as well.
Thanks
Ben
Quote from: Ben Buse on February 19, 2018, 03:34:39 AM
"Convert All PrbImg Files in Folder to ASCII" is excellent.
I can now open them in ImageJ using open TextImage.
Two things
(1) Any chance you can remove the first line [the file name] for I have to delete it before opening it in ImageJ?
(2) It be really good if you could do this for grd files as well.
Hi Ben,
I can certainly remove the filename at the beginning of the file.
On (2), you want all PrbImg files converted to GRD as well? Should that be a separate menu or do you always want ASCII text and GRD files output together?
john
Quote from: John Donovan on February 19, 2018, 08:05:20 AM
On (2), you want all PrbImg files converted to GRD as well? Should that be a separate menu or do you always want ASCII text and GRD files output together?
john
Hi John,
No what I meant was grd -> ASCII. So I can open the quant maps in imagej - would this be possible?
Thanks
Ben
Quote from: Ben Buse on February 19, 2018, 08:20:44 AM
Quote from: John Donovan on February 19, 2018, 08:05:20 AM
On (2), you want all PrbImg files converted to GRD as well? Should that be a separate menu or do you always want ASCII text and GRD files output together?
john
Hi John,
No what I meant was grd -> ASCII. So I can open the quant maps in imagej - would this be possible?
Thanks
Ben
Ah, OK. I'll make a separate menu for GRD to ASCII conversion.
One more question: I'll remove the filename at the beginning of the ASCII file, but would it be helpful to include the X/Y pixel dimensions in the ASCII files? That is the first line could be "800 <tab> 600" indicating that the map is 800 pixels wide by 600 pixels high...
Since Remi originally requested this I hope this is helpful for him as well.
john
Not for imagej import (text image) - you don't want any first line header
That be great
Thanks
Ben
Quote from: Ben Buse on February 19, 2018, 10:58:04 AM
Not for imagej import (text image) - you don't want any first line header
That be great
Thanks
Ben
In that case maybe I'll make the file with a .txt extension with just the intensities? Then I can also make the .dat file with some header information.
Dear John,
Thanks for adding this extension in extension in CalcImage. It certainly makes export easier.
The reason we are using directly count maps in Matlab is that we have a developed an alternative procedure to quantify mapping not relying on the usual ZAF quantification method (which would be punishingly long for decent size mapping). MATLAB also allows us integrating other image processing steps very specific to our needs in a single automated script. We also do batch processing of multiple ASCII files. We still need to explore the possibilities of CalcImage since we only recently implemented the full ProbeSoftware on our machine.
Having a header in the ASCII/txt file is for us not an issue since most advanced soft can easily discriminate it from the data. Experience tells us that is actually quite handy to have a .txt file containing the data and a header that one can quickly open to check main conditions of acquisitions. That allows us to spot rather quickly possible mistakes in the acquisitions settings. A scripting program like MATLAB can also automatically extract some data from the header that may be useful for the processing. In our case, we need for example to normalize for the beam current so the script get the current used for the map in the header itself. Below an example of the header that PeakSight generates when data is exported to a .txt file. As you can see most basic acquisition parameters are included. Something similar would of course be very nice to have as an export from the ProbeSoftware suite.
FileName : no name (Current dataset : 1)
Signal(s) Used : Vs1 SE , Vs2 BSE Z, Cr Ka, O Ka, Pu Mb, U Ma
Spectromers Conditions : Sp1 PET, Sp2 PC1, Sp3 PET, Sp4 PET
Full Spectromers Conditions : Sp1 PET(2d= 8.75,K= 0.000144), Sp2 PC1(2d= 60,K= 0.01483), Sp3 PET(2d= 8.75,K= 0.000144), Sp4 PET(2d= 8.75,K= 0.000144)
Column Conditions : Cond 1 : 20.2keV 202.6123nA
Date : 16-oct-2017
User Name : sx
Setup Name : Bv 20170831.impSet
DataSet Comment : Image Mosaic
Comment :
Analysis Date : 16/10/2017 09:28:49
Project Name :
Sample Name :
Analysis Parameters :
Sp Elements Xtal Position Bias Gain Dtime Blin Wind Mode
Sp1 Cr Ka PET 26190 1318 1042 3 2124 2876 Auto
Sp2 O Ka PC1 39940 1510 770 3 486 4514 Auto
Sp3 Pu Mb PET 40077 1847 982 3 1132 2726 Auto
Sp4 U Ma PET 44667 1849 988 3 946 2584 Auto
Peak Position : Sp1 26190, Sp2 39940, Sp3 40077, Sp4 44667
Start : X = -3300 Y = 130 Z = -218
Stop :
Dwell Time : 0.02 Sec
Acquisition type : Stage Grid
Step Number : 1024
Line Number : 1024
Beam Size : 0 µm
Step Size : 1.
Y Unit : cts
Image : Pu Mb
Hi Remi,
There are many ways to get this sort of information out of Probe for EPMA and Probe Image.
For example one can simply open the PrbImg file in any text editor and all the conditions for that map are there in ASCII. The following is a sample of some of that information (it's just a Windows INI file format):
[ProbeImage]
ImageFile=FeCu_Sulfide-1_00050_SP1_Si_PET_.tif
Type=prbimg
Version=1.2.0
Notes=
Probe=Demo Microprobe JEOL
InstrumentModel=Demo
ScanOrientation=Cartesian
ScanType=Beam
[BasicInfo]
Author=
Computer=
Description=
Make=
Model=
Organization=
[UUID]
Application=d9f25482-54b8-4f10-a4ca-cffcee884cfe
ImageSet=d3021183-b051-4958-ad6a-f5187090f4a7
Image=ecfe46be-29a0-4434-a342-7ca13e4502fa
[Registration]
X1Pixel=63
Y1Pixel=63
X2Pixel=0
Y2Pixel=0
X3Pixel=63
Y3Pixel=0
X1Real=48.9564
Y1Real=49.0423
Z1Real=8
X2Real=49.0423
Y2Real=48.9564
Z2Real=8
Z3Real=8
[ColumnConditions]
BeamCurrent=1e-008
BeamSize=1
ChannelNumber=1
ChannelLabel=SP1
Crystal=1
CrystalName=PET
Element=14
HighVoltage=15
Line=1
LineName=Ka
NegOffset=-4.41
Magnification=4353
OffsetType=0
PixelTime=20
PosOffset=4.411
Position=227.844
SignalName=SP1
ScanRotation=2.21175
TakeoffAngle=40
Bias=1674
Baseline=1
Deadtime=0
Gain=32
Mode=0
Window=9
For a summary of the map setup for all the mapped elements, since the map setup is based on the sample setup in Probe for EPMA, simply use the Report button in the Analyze! window. Here is a post that discusses this report output:
http://smf.probesoftware.com/index.php?topic=42.msg5982#msg5982
The Report information can also be exported to Excel. An example of this sample setup output in tabular format is attached below in both ASCII (.DAT) and Excel (.XLXS) formats.
Later today I will post about the new export menus I've added to CalcImage along the lines that you and Ben requested.
Hi Remi and Ben,
I added several new file conversion menus to the CalcImage "log window" as seen here:
(https://smf.probesoftware.com/gallery/1_20_02_18_1_22_16.png)
Basically for the conversion to ASCII, the .TXT file contains only the intensities as tab delimited values. The .DAT file contains a number of useful values in addition to the tab delimited intensities as seen here in this example:
"C:\UserImages\Carpenter\Wi-108-2b_00423_SP1_Na_TAP_.prbimg"
800 800
15 0.05
100.05 100.05
14.331 11.934 53.6757 51.2787
Where the first line is the full path to the file, the second line is the X and Y pixel dimensions, the third line is the keV and pixel dwell time in seconds, the fourth line is the before and after beam current in nA, and the fifth line is the stage extents of the map in millimeters (XMIN, XMAX, YMIN, YMAX).
The latest version of PFE (update from the Help menu), contains these new CalcImage menus. Please let me know if you need any additional changes to this output.