Hi,
We all love CalcImage, and the beautiful maps extracted using the software Surfer from Golden Software is great... However, I found many of my users not having access to Surfer, and either they have to treat their maps using my reprocessing computer, or finding a third party software to do the trick. John suggested that ImageJ should be able to read GRD files, but it would be even better if CalcImage could have an image exportation tool in BMP, or even better in TIF format.
Even more ideally, this little image exportation tool should come with some minimal ability to change the threshold on the intensity of the map (or the weight-% or the atomic proportion...), and to do other basic things, notably choosing the gradient color and doing some minimal math on the maps (what can already be done in CalcImage, but what will generate another DAT or GRD file, and not an actual image if I am correct...).
I envision the following:
- Have some kind of routine that allows the user to constrain the minimum & maximum intensity (Z) value. The value will be either an intensity or a weight-% value, so this parameter will have to be a float number.
- There could be additional options, like the color scale (grayscale, rainbow... you already have these options in the software)... Maybe there are packages similar to the graph options in PfE that could be integrated into CalcImage?
- Once the user has set these values, the Z-value is scaled back from 0 to 255, 0 being the minimum intensity the user entered and 255 being the maximum. You could even think about have a toggle box for the log scale output (for when you are mapping an element present as minor in one phase and major in the other...).
- Export these values in a TIF or BMP file.
Of course, there should also be a batch option to allow for the treatment of all element maps at once. Maybe the threshold value could be saved in a little text file, and be recalled when the batch process is activated?
BTW, it would be good one day to move away from that decaying Bitmap format. Way to heavy. TIF is the best way to go, you can even use lossless compression and end with files 4-5x lighter than BMP. Something for Probe for EPMA version 20 - The Golden Xtreme Plus Edition :D ?
I honestly have no idea how you generate images through a script in VB or so, so maybe what I personally think is simple in my mind might be a hassle to code...
Cheers,
Julien
Hi Julien,
What I've been doing is converting the *.GRD files to 16 bit TIFFS and working with them in imagej. This works well because the pixel values of the 16 bit images are the real data (i.e. oxide wt%, detetction limit etc). Once I've got the data in imagej I can do all the manipulation and whatever else I want to do. My favourite is to throw all the quant maps into a stack and use a quick macro to calculate average compositions for a selected area. I've been using imagej since it first became available and I'm much happier using it
The conversion is done using an R script I've cobbled together from some code initially posted to the forum by Ben Buse, I can send it to you if you like.
Cheers
Glenn
Quote from: glennpoirier on December 12, 2017, 05:50:24 AM
Hi Julien,
What I've been doing is converting the *.GRD files to 16 bit TIFFS and working with them in imagej. This works well because the pixel values of the 16 bit images are the real data (i.e. oxide wt%, detetction limit etc). Once I've got the data in imagej I can do all the manipulation and whatever else I want to do. My favourite is to throw all the quant maps into a stack and use a quick macro to calculate average compositions for a selected area. I've been using imagej since it first became available and I'm much happier using it
The conversion is done using an R script I've cobbled together from some code initially posted to the forum by Ben Buse, I can send it to you if you like.
Hi Glenn,
Your R script converts the GRD files to 16 bit TIFF? Does it convert by file or by folder?
Can you post it here?
john
Hi John,
It converts by folder and produces 32 bit tiffs (not 16 like I said earlier) as well as PNGs with a colour scale. It does what I need and nothing else so please don't laugh at my code ;)
The script is meant to run in RStudio, you could probably run it in the R command line but I haven't tried. You'll need to install the sp, raster and rasterVis packages.
Set the working directory in RStudio to the directory containing the folders you want to convert and run the code. If it doesn't work the first time try again, the first run occasionally doesn't work (library issues?). I haven't lost any data but you should be careful.
Obviously this is an absolute minimal piece of code, if anyone makes improvements please post them! I'd especially like to know why the levelplot function doesn't work.
Cheers
Glenn
PS: If anyone has succeded in opening GRD file in ImageJ I'd love to know how.
There are also scripts for surfer's scripter floating around in the forum that convert whole directories of grd files to imageJ readable formats with real data (counts) in them. Easy to convert and then everything can be done in imageJ.
I don't know if this is any help, but attached below is the VB code I use to read and write Surfer GRD files.
Note that it supports both the older (up to Surfer 6) format and the newer (Surfer 7 and higher) GRD formats. We can all probably ignore the older format as I doubt that anyone is still running Surfer 6 or older...
Quote from: Julien on December 11, 2017, 01:34:29 PMHi,
We all love CalcImage, and the beautiful maps extracted using the software Surfer from Golden Software is great... However, I found many of my users not having access to Surfer, and either they have to treat their maps using my reprocessing computer, or finding a third party software to do the trick. John suggested that ImageJ should be able to read GRD files, but it would be even better if CalcImage could have an image exportation tool in BMP, or even better in TIF format.
Even more ideally, this little image exportation tool should come with some minimal ability to change the threshold on the intensity of the map (or the weight-% or the atomic proportion...), and to do other basic things, notably choosing the gradient color and doing some minimal math on the maps (what can already be done in CalcImage, but what will generate another DAT or GRD file, and not an actual image if I am correct...).
I envision the following:
- Have some kind of routine that allows the user to constrain the minimum & maximum intensity (Z) value. The value will be either an intensity or a weight-% value, so this parameter will have to be a float number.
- There could be additional options, like the color scale (grayscale, rainbow... you already have these options in the software)... Maybe there are packages similar to the graph options in PfE that could be integrated into CalcImage?
- Once the user has set these values, the Z-value is scaled back from 0 to 255, 0 being the minimum intensity the user entered and 255 being the maximum. You could even think about have a toggle box for the log scale output (for when you are mapping an element present as minor in one phase and major in the other...).
- Export these values in a TIF or BMP file.
Of course, there should also be a batch option to allow for the treatment of all element maps at once. Maybe the threshold value could be saved in a little text file, and be recalled when the batch process is activated?
I honestly have no idea how you generate images through a script in VB or so, so maybe what I personally think is simple in my mind might be a hassle to code...
Julien (and several others) have requested an image export option from CalcImage that does not rely on Golden' Software's Surfer app.
Well, better late than never, we now have an option for those who don't have access to the Surfer application (though please remember, that students can get access to Surfer for only $50 per year)...
https://smf.probesoftware.com/index.php?topic=249.msg13408#msg13408
What we did was hire a developer (Mia Kraft) to write a small app that can take a "Classify" DAT file from CalcImage quantification and create images with XY scales and a data color scale. The app is called Classify Image Exporter and can also be run by itself or from a menu in CalcImage.
Here is the new menu in CalcImage v. 14.1.9:
(https://smf.probesoftware.com/gallery/1_15_05_25_6_11_37.png)
Note that one can select any "Classify" output file (oxide, atomic, formula, etc.) from CalcImage, and the Classify Image Exporter app will allow one to customize the XY tick marks, data scale and blanking color (for those maps with blanking values) and export them to PNG or JPG files:
(https://smf.probesoftware.com/gallery/1_15_05_25_6_13_52.png)
Here's an example of a 2K export:
(https://smf.probesoftware.com/gallery/1_15_05_25_6_12_01.png)
So please download v. 14.1.9 of Probe for EPMA and let us know what you think...
Hi John,
This is really nice, and will be very useful. It's great the way you can set max and min colour, title and subtitle before exporting.
Just as an extra note, you can change the file extension, or selected one from the drop-down menu during export (Just under the file name).
Currently supported file types are: PNG, JPEG, TIFF, WEBP, and BMP.
Color palettes are non-intuitive- at least to me!
When we first started our quant mapping software I created a color palette which we called rainbow.fc:
(https://smf.probesoftware.com/gallery/1_19_05_25_5_34_11.png)
Now one can point out, as many have, that we shouldn't even be using "rainbow" palettes in the first place, because they are not intuitive for a couple of reasons at least:
https://smf.probesoftware.com/index.php?topic=1003.0
First, if you have black on both ends of the palettes it can be hard to tell what color is high and what color is low. I choose to modify the typical rainbow palette and remove the black colors as seen here:
(https://smf.probesoftware.com/gallery/1_19_05_25_5_34_31.png)
But, probably because I'm such as nerd, this rainbow2.fc file has red as the low values and blue as the high values... because blue is a higher energy photon, right? Hey, it made sense to me at the time!
However, Golden Software has a different perspective. They made blue the low values and red the high values:
(https://smf.probesoftware.com/gallery/1_19_05_25_5_33_13.png)
and here is their rainbow2 CLR file:
(https://smf.probesoftware.com/gallery/1_19_05_25_5_33_48.png)
which I personally like because black is the low value and white is the high value, though it still has blue colors as "lower" than red values.
In the past, we defaulted the Surfer palette to use the rainbow2.clr file, because it seemed more intuitive, but starting with v. 16 of Golden Software's Surfer app, they stopped providing discrete .CLR files, except for the rainbow.clr file.
So when we run Surfer scripts they now default to rainbow.clr instead of rainbow2.clr. But now we've created a rainbow2.clr file (attached below) which you can copy to your v 16 Surfer application folder, usually C:\Program Files\Golden Software\Surfer\Samples. If you do this, the latest GridXY scripts in CalcImage will recognize this, and specify that palette instead for the color palette in Surfer.
Please note that the Classify Image Exporter application is not meant to replace Golden Software's Surfer app, as it is very customizable as shown here:
https://smf.probesoftware.com/index.php?topic=55.msg2107#msg2107
https://smf.probesoftware.com/index.php?topic=73.msg8998#msg8998
But for those that do not have a multiple user license for Surfer, it should be very helpful.
Thank you John, that's very interesting how the different colour palettes developed and what is a high intensity colour. Any chance anyone has the surfer rainbow and surfer rainbow2 as .fc files and could upload here?
Here's a python script for converting ImageJ lut tables (where the format is just 3 or 4 columns of numbers with or without header; not all the lut have the same format) into .fc
import pandas as pd
import tkinter
import tkinter.filedialog
lutlocation = tkinter.filedialog.askopenfile(title="select imagej lut")
savelocation = tkinter.filedialog.asksaveasfilename(title="select save location")
df = pd.read_csv(lutlocation.name, sep="\s+", header=None)
if df.iat[0,0] == 'Index':
df = pd.read_csv(lutlocation.name, sep="\s+")
f = open(savelocation,"a")
f.write(" False color description for MicroImage"+"\n")
f.write(" BEGIN Items"+"\n")
f.write(" Interpolate=0"+"\n")
for x in range(0,len(df)):
if len(df.columns) == 3:
r = df.iat[x,0]
g = df.iat[x,1]
b = df.iat[x,2]
if len(df.columns) == 4:
r = df.iat[x,1]
g = df.iat[x,2]
b = df.iat[x,3]
rgbdecimal = b * 65536 + g * 256 +r
f.write(" Item= "+str(x)+" "+str(x)+" "+str(rgbdecimal)+"\n")
f.write(" END Items")
f.close()
sepia
(https://smf.probesoftware.com/gallery/453_20_05_25_1_50_10.png)
jet
(https://smf.probesoftware.com/gallery/453_20_05_25_1_50_34.png)
001fire
(https://smf.probesoftware.com/gallery/453_20_05_25_2_45_19.png)
Note that with this Classify Image Exporter app, you can also export other quantitative data from CalcImage such as formula basis maps:
(https://smf.probesoftware.com/gallery/1_16_05_25_1_34_20.png)
Also note that display of blanked pixels is also supported such as this analytical sensitivity map:
(https://smf.probesoftware.com/gallery/1_16_05_25_1_34_04.png)
Quote from: Ben Buse on May 19, 2025, 11:46:39 PMThank you John, that's very interesting how the different colour palettes developed and what is a high intensity colour. Any chance anyone has the surfer rainbow and surfer rainbow2 as .fc files and could upload here?
Just FYI, one can save any CLR file from Surfer from the Color Map Editor dialog:
(https://smf.probesoftware.com/gallery/1_21_05_25_8_14_55.png)
Also note that all the Surfer color map palettes are described in the Surfer ColorMapPresets.toml file located in the C:\Program Files\Golden Software\Surfer\AppData folder:
(https://smf.probesoftware.com/gallery/1_19_05_25_5_35_04.png)
Finally note this KnowledgeBase article from Golden Software describing how to set a color scale by default in Surfer:
https://support.goldensoftware.com/hc/en-us/articles/226807128-Always-fill-contours-and-show-a-color-scale-by-default-in-Surfer
though note that this setting is over ridden by the CalcImage Surfer Scripts unless these lines in the scripts are commented out in the custom scrips, for example GridXY_Custom1.bas:
https://smf.probesoftware.com/index.php?topic=305.msg1567#msg1567
Quote from: Ben Buse on May 19, 2025, 11:46:39 PMAny chance anyone has the surfer rainbow and surfer rainbow2 as .fc files and could upload here?
Attached.
I've added the details to the instructions on my post here (https://smf.probesoftware.com/index.php?topic=1615.msg13458#msg13458) about how to create .FC files in R using any colours you want.
All you need to do is take the rgb values from John's tip about exporting colours from Surfer (above) and replace the "0x" with "#" to get the colour points e.g. "0xff6600" becomes "#ff6600"
Thanks JonF, that's good to know. And thanks for the files, together with the files in your other post. I'd have thought code only works if colors evenly spaced along color scale, if your simply stating color table a list of colors, which isn't quite true above. But now have plenty of color tables from R, from ImageJ, and from Python, and several good rainbow luts. (Interpolation between Irregularly spaced colors could be done using cdict in matplotlib in python, but don't feel currently inclined https://matplotlib.org/stable/api/_as_gen/matplotlib.colors.LinearSegmentedColormap.html (https://matplotlib.org/stable/api/_as_gen/matplotlib.colors.LinearSegmentedColormap.html))
Quote from: Ben Buse on May 21, 2025, 02:58:22 PMI'd have thought code only works if colors evenly spaced along color scale, if your simply stating color table a list of colors, which isn't quite true above.
Ah, good point.
I've created a new .FC file for the rainbow2 colour palette where the colours are the same layout as the ones in Surfer - and I actually opened Surfer to check this time!
(https://smf.probesoftware.com/gallery/796_22_05_25_1_39_45.jpeg)
Unevenly spaced rainbow2 colour palette file attached below, and the contents of the file can be opened in notepad/text editor to show this:
False color description for MicroImage
BEGIN Items
Interpolate=1
Item= 0 0 0
Item= 28 28 16711680
Item= 66 66 16776960
Item= 124 124 39168
Item= 150 150 65535
Item= 204 204 255
Item= 244 244 16777215
Item= 255 255 16777215
END ItemsI mostly did this one by hand because it was more effort coding it in!
All you need to do is:
- set interpolate to 1
- convert the fractional values that Surfer gives you (e.g. 0.5999999..) and multiply that by 255, then take the integer value of what comes back (0.59*255 = ~150) and put that in the first two columns
- convert the hexadecimal colour code (e.g. 0xff6600) to decimal - I used the col2rgb function in my R script described above, but there are plenty of websites that will do the same for you
Here's rainbow, too.
Also tested in Surfer to make sure it's the same ;D
That's nice and a great way of converting it
Very helpful, it makes it easy to write a generic conversion script to convert .clr to .fc
.clr file
ColorMap 1 1
0 153 102 255
20 0 0 255
40 0 255 0
60 255 255 0
80 255 102 0
100 255 0 0
becomes .fc file
False color description for MicroImage
BEGIN Items
Interpolate=1
Item= 0 0 16737945
Item= 20 20 16711680
Item= 40 40 65280
Item= 60 60 65535
Item= 80 80 26367
Item= 100 100 255
END Items
Where code
import pandas as pd
import tkinter
import tkinter.filedialog
lutlocation = tkinter.filedialog.askopenfile(title="select surfer .clr file")
savelocation = tkinter.filedialog.asksaveasfilename(title="select save location")
df = pd.read_csv(lutlocation.name, skiprows=1, sep="\s+", header=None)
f = open(savelocation,"a")
f.write(" False color description for MicroImage"+"\n")
f.write(" BEGIN Items"+"\n")
f.write(" Interpolate=1"+"\n")
for x in range(0,len(df)):
r = df.iat[x,1]
g = df.iat[x,2]
b = df.iat[x,3]
rgbdecimal = b * 65536 + g * 256 +r
f.write(" Item= "+str(df.iat[x,0])+" "+str(df.iat[x,0])+" "+str(rgbdecimal)+"\n")
f.write(" END Items")
f.close()
Or if instead of 0 to 100, recast it as 0 to 255
False color description for MicroImage
BEGIN Items
Interpolate=1
Item= 0.0 0.0 16737945
Item= 51.0 51.0 16711680
Item= 102.0 102.0 65280
Item= 153.0 153.0 65535
Item= 204.0 204.0 26367
Item= 255.0 255.0 255
END Items
And
import pandas as pd
import tkinter
import tkinter.filedialog
lutlocation = tkinter.filedialog.askopenfile(title="select surfer .clr file")
savelocation = tkinter.filedialog.asksaveasfilename(title="select save location")
df = pd.read_csv(lutlocation.name, skiprows=1, sep="\s+", header=None)
f = open(savelocation,"a")
f.write(" False color description for MicroImage"+"\n")
f.write(" BEGIN Items"+"\n")
f.write(" Interpolate=1"+"\n")
for x in range(0,len(df)):
r = df.iat[x,1]
g = df.iat[x,2]
b = df.iat[x,3]
rgbdecimal = b * 65536 + g * 256 +r
f.write(" Item= "+str(255*((df.iat[x,0])/100))+" "+str(255*((df.iat[x,0])/100))+" "+str(rgbdecimal)+"\n")
f.write(" END Items")
f.close()
Interestingly in the case of rainbow2 - interpolate in classify extract image works different than in surfer - giving a different appearance.
Because interpolate ignores the position number in .fc in classify extract image, and simply divides equally by the number of entries/rows in the order given.
So
False color description for MicroImage
BEGIN Items
Interpolate=1
Item= 0 0 16737945
Item= 10 10 255
Item= 255 255 255
END Items
gives the same appearance as
False color description for MicroImage
BEGIN Items
Interpolate=1
Item= 0 0 16737945
Item= 85 10 255
Item= 255 255 255
END Items
So to instead to avoid interpolation in classify extract image for non even spacing, the following does the interpolation in python matplotlib to convert surfer .clr to .fc, where the list of colours includes their position along the scale
import pandas as pd
import tkinter
import tkinter.filedialog
import matplotlib
lutlocation = tkinter.filedialog.askopenfile(title="select surfer .clr file")
savelocation = tkinter.filedialog.asksaveasfilename(title="select save location")
df = pd.read_csv(lutlocation.name, skiprows=1, sep="\s+", header=None)
f = open(savelocation,"a")
f.write(" False color description for MicroImage"+"\n")
f.write(" BEGIN Items"+"\n")
f.write(" Interpolate=0"+"\n")
storey = -1
testcolours=[]
#read clr file to generate color map using colours and their position along the scale
for x in range(0,(len(df)-0)):
test=(df.iat[x,1]/255,df.iat[x,2]/255,df.iat[x,3]/255)
test2=(df.iat[x,0]/100,test)
testcolours.append(test2)
#cmap=matplotlib.colors.LinearSegmentedColormap.from_list(name='testmap',colors=[(0,'red'),(1,'blue')],N=255)
cmap=matplotlib.colors.LinearSegmentedColormap.from_list(name='testmap',colors=testcolours,N=255)
norm = matplotlib.colors.Normalize(vmin=0, vmax=255)
#write a file from 0 to 255 with color from colour map
for x in range(0,255):
#cmap() returns rgba; [] specifies r or g or b or a; norm(x) is 0 to 255 given as 0 to 1 for color map
r = cmap(norm(x))[0]
g = cmap(norm(x))[1]
b = cmap(norm(x))[2]
#rgb are in range 0 to 1
rgbdecimal = int(b*255) * 65536 + int(g*255) * 256 + int(r*255)
f.write(" Item= "+str(x)+" "+str(x)+" "+str(rgbdecimal)+"\n")
f.write(" END Items")
f.close()
Here's rainbow and rainbow 2, and others in zip
The latest version of Probe for EPMA (v. 14.2.3) has a new version of the Classify Image Exporter (v.1.2.1) for exporting maps as images in the CalcImage software:
(https://smf.probesoftware.com/gallery/1_15_05_25_6_11_37.png)
In the new version Mia fixed a couple of minor bugs and improved support for wide or tall aspect ratio maps, by adding a separate field for the Y axis increment, for example as seen here:
(https://smf.probesoftware.com/gallery/1_10_07_25_6_54_26.png)
Having separate controls for the X and Y axes allows for better graphics especially when the maps are very tall or wide as seen here:
(https://smf.probesoftware.com/gallery/1_10_07_25_6_54_46.png)
Very nice, I'll have to give it a try. What controls the title font size?
Quote from: Ben Buse on July 10, 2025, 08:59:27 AMVery nice, I'll have to give it a try. What controls the title font size?
As it stands the font size is automatically sized and positioned based on the resolution and what I saw as similar to historical examples.
Quote from: Ben Buse on July 10, 2025, 08:59:27 AMVery nice, I'll have to give it a try. What controls the title font size?
As Mia said, the Classify Image Exporter app auto sizes all text, but if you want more control over image fonts and sizes, you should use the Surfer output options:
https://smf.probesoftware.com/index.php?topic=55.msg509#msg509
Basically you output your maps using the Surfer script option, run the script BAS file, then locate the .SRF file for that export and open it in Surfer and manually edit your maps as desired. Then export your images from Surfer.