Image export option for CalcImage

[Julien]

Hi,

We all love CalcImage, and the beautiful maps extracted using the software Surfer from Golden Software is great... However, I found many of my users not having access to Surfer, and either they have to treat their maps using my reprocessing computer, or finding a third party software to do the trick. John suggested that ImageJ should be able to read GRD files, but it would be even better if CalcImage could have an image exportation tool in BMP, or even better in TIF format.

Even more ideally, this little image exportation tool should come with some minimal ability to change the threshold on the intensity of the map (or the weight-% or the atomic proportion...), and to do other basic things, notably choosing the gradient color and doing some minimal math on the maps (what can already be done in CalcImage, but what will generate another DAT or GRD file, and not an actual image if I am correct...).

I envision the following:

• Have some kind of routine that allows the user to constrain the minimum & maximum intensity (Z) value. The value will be either an intensity or a weight-% value, so this parameter will have to be a float number.


• There could be additional options, like the color scale (grayscale, rainbow... you already have these options in the software)... Maybe there are packages similar to the graph options in PfE that could be integrated into CalcImage?


• Once the user has set these values, the Z-value is scaled back from 0 to 255, 0 being the minimum intensity the user entered and 255 being the maximum. You could even think about have a toggle box for the log scale output (for when you are mapping an element present as minor in one phase and major in the other...).


• Export these values in a TIF or BMP file.

Of course, there should also be a batch option to allow for the treatment of all element maps at once. Maybe the threshold value could be saved in a little text file, and be recalled when the batch process is activated?

BTW, it would be good one day to move away from that decaying Bitmap format. Way to heavy. TIF is the best way to go, you can even use lossless compression and end with files 4-5x lighter than BMP. Something for Probe for EPMA version 20 - The Golden Xtreme Plus Edition ?

I honestly have no idea how you generate images through a script in VB or so, so maybe what I personally think is simple in my mind might be a hassle to code...

Cheers,

Julien

[glennpoirier]

Hi Julien,
What I've been doing is converting the *.GRD files to  16 bit TIFFS and working with them in imagej. This works well because the pixel values of the 16 bit images are the real data (i.e. oxide wt%, detetction limit etc). Once I've got the data in imagej I can do all the manipulation and whatever else I want to do. My favourite is to throw all the quant maps into a stack and use a quick macro to calculate average compositions for a selected area. I've been using imagej since it first became available and I'm much happier using  it

The conversion is done using an R script I've cobbled together from some code initially posted to the forum by Ben Buse, I can send it to you if you like.


Cheers

Glenn

[John Donovan]

Hi Julien,
What I've been doing is converting the *.GRD files to  16 bit TIFFS and working with them in imagej. This works well because the pixel values of the 16 bit images are the real data (i.e. oxide wt%, detetction limit etc). Once I've got the data in imagej I can do all the manipulation and whatever else I want to do. My favourite is to throw all the quant maps into a stack and use a quick macro to calculate average compositions for a selected area. I've been using imagej since it first became available and I'm much happier using  it

The conversion is done using an R script I've cobbled together from some code initially posted to the forum by Ben Buse, I can send it to you if you like.

Hi Glenn,
Your R script converts the GRD files to 16 bit TIFF?   Does it convert by file or by folder?

Can you post it here?
john

[glennpoirier]

Hi John,
It converts by folder and produces 32 bit tiffs (not 16 like I said earlier) as well as PNGs with a colour scale. It does what I need and nothing else so please don't laugh at my code

The script is meant to run in RStudio, you could probably run it in the R command line but I haven't tried. You'll need to install the sp, raster and rasterVis packages.

Set the working directory in RStudio to the directory containing the folders you want to convert  and run the code. If it doesn't work the first time try again, the first run occasionally doesn't work (library issues?). I haven't lost any data but you should be careful.

Obviously this is an absolute minimal piece of code, if anyone makes improvements please post them! I'd especially like to know why the levelplot function doesn't work.

Cheers

Glenn

PS: If anyone has succeded in opening GRD file in ImageJ I'd love to know how.

[Philipp Poeml]

There are also scripts for surfer's scripter floating around in the forum that convert whole directories of grd files to imageJ readable formats with real data (counts) in them. Easy to convert and then everything can be done in imageJ.

[John Donovan]

I don't know if this is any help, but attached below is the VB code I use to read and write Surfer GRD files.

Note that it supports both the older (up to Surfer 6) format and the newer (Surfer 7 and higher) GRD formats.  We can all probably ignore the older format as I doubt that anyone is still running Surfer 6 or older...

[John Donovan]

Hi,

We all love CalcImage, and the beautiful maps extracted using the software Surfer from Golden Software is great... However, I found many of my users not having access to Surfer, and either they have to treat their maps using my reprocessing computer, or finding a third party software to do the trick. John suggested that ImageJ should be able to read GRD files, but it would be even better if CalcImage could have an image exportation tool in BMP, or even better in TIF format.

Even more ideally, this little image exportation tool should come with some minimal ability to change the threshold on the intensity of the map (or the weight-% or the atomic proportion...), and to do other basic things, notably choosing the gradient color and doing some minimal math on the maps (what can already be done in CalcImage, but what will generate another DAT or GRD file, and not an actual image if I am correct...).

I envision the following:

• Have some kind of routine that allows the user to constrain the minimum & maximum intensity (Z) value. The value will be either an intensity or a weight-% value, so this parameter will have to be a float number.
• There could be additional options, like the color scale (grayscale, rainbow... you already have these options in the software)... Maybe there are packages similar to the graph options in PfE that could be integrated into CalcImage?
• Once the user has set these values, the Z-value is scaled back from 0 to 255, 0 being the minimum intensity the user entered and 255 being the maximum. You could even think about have a toggle box for the log scale output (for when you are mapping an element present as minor in one phase and major in the other...).
• Export these values in a TIF or BMP file.

Of course, there should also be a batch option to allow for the treatment of all element maps at once. Maybe the threshold value could be saved in a little text file, and be recalled when the batch process is activated?

I honestly have no idea how you generate images through a script in VB or so, so maybe what I personally think is simple in my mind might be a hassle to code...

Julien (and several others) have requested an image export option from CalcImage that does not rely on Golden' Software's Surfer app.

Well, better late than never, we now have an option for those who don't have access to the Surfer application (though please remember, that students can get access to Surfer for only $50 per year)...

https://smf.probesoftware.com/index.php?topic=249.msg13408#msg13408

What we did was hire a developer (Mia Kraft) to write a small app that can take a "Classify" DAT file from CalcImage quantification and create images with XY scales and a data color scale. The app is called Classify Image Exporter and can also be run by itself or from a menu in CalcImage.

Here is the new menu in CalcImage v. 14.1.9:



Note that one can select any "Classify" output file (oxide, atomic, formula, etc.) from CalcImage, and the Classify Image Exporter app will allow one to customize the XY tick marks, data scale and blanking color (for those maps with blanking values) and export them to PNG or JPG files:



Here's an example of a 2K export:



So please download v. 14.1.9 of Probe for EPMA and let us know what you think...

[Ben Buse]

Hi John,

This is really nice, and will be very useful. It's great the way you can set max and min colour, title and subtitle before exporting.

[Mia Kraft]

Just as an extra note, you can change the file extension, or selected one from the drop-down menu during export (Just under the file name).

Currently supported file types are: PNG, JPEG, TIFF, WEBP, and BMP.

[John Donovan]

Color palettes are non-intuitive- at least to me!

When we first started our quant mapping software I created a color palette which we called rainbow.fc:



Now one can point out, as many have, that we shouldn't even be using "rainbow" palettes in the first place, because they are not intuitive for a couple of reasons at least:

https://smf.probesoftware.com/index.php?topic=1003.0

First, if you have black on both ends of the palettes it can be hard to tell what color is high and what color is low. I choose to modify the typical rainbow palette and remove the black colors as seen here:



But, probably because I'm such as nerd, this rainbow2.fc file has red as the low values and blue as the high values... because blue is a higher energy photon, right?  Hey, it made sense to me at the time!

However, Golden Software has a different perspective. They made blue the low values and red the high values:



and here is their rainbow2 CLR file:



which I personally like because black is the low value and white is the high value, though it still has blue colors as "lower" than red values.

In the past, we defaulted the Surfer palette to use the rainbow2.clr file, because it seemed more intuitive, but starting with v. 16 of Golden Software's Surfer app, they stopped providing discrete .CLR files, except for the rainbow.clr file.

So when we run Surfer scripts they now default to rainbow.clr instead of rainbow2.clr.  But now we've created a rainbow2.clr file (attached below) which you can copy to your v 16 Surfer application folder, usually C:\Program Files\Golden Software\Surfer\Samples. If you do this, the latest GridXY scripts in CalcImage will recognize this, and specify that palette instead for the color palette in Surfer.

Please note that the Classify Image Exporter application is not meant to replace Golden Software's Surfer app, as it is very customizable as shown here:

https://smf.probesoftware.com/index.php?topic=55.msg2107#msg2107

https://smf.probesoftware.com/index.php?topic=73.msg8998#msg8998

But for those that do not have a multiple user license for Surfer, it should be very helpful.

[Ben Buse]

Thank you John, that's very interesting how the different colour palettes developed and what is a high intensity colour. Any chance anyone has the surfer rainbow and surfer rainbow2 as .fc files and could upload here?

[Ben Buse]

Here's a python script for converting ImageJ lut tables (where the format is just 3 or 4 columns of numbers with or without header; not all the lut have the same format) into .fc

Code Select Expand

import pandas as pd
import tkinter
import tkinter.filedialog
lutlocation = tkinter.filedialog.askopenfile(title="select imagej lut")
savelocation = tkinter.filedialog.asksaveasfilename(title="select save location")
df = pd.read_csv(lutlocation.name, sep="\s+", header=None)
if df.iat[0,0] == 'Index':
    df = pd.read_csv(lutlocation.name, sep="\s+")
f = open(savelocation,"a")
f.write(" False color description for MicroImage"+"\n")
f.write(" BEGIN Items"+"\n")
f.write(" Interpolate=0"+"\n")
for x in range(0,len(df)):
    if len(df.columns) == 3:
        r = df.iat[x,0]
        g = df.iat[x,1]
        b = df.iat[x,2]
    if len(df.columns) == 4:
        r = df.iat[x,1]
        g = df.iat[x,2]
        b = df.iat[x,3]
    rgbdecimal = b * 65536 + g * 256 +r
    f.write(" Item= "+str(x)+" "+str(x)+" "+str(rgbdecimal)+"\n")
f.write(" END Items")
f.close()

sepia


jet


001fire

[John Donovan]

Note that with this Classify Image Exporter app, you can also export other quantitative data from CalcImage such as formula basis maps:



Also note that display of blanked pixels is also supported such as this analytical sensitivity map:

[John Donovan]

Thank you John, that's very interesting how the different colour palettes developed and what is a high intensity colour. Any chance anyone has the surfer rainbow and surfer rainbow2 as .fc files and could upload here?

Just FYI, one can save any CLR file from Surfer from the Color Map Editor dialog:



Also note that all the Surfer color map palettes are described in the Surfer ColorMapPresets.toml file located in the C:\Program Files\Golden Software\Surfer\AppData folder:



Finally note this KnowledgeBase article from Golden Software describing how to set a color scale by default in Surfer:

https://support.goldensoftware.com/hc/en-us/articles/226807128-Always-fill-contours-and-show-a-color-scale-by-default-in-Surfer

though note that this setting is over ridden by the CalcImage Surfer Scripts unless these lines in the scripts are commented out in the custom scrips, for example GridXY_Custom1.bas:

https://smf.probesoftware.com/index.php?topic=305.msg1567#msg1567

[JonF]

Any chance anyone has the surfer rainbow and surfer rainbow2 as .fc files and could upload here?

Attached.

I've added the details to the instructions on my post here about how to create .FC files in R using any colours you want.

All you need to do is take the rgb values from John's tip about exporting colours from Surfer (above) and replace the "0x" with "#" to get the colour points e.g. "0xff6600" becomes "#ff6600"