Hi all,
Today I came into the lab and noticed that the Thermo Pathfinder was not showing the correct mag or stage positions on our FEI Quanta instrument. So of course I re-started Pathfinder because if someone had re-started the FEI software while the Thermo software was running, it will break the DCOM connection. But when the Thermo Pathfinder re-started I got the error "Failed to Connect" and then "Connection Not made".
Now you might ask, why am I posting this in the PictureSnapApp board? Well, mostly because there is no FEI board on this forum, but also because when I start PictureSnapApp I also get the error "Permission Denied", so it seems the DCOM connection is now broken in general for all apps.
So I re-started the Quanta computer and software, but still no dice. I can ping the Quanta microscope computer at 192.168.0.1 from the Thermp?PSA computer fine, but both PictureSnapApp and Pathfinder refuse to connect to the FEI computer over the local 192.168.0.1 connection.
I should probably also re-start the Thermo/PSA computer but it's in the middle of a Monte Carlo calculation for a paper and I hate to interrupt it. Maybe tomorrow.
In the meantime, does anyone have any suggestions as to why the DCOM connection is suddenly broken, but the user logins for both computers are still the same?
Have there been any recent updates to the system(s)?
It is a long shot, but you might want to verify that DCOM is enabled on both systems.
Check:
HKEY_LOCAL_MACHINE\Software\Microsoft\OLE\EnableDCOM
Quote from: jrminter on August 28, 2018, 07:13:25 AM
Have there been any recent updates to the system(s)?
It is a long shot, but you might want to verify that DCOM is enabled on both systems.
Check:
HKEY_LOCAL_MACHINE\Software\Microsoft\OLE\EnableDCOM
Hi John,
John here. I checked both computers and both their registry entries show a "Y" in that field.
Still trying various things but no luck so far.
Quote from: UofO EPMA Lab on August 28, 2018, 10:20:20 AM
Quote from: jrminter on August 28, 2018, 07:13:25 AM
Have there been any recent updates to the system(s)?
It is a long shot, but you might want to verify that DCOM is enabled on both systems.
Check:
HKEY_LOCAL_MACHINE\Software\Microsoft\OLE\EnableDCOM
Hi John,
John here. I checked both computers and both their registry entries show a "Y" in that field.
Still trying various things but no luck so far.
Well I've tried re-running the configure.cmd file for the DCOM configuration and that worked, but only after I set the user access control to Never Notify (I really don't like that so many apps require that this get turned off- that's not a good thing!).
Then I uninstalled and re-installed the FEI DCOM installer msi, but when I run the FEI test DCOM connection app it says connection failed. As does the Thermo AutomationClient app and PictureSnapApp. In fact the PictureSnapApp reports "Permission Denied" so that information tells me that the FEI microscope computer is refusing the connection for some reason. I also tried from an admin account and adding the FEI test connection app through the firewall exceptions but still the DCOM connection isn't working.
But I can ping it and the user logins are the same on both computers, so I am at a loss...
It's so weird, it was all working fine last week.
I heard from Thermo that another system of theirs stopped working all of a sudden last week also.
I'm suspecting a recent Microsoft update is killing DCOM for some reason.
john
Is a system restore an option?
Quote from: jrminter on August 29, 2018, 02:40:11 PM
Is a system restore an option?
Only as a last resort.
I did notice that I'm getting a new error when the computer is rebooted:
APPCRASH: SupportAssistAgent.exe
Could this have anything to do with these DCOM issues?
john
This is a Dell PC, right? https://www.file.net/process/supportassistagent.exe.html (https://www.file.net/process/supportassistagent.exe.html) suggests that supportassistagent.exe is part of Dell SupportAssistAgent software. This makes me think the MS patch has caused a lot of collateral damage. https://www.computerworld.com/article/3290465/microsoft-windows/stung-by-a-festering-pile-of-bugs-on-patch-tuesday-ms-releases-27-more-patches.html (https://www.computerworld.com/article/3290465/microsoft-windows/stung-by-a-festering-pile-of-bugs-on-patch-tuesday-ms-releases-27-more-patches.html) suggests Microsoft patches have been pretty buggy lately... They claim corporate IT folks have been reluctant to install them... Sorry to be the bearer of bad news...
Quote from: jrminter on August 29, 2018, 06:00:13 PM
This is a Dell PC, right? https://www.file.net/process/supportassistagent.exe.html (https://www.file.net/process/supportassistagent.exe.html) suggests that supportassistagent.exe is part of Dell SupportAssistAgent software. This makes me think the MS patch has caused a lot of collateral damage. https://www.computerworld.com/article/3290465/microsoft-windows/stung-by-a-festering-pile-of-bugs-on-patch-tuesday-ms-releases-27-more-patches.html (https://www.computerworld.com/article/3290465/microsoft-windows/stung-by-a-festering-pile-of-bugs-on-patch-tuesday-ms-releases-27-more-patches.html) suggests Microsoft patches have been pretty buggy lately... They claim corporate IT folks have been reluctant to install them... Sorry to be the bearer of bad news...
Hi John,
Yes, it is a Dell PC. Yes, it's a Win 7 system. Yes, I probably ran a Windows update last week.
Sigh...
I read the link and they did not mention DCOM specifically as being affected by these updates, but a number of TCP/IP issues were mentioned, so these recent updates could very well be the problem. Let's hope Microsoft re-fixes these issues soon!
john
I feel your pain. I had issues with Windows backup failing on my Win 7 box. I hadn't run a Windows backed up for a while (I had everything important backed up manually...) and had several (long) backup attempts fail (at the end, of course) with a very unhelpful error code... Even running checkdsk didn't help. I tried several solutions from the web (such as backup from safe mode), but none worked properly. This was a long process...
On day 3, I unplugged all external drives but the main backup drive, deleted all the old backups on that drive, and ran a backup that passed. I have done two more since after updating programs. I just noticed that I have't created a system restore point since doing that, so I just started one...
Guess given MS's recent history, I think I will create a restore point before applying any major MS patch...
I could never get the Windows Backup in Windows 7 to work properly so we're still using Norton Ghost which works great.
In the last 10 or 15 years, there's been half a dozen times when a hard drive failed in our lab (we do have 8 or so PCs that are always on), and we had to restore the drive from a backup. 30 minutes later and we're up and running again, so well worth it.
That reminds me that I should update Norton Ghost so we can start using it on the new Windows 10 system we have.
I wonder if it's worth contacting FEI to see if they've run into this Windows 7 DCOM issue recently? Has anyone reading this had a similar problem with third party communications (usually EDS systems for reading the keV, mag, stage, etc.) with their FEI instruments, recently?
In talking with Rich Westphall from Thermo support they found that on their Thermo system that stopped communicating with an FEI instrument at Penn State, they had to re-install the FEI xT software on the microscope computer (MPC), to get the DCOM talking again with the support computer (SPC).
I've located the FEI install disks and will attempt a re-install of the FEI software next, but in the meantime I discovered that one should not set the User Access Control to "Never Notify" on the Thermo/PictureSnapApp computer. Because if you do, then when you run the installer, the operating system doesn't pop up the prompt for the admin login, and the installer instead simply gives an error that the update needs to be run by an administrator.
This of course is on a user login that is *not* an administrative account.
Quote from: Probeman on August 27, 2018, 03:50:54 PM
Today I came into the lab and noticed that the Thermo Pathfinder was not showing the correct mag or stage positions on our FEI Quanta instrument. So of course I re-started Pathfinder because if someone had re-started the FEI software while the Thermo software was running, it will break the DCOM connection. But when the Thermo Pathfinder re-started I got the error "Failed to Connect" and then "Connection Not made".
Now you might ask, why am I posting this in the PictureSnapApp board? Well, mostly because there is no FEI board on this forum, but also because when I start PictureSnapApp I also get the error "Permission Denied", so it seems the DCOM connection is now broken in general for all apps.
So I re-started the Quanta computer and software, but still no dice. I can ping the Quanta microscope computer at 192.168.0.1 from the Thermp?PSA computer fine, but both PictureSnapApp and Pathfinder refuse to connect to the FEI computer over the local 192.168.0.1 connection.
I should probably also re-start the Thermo/PSA computer but it's in the middle of a Monte Carlo calculation for a paper and I hate to interrupt it. Maybe tomorrow.
In the meantime, does anyone have any suggestions as to why the DCOM connection is suddenly broken, but the user logins for both computers are still the same?
Just an update on the DCOM wars on our FEI Quanta Win 2000 computer...
We finally got the DCOM connection working again, but to be honest we really aren't exactly sure how it happened or what we did to fix the problem! Yup, not very satisfying, but here's our best guess now that the smoke has cleared:
To begin with we think that the issue started when we temporarily connected the Quanta MPC to the Internet to allow the Norton Ghost software to renew it's subscription. Norton had been complaining that our license had expired so I thought, what's the harm in connecting an Internet cable to the PC? I'll just change the IP address from static private to a dynamic address on the campus network, then restore the 192.168.0.1 static address afterwards. And we were able to renew the Ghost subscription and then I reset the connection back to it's original values, and was able to ping the computer through the 192.168.0.x subnet afterwards so all seemed fine network wise.
But apparently something was not happy and the DCOM would not connect even though we went through all the many steps provided to us by Thermo and FEI support. We then decided to try and get a duplicate hard disk we had made a few years ago, but for some other reason every time we tried to boot it up, it would get to the Windows 2000 logo and then blue screen.
Our instrument engineer tried everything he could think of but still no dice. So then we called in the campus IT people to try and help us get the duplicate disk bootable, and so I demonstrated the DCOM problem on the original working disk, and lo and behold, the DCOM connections all worked perfectly!
The only thing we can think of is that somehow in the process of trying to re-boot the computer with a different disk and having to go into the ROM settings we somehow got the network card working properly again. But who knows?
Jeez, what a pain, but PictureSnapApp and Thermo Pathfinder all seem happy now connecting to the Xt software on the Quanta MPC. I just hope this never happens to anyone else!
This is somewhat related to the "DCOM hell" we recently went through on our FEI Quanta FEG instrument. I'm posting it here in the hopes that it helps someone else avoid our pain and suffering.
Because this instrument is a "Mark I" model, the MPC (microscope PC) is still Win2K and there is no upgrade path for it. However, it is not on the Internet, only the private network so no worries there.
Meanwhile the SPC (support PC) is on the Internet so students can upload/download their image files, and although I updated it from Win2K to WinXP some time ago, that apparently isn't enough any longer. Because recently the IT people said we had to update the FEI "Support" computer because they didn't want any WinXP computers on the campus network.
So we talked about Win7 vs. Win10 and because Win7 is not supposed to be supported by Microsoft after next year I decided to go with Win10. I mean the only thing we were using it for was as a file server with a shared folder connection to the MPC over the private network. What could go wrong?
So the IT guy did a Win10 clean install and I re-installed a few apps but when we tried to create a shared folder it was weird. We could not connect to a shared folder on the new Win10 system from the Win2K system even though we could ping each of them from the other.
The IT guy called me back today to tell me he had figured it out. Apparently Win10 only support SMB v.2 which is the transport protocol used for sharing folders, while Win2K only supports SMB v.1. So they will never learn to share! :(
But since Win7 supports both SMB v.1 and SMB v.2, we were able to create a shared data folder on the PictureSnapApp/Thermo Pathfinder computer that we could connect to from the MPC.
At least until after next year when the campus will start bugging us to upgrade from Win7 to Win10. >:(
Edit by John: just learned from our IT guy that XP also only supports SMB v.1, so apparently one cannot connect to a shared folder on Windows 10 from a WinXP computer either.
Thanks for the update. I feel your pain. Does your university IT group have a variance process for these situations? That was one highlight of our corporate process. Our IT folks were sympathetic when they understood that there was no available upgrade for our microscope PC and that replacement of the microscope would have cost much more than $100K. They were willing to grant us a variance that was signed by senior IT management and gave us the best support they could. Like your situation, ours included buffer computers with all the service patches and anti-virus modules that were certified. I'd also check to see that you had the Enterprise version of Win-10 Pro on those systems so that your IT folks can verify the MS updates and schedule them at opportune times. I keep seeing new complaints about MS updates causing problems. We both know all too well the hours of lost productivity caused by botched updates.
Hi John,
Yeah, keeping all these computers updated and backed up is more of a pain than it should be for sure.
I have not heard of a "variance process" but if we keep the computer on a private network they really don't care. The problem then becomes keeping the anti-virus and backup software licenses updated.
I mention anti-virus because not all viruses come from the Internet, we've had students bring them in on USB sticks. And some of our Norton Ghost installs need to check for a current subscription or they won't run backups. That's how we killed the DCOM on the Quanta MPC last month. Next time I try that I will install a separate network card for the campus network to be used temporarily.
More pain will come after next year when the campus will want us to update our Win7 computers to Win10. Fortunately while upgrading from WinXP requires a clean install of Win7 or Win10 and re-installation of all apps, our IT person says that upgrading from Win7 to Win10 is pretty straight forward as the apps will not need to be re-installed. We'll see! One issue I foresee is the shared folder issue when our Win2K Quanta MPC needs to connect to a computer on the Internet for file sharing due to the SMB v1 vs. SMB v2 issue previously described.
The other issue is that Symantec does not currently have a Ghost backup/recovery version for Win10. So we're using the built-in Windows backup, but according to our IT person it's non-trivial to utilize Win10 Backup to restore a crashed hard drive- which can be a real life saver in the lab for computers with complex installs of OEM software for instrument operation.
Hard drives are more reliable than ever, but still Norton Ghost has been utilized several times in the last 5 to 10 years out of a dozen or so computers that I manage.
I've heard about the Win10 update issues, but I haven't been personally affected yet. That said I only have Win10 on one of our file servers and it only needs to run an FTP server. Plus I do have Win10 on my personal laptop, but I only use it when traveling. Hopefully Microsoft will get its act together before we are forced to update all our workstations from Win7 to Win10 next year.
The best news is that CalcZAF, Probe for EPMA and Probe Image all seem to run fine on Windows 10, so that's a relief at least!
Looks like you might be able to turn smb v1 back on in win 10:
https://support.microsoft.com/en-gb/help/4034314/smbv1-is-not-installed-by-default-in-windows
Or you can fiddle with the smb v2 requirements so that it plays nice with smb v1.
Quote from: JonF on October 24, 2018, 01:05:30 PM
Looks like you might be able to turn smb v1 back on in win 10:
https://support.microsoft.com/en-gb/help/4034314/smbv1-is-not-installed-by-default-in-windows
Or you can fiddle with the smb v2 requirements so that it plays nice with smb v1.
Holy crap! :o
How come my IT person didn't know this? >:(
Thanks, Jon.
Quote from: JonF on October 24, 2018, 01:05:30 PM
Looks like you might be able to turn smb v1 back on in win 10:
https://support.microsoft.com/en-gb/help/4034314/smbv1-is-not-installed-by-default-in-windows
Or you can fiddle with the smb v2 requirements so that it plays nice with smb v1.
Our IT guy said that if we turn SMB v1 back on a Windows10 system, the campus will still cut off Internet access to that computer. I don't know exactly how they detect SMB v.1 is available, but I suspect that modifying SMB "so it plays nice with smb v1" would result in the same response from them.
I know, I know, they're just trying to protect us...
Ned Pile (The "owner" of SMB at Microsoft ) published this explanation of why he thought it was necessary to quit using SMB1: https://blogs.technet.microsoft.com/filecab/2016/09/16/stop-using-smb1/ (https://blogs.technet.microsoft.com/filecab/2016/09/16/stop-using-smb1/).
The key quote:
QuoteThe nasty bit is that no matter how you secure all these things, if your clients use SMB1, then a man-in-the-middle can tell your client to ignore all the above. All they need to do is block SMB2+ on themselves and answer to your server's name or IP. Your client will happily derp away on SMB1 and share all its darkest secrets unless you required encryption on that share to prevent SMB1 in the first place. This is not theoretical – we've seen it. We believe this so strongly that when we introduced Scaleout File Server, we explicitly prevented SMB1 access to those shares!
I'm not certain how likely such an attack is, but I see why IT organizations are concered... I don't see any easy answers...
Yeah, I'd guess so. They'll be disabling smb v1 because its got a fairly big security hole, linked to a lot of the more recent ransomware attacks.
Essentially, you've got the Win2k box on the Quanta that can only communicate via smb.v1, a support PC that needs smb.v1 to communicate with the Win2k Quanta PC and a requirement that all university network connected PCs have smb.v1 disabled and a minimum of smb.v2. Bit of a tricky one!
The only way I could think of doing it would require a second support PC (assuming that you can't enable smb.v1 on a per interface basis, which samba in Linux may be able to do), something along the following lines:
Quanta (Win2k) -smb.v1-> 1st Support PC (Win7,10w/smb.v1/Linux) -smb.v2-> 2nd Support PC (Win10 w/smb.v1 disabled) -smb.v2-> University Network
Both support PCs would need at least two NICs. The first support PC would be able to communicate via both smb.v1 and smb.v2, the 2nd support PC via smb.v2 and above. You could have the data cascading through the support PCs (possibly deleted from the 1st support PC to remove the storage requirement, depending on how you have the data policy configured). You could also make all the shares read only, adding an extra bit of protection.
To get round the requirement for regular antivirus updates, I'd disable all USB ports (via group policy) and ban USB pen drive access to any of the instrument or support PCs, making everyone pull the data from the network share (whether 2nd support PC or a data store on the university network), and preferably have the 1st support PC as a Linux machine also acting as a hardware firewall. I'd image a working version of the Quanta PC win2k installation, and keep that in a safe place and ensure that all the data that changes is sent off the instrument PC (removing the Norton Ghost requirement).
If going with windows for the support PCs, I'd recommend getting a backup/copy program that supports the volume shadow copy service, or write a script that creates the copy, robocopy to the network share then deletes the shadow copy - this should avoid any issues with locking a file currently having data written to it.
Quote from: JonF on October 27, 2018, 09:14:12 AM
The only way I could think of doing it would require a second support PC (assuming that you can't enable smb.v1 on a per interface basis, which samba in Linux may be able to do), something along the following lines:
I was just googling something else and came across the answer to this: yes, you can bind samba to interfaces under Linux and have multiple instances of samba installed. The SMB settings in Windows are set for the whole OS (rather than NIC specific).
Linux is probably the way I'd go with this - it'll give you more flexibility on the networking side of things. I'd set up a linux server with two physical NICs - one internal to the lab with SMBv1 enabled to talk to the quanta; and one facing the external network with SMBv1 disabled.
A Windows-only alternative would be to run a virtual machine with an SMBv1 enabled guest having access only to the internal NIC and the host OS pushing the data to offsite storage.
Quote from: JonF on December 03, 2018, 01:26:19 AM
Quote from: JonF on October 27, 2018, 09:14:12 AM
The only way I could think of doing it would require a second support PC (assuming that you can't enable smb.v1 on a per interface basis, which samba in Linux may be able to do), something along the following lines:
I was just googling something else and came across the answer to this: yes, you can bind samba to interfaces under Linux and have multiple instances of samba installed. The SMB settings in Windows are set for the whole OS (rather than NIC specific).
Linux is probably the way I'd go with this - it'll give you more flexibility on the networking side of things. I'd set up a linux server with two physical NICs - one internal to the lab with SMBv1 enabled to talk to the quanta; and one facing the external network with SMBv1 disabled.
A Windows-only alternative would be to run a virtual machine with an SMBv1 enabled guest having access only to the internal NIC and the host OS pushing the data to offsite storage.
You are smart! :)
Seriously, it may come to running a virtual machine as Windows 10 only supports SMBv2. For now we are OK as Windows 7 supports both SMB versions. But IT tells us that Windows 7 will only be supported until the end of 2019. Of course I've noticed that Microsoft is still releasing security updates for Windows XP, so maybe it won't happen that fast.
Then one has to consider that Microsoft has been struggling with some recent Windows 10 updates... Anyway, hopefully by the time we have to update this Windows 7 support computer to Windows 10, we'll get a new FEG SEM with new instrument software and won't have to run Windows 2000! ;D
Hey, one can always dream, right?
Quote from: Probeman on October 09, 2018, 04:20:40 PM
Quote from: Probeman on August 27, 2018, 03:50:54 PM
Today I came into the lab and noticed that the Thermo Pathfinder was not showing the correct mag or stage positions on our FEI Quanta instrument. So of course I re-started Pathfinder because if someone had re-started the FEI software while the Thermo software was running, it will break the DCOM connection. But when the Thermo Pathfinder re-started I got the error "Failed to Connect" and then "Connection Not made".
Now you might ask, why am I posting this in the PictureSnapApp board? Well, mostly because there is no FEI board on this forum, but also because when I start PictureSnapApp I also get the error "Permission Denied", so it seems the DCOM connection is now broken in general for all apps.
So I re-started the Quanta computer and software, but still no dice. I can ping the Quanta microscope computer at 192.168.0.1 from the Thermp?PSA computer fine, but both PictureSnapApp and Pathfinder refuse to connect to the FEI computer over the local 192.168.0.1 connection.
I should probably also re-start the Thermo/PSA computer but it's in the middle of a Monte Carlo calculation for a paper and I hate to interrupt it. Maybe tomorrow.
In the meantime, does anyone have any suggestions as to why the DCOM connection is suddenly broken, but the user logins for both computers are still the same?
Just an update on the DCOM wars on our FEI Quanta Win 2000 computer...
We finally got the DCOM connection working again, but to be honest we really aren't exactly sure how it happened or what we did to fix the problem! Yup, not very satisfying, but here's our best guess now that the smoke has cleared:
To begin with we think that the issue started when we temporarily connected the Quanta MPC to the Internet to allow the Norton Ghost software to renew it's subscription. Norton had been complaining that our license had expired so I thought, what's the harm in connecting an Internet cable to the PC? I'll just change the IP address from static private to a dynamic address on the campus network, then restore the 192.168.0.1 static address afterwards. And we were able to renew the Ghost subscription and then I reset the connection back to it's original values, and was able to ping the computer through the 192.168.0.x subnet afterwards so all seemed fine network wise.
But apparently something was not happy and the DCOM would not connect even though we went through all the many steps provided to us by Thermo and FEI support. We then decided to try and get a duplicate hard disk we had made a few years ago, but for some other reason every time we tried to boot it up, it would get to the Windows 2000 logo and then blue screen.
Our instrument engineer tried everything he could think of but still no dice. So then we called in the campus IT people to try and help us get the duplicate disk bootable, and so I demonstrated the DCOM problem on the original working disk, and lo and behold, the DCOM connections all worked perfectly!
The only thing we can think of is that somehow in the process of trying to re-boot the computer with a different disk and having to go into the ROM settings we somehow got the network card working properly again. But who knows?
Jeez, what a pain, but PictureSnapApp and Thermo Pathfinder all seem happy now connecting to the Xt software on the Quanta MPC. I just hope this never happens to anyone else!
Well our PictureSnapApp and Thermo Pathfinder softwares are once again, not connecting to the FEI Quanta software over the DCOM connection from our Win 7 computer.
As you all can see from the above previous post, we got it working a couple of years ago when we started getting this "Permission Denied" error, but aren't quite sure how we fixed it! I suspect that on the Win XP computer which is running the Quanta software, there is a network setting not allowing the DCOM connection for some reason.
Any suggestions where to look?