PDA

View Full Version : [FIXED] tem-win64.exe using all system mem and crashing/unresponsive on Maven Update



Paul Priest
18 Nov 2015, 9:50 AM
Currently my developers were not able to effectively validate the capabilities, because with our large project, doing a Maven (m2e plugin from Eclipse.org included with JEE edition of Eclipse) 'Update Project' from within Eclipse resulted in the tem-win64.exe process growing to over a 1.5GB of mem and then Eclipse becomes unresponsive or crashes or needing to be all killed.
Everyone encountered this and promptly uninstalled the plugin.


We have no other problems with Eclipse and Stability.
We are developing combined Javascript/Java projects and using Mars.

--
Also, opening our project (or opening Eclipse with an open project) with the Sencha nature adds ~15 seconds to what would otherwise be only a couple of seconds for the indexing.

Chris.OBrien
18 Nov 2015, 6:55 PM
Currently my developers were not able to effectively validate the capabilities, because with our large project, doing a Maven (m2e plugin from Eclipse.org included with JEE edition of Eclipse) 'Update Project' from within Eclipse resulted in the tem-win64.exe process growing to over a 1.5GB of mem and then Eclipse becomes unresponsive or crashes or needing to be all killed.
Everyone encountered this and promptly uninstalled the plugin.

We did encounter a problem in the past few weeks with our EA release when folders are moved/updated/deleted. It was firing a large number of events and flood our tern background process with update calls. I just put the finishing changes on this today to handle folders in bulk and do one call to the indexing process. It will be ready in our upcoming GA release.

Hopefully this fixes the problem with a Maven update. Do you know what files/folders the maven update might be modifying? I'm assuming a parent folder of some sort. Our events should have only been sent for .js files anyway, so I'm a bit curious what



Also, opening our project (or opening Eclipse with an open project) with the Sencha nature adds ~15 seconds to what would otherwise be only a couple of seconds for the indexing.

When you say "only a couple of seconds for the indexing" do you mean standard Eclipse indexing? The reason our indexing is a bit more time consuming, is that it's going over the entire JavaScript to analyze Ext JS and your related JavaScript. We do have performance improvements for this indexing ready to go for GA as well.

Paul Priest
19 Nov 2015, 1:52 AM
We did encounter a problem in the past few weeks with our EA release when folders are moved/updated/deleted. It was firing a large number of events and flood our tern background process with update calls. I just put the finishing changes on this today to handle folders in bulk and do one call to the indexing process. It will be ready in our upcoming GA release.

Hopefully this fixes the problem with a Maven update. Do you know what files/folders the maven update might be modifying? I'm assuming a parent folder of some sort. Our events should have only been sent for .js files anyway, so I'm a bit curious what

It might well be triggering events for every file - is there a simple way to log the events that you trigger off of?
I'd be happy to test a debug or beta version to pin this down.



When you say "only a couple of seconds for the indexing" do you mean standard Eclipse indexing? The reason our indexing is a bit more time consuming, is that it's going over the entire JavaScript to analyze Ext JS and your related JavaScript. We do have performance improvements for this indexing ready to go for GA as well.

Yes, the cost of the Sencha indexing is very high as an overhead relative to Eclipse's built-in processing and indexing for launching or opening projects. It would be less impactful if the indexing could somehow be preserved across closing/opening projects and Eclipse (with the asusmption that nothing has touched files whilst they were not being monitored - and explicit 'reindexing' option would take care of that).

Chris.OBrien
19 Nov 2015, 5:38 AM
It might well be triggering events for every file - is there a simple way to log the events that you trigger off of?
I'd be happy to test a debug or beta version to pin this down.



You can go to the Eclipse Preferences, and find the Sencha section, and check "Enable Debug Logging". You can see our logging from either the workspace/.metadata/.log file, or from within Eclipse's "Error Log" view. You can 'group by' Plug-in from the dropdown arrow icon at top left of the view.

You might have to use the .log file if Eclipse itself hangs in this scenario.


We do log output regarding JavaScript files, such as: Detected JavaScript file change or Detected JavaScript file removed. These should be the only things sending data to our backend during a resource change like this. I'm curious what you'll see.






Yes, the cost of the Sencha indexing is very high as an overhead relative to Eclipse's built-in processing and indexing for launching or opening projects. It would be less impactful if the indexing could somehow be preserved across closing/opening projects and Eclipse (with the asusmption that nothing has touched files whilst they were not being monitored - and explicit 'reindexing' option would take care of that).


What environment are you in? We do write our indexing cache to the disk to make subsequent indexing efforts faster, and load the cache whenever we can.


We did encounter a problem in EA for Windows where it wasn't properly stopping the backend process, so our cache was never getting written out. This caused the full re-index every single time. This has been fixed for GA as well

Paul Priest
19 Nov 2015, 6:49 AM
You can go to the Eclipse Preferences, and find the Sencha section, and check "Enable Debug Logging". You can see our logging from either the workspace/.metadata/.log file, or from within Eclipse's "Error Log" view. You can 'group by' Plug-in from the dropdown arrow icon at top left of the view.


Okay, so just enable the Sencha nature outputs a few hundred lines - one per JS file indicating it's sending it to be indexed.
If I trigger a Maven Update, it outputs thousands and thousands of lines of the following (4 per file), getting slower all the time (presumably blocking on a response), and ultimately just stops - whilst the Eclipse Maven Update process is still technically in-progress.

Each JS file appears to appear twice in the logs for a single update before it gives up.


Sending JSON to Tern (http://...): {"files": [{"type": "full", ....
Tern myProject [1234] - parsing c:\...
Detected Javascript file change foo.js delta.kind: 4
Response from Tern server: {}






What environment are you in? We do write our indexing cache to the disk to make subsequent indexing efforts faster, and load the cache whenever we can.

We did encounter a problem in EA for Windows where it wasn't properly stopping the backend process, so our cache was never getting written out. This caused the full re-index every single time. This has been fixed for GA as well

We're on Windows. That certainly sounds like a/the issue. If there's an update to the EA, or a beta of the GA you want me to verify, shout.

Chris.OBrien
19 Nov 2015, 9:55 AM
We're on Windows. That certainly sounds like a/the issue. If there's an update to the EA, or a beta of the GA you want me to verify, shout.

Note that even though we've fixed the problem with it writing the cache in Windows, there will still be an "Indexing..." period when opening a project, since we have to load that cache into memory. However, it should go much faster than the initial index. The larger the project, the more of a difference you'll see.

We'll also see the Indexing... dialog when folders and a large number of files are moved/renamed/changed to make sure that everything stays updated, but this should be even quicker then opening the project.

Chris.OBrien
19 Nov 2015, 10:56 AM
Each JS file appears to appear twice in the logs for a single update before it gives up.


Sending JSON to Tern (http://...): {"files": [{"type": "full", ....
Tern myProject [1234] - parsing c:\...
Detected Javascript file change foo.js delta.kind: 4
Response from Tern server: {}



"delta.kind: 4" means the file itself is being changed, not the parent folders. If the files were moved, or parent folders were moved/renamed, it would actually be a delete and re-add. This makes it seem that the Maven update is actually touching every single .js file out there.

So I just setup a test project using the EA plugin release, and just configuring it to be a Maven project triggered the same updates, grinding my workspace to a halt. Reopening and doing an Update maven project does the same, so I am seeing what you experienced.

I'm still seeing the same thing in our latest code as well. I just linked this thread with one of our JIRA tickets, and will be fixing this for GA.

Thanks for your help!

Chris.OBrien
3 Dec 2015, 6:56 AM
Our GA release is now available from the Sencha Support Portal and Eclipse Marketplace (http://marketplace.eclipse.org/content/sencha-eclipse-plugin), which should fix these problems you were experiencing with Maven updates.

Paul Priest
3 Dec 2015, 7:15 AM
I can confirm the Maven issue is fixed in the GA.
We're experiencing a variety of other issues. It's working well for some, and not at all for others.
I'll followup with a new thread.
Thanks.