PDA

View Full Version : [FIXED] Indexing whenever workspace is loaded



firefoxSafari
3 Nov 2015, 11:10 AM
I've noticed a fairly lengthy indexing operation happen every time I open a workspace for a Sencha app. It seems like this should just be done once or at least be quicker when loading an existing workspace?

Chris.OBrien
3 Nov 2015, 11:46 AM
The first time a project is created and opened, the indexing will take longer since it is parsing all of your files, as well as the SDK, and creating a local index cache.

Any time a project or workspace is opened after a cache already exists, we still have to load the cache, but not parse the files, so it should be quicker than the first indexing. All this depends on the size of your project, machine, etc.

ssamayoa
3 Nov 2015, 12:17 PM
Mmmm...

Any chances that you may have pre-indexed the sdk and index only the app's files?

I have a decent laptop (i5 / 12G RAM / SDD) and take a lot of time.

Regards.

firefoxSafari
3 Nov 2015, 2:16 PM
I'm on desktop with i7 and 16 GB and still seems rather slow. Don't notice that it's that much faster when opening the workspace versus just creating a new project. This is just for the base project generated from new Ext JS App Project - haven't added any custom code.

bholyoak
3 Nov 2015, 3:25 PM
I created an application which then builds and the progress tab shows "Sencha indexing PROJECTNAME". After chugging away for several minutes it stopped at 66% and never finished (I left it running for an hour and a half to make sure... CPU maxing, fans running like jet engines, etc).

I shutdown Zend Studio and re-launched it. The project brought up "Sencha indexing PROJECTNAME" again in the Progress tab, which took roughly the same amount of time it took before to make it to 66%, where it promptly stops progressing but keeps running the CPU high.

Chris.OBrien
3 Nov 2015, 7:46 PM
I created an application which then builds and the progress tab shows "Sencha indexing PROJECTNAME". After chugging away for several minutes it stopped at 66% and never finished (I left it running for an hour and a half to make sure... CPU maxing, fans running like jet engines, etc).

I shutdown Zend Studio and re-launched it. The project brought up "Sencha indexing PROJECTNAME" again in the Progress tab, which took roughly the same amount of time it took before to make it to 66%, where it promptly stops progressing but keeps running the CPU high.

What happens if you try to cancel it when it is stuck?

Can you go to the Eclipse Preferences, select Sencha from the left, and then click the checkbox to enable debug logging?

Then, try to reopen the project, and when it gets stuck at 66%, open up your workspace's .metadata/.log file. Can you please send me a PM with the last page or so of the log?

Chris.OBrien
3 Nov 2015, 8:16 PM
I'm on desktop with i7 and 16 GB and still seems rather slow. Don't notice that it's that much faster when opening the workspace versus just creating a new project. This is just for the base project generated from new Ext JS App Project - haven't added any custom code.


I have a decent laptop (i5 / 12G RAM / SDD) and take a lot of time.

Just to get a feel for it, how long are you two saying it's taking? 2 minutes? 10 minutes?

What OS and architecture are your machines?

firefoxSafari
4 Nov 2015, 7:00 AM
Just to get a feel for it, how long are you two saying it's taking? 2 minutes? 10 minutes?

What OS and architecture are your machines?

I'm on Windows 7 64 bit i7 16 GB.

There's some variance, but the indexing dialog is up for between 45-60 seconds when I load a workspace. I've timed how long it's up during new project creation, and it appears to be about the same. That's why I question whether the optimization to not index as much on the second pass is working.

ssamayoa
4 Nov 2015, 7:21 AM
Create new Ext App Project:
- Generating: 24s, comprehensible because copies the whole SDK.
- Indexing: ~1 minute

Open eclipse's workspace containing Sencha project:
- Indexing: ~5s

Note that I have an SSD, lot of RAM and linux uses that RAM for cache.

@firefoxSafari:
Do you have a conventional HDD or SSD?

firefoxSafari
4 Nov 2015, 7:28 AM
Note that I have an SSD, lot of RAM and linux uses that RAM for cache.

@firefoxSafari:
Do you have a conventional HDD or SSD?

Machine I'm trying this on is conventional HDD, not SSD.

Chris.OBrien
4 Nov 2015, 7:36 AM
I'm on Windows 7 64 bit i7 16 GB.

There's some variance, but the indexing dialog is up for between 45-60 seconds when I load a workspace. I've timed how long it's up during new project creation, and it appears to be about the same. That's why I question whether the optimization to not index as much on the second pass is working.

What are you doing when you close a workspace? Quitting Eclipse outright? (Not saying this is wrong, just trying to figure out what might cause this)

Can you try closing a project from within Eclipse, and re-opening to see if it behaves any different? We currently write our index cache when the our background indexing process ends, and it might be getting killed before it has a chance to write out the cache.

Also, can you see inside your project, there should be a .sencha-ide folder. Can you tell me what is inside it? If there's just a status.json file, and nothing else, then the cache is not being written out properly.

Chris.OBrien
4 Nov 2015, 7:40 AM
Create new Ext App Project:
- Generating: 24s, comprehensible because copies the whole SDK.
- Indexing: ~1 minute

Open eclipse's workspace containing Sencha project:
- Indexing: ~5s

Note that I have an SSD, lot of RAM and linux uses that RAM for cache.

Ok, that's looking good, we're seeing about the same results. That 1 minute for the generating is appropriate, since it indexes the entire Ext SDK.

Glad there's a difference between the initial generate and re-opening the project. Thanks.

firefoxSafari
4 Nov 2015, 8:18 AM
What are you doing when you close a workspace? Quitting Eclipse outright? (Not saying this is wrong, just trying to figure out what might cause this)

Can you try closing a project from within Eclipse, and re-opening to see if it behaves any different? We currently write our index cache when the our background indexing process ends, and it might be getting killed before it has a chance to write out the cache.

Also, can you see inside your project, there should be a .sencha-ide folder. Can you tell me what is inside it? If there's just a status.json file, and nothing else, then the cache is not being written out properly.

To close, I normally just press the big red x on the window. I've tried File -> Exit and closing and opening a project - no change.

I found the .sencha-ide folder and it only contains the status.json file, so the cache isn't being written out.

I looked at cpu usage after indexing and before closing Eclipse and it didn't look like Eclipse was taking any cpu resources.

I enabled Sencha debug logging from preferences and looked at the Error Log. Just info messages... nothing that looks like anything failed.

Chris.OBrien
4 Nov 2015, 8:21 AM
To close, I normally just press the big red x on the window. I've tried File -> Exit and closing and opening a project - no change.

I found the .sencha-ide folder and it only contains the status.json file, so the cache isn't being written out.

I looked at cpu usage after indexing and before closing Eclipse and it didn't look like Eclipse was taking any cpu resources.

I enabled Sencha debug logging from preferences and looked at the Error Log. Just info messages... nothing that looks like anything failed.

We've got some enhancements on deck to write the cache out earlier, before the process is terminated, so this will improve in the future.

I'll have to fire up a Win 7 VM and attempt to recreate it myself. What version of Java are you running Eclipse under?

ssamayoa
4 Nov 2015, 8:25 AM
Ok, that's looking good, we're seeing about the same results. That 1 minute for the generating is appropriate, since it indexes the entire Ext SDK.

Glad there's a difference between the initial generate and re-opening the project. Thanks.

Yes, is OK.

But I suspect that the difference with my times against firefoxSafari's is the use of SSD and because you get similar results seems that you use SSD also so you should do your speed tests on computers with conventional HDD.

@firefoxSafari
Go for SSD!
Having an i7 with 16G of RAM with a conventional HDD is like having a Ferrari with bicycle tires!

Regards.

firefoxSafari
4 Nov 2015, 8:25 AM
We've got some enhancements on deck to write the cache out earlier, before the process is terminated, so this will improve in the future.

I'll have to fire up a Win 7 VM and attempt to recreate it myself. What version of Java are you running Eclipse under?

jdk1.7.0_79

ssamayoa
4 Nov 2015, 8:29 AM
jdk1.7.0_79

Even if you still code for Java 7 better use Java 8: there are several speed improvements and the odious PermGen is no longer there!

Regards.

Chris.OBrien
5 Nov 2015, 8:16 AM
jdk1.7.0_79

Thanks, I was able to recreate it in Windows 7 with Java 1.8. It is not an issue for me in Linux or OSX.

I'll open a ticket so we can make sure the process is properly shutdown and the cache gets updated.

bholyoak
11 Nov 2015, 9:06 AM
What happens if you try to cancel it when it is stuck?

Can you go to the Eclipse Preferences, select Sencha from the left, and then click the checkbox to enable debug logging?

Then, try to reopen the project, and when it gets stuck at 66%, open up your workspace's .metadata/.log file. Can you please send me a PM with the last page or so of the log?

If I cancel it, it seems to cancel pretty immediately, so it's not "freezing".

I enabled the debugging and set it to work indexing again. I let it go for over an hour and a half last night before I got impatient and went to cancel it. Right when I cancelled it, it ticked to 67%... figures.

It is in a very large project with a huge amount of other PHP scripts, templates, and other Javascript Libraries, so maybe it just needs to run for *hours*. I'll let it run longer and see if it goes any faster once it gets past 66%.

I'm really hoping that once it is done indexing that it won't have to go through this process everytime I open Zend Studio, or it's just going to be unusable.

Chris.OBrien
11 Nov 2015, 9:13 AM
If I cancel it, it seems to cancel pretty immediately, so it's not "freezing".

I enabled the debugging and set it to work indexing again. I let it go for over an hour and a half last night before I got impatient and went to cancel it. Right when I cancelled it, it ticked to 67%... figures.

It is in a very large project with a huge amount of other PHP scripts, templates, and other Javascript Libraries, so maybe it just needs to run for *hours*. I'll let it run longer and see if it goes any faster once it gets past 66%.

I'm really hoping that once it is done indexing that it won't have to go through this process everytime I open Zend Studio, or it's just going to be unusable.

Hmm, it definitely shouldn't take that long, something is clearly wrong. We've done some tests with large projects, and nothing should really exceed 5 minutes or so. This shouldn't have anything to do with Zend itself, since our background process scans the JavaScript regardless to the IDE plugin. Our process also only scans .js files, so any extra templates or PHP files shouldn't have any impact.

Was there anything in the debug output? Anything saying Parsing or error?

Also, if you check the status.json file inside the .sencha-ide folder of your project, what does it say? This tracks the initial process of our background process, and is used to generate the progress bar.

bholyoak
11 Nov 2015, 12:13 PM
I sent you a PM with the info you requested.