Ext Direct Java based implementation

[pagullo]

can not call a directmethod when a string param contains \n \r etc?
why?

Make our life easier and provide additional info, please: js & java source code, exception traces, request/response logs, error logs, etc.

Take into account that the one in charge of JSON processing is the GSON library, so it might be due to some issue with it, be it a bug or just we having to adjust GSON in some way -or just using it the wrong way.

[Whatty]

Hi!

I didn't do it the way you suggest because poll methods need to be taken into account too, and I implemented them so they are 'global', i.e., they do not belong to a RegisteredAction, and therefore I can't pass the action name to the dispatch method for poll methods (sigh).

But I can pass the class, and it is really useful sometimes, for example to process custom annotations -which is what I do to support stateful actions.

As to way poll methods do not belong to a RegisteredAction, that made lots of sense when we had no stateful actions, as it made implementing some things much easier back then. Changing that now would change the published API...but I might do, because that part of the API is rarely used, to be fair...On the other hand, I was aware that if I had made that change, I would have had the RegisteredAction available, and passing that to dispatch would make additional info available, including the action name you want...

So, I need to consider carefully how or whether to pass the additional info you suggest. Let me think about that, ok?

Thanks for the feedback

Best regards,

If you take a look at my post with the Session Aware Spring integrator you will quickly see the problem that I face.

In the class

SessionAwareSpringDispatcher

you will see that I had to essentially override a number of methods in not so nice manner.

I understand that you are supporting a more generic framework so anything that you could do to provide additional hooks or information into the generic framework would be appreciated.

Thanks

Whatty

[GregT]

Hi Pedro,

I get this OOM error from Tomcat a few times a day. I am wondering if it is a leak in the YUI Compressor, and that it happens proportional to how often you recompile the classes that DJN is looking for. I assume this triggers a rebuild of Api.js, which is the only thing on my system that I know of that might run YUI. Any thoughts? Should I try to track down a better compressor? Do you have any experience with this problem? First few lines of the stack trace follows.

Thanks,
Greg

Nov 27, 2009 12:55:31 PM org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.util.Arrays.copyOf(Unknown Source)
at java.util.ArrayList.ensureCapacity(Unknown Source)
at java.util.ArrayList.add(Unknown Source)
at com.yahoo.platform.yui.compressor.JavaScriptCompressor.<clinit>(JavaScriptCompressor.java:69)
at com.softwarementors.extjs.djn.jscodegen.Minifier.minify(Minifier.java:42)

[pagullo]

Hi Pedro,

I get this OOM error from Tomcat a few times a day. I am wondering if it is a leak in the YUI Compressor, and that it happens proportional to how often you recompile the classes that DJN is looking for. I assume this triggers a rebuild of Api.js, which is the only thing on my system that I know of that might run YUI. Any thoughts? Should I try to track down a better compressor? Do you have any experience with this problem? First few lines of the stack trace follows.

Thanks,
Greg

Nov 27, 2009 12:55:31 PM org.apache.catalina.core.ApplicationContext log
SEVERE: StandardWrapper.Throwable
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.util.Arrays.copyOf(Unknown Source)
at java.util.ArrayList.ensureCapacity(Unknown Source)
at java.util.ArrayList.add(Unknown Source)
at com.yahoo.platform.yui.compressor.JavaScriptCompressor.<clinit>(JavaScriptCompressor.java:69)
at com.softwarementors.extjs.djn.jscodegen.Minifier.minify(Minifier.java:42)

Greg,

No, I haven't seen this problem before. Ugly one .

Difficult to know whether this is some issue with the YUI Compressor, the way DJN handles the compressor, or even your app: this last possibility might sound crazy, but there might be some app-related memory leak that is made visible only when the compressor starts working because it is very memory hungry and hence is the first piece of code to reach some upper limit.

Can you post the whole stack trace?

On the other hand, how much memory does Tomcat have? Have you tried to modify the amount of memory Tomcat uses? Can you monitor how memory is used by Tomcat? Is there some trend?

If at all possible, check memory usage with the 'minify' option set to true and then to false, just to check the trend in memory usage in both scenarios.

Hmmm... Why don't you patch the DJN code that minifies the js code to log the amount of memory used by the VM before/after every compression? Maybe you will be able to find some trend...

Everything else failing, the workaround will be to set the 'minify' option to false.

Let us know how it goes.

[Ramez]

Hi Pedro,

We recently moved all our Prototype based AJAX calls over to using Ext.Direct with DirectJNgine. Most of the conversion process was relatively straight forward but we have run into a problem with a few calls that rely on values pulled from a client's session.

To facilitate this we implemented a Filter that pulls the session out for later use by various Direct methods. This approach seems to work for the most part, but we've noticed an issue with the Filter not being called when a call that relies on it is batched with another call. If filter is not called, when a method goes to get the session value as stored by the Filter, it will be null since doFilter is not being called.

Here's what the stack trace looks like when the Filter is properly called and we are able to extract the session values for use in the Direct method:

MyClass.mySessionReliantMethod(String) line: 967
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]    
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39    
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25    
Method.invoke(Object, Object...) line: 597    
Dispatcher.invokeMethod(Method, Object, Object[]) line: 113    
Dispatcher.dispatch(Class<?>, Method, Object[]) line: 55    
JsonRequestProcessor(RequestProcessorBase).dispatch(Class<?>, Method, Object[]) line: 219    
JsonRequestProcessor(RequestProcessorBase).dispatch(String, String, Object[]) line: 230    
JsonRequestProcessor.processIndividualRequest(JsonRequest, boolean, int) line: 368    
JsonRequestProcessor.processIndividualRequestsInThisThread(JsonRequest[]) line: 127    
JsonRequestProcessor.process(Reader, Writer) line: 109    
RequestRouter.processJsonRequest(Reader, Writer) line: 77    
DirectJNgineServlet.doPost(HttpServletRequest, HttpServletResponse) line: 377    
DirectJNgineServlet(HttpServlet).service(HttpServletRequest, HttpServletResponse) line: 637    
DirectJNgineServlet(HttpServlet).service(ServletRequest, ServletResponse) line: 717    
ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 290    
ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206    
SessionFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 57    
ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 235    
ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206    
EncodingUTF8Filter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 25    
ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 235    
ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206    
LocalAddrFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 52    
ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 235    
ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206    
StandardWrapperValve.invoke(Request, Response) line: 233    
StandardContextValve.invoke(Request, Response) line: 191    
SSLWithFormFallbackAuthenticator.invoke(Request, Response) line: not available    
StandardHostValve.invoke(Request, Response) line: 128    
ErrorReportValve.invoke(Request, Response) line: 102    
StandardEngineValve.invoke(Request, Response) line: 109    
CoyoteAdapter.service(Request, Response) line: 293    
Http11Processor.process(Socket) line: 849    
Http11Protocol$Http11ConnectionHandler.process(Socket) line: 583    
JIoEndpoint$Worker.run() line: 454    
Thread.run() line: 619 [local variables unavailable]

Here's what the stack trace looks like when the Filter is not called, as is the case when calls are batched together:

MyClass.mySessionReliantMethod(String) line: 967    
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]    
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39    
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25    
Method.invoke(Object, Object...) line: 597    
Dispatcher.invokeMethod(Method, Object, Object[]) line: 113    
Dispatcher.dispatch(Class<?>, Method, Object[]) line: 55    
JsonRequestProcessor(RequestProcessorBase).dispatch(Class<?>, Method, Object[]) line: 219    
JsonRequestProcessor(RequestProcessorBase).dispatch(String, String, Object[]) line: 230    
JsonRequestProcessor.processIndividualRequest(JsonRequest, boolean, int) line: 368    
JsonRequestProcessor.access$000(JsonRequestProcessor, JsonRequest, boolean, int) line: 66    
JsonRequestProcessor$1IndividualRequestProcessor.call() line: 148    
JsonRequestProcessor$1IndividualRequestProcessor.call() line: 140    
FutureTask$Sync.innerRun() line: 303    
ParallelTask$BoundedFuture(FutureTask<V>).run() line: 138 [local variables unavailable]    
ThreadPoolExecutor$Worker.runTask(Runnable) line: 886    
ThreadPoolExecutor$Worker.run() line: 908 [local variables unavailable]    
Thread.run() line: 619

We'd like to keep the ability to batch calls and the optimal solution would be that ability to ensure that requests that are batched are filtered as expected. Another option would be if we could force this one method not to be batched.

We are using version DirectJNgine 1.0 due to licensing constraints incurred by 1.1's move to GPLv3.

Any help you can give would be greatly appreciated.

[pagullo]

Hi!

The issue you have is probably due to the fact that you are (probably) storing your request, etc. data in a thread local variable, BUT that will not be available to the newly created threads that run the individual requests in a batch.

Frankly speaking, processing batched request via multiple threads is an optimization you will probably not really need (I know of no other Ext Direct stack that does that, in fact, and nobody has complained so far...). So, the easier way to solve this is to set the corresponding configuration setting to false -as I already mentioned in other post.

That said, I have already implemented the equivalent of the filter you developed due to popular demand, of course supporting multithreaded batched requests processing. I will publish this one in alpha 2 of Ext Direct 1.2 real soon.

You need not worry about the license change in 1.1, it is both a silly and unintended mistake I already discussed in this thread: 1.2 final will be published with an LGPL v3 license, which is what I always meant .

Therefore, you might want to keep using whatever you have now with the workaround mentioned above, and wait for 1.2 final and use the new built-in mechanism for accesing the request, response, session, servlet context and servlet configuration .

Best regards,

[Ramez]

Thanks for the support Pedro.

Your assumption about storing the session in a thread local variable was correct.

After adding the init-param to disable multithreading of batch requests to our web.xml, all Ext.Direct calls succeed as expected.

Thanks for the heads up the regards to the planned change in licensing for 1.2. Do you have any idea of the time frame in which the 1.2 alpha might be available?

[maho2nd]

Hi,

I like to send some requests as a batch to the server (no multithreading!).

Is something like this possible?

public class TestAction {

private int calls = 0;

@DirectMethod
public String doEcho( String data ) {
calls ++;

if(calls == 5)
    //do something

return data;
}

public String doSomething( String data ) {
calls ++;

if(calls == 5)
    //do something

return data;
} 

Edit:
I think I need something like this: @ActionScope(scope=Scope.BATCH)

[pagullo]

Hi,

I like to send some requests as a batch to the server (no multithreading!).

Is something like this possible?

public class TestAction {

private int calls = 0;

@DirectMethod
public String doEcho( String data ) {
calls ++;

if(calls == 5)
    //do something

return data;
}

public String doSomething( String data ) {
calls ++;

if(calls == 5)
    //do something

return data;
} 

Edit:
I think I need something like this: @ActionScope(scope=Scope.BATCH)

Yes, it is, but only from DJN 1.2 onwards, using either session/application scoped actions or the new WebContext class. 1.2 is currently in alpha, check post #155 to get it and know what's there.

Of course, since what I have is an alpha, things might change, but I hope not that much.

BTW, in alpha 1 there is no support for multithreaded batched request, set the corresponding configuration flag to false...

[maho2nd]

Yes, it is, but only from DJN 1.2 onwards, using either session/application scoped actions or the new WebContext class. 1.2 is currently in alpha, check post #155 to get it and know what's there.

Of course, since what I have is an alpha, things might change, but I hope not that much.

BTW, in alpha 1 there is no support for multithreaded batched request, set the corresponding configuration flag to false...

Thanks for your response!

I read the post #155 but i thought Scope.Applications would save the private variables for the whole runtime and not only for a batch request. Because I need it only for the batch request.

I tested the Scope.Applications Annotation by modify your demo , but it didn't worked:

@ActionScope(scope=Scope.APPLICATION)
public class DirectStoreDemo {
    private int count = 0;
   
   [..] 
   
  @DirectMethod
  public synchronized void addCounter() {
      
    this.count++;

  }
  
  @DirectMethod
  public synchronized List<Experience> djn_loadExperienceData() {
      
      this.count++;
    
    List<Experience> items = new ArrayList<Experience>();
    Collections.addAll( items,
      new Experience( String.valueOf(count), "", "Programming, design and analysis in many projects, using Java, C#, C++, Smalltalk, Delphi, C"),[...]
    );
    return items;
  } 

web.xml:

<init-param>
      <param-name>batchRequestsMultithreadingEnabled</param-name>
      <param-value>false</param-value> 
    </init-param> 

In my script I call addCounter() multiple times an then I load a Grid with the data of
loadExperienceData. But String.valueOf(count) is always 1. Firebug shows that all calls are send in 1 request.

Can you help me?