1. #1
    Sencha User
    Join Date
    Feb 2013
    Posts
    8
    Vote Rating
    0
    rattler is on a distinguished road

      0  

    Default Unanswered: Grid rendering very slow on adding data every second

    Unanswered: Grid rendering very slow on adding data every second


    I am trying to make a live dashboard application with grid and charts on it. The data is supplied by rest service in JSON format.

    I am using the following code to add the data to store. The grid and charts get automatically refreshed once the data is added to the store. I am using Ext.data.Store , Ext.grid.Panel and Ext.chart.Chart.

    The problem is that the grid becomes unresponsive after just a few seconds and scrolling hangs. I have also tried to refresh it every 10 seconds, still it becomes unresponsive after 10-15 seconds. I am adding approximately 6 rows every second in the grid.

    Code:
    setInterval( function() {		d3.json("http://localhost:8080/output-console/rest/getCSVList", function(jsondata) {
    store.add(jsondata);
    });
    }, 1000 );
    Can someone please help and advise if there is a better way to do it?

    PS: I am using D3 just because I was using the same in my previous testing. If someone thinks, that can be the issue, i can replace it.

  2. #2
    Sencha User
    Join Date
    Mar 2012
    Location
    Oklahoma, USA
    Posts
    52
    Answers
    10
    Vote Rating
    8
    sdt6585 will become famous soon enough

      0  

    Default


    What exactly is coming back in that json? If it's the entire store worth of data, I could see that causing problems. You will need a way to only add the new records rather than updating the entire thing. Maybe a webservice call that you send the timestamp of your last request and it sends back a json object that contains an add and remove object so you can add and remove as necessary. Or if it's reading from a data source that generally only gets new data, maybe just send the time stamp in and get back an object with all the new records.

  3. #3
    Sencha User
    Join Date
    Feb 2013
    Posts
    8
    Vote Rating
    0
    rattler is on a distinguished road

      0  

    Default


    Yes, I am just sending the delta updates via that JSON call, i.e Only the new rows are being sent from the server. Then I am adding those rows to the data store using store.add().

  4. #4
    Sencha User
    Join Date
    Mar 2012
    Location
    Oklahoma, USA
    Posts
    52
    Answers
    10
    Vote Rating
    8
    sdt6585 will become famous soon enough

      0  

    Default


    Well...without having it to play with I'm probably not much more help. If you're concerned that using the d3.json call has performance issue, you could certainly switch to Ext.Ajax.request() or configure a proxy on the store and just call store.sync(). If you get a chance and it's not confidential info in that request, you might stick a demo up on jsfiddle so someone can trace through the calls to find the performance culprit. Good luck.

  5. #5
    Sencha - Support Team scottmartin's Avatar
    Join Date
    Jul 2010
    Location
    Houston, Tx
    Posts
    9,010
    Answers
    667
    Vote Rating
    460
    scottmartin has a brilliant future scottmartin has a brilliant future scottmartin has a brilliant future scottmartin has a brilliant future scottmartin has a brilliant future scottmartin has a brilliant future scottmartin has a brilliant future scottmartin has a brilliant future scottmartin has a brilliant future scottmartin has a brilliant future scottmartin has a brilliant future

      0  

    Default


    You will need to provide a test case for us to continue ...

    I am adding approximately 6 rows every second in the grid
    Are you checking for collision of your data? Once every second it a bit much .. at your 10 second interval, are you only returning 6 new records? if you stop your interval timer, does the grid become responsive?

    Scott.

  6. #6
    Sencha User
    Join Date
    Feb 2013
    Posts
    8
    Vote Rating
    0
    rattler is on a distinguished road

      0  

    Default


    I have posted the code , couple of times. But my post keeps getting lost
    ---
    I am doing a proof of concept actually powering Grid and charts using the same data source. (Still needs to figure out how to do grouping for a time series, whereas same data would be without grouping in the grid).


    The actual dashboard I intend to built can have multiple charts and grids using same or different data sources. They will be based on the push model (may be comet). And its quite possible to have this amount of data flowing in that dashboard.


    To answer your second question, its approximately 6 records per second and will be 60 records for 10 seconds. The page becomes completely unresponsive after some time and looks like it becomes CPU bound. The CPU utilisation goes upto 100% for one core from 15-20% without the chart (with just a grid).


    Please see the code below:
    HTML:


    Code:
    <html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
        <title>Infinite Scroll Grid Example</title>
    
    
    
    
        <link rel="stylesheet" type="text/css" href="../extjs/resources/css/ext-all.css"/>
        <link rel="stylesheet" type="text/css" href="../extjs/example.css" />
    
    
    
    
        <!-- GC -->
    
    
    
    
        <script type="text/javascript" src="../extjs/ext-all.js"></script>
    
    
    
    
        <script type="text/javascript" src="infinite-scroll.js"></script>
        
        <script src="../vendor/d3.min.js"></script>
    <script src="../vendor/d3.layout.min.js"></script>
    <script src='http://d3js.org/d3.v2.js'></script>
        
    </head>
    <body>
        <h1>Infinite Scrolling</h1>
       
    </body>
    </html>

    JavaScript File
    Code:
    // Set up a model to use in our Store
    	Ext.require('Ext.chart.*');
    Ext.require(['Ext.Window', 'Ext.fx.target.Sprite', 'Ext.layout.container.Fit', 'Ext.window.MessageBox']);
    
    
    Ext.onReady(function(){
    Ext.define('Ext.app.GridPortlet', {
        extend: 'Ext.data.Model',
        fields: [
            {name: 'name', type: 'string'},
            {name: 'price',  type: 'float'},
            {name: 'time',       type: 'double'},
        ]
    });
    
    
    var store = Ext.create('Ext.data.Store', {
        model: 'Ext.app.GridPortlet',
        autoLoad: true,
        pageSize: 4,
    //    groupField: 'name',
        proxy: {
            type: 'ajax',
            url : 'http://localhost:8080/output-console/rest/getCSVList',
            reader: {
                type: 'json',
                root: 'stocks',
                totalProperty: 'count'
            }
        }
    });
    var grid = Ext.create('Ext.grid.Panel', {
        renderTo: Ext.getBody(),
        store: store,
        width: 400,
        height: 200,
        title: 'Application Users',
        invalidateScrollerOnRefresh: false,
        columns: [
            {
                text: 'name',
                width: 100,
                sortable: false,
                hideable: false,
                dataIndex: 'name'
            },
            {
                text: 'price',
                width: 150,
                dataIndex: 'price',
                
            },
            {
                text: 'time',
                flex: 1,
                dataIndex: 'time'
            }
        ]
    //    ,features: [{ftype:'grouping'}]
    });
    var chart = Ext.create('Ext.chart.Chart', {
        xtype: 'chart',
        style: 'background:#fff',
        animate: true,
        width: 900,
        height: 500,
        store: store,
        shadow: true,
        theme: 'Category1',
     renderTo: Ext.getBody(),
        legend: {
            position: 'right'
        },
        axes: [{
            type: 'Numeric',
            minimum: 0,
            position: 'left',
            fields: ['price'],
            title: 'Number of Hits',
            minorTickSteps: 1,
            grid: {
                odd: {
                    opacity: 1,
                    fill: '#ddd',
                    stroke: '#bbb',
                    'stroke-width': 0.5
                }
            }
        }, {
            type: 'Category',
            position: 'bottom',
            fields: ['name'],
            title: 'Month of the Year'
        }],
        series: [{
            type: 'line',
            highlight: {
                size: 7,
                radius: 7
            },
            axis: 'left',
            xField: 'time',
            yField: 'price',
            markerConfig: {
                type: 'cross',
                size: 4,
                radius: 4,
                'stroke-width': 0
            }
        }]
    });
    setInterval( function() {
    		d3.json("http://localhost:8080/output-console/rest/getCSVList", function(jsondata) {
    			var stocks = jsondata.stocks;
    			for(var i =0;i<stocks.length;i++){
    				store.add(stocks[i]);	
    			}
    			
    		});
    //	Ext.Ajax.request({
    //	    url: 'http://localhost:8080/output-console/rest/getCSVList',
    //	    success: function(response){
    //	        var stock = response.responseText;
    //			for(var i =0;i<stock.stocks.length;i++){
    //			store.add(stocks[i]);	
    //		}
    //	        
    //	    }
    //	});
    }, 3000 );
    
    
    
    
    });

    Sample JSON Data:


    Code:
    {"stocks":[{"name":"IBM","price":105.0,"time":1.360986119E9},{"name":"Facebook","price":314.0,"time":1.360986119E9},{"name":"Google","price":529.0,"time":1.360986119E9},{"name":"IBM","price":105.0,"time":1.360986119E9},{"name":"Facebook","price":314.0,"time":1.360986119E9},{"name":"Google","price":529.0,"time":1.360986119E9},{"name":"IBM","price":105.0,"time":1.360986119E9},{"name":"Facebook","price":314.0,"time":1.360986119E9},{"name":"Google","price":529.0,"time":1.360986119E9}],"count":9}

  7. #7
    Sencha - Ext JS Dev Team evant's Avatar
    Join Date
    Apr 2007
    Location
    Sydney, Australia
    Posts
    16,937
    Answers
    460
    Vote Rating
    635
    evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute

      0  

    Default


    The loading code doesn't make a whole lot of sense. You're looping over each item and calling loadData, why is that?
    Evan Trimboli
    Sencha Developer
    Twitter - @evantrimboli
    Don't be afraid of the source code!

  8. #8
    Sencha User
    Join Date
    Feb 2013
    Posts
    8
    Vote Rating
    0
    rattler is on a distinguished road

      0  

    Default


    Quote Originally Posted by evant View Post
    The loading code doesn't make a whole lot of sense. You're looping over each item and calling loadData, why is that?
    I am trying to build a real time dashboard. This code is continuously fetching delta updates from the backend. In this example, it is fetching stock price updates for 3 stocks from the backend (Tomcat based REST service) , every second.

    I intend to display all the updates in a grid and showcase them on a series chart. Grid is working perfectly fine, but when i add a series chart, its not able to cope up with these updates and the browser tab keeps crashing. I am using chrome, but have also tried it on safari.

    I expect it to crash after a while when the data becomes huge. I understand in a real world scenario, I will have to manage it differently. But my concern is that, even after 10-15 seconds of data, the page becomes unresponsive. I tried the same with D3/Rickshaw library earlier and the series use to work fine for 10-15 mins.

    PS: My Bad, the code I pasted above is not correct. I am using Store.add actually. It was not taking my json object as such, so I had to loop through. I will edit the post above

    UPDATE:
    I have removed the loop and using this now, changed the update from 1 to 3 seconds or 10 seconds.

    <CODE>
    setInterval( function() {
    d3.json("http://localhost:8080/output-console/rest/getCSVList", function(jsondata) {
    var stocks = jsondata.stocks;
    //for(var i =0;i<stocks.length;i++){
    store.add(stocks);
    //}
    });

    }, 3000 );
    </CODE>
    Still the page is becoming completely unresponsive. Do we have any examples of live dashboard being using EXTJS where a grid and chart is sharing a data source ?

  9. #9
    Sencha - Ext JS Dev Team evant's Avatar
    Join Date
    Apr 2007
    Location
    Sydney, Australia
    Posts
    16,937
    Answers
    460
    Vote Rating
    635
    evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute evant has a reputation beyond repute

      0  

    Default


    You should call suspend/resumeEvents on the store, then refresh the view if you're bulk adding multiple records.
    Evan Trimboli
    Sencha Developer
    Twitter - @evantrimboli
    Don't be afraid of the source code!

  10. #10
    Sencha User
    Join Date
    Feb 2013
    Posts
    8
    Vote Rating
    0
    rattler is on a distinguished road

      0  

    Default


    I have added suspend/resume events as shown below. But the grid/chart is not refreshing automatically. Do i need to explicitly call refresh on the whole view or grid/chart ? Sorry, I am new to EXTJS.


    Also I am not doing any bulkloading, this code is polling data from the server and adding the delta updates to datastore (approx 5-6 rows per second) .
    <CODE>
    setInterval( function() {
    store.suspendEvents();
    d3.json("http://localhost:8080/output-console/rest/getCSVList", function(jsondata) {
    var stocks = jsondata.stocks;
    //for(var i =0;i<stocks.length;i++){
    store.add(stocks);
    store.resumeEvents();
    //}


    });
    </CODE>
    Quote Originally Posted by evant View Post
    You should call suspend/resumeEvents on the store, then refresh the view if you're bulk adding multiple records.

Thread Participants: 4

Tags for this Thread