1. #1
    Sencha - Community Support Team mschwartz's Avatar
    Join Date
    Nov 2008
    Location
    San Diego, Peoples' Republic of California
    Posts
    2,040
    Vote Rating
    7
    mschwartz will become famous soon enough

      0  

    Default Using SilkJS ORM and ExtJS Grids Together

    Using SilkJS ORM and ExtJS Grids Together


    SilkJS is a very fast JavaScript based HTTP server (http://silkjs.org) that was designed to work with ExtJS. All code for the server is written in JavaScript, including almost the entirety of the server itself. You can check out the server's github repository: http://github.com/mschwartz/SilkJS and the repository for this demo at http://github.com/mschwartz/SilkJS-ext.

    The SilkJS Install.js script creates a /usr/share/SilkJS directory with the HTTP server, library files, and default documentRoot. It also installs the main HTTP server in /usr/local/bin/httpd-silk.js.

    Let's examine the SilkJS-ext repository first.

    config.js:
    Code:
    Config.mysql = {
    	host: 'localhost',
    	user: 'mschwartz',
    	passwd: '',
    	db: 'ext3'
    };
    Config.documentRoot = 'docroot';
    This file overrides the default Config object, specifying MySQL connection information, and a documentRoot for this example.

    The actual Server-Side JavaScript code is implemented in < 60 lines of code, in Server.js:
    Code:
    SQL = new MySQL();
    SQL.connect();
    
    
    Schema.add({
    	name: 'Users',
    	fields: [
    		{ name: 'userId', type: 'int', autoIncrement: true, defaultValue: 0 },
    		{ name: 'username', type: 'varchar', size: 64, header: 'User Name', width: 128, autoExpand: true, editable: true },
    		{ name: 'password', type: 'varchar', size: 64, header: 'Password', serverOnly: true, editable: true },
    		{ name: 'email', type: 'varchar', size: 128, header: 'E-Mail Address', width: 128, editable: true },
    		{ name: 'created', type: 'int', defaultValue: function() { return parseInt(new Date().getTime() / 1000, 10); }, header: 'Created', width: 150, format: 'DateTime', editable: false }
    	],
    	primaryKey: 'userId',
    	indexes: [
    		'username',
    		'email'
    	]
    });
    
    
    function main_action() {
    	res.write([
    		'<!doctype html>',
    		'<html>',
    		'	<head>',
    		'		<title>Schema / ExtJS Demo</title>',
    		'		<link rel="stylesheet" type="text/css" href="/ext-3.4.0/resources/css/ext-all.css" />',
    		'	</head>',
    		'	<body>',
    		'	<script type="text/javascript" src="/ext-3.4.0/adapter/ext/ext-base.js"></script>',
    		'	<script type="text/javascript" src="/ext-3.4.0/ext-all-debug.js"></script>',
    		'	<script type="text/javascript" src="/client/Ext.ux.SchemaGrid.js"></script>',
    		'	<script type="text/javascript" src="/client/ViewPort.js"></script>',
    		'	<script type="text/javascript">',
    		'		Schemas = ' + Json.encode(Schema.getSchemaExtJs()) + ';',
    		'	</script>',
    		'	</body>',
    		'</html>'
    	].join('\n'));
    	res.stop();
    }
    function Server_action() {
    	switch (req.data.method) {
    		case 'listUsers':
    			Json.success(Schema.list('Users', {}, function(o) {
    				o = Schema.clean(o);
    			}));
    		case 'editUser':
    			var example = Json.decode(req.data.example);
    			example.userId = example.userId || 0;
    			Schema.putOne('Users', example);
    			Json.success();
    		case 'deleteUsers':
    			var examples = Json.decode(req.data.examples);
    			forEach(examples, function(example) {
    				Schema.remove('Users',example);
    			});
    			Json.success();
    	}
    }
    The first thing it does is create a global MySQL object, named SQL. This SQL object is used by the ORM to generate queries. With the ORM, you rarely have to generate any queries yourself, and even then, the ORM does most of the work.

    The Schema.add() method is called to install a "Schema" in the ORM. The Schema is defined as a JavaScript object. The definition above shows only a part of what's possible. The name member is the name of the table in the database. The fields array is an array of field definitions. The primaryKey member is the primary key of the table, and the indexes array is an array of additional indexes to be created for the table.

    For the fields array, each item is an object with a few required members and some optional ones. The required ones are name, and type. If type is 'varchar', then size is also required. Only one of the fields may be autoIncrement: true, and is typically the primaryKey of the table as well. This autoIncrement field is also typically the ID field of ExtJS DataStores on the client-side.

    The defaultValue member may be an expression or a function. The defaultValue member is used to set a default value for the field when creating a new entry in the database. As you can see in the example above, we have a 'created' field of type 'int' (a Unix-style timestamp) that has a defaultValue of a function that creates a Unix timestamp for the current time.

    One additional members may be specified. Note the password field is marked serverOnly: true - this is to specify that the value of this field should not be sent to the client (we'll see this later).

    Every other member of a field definition is ignored by the ORM, currently, and can be used for your application for whatever purposes you like. In this case, I specify information that will help us generate ExtJS Grid columns, and form fields for editing records. For example, username is autoExpand: true, and fields with header: 'some string' will show in the grids, etc.

    The SilkJS HTTP server will call main_action() (if it is defined) if / is the specified URL. Here we simply write a typical HTML document to the browser. The output HTML looks like this:

    Code:
    <!doctype html>
    <html>
    <head>
    <title>Schema / ExtJS Demo</title>
    <link rel="stylesheet" type="text/css" href="/ext-3.4.0/resources/css/ext-all.css" />
    </head>
    <body>
    <script type="text/javascript" src="/ext-3.4.0/adapter/ext/ext-base.js"></script>
    <script type="text/javascript" src="/ext-3.4.0/ext-all-debug.js"></script>
    <script type="text/javascript" src="/client/Ext.ux.SchemaGrid.js"></script>
    <script type="text/javascript" src="/client/ViewPort.js"></script>
    <script type="text/javascript">
    Schemas = {"Users":{"name":"Users","fields":[{"name":"userId","type":"int","autoIncrement":true,"defaultValue":0},{"name":"username","size":64,"header":"User Name","width":128,"autoExpand":true,"editable":true},{"name":"password","size":64,"header":"Password","serverOnly":true,"editable":true},{"name":"email","size":128,"header":"E-Mail Address","width":128,"editable":true},{"name":"created","type":"int","header":"Created","width":150,"format":"DateTime","editable":false}],"primaryKey":"userId"}};
    </script>
    </body>
    http://en.wikipedia.org/wiki/Query_by_Example.

    All records in SilkJS and ExtJS are effectively just JavaScript objects, each member is a column/value pair from the database. But not every field from the database must exist in these objects! A partial record of this form is what I call an "example." If you call Schema.find(example), it will do a SELECT query based upon the fields you do specify in the example. For example, Schema.find({ username: '%msch%' }) will return an array of records from the DB that have a username with 'msch' somewhere in it. Schema.putOne({ username: 'mschwartz' }) will store a new record in the database with defaultValues for the other fields and username set to 'mschwartz'. The only way I can explain examples is with examples! :)

    So editUser expects the client to build an example and send it over. If editing an existing record, the example will have a userId, and the record will be updated by Schema.putOne(). If adding a new record, the example will have no userId or a userId set to 0.

    The deleteUsers method expects an array of examples. It calls Schema.remove(example) on each element of the array, which deletes the specified rows from the DB. Schema.remove() can be dangerous if you're not careful about the examples you give it. In this case, the array of examples would be some multiple selections on the grid on the client side.

    The client-side ExtJS code is in docroot/client and accessed via /client/whatever paths in the HTML. In this case, we have just two files, Viewport.js and Ext.ux.SchemaGrid.js.

    ViewPort.js is rather simple:

    Code:
    Ext.onReady(function() {
    	var items = [];
    	for (var i in Schemas) {
    		var schema = Schemas[i];
    		items.push({
    			xtype: 'ext-ux-schemagrid',
    			title: i,
    			schema: schema,
    			method: 'list'+i
    		});
    	}
    
    
    	new Ext.Viewport({
    		layout: 'fit',
    		items: [
    			{
    				xtype: 'tabpanel',
    				activeTab: 0,
    				items: items
    			}
    		]
    	})
    });
    It loops through the global Schemas and creats an items array of Panel configs, one Schema Grid per panal. The Ext.ViewPort is simply a Ext.TabPanel with one tab per panel.

    The real work on the client is done by Ext.ux.SchemaGrid.js:
    Code:
    Ext.ux.SchemaGrid = Ext.extend(Ext.grid.GridPanel, {
    	initComponent: function() {
    		var me = this,
    			schema = me.initialConfig.schema,
    			fields = schema.fields,
    			id = Ext.id();
    
    
    		var store = new Ext.data.JsonStore({
    			root: 'list',
    			totalProperty: 'count',
    			idProperty: schema.primaryKey,
    			url: '/Server',
    			baseParams: {
    				method: me.initialConfig.method,
    			},
    			fields: fields,
    			listeners: {
    				load: function() {
    					me.updateToolbar();
    				}
    			}
    		});
    		var sm = new Ext.grid.RowSelectionModel({
    			singleSelect: false,
    			listeners: {
    				rowselect: function() {
    					me.updateToolbar();
    				},
    				rowdeselect: function() {
    					me.updateToolbar();
    				}
    			}
    		});
      		var columns = [],
    			fieldHash = {};
    		var autoExpandColumn = undefined;
    		for (var i in fields) {
    			var field = fields[i];
    			fieldHash[field.name] = field;
    			if (field.header && !field.serverOnly) {
    				if (field.autoExpand) {
    					autoExpandColumn = field.name;
    				}
    				columns.push({
    					header: field.header,
    					id: field.name,
    					format: field.format,
    					dataIndex: field.name,
    					width: field.width,
    					renderer: me.renderField
    				});
    			}
    		}
    		var config = {
    			sm: sm,
    			store: store,
    			columns: columns,
    			autoSizeColumns: true,
    			autoFill: true,
    			stripeRows: true,
    			trackMouseOver: true,
    			loadMask: true,
    			autoExpandColumn: autoExpandColumn,
    			buttonId: id
    		};
    		Ext.apply(this, Ext.apply(this.initialConfig, config));
    		me.tbar = new Ext.Toolbar({
    			items: [
    				{
    					text: 'Add',
    					id: 'add-'+id,
    					handler: function() {
    						me.editRecord({});
    					}
    				},
    				{
    					text: 'Edit',
    					id: 'edit-'+id,
    					handler: function() {
    						me.editRecord(me.getSelectionModel().getSelected().data);
    					}
    				},
    				{
    					text: 'Delete',
    					id: 'delete-'+id,
    					handler: function() {
    						me.deleteRecords();
    					}
    				}
    			]
    		});
    		me.bbar = new Ext.PagingToolbar({
    			store: store,
    			pageSize: 25,
    			displayInfo: true,
    			displayMsg: 'Displaying ' + me.initialConfig.schema.name + '{0} - {1} of {2}',
    			beforePageText: 'Page',
    			emptyMsg: 'Nothing to Display'
    		});
    		Ext.ux.SchemaGrid.superclass.initComponent.apply(me, arguments);
    		me.on({
    			'render': function() {
    				store.reload();
    			}
    		});
    		me.fieldHash = fieldHash;
    	},
    	updateToolbar: function() {
    		var me = this,
    			id = this.buttonId;
    		var selections = me.getSelectionModel().getSelections();
    		if (selections.length) {
    			Ext.getCmp('delete-'+id).enable();
    			if (selections.length == 1) {
    				Ext.getCmp('edit-'+id).enable();
    			}
    			else {
    				Ext.getCmp('edit-'+id).disable();
    			}
    		}
    		else {
    			Ext.getCmp('delete-'+id).disable();
    			Ext.getCmp('edit-'+id).disable();
    		}
    	},
    	editRecord: function(record) {
    		var me = this,
    			id = Ext.id(),
    			schema = me.initialConfig.schema,
    			fields = schema.fields,
    			primaryKey = schema.primaryKey;
    		record = record || {};
    		var items = [];
    		for (i in fields) {
    			var field = fields[i];
    			if (field.header && field.editable) {
    				items.push({
    					xtype: 'textfield',
    					fieldLabel: field.header,
    					anchor: '100%',
    					id: field.name + '-' + id,
    					name: field.name,
    					value: record[field.name]
    				})
    			}
    		}
    		var dialog = new Ext.Window({
    			title: record[primaryKey] ? 'Edit Record' : 'Add Record',
    			width: 640,
    			height: 480,
    			modal: true,
    			layout: 'fit',
    			items: [
    				{
    					xtype: 'form',
    					frame: true,
    					labelWidth: 150,
    					items: items
    				}
    			],
    			buttonAlign: 'center',
    			buttons: [
    				{
    					text: 'OK',
    					handler: function() {
    						for (var i=0, len = items.length; i<len; i++) {
    							var cmp = Ext.getCmp(items[i].id);
    							if (cmp) {
    								record[items[i].name] = cmp.getValue();
    							}
    						}
    						Ext.Ajax.request({
    							url: '/Server',
    							params: {
    								method: 'editUser',
    								example: Ext.encode(record)
    							},
    							success: function(response) {
    								dialog.close();
    								me.store.reload();
    							}
    						});
    					}
    				},
    				{
    					text: 'Cancel',
    					handler: function() {
    						dialog.close();
    					}
    				}
    			]
    		});
    		dialog.show();
    	},
    	deleteRecords: function() {
    		var me = this,
    			records = me.getSelectionModel().getSelections(),
    			len = records.length;
    		var examples = [];
    		for (var i=0; i<len; i++) {
    			examples.push(records[i].data);
    		}
    		Ext.Ajax.request({
    			url: '/Server',
    			params: {
    				method: 'deleteUsers',
    				examples: Ext.encode(examples)
    			},
    			success: function(response) {
    				me.store.reload();
    			}
    		});
    	},
    	renderField: function(value, p, r) {
    		var me = this,
    			format = me.format;
    		switch (format) {
    			case 'DateTime':
    				return new Date(value*1000).toString();
    			default:
    				return value;
    		}
    	}
    });
    
    
    Ext.reg('ext-ux-schemagrid', Ext.ux.SchemaGrid);
    I don't want to go into great detail about how this custom component works, but I will explain some of the things it does. I figure you all know how ExtJS works...




    SchemaGrid extends Ext.grid.GridPanel and takes as additional config options a Schema, and a method. Those are set in the ViewPort.js code.

    In the initComponent method, the URL of the JsonStore is set to /Server (so Server_action() is called on the SilkJS side), and baseParams method is set to the SchemaGrid's config.method, listUsers in our case. The JsonStore's fields are directly used from the Schema's fields array (no muss, no fuss!).

    After setting up the SelectionModel, the code loops through the Schema's fields and generates column definitions for the grid. Those fields that have a header member and are not serverOnly will have columns displayed. The renderer for each column is a single method in the SchemaGrid class. The default action for our renderer is to render the created/timestamp in human readable format. An autoExpand column may be specified in the Schema definition, and it is handled automatically by this code.

    When the JsonStore is loaded or the user clicks on items in the grid, the updateToolbar() method is called to conditionally enable and disable the edit and delete buttons. Basically, you can only edit if ONE row is selected, and you can't delete unless at least one row is selected.

    The editRecord() method dynamically creates a form in a modal dialog window. If a record is passed in, the dialog title says "Edit record" otherwise "Add record." The dialog/form can be used to add new records (click the add button) or edit existing records (select a record, click edit button).

    When the OK button is clicked on the dialog, the record (EXAMPLE) is updated with the form fields' values and the Ext.Ajax.request() made to call editUser to update the database.

    The deleteRecords() method is called to delete one or more selected records in the grid. It creates the array of examples and calls Ext.Ajax.request() to invoke deleteUsers on the server side.

    So let's see it in action!

    The attached snapshot1.png is the application when it first starts.

    The attached snapshot2.png is the application after I click the add button.

    The attached snapshot3.png is the application after I've added a few records.

    The attached snapshot4.png is the application after I've selected the bbb record and clicked edit.
    Attached Images

  2. #2
    Sencha - Community Support Team mschwartz's Avatar
    Join Date
    Nov 2008
    Location
    San Diego, Peoples' Republic of California
    Posts
    2,040
    Vote Rating
    7
    mschwartz will become famous soon enough

      0  

    Default


    So what's so neat about this?

    If you edit the Schema definition at the top of Server.js, those changes will immediately be reflected in the database when you either restart the server or the browser makes a new request (reload the browser, click the reload button in the paging toolbar).

    What do I mean? If you change the size of password field to 128, the ORM will submit the appropriate ALTER TABLE statements to MySQL to change the size of the field. If you add a field to the fields array, it will be added to the table in the DB as well. If you delete a field, it will be deleted in the DB. If you rename a field, it will be renamed in the DB.

    If you add a second Schema definition at the top of Server.js, the table will be created next access. If you reload your browser, you will see TWO tabs, one for each table, and the CRUD operations work on each.

    Outside of the Schema definitions, which are compact in their own right, and the main_action() method that spews out the HTML for the app, the entire CRUD operations on the server side consist of under 20 lines of code.

    The entire demo took me under 1 hour to create, all but 2-3 minutes was spent writing the client-side code.

  3. #3
    Sencha User edykstra's Avatar
    Join Date
    Feb 2009
    Posts
    97
    Vote Rating
    0
    edykstra is on a distinguished road

      0  

    Default Can SilkJS and ExtJS share models and other code?

    Can SilkJS and ExtJS share models and other code?


    Hey mschwartz,

    I have used ExtJS a lot for complex desktop (RIA) type apps. These projects have been Rails on the server side. I see a LOT of code duplication to define models etc. on each side.

    I have a new project being planed, and I will be doing both the server and client side code. While I can poke around a bit in Rails, there is MUCH for me to learn still, and I am not sure I want to climb that learning curve. The time may be better spent learning NodeJS and related technologies.

    The app will have Responsive and Mobile interfaces for 'consumers' and a very complex RIA 'webtop' in ExtJS for 'administrators'. There will also be a very scaled down mobile interface for 'administrators' too.

    I don't see the need for 'real-time' via sockets or Meteor etc., but some of the requirements make me think that is a possibility. Mostly, I see for those few instances, I could use ExtJS polling initially, but that may not scale well server side if this app becomes popular.

    So, I guess I am asking if you could give an update to this thread. Is SilkJS still the way to go? Does it allow common model code for client/server side with ExtJS 4+? Any other suggestions working with ExtJS and server side JS?

    Thanks!

    Eric

  4. #4
    Sencha - Community Support Team mschwartz's Avatar
    Join Date
    Nov 2008
    Location
    San Diego, Peoples' Republic of California
    Posts
    2,040
    Vote Rating
    7
    mschwartz will become famous soon enough

      0  

    Default


    Quote Originally Posted by edykstra View Post
    Hey mschwartz,

    I have used ExtJS a lot for complex desktop (RIA) type apps. These projects have been Rails on the server side. I see a LOT of code duplication to define models etc. on each side.

    I have a new project being planed, and I will be doing both the server and client side code. While I can poke around a bit in Rails, there is MUCH for me to learn still, and I am not sure I want to climb that learning curve. The time may be better spent learning NodeJS and related technologies.

    The app will have Responsive and Mobile interfaces for 'consumers' and a very complex RIA 'webtop' in ExtJS for 'administrators'. There will also be a very scaled down mobile interface for 'administrators' too.

    I don't see the need for 'real-time' via sockets or Meteor etc., but some of the requirements make me think that is a possibility. Mostly, I see for those few instances, I could use ExtJS polling initially, but that may not scale well server side if this app becomes popular.

    So, I guess I am asking if you could give an update to this thread. Is SilkJS still the way to go? Does it allow common model code for client/server side with ExtJS 4+? Any other suggestions working with ExtJS and server side JS?

    Thanks!

    Eric
    Hey Eric,

    I'm absolutely using SilkJS with the ORM to implement the server side for ExtJS4.

    https://github.com/mschwartz/SilkJS-extjs4

    This demo implements Ext.ux.SchemaGrid class. The Grid implements a toolbar with CRUD buttons (new, edit, delete), and a popup window with form in it to edit records. The grid columns, form fields, and the database schema (table definitions) are all defined on the server side. If you edit the server side code and add columns, delete columns, etc., the database is altered (alter table) accordingly and you'll see new columns and form fields on the client side to match.

    I've started with that demo to create two different ExtJS projects in the past few months.

    If you have any issues getting any of this to run, PM me.

    As for polling, I measure 0-4ms for some pretty complicated short poll requests using SilkJS on a micro type virtual machine instance. In theory, you should be able to get at least 250 clients polling at 1/second, 500 at 2/second, etc. Scale it like you would apache (server farm behind load balancer) and you can do a huge number of clients.

    The http://silkjs.net/ WWW site is entirely served by SilkJS.

    Cheers