PDA

View Full Version : Large Data Sets



JackieLip
17 Apr 2012, 7:03 AM
What is the best option for storing large data sets for use? I am talking about a few hundred thousand to a million rows.

Thanks!

scottmartin
17 Apr 2012, 7:06 AM
I am going to present the obvious .. a database? MySQL, PostgreSQL, etc? Did you have more to this question?

Regards,
Scott.

JackieLip
17 Apr 2012, 7:10 AM
Well currently my data set is in a form of a text file. I could change it to an access file. I could store it on my SQL server. I could even convert it to a pdf file if it helps (obvious joke...) I am more interested with what is the best and easiest option or even what most people with large data sets do.

Thanks again.

scottmartin
17 Apr 2012, 7:56 AM
I vote for a SQL server.
No need to stumble through a large text file.. Users can only look at so much data at once .. so return what is needed, fast as possible.

Scott.

JackieLip
17 Apr 2012, 7:59 AM
Cool Beans! Thanks. Could you forward me to some documentation about implementing SQL server using java script. I only have experience doing this with asp.net. Currently running through some of the Sencha Touch 2 MVC examples and instead of XML I want to use SQL Server (as you suggested).

Thanks again!

scottmartin
17 Apr 2012, 8:12 AM
I usually suggest using a server based language such has PHP,Java (Spring), etc. You can have a look at
node.js or silkjs and see if they help.

Regards,
Scott.