Results 1 to 7 of 7

Thread: Can you and should you do screen scraping

  1. #1

    Default Can you and should you do screen scraping

    One of the developers is suggesting that we allow the server to generate all of the html on the server and just display that into the widgets that are in Ext JS. Being that we are new to the world of Ext JS, is this possible and if it is, is it even a good idea? I am initially thinking no! Kind of like the idea of taking a Mercedes and sticking seats from a Pinto in it. Sure it can be done but its just not right.

    The data that would be in the generated html could be anything such as static text, links, layouts, content with javascript popups etc.

    Some input on this would be greatly appreciated.

  2. #2
    Sencha User jay@moduscreate.com's Avatar
    Join Date
    Mar 2007
    Location
    DC Area =)
    Posts
    16,364

    Default

    that is one way of doing things, but are you sure?

    That said, what does that have to do with "screen scraping"
    http://en.wikipedia.org/wiki/Data_sc...creen_scraping

  3. #3
    Sencha User realjax's Avatar
    Join Date
    Aug 2009
    Location
    Netherlands
    Posts
    412

    Default

    Well,

    - either generate 1000 html docs for 1000 users on your server or
    - have 1000 clients take care of it themselves..

  4. #4

    Default

    Quote Originally Posted by [email protected] View Post
    that is one way of doing things, but are you sure?

    That said, what does that have to do with "screen scraping"
    http://en.wikipedia.org/wiki/Data_sc...creen_scraping
    "Screen scraping" The other developers words. I thought it odd but that's how he described it.

    So, if it can be done, what should I be looking for keyword searching in the forum to find examples of how this is to be done?

    And the 2nd part of my question is should it be done?

  5. #5
    Sencha User realjax's Avatar
    Join Date
    Aug 2009
    Location
    Netherlands
    Posts
    412

    Default

    Quote Originally Posted by BlackBerry View Post
    And the 2nd part of my question is should it be done?
    No, see my previous post and think about it.

    Besides: I'm guessing you'll run in to all sorts of DOM related issues sooner then later.

  6. #6

    Default

    realjax,

    Either way its a thousand hits against the server by a thousand users. But my thoughts are relatively the same I believe as yours.
    1. At some point we are going to run into issues where we have the visual content being done in two locations (client, server)
    2. Let the client machines do more work instead of just pulling views.

    I told the other developer that the content from the server should be strictly subject content and not display and that by providing only the requested information and not a display of that information the we can manage how we display that information across any system or even provide the information as a service. (e.g. If I make a request for a users first and last name, then the server should only return the users first and last name in a predefined structure and not html injected code on how to display since I am not asking for the information and how to display it)

    I also told him that I would prefer to get the information back as JSON and not XML.

    Has anyone been down this road and run into known issues of allowing the server to generate the display code to push into the widgets?

  7. #7

    Default

    I suggest that you should "look around" and "kick the tires" a little bit more, before striking a decision in the matter.

    Categorically, this kind of web-application is "server-provided client-server." In other words:


    1. The initial "programming" is obtained from the server, at a particular URL. Whatever the server chooses to supply is what it gets.
    2. The client-side then obtains additional source-code by following the supplied links. Somewhere, another server supplies every one of those responses.
    3. The client-side then takes a more active role in the process, executing local programs and making periodic Ajax-based requests... thereby requesting that the server-side perform certain tasks and/or to supply certain data.
    4. The client-side can also obtain more JavaScript code (from whatever source) and execute it, literally incorporating it into its client-side operating environment which consists of both code-objects and data.


    So it's rather a mix of points-of-view: the server is in full control of what is possible, but the client is in charge of what is attempted. The end-user is, by and large, in charge of what takes place, but within the limits proscribed by both client and server.

    I very, very respectfully suggest that you need to "review the lay of the land," and the nature of the technology and its options, as your next priority before embarking upon a course of action. This is a rich landscape. Some things shine brightly; some await discovery.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •