Executing multi-valued web service calls from javascript

Monday, October 31, 2005

I'm currently working hard on a web based business system for a theatre agency here in Denmark. Client-side business logic communicates through web service calls with server-side business logic.

I'm using the web service proxies that Matthias Hertels AJAX Engine generates, but I was unable to use ajax.Start to execute multi-valued calls.

Maybe I overlooked something? :) Anyway, I came up with a fix for it...

This an example of a multi-valued call:

  ...
  var Add = {
    delay: 200,
    prepare: function() { return [34, 23]; },
    call: proxies.AddService.Add,
    finish: function (p) {
      alert("Sum: " + p);
    },
    onException: proxies.alertException
  };

  ajax.Start(Add);
  ...

I changed some lines in Ajax.js. After this:

      ...      
      // start the call
      ca.call.func = ajax.Finsh;
      ca.call.onException = ajax.Exception;

I added this:

      // multi-valued call (new)
      if ((typeof(data)=="object") && (data!=null) && data.length) {
        ca.args = new Object();
        var aArg = new Array();
        for (var i = 0; i<data.length; i++) {
          ca.args["a" + i] = data[i];
          aArg[aArg.length] = "a" + i;
        }
        with (ca.args) {
          eval("ca.call(" + aArg.join(",") + ")");
        }
      }
      // single-valued call
      else { 
        ca.call(data);
      }      
      ...

And it worked. I hope you can use this.

OutPost: Near future roadmap

Wednesday, October 19, 2005

As soon as I have a version of OutPost that is ASP.NET version independent (I'm working on that now) I'll optimize more on the amount of data sent back to the client.

I have just added support for Netscape 8 and Opera 8.5 in my development version, and I'll post a new beta version of that soon. I also got OutPost working nicely with the Personal Web Starter Kit for ASP.NET 2.0 RC.

OutPost 1.1 beta (ASP.NET 2.0 only)

Saturday, October 15, 2005

Great news for beta testers: You can download the 1.1 beta ASP.NET 2.0 version of OutPost now.

It doesn't work in FireFox yet, because I haven't been able to make the tags behave like they should.

Here are some of the new features / bug fixes:

1.1.000 - 20051014 1700 twj
Support for HTTPS/SSL.
Support for Visual Studio 2005 Release Candidate.

1.0.002 - 20051007 1125 twj
Bug fixed in javascript Serializer: Unicode characters were not escaped properly.
Optimization of the JavaScript property by using StringBuilder.
Default values are stored before PostBack and restored afer PostBack.

1.0.001 - 20051004 1825 twj
Bug fixed in constructor: PartsNeeded is only initialized as a StringArray if the "PartsNeeded" header is set.
Bug fixed in window_onload function: Script error if no forms present on page.
Added version comment to client-framework javascript.

1.0.000 - 20051003 0149 twj
Initial article release.

OutPost: Performance Tuning and Optimization

Wednesday, October 12, 2005

The same principles for Performance Tuning and Optimization apply when developing with OutPost. You optimize by taking advantage of the view-state, page level caching, page fragment caching, application caching, session caching and the web server Cache.

With OutPost you can actually get further performance improvements:
- Splitting up your page and having more web forms on the same page, like in "The iGoogle Demo", is a great idea.
- Finding the balance of when to use the view-state and when not to is also important and would often require you to test your application and analyze the load times to find the right settings.
- Setting the IDs of the HTML elements that frequently get updated is a good idea, because OutPost compares the innerHTML of elements with IDs.

I'll grant you that AJAX is much nicer to the server. But I would say that OutPost is much nicer to the developer

I like the simple structure of ASP.NET Web Forms and the way that you can develop ASP.NET controls for different browsers (with or without DHTML, depending on the capabilities of the browser).

The improvements to ASP.NET in upcoming versions (e.g. the view-state) will also have impact on the performance in OutPost web applications.

The improvements that you mention in ASP.NET 2.0 that you can write asynchronous pages will also work in combination with OutPost.

ATLAS will also work in combination with OutPost to provide extra RIA features to up-level browsers.

OutPost: Future Improvements

Tuesday, October 11, 2005

I am working on an easy way to make web service calls from JavaScript and I will include that in OutPost, so that you don't have create an invisible web form with text boxes and buttons if you want to call the server. I think it will be inspired heavily by ATLAS. I would really like to include more ATLAS features in OutPost.

OutPost: How I got the Idea

Monday, October 10, 2005

It is a novel solution. I got the idea, while I was developing the framework for a web site editor called UpFront in 2002, where the user designs his pages by inserting text, images, data lists, web forms and navigation menus on the pages.

The language was ASP (JavaScript) on the server-side and JavaScript on the client. I had two parallel frameworks to create data lists, web forms and navigation menus (one on the client-side and one on the server-side). The client-side framework was used when dynamically creating or updating the properties of a data list, web form or navigation menu in the editor (with heavy use of DHTML and XmlHttp calls to retrieve dataset in JSON - developed on top of JSRS). The server-side framework was used when rendering HTML on the first page load.

I found out that I could trash the client-side framework by calling an ASP page that returned the HTML and JavaScript needed over XmlHttp. This was a huge improvement i productivity, because I had one framework less to maintain and I could develop new "controls" faster.

This solution also gave me the advantage that I could gradually convert to C#-compiled code and I did so by calling a .NET web service over XmlHttp for the data list and navigation menus to start of with, and I still used a page (now an ASP.NET page) to render data lists and navigation menus. Now I could take advantage of the ASP.NET DataGrid for my lists, but in order to make it work properly I had to support the view-state and the server-events, so I examined the headers and the body of post backs to ASP.NET pages (using Fiddler and other HTTP debugging tools) and I was able to simulate it. So now I had my data lists and navigation menus rendered like stand-alone .NET pages with support for State Management, Server Events, Page-Level Caching, Page Fragment Caching, Session, Application and Data Cache. The HTML that the page generated contained the view-state, but I didn't have to return that to the client, since it's not used for anything on the client, so I stored it in Application together with the HTML. In that way I could compare HTML from request to request and didn't have to send more than necessary to the client. The view-state was combined with the form data from the client to make the request to the ASPX page.

So that's how I got the idea. As you can see I started out with Remote Scripting and DHTML (AJAX!). I moved away from that primarily to get productivity gains. OutPost is cross-browser compatible because it relies on ASP.NET controls that are cross-browser compatible. That also means that when you use OutPost on an up-level browser JavaScript will be transmitted on the first page load to take advantage of the capabilities in the client browser, and less HTML will be transmitted on post backs or no post backs will be necessary at all. So it will not be a screen scraping on up-level browsers. If the client browser is a downlevel-browser more HTML will be transmitted. In the examples in the demo project I have set the Page attribute clienttarget="downlevel" to force it to make a post back, but you can remove that attribute and no post backs will be made.

OutPost: Creating Interactive Web Applications

Sunday, October 09, 2005

When you are creating interactive web applications with AJAX (any AJAX framework), you need a solid understanding of the DOM and CSS in different browsers. DHTML can be a challenge for any developer - you have to test your application rigorously in different browsers. And browsers do change. You have to write the DHTML for your controls and you have to make it cross-browser compatible yourself. ATLAS lends the developer a hand by introducing its layers of DOM compatibility. The DOM is extended by adding methods to the document and window element (as I describe it in the article). And that is great if you need to write DHTML! But with OutPost you don't! You just have to use stardard ASP.NET controls that are able to output HTML for downlevel browsers. If the browser is up-level it outputs HTML+JavaScript, and if the browser is down-level is outputs HTML. If you installed the demo project, you can try this out by removing the clienttarget="downlevel" attribute in the Page directive of Validator7.aspx or Validator8.aspx. You'll then notice that with up-level browsers fewer XmlHttp-calls are now made, because the validation is done with JavaScript.

OutPost: Post Back Over XmlHttp

Wednesday, October 05, 2005

Perform real Post Backs of ASP.NET Web Forms without reloading the entire page.
Keywords: AJAX, XmlHttp, WebForms, Post Back, DHTML, Web Service, OutPost.

A screenshot of The iGoogle Demo
Screenshot of "The iGoogle Demo" in the demo project

Introduction

This article assumes that you are familiar with ASP.NET WebForms. You might also want to know the basics about AJAX. This is a quote from Wikipedia on AJAX:

Ajax applications look almost as if they reside on the user's machine, rather than across the Internet on a server. The reason: pages get updated, not entirely refreshed.

“Every user action that normally would generate an HTTP request takes the form of a JavaScript call to the Ajax engine instead”, wrote Jesse James Garrett, in the essay that first defined the term. “Any response to a user action that doesn’t require a trip back to the server — such as simple data validation, editing data in memory, and even some navigation — the engine handles on its own. If the engine needs something from the server in order to respond — if it’s submitting data for processing, loading additional interface code, or retrieving new data — the engine makes those requests asynchronously, usually using XML, without stalling a user’s interaction with the application.”

Traditional web applications essentially submit forms, completed by a user, to a web server. The web server responds back by sending a new web page. Because the server must submit a new page each time, applications run more slowly and awkwardly than their native counterparts.

Ajax applications, on the other hand, can send requests to the web server to retrieve only the data that is needed, usually using SOAP or some other XML-based web services dialect. On the client, JavaScript processes the web server response. The result is a more responsive interface, since the amount of data interchanged between the web browser and web server is vastly reduced. Web server processing time is also saved, since much of it is done on the client.

AJAX is a maintenance nightmare

Today you have to create web applications that work in both IE and Firefox, and that is definitely doable but also pretty time consuming and a maintenance nightmare.

AJAX has become very popular, because it answers a need for speed and responsiveness in web applications and there has also been the "Google" effect:

- "We create interactive web applications like Google does and they use AJAX too",

yeah, but it is not so cool, if you use 95% of the time on testing in different browsers.

AJAX falls short on two major points: It has no support of the state management in ASP.NET Web Forms and it has no support for cross-browser web applications. The goal of this article is to solve these two issues.

Example

Lets say you have a tree-structure displayed to the user (like in Windows Explorer) and you only want to retrieve and display the sub-folders when the user expands a folder in the tree.

With AJAX you would call a web service to retrieve an array containing the subfolder names, links and icons and create the tree nodes using DHTML. You would have to create the web service methods, write cross-browser javascript to make it work and have hidden fields in your ASP.NET WebForm containing the state of the control.

What OutPost does is to perform a real Post Back through a Web Service that also manages the view-state (and keeps it on the web server). The InnerHtml of the modified elements are sent back to the client and displayed in the browser. In that way the whole process is generalized and you don't need to create Web Service methods, DHTML javascripts or add hidden fields for your ASP.NET Controls.

So obviously you get considerable productivity gains by using OutPost to AJAX-enable your web applications.

What makes OutPost unique?

OutPost is a new angle on AJAX. Other AJAX frameworks use specifically tailored Web Services, that you have to write yourself to return the data you need. And none of the frameworks fully support the benefits of the ASP.NET Framework. With AJAX you have to re-think and re-write your code.

Unique advantages of OutPost

  • A single .js include file is enough for OutPost.
  • OutPost converts any ASP.NET WebForm into an AJAXed WebForm without any modifications to the code.
  • It has cross-browser support built-in.
  • The Post Backs are 100% real and server-side events are triggered, as you'd expect them to trigger.
  • Multiple server-side WebForms (.aspx-pages) are supported side by side on the same page.
  • Excess traffic between client and server is eliminated (the view-state stays on the server and only modified HTML is sent back to the client).
  • A lightweight javascript client-framework makes it possible to add, remove and Post Back WebForms by script.

Other OutPost features

  • Output caching is supported on WebForm-level.
  • Unicode characters are supported.
  • A WebForm has its own Session.
  • No custom controls are needed.
  • Main pages and WebForms can be designed in VS.NET.

OutPost-enabling a single WebForm

The decision to add "AJAX" to your pages is not a question of "do we have the time and skill to implement it". Instead it becomes a design question.

To add OutPost and AJAX-enable a WebForm, all you need is to write the following in the beginning of your .aspx file:

  <script src="/OutPost/OutPost.js.aspx" type="text/javascript"></script>

But let us just zoom out for a second:

Background

When developing interactive applications for the web, you have different models to use, and two of them are shown below. They can be combined in many ways, but if they are not combined the following is a non-pre-emptive comparison of their advantages and disadvantages:

ASP.NET WebForms

+FAST AND SAFE: Compiled code, type safe code model.
+OPEN OOP MODEL: Object oriented programming model, open extendable classes, reusable code.
+FRAMEWORK: Has an extensible framework with a fair amount of rapid development tools and web controls ready to use.
+STATE MANAGED: It manages the state for you using viewstate post backs.
+SERVER-EVENTS: An event model takes care of the communication between client and server.
+CROSS-BROWSER: The HTML and javascript it produces automatically match the capabilities of the target browser.
+CACHED OUPUT: Pages, controls and datasets can be cached in a variety of ways.
+FAST STARTUP: Only the HTML needed for displaying the current page is transmitted to the client.
-HEAVY TRAFFIC: When posting back, the whole web form including the viewstate is uploaded to the server and the whole page is then sent back to the client.
-RELOADS: When posting back the whole page is refreshed and especially on slow connections and or slow cpu's this makes the page flicker.

AJAX

+LIGHT TRAFFIC: State is not transmitted back to the server, only the needed data is transmitted
+NO RELOADS: The page is updated by using DHTML (manipulating the DOM with javascript).
-NO SERVER-EVENTS: Communication with the server is done by directly calling different web service methods.
-SLOW AND UNSAFE: Jscript is not a compiled language and testing is pretty difficult on some devices.
-SLOW STARTUP: Huge amounts of javascript is usually needed for client controls and these files are sent to the client on startup.
-NOT CROSS BROWSER: Making sure the DHTML is cross browser is completely in the hands of the developer.
-NO FRAMEWORK: You have to put together your own client-control framework to manipulate the DOM.

OutPost - The best of both worlds

In OutPost the advantages of ASP.NET WebForms and AJAX are combined. OutPost takes care of the following:

  1. Need for cross-browser DHTML is eliminated.
  2. A real Post Back is executed and server-side events are fired.
  3. Excess traffic is eliminated.
  4. Page reloads are eliminated.

That easy, right? Yes, let's take it step by step.

Step by step

The steps below describe the life of a Post Back when using OutPost:

  1. Need for cross-browser DHTML is eliminated

    • A main page loads the Web Form Page and in that process (on the server) the HTML and ViewState are stored for later use (it is used again on the server when posting back the WebForm). Also the needed link(s) to javascript file(s) are inserted in the HEAD section of the HTML (extra cross-browser compatibility layer scripts are added if necessary).

      When loading a WebForm, you can decide whether you want all the HTML or only the outerHTML of the form. If you want several forms loaded onto the same main page, you would only need the outerHTML of each form and add that, where you'd like on the page (as long as it is not inside the page's own web form).

      If you load a single WebForm you could go for the HTML of the whole page and simply let that be the contents of the main page. The main page does not necessarily need to have its own web form.

    • The above is optional and you can insert the javascript link in your code yourself and use the server-controls to add the cross-browser compatibility layers if needed.

      The compatibility layers are added by the files AtlasCompat.js and AtlasCompat2.js from Atlas (the Microsoft ASP.NET 2.0 AJAX platform) and they ensure that the OutPost runtime works the same in all browsers. For example on the window and the document elements the attachEvent and detachEvent functions are added, so the DHTML in the OutPost runtime can be the same.

      If you don't use a main page, the HTML structure and the ViewState of the first call will not be stored on the server and the first post back will have to send the ViewState to the server and the inner HTML of the whole web form will also be sent back to the client on the first Post Back.

  2. A real Post Back is executed and server-side events are fired

    The PostBack of the form is replaced by out-of-band calls to a .NET Web Service. The calls are made from javascript without reloading the page. This is done by posting a SOAP-envelope to the Web Service using XmlHttp. The SOAP-envelope contains the Uri of the WebForm Page (the action attribute of the form element) and the contents of the input fields in the WebForm (but not the ViewState).

    The Web Service then executes the real Post Back using HttpWebRequest and HttpWebResponse. Headers containing user agent, language and session cookie are sent on to the WebForm Page.

  3. Excess traffic is eliminated

    The HTML read from the HttpWebResponse is parsed and the tags/elements that have an ID gets picked out and put into a tree structure. This step only works if the HTML structure is correct. If the structure is not correct the inner HTML of the whole web form is sent back to the client (not the ViewState though).

    The HTML tree structure of the new post back is compared to the HTML tree structure of the last post back. This is done in a bottom-up way, so first the contents of the form without any inner HTML of sub nodes are compared. If no differences found, the inner HTML of the next sub nodes (with the same id and id-index) from each tree are compared without the inner HTML of their sub nodes. And so forth.

  4. Page reloads are eliminated

    The inner HTML of the elements found to be different in the previous step are returned to the client. The DOM is updated, element by element, using cross browser DHTML.

    It might seem as a bit of a crude way to just send the modified innerHTML fragments back to the client, but it really makes sense when developing cross-browser web applications.

Using the code

A single WebForm

To add OutPost and AJAX-enable a WebForm, all you need is to write the following in the beginning of your .aspx file:

  <script src="/OutPost/OutPost.js.aspx" type="text/javascript"></script>

Using a main page to load the WebForm

To load a page from a main page and OutPost-enable it you write the following in your Page_Load method:

    private void Page_Load(object sender, System.EventArgs e) {
      Page.Controls.Add(OutPost.Core.OutPost.GetLiteralControl(ResolveUrl("Validator7.aspx")));
    }

This (when placed in the top) adds cross-browser scripts on demand, based on the client User-Agent:

  <%@ Register TagPrefix="outpost" Namespace="OutPost.Controls" Assembly="OutPost" %>
  <outpost:script Runat="server" Path="~/ScriptLibrary/AtlasCompat.js" Browsers="Netscape,Mozilla,Firefox,Apple,Safari,Mac" />
  <outpost:script Runat="server" Path="~/ScriptLibrary/AtlasCompat2.js" Browsers="Apple,Safari,Mac" />

Having two forms on the same page that depend on each other

Using javascript it is possible to let two forms depend on each other:

  <script type="text/javascript">
    //OutPost7.aspx

    function document_onkeyup() {
      if (event.srcElement==Forms[0].TextBox1) {
        Forms[1].TextBox1.value = event.srcElement.value;
      }
    }

    function document_onclick() {
      if (event.srcElement==Forms[0].Button1) {
        Forms[1].Button1.bClicked = true;
        Forms[1].doPostBack('','');
      }
    }
    
    document.attachEvent("onclick",document_onclick);
    document.attachEvent("onkeyup",document_onkeyup);
  </script>

Using the Client Framework

The following example shows how you can dynamically add, remove and Post Back different forms on a page (See "The iGoogle Demo" in the demo project and the screen shot from the demo in the beginning of this article).

  //RssFeed.js - "The iGoogle Demo"

  function DeleteBox() {
    var oNodeForm = null;
    if (oEventNode) {
      oNodeForm = misc_getForm(oEventNode);
      oEventNode = null;
    }
    if (oNodeForm!=null) {
      oNodeForm.parentNode.parentNode.removeChild(oNodeForm.parentNode);
      Forms[oNodeForm.iIndex] = null;
    }
    var sAction = unescape(oNodeForm.action);
    FormMain.bAddBox = false;
    FormMain.doPostBack("oButtonDelete",sAction);
  }
  
  function AddBox() {
    var sUri = "/OutPost/Demo/RssFeed.aspx?rss="+escape(document.getElementById('FormMain').oTextBoxAddRssUri.value);
    
    for (var i in Forms) {
      if ((Forms[i]!=null) && (unescape(Forms[i].action)==unescape(sUri))) {
        alert("The feed has already been added");
        return;
      }
    }
  
    var oNodePanel = document.createElement("DIV");
    oNodePanel.className = "cPage";
    
    var oNodeColumn = document.getElementById("oTableCell2");
    if (document.getElementById("oTableCell1").childNodes.length<oNodeColumn.childNodes.length) {
      oNodeColumn = document.getElementById("oTableCell1");
    }
    if (document.getElementById("oTableCell3").childNodes.length<oNodeColumn.childNodes.length) {
      oNodeColumn = document.getElementById("oTableCell3");
    }
    
    oNodeColumn.appendChild(oNodePanel);
    
    var oNodeForm = createForm("Form1", sUri, AddBox_onerror);
    oNodePanel.appendChild(oNodeForm);
    document.getElementById('FormMain').oTextBoxAddRssUri.value = "";
  }
  
  function AddBox_onerror() {
    alert("Error adding RSS feed");
  }
  
  function FormMain_onsuccess() {
    this.base_onsuccess();
    if (this.bAddBox!=false) {
      AddBox();
    }
    this.bAddBox = true;
  }
  
  function window_onload() {
    FormMain.onsuccess = FormMain_onsuccess;
  }
  
  window.attachEvent("onload", window_onload);

The Future of OutPost

  • Making it possible to send back changes on a smaller scale (like only setting the value of an input or only changing one option in a dropdown list).
  • I would like to incorporate simple AJAX behaviour, so that you can interact with business logic on the web server.
  • Solving the Bookmark issue. It is not possible to bookmark a WebForm page and I'm trying to find a solution to that.
  • Solving the Back-button issue. It is not possible to use Back-Button and I'm trying to find a solution to that also.

History

  • Sep 2005 - OutPost version 1.0 - Added cross-browser support, lightweight client-framework,
  • May 2005 - OutPost version 0.5 - Single web form supported, IE only.
  • Nov 2004 - First initial testing performed.