XML, Web Services, SOAP, and transmission protocols

Intro

In the pursuit of integration, I've found myself needing to integrate with clients with different integration needs/capabilities. In doing so I found some interesting things about .NET as well as just web services in general.

Communication Protocols

Although most web services are consumed over HTTP as the transmission protocol, not all communicate through SOAP. While generally the standard is communication through SOAP, some communicate over HTTP.

I recently found .NET does have the ability to use HTTP POST and GET as communication protocols as well, which seems expected. However, as of .NET 1.1 SOAP is the primary communication protocols and while HTTP and GET are supported, they are disabled by default. It turns out that it is very simple to enable them, only requiring the following change in your application's "web.config" file:

<webservices>
  <protocols>
    <add name="HttpPost">
    <add name="HttpGet">
  </protocols>
</webservices>

This makes integration with many legacy systems much more possible- and especially other systems that are not on the .NET platform.

A page built to make a request to a web service using HTTP as both the transmission and communication protocol may look as such:

<form action="mywebservice.asmx/WebServiceMethod" method="post">
  <input name="parameterName" value="param1" />
  <input name="parameterName2" value="param2" />
  
  <input type="submit" value="Submit!" />
</form>

If you were developing your web service on .NET all you'd have to do is develop as normal and modify your web.config as described. You may also consume a web service using GET and query parameters.

One of the scenarios I've found myself in is a client desiring to consume the web service using HTTP as both the transmission and communication protocols, however passing XML as a "parameter" effectively using XML as a communication protocol as well. I found this is relatively simple and easy to work with as long as the XML is url-encoded as one of the form's fields, and is decoded once loaded. (In .NET you would simply use HttpUtility.UrlDecode()).

You can then load this string in an XmlDocument and manage it easily using the object structure in .NET. When returning a response- if the client is able to accept an XML response- using .NET you can simply return the XmlDocument object. The XmlDocument Object can be found in the System.Xml namespace. do not return an XML-formatted string if you have an XmlDocuments.


XML and SQL Server 2005

In the midst of all this, I found myself storing XML. I realized there was an XML data type in SQL Server, and so I did a bit of research on it. I decided to use it and I was pretty amazed.

By storing data as XML in SQL Server 2005 using the XML data type, you can actually use XQuery to write queries that take advantage of the XML data structure. It's almost like running your main query and s asubquery to filter the XML data. It make it very practical and simple to retrieve and store XML data.

A SQL Query using XQuery might look like:

SELECT OrigXmlRequest.query('/Notes//child::*')
FROM YardiRequest

This would return all the XML child nodes and their descendants of 'Notes'.

XML Data in SQL Server 2005 is stored in the same was a varchar(MAX) is, and has a max size limit of 2 gigs. (Which if you're hitting anywhere close to that 2 GIG limit when storing an XML file, you should probably store the data normalized in tables)

One thing I ran into that I would like to point out when storing the XML data from .NET into SQL was the encoding format used. I kept receiving an encoding exception, until I realized the encoding needed to be changed from UTF-8 to UTF-16 in order to be stored in the database. If you have an XmlDocument object, an easy way to do this would be to just edit the first child and the you will be able to store the data. e.g.:

xmlDcRequest.LoadXml(myXmlString);
// Change encoding to UTF-16 for storage in SQL Server
xmlDcRequest.FirstChild.InnerText = xmlDcRequest.FirstChild.InnerText.ToLower().Replace("utf-8", "UTF-16");

...

// Adding the XML to a stored procedure parameter to store the XML data:
sqlCmd.Parameters.Add("@XMLRequest", SqlDbType.Xml).Value = xmlDcRequest.InnerXml.ToString();

Final Notes on Web Service Development

I think one of the biggest lessons I am learning from putting together web services to service other clients is the cruciality of writing code as bulletproof as possible. Of course, you should always be doing this anyways. Make sure your web service is returning clear error codes and descriptions when errors do occur to make development for the client straightforward. Clear communication and a detailed scope of work are essential. Even though these are simple things, it seems many times these things are missed.

- Jheatt