Friday, January 16, 2009

XML, Web Services, SOAP, and transmission protocols


In the pursuit of integration, I've found myself needing to integrate with clients with different integration needs/capabilities. In doing so I found some interesting things about .NET as well as just web services in general.

Communication Protocols

Although most web services are consumed over HTTP as the transmission protocol, not all communicate through SOAP. While generally the standard is communication through SOAP, some communicate over HTTP.

I recently found .NET does have the ability to use HTTP POST and GET as communication protocols as well, which seems expected. However, as of .NET 1.1 SOAP is the primary communication protocols and while HTTP and GET are supported, they are disabled by default. It turns out that it is very simple to enable them, only requiring the following change in your application's "web.config" file:
<add name="HttpPost">
<add name="HttpGet">

This makes integration with many legacy systems much more possible- and especially other systems that are not on the .NET platform.

A page built to make a request to a web service using HTTP as both the transmission and communication protocol may look as such:

<form action="mywebservice.asmx/WebServiceMethod" method="post">
<input name="parameterName" value="param1" />
<input name="parameterName2" value="param2" />

<input type="submit" value="Submit!" />

If you were developing your web service on .NET all you'd have to do is develop as normal and modify your web.config as described. You may also consume a web service using GET and query parameters.

One of the scenarios I've found myself in is a client desiring to consume the web service using HTTP as both the transmission and communication protocols, however passing XML as a "parameter" effectively using XML as a communication protocol as well. I found this is relatively simple and easy to work with as long as the XML is url-encoded as one of the form's fields, and is decoded once loaded. (In .NET you would simply use HttpUtility.UrlDecode()).

You can then load this string in an XmlDocument and manage it easily using the object structure in .NET. When returning a response- if the client is able to accept an XML response- using .NET you can simply return the XmlDocument object. The XmlDocument Object can be found in the System.Xml namespace. do not return an XML-formatted string if you have an XmlDocuments.

XML and SQL Server 2005

In the midst of all this, I found myself storing XML. I realized there was an XML data type in SQL Server, and so I did a bit of research on it. I decided to use it and I was pretty amazed.

By storing data as XML in SQL Server 2005 using the XML data type, you can actually use XQuery to write queries that take advantage of the XML data structure. It's almost like running your main query and s asubquery to filter the XML data. It make it very practical and simple to retrieve and store XML data.

A SQL Query using XQuery might look like:

SELECT OrigXmlRequest.query('/Notes//child::*')
FROM YardiRequest
This would return all the XML child nodes and their descendants of 'Notes'.

XML Data in SQL Server 2005 is stored in the same was a varchar(MAX) is, and has a max size limit of 2 gigs. (Which if you're hitting anywhere close to that 2 GIG limit when storing an XML file, you should probably store the data normalized in tables)

One thing I ran into that I would like to point out when storing the XML data from .NET into SQL was the encoding format used. I kept receiving an encoding exception, until I realized the encoding needed to be changed from UTF-8 to UTF-16 in order to be stored in the database. If you have an XmlDocument object, an easy way to do this would be to just edit the first child and the you will be able to store the data. e.g.:

// Change encoding to UTF-16 for storage in SQL Server
xmlDcRequest.FirstChild.InnerText = xmlDcRequest.FirstChild.InnerText.ToLower().Replace("utf-8", "UTF-16");

// Adding the XML to a stored procedure parameter to store the XML data:
sqlCmd.Parameters.Add("@XMLRequest", SqlDbType.Xml).Value = xmlDcRequest.InnerXml.ToString();

Final Notes on Web Service Development

I think one of the biggest lessons I am learning from putting together web services to service other clients is the cruciality of writing code as bulletproof as possible. Of course, you should always be doing this anyways. Make sure your web service is returning clear error codes and descriptions when errors do occur to make development for the client straightforward. Clear communication and a detailed scope of work are essential. Even though these are simple things, it seems many times these things are missed.

- Jheatt

Saturday, January 10, 2009

SQL Server Reporting Services and Headers requiring dataset fields.

Any search for "fields in headers SSRS" in Google will come up with many search results with pages addressing workarounds for one of Microsoft SQL Server Reporting Service's biggest weakness: the inability to add fields to the header or footer from a dataset.

This limitation makes some sense in that a dataset returns rows of fields and so the problem might lie in identifying which row should be displayed, however one should be able to define exactly what to display- the first or last row or maybe an aggregate of the rows.

All that to say, this is yet another workaround I haven't seen elsewhere. It addresses the problem of maintaining data in the header regardless of what page the report is being viewed on.

Why another work around? I tend to avoid storing fields in internal parameters because I've had problems with it before, and shared variables are a horrible solution in a production enviroment because of the possibility of concurrently running reports. It seems like these are always the solutions given to this issue.

So here goes: the way I've solved it is by making a group within the table displaying my dataset, and then using the footer of the group to store a list box, (if you are needing to use the footer, simply extend it to add another footer row right below it). In that list box you can place another table or fields and you can even have this list box pull from another datasource. When done, set the group footer's "Hidden" property to "True" so that it doesn't show on the rendered report, and you're set. You can now reference these fields from the header or footer like so:


And since it's an expression, you can do whatever you want. You can also use aggregates and such- just make sure you modify the expression on the field/table pulling directly from the dataset, and not the field in the header/footer that will actually be displaying the data.

The only major problem I've run into with this approach (that I am still working on) is that the fields in the header will export as blank if I export the report as a PDF. It will do fine in HTML, or even Excel. (Which is funny since most rendering errors seem to be tied to Excel instead of the other way around).

If any solution is found to this issue, or if someone can explain to me the PDF rendering process and why these fields show up as blank, it would be greatly appreciated!

Hopefully this post was of some use to those that do not need to export in PDF...

- Jheatt

HTTPS, HTTP, and form posting.

So this is really more of a quick blog post for my own benefit of recording a solution. HTTPS and form posting.

I haven't really done a whole lot with HTTPS and I'm still sort of new to it. I was trying to do a form post cross-domain between a beta server and my computer and it wouldn't work. It would run fine locally, but not with the BETA server. The funny thing is it ran fine when I tried running it locally on the BETA server or on my development machine, but I couldn't post from my development machine to the BETA server- so it seemed like it was a cross-domain security issue.

All the POST fields would come up blank- actually they wouldn't even be set, query parameters would be fine of course.

Another fellow developer I work with pointed out I should be posting back with HTTPS. The form was posting to a HTTP address, but once it loaded it showed HTTPS. That was the only problem. I couldn't believe it- so simple! I guess I needed to be humbled.

So I still don't understand HTTPS completely, but I will definitnely be researching. My guess is that the POST didn't go through since the HTTPS server was addressed as HTTP in the original form POST and so the POST didn't go through all the SSL encryption action and hence it turned up empty on the server side... any one care to give me the technical explenation? I guess I'll find out more when I research.

- Jheatt

New Job; New Obstacles

So I switched jobs, and this new place is totally a Microsoft Shop. It's different than what I'm used to, but in a good way. I like how we are working with the latest tech for the most part, including MS Visual Studio 2008- love it.

Doing nothing but .NET (Both VB and C#) with the exception of a few third-party controls (Telerik) this will was sort of a new field for me with new obstacles. I'm excited. I was looking forward to working with a .NET development team and loving it so far.

So I know I haven't written a tech blog forever but I'm starting to realize the importance of this blog to more than just giving back to the community- more like a way to remember all the problems and solutions I've come accross. Sometimes I'll run into the same problem I've ran in the past but will forget the solution, so it's easy to see how a blog would be a great way to retrieve this information while helping others as well.

Anyway, just preparing the way for my first post as a software engineer here at Trak-1. Happy coding...

- Jheatt